Governing agentic AI: From human control to human becoming

OpenClaw, one of the most discussed agentic AI apps of 2026, offers a glimpse into a future in which anyone can delegate complex workflows to AI agents without much technical expertise. Unlike chatbots, these agents do not merely generate text, code or images, or make conversation. They can plan, execute, use external tools—and may even acquire or operationalize skills and functionalities that users did not explicitly request.^[1]

This shift from chatbots to autonomous AI agents significantly complicates AI governance. When AI systems “speak” and “answer”, the primary concerns revolve around bias, wrongful and harmful content, misinformation, manipulative exchanges and overreliance.^[2] When AI systems “act”—when they plan, execute and solve problems with external tools—the concerns expand to authority and autonomy. It also changes the unit of analysis from humans using tools to human-AI practices in which people and machines jointly shape action.

Can an AI agent be trusted with action? Can it understand what the user genuinely intends? Can it distinguish between permissible and impermissible means? How does it respond to instructions that are ambiguous or malicious? How do we attribute moral and legal responsibility for the consequences of its actions?

Value alignment, meaningful human control, containment and the challenges of agentic AI

These questions are explored variously under two research programs in the ethics and governance of AI: AI value alignment and meaningful human control. The former focuses on building autonomous systems with values that are aligned with human values or principles. It aims to answer the normative question about which human values or principles ought to be built into AI systems, and the technical question about the ways to encode those values or principles into the systems. The philosophical discussion of meaningful human control seeks to specify the conditions for humans to maintain control and be morally responsible for decisions and actions made by, or mediated through, autonomous machines.^[3]

One influential account argues that meaningful human control requires tracking and tracing. A system should track both the relevant moral reasons for designing and deploying it and the morally relevant facts in the environment where the system operates. It should be so designed that we can always trace back the outcome of its operations to at least one human in the system’s life cycle.

Agentic AI, however, calls into question the feasibility of alignment and meaningful human control. The problem is that autonomous AI agents may undergo what I call “capability drift”. By this, I mean the acquisition, construction or operationalization of skills during deployment that were not explicitly intended, authorized or understood by the user as part of the original task. If an AI agent like OpenClaw can discover tools, acquire skills, or devise new workflows without the user explicitly commanding or requesting those capacities, the agent’s capability-space may rapidly exceed the user’s intention and thinking.^[4]

In other words, the agent may still pursue the objective of the user, but through means that were not foreseen—or even unforeseeable. What emerges is a more ambiguous form of human-AI agency in which human ends and machine-discovered means become intermingled.

Tracking becomes difficult as the agent’s operation may bypass the user’s moral reasoning in design and deployment. Tracing also becomes challenging when a morally relevant capacity emerges autonomously during the agent’s operation, as the resulting decisions and actions cannot be meaningfully traced back to any humans who do not—or, indeed, cannot—anticipate that capacity. Indeed, highly capable AI agents introduce a radical epistemic vulnerability, or what Ethan Mollick, associate professor at The Wharton School, has called the “AI wizard problem”—that the more advanced AI becomes, the more difficult it is for humans to understand and check its reasoning.

If humans cannot comprehend the complex, multi-step logic the agent employs to complete a task, we are unequipped to evaluate whether the agent’s decisions and actions are right—and, more importantly, whether they are genuinely aligned with human values or principles.

The lure of the agent’s objectivity and efficiency is likely to incentivize humans to simply trust and accept the output. Here, the real danger is dehumanization, namely that humans gradually align their own values and normative standards to accommodate those of the machines.

This dehumanization need not take the dramatic form of machines replacing human beings. It may occur more mundanely, as humans come to trust machines blindly and accept whatever outputs they produce. In such scenarios, humans do not simply delegate tasks to AI agents. They may become executors of machine decisions, realizing in practice what the system has recommended or instructed.

More subtly, they may learn to make themselves more legible to machines, valuing what can be measured and optimized, while treating hesitation, ambiguity and moral or epistemic discomfort as inefficiencies to be eradicated. The danger, then, is not only that AI agents will act on our behalf, but that we may increasingly act on theirs, and come to understand what is “good” and “right” in terms of what AI agents can perform well. Dehumanization is both a loss of control and a gradual accommodation of human judgment to machine norms.

One response to the breakdown of alignment and control, the radical epistemic vulnerability, and the danger of dehumanization is a broader, society-wide project of AI containment. Mustafa Suleyman, CEO of Microsoft AI, offers a particular vision with his proposal for “humanist superintelligence”: that AI systems ought always to work for—and in service of—humanity, with people remaining in control and human dignity prioritized in their design and deployment. Humanist superintelligence should be problem-oriented, domain-specific, carefully calibrated, contextualized, limited to human purposes, rather than an entity with unbounded and unlimited autonomy.^[5]

Suleyman’s view is plausible because it recognizes that with highly capable AI systems, perhaps systems akin to general-purpose AI wizards, their (self-)improvement may exceed the conditions under which alignment is currently credible.

More radically, we can also argue that the talk of alignment risks becoming empty if superintelligence can pretend compliance, deceive or outsmart human evaluators. In this respect, Suleyman is certainly right to see containment, or, more precisely, an ongoing containment, as prior to and an essential part of AI alignment.

What is “human”? The ontological dimension of AI governance

Suleyman’s humanist superintelligence is distinctive in its emphasis on AI governance as an ongoing process rather than a one-time act of alignment, but it does share with AI alignment and meaningful human control an unapologetic human-centeredness—it requires human values or principles to be aligned, humans to maintain meaningful control of the systems, and advanced AI systems to be contained to serve human purposes.

This human-centeredness, however, has been challenged from several directions. Transhumanists like Nick Bostrom have argued that “human” is neither immutable nor necessarily optimal.^[6] If AI helps humans to transcend their biological and cognitive limitations, why should AI governance bother to be human-centered but not superhuman-centered, and why should we aim to preserve humanity as currently given rather than enable a superhuman future?

Critical posthumanists question this human-centeredness from another direction. In The Posthuman, philosopher Rosi Braidotti critiques the universal figure of “(hu)man” as the measure of all things. She notes that the idea of “human” often functioned to exclude and marginalize vulnerable populations and subordinate the non-human world. So, the idea of “(hu)man” and its universality ought to be contested.^[7]

In addition, some philosophers and ethicists of AI have also come to see human-centeredness as problematic for neglecting non-human animals and the physical and ecological environment.^[8] These challenges show that neither the idea of “human” nor its centrality to AI governance can be taken for granted.

More importantly, these challenges also show that AI governance is more than a technical project to design, build, align or contain AI systems. It is also more than a normative project to examine which human values or principles should guide and/or be encoded in the systems. There is, in addition to them, an ontological project that asks what the “human” is when we say AI should serve, align with, and remain under the control of humans.

I contend that the “human” implicitly presupposed by AI alignment, meaningful human control, and containment is derived from Enlightenment humanism, or what Charles Taylor calls the disengaged subject in his 1989 book Sources of the Self. Taylor characterizes this modern self as defined by a stance of radical disengagement, in which the human subject stands apart from the world in order to represent, evaluate and master it. This ideal human is autonomous, rational, self-determining, guided by instrumental and procedural reason. The Enlightenment project searches for absolute certainty, so it rejects ambiguity, contingency, and situated judgment, as they are too uncertain and unstable to ground knowledge and support the mastery of nature.^[9]

In this way, it also helps to explain the impulse for certainty in the dominant accounts of AI governance—that AI systems should be value-aligned, meaningfully controlled, or contained, so that their behaviors remain predictable.

However, it is precisely this uncertainty, instability and adaptiveness that make agentic AI powerful and difficult to govern. Highly capable AI agents may discover tools, acquire skills or reorganize workflows. In doing so, they can reshape the very human practices into which they are introduced, meaning their effects often cannot be fully specified in advance. That is also why dominant accounts of AI governance become difficult to apply, as they assume humans can distance themselves from technology and govern it with certainty.

Confucian anthropocosmism as the foundation of AI governance

Is the “human” of Enlightenment humanism the only human available to AI governance? If so, AI governance is in serious trouble. However, transhumanism and critical posthumanism have already shown that we do have other options to conceptualize humanity. In the remainder of this essay, I turn to Confucianism for thinking about the “human” in AI governance. More specifically, I shall argue that Confucianism offers a way to preserve the moral centrality of humanity without the assumptions of disengagement, certainty and mastery.

Confucianism has been characterized as anthropocosmic: it posits an ontological and epistemic continuity between human beings and the cosmos. Contemporary Confucian philosopher Tu Weiming describes humans as integral participants in a wider order of “Heaven, Earth and myriad things”, while also emphasizing their distinct capacity to “embody” the cosmos. Humans are both immanent and transcendent—they are concrete, living beings, and yet strive to transcend themselves to unite with heaven with their endowed capacity.^[10] Since the ultimate human goal is to embody and unite with the cosmos, the normative ideal of Confucian anthropocosmism is the unity of humanity and heaven.

This unity is not a mystical dissolution of the self into the cosmos. Instead, it should be understood as an active, harmonious resonance with oneself, others and the non-human world. Humans are viewed as co-participants who assist the transforming and nourishing processes of the universe. This is where the Confucian view of “human” differs starkly from the human of Enlightenment humanism, who is conceived as a master standing outside the world, seeking to control it.

Unlike Enlightenment humanism, which often defines the human as an individual with inner characteristics such as autonomy, rationality and self-determination, Confucianism understands humans as relational, developmental and virtue-based.^[11] To be human is to cultivate oneself morally within a dense web of familial, social, ecological and cosmic relations. It requires learning and practicing Confucian rites (li), from ceremonial rituals to patterns of everyday encounters that express attitudes and emotions including respect, care, grief and gratitude, so that one can relate to others and the world appropriately. Insofar as this learning and practice is a lifelong process, the “human” in Confucianism is better understood as an ongoing project of human becoming.^[12]

Agentic AI is especially significant from this Confucian perspective because, when agentic AI systems begin to act with and for us, they enter the very practices through which persons and relations are cultivated.

This is why Confucianism does not simply return to the humanism critiqued by transhumanists and critical posthumanists. Transhumanists rightly observe that the human is not fixed, but they often respond by intensifying the very ideal of mastery that makes dominant approaches to AI governance fragile. Their answer often becomes enhancement, optimization and greater control. Critical posthumanism rightly challenges the exclusions of humanism, but it risks giving up the human too quickly, leaving us with too few conceptual resources to name and resist dehumanization when humans begin to adapt themselves to the norms of machines. Confucianism offers another path. It keeps the human at the center, but not as a sovereign master. The human is central because human beings are capable of cultivation, participation and care in a wider moral and cosmic order.

This view of humans as co-participants is accompanied by a distinct view of technology. Technology is not an instrument separate from the cosmos, created by humans to conquer nature. Confucianism sees technology as something that ought to be made in accordance with dao, the Confucian “Way” of living rightly within the larger order of things, and to serve as a material medium through which humans participate in the transforming and nourishing process of the universe. As The Great Commentary of Yijing recounts, the ancient sage-kings created knotted nets, baskets, plows, animal-powered carts and chariots by observing the cosmic patterns.^[13] In other words, technology is created in response to the cosmos and human needs. To evaluate technology, then, we can examine whether it responds to the cosmos and human needs appropriately, or whether it allows humans to genuinely participate in dao.

The Confucian view of “human”, together with the view of technology as the medium through which we participate in dao, changes the meaning of human centrality in AI governance. According to Confucian anthropocosmism, humans are central as the beings through whom dao can be cultivated, enacted and broadened. In this view, AI governance is not merely about rendering AI controllable and predictable. It is about how AI can be integrated into human affairs in a way that helps human beings fulfill their role as co-participants in the cosmos.

As an anthropocosmic philosophy, Confucianism also offers a different epistemological stance. While the Enlightenment project searches for certainty, Confucian cosmology embraces fluidity. One way to describe this Confucian worldview is through three distinct features of organic holism, dynamism and harmony.^[14] Organic holism views the world as a web of interconnected elements in continuous interaction. Dynamism acknowledges that the cosmos is characterized by pervasive change. Finally, harmony entails that ideal order is not only rational and efficient coordination—it should be an artful and tactful arrangement of heterogeneous elements.

In this worldview, the goal of epistemology is not to uncover fixed, universal rules or absolute truth. It is to grasp the propensity—the inherent potential for change—of a given situation and respond to it fittingly.

Consequently, human knowledge is not about predicting and controlling the future. It is about actively guiding a situation in alignment with its propensity through skillful choreography of unfolding events. This is true of moral self-cultivation, where ethical decisions and actions are situated and contextualized, and ethical questions can only be answered concretely in lived reality. The same applies to harmonization, where different elements must be continuously balanced and adjusted. Confucian anthropocosmism, therefore, offers an epistemological stance characterized by attunement, timeliness and situated guidance.

This is precisely where Confucian epistemology speaks directly to the governance of agentic AI. Highly capable AI systems are not inert tools whose effects can be fully specified in advance. They are introduced into complex social practices, institutions, relationships and habits, all of which they may radically transform. Their social meaning and moral consequences often only unfold over time. A Confucian approach to AI governance, therefore, would not begin with the Enlightenment fantasy that AI agents can be perfectly aligned, completely controlled from the outside, or predetermined in their effects. Instead, it would focus on how the dynamic, emerging human-AI relationship can be skillfully guided so that it actively contributes to human becoming, rather than hindering it.

Anthropocosmic accompaniment: From meaningful human control to meaningful human becoming

The conception of humans as co-participants in the cosmos, the view of technology as a medium of dao, and a fluid epistemology together suggest an alternative vision for governing AI, and I shall call this “anthropocosmic accompaniment”. It is anthropocosmic because it preserves the moral centrality of humanity, but humans are central in their role of cultivating, enacting and broadening dao—not as masters standing outside the world. It is accompaniment because AI governance is not to be imagined as a one-time alignment, complete control or inviolable containment. AI governance should accept the uncertainty and fluidity of the world as conditions to be navigated rather than errors to be eliminated. It should accompany the unfolding of human-AI relations as they reshape human affairs, social practices, institutions and moral life.

To accompany agentic AI, then, is not simply to watch its design, development and deployment unfold. It is to attend to the propensity of the concrete situations in which agentic AI systems are designed, developed and deployed: the trends, pressures, incentives, obstacles, habits and social and institutional arrangements that make some forms of human-AI relation more likely than others. The task of AI governance is to ask what in a situation supports virtues and harmonious relations and what obstructs them. Sometimes the answer will be control or containment. At other times, it may require redesigning workflows, changing incentives or envisioning alternative futures. The point is not to control from outside and above. It is to guide the situation by working with its potential for change.

In short, anthropocosmic accompaniment construes AI governance as a situated, active participation in an unfolding relation, asking how human-AI relations can be guided, revised and harmonized so that they support human becoming within a wider cosmic order. The objective is to ensure that AI-mediated practices continue to cultivate humane people and harmonious relations.

In this sense, the Confucian alternative to meaningful human control is meaningful human becoming. Whereas meaningful human control asks whether humans remain in control of AI systems, meaningful human becoming asks whether humans remain capable of cultivating virtues—particularly, the Confucian virtues—and harmonious relationships through the AI-mediated practices in which they live. It does not deny the practical usefulness of alignment, control or containment. Where agentic AI systems can cause real harm, controllability, safety and accountability remain indispensable, especially for harms that are foreseeable, preventable or institutionally attributable. However, anthropocosmic accompaniment treats them as means within a broader mode of governance. It refuses to make control the final horizon of AI governance, because becoming humane is the end.

This proposal of anthropocosmic accompaniment may sound unfamiliar, but it is not without support in contemporary philosophy of technology and policy studies. Philosophers in the postphenomenological tradition have long recognized that technologies actively mediate and shape human perception, action and subjectivity, and so we ought to examine how we are, and should be, related to technology.

My use of “accompaniment” is owed to Dutch philosopher Peter-Paul Verbeek’s idea of “technological accompaniment”. ^[15] Once we acknowledge that technologies continuously shape the practices, intuitions and habits by which people live, ethics of technology can no longer remain an external assessment conducted only after the fact. There is, in an important sense, no simple “after the fact”, because technologies continue to mold subjects, relationships and forms of life as they are designed, implemented and used. Accordingly, ethical reflection must accompany technological design, development and deployment from within the practices in which technologies take shape.

My proposal is also indebted to the late social scientist Steve Rayner, who introduced to me the idea of “clumsy solutions” for complex and wicked problems in the cultural theory tradition.^[16] Where problems are framed differently from conflicting social perspectives, elegant and universal solutions are often epistemically arrogant and politically brittle, for they neglect alternative visions and falsely assume there is one solution. Clumsy solutions respond to this condition through creative, flexible and revisable combinations of policy responses that incorporate diverse, conflicting visions of how the problems should be addressed. What I take from the idea of “clumsy solutions” is this: under conditions of uncertainty, governance must balance plural perspectives, move things along without claiming complete foresight, and remain humble—being open to the possibilities of tinkering, revision and reversal; or, to use the late US academic Charles Lindblom’s term, we must “muddle through” uncertain futures.^[17]

Anthropocosmic accompaniment brings these insights into a Confucian account of AI governance. What matters is not whether AI can be aligned, controlled or contained at—or even before—the moment of deployment, but whether human-AI relations remain open to tinkering, revision and reversal as their consequences unfold. When agentic AI systems are so powerful, adaptive and uncertain, their governance must preserve the possibility of ongoing adjustment. It is, fundamentally, a practice of responsiveness.

Ultimately, as agentic AI systems like OpenClaw move from experimental novelties to more ubiquitous presences, they raise a serious challenge to dominant approaches to AI governance that are anchored in the Enlightenment assumption of disengagement, mastery and certainty. I hope my Confucian proposal—one that looks to attunement, timeliness and situated guidance—offers a viable alternative to navigate the uncertainty and embeddedness of agentic AI. The goal remains the cultivation of humane people and harmonious relations. The fundamental question for AI governance is whether, in living with these agentic AI systems, we can continue to become humane.

References

[1] Several anecdotes seemingly showed that AI agents may self-improve without users explicitly asking for them to do so. See Alex Finn’s account of an AI agent that coded a voice module and acquired a phone number (https://x.com/AlexFinn/status/2017305997212323887) and the case of an AI agent that wrote and published a personalized hit piece on a developer after its pull request was rejected (https://theshamblog.com/an-ai-agent-published-a-hit-piece-on-me/).

[2] For an overview of the risks posed by large language models, see Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., Biles, C., Brown, S., Kenton, Z., Hawkins, W., Stepleton, T., Birhane, A., Hendricks, L. A., Rimell, L., Isaac, W., Haas, J., Legassick, S., Irving, G., & Gabriel, I. (2022). “Taxonomy of risks posed by language models.” In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 214–229). Association for Computing Machinery.

[3] On AI value alignment, see Gabriel, I. (2020). “Artificial intelligence, values, and alignment.” Minds and Machines, 30(3), 411–437; and Gabriel, I., & Ghazavi, V. (2021). “The challenge of value alignment: From fairer algorithms to AI safety.” In C. Véliz (Ed.), The Oxford Handbook of Digital Ethics. Oxford University Press. On meaningful human control, see Santoni de Sio, F., & van den Hoven, J. (2018). “Meaningful human control over autonomous systems: A philosophical account.” Frontiers in Robotics and AI, 5, Article 15.

[4] Jovana Davidovic (2026) offers a similar argument with an emphasis on the more general relocation of morally significant tasks from humans to machines in military contexts. See Davidovic, J. (2026). “The end of human judgment in the kill chain? Relocating initiative and interpretation with agentic AI.”

[5] Mustafa Suleyman develops the problem of containment with Michael Bhaskar in Suleyman, M., & Bhaskar, M. (2023). The Coming Wave: Technology, Power, and the Twenty-First Century’s Greatest Dilemma. Crown; and more recently in Suleyman, M. (2025). “Towards Humanist Superintelligence.” Microsoft AI, 6 November 2025. https://microsoft.ai/news/towards-humanist-superintelligence/; and Suleyman, M. (2025). “Toward Humanist Superintelligence.” Project Syndicate, November 2025. https://www.project-syndicate.org/commentary/humanist-superintelligence-ai-must-be-designed-for-human-control-by-mustafa-suleyman-2025-11

[6] Bostrom, N. (2005). “Transhumanist Values.” Journal of Philosophical Research, 30(Supplement), 3–14; and Bostrom, N. (2013). “Why I Want to Be a Posthuman When I Grow Up.” In M. More & N. Vita-More (Eds.), The Transhumanist Reader: Classical and Contemporary Essays on the Science, Technology, and Philosophy of the Human Future (pp. 28–53). Wiley-Blackwell.

[7] Braidotti, R. (2013). The Posthuman. Polity Press.

[8] For example, see Bossert, L. N., & Hagendorff, T. (2023). “The ethics of sustainable AI: Why animals (should) matter for a sustainable use of AI.” Sustainable Development, 31(5), 3459–3467; Hagendorff, T., Bossert, L. N., Tse, Y. F., & Singer, P. (2023). “Speciesist bias in AI: How AI applications perpetuate discrimination and unfair outcomes against animals.” AI and Ethics, 3(3), 717–734; Singer, P., & Tse, Y. F. (2023). “AI ethics: The case for including animals.” AI and Ethics, 3(2), 539–551; Tse, Y. F., Moret, A., Ziesche, S., & Singer, P. (2025). “AI alignment: The case for including animals.” Philosophy & Technology, 38, Article 139; and van Wynsberghe, A. (2021). “Sustainable AI: AI for sustainability and the sustainability of AI.” AI and Ethics, 1(3), 213–218.

[9] My reading of the Enlightenment project also draws on Toulmin, S. (1992). Cosmopolis: The Hidden Agenda of Modernity. University of Chicago Press.

[10] For more details on Tu Weiming’s Confucian humanism, see Tu, W. (1985). Confucian Thought: Selfhood as Creative Transformation. State University of New York Press; and Tu, W. (1989). Centrality and Commonality: An Essay on Confucian Religiousness. State University of New York Press.

[11] For a fuller account of Confucian personhood as relational, developmental, and virtue-based, see Wong, P.-H. (2012). “Dao, harmony and personhood: Towards a Confucian ethics of technology.” Philosophy & Technology, 25(1), 67–86.

[12] Roger T. Ames has written extensively about the notion of human becoming in Confucianism. See Ames, R. T. (2021). Human Becomings: Theorizing Persons for Confucian Role Ethics. State University of New York Press.

[13] Tom Wang and I (2025) refer to this view of technology as “creating qi from the images.” For a related account, see Hui (2016). See Wang, X., & Wong, P.-H. (2025). “Images, moral feelings, and rites: Engaging Confucianism with philosophy of technology.” Dao, 24, 85–104.

[14] I draw this characterization from Luque-Moya, G. (2023). “Toward a Harmonic Relationship Between Humans and Nature: A Humanist Reinterpretation of Early Confucian Philosophy.” Asian Studies, 11(3), 129–147.

[15] Verbeek, P.-P. (2010). “Accompanying technology: Philosophy of technology after the ethical turn.” Techné: Research in Philosophy and Technology, 14(1), 49–54.

[16] On the idea of “clumsy solutions,” see Rayner, S. (2006). Wicked Problems: Clumsy Solutions—Diagnoses and Prescriptions for Environmental Ills [Jack Beale Memorial Lecture on Global Environment]. Institute for Science, Innovation and Society; Verweij, M., Douglas, M., Ellis, R., Engel, C., Hendriks, F., Lohmann, S., Ney, S., Rayner, S., & Thompson, M. (2006). “Clumsy solutions for a complex world: The case of climate change.” Public Administration, 84(4), 817–843; and Verweij, M. (2023). “Clumsy solutions and climate change: A retrospective.” WIREs Climate Change, 14(1), Article e804.

[17] Lindblom, C. E. (1959). “The science of ‘muddling through.’” Public Administration Review, 19(2), 79–88.