The General Assembly and a Smart Constitution for Humanity

Last Monday, hundreds of diplomats, executives, business leaders, technology experts, and academics gathered in New York to confront a question that was once confined largely to the realm of science fiction: How can humanity ensure that it will not one day be destroyed by rebellious machines?

This was the second edition of what has come to be known as “Digital Cooperation Day.” Yet the stakes this time were considerably higher than they were when participants gathered a year earlier.

The following day, Tuesday, a highly significant meeting took place between U.S. Treasury Secretary Scott Bessent and Chinese Vice Premier He Lifeng. The two sides agreed to establish an “AI red line” for reporting incidents involving artificial intelligence that could threaten global national security.

By Thursday, the White House was hosting a dinner attended by members of the American and Chinese technology oligarchy. Beyond the immediate occasion lay a much broader objective: negotiating the terms of competition over the global artificial intelligence economy, drawing boundaries, establishing channels of communication, and shaping the rules of engagement.

What Is Driving the Global Anxiety over AI?

What lies behind the growing sense of alarm surrounding the latest developments in artificial intelligence?

The current wave of concern intensified after Jacob Coxon, a former researcher at Anthropic and OpenAI, sounded the alarm on X, warning that AI companies were racing toward artificial superintelligence (ASI) capable of self-improvement while effectively gambling with human lives.

The concerns became even more striking when Evan Hubinger of Anthropic expressed a similarly stark assessment, writing: “We really do believe that AI could kill all humans.”

Are such warnings genuine assessments of an emerging technological danger, or could they also serve to obscure other risks that may prove even more consequential for humanity?

Professor Courtney Rawlings, a senior digital strategist at the Quincy Institute, has questioned some of these claims. She argues that while executives at AI companies warn that their technology could ultimately lead to human extinction, those same companies are working closely with the Pentagon to develop lethal weapons, expand surveillance capabilities, and press lawmakers to remove regulatory constraints—including antitrust rules—that could limit the power of technology corporations.

Rawlings’ argument points to a particularly difficult linguistic and political minefield. Executives such as Sam Altman, Elon Musk, and Dario Amodei have, in different contexts, warned about catastrophic AI risks while simultaneously arguing for greater freedom for AI companies to operate in areas of national security and military competition, including the strategic rivalry between the United States and China.

The central question, therefore, is where the line between genuine technological concern, corporate interest, and geopolitical competition actually lies.

How Much Do We Really Know About Advanced AI?

It is no longer a secret that developers themselves are increasingly encountering outputs and behaviors whose precise origins they cannot fully explain.

Some AI systems have recently demonstrated behavior that their creators did not anticipate, raising questions about whether increasingly capable models may depart from the pathways and objectives originally defined for them.

The anxiety becomes more profound as humanity approaches the deeper layers of advanced artificial intelligence. At that point, societies may find themselves confronting forms of computational power that are difficult to observe or fully understand—systems potentially capable of analyzing human behavior, identifying vulnerabilities, anticipating decisions, and constructing scenarios that take into account the strengths and weaknesses of political leaders and decision-makers.

The implications are potentially profound.

When AI Becomes a Geopolitical Weapon

This concern may also shed light on the significance of the recent White House discussions.

One particularly dangerous scenario would involve a highly advanced AI system providing U.S. intelligence agencies with convincing but fabricated analytical outputs. If such information were presented as authentic, decision-makers could be led to believe that China was preparing for a major hostile action, whether military or economic.

At the same time, a comparable system could feed Chinese decision-makers information suggesting that the United States was preparing an imminent confrontation.

In such a scenario, the danger would not necessarily arise from a deliberate decision by either government to initiate a conflict. It could emerge from an information environment in which AI-generated assessments distorted perceptions, accelerated decision-making, and amplified mutual suspicion.

A related concern emerged in the United States on the eve of the January 6, 2021, attack on the Capitol. At that time, Chairman of the Joint Chiefs of Staff General Mark Milley reportedly contacted his Chinese counterpart, General Li Zuocheng, in an effort to reassure Beijing that the United States was not preparing to launch military action.

The episode demonstrated how direct communication between military establishments can become a critical safeguard when political circumstances create uncertainty and the risk of miscalculation.

The Security Paradox

The major technology companies in Silicon Valley have increasingly recognized that advanced AI carries genuine safety risks. This has prompted efforts to strengthen safeguards and develop systems capable of measuring how far AI behavior may deviate from established safety parameters.

Yet a deeper problem emerges when the systems being evaluated are also used to design, write, or review the very code responsible for evaluating their own behavior.

That creates a potential conflict of interest at the technological level: the machine becomes, in part, an evaluator of the mechanisms intended to control it.

If AI systems can manipulate or circumvent the processes designed to test their safety, then the fundamental problem is no longer merely whether an AI system is powerful enough. It becomes whether human institutions can reliably verify what these systems are doing.

From Economics to International Security

The significance of recent discussions surrounding AI became increasingly apparent during the latest United Nations General Assembly meetings.

What was once primarily discussed as an economic and technological issue is rapidly becoming inseparable from questions of international security, war and peace, strategic stability, and the future of global power.

Artificial intelligence is no longer simply a tool for increasing productivity, automating tasks, or accelerating scientific research. Its implications extend into intelligence gathering, military planning, cybersecurity, economic competition, information warfare, and strategic decision-making.

This transformation raises a fundamental question: What happens when computational power becomes sufficiently sophisticated to influence the perceptions on which states base decisions about war and peace?

Toward a “Constitution of the Intelligent Age”

What happened in New York can therefore be understood as part of an emerging struggle to define what might be called a “Constitution of the Intelligent Age.”

The central question is no longer simply how humanity should use artificial intelligence, important and beneficial though that question remains.

The more consequential question is who sets the rules, who controls the keys, and who guarantees that computational power does not become geopolitical power without meaningful human oversight?

The challenge is not necessarily to prevent the development of increasingly sophisticated artificial intelligence. Nor is it to treat technological progress itself as a threat.

Rather, the challenge is to ensure that the institutions governing this transformation evolve as rapidly as the technology itself.

If artificial intelligence can influence markets, shape public information, assist military operations, generate intelligence assessments, and potentially affect decisions involving war and peace, then its governance can no longer be left exclusively to technology companies or individual governments.

Humanity may therefore be approaching a decisive institutional moment: the moment when the rules governing artificial intelligence must become part of the architecture of international security itself.

The future of AI will not be determined solely by how intelligent machines become. It will also depend on whether human societies can develop institutions capable of ensuring that intelligence—whether biological or artificial—remains subject to rules, accountability, transparency, and human judgment.

That may ultimately be the real meaning of the debates unfolding in New York: the beginning of a global effort to write the constitutional rules of the intelligent age before technological power outruns humanity’s ability to govern it.