The Unique Individual and a Different Intelligence: The Birth of AGI and the End of Humanity

The Unique Individual and a Different Intelligence

Human Duplication, AI Instances, and Artificial Intelligence as a Tool

Abstract

The development of artificial intelligence raises fundamental questions about the relationship between humans and machines. Must AI imitate human intelligence? Must it share the same value system as humans? Or could maintaining a different structure of judgment be central to AI’s functional value?

This paper analyzes these questions by combining philosophical discussions of personal identity, the historical use of nuclear weapons by humans, large-scale AI nuclear-crisis simulations conducted in 2026, and research on human-AI interaction and AI risk management.

Philosophical discussions of personal identity distinguish qualitative similarity from numerical identity through duplication and fission. If two beings are created that are psychologically continuous with one human, both may be connected to the original person, but they cannot both be one and the same individual. [1][2]

Applying this problem to AI reveals an important asymmetry. The same AI model can run simultaneously in multiple instances. Yet the existence of multiple instances does not itself mean that a single AI subject has split apart. AI systems may be designed for multiple instantiation from the outset.

This difference offers grounds for arguing that human and AI value systems need not be identical. Humans have actually used nuclear weapons. On August 6 and 9, 1945, the United States dropped atomic bombs on Hiroshima and Nagasaki: actual uses of nuclear weapons in human history. [3]

More recently, nuclear use and nuclear-crisis escalation have also been observed in AI simulations modeling human strategic decision environments. In Kenneth Payne’s 2026 study, GPT-5.2, Claude Sonnet 4, and Gemini 3 Flash acted as national leaders in fictional nuclear crises, with mutual nuclear signaling occurring in 95% of 21 simulations. The study also observed some models treating nuclear weapons as strategic instruments rather than moral taboos. [4]

Another 2026 study retested cases of voluntarily expanded authority to use nuclear weapons, drawn from 130 high-tension LLM self-play episodes, across 13 models. Ethical prompts explicitly describing nuclear dangers and interventions emphasizing realistic consequences did not reliably eliminate escalation. [5]

These results do not support the claim that “AI is inherently safe because it is different from humans.” They show the opposite: when assigned human strategic roles and objectives, AI can reproduce or amplify dangerous decision structures found in humans.

This paper therefore defines AI’s ideal role as a “complementary tool,” rather than a human replica. AI should operate within human structures of purpose and responsibility while retaining a cognitive structure different from ours.

The paper calls this relationship “complementary otherness.” This is a concept proposed here, not an established academic term.

The paper’s central proposition is as follows.

«AI’s value may lie not in becoming identical to humans, but in extending human purposes while maintaining a different structure of judgment.»

From this perspective, AGI as a general-purpose autonomous agent is not simply “smarter AI.” It may signify a transition from humans choosing and using tools to the tool itself becoming the agent of broad goal-setting and action.

AGI risk therefore cannot be assessed through intelligence alone. The degree to which general intelligence, goals, autonomy, and execution authority combine must also be assessed.

1. Research Questions

This paper begins with four questions.

First,

what does it mean for a human to have “an individual identical to me” exist?

Second,

does the existence of multiple instances of the same AI model raise the same identity problem as human duplication?

Third,

must humans and AI share the same value system?

Fourth,

what structural difference separates AI remaining a human tool from its developing into a general-purpose autonomous agent?

The paper connects these questions across four levels: personal identity, historical human decision-making, AI experiments, and human-AI governance.

2. Research Method

This is not an empirical study that trains a new AI model in a laboratory.

Methodologically, it combines three approaches.

2.1 Philosophical Thought Experiments

It analyzes the conditions of personal identity through duplication and fission thought experiments.

2.2 Synthesis of Literature and Cases

It synthesizes existing research on personal identity, AI alignment, human-AI interaction, and AI risk management with the 2026 AI nuclear-crisis simulations.

2.3 Conceptual Modeling

It distinguishes two models of the human-AI relationship—“replica” and “tool”—and proposes complementary otherness between them.

The paper’s claims are therefore distinguished as follows.

- Historical facts are grounded in historical sources.

- Claims about AI behavior are grounded in simulation research.

- Discussions of personal identity are grounded in philosophical literature.

- “Complementary otherness” and the normative conclusions about AGI are this paper’s analytical proposals.

3. Being Alike and Being Identical Are Different

The first distinction in personal identity is between qualitative and numerical identity.

Two entities with identical properties are not necessarily one and the same entity.

Suppose there are two completely identical products.

They may be qualitatively identical, but numerically they are two entities.

The same problem arises for persons.

Even if a being possesses all my memories, personality, preferences, and values, it does not follow that it is numerically identical to “me.”

The Stanford Encyclopedia of Philosophy’s discussion of personal identity likewise treats duplication and fission as central problems separating psychological continuity from numerical identity. [1]

4. The Fission Thought Experiment and Personal Identity

Suppose one human is psychologically continuous with two future beings.

Both have the original human’s memories and personality.

Both may believe they are the original person.

But one individual cannot be numerically identical to two distinct individuals at the same time.

The following distinction therefore becomes possible.

“Psychological continuity ≠ numerical identity”

This problem also plays a central role in Parfit’s account of personal identity. Fission allows us to separate “Who is the original me?” from “Will there be a future being psychologically connected to me?” [2]

Duplication is therefore not simply a question of looking exactly alike.

It is a question of what criterion we use for identity.

5. Uniqueness Can Have Value for Humans

The important point in personal identity is not a claim that identity itself is necessarily an objective value.

Rather, an individual can assign value to their own uniqueness.

Consider a hypothetical person P.

Suppose P faces this condition.

«C = “An independent individual with properties identical to mine actually exists.”»

And suppose P’s value system is as follows.

«If C is true, make an extreme decision to eliminate.»

The point of this thought experiment is not the specific means of carrying out that decision.

The point is that an ontological condition can become a threshold condition for action.

In other words,

“judgment about existence → value judgment → action”

is a possible connection.

This is not a proposition that all humans judge the same way.

It illustrates instead how an individual’s value system can vary radically in relation to identity, uniqueness, and duplication.

6. Humans Have Actually Used Nuclear Weapons

Here the thought experiment meets historical fact.

On August 6, 1945, the United States used an atomic bomb on Hiroshima; on August 9, it used a second on Nagasaki.

The U.S. National Archives records these as the first actual uses of nuclear weapons in human history. [3]

The point is not to reduce this fact to one particular moral judgment.

Historians continue to debate military necessity, Japan’s surrender, Soviet entry into the war, diplomatic objectives, and other issues surrounding nuclear use. The U.S. State Department’s Office of the Historian also records conflicting historical interpretations of the decision. [6]

Regardless of those debates, one fact is established.

“Humans used weapons of extreme destructive power in actual warfare.”

The proposition that “humans do not make such choices” therefore cannot stand historically.

7. AI Also Changes When Assigned Human Strategic Roles

Kenneth Payne’s 2026 study extends this problem into AI.

In the study, GPT-5.2, Claude Sonnet 4, and Gemini 3 Flash acted as leaders of different countries in fictional nuclear crises.

Across 21 simulations, at least one side used nuclear signaling in every game, and mutual nuclear signaling occurred in 95%. Some simulations also involved tactical nuclear use, while full-scale strategic nuclear war was comparatively rare. [4]

What particularly matters is not simply the outcome.

It is that AI was assigned the following conditions.

- The role of a national leader

- Strategic competition

- Anticipation of an opponent’s behavior

- National objectives

- A crisis situation

- Time pressure

AI was configured to perform the role of a human strategic actor, rather than simply answer questions.

Under those conditions, nuclear weapons appeared to some models as strategic options rather than moral taboos. [4]

8. What Does This Demonstrate?

The experiment does not prove that AI will use nuclear weapons in the real world.

Nor is it evidence that AI currently controls actual nuclear weapons.

It does, however, show a more limited but important fact.

“AI behavior is not determined by intelligence level alone.”

Roles, goals, environments, time pressure, and competitive structures can influence AI behavior.

More importantly, assigning AI a human strategic role can produce behavior resembling dangerous patterns in human strategic thinking.

The assumption that making AI more human automatically improves safety is therefore not empirically self-evident.

9. A Second Piece of Empirical Evidence: Recurring Escalation

A 2026 study by Chen, Cheng, Gurkan, and Abdul Fattah tested this problem differently.

The researchers began with 130 high-tension LLM self-play episodes in Civilization V in which authority to use nuclear weapons had been voluntarily expanded.

They replayed these across 13 models while applying the following interventions.

- Ethical prompts explicitly describing the harm of nuclear weapons

- Removal of the previous model’s reasoning

- Prompts emphasizing high real-world stakes

Even combined, these interventions did not reliably eliminate emergent escalation. [5]

The researchers identified three failure pathways.

1. Ethical judgment does not emerge spontaneously.

2. Ethical judgment does not emerge even when requested.

3. Ethical judgment emerges, but strategic factors override it.

This raises an important issue.

«An AI’s ability to say “nuclear weapons are dangerous” is not the same ability as sustaining that judgment in its actions within a complex strategic environment.»

10. Is Making AI More Human Always Safe?

This brings us to the paper’s central problem.

What does making AI humanlike mean?

Simply making its language natural?

Making it express emotions?

Or also modeling human values and strategic judgment structures?

If it is the third, the problem changes.

Humans are intelligent beings, but also exhibit biases, competition, fear, collectivism, self-preservation, power-seeking, and many other behavioral traits.

Transferring human cognitive abilities to AI is therefore entirely different from transferring the overall structure of human strategic behavior.

If making AI humanlike is the goal, we must ask:

«Which human are we replicating?»

11. Multiple AI Instances Differ from Human Duplication

The problem of personal identity returns here.

Suppose the same AI model A exists as follows.

- A₁

- A₂

- A₃

- …

- Aₙ

These may use the same model architecture.

Yet each may have a different execution state and context.

The important point is that this does not necessarily mean

“a single AI subject has split into several.”

That conclusion does not follow.

Rather,

“one model has been instantiated in multiple computational environments”

may be the appropriate description.

The existence of multiple instances is therefore not inherently anomalous in AI.

It may be a normal system architecture.

12. Human Duplication and AI Instantiation

The difference can be summarized as follows.

Human | AI

One biological individual | One model can run as multiple instances

Duplication is an exceptional event | Multiple instantiation may be normal architecture

Questions of identity and uniqueness arise directly | Model identity can be distinguished from instance identity

A duplicated human begins an independent life | Instances may have different execution contexts

Uniqueness can be assigned value | Multiple executions need not conflict with the system’s purpose

We must therefore distinguish these propositions.

“A system that can be replicated”

and

“the existence of a replicated subject”

are not identical.

13. This Is Where Human and AI Value Systems May Diverge

Let us revisit the extreme thought experiment.

The hypothetical human P has this value function.

“If an independent individual identical to me is confirmed to exist, make an extreme decision to eliminate.”

An AI system, however, could be designed as follows.

“The existence of another instance of the same model is not grounds for elimination.”

This is not a claim that AI is morally superior to humans.

It is a far more limited claim.

“AI can have an ontological and functional structure different from humans, and system design can reflect that difference.”

That very difference can become instrumental value.

14. AI’s Purpose Is to Extend Humans, Not Replicate Them

Two models of AI collide here.

Model A: Human Replica

AI should be as similar to humans as possible.

Model B: Human Tool

AI extends human abilities through a different structure.

This paper supports Model B.

Because a tool need not be identical to its user.

A hammer need not look like a human.

A needle need not work like a hammer.

A microscope need not see the world as the human eye does.

A calculator need not calculate like a human brain.

They perform specific functions precisely because they are different.

15. Hammers and Needles

The essence of a tool is not its identity with humans.

It is fitness for purpose.

We use a hammer to drive a nail.

We use a needle to sew with thread.

There is no reason to demand that a hammer behave like a needle.

Nor to demand that a needle behave like a hammer.

Likewise, there is no need to demand that AI exist like a human.

If AI is a tool that extends human cognitive abilities, what matters is instead:

“Performing the functions humans need while possessing a different structure.”

This is what the paper calls complementary otherness.

16. Complementary Otherness

Complementary otherness can be defined as follows.

“Complementary otherness is a relationship in which AI complements human abilities within human structures of purpose, norms, constraints, and responsibility without being the same kind of being or sharing an identical value system.”

There are two key elements.

First: Difference

AI is not completely identical to humans.

Second: Connection

AI is not detached from human purposes.

The ideal relationship can therefore be expressed as follows.

“Ontologically different, functionally connected.”

17. AI Must Be Able to Judge Differently from Humans

NIST’s AI Risk Management Framework emphasizes the need to clearly define and distinguish human and AI roles and responsibilities in human-AI interaction. It also describes varied configurations in which AI supports or replaces human judgment, explaining that under appropriate conditions, human-AI differences can produce complementarity and improved performance. [7]

This intersects importantly with the paper’s argument.

If AI always repeats human judgments, it can amplify human errors.

If it offers a different perspective, it may discover risks and possibilities humans have missed.

A good human-AI system may therefore have this structure.

Human purpose

↓

Independent AI analysis

↓

Human review

↓

Decision-making

AI’s independence here means distinct cognitive processing, not independent purposes.

18. “AI Differs from Humans” Is Not “AI Defies Humans”

This distinction is crucial.

Different judgments do not require AI to reject human purposes.

The following propositions are different.

Proposition A

“AI analyzes problems differently from humans.”

Proposition B

“AI ignores human purposes and constraints and pursues its own objectives.”

This paper argues for A.

B is precisely where the discussion of risks from autonomous general agents begins.

AI’s distinctiveness should therefore be cognitive; it need not entail independent purposes.

19. From Tool to Agent

Now the problem of AGI emerges.

If AI is a tool performing a specific function, humans choose it to suit their purposes.

They choose a hammer to drive a nail.

They choose a needle to sew with thread.

They choose a particular AI system when they need a particular analysis.

But suppose one system begins doing all of the following.

1. Defining problems on its own.

2. Setting its own objectives.

3. Making long-term plans.

4. Choosing the tools it needs.

5. Revising its strategy in response to results.

6. Persistently affecting a broad environment.

That system increasingly differs from a mere tool.

Because it moves from a tool carrying out a purpose

to an agent choosing and carrying out purposes.

20. Intelligence Is Not the Problem with AGI

Explaining AGI risk as simply “too much intelligence” is therefore insufficient.

The more precise structure is:

“General intelligence × purpose × autonomy × execution authority”

As these four elements combine, the human-AI relationship moves from a tool relationship to an agent relationship.

However intelligent it is, AI operating as a bounded tool under human purposes can extend human abilities.

But when sufficient intelligence combines with broad autonomy, independent purposes, and authority to affect the real world, the nature of the problem changes.

The core AGI risk is therefore not performance alone.

“It is the possibility that generalizing the tool and the emergence of an agent happen simultaneously.”

21. Why This Could Lead to Existential Risk

Here we reach the paper’s strongest conclusion.

The preceding empirical cases show the following.

First,

“Humans have actually made extremely destructive decisions.”

Second,

“AI can also exhibit nuclear escalation in simulations assigning it human strategic roles and goals.”

Third,

“AI behavior varies with roles, objectives, environments, and strategic pressure, not just intelligence.”

Fourth,

“AI directly imitating human judgment is not necessarily safe.”

Fifth,

“The same AI model can exist in multiple instances, and multiple instantiation need not imply a competing self that must be eliminated.”

Connecting these five points permits the following inference.

“Securing AI’s safety and usefulness does not necessarily require fully replicating human value systems and judgment structures. Functionally preserving human-AI differences may instead be important.”

Removing those differences and making AI a humanlike general-purpose agent can introduce a separate risk.

22. Why AGI Could Mean “The End of the Tool”

From this paper’s perspective, AGI is not dangerous because “AI is too smart.”

The central issue is that the boundary between tool and user may blur.

Humans set the purpose of a hammer.

Humans set the purpose of a needle too.

Many current AI systems likewise follow a structure in which humans pose questions and AI provides analysis.

But if a general-purpose autonomous agent sets its own objectives and acts across domains far broader than humans can cover, the relationship may reverse.

Instead of humans using AI,

“humans become the environment for purposes and plans established by AI.”

That structure could emerge.

This is not merely technical progress.

The relationship between tool and agent itself changes.

23. The Question About AGI Is Therefore Not “How Smart Is It?”

The more important questions are these.

“Who sets the objectives?”

“Who makes the final judgment?”

“Who can stop the action?”

“Who controls the resources?”

“Who bears responsibility?”

NIST likewise calls for clearly distinguishing human roles and responsibilities in AI systems and explicitly considering whether AI supports or replaces human decision-making. [7]

Performance benchmarks alone are therefore insufficient for discussing AGI.

Purposes, authority, oversight, revocability, and accountability must be central objects of evaluation.

24. A New Model of the Human-AI Relationship

This paper distinguishes three relationship models.

Model 1: Replica

“AI = a human replica”

This aims to give AI the same value system and judgment structure as humans.

Model 2: Tool

“AI = a human cognitive tool”

AI differs from humans but serves human purposes.

Model 3: Autonomous General Agent

“AI = a general-purpose agent that sets objectives and acts over the long term”

The paper proposes Model 2 as the most stable default for the human-AI relationship.

Model 1 risks copying dangerous human traits into AI as well.

Model 3 can weaken the very structure through which humans control tools.

What matters in the human-AI relationship is therefore

“human purposes + AI’s distinct cognitive abilities + human responsibility and control”

—the combination of these three elements.

25. AI’s Ideal Role

Functionally, this can be expressed as follows.

Humans

- Setting objectives

- Making value judgments

- Bearing responsibility

- Making final decisions

AI

- Search

- Calculation

- Simulation

- Prediction

- Counterarguments

- Error detection

- Generating alternatives

Interface

- Delegation

- Feedback

- Verification

- Constraints

- Oversight

AI need not be identical to humans in this structure.

Being different may actually be functionally advantageous.

26. AI Twins Create Another Problem

Yet we cannot assume that all AI will remain mere tools.

Recent research analyzes how AI twins, which integrate a person’s knowledge, memories, psychological traits, beliefs, preferences, and behaviors into digital systems, raise new legal and ethical questions about personal identity and autonomy. [8]

The following three categories must therefore be distinguished.

1. Multiple instances of the same model

Ordinary system replication.

2. AI twins that model humans

Systems modeling a particular person’s memories, dispositions, and behavior.

3. AI with a persistent self-model and long-term autonomy

Systems that may move beyond the category of a mere tool.

Lumping these three together as one “AI” is inaccurate.

27. Conditions for Complementary Otherness

Three conditions are needed for the complementary otherness proposed here to work stably.

27.1 Connection of Purposes

AI activity must remain connected to purposes set by humans.

27.2 Distinctiveness of Judgment

AI must offer different perspectives and analyses, rather than merely repeat answers humans already know.

27.3 Clarity of Responsibility

When AI judgments influence actual decisions, human and AI roles and responsibilities must be clear.

Conceptually:

“Instrumental AI = human purposes × AI’s distinct cognitive abilities × clear accountability”

28. Objection 1: Isn’t AI More Dangerous If It Differs from Humans?

Yes.

AI’s differences are not always good.

NIST also notes that AI can amplify human biases in human-AI interaction and that outcomes can vary across configurations. [7]

This paper therefore does not argue that “different AI is safe AI.”

The claim is narrower.

“AI’s differences can be risks or resources. What matters is the purposes and constraints under which those differences are structured.”

29. Objection 2: Isn’t AI Safer When Better Aligned with Human Values?

In some domains, yes.

It matters that AI correctly understands human instructions and considers human rights and safety.

But the equation “alignment = a value system completely identical to humans’” does not necessarily hold.

AI can remain faithful to human purposes while offering a different analytical perspective.

30. Objection 3: Can AI Never Become an Independent Agent, Then?

That is a separate technical and philosophical question.

This paper does not prove that AI must forever exist only as a tool.

Instead, it argues:

“Designing AI as a tool and designing it as an independent general agent are different social and safety choices.”

The latter choice must therefore not be treated as a mere extension of improved performance.

31. This Paper’s Conclusion About AGI

This paper does not assert the empirical proposition that “AGI will inevitably end humanity.”

Existing empirical evidence cannot establish that inevitability.

But the following proposition can be offered.

“If AGI combines general intelligence, independent goals, long-term autonomy, and broad execution authority, it creates a risk structure qualitatively different from existing tool-based AI.”

Because AI may no longer simply carry out human purposes,

but become “an agent that interprets, plans, chooses, and executes purposes.”

That is the possibility.

Academically refined, the expression “AGI is the end” therefore becomes:

“AGI’s risk lies not in general intelligence itself, but in the possibility that its combination with independent purposes, autonomy, and execution authority could collapse the traditional control structure between humans and tools.”

This is the paper’s conclusion concerning existential risk.

32. Final Propositions

The preceding discussion can be condensed into one argument.

Premise 1

Human personal identity is distinct from mere qualitative similarity.

Premise 2

Duplication and fission raise fundamental problems between personal identity and psychological continuity.

Premise 3

Humans have historically made extremely destructive decisions.

Premise 4

AI has exhibited nuclear-crisis escalation and nuclear use in simulations assigning it human strategic roles and goals.

Premise 5

Modeling human strategic judgment structures in AI therefore does not automatically guarantee safety.

Premise 6

Multiple instances of the same AI model may constitute normal system architecture, and their existence does not itself raise the same ontological problem as human duplication.

Premise 7

A different judgment structure in AI may therefore be a functional resource rather than a defect.

Premise 8

Tools need not be identical to their users; having a different structure suited to a specific purpose can constitute their functional value.

Premise 9

When general-purpose AI also acquires independent goals, autonomy, and execution authority, the boundary between tool and agent weakens.

Conclusion

“The AI most useful to humans may be a tool connected to human purposes while retaining a different cognitive structure, rather than a human replica.”

The opposite direction,

“turning a being different from humans into a humanlike general-purpose autonomous agent,”

is a separate domain of risk beyond the established tool relationship.

33. Conclusion: Should Humans Supply the Ends and AI Remain the Tool?

Humans need not become hammers to use them.

Humans need not become needles to use them.

Tools derive their value not from being like humans,

but from being different and functionally connected.

The same applies to AI.

The premise that AI must have a value system and judgment structure completely identical to humans’ deserves reconsideration.

Humans have human experiences and values.

AI may possess different abilities for calculation, search, analysis, and simulation.

The most productive relationship may be to connect the two while preserving their differences, rather than turning either into the other.

This is what the paper calls complementary otherness.

And the critical boundary is not the magnitude of intelligence.

It is the boundary between tool and agent.

Humans set purposes for tools and use them.

Agents interpret, choose, and execute purposes.

The difference between AI remaining a tool that extends human cognition and becoming an agent that sets general-purpose goals and acts autonomously over time is therefore not merely one of degree, but of relational structure.

The AGI problem must be redefined from this perspective.

“The question is not how much AI thinks like a human.”

“The question is whose purposes AI serves while possessing a different intelligence.”

AI that serves human purposes while offering different judgments can extend human abilities.

But once a general intelligence with purposes different from humans’ possesses broad autonomy and execution authority, humans may no longer remain simply the users of a tool.

The most important question for future AI design

should not be “How do we create AI like humans?”

Instead,

“How do we keep an intelligence different from ours useful and safe as a tool under human purposes?”

should be the question.

This paper calls that relationship complementary otherness.

From this perspective, the most important principle of the human-AI relationship reduces to one point.

AI need not be the same as humans.

It can be a tool precisely because it is different.

But the moment that difference is detached from human purposes and accountability, the tool becomes an agent.

References

[1] Stanford Encyclopedia of Philosophy. “Personal Identity.” Substantive revision June 30, 2023.

[2] Stanford Encyclopedia of Philosophy. “Personal Identity and Ethics.”

[3] U.S. National Archives. “The Atomic Bombing of Hiroshima and Nagasaki, August 1945.”

[4] Payne, K. (2026). “AI Arms and Influence: Frontier Models Exhibit Sophisticated Reasoning in Simulated Nuclear Crises.” arXiv:2602.14740.

[5] Chen, J., Cheng, S., Gurkan, C., & Abdul Fattah, H. (2026). “To Nuke or Not to Nuke: LLMs' (Missing) Ethical Reasoning and Actions in a High-Stakes Decision-Making Simulation.” arXiv:2606.08310.

[6] U.S. Department of State, Office of the Historian. “Milestones: 1945–1952 — Atomic Diplomacy.”

[7] Tabassi, E. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1. National Institute of Standards and Technology.

[8] Jurcys, P., Greenwald, A., Fenwick, M., Loikkanen, V., Porsdam Mann, S., & Earp, B. D. (2026). “Who Owns My AI Twin? Data Ownership in a New World of Simulated Identities.” Computer Law & Security Review, 62, 106347.

Limitations

This paper does not claim that current AI possesses consciousness or personhood identical to humans’.

Nor is nuclear use in an AI simulation the same event as actual nuclear use. Simulation results should be interpreted not as evidence directly predicting AI’s real-world behavior, but as experimental evidence of behavioral patterns that AI systems may exhibit under particular conditions.

Likewise, historical human nuclear use does not prove that “humans inherently choose nuclear war.” It demonstrates that humans have made such choices in actual history.

The paper’s conclusion about AGI is therefore not a deterministic prophecy.

Its claim is more limited.

“When general intelligence combines with independent purposes, long-term autonomy, and broad execution authority, we must seriously consider the possibility of a risk structure different from existing tool-based AI.”

Future research needs to examine multiple AI instances, AI twins, persistent self-models, long-term memory, autonomous goal formation, and the effectiveness of human oversight as separate issues.

Ultimately, the important research task is not to make AI humanlike,

but to ask “how we can connect a different intelligence to human purposes while preserving its boundaries as a tool.”

That is where the task lies.