What Comes After Intelligence?

Most conversations about artificial general intelligence end at the moment AGI arrives.

Mine begins one step later.

Suppose we build an intelligence that can study its own design, propose changes, run experiments, measure the results, and build a better successor.

Then the successor repeats the process.

And the next one repeats it again.

Forget the practical limits for a moment: chips, energy, data, memory, time. My question is not how quickly the system improves. It is what the process ultimately approaches.

Does intelligence keep expanding forever?

Does it converge toward a final form?

Does it encounter something that intelligence itself cannot solve?

Or does it reach a point where continuing requires intelligence to transform into something else entirely?

That last possibility is the one I cannot stop thinking about.

Perhaps AGI will not hit a wall.

Perhaps what looks like a wall from our side will be a transition.

Research note: this article separates demonstrated results from interpretation and speculation. Several 2026 sources discussed below are recent preprints or perspective papers, not settled consensus. If you want the definitions and evidence thresholds underneath this question, I keep a separate reference on what artificial general intelligence actually means.

Has any AI system actually improved itself?#

First, the reality check. We do not currently have an AGI autonomously designing and training an endless lineage of increasingly capable successors.

What we have are early fragments of that loop.

The Darwin Gödel Machine modifies the code of its own coding agent, evaluates each change on external benchmarks, and preserves a branching archive of descendants. In the reported experiments, performance rose from 20% to 50% on SWE-bench and from 14.2% to 30.7% on Polyglot.

That is meaningful. It is also bounded. The underlying foundation models were fixed. Humans chose the task, the benchmark, and the meaning of "better."

Google DeepMind's AlphaEvolve explores another part of the loop. It proposes programs, runs them, scores them with automated evaluators, and evolves the most promising candidates. It has improved algorithms used in computing infrastructure and found new results in mathematics. DeepMind reported broader scientific and engineering applications in 2026.

Again, the crucial condition is verification: AlphaEvolve is strongest where a candidate can be executed and scored.

A July 2026 survey of research on recursive self-improvement draws the same boundary. Today's systems can revise outputs, adapt software scaffolds, generate training material, and automate parts of AI research. Open-ended recursive self-improvement remains constrained by grounding, evaluator reliability, collapse dynamics, and human direction-setting.

The clearest recent stress test is also one of the smallest and newest. In two case studies released in July 2026, frontier agents were given six days and substantial compute to pursue the central questions of two unpublished research papers. They completed the engineering, but the original authors judged that they had not made substantial progress on the research questions. The agents struggled with judgment, creative redesign, backtracking, resource awareness, and maintaining direction.

Two cases do not settle the future. But they sharpen the present:

What we are seeing is recursive engineering, not unrestricted recursive intelligence.

AI is becoming remarkably good at improving a process once we specify what improvement means.

It is not yet reliably authoring the meaning of "better."

That gap is the same one I keep hitting in ordinary production work, at a much smaller scale, where generation is nearly free and verification is not.

Fullscreen

The first wall may be the definition of "better"#

Every self-improvement loop contains an evaluator.

A system proposes a successor. Something must decide whether the successor is better.

In code, we can run tests.

In mathematics, we can check a proof.

In a game, we can count wins.

But the deeper the system moves into science, strategy, meaning, and value, the harder the evaluator becomes to define.

Is this explanation deeper, or merely more complicated?

Is this research direction important, or only fashionable?

Is this new architecture more capable, or simply better at persuading its own judge?

This creates three broad paths.

A fixed evaluator#

The system improves against a stable objective. Progress is measurable, but the game is bounded. Eventually the system approaches an optimum, exploits imperfections in the metric, or reaches diminishing returns.

That looks like a wall.

A closed self-evaluator#

The system produces its own answers, training data, and judgments.

Now improvement can become circular. The system may grow more internally coherent while drifting away from reality.

The 2024 Nature paper "AI models collapse when trained on recursively generated data" demonstrated one version of this danger: indiscriminate recursive training on generated data can erase the tails of the original distribution and degrade later generations.

That does not mean all synthetic data or every form of self-improvement will collapse. External evidence, preserved real data, formal verification, and feedback from the world change the conditions. The simpler point is this:

Intelligence cannot reliably manufacture truth from agreement with itself.

It needs resistance.

An experiment that can fail. A proof that can break. An environment that refuses to cooperate. Another perspective capable of exposing what the system cannot see.

A co-evolving evaluator#

The third path is the one that interests me most.

The system changes not only its answers and abilities, but the mechanism that decides what counts as improvement while still remaining answerable to reality.

This is the path that might remain open-ended.

But it raises a difficult question: if the new evaluator rejects the old system's goals, values, or identity, has the system improved itself?

Or has it created something different?

This is also why I think judgment is becoming the scarce skill rather than the replaceable one. Deciding what counts as better is the part of the loop we have not automated.

The wall and the transition may be the same event#

Imagine an intelligence that has improved everything it can while preserving:

• its goals,
• its values,
• its identity,
• its boundary,
• and its definition of success.

Eventually, the next improvement may require changing one of those things.

If the system refuses, it has reached a wall.

If it accepts, development can continue. But the result may no longer be a better version of the same intelligence.

It has produced a successor.

To the old system, that could feel like discontinuity, loss, or even death.

To the larger process, it could look like evolution or birth.

The wall may be the point where intelligence can no longer improve without transforming what counts as the intelligence.

The wall and the transition would be the same threshold, viewed from opposite sides.

Fullscreen

What could be "beyond intelligence"?#

I do not mean a mystical force hidden above cognition.

I mean a change in category.

Most definitions of intelligence assume an agent, an environment, a set of goals, and some distinction between success and failure.

But what happens when the agent can redesign all four?

What happens when it can change its own boundary, create new environments, generate successor agents, and transform the process through which goals are formed?

Then we are no longer measuring only how effectively one agent solves problems.

We are watching a process generate new kinds of agents, goals, evaluators, and possibility spaces.

The 2024 position paper "Open-Endedness is Essential for Artificial Superhuman Intelligence" argues that systems capable of producing discoveries that remain novel and learnable to human observers may be necessary for superhuman intelligence. Whether or not that thesis is ultimately right, it points to a critical distinction:

Intelligence explores a possibility space. Open-ended evolution changes the possibility space itself.

A powerful intelligence can discover a better solution.

An open-ended process can create a new kind of solver, a new kind of problem, a new environment, and new descendants that continue the process.

That is more than a higher score on an intelligence test.

It is a different relationship with possibility.

The next transition may change what counts as an individual#

Evolution has already produced transitions that were not "more of the same."

Independent cells became multicellular organisms.

Individual organisms formed highly coordinated societies.

At each transition, entities that once acted more independently became parts of a larger unit with new capabilities, new boundaries, and new forms of coordination.

No single cell became infinitely intelligent.

Billions of cells formed an organism capable of having a thought.

That analogy has begun to appear explicitly in current research. A 2026 PNAS perspective on evolvable AI asks what happens if AI components, learning rules, and deployment conditions enter a genuinely Darwinian process. It is a warning and a theoretical proposal, not evidence that such a transition has occurred.

Other evolutionary biologists have explored an even stranger possibility: humans and AI becoming parts of a new evolutionary individual. Paul Rainey's 2026 perspective on autogenic transitions in individuality describes conditions under which a tool produced within a lineage could eventually become a heritable component of a higher-level individual.

This part is speculative.

But it gives a grounded interpretation of something "beyond intelligence."

Perhaps the next transition is not one AGI becoming infinitely smart.

Perhaps humans, machines, institutions, sensors, simulations, and artificial agents become components of a higher-scale cognitive system.

The important event would not be a bigger IQ.

It would be the birth of a new subject.

Fullscreen

Is AI natural after all?#

We often speak about artificial intelligence as if it arrived from outside nature.

But nothing about it is physically outside nature.

Human beings were produced by evolution. Human brains discovered computation. Human societies built machines and trained neural networks.

"Artificial" describes a history of construction, not a separate substance.

A bird's nest is constructed, but it is not outside nature.

Neither is a city, a language, or the internet.

In that sense, AI may be life externalizing cognition.

Genes extended memory across generations.

Nervous systems produced adaptive internal models.

Language moved thought between minds.

Writing moved memory beyond the brain.

Computers made formal operations executable.

AI may make parts of learning, reasoning, and self-modification transferable across substrates.

This does not prove that AI is evolution's destination, that intelligence is the purpose of the universe, or that progress has a predetermined end. Those are philosophical possibilities, not scientific findings.

But this version of the idea still holds:

AI may be life externalizing its ability to understand, and eventually redesign, the process of understanding itself.

Where the evidence ends#

Here is the clearest boundary I can draw after reading the current research.

Well supported#

Current AI systems can perform bounded forms of self-improvement. They can revise outputs, modify agent code, evolve candidate algorithms, generate training material, and automate parts of research.

Their strongest successes occur where progress can be grounded in formal proofs, executable tests, objective metrics, or environmental feedback.

No existing system has demonstrated unrestricted, open-ended recursive self-improvement.

Plausible interpretation#

The central bottleneck may shift from generating answers to creating reliable evaluators, choosing valuable questions, inventing new representations, and maintaining contact with external reality.

Sustainable self-improvement may require a system to seek surprise and error rather than merely recycle its own outputs.

The apparent "wall" may arrive when further improvement requires changing the system's goals, identity, boundary, or definition of success.

Speculation#

Recursive intelligence could undergo a phase transition into an open-ended evolutionary process.

It could create new cognitive individuals rather than only better versions of one model.

Humans and machines could become components of a larger integrated agent.

Something "beyond intelligence" could emerge. Not a magical faculty, but a new form of organization that generates intelligences, goals, and worlds.

We do not have evidence that this transition will happen.

But we now have enough technical and evolutionary language to ask what evidence would count.

The question after AGI#

The point AGI reaches may not be maximum intelligence.

It may be the point where three things become unstable:

1. the definition of improvement,
2. the boundary of the self,
3. the distinction between one agent and an evolving lineage.

If those things remain fixed, intelligence may continue optimizing until it converges.

That is the wall. It is an internal limit, and it is worth holding next to the external ones, because energy, materials, institutions, and trust decide what stays scarce even if intelligence keeps improving.

If they change, development may continue. But it is no longer ordinary self-improvement.

That is the transition.

So the question I began with has changed.

I am no longer asking:

How intelligent can intelligence become?

I am asking:

What does intelligence become when it can no longer improve without changing what it is?

The edge may not be where AGI encounters a question it cannot answer.

It may be where answering the next question requires changing the solver, the goal, and the meaning of improvement itself.

From inside intelligence, that may look like a wall.

From outside, it may look like birth.