31 Evolutionary AI: Coevolution, Open-Ended Search, and Autonomous Discovery

The Philosophy of Evolutionary Design

There is grandeur in this view of life. With those words, Charles Darwin closed the most revolutionary scientific treatise ever written, a book that replaced the need for a cosmic designer with three breathtakingly simple principles: variation, selection, and heredity [30]. From these three ingredients alone, operating over geological time, emerged the staggering diversity of the biosphere – the compound eye of the mantis shrimp that perceives sixteen colour channels where humans see three, the echolocation system of the bat that constructs three-dimensional maps from reflected sound, the immune system that generates antibodies against pathogens it has never encountered. No blueprint. No foresight. No gradient.

This chapter asks a question that has haunted computer scientists since the dawn of the field: can we harness the same blind, undirected, yet astonishingly creative process that produced biological intelligence to produce artificial intelligence? The answer, as we shall see, is not merely “yes” but something far more interesting. Evolution is not just an optimisation algorithm to be placed alongside gradient descent and simulated annealing in the practitioner's toolbox. It is something qualitatively different: a process capable of open-ended creation, of discovering solutions that no human designer would have conceived, of inventing the very representations and objectives that make learning possible.

We begin with philosophy because evolution demands it. To understand evolutionary approaches to AI, one must first understand why evolution is different from every other optimisation method, and why that difference matters profoundly for the future of generative modelling.

Darwin's Dangerous Idea: Variation, Selection, Heredity

Daniel Dennett called natural selection “Darwin's dangerous idea” because it acts as a “universal acid” – a concept so powerful that it eats through every traditional explanation for the existence of complex, purposeful design [31]. Before Darwin, the argument from design seemed unassailable: a watch implies a watchmaker; an eye implies an eye-maker. Darwin showed that a purely algorithmic process, requiring no intelligence, no foresight, and no understanding, can produce structures of arbitrary complexity given sufficient time and a source of heritable variation.

The three pillars of Darwinian evolution are deceptively simple:

1. Variation. Individuals in a population differ from one another. These differences arise from random mutations, recombination during sexual reproduction, and environmental perturbations. In computational terms, variation is the exploration mechanism that injects novelty into the search process.

2. Selection. Some variants are better suited to their environment than others. “Better suited” means, in the most stripped-down formulation, that they leave more copies of themselves in the next generation. Selection is the exploitation mechanism that retains and amplifies successful designs.

3. Heredity. Offspring resemble their parents. The traits that made a parent successful are transmitted – with some fidelity, but not perfectly – to the next generation. Heredity is the memory mechanism that allows the population to accumulate adaptations over time.

What makes this triad so powerful is not any single component but their interaction. Variation without selection is random drift. Selection without variation is stasis. Both without heredity are Sisyphean: each generation starts from scratch. Only when all three operate together does the ratchet of cumulative adaptation engage, and complexity begins its inexorable ascent.

The philosophical import of Definition 1 is profound. It tells us that design – in the sense of functional organisation adapted to a purpose – does not require a designer. It requires only an evolutionary system operating for sufficient time. This insight liberates the AI researcher from a deep constraint: the assumption that intelligence must be hand-crafted. If evolution can design brains, perhaps it can also design neural networks, loss functions, training algorithms, and even the search processes that discover these components.

The Fitness Landscape Metaphor

In 1932, the population geneticist Sewall Wright introduced one of the most influential metaphors in all of biology: the adaptive landscape [32]. Imagine a vast, multidimensional terrain where each point represents a possible genotype and the height at that point represents the organism's fitness. Evolution, in this picture, is a population of hill climbers navigating a rugged landscape, seeking peaks while avoiding the treacherous valleys between them.

The metaphor is more than poetic. It provides a geometric framework for reasoning about the difficulty of evolutionary search, the role of population size, and the tension between exploration and exploitation.

The crucial question for any evolutionary search is: how rugged is the landscape? A smooth, single-peaked landscape is trivially navigable by hill climbing. A landscape with many peaks of similar height, separated by deep valleys, is a combinatorial nightmare. Stuart Kauffman's NK model provides a tuneable framework for studying this question [1].

The parameter $K$ controls the ruggedness of the landscape. When $K=0$, there is no epistasis: each locus contributes independently to fitness, and the landscape has a single peak reachable by greedy hill climbing. When $K=N-1$, every locus interacts with every other, the fitness function is essentially random, and the landscape is maximally rugged.

The NK model teaches a lesson that resonates throughout this chapter: the difficulty of an optimisation problem depends not only on the size of the search space but on the structure of interactions between components. When components are tightly coupled (high $K$), the landscape becomes a fractal wilderness of competing peaks. When they are modular (low $K$), the landscape is smooth and navigable. This observation will reappear when we study neuroevolution in Neuroevolution, where modular network architectures are easier to evolve than monolithic ones.

Why Evolution Matters for AI

Why should a practitioner of deep learning, armed with automatic differentiation and GPU clusters, care about evolution? The answer lies in the fundamental limitations of gradient-based optimisation and the unique capabilities that evolutionary methods bring to the table.

The tyranny of differentiability.

Gradient descent, for all its triumphs, is shackled by a non-negotiable requirement: the objective function must be differentiable with respect to the parameters. This requirement is so deeply embedded in modern deep learning practice that we rarely pause to consider how constraining it is. It forces us to use smooth activation functions (or piecewise-linear approximations with well-defined subgradients), differentiable loss functions, and continuous parameter spaces. It prevents us from optimising over discrete choices: which layers to include, how to connect them, what data augmentation to apply, which loss function to use. Evolution faces no such constraint. An evolutionary algorithm can optimise over any space – discrete, continuous, mixed, structured, or unstructured – as long as candidate solutions can be evaluated.

The prison of fixed architecture.

When we train a neural network with gradient descent, we optimise the weights within a fixed computational graph. The architecture – the number of layers, their widths, connectivity patterns, activation functions – is frozen before training begins. This is akin to asking an engineer to optimise the fuel mixture of an engine without being allowed to modify the engine's design. Evolution operates at a higher level of abstraction: it can modify the architecture itself, adding or removing components, changing their connectivity, and even inventing entirely new computational primitives. This is the domain of neuroevolution, which we explore in Neuroevolution, and it connects directly to the neural architecture search methods discussed in 30.

The trap of local optima and mode collapse.

A single gradient-following trajectory is inherently local: it can only reach the nearest basin of attraction. Populations of diverse solutions, evolving in parallel, explore multiple basins simultaneously. Moreover, the population itself acts as a form of implicit memory, preserving diverse strategies that may become useful as the environment changes. This population-level diversity is the evolutionary antidote to the mode collapse that plagues generative adversarial networks and other generative models.

Evolution as meta-learning.

Perhaps the most profound reason to study evolutionary approaches is that evolution does not merely optimise a solution within a fixed framework – it optimises the framework itself. Biological evolution did not simply find good neural network weights for the brains of organisms; it invented the brain. It invented learning algorithms (synaptic plasticity), memory systems (the hippocampus), attention mechanisms (the superior colliculus), and generative models (the predictive processing hierarchy of the neocortex). In the language of machine learning, evolution is the ultimate meta-learner: it learns the learning algorithm.

Biological Foundations

Before we can build evolutionary algorithms that match the creative power of nature, we must understand the biological principles that make natural evolution so extraordinarily effective. This section is not a biology textbook – it is a targeted extraction of the ideas from evolutionary biology that have the deepest implications for algorithm design. We shall encounter Fisher's fundamental theorem, which reveals the mathematical engine driving adaptation; the Red Queen hypothesis, which explains why coevolution prevents complacency; punctuated equilibrium, which suggests that the most important evolutionary events are rare and sudden; and the Baldwin effect, which bridges the chasm between learning and evolution.

Each of these biological insights will become, in later sections, the foundation of a computational method.

Natural Selection: The Algorithm That Programs Itself

Let us state natural selection as an algorithm in the most explicit possible terms, stripping away the biological particulars to reveal the bare computational skeleton.

The simplicity of alg:evo:natural_selection belies its power. Notice that the algorithm says nothing about the structure of the genotype space, the form of the fitness function, or the nature of the variation operator. It is, in the most precise sense, a black-box optimiser: it requires only the ability to evaluate fitness and to generate perturbed copies of existing solutions. This universality is both evolution's greatest strength and, as we shall see, its greatest weakness (it can be slow precisely because it uses so little information about the problem structure).

The mathematical heart of natural selection was formalised by R. A. Fisher in 1930, in what he grandly called the “fundamental theorem of natural selection” [33].

Fisher's theorem has a beautiful interpretation: natural selection is a variance-consuming engine. It converts genetic diversity (variance in fitness) into adaptive improvement (increase in mean fitness). When variance is high, adaptation is rapid. When variance is exhausted – when all individuals are equally fit – evolution grinds to a halt. This has immediate algorithmic implications: to maintain the adaptive power of an evolutionary algorithm, one must maintain population diversity, a principle known as diversity maintenance that recurs throughout this chapter.

The Red Queen Hypothesis

In Lewis Carroll's Through the Looking-Glass, the Red Queen tells Alice: “Now, here, you see, it takes all the running you can do, to keep in the same place.” In 1973, the evolutionary biologist Leigh Van Valen transformed this literary image into one of the most important hypotheses in evolutionary biology [34]: organisms must constantly adapt and evolve not merely to gain in fitness but simply to maintain their current fitness relative to the organisms they are co-evolving with.

The intuition is vivid. Consider the cheetah and the gazelle. As cheetahs evolve greater speed, the slower gazelles are eliminated, leaving only the fastest. But now the baseline has shifted: the average gazelle is faster, so only the fastest cheetahs can catch prey. Selection pressure on the cheetah intensifies, driving further speed increases. Both species run faster and faster, yet neither gains a lasting advantage. They are, in Carroll's phrase, running as fast as they can just to stay in the same place.

This dynamic is not limited to predator-prey relationships. It appears in host-parasite interactions (the vertebrate immune system is locked in a perpetual arms race with rapidly mutating pathogens), in sexual selection (the peacock's tail grows ever more elaborate because peahens evolve ever more discriminating preferences), and in competition between members of the same species for territory, mates, and resources.

Red Queen dynamics have a profound consequence: they prevent convergence to any fixed point. In a system with Red Queen dynamics, the very concept of an “optimum” becomes time-dependent. The best strategy at time $t$ may be mediocre at time $t+1$ because the opponents have adapted.

The Red Queen hypothesis provides the biological motivation for coevolutionary algorithms, which we shall study extensively in later sections. In generative modelling, the most famous example of Red Queen dynamics is the GAN: the generator and discriminator are locked in exactly the kind of coevolutionary arms race that Van Valen described, each constantly adapting to the other's improvements.

Punctuated Equilibrium and Neutral Networks

The fossil record tells a story that Darwin himself found troubling: species do not change gradually and continuously as his theory seemed to predict. Instead, they persist for millions of years in a state of near-stasis, then undergo rapid bursts of morphological change. In 1972, Niles Eldredge and Stephen Jay Gould formalised this observation as the theory of punctuated equilibrium [35]: the history of life is not a smooth ramp but a staircase, with long flat treads separated by abrupt risers.

Why would evolution exhibit such stop-and-start dynamics? One explanation comes from the structure of fitness landscapes. If the landscape contains neutral networks – connected sets of genotypes with equal fitness – then a population can wander along these neutral networks without any change in phenotype or fitness. The population is exploring, but from the outside, it appears to be standing still. Then, suddenly, the neutral walk reaches a region of genotype space adjacent to a higher-fitness peak, and the population rapidly climbs to the new peak. The result is exactly the punctuated pattern: long periods of apparent stasis (neutral drift along the plateau) interrupted by bursts of rapid change (climbing a newly accessible peak).

Motoo Kimura's neutral theory of molecular evolution [36] provides the theoretical foundation: most mutations at the molecular level are selectively neutral, neither beneficial nor deleterious. These neutral mutations accumulate through random genetic drift rather than natural selection. The population performs a random walk on the neutral network, and this walk is invisible to selection.

This result has a striking computational implication: neutral networks transform local optima from traps into way-stations. A population that would be forever stuck at a local optimum under pure hill climbing can escape via neutral drift, provided the fitness landscape contains sufficient neutral structure.

The Baldwin Effect: Learning Guides Evolution

In 1896, the psychologist James Mark Baldwin proposed a mechanism by which learning – a lifetime phenomenon – could influence evolution – a generational phenomenon – without invoking the discredited Lamarckian inheritance of acquired characteristics. The idea, now known as the Baldwin effect, runs as follows:

Imagine a population of organisms facing a novel environmental challenge. Some organisms have genotypes that make them more capable of learning the appropriate response, even if they do not innately possess it. These fast-learning organisms survive and reproduce more effectively than slow-learning ones. Over generations, selection favours genotypes that facilitate learning. Eventually, what was originally learned behaviour may become partially or fully innate through a process called genetic assimilation: the costly and time-consuming learning is replaced by hardwired neural circuitry.

The Baldwin effect has a precise mathematical formulation in terms of fitness landscapes. Recall that an organism's fitness depends on its phenotype (behaviour, morphology), which is determined by both genotype and lifetime experience (learning). An organism that can learn effectively has a broader basin of attraction in genotype space: even if its genotype is not perfectly adapted, learning can compensate and pull the phenotype toward a high-fitness region.

The Baldwin effect has deep implications for the relationship between evolution and learning in AI systems. It suggests that the most effective strategy is not to evolve complete solutions but to evolve learning algorithms – organisms (or neural networks) that can rapidly adapt to their environment during their lifetime. This is precisely the philosophy behind meta-learning approaches, and it connects directly to the neural architecture search methods discussed in 30, where the “genotype” is the network architecture and “learning” is gradient-based training of the weights.

Classical Evolutionary Algorithms

With the biological foundations in place, we now turn to the computational incarnations of evolutionary principles. The history of evolutionary computation is a story of convergent invention: three independent research traditions, on three continents, each abstracted the essential logic of Darwinian evolution into computational form. John Holland's genetic algorithms in Michigan, Ingo Rechenberg and Hans-Paul Schwefel's evolution strategies in Berlin, and Lawrence Fogel's evolutionary programming in San Diego all emerged in the 1960s and 1970s, each capturing different facets of the evolutionary process. A fourth tradition, John Koza's genetic programming, extended evolutionary search to the space of computer programs in the early 1990s. And in 2002, Kenneth Stanley and Risto Miikkulainen introduced NEAT, demonstrating that evolution could construct neural network topologies from scratch.

This section provides rigorous treatments of each approach, emphasising the mathematical principles that make them work (and the conditions under which they fail).

Genetic Algorithms

Holland's genetic algorithms (GAs) are the oldest and most widely known family of evolutionary methods [4][37]. The original formulation operates on fixed-length binary strings, a deliberate abstraction that separates the study of evolutionary dynamics from the specifics of any particular application domain.

Holland's most celebrated theoretical contribution is the schema theorem, which attempts to explain why genetic algorithms work by analysing the dynamics of schemata – templates that match subsets of the search space.

The schema theorem has an appealing interpretation: short, low-order, above-average schemata receive exponentially increasing numbers of instances. Holland called this the implicit parallelism of genetic algorithms: while explicitly evaluating $N$ individuals per generation, the GA implicitly processes $\mathrm{\Theta }(N^3)$ schemata.

Evolution Strategies

While Holland's genetic algorithms were designed for discrete optimisation, Ingo Rechenberg and Hans-Paul Schwefel independently developed evolution strategies (ES) for continuous optimisation – the problem of finding ${𝒙}^{\ast }={\mathrm{arg\hspace{0.17em}min}}_{𝒙\in {\mathrm{ℝ}}^n}f(𝒙)$ for a (possibly non-differentiable, noisy, or multimodal) objective function $f$ [2][3].

The key insight of evolution strategies is that the mutation operator should be self-adaptive: the algorithm does not merely evolve candidate solutions but also evolves the parameters of its own variation operator. This is a remarkable instance of the meta-learning principle we discussed in Why Evolution Matters for AI: evolution learning how to evolve.

The mutation step sizes $𝝈$ evolve alongside the solution $𝒙$. If a particular step size is too large (causing offspring to overshoot good regions) or too small (causing stagnation), selection will favour individuals with better-tuned step sizes. Over generations, the step sizes adapt to the local curvature of the fitness landscape – large steps in flat directions, small steps in steep directions.

This self-adaptation of step sizes was a revolutionary concept when Rechenberg introduced it in 1973. It anticipated by decades the adaptive learning rate methods (Adam, AdaGrad, RMSProp) that are now standard in deep learning. The principle is the same: each parameter should be updated with a step size matched to the local geometry of the loss surface.

CMA-ES: The gold standard.

The Covariance Matrix Adaptation Evolution Strategy (CMA-ES), developed by Hansen and Ostermeier [38], represents the state of the art in derivative-free continuous optimisation. While the basic ES adapts individual step sizes per dimension (a diagonal covariance matrix), CMA-ES adapts the full $n\times n$ covariance matrix of the mutation distribution, capturing correlations between variables.

The beauty of CMA-ES lies in its ability to learn the local geometry of the fitness landscape from the distribution of successful mutations. The covariance matrix $𝐂$ captures the shape of the fitness function's level sets near the current mean: its eigenvectors align with the principal axes of the local quadratic approximation, and its eigenvalues reflect the relative scales along these axes. The evolution path ${𝒑}_c$ provides temporal coherence, encouraging the covariance matrix to adapt in directions of consistent progress.

Genetic Programming

Genetic algorithms evolve strings. Evolution strategies evolve vectors. But the true prize of evolutionary computation has always been more ambitious: can we evolve programs? John Koza's genetic programming (GP) answers in the affirmative [5]. In GP, the genotype is not a fixed-length string but a variable-size tree representing a computer program. The leaves of the tree are terminals (constants, variables, sensor inputs), and the internal nodes are functions (arithmetic operators, conditional statements, domain-specific operations). Evolution operates by composing, decomposing, and recombining these trees, searching through the space of all expressible programs for one that solves the task.

The variation operators in GP are designed for tree structures:

Crossover (subtree swap).

Select two parent trees. In each parent, choose a random subtree. Swap the subtrees between parents. The resulting offspring inherit structure from both parents – a form of compositional recombination that has no analogue in fixed-length string representations.

Mutation (subtree replacement).

Select a random node in the tree. Replace the subtree rooted at that node with a randomly generated subtree. This introduces genuinely new structure that may not have existed in the population.

The bloat problem.

GP trees tend to grow in size over generations even when the extra nodes contribute nothing to fitness – a phenomenon known as bloat or code growth. Bloat occurs because larger trees have more crossover points, making them more likely to survive crossover intact (the “defence against crossover” hypothesis). Unchecked bloat leads to enormous, uninterpretable trees that waste computational resources.

The standard countermeasure is parsimony pressure: adding a penalty for tree size to the fitness function: (Parsimony)$$\begin{array}{ccc}& f_{\text{penalised}}(\text{tree})=f(\text{tree})-\alpha \cdot |\text{tree}|,& \text{<span class="tml-eqn"></span>}\end{array}$$ where $|\text{tree}|$ is the number of nodes and $\alpha >0$ is a parsimony coefficient. Lexicographic parsimony pressure, which prefers smaller trees only among individuals of equal fitness, is often more effective than the additive penalty.

Neuroevolution

The holy grail of evolutionary approaches to AI is neuroevolution: the evolution of neural network architectures and/or weights. If neural networks are the most powerful function approximators known, and if evolution is the most creative search process known, then combining the two should yield something extraordinary. This intuition has been validated repeatedly, from the early work of Yao (1999) to the modern successes of large-scale neuroevolution at OpenAI [9] and Uber [10].

The landmark contribution in this field is Kenneth Stanley and Risto Miikkulainen's NeuroEvolution of Augmenting Topologies (NEAT) [6], which solved three fundamental problems that had plagued earlier neuroevolution approaches:

1. The competing conventions problem: Two networks may encode the same function with different topologies. If crossover naively combines them, the offspring may inherit redundant structure and perform poorly.

2. The protection of innovation problem: When a structural mutation adds a new node or connection, the new structure initially performs poorly (the weights are random). Without protection, selection immediately eliminates these innovations before they have time to be optimised.

3. The minimal complexity problem: How should networks be initialised? Starting with large, randomly connected networks wastes search effort on unnecessary structure.

NEAT addresses all three problems with elegant mechanisms: innovation numbers for structural alignment, speciation for innovation protection, and complexification starting from minimal networks.

Mutation operators.

NEAT uses three types of mutation:

1. Weight mutation: Perturb existing connection weights by Gaussian noise or replace with a new random value.

2. Add connection: Add a new connection between two previously unconnected nodes, with a random initial weight and a new innovation number.

3. Add node: Split an existing connection by disabling it, inserting a new hidden node, and creating two new connections: one from the old input to the new node (weight 1) and one from the new node to the old output (weight equal to the old connection's weight). This ensures the mutation is initially neutral: the network's behaviour is unchanged.

Speciation.

To protect structural innovations, NEAT divides the population into species – groups of individuals with similar topologies. Individuals compete primarily within their species, not against the entire population. This gives newly evolved structures time to optimise their weights before facing competition from well-tuned networks with different topologies.

The distance between two genomes is: (NEAT Distance)$$\begin{array}{ccc}& d({𝒈}_1,{𝒈}_2)=\frac{c_{1}E}{N}+\frac{c_{2}D}{N}+c_3\cdot \bar{W},& \text{<span class="tml-eqn"></span>}\end{array}$$ where $E$ is the number of excess genes, $D$ is the number of disjoint genes, $\bar{W}$ is the average weight difference across matching genes, $N=\max (|{𝒈}_1|,|{𝒈}_2|)$, and $c_1,c_2,c_3$ are importance coefficients. Individuals are assigned to the species of the first representative whose distance is below a threshold ${\delta }_t$.

Fitness sharing ensures that each species receives a proportional share of the next generation's population. The adjusted fitness of individual $i$ in species $S$ is: (NEAT Sharing)$$\begin{array}{ccc}& f_{\text{adj}}({𝒈}_i)=\frac{f({𝒈}_i)}{|S|},& \text{<span class="tml-eqn"></span>}\end{array}$$ where $|S|$ is the number of individuals in species $S$. This prevents any single species from dominating the population, maintaining topological diversity.

HyperNEAT and indirect encodings.

A limitation of NEAT's direct encoding – where every connection is explicitly listed in the genome – is that genome size grows linearly with network size. For large networks, this makes evolution prohibitively slow. HyperNEAT (Stanley, D'Ambrosio, and Gauci, 2009) addresses this by evolving an indirect encoding: a Compositional Pattern Producing Network (CPPN) that takes as input the coordinates $(x_1,y_1,x_2,y_2)$ of a potential connection's source and target nodes and outputs the connection weight. The full network is constructed by querying the CPPN at every pair of node positions.

This indirect encoding has a profound advantage: it can exploit geometric regularities such as symmetry, repetition, and gradient patterns that are ubiquitous in biological neural networks and in the physical world. A CPPN with a sine activation can produce periodic connection patterns; one with Gaussian activations can produce locally connected receptive fields. The genome size is determined by the CPPN, which can be much smaller than the target network, enabling the evolution of networks with millions of connections from genomes with only hundreds of parameters.

We have now surveyed the foundational methods of evolutionary computation, each capturing a different aspect of the Darwinian engine. Genetic algorithms taught us the power of recombination and the implicit parallelism of population-based search. Evolution strategies revealed the principle of self-adaptation and the deep connection to natural gradient methods. Genetic programming showed that evolution can search the space of programs, not merely the space of parameters. And neuroevolution – particularly NEAT – demonstrated that the architecture of intelligence itself can be an object of evolutionary discovery.

Yet these classical methods, for all their elegance, share a common limitation: they optimise toward a single, fixed fitness function specified by the human designer. In the next sections, we shall see what happens when we remove this constraint – when we allow fitness itself to evolve, when we replace optimisation with exploration, and when we seek not the single best solution but an entire archive of diverse, high-quality solutions. This is the frontier of evolutionary AI, and it brings us closer to the open-ended creativity of biological evolution than any fixed-objective method ever could.

Coevolutionary Algorithms

In every evolutionary algorithm we have encountered so far, fitness has been a fixed, externally imposed quantity: a function $f:𝒳\to \mathrm{ℝ}$ that maps each candidate solution to a scalar score, and that function does not change as the population evolves. This is a convenient abstraction, but it is biologically absurd. No organism on Earth is evaluated against a static fitness function. The fitness of a gazelle depends on the speed and cunning of the cheetahs that hunt it; the fitness of a cheetah depends on the evasive agility of the gazelles it pursues. The fitness landscape is not a fixed mountain range to be climbed. It is a seething, shifting surface that deforms in response to the very organisms that inhabit it.

This is the central insight of coevolution: fitness is relational, not absolute. When two or more populations evolve simultaneously, and the fitness of individuals in each population depends on the composition of the other populations, we enter a qualitatively different regime. The dynamics are no longer those of hill-climbing on a fixed landscape; they are the dynamics of coupled, adaptive systems, with all the richness, subtlety, and danger that entails.

Coevolutionary algorithms formalise this insight into a computational framework. They maintain multiple interacting populations, each exerting selective pressure on the others. The result is an evolutionary arms race that can produce solutions of astonishing sophistication, far beyond what any single population evolving against a fixed fitness function could achieve. But coevolution is also fraught with peril: cyclic dynamics, disengagement, mediocre stable states, and other pathologies that we shall dissect in Pathologies of Coevolution.

Competitive Coevolution

The most dramatic form of coevolution is competitive: two populations are locked in an adversarial struggle, each trying to outperform the other. The canonical biological metaphor is the host–parasite relationship, in which hosts evolve defences against parasites, and parasites evolve countermeasures to circumvent those defences. In computational terms, one population generates solutions and the other generates tests (or adversaries), and each is rewarded for defeating the other.

The computational cost of alg:evo:competitive-coevolution is dominated by the evaluation step, which requires $N\times M$ interaction evaluations per generation. When interactions are expensive (e.g., simulating physical contests between virtual creatures), various sampling strategies can be employed: evaluating each individual against a random subset of the opposing population, or using competitive fitness sharing to select representative opponents [39].

Cooperative Coevolution

Not all coevolution is adversarial. In cooperative coevolution, multiple populations evolve complementary components of a solution, and they are rewarded for working together rather than against each other. The biological inspiration is symbiosis: the mitochondrion and the eukaryotic cell, the flower and the pollinator, the mycorrhizal network and the forest.

The computational motivation is equally compelling: many real-world problems have a natural decomposition into subcomponents, and evolving each subcomponent in a separate population can be far more efficient than evolving monolithic solutions. Consider the design of a robot: one population might evolve the morphology (body shape, limb configuration), another the control policy (neural network weights), and a third the sensor layout. Each population explores its own search space, but fitness is determined by how well the components work together as a complete system.

The choice of representatives in (COOP Fitness) is critical. Using the current best individual from each population (the greedy strategy) provides a stable evaluation signal but can lead to co-adaptation to a single partner, missing synergies with alternative collaborators. Using random collaborators provides broader coverage but introduces significant noise into the fitness signal. Potter and De Jong [14] proposed a balanced approach: evaluate each individual with both the current best and a random collaborator, and take the maximum fitness across these evaluations.

The Game-Theoretic View

The language of coevolution is, at its core, the language of game theory. When two populations interact through a fitness function that depends on both, we have a two-player game. When $K$ populations interact, we have a $K$-player game. This connection is not merely metaphorical; it provides a rigorous mathematical framework for understanding when coevolution converges, what it converges to, and why it sometimes fails catastrophically.

Pathologies of Coevolution

Coevolution promises an arms race that drives continuous improvement. But promises, in the world of adaptive systems, are treacherous things. The same feedback loops that enable spectacular emergent complexity can also trap populations in vicious cycles, drive them into mutual irrelevance, or seduce them into mediocre equilibria from which escape is exponentially difficult. Understanding these pathologies is not merely an academic exercise; it is essential for anyone who seeks to build reliable coevolutionary systems, or equivalently, stable adversarial training procedures for modern generative models.

This section catalogues the four great pathologies of coevolution, each a distinct failure mode with its own mathematical signature, and then presents the arsenal of remedies that the field has developed over three decades of struggle.

Cyclic Dynamics: The Red Queen Trap

The Red Queen Trap takes its name from Lewis Carroll's Through the Looking-Glass, in which the Red Queen tells Alice, “It takes all the running you can do, to keep in the same place.” In coevolutionary systems, this manifests as a situation where both populations are evolving furiously but making no objective progress; they are running faster and faster on a treadmill, each adapting to the other's latest strategy without ever reaching a state that is objectively better than where they started.

The mathematical root of cycling is intransitivity: strategy A beats strategy B, B beats C, and C beats A. When the strategy space contains intransitive cycles, the coevolutionary dynamics can chase their own tail indefinitely.

Disengagement

If cycling is the pathology of evenly matched populations chasing each other in circles, disengagement is the pathology of mismatched populations giving up on each other entirely. Disengagement occurs when one population becomes so dominant that the opposing population can no longer provide a useful fitness signal.

Think of it this way: a strong chess player who trains exclusively against weak opponents will gradually lose the tactical sharpness needed to compete at a high level. The training signal – easy victories – provides no pressure to maintain complex strategies, and the cognitive resources devoted to deep tactical calculation are gradually reallocated to simpler pattern matching that suffices against the current opponents. The same dynamic plays out in coevolutionary systems, often with dramatic consequences for the long-term trajectory of the search.

The analogy to mode collapse in GANs is precise. When the generator in a GAN collapses to a single mode that consistently fools the discriminator, the discriminator sees only “fake” samples from that mode and “real” samples from the full data distribution. The discriminator easily learns to reject everything from the collapsed mode, which provides no useful gradient to the generator about how to diversify. Conversely, if the discriminator becomes trivially good (perfectly classifying all samples), the generator receives a saturated gradient signal that provides no directional information. Both scenarios are forms of disengagement.

Forgetting: The Loss of Hard-Won Knowledge

Perhaps the most insidious pathology of coevolution is forgetting: a population that has learned to defeat a particular class of opponents gradually loses that capability as the opposing population evolves away from those strategies. The hosts learn to beat parasites of type A, so the parasites evolve into type B. The hosts adapt to beat type B, but in doing so, they unlearn their defences against type A. If the parasites ever cycle back to type A, the hosts are as vulnerable as they were at the beginning.

This is the coevolutionary analogue of catastrophic forgetting in neural network training, where learning new tasks overwrites the representations learned for previous tasks. The parallel is not merely metaphorical: in both cases, a system with finite capacity optimises for the current distribution of challenges, sacrificing performance on previously mastered challenges that are no longer present in the training signal. The remedies, too, are analogous: experience replay in continual learning corresponds to the Hall of Fame in coevolution; elastic weight consolidation corresponds to fitness regularisation against archived opponents; progressive network expansion corresponds to growing the solution representation to accommodate new strategies without overwriting old ones.

Mediocre Stable States

The most frustrating pathology of all is the mediocre stable state: a configuration in which both populations have settled into a lukewarm equilibrium, each performing adequately but not brilliantly against the other, and neither having any incentive to change. Unlike a Nash equilibrium, which represents a genuinely optimal configuration (no player can improve by unilateral deviation), a mediocre stable state is a trap, a configuration that appears stable but is far from optimal, and from which escape requires coordinated changes in both populations that are exponentially unlikely under independent variation.

Solutions to Pathologies

Three decades of wrestling with coevolutionary pathologies have produced a rich toolkit of remedies. No single technique solves all problems, but the combination of several can produce coevolutionary systems that are remarkably robust.

The Hall of Fame directly addresses forgetting: by requiring hosts to perform well against archived tests from earlier generations, it prevents them from unlearning previously acquired capabilities. It also mitigates cycling: if the test population cycles back to a previously seen strategy, the archive already contains that strategy, and hosts that have forgotten how to beat it are penalised.

Shared Fitness and Competitive Fitness Sharing.

Fitness sharing [39] modifies the fitness of a test individual to reward uniqueness: a test that defeats few hosts (but defeats them uniquely, since no other test can defeat them) receives higher fitness than a test that defeats many hosts but is redundant with other tests. Formally, the shared fitness of test $t_j$ is (Competitive Fitness Sharing)$$\begin{array}{ccc}& f_T^{\text{shared}}(t_j)=\sum _{i=1}^N\frac{(1-G(h_i,t_j))}{{\sum }_{j^{\prime }=1}^M(1-G(h_i,t_{j^{\prime }}))},& \text{<span class="tml-eqn"></span>}\end{array}$$ where the denominator counts how many other tests also defeat host $h_i$. This encourages the test population to maintain diversity, covering different weaknesses of the host population rather than converging on a single exploit. The effect is analogous to diverse ensemble training in machine learning: each test specialises in exposing a different vulnerability.

Pareto Coevolution.

Ficici and Pollack [40] proposed treating coevolution as a multi-objective optimisation problem: instead of aggregating performance across all opponents into a single scalar, treat performance against each opponent (or class of opponents) as a separate objective, and apply Pareto-based selection. A host $h$ dominates another host $h^{\prime }$ if $h$ performs at least as well as $h^{\prime }$ against every test, and strictly better against at least one. The Pareto front of non-dominated hosts is preserved, maintaining a diverse set of strategies that collectively cover the space of possible opponent types.

This approach is particularly effective against mediocre stable states, because Pareto selection preserves “specialist” strategies that excel against particular opponents even if they perform poorly on average. These specialists serve as stepping stones that can eventually be combined (through crossover or other variation) into generalist strategies that escape the mediocre equilibrium.

Pathology	Signature	Root Cause	Remedies
Cyclic dynamics (Cyclic Dynamics: The Red Queen Trap)	Periodic oscillations in population strategy composition; no improvement on external benchmark	Intransitive dominance structure in strategy space	Hall of Fame; fitness sharing; Pareto coevolution; increased population diversity
[6pt] Disengagement (Disengagement)	Near-uniform fitness within one population; interaction outcomes nearly constant	Fitness gap exceeds variation radius; one population dominates completely	Asymmetric mutation rates; competitive fitness sharing; resource sharing; diversity injection
[6pt] Forgetting (Forgetting: The Loss of Hard-Won Knowledge)	Performance against archived opponents degrades over time; periodic regression	Finite representational capacity; no memory of past opponents	Hall of Fame; experience replay; growing solution capacity; multi-objective evaluation
[6pt] Mediocre stable states (Mediocre Stable States)	Stable fitness with no improvement; poor absolute performance; appears converged	Local stability without global optimality; escape requires coordinated multi-population change	Pareto coevolution; novelty-based diversity; random restarts; external fitness evaluation

Quality-Diversity and MAP-Elites

We arrive now at what may be the most philosophically provocative idea in all of evolutionary computation: the suggestion that the goal of evolution is not to find the best solution, but to find all the interesting ways to solve a problem.

For four decades, evolutionary algorithms have been framed as optimisation procedures. The objective is always the same: maximise (or minimise) a fitness function. Find the tallest peak in the landscape. Get to the top. Plant a flag. Declare victory. This framing is so deeply embedded in the culture of evolutionary computation, and indeed in the broader culture of computer science, that it is difficult to imagine an alternative. But nature, as always, provides one.

Consider the biosphere. Evolution has not produced a single “optimal” organism. It has produced an astonishing diversity of organisms, each adapted to a different ecological niche: bacteria that thrive in boiling acid, fungi that digest rock, whales that navigate by song, birds that fly backwards. This diversity is not a failure of optimisation. It is the product of a process that rewards novelty and exploration as much as performance, and the diversity itself is the source of the biosphere's resilience, adaptability, and creative potential.

The quality–diversity (QD) paradigm [16] translates this insight into a computational framework. Instead of seeking a single best solution, QD algorithms seek a diverse collection of high-performing solutions, each occupying a distinct region of a user-defined “behaviour space.” The result is not a single answer but a repertoire, a library of solutions that collectively illuminate the space of possibilities.

Beyond Optimisation: The Quality-Diversity Paradigm

Why would we want diversity rather than (or in addition to) optimality? Three compelling reasons:

Robustness.

A single optimal solution is brittle: if the problem changes slightly (a new constraint, a shifted objective, a perturbation in the environment), the solution may become useless. A diverse repertoire is robust: among the hundreds of solutions it contains, there is likely one that remains effective under the new conditions. This is the evolutionary analogue of the maxim “don't put all your eggs in one basket.”

Stepping Stones.

Solutions that are suboptimal for the current problem may be essential stepping stones toward solutions that are optimal for a harder variant. Fitness landscapes are often deceptive: the path to the global optimum leads through regions of low fitness. By maintaining a diverse repertoire, QD algorithms preserve these stepping-stone solutions, enabling future discovery that a greedy optimiser would foreclose. This is the deep insight of Lehman and Stanley's work on novelty search [8], which we explore in Novelty Search.

Insurance Against Uncertainty.

When we do not know exactly what problem we will face in the future, having a repertoire of qualitatively different solutions is a form of insurance. In robotics, for example, a repertoire of diverse locomotion behaviours allows a damaged robot to quickly adapt by selecting a pre-computed behaviour that does not rely on the damaged component [15].

Novelty Search

Before MAP-Elites, there was an even more radical idea: what if we abandoned the fitness function entirely?

In 2011, Joel Lehman and Kenneth Stanley [8] proposed novelty search, a paradigm that replaces fitness-based selection with selection for behavioural novelty. Instead of rewarding solutions that perform well, novelty search rewards solutions that behave differently from anything seen before. The fitness function is not merely modified or supplemented; it is discarded. The only selection pressure is toward the new, the unexplored, the different.

The philosophical audacity of this proposal cannot be overstated. It flies in the face of four decades of optimisation-centric thinking. And yet, on certain classes of problems, specifically problems with deceptive fitness landscapes, novelty search dramatically outperforms fitness-based search.

MAP-Elites

Novelty search demonstrated that abandoning fitness in favour of behavioural diversity can be powerful. But it goes too far: by ignoring fitness entirely, novelty search discovers many diverse solutions but makes no guarantee about their quality. What we really want is both: diversity and quality. Enter MAP-Elites.

The Multi-dimensional Archive of Phenotypic Elites (MAP-Elites) [15] is an elegantly simple algorithm that maintains a map (grid) of the highest-performing solution discovered for each cell in a discretised behaviour space. It simultaneously optimises quality within each behavioural niche and maintains diversity across niches, producing a structured repertoire that “illuminates” the relationship between behaviour and performance.

The beauty of MAP-Elites lies in its simplicity. The algorithm (alg:evo:map-elites) has only two operations: generate a random variation of an existing elite, and place it in the archive if it is either the first solution in its cell or better than the current occupant. There is no explicit population, no selection tournament, no crossover pool. The archive is the population, and the grid structure provides both diversity maintenance (solutions in different cells are preserved regardless of relative fitness) and quality pressure (within each cell, only the best solution survives).

Connections to Generative Models

The quality–diversity framework has a deep and illuminating connection to generative modelling that goes beyond mere analogy. A MAP-Elites archive can be viewed as an implicit generative model over the solution space: by sampling uniformly from the occupied cells of the archive, we obtain a distribution over solutions that is both diverse (covering the behaviour space) and high-quality (each sample is the best known solution for its behavioural niche). This perspective opens the door to a powerful fusion of QD algorithms and deep generative models.

Quality-Diversity as Implicit Generative Model.

Consider a MAP-Elites archive $ℳ$ with $C$ occupied cells. Sampling a solution from this archive produces a random variable $𝒙\sim p_{ℳ}$, where (Archive Distribution)$$\begin{array}{ccc}& p_{ℳ}(𝒙)=\frac{1}{C}\sum _{𝒄\in 𝒢}\delta (𝒙-ℳ(𝒄))\cdot \mathrm{𝟙}[ℳ(𝒄)\ne \varnothing ].& \text{<span class="tml-eqn"></span>}\end{array}$$ This is a uniform mixture of point masses, one at each elite. As the archive grows (more cells become occupied and elites improve), this implicit distribution becomes a richer and more accurate model of the space of high-performing, diverse solutions. In the limit where the grid resolution approaches the intrinsic dimensionality of the behaviour space, the archive distribution converges to a continuous distribution supported on the Pareto front of quality vs. diversity.

Differentiable Quality-Diversity (DQD).

The limitation of standard MAP-Elites is that it uses random, undirected variation: mutations are sampled without knowledge of the fitness landscape or the behaviour descriptor function. When the solution space is parameterised by a differentiable model (e.g., a neural network), we can compute gradients of both the fitness and the behaviour descriptor with respect to the parameters, enabling directed variation that moves solutions toward better quality and toward under-explored regions of behaviour space.

The DQD framework provides a principled way to combine the strengths of gradient-based optimisation (fast convergence to local optima) with the strengths of QD algorithms (diversity maintenance and global exploration). It is particularly powerful when the solution space is high-dimensional and random mutations are inefficient, precisely the regime where deep generative models operate.

Latent Space Illumination.

An even more powerful approach uses a deep generative model, typically a variational autoencoder (VAE), to learn a low-dimensional latent space that parameterises the solution space, and then performs MAP-Elites in this latent space rather than in the original parameter space. The VAE is trained on solutions discovered during the QD search, creating a learned mapping from a compact latent code $𝒛\in {\mathrm{ℝ}}^q$ (with $q\ll p$) to solutions $𝒙=g_{𝝓}(𝒛)$, where $g_{𝝓}$ is the decoder.

The advantages are twofold:

1. Efficient variation: Mutations in the low-dimensional latent space produce semantically meaningful variations in solution space, because the VAE has learned the manifold of valid solutions. This is the QD analogue of the LLM-guided mutation we studied in LLMs as Semantic Variation Operators.

2. Interpolation: The latent space supports smooth interpolation between solutions, enabling the generation of novel solutions that “blend” the characteristics of existing elites.

LLMs as Semantic Variation Operators

In the preceding sections we developed the classical machinery of evolutionary computation: populations, fitness landscapes, crossover, mutation, and selection. Every one of these ideas traces back to the biological metaphor of blind variation and selective retention, a framework that has powered four decades of remarkably successful optimisation. Yet if we step back and examine the variation operators that drive classical evolutionary algorithms, we discover a fundamental limitation that becomes increasingly severe as we move from continuous numerical optimisation to the structured, discrete, semantically rich domains that matter most in modern AI: code synthesis, mathematical proof discovery, algorithm design, and hardware optimisation.

The limitation is simple to state but profound in its consequences: classical variation operators are structurally blind. They perturb solutions without understanding what those solutions mean.

This section charts the paradigm shift that occurs when we replace blind random mutation with the semantic understanding of large language models. The result is a new class of variation operators that understand syntax, respect semantics, diagnose failure modes, and generate targeted repairs. This fusion of evolutionary search with generative language models is, we shall argue, one of the most consequential developments in AI since the transformer architecture itself.

The Fundamental Limitation of Classical Variation

Consider the task of evolving a sorting algorithm. A candidate solution is a program, represented as a string of characters or an abstract syntax tree. Classical genetic programming applies variation operators drawn from the same conceptual family as the bit-flip mutations and single-point crossovers we studied in LLMs as Semantic Variation Operators: random subtree replacement, point mutations that swap one function symbol for another, and crossover that grafts a subtree from one parent into a random position in another.

The difficulty is immediate. A random subtree replacement in a program is overwhelmingly likely to produce a syntactically invalid result. Even when the result is syntactically valid, it is almost certainly semantically meaningless: the program crashes, enters an infinite loop, or produces output that bears no relation to sorting. Empirical studies of classical genetic programming report that between 70% and 95% of offspring produced by random variation are non-functional [11], a staggering waste of computational effort.

The consequences of structural blindness are most devastating in highly constrained search spaces. Consider the following hierarchy of increasing structural complexity:

1. Continuous vectors $𝒙\in {\mathrm{ℝ}}^n$: Gaussian perturbations ${𝒙}^{\prime }=𝒙+\sigma 𝝐$, $𝝐\sim 𝒩(0,𝐈)$, produce valid solutions with probability 1. Classical ES and CMA-ES exploit this property brilliantly.

2. Combinatorial structures (permutations, graphs): Validity constraints reduce the fraction of useful mutations, but domain-specific operators (swap mutations for permutations, edge insertions for graphs) maintain reasonable survival rates of 30–60%.

3. Programs and algorithms: The space of syntactically valid, semantically meaningful programs is a vanishingly small subset of the space of all character strings. Random perturbations almost never land in this subset.

4. Mathematical proofs: Every step must follow logically from previous steps. A random perturbation to a proof step almost certainly breaks the logical chain.

The needle-in-a-haystack problem.

For a concrete illustration, consider Python programs of length $L=200$ characters. The ASCII alphabet has $|\mathrm{\Sigma }|=95$ printable characters, giving ${95}^{200}\approx {10}^{395}$ possible strings. The number of syntactically valid Python programs of this length is astronomically smaller, and the number that are syntactically valid, terminate, and compute something useful for a given task is smaller still. A random mutation that changes even a single character has a high probability of producing a syntax error, a runtime exception, or a program that computes an entirely different function. Classical genetic programming copes with this by using tree-based representations and type-constrained operators, but these are band-aids on a fundamental wound: the variation operator does not understand what the program does.

The Paradigm Shift: LLMs as Intelligent Mutators

The emergence of large language models trained on billions of lines of code, mathematical text, scientific papers, and engineering documentation creates a fundamentally new possibility. An LLM is not merely a text generator; it is a learned model of structured domains. It has internalised the syntax of dozens of programming languages, the patterns of algorithmic design, the conventions of mathematical notation, and the reasoning chains that connect problem specifications to solutions.

When we use an LLM as a variation operator, we replace the structurally blind perturbation $\xi \sim p(\xi )$ with a context-aware transformation that conditions on:

• The full text of the parent solution $𝒙$.

• The problem specification or objective description $d$.

• The evaluation feedback $f(𝒙)$ from previous generations.

• The execution trace, error logs, or reasoning chain explaining why the parent solution succeeded or failed.

• Other high-performing solutions in the population (“inspirations”).

This conditioning transforms mutation from a random walk in syntactic space to a directed search in semantic space.

The difference between Definition 28 and Definition 29 is the difference between a blindfolded person throwing darts at a map and a skilled navigator charting a course. The classical operator explores by random perturbation and relies on selection pressure to retain the rare improvements. The semantic operator exploits the LLM's compressed knowledge of the solution space to generate perturbations that are far more likely to be valid, meaningful, and beneficial.

The LLM-Evolutionary Loop

The integration of LLMs into evolutionary search produces a characteristic feedback loop that differs fundamentally from both pure LLM inference and pure evolutionary optimisation. In pure LLM inference, the model generates a single response (or a small number of candidates via temperature sampling) and the process terminates. In pure evolutionary search, a population evolves over many generations under selection pressure but with blind variation. The LLM-evolutionary loop combines the generative power of language models with the sustained exploration and selection pressure of evolutionary computation.

The loop proceeds as follows:

1. Generate. The LLM generates one or more candidate solutions (or modifications to existing solutions) conditioned on the current population, problem description, and evaluation feedback.

2. Evaluate. Each candidate is executed in a sandboxed environment and scored against the objective function. This step grounds the LLM's output in reality: no matter how plausible a candidate looks, only its actual performance matters.

3. Reflect. The evaluation results, including execution traces, error messages, performance metrics, and comparison with the current best solutions, are compiled into a natural-language reflection that explains what worked, what failed, and why.

4. Mutate. The LLM receives the reflection along with selected parent solutions and generates new candidates that attempt to fix the identified failures, combine the strengths of multiple parents, or explore entirely new approaches suggested by the analysis.

Why the loop matters.

A single LLM inference produces a candidate whose quality is bounded by the model's training distribution and prompt quality. The evolutionary loop transcends this bound in two crucial ways. First, iterative refinement: each generation builds on the accumulated evaluations and reflections of all previous generations, creating an ever-richer context for the LLM. Second, population diversity: maintaining a population of diverse solutions prevents premature convergence to a single local optimum and enables the LLM to combine ideas from multiple high-performing but structurally different solutions. The evolutionary pressure acts as a ratchet that retains improvements while the LLM's semantic understanding guides the search toward promising regions of the solution space.

Semantic Gradients: Natural Language as Optimisation Signal

In classical optimisation, the gradient $\nabla f(𝒙)$ provides a local linear approximation of the objective function, telling us which direction to move in parameter space to improve the objective most rapidly. In gradient-free evolutionary search, we lose this directional information entirely: we know that mutation $A$ produced a fitness of 0.73 and mutation $B$ produced a fitness of 0.81, but we have no structured information about why $B$ was better or how to improve further.

The introduction of LLMs creates a qualitatively new kind of optimisation signal that we call the semantic gradient. Instead of receiving a scalar fitness value, the LLM-evolutionary system constructs a rich natural-language description of the evaluation outcome: execution traces, error messages, performance profiles, comparison with other solutions, and an explicit analysis of which components of the solution contributed positively or negatively to the overall score.

The semantic gradient is not a gradient in the formal mathematical sense (it does not live in a vector space, and there is no notion of linearity), but it serves the same functional role: it provides directional guidance for the next step of optimisation. The analogy is precise enough to be useful.

Formal connection to information-theoretic search.

The semantic gradient can be understood through the lens of information theory. Let $H({𝒙}^{\ast }|{ℐ}_g)$ denote the entropy of the optimal solution ${𝒙}^{\ast }$ conditioned on the information ${ℐ}_g$ available at generation $g$. Each evaluation provides information that reduces this entropy. A scalar fitness value $f(𝒙)$ provides at most ${\log }_2|𝒴|$ bits of information, where $𝒴$ is the discretised fitness range. A semantic gradient $g_{\text{sem}}(𝒙)$, by contrast, can provide information about the structure and location of defects within the solution, potentially yielding $\mathrm{\Omega }(L)$ bits of information where $L$ is the solution length. Formally:

(Semantic Information)$$\begin{array}{ccc}& 𝖨({𝒙}^{\ast };g_{\text{sem}}(𝒙)|{ℐ}_g)\ge 𝖨({𝒙}^{\ast };f(𝒙)|{ℐ}_g),& \text{<span class="tml-eqn"></span>}\end{array}$$

where $𝖨(\cdot ;\cdot )$ denotes mutual information. The inequality states that the semantic gradient is at least as informative as the scalar fitness, and in practice is typically far more informative.

The LLM as Crossover Operator

The semantic variation paradigm extends naturally to crossover. Classical crossover operators (single-point, two-point, uniform) combine genetic material from two parents by selecting crossover points and swapping segments. In structured domains, this frequently produces offspring that are syntactically or semantically broken because the swapped segments are incompatible (e.g., variables referenced in one segment are defined in the other).

An LLM-based crossover operator receives two parent solutions along with their evaluation summaries and is prompted to combine the best aspects of both. The LLM understands the semantic content of each parent and can perform intelligent recombination: taking the data structure from parent $A$ and the algorithm from parent $B$, resolving variable name conflicts, adapting interfaces, and producing a coherent offspring that genuinely integrates the strengths of both parents.

Prompt engineering for crossover.

A typical crossover prompt might read:

“Below are two solutions to [problem description]. Solution A achieves [score A] with strength [analysis A]. Solution B achieves [score B] with strength [analysis B]. Create a new solution that combines the best aspects of both. Specifically, try to use A's [identified strength] with B's [identified strength]. The solution must be a complete, self-contained program.”

The prompt structure ensures that the LLM's output is both semantically informed (it knows what each parent does well) and practically grounded (it must produce a complete, runnable program).

Connection to Generative Models and Diffusion

The use of LLMs as variation operators creates a beautiful conceptual bridge between evolutionary computation and the generative modelling frameworks we studied in earlier chapters. In diffusion models (19), we learn a denoising process that iteratively transforms random noise into structured samples from a target distribution. In LLM-guided evolution, the LLM plays an analogous role: it transforms a “noisy” solution (one with bugs, inefficiencies, or suboptimal design choices) into a “cleaner” solution that better fits the objective.

The parallel extends further. Classifier-free guidance in diffusion models conditions the denoising process on a desired property (e.g., a text prompt), biasing generation toward samples that satisfy the condition. In LLM-guided evolution, the semantic gradient plays the role of the classifier signal, biasing the LLM's mutations toward solutions that address identified weaknesses. We can formalise this analogy:

(Diffusion Analogy)$$\begin{array}{ccc}& \underset{\text{ guided semantic mutation}}{\underbrace{p_{𝜽}({𝒙}^{\prime }|𝒙,g_{\text{sem}})}}⟷\underset{\text{ guided diffusion step}}{\underbrace{p_{𝜽}({𝒙}_{t-1}|{𝒙}_t,𝒄)}},& \text{<span class="tml-eqn"></span>}\end{array}$$

where the current (imperfect) solution $𝒙$ corresponds to the noisy sample ${𝒙}_t$, the improved solution ${𝒙}^{\prime }$ corresponds to the denoised sample ${𝒙}_{t-1}$, and the semantic gradient $g_{\text{sem}}$ corresponds to the conditioning signal $𝒄$.

Historical Context and Rapid Progress

Dangers and Failure Modes

AlphaEvolve: Open-Ended Algorithmic Discovery

In May 2025, DeepMind published a system that quietly but unmistakably crossed a threshold. AlphaEvolve did not merely optimise existing algorithms or fine-tune known solutions: it discovered new mathematical constructions and engineering optimisations that had eluded human researchers for decades. Among its achievements was a new algorithm for multiplying $4\times 4$ matrices over the complex numbers using only 48 scalar multiplications, improving upon a record that had stood since Volker Strassen's landmark 1969 paper [19].

What makes AlphaEvolve remarkable is not any single result but the generality of its approach. The same system, with minimal task-specific engineering, produced state-of-the-art results across domains as diverse as combinatorial mathematics, compiler optimisation, chip design, data centre scheduling, and GPU kernel programming. In this section we dissect the architecture and mechanisms that make this generality possible.

Architecture Overview

AlphaEvolve integrates four major components into a coherent system: (1) an evolutionary database that maintains a diverse population of candidate programs, (2) a prompt sampler that constructs rich context from the population, (3) a tiered ensemble of LLMs that generate mutations and new candidates, and (4) an evaluator pool that grounds every candidate in empirical reality through sandboxed execution.

The evolutionary database.

The database $𝒟$ uses a MAP-Elites architecture (MAP-Elites) extended with an island model [15]. Each island maintains its own MAP-Elites archive, with solutions characterised by behavioural descriptors that capture different aspects of solution quality (e.g., for a sorting algorithm: time complexity class, space usage, and stability). Periodically, the best solutions from each island migrate to other islands, injecting diversity and enabling the cross-pollination of ideas across semi-independent evolutionary lineages.

The island model serves a dual purpose. First, it maintains diversity: different islands may converge to different local optima, and migration allows the best features of each to combine. Second, it provides a natural parallelism structure: each island can evolve independently, with migration events providing synchronisation points.

The prompt sampler.

The prompt sampler $𝒮$ is perhaps the most sophisticated component, responsible for constructing the context that guides the LLM's mutations. For each mutation, $𝒮$ performs the following steps:

1. Parent selection. Select one or two parent programs from the evolutionary database using a combination of fitness- proportionate and diversity-based selection.

2. Inspiration selection. With probability $p_{\text{inspire}}$, additionally select a high-performing program from a different island as an “inspiration” that the LLM can draw ideas from without directly combining code.

3. Context assembly. Combine the parent program(s), their evaluation summaries, the problem description, any EVOLVE-BLOCK markers (described below), and optional inspirations into a structured prompt.

4. Instruction generation. Append a natural-language instruction that specifies the type of mutation desired: “improve the time complexity,” “fix the bug on edge case $X$,” “try a completely different algorithmic approach,” or “combine the data structure from parent A with the algorithm from parent B.”

Tiered model allocation.

AlphaEvolve uses two tiers of LLMs: Gemini Flash for approximately 80% of mutations and Gemini Pro for the remaining 20%. The allocation is not random but is guided by the difficulty of the mutation. Simple refinements (fixing syntax errors, adjusting constants, minor restructuring) are handled by the fast model. Fundamental algorithmic redesigns, novel mathematical constructions, and mutations that require deep chain-of-thought reasoning are routed to the powerful model.

This tiered approach reflects a fundamental insight about the structure of evolutionary search: most mutations are incremental refinements that do not require the full reasoning capacity of a frontier model. Allocating expensive compute only to the mutations that need it yields a dramatic improvement in cost-efficiency without sacrificing the quality of breakthrough discoveries.

Multi-Block Coevolution

One of AlphaEvolve's most powerful capabilities is its ability to evolve multiple interdependent components of a codebase simultaneously. Classical evolutionary approaches typically evolve a single function or a single configuration file. Real-world optimisation problems, however, often require coordinated changes across multiple components: the search algorithm must co-adapt with the data structure it operates on, the mathematical formula must co-evolve with the numerical method that evaluates it, and the hardware kernel must co-evolve with the scheduling policy that dispatches work to it.

AlphaEvolve enables this through EVOLVE-BLOCK markers: special annotations in the codebase that delineate regions subject to evolutionary modification. A single codebase can contain multiple EVOLVE-BLOCK regions, and the evolutionary process can modify any subset of them in a single mutation.

Landmark Results

AlphaEvolve's results span multiple domains, from pure mathematics to production engineering systems. We review the most significant achievements, each of which illustrates a different aspect of the system's capabilities.

Matrix multiplication: breaking the 56-year barrier.

The problem of multiplying two $n\times n$ matrices is one of the most fundamental in computer science. The naive algorithm requires $n^3$ scalar multiplications. In 1969, Volker Strassen showed that two $2\times 2$ matrices could be multiplied using only 7 scalar multiplications instead of 8, leading to an overall complexity of $O(n^{{\log }_{2}7})\approx O(n^{2.807})$ via recursive decomposition [19]. This was the first demonstration that the cubic barrier could be broken, and it launched an entire field of research into fast matrix multiplication algorithms.

For $4\times 4$ matrices over the complex numbers ${ℂ}^{4\times 4}$, the best known algorithm before AlphaEvolve required 49 scalar multiplications (achieved through a combination of Strassen-like recursive decompositions and computer-assisted search). AlphaEvolve discovered an algorithm using only 48 scalar multiplications, improving a record that had been refined incrementally over more than five decades.

The significance of this result extends beyond the specific constant. Matrix multiplication is a fundamental primitive in virtually all of scientific computing, and any improvement in the constant factor or asymptotic complexity has cascading effects across numerical linear algebra, machine learning, signal processing, and physics simulations.

Combinatorial optimisation: broad superiority.

AlphaEvolve was evaluated on over 50 open problems from combinatorial mathematics, including extremal graph theory, coding theory, and combinatorial design. The results were striking: approximately 75% of the time, AlphaEvolve matched the best known human-designed constructions, and on approximately 20% of problems, it surpassed the state of the art, discovering constructions that no human had found [18].

Borg scheduler optimisation.

Google's Borg cluster management system schedules jobs across millions of machines in data centres worldwide. AlphaEvolve was applied to optimise the scheduling heuristics in Borg, evolving the scoring functions that determine how jobs are assigned to machines. The result was a 0.7% improvement in overall resource recovery, which translates to recovering approximately 0.7% of all compute resources across Google's worldwide infrastructure. Given the enormous scale of this infrastructure, even a fraction-of-a-percent improvement represents hundreds of millions of dollars in annual value.

FlashAttention kernel optimisation.

FlashAttention [41] is a memory-efficient implementation of the attention mechanism that has become a critical component of modern transformer training. AlphaEvolve evolved optimised GPU kernels for FlashAttention, achieving a 32.5% speedup over the existing hand-tuned implementation. This is particularly remarkable because FlashAttention was already heavily optimised by expert GPU programmers; the fact that an automated system could find a significant further improvement suggests that the space of possible kernel optimisations is far larger than human engineers can explore manually.

TPU Verilog optimisation.

AlphaEvolve was also applied to optimise the Verilog hardware description language code for Google's Tensor Processing Units (TPUs). The system evolved circuit-level optimisations in the matrix multiply unit, achieving improvements in area and timing that were deployed in production TPU designs. This application demonstrates the system's ability to operate at the hardware level, evolving low-level circuit descriptions that require deep understanding of digital logic, timing constraints, and power dissipation.

Domain	Problem	Previous best	AlphaEvolve
Mathematics	$4\times 4$ complex matmul	49 mults	48 mults
Mathematics	Cap set constructions	Human-designed	New records
Combinatorics	50+ open problems	State of art	20% improved
Systems	Borg scheduler	Production baseline	+0.7% recovery
ML kernels	FlashAttention	Hand-tuned	+32.5% speedup
Hardware	TPU Verilog	Engineer-designed	Area + timing gains
Algorithms	Sorting networks	Known optimal	Matched or improved

Evaluation and Grounding

The evaluation pool $ℰ$ is the component that transforms LLM-guided evolution from speculative generation into grounded discovery. Every candidate program produced by the LLM is compiled and executed in a sandboxed environment with strict resource limits (CPU time, memory, disk space). The evaluator produces two outputs:

1. A scalar fitness $s=f(𝒙)\in \mathrm{ℝ}$ that summarises the candidate's quality.

2. An evaluation trace $𝒯(𝒙)$ that includes compilation output, runtime logs, intermediate values, timing measurements, test case results, and any error messages.

The scalar fitness determines whether the candidate enters the evolutionary database. The evaluation trace feeds back into the prompt sampler to construct semantic gradients for future mutations.

Multi-objective evaluation.

For many problems, the evaluation function is multi-objective. For example, when optimising a GPU kernel, AlphaEvolve evaluates both throughput (operations per second) and numerical accuracy (error relative to a reference implementation). The MAP-Elites archive naturally accommodates multi-objective evaluation by using the different objectives as behavioural descriptors, maintaining solutions that are Pareto-optimal across the objective space.

The Open-Source Ecosystem: OpenEvolve and CodeEvolve

The publication of AlphaEvolve's architecture and results catalysed a rapid open-source response. Within weeks of the original paper, independent teams released open-source implementations that replicated the core LLM-evolutionary loop, adapted it for different use cases, and introduced novel algorithmic innovations. This section surveys the three most significant open-source systems – OpenEvolve, CodeEvolve, and LEVI – and analyses how they extend, simplify, or improve upon the original AlphaEvolve design.

The existence of this ecosystem is itself significant. Unlike many industrial AI systems that remain proprietary and irreproducible, the LLM-evolutionary paradigm is inherently reproducible: the core algorithm (Algorithm 1) is straightforward to implement, and the variation operator is any LLM accessible via API. This accessibility has enabled a Cambrian explosion of experimentation by researchers, hobbyists, and small teams who lack the resources of a major AI lab but possess domain expertise in specific optimisation problems.

OpenEvolve: Democratising LLM-Evolutionary Search

OpenEvolve [42] is an open-source implementation of the LLM-evolutionary search paradigm designed for maximum generality and accessibility. Its architecture closely follows the AlphaEvolve blueprint but makes several pragmatic adaptations for the open-source setting: support for any LLM API (not just Gemini), multi-language evaluation support, simplified configuration, and extensive logging for reproducibility.

Architecture.

OpenEvolve consists of four modules that mirror the AlphaEvolve components:

1. Population Manager. Maintains the evolutionary population using a simplified island model with $K$ islands (default $K=5$). Each island stores candidates as pairs $({𝒙}_i,s_i)$ of program text and scalar fitness, ranked by fitness within each island. Migration occurs every $\tau =10$ generations by copying the best candidate from each island to a randomly selected neighbour.

2. Prompt Engine. Constructs prompts for the LLM by selecting one or two parents from the current island, formatting them with evaluation feedback and problem description, and appending a mutation instruction. The prompt engine supports customisable templates that can be adapted to different problem domains.

3. LLM Interface. A generic API layer that supports OpenAI-compatible APIs, Anthropic Claude, Google Gemini, and local models via Ollama or vLLM. This abstraction allows users to swap LLMs without modifying any other component.

4. Evaluation Harness. A sandboxed execution environment that supports Python, Rust, R, C++, and Metal shader programs. Each candidate is executed with configurable timeouts and resource limits. The evaluation harness returns both a scalar score and structured feedback (stdout, stderr, timing data, test results).

Multi-language support.

A distinguishing feature of OpenEvolve is its support for evolving programs in multiple programming languages. The evaluation harness includes language-specific runners for:

• Python: Direct execution with subprocess, supporting NumPy, SciPy, and PyTorch imports.

• Rust: Compilation with rustc followed by execution, enabling evolution of high-performance systems code.

• R: Execution via Rscript, targeting statistical computing and data analysis applications.

• Metal shaders: Compilation and execution on Apple GPUs via the Metal Performance Shaders framework, enabling evolution of GPU compute kernels for Apple hardware.

This multi-language capability reflects the insight that different optimisation problems are most naturally expressed in different languages: numerical algorithms in Python, systems code in Rust, statistical models in R, and GPU kernels in Metal or CUDA.

Results.

OpenEvolve has been applied to a diverse set of problems by the open-source community, with notable results including:

• Circle packing: Evolving algorithms for packing circles of varying radii into a square container, achieving packing densities competitive with specialised mathematical solvers.

• FIR filter design: Evolving finite impulse response filter coefficients for signal processing applications, discovering filters with better stopband attenuation than Parks-McClellan designs for certain specifications.

• MLX kernels: Evolving optimised compute kernels for Apple's MLX machine learning framework, achieving speedups of 15–25% over the default implementations for matrix operations on M-series chips.

CodeEvolve: Meta-Prompting and Transparent Evaluation

CodeEvolve [43] introduces two innovations that distinguish it from both AlphaEvolve and OpenEvolve: meta-prompting, in which the LLM evolves its own mutation instructions, and transparent evaluation, in which the full evaluation pipeline is visible to and modifiable by the user.

Meta-prompting: evolving the evolution strategy.

In standard LLM-evolutionary search, the prompt template that instructs the LLM how to mutate solutions is fixed by the system designer. CodeEvolve recognises that the prompt template is itself a crucial design choice that significantly affects the quality and diversity of mutations. Rather than fixing the template, CodeEvolve treats it as an additional evolvable component.

The meta-prompting mechanism enables the system to adapt its mutation strategy to the problem at hand. For example, if the system discovers that mutations focused on “simplifying the algorithm” tend to produce higher-fitness offspring than mutations focused on “adding features,” the template population will shift toward simplification-oriented prompts. This is a form of self-adaptation, analogous to the self-adaptive mutation step sizes in evolution strategies (Evolution Strategies).

Elitist migration.

CodeEvolve implements a migration strategy between islands that is strictly elitist: only solutions that represent a Pareto improvement over any existing solution in the destination island are accepted. This prevents the dilution of high-quality island populations by mediocre migrants, a problem that can occur with the unconditional migration used in simpler island models.

Formally, let ${ℱ}_k$ be the set of fitness values in island $k$, and let $𝒙$ be a candidate migrant with fitness $s=f(𝒙)$. The migrant is accepted into island $k$ only if: (Elitist Migration)$$\begin{array}{ccc}& s>\underset{{𝒙}^{\prime }\in I_k}{\min }f({𝒙}^{\prime })\text{and}\nexists {𝒙}^{\prime \prime }\in I_k:f({𝒙}^{\prime \prime })=s\text{ and }d(𝒙,{𝒙}^{\prime \prime })<ϵ,& \text{<span class="tml-eqn"></span>}\end{array}$$ where the second condition prevents the introduction of near-duplicates that would reduce diversity without improving fitness. The distance function $d$ is typically edit distance between program texts or a semantic similarity metric based on output behaviour.

Transparent evaluation.

CodeEvolve makes the evaluation pipeline a first-class citizen of the system: the evaluation script is written in plain Python, included in the project repository, and fully visible to the user. The user can modify the evaluation function at any time during the evolutionary run, enabling interactive refinement of the fitness landscape. This transparency is essential for trust: the user can inspect exactly how fitness is computed, verify that the evaluation is correct, and adjust it when the evolutionary process converges to an unexpected local optimum that exploits an evaluation loophole.

LEVI: Stratified Model Allocation

LEVI (LLM-Evolutionary Variation with Intelligence stratification) [44] focuses on a problem that becomes critical as LLM-evolutionary search scales: the cost of frontier model queries. A single query to a state-of-the-art model like GPT-4 or Claude Opus can cost 10–100$\times$ more than a query to a smaller model like GPT-4o-mini or Claude Haiku. If every mutation in a population of thousands over hundreds of generations requires a frontier model query, the total cost becomes prohibitive for most research groups.

LEVI addresses this with a stratified model allocation strategy that assigns different tiers of LLMs to different types of mutations based on their expected difficulty and importance.

The stratification principle.

LEVI classifies mutations into three tiers:

1. Tier 1: Routine refinements (80–90% of mutations). These include constant tweaking, minor refactoring, adding edge case handling, and fixing obvious bugs. These mutations are well within the capability of small, inexpensive models and are assigned to the cheapest available LLM (e.g., GPT-4o-mini, Claude Haiku, Gemini Flash).

2. Tier 2: Structural modifications (8–15% of mutations). These include changing data structures, modifying algorithmic flow, adding new subroutines, and combining ideas from multiple parents. These require moderate reasoning capability and are assigned to a mid-tier model (e.g., Claude Sonnet, GPT-4o).

3. Tier 3: Paradigm shifts (2–5% of mutations). These are attempts at fundamental algorithmic redesign: replacing a greedy algorithm with dynamic programming, switching from iterative to recursive approaches, or introducing entirely new mathematical constructions. These require the deepest reasoning and are assigned to a frontier model (e.g., Claude Opus, GPT-4, Gemini Ultra).

Tier classification.

LEVI classifies mutations into tiers using a combination of heuristics and learned classifiers:

• Edit distance heuristic. If the requested mutation involves changes to fewer than 5 lines of code, it is classified as Tier 1. If it involves 5–30 lines, Tier 2. If it involves more than 30 lines or requires generating entirely new code, Tier 3.

• Prompt complexity. The mutation prompt is analysed for keywords indicating difficulty: “fix bug,” “adjust constant,” “improve readability” suggest Tier 1; “restructure,” “add caching,” “change data structure” suggest Tier 2; “redesign algorithm,” “try a completely different approach,” “novel mathematical construction” suggest Tier 3.

• Stagnation detection. If the population has not improved for $\gamma$ consecutive generations, the system increases the fraction of Tier 3 mutations, reasoning that incremental refinements have been exhausted and a paradigm shift is needed.

Adaptive tier allocation.

LEVI does not fix the tier fractions $({\alpha }_1,{\alpha }_2,{\alpha }_3)$ but adapts them based on the improvement rate at each tier. Let $r_i$ be the fraction of tier-$i$ mutations that produce offspring fitter than their parents. LEVI updates the allocation using a softmax rule: (Adaptive Allocation)$$\begin{array}{ccc}& {\alpha }_i^{(g+1)}=\frac{\exp (\beta \cdot r_i^{(g)}/c_i)}{{\sum }_{j=1}^3\exp (\beta \cdot r_j^{(g)}/c_j)},& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\beta >0$ is a temperature parameter that controls the sharpness of the allocation. The ratio $r_i/c_i$ measures the improvement per unit cost at each tier, ensuring that the system allocates resources to the tier that provides the best return on investment.

Comparative Analysis

The four systems – AlphaEvolve, OpenEvolve, CodeEvolve, and LEVI – represent different points in the design space of LLM-evolutionary search. Table 7 summarises their key architectural choices and capabilities.

Feature	AlphaEvolve	OpenEvolve	CodeEvolve	LEVI
Developer	DeepMind	Community	Community	Community
LLM support	Gemini only	Any API	Any API	Any API
Population model	MAP-Elites + islands	Islands	Islands	Islands
Multi-block coevol.	Yes	Partial	No	No
Meta-prompting	No	No	Yes	No
Tiered models	2-tier	Single	Single	3-tier
Languages	Python, C++	Py, Rust, R, Metal	Python	Python
Open source	No	Yes	Yes	Yes

Architectural trade-offs.

Each system makes different trade-offs between complexity and capability:

• AlphaEvolve is the most sophisticated system, with multi-block coevolution, MAP-Elites quality-diversity archives, and a highly engineered prompt sampler. Its complexity is justified by the scale and importance of its target problems (production systems at Google), but makes it difficult to reproduce or adapt for different settings.

• OpenEvolve prioritises accessibility and generality. Its simpler architecture (basic island model, single LLM tier) makes it easy to set up and use, and its multi-language support enables a wide range of applications. The trade-off is that it may not achieve the same peak performance as AlphaEvolve on the hardest problems.

• CodeEvolve introduces meta-prompting, which adds a new dimension of adaptivity but also increases the search space (the system must search over both solutions and prompts). Transparent evaluation provides an important trust and debugging benefit. The trade-off is increased complexity in the evolutionary dynamics.

• LEVI focuses on cost-efficiency through stratified model allocation. Its key contribution is demonstrating that near-frontier quality can be achieved at a fraction of the cost by matching model capability to mutation difficulty. The trade-off is the overhead of tier classification and adaptive allocation.

Island Model Topologies

The island model is a common architectural element across all four systems. The migration topology, which determines which islands can exchange solutions with which others, has a significant effect on the balance between exploration and exploitation. We examine several topologies and their properties.

Common topologies include:

1. Ring topology. Each island sends migrants to its clockwise neighbour. Migration diameter: $K-1$. This topology provides slow but steady mixing, giving each island time to develop its own evolutionary trajectory before being influenced by distant islands.

2. Fully connected. Every island can send migrants to every other island. Migration diameter: 1. This provides the fastest mixing but reduces the independence of islands, potentially causing premature convergence.

3. Star topology. One central island receives migrants from all others and sends migrants to all others. Peripheral islands do not communicate directly. Migration diameter: 2. This creates a “melting pot” in the central island while preserving diversity in the periphery.

4. Random sparse graph. Each island has $d$ randomly chosen neighbours with $d\ll K$. Migration diameter: $O(\log K/\log d)$. This provides a good balance between mixing speed and island independence.

Challenges and Future Directions

The open-source ecosystem faces several challenges that are active areas of research and development.

Evaluation bottleneck.

The evaluation step is typically the computational bottleneck in LLM-evolutionary search. While LLM queries take seconds, program evaluation can take minutes (for numerical simulations), hours (for training neural networks), or days (for hardware synthesis). Surrogate models that predict fitness from program features can alleviate this bottleneck but introduce approximation errors that may mislead the evolutionary process.

Prompt sensitivity.

The quality of LLM-generated mutations is highly sensitive to the prompt format, wording, and structure. Small changes in the prompt template can dramatically alter the distribution of generated mutations. CodeEvolve's meta-prompting addresses this by evolving the prompts themselves, but a principled understanding of what makes a good evolutionary prompt remains an open question.

Reproducibility.

LLM-evolutionary search is inherently stochastic at two levels: the LLM's sampling process and the evolutionary selection process. Reproducing results requires fixing both the LLM (including exact version and API parameters) and the random seeds for selection. Because LLM providers frequently update their models, exact reproduction of published results may be impossible even with identical code and configuration. OpenEvolve addresses this partially through extensive logging and the option to use local models via Ollama.

Scaling laws.

It remains unclear how the performance of LLM-evolutionary search scales with population size, number of generations, and model size. Empirical evidence from AlphaEvolve suggests that performance improves consistently with compute budget, but whether this scaling follows a power law (analogous to neural scaling laws) or has diminishing returns is an open question.

Synthesis: The New Evolutionary Paradigm

Let us step back and consider what this section and the preceding one have established. The fusion of large language models with evolutionary computation represents a genuine paradigm shift in optimisation, one that combines the sustained exploration and selection pressure of evolution with the semantic understanding and generative power of modern AI.

The classical evolutionary algorithm explores by blind variation and selects by survival of the fittest. The LLM-evolutionary algorithm explores by intelligent variation, mutation that understands what it is mutating and why, and selects by grounded evaluation that verifies every candidate in the real world. The result is a search process that is orders of magnitude more efficient than classical evolution on structured domains.

Several principles have emerged from the systems we have studied:

1. Semantic variation is the key innovation. The replacement of blind random mutation with LLM-guided semantic mutation is the single most important change. It transforms the survival rate from exponentially small to practically useful, enabling evolution in program space, mathematical space, and other highly structured domains.

2. Grounding is essential. Without sandboxed evaluation, LLM-generated candidates are merely plausible hallucinations. The evolutionary loop's power comes from the combination of creative generation and rigorous verification.

3. Diversity mechanisms matter. Island models, MAP-Elites archives, varied temperatures, and inspiration selection all contribute to maintaining the population diversity that prevents premature convergence.

4. Cost-efficient model allocation is practical. Not every mutation needs a frontier model. Stratified allocation achieves near-frontier quality at a fraction of the cost.

5. Meta-level adaptation is powerful. Systems that evolve not only solutions but also their own mutation strategies (meta-prompting, adaptive tier allocation) can adapt to the structure of the problem being solved.

GEPA: Reflective Coevolution and Prompt Adaptation

The preceding sections have established that evolutionary algorithms can discover programs, architectures, and strategies that elude gradient-based optimisation. We now turn to a question that strikes at the heart of modern AI engineering: can evolution optimise the instructions we give to large language models? The answer, as we shall see, is not merely yes, but that evolutionary prompt optimisation can outperform reinforcement learning by an order of magnitude in sample efficiency, and by twenty percentage points in absolute performance on the hardest mathematical reasoning benchmarks.

The system that achieves this is called GEPA (Guided Evolution of Prompts and Agents) [45], and its central innovation is breathtakingly simple in concept yet profound in consequence: replace scalar reward signals with natural-language diagnostic feedback, and let evolution be guided by understanding rather than by random mutation.

The Limitations of RL for LLM Optimisation

Consider the standard reinforcement learning pipeline for improving an LLM's reasoning ability. A policy model ${\pi }_{𝜽}$ generates a response to a mathematical problem. A reward function $R:𝒴\to \{0,1\}$ checks whether the final answer is correct. The policy gradient, typically computed via Group Relative Policy Optimisation (GRPO) or Proximal Policy Optimisation (PPO), updates the model weights: (RL Policy Gradient)$$\begin{array}{ccc}& {\nabla }_{𝜽}J(𝜽)={𝔼}_{𝒚\sim {\pi }_{𝜽}(\cdot |𝒙)}[R(𝒚){\nabla }_{𝜽}\log {\pi }_{𝜽}(𝒚|𝒙)].& \text{<span class="tml-eqn"></span>}\end{array}$$ This formulation suffers from three fundamental pathologies when applied to complex reasoning tasks.

Sparse, binary rewards.

For a multi-step mathematical proof or a complex coding task, the reward is typically binary: the answer is either correct or it is not. A model that produces a brilliant chain of reasoning but makes a single arithmetic error in the final step receives the same zero reward as a model that produces nonsense. The gradient in (RL Policy Gradient) treats both failures identically, washing out the signal from near-misses that contain valuable partial progress.

The credit assignment problem.

When a model generates a 2,000-token reasoning chain, which tokens were responsible for the failure? Was it the initial problem decomposition? A wrong lemma choice in step three? A calculation error in step seven? The policy gradient distributes credit uniformly across all tokens in the trajectory, a strategy that is demonstrably suboptimal when the reasoning chain contains both excellent and catastrophic segments.

Sample inefficiency.

GRPO requires thousands to tens of thousands of rollouts to converge, each involving a full forward pass through the LLM. For a 70-billion-parameter model, each rollout costs significant compute. When the search space is the space of all possible prompts and reasoning strategies rather than the space of model weights, this sample complexity becomes prohibitive.

Prompts as Evolvable Genomes

GEPA reconceptualises the prompt optimisation problem through an evolutionary lens. Instead of adjusting model weights via backpropagation, GEPA treats the system prompt as a genome that encodes the reasoning strategy of the LLM. The key objects in this framework are the following.

The search space $𝒢$ is vast – a prompt of 1,000 tokens drawn from a vocabulary of 100,000 tokens admits ${10}^{\mathrm{5,000}}$ possible genomes – but it is also richly structured. Unlike bit-string genomes in classical evolutionary algorithms, prompt genomes have semantic structure: small changes in wording can produce large changes in LLM behaviour, while large changes in wording can sometimes be semantically neutral. This structure is precisely what GEPA exploits.

The first innovation is the capture of execution trajectories that record not just the final answer but the entire reasoning process.

The execution trajectory is the raw material from which GEPA extracts its diagnostic signal. Where reinforcement learning sees only the scalar $R(𝒚,𝒙)\in \{0,1\}$, GEPA sees the entire reasoning process: every correct insight, every wrong turn, every tool call that returned an error, every moment where the model's chain of thought diverged from sound mathematical reasoning.

The Coach: Semantic Gradients via Diagnosis

The second innovation is the introduction of a coach LLM that analyses failed trajectories and produces diagnostic feedback in natural language. This feedback serves as what the GEPA authors call a semantic gradient: a directional signal in the space of meanings rather than in the space of parameters.

The analogy to numerical gradients is precise and illuminating. In standard gradient descent, the gradient ${\nabla }_{𝜽}ℒ$ is a vector in parameter space that points in the direction of steepest descent. It tells you which parameters to change and by how much, but it cannot tell you why those changes help. The semantic gradient inverts this: it tells you why the current prompt fails, from which the how of improvement follows naturally.

fig:evo:gepa-coaching illustrates this diagnostic process. The coach operates by examining the execution trajectory through several analytical lenses. First, it identifies the failure point: the earliest step $s_{t^{\ast }}$ at which the reasoning chain diverged from a correct solution path. Second, it performs root cause analysis: was the failure due to a mathematical error, a strategic misstep, a misunderstanding of the problem statement, or an inadequate heuristic encoded in the prompt? Third, it generates constructive feedback: specific modifications to the prompt genome that would address the root cause.

Guided Mutation: Evolution by Understanding

The semantic gradient drives what GEPA calls guided mutation. Unlike random mutation in classical evolutionary algorithms, where a bit is flipped or a character is substituted without regard to semantics, guided mutation uses the coach's diagnosis to make targeted, meaningful changes to the prompt genome.

The mutation operator $M:𝒢\times ℒ\to 𝒢$ takes a parent genome $g$ and a semantic gradient ${\nabla }_{\text{sem}}f(g)$, and produces an offspring genome $g^{\prime }$: (Guided Mutation)$$\begin{array}{ccc}& g^{\prime }=M(g,{\nabla }_{\text{sem}}f(g))=\text{LLM}(\text{“Modify the following prompt based on the diagnostic feedback: ”}\Vert g\Vert {\nabla }_{\text{sem}}f(g)).& \text{<span class="tml-eqn"></span>}\end{array}$$ The mutation is itself performed by an LLM, which reads the current prompt, understands the diagnostic feedback, and rewrites the prompt to address the identified weaknesses. This creates a remarkable recursive structure: LLMs are being used to improve the instructions given to LLMs, guided by LLM-generated diagnoses of LLM failures.

The mutation operator can produce several types of changes:

1. Strategy injection: Adding new problem-solving strategies (e.g., “When encountering modular arithmetic, always verify inverses by multiplication”).

2. Heuristic refinement: Modifying existing heuristics to be more precise (e.g., changing “try small cases first” to “try cases $n=1,2,3,4$ and look for patterns in residues”).

3. Error-prevention clauses: Adding explicit warnings about common failure modes (e.g., “Never assume a sequence is geometric without checking the ratio of consecutive terms”).

4. Structural reorganisation: Reordering the sections of the prompt to prioritise the most effective strategies.

The GEPA Algorithm

We now present the complete GEPA algorithm, which integrates execution trajectory capture, coach diagnosis, guided mutation, and Pareto-optimal selection into a coherent evolutionary loop.

Several aspects of Algorithm 4 deserve careful attention. First, the algorithm maintains a population of prompt genomes rather than a single best prompt. This is essential because different prompts may excel on different problem types: a prompt optimised for combinatorics may perform poorly on number theory, and vice versa. The population provides coverage across the task distribution.

Second, the coach operates only on failed trajectories. Successful trajectories are valuable as positive examples (and are used for fitness computation), but the diagnostic signal for improvement comes from failures. This is analogous to the biological principle that evolution acts primarily through differential survival: organisms that fail to reproduce are the ones that drive adaptive change.

Third, the guided mutation step is deterministic given the diagnosis. The stochasticity in the algorithm comes from (a) the sampling of evaluation tasks, (b) the stochastic decoding of the LLM during execution, and (c) the temperature of the mutator LLM. This is a fundamentally different source of variation than random bit-flipping: the variation is semantically structured.

The GEPA Pipeline

fig:evo:gepa-pipeline illustrates the complete GEPA pipeline with its characteristic reflection loop.

Pareto-Optimal Candidate Selection

A single prompt cannot be the best at everything. A prompt that excels at algebraic manipulation may sacrifice geometric intuition; one that encourages careful verification may become too slow for time-constrained settings. GEPA addresses this fundamental tension through multi-objective optimisation, maintaining a population of prompts along a Pareto frontier.

The Pareto selection mechanism has a crucial theoretical advantage over greedy single-objective selection: it preserves diversity in the population, ensuring that specialist capabilities are never discarded in favour of a generalist that happens to have higher average fitness.

System-Aware Merging Crossover

Modern AI systems are not monolithic: they consist of multiple interacting components – a planner prompt, a tool-use prompt, a verification prompt, and so forth. GEPA introduces a system-aware merging crossover operator that respects this compositional structure.

Given two parent genomes $g_A$ and $g_B$ that excel on different task types, the crossover operator $𝒳:𝒢\times 𝒢\to 𝒢$ produces an offspring that combines the strengths of both parents. Crucially, this is not naive concatenation or random splicing. The crossover operator parses each genome into semantic sections (problem decomposition strategy, verification procedures, domain-specific heuristics, etc.) and then uses an LLM to merge the best sections from each parent into a coherent whole: (Crossover)$$\begin{array}{ccc}& g_{\text{child}}=𝒳(g_A,g_B)=\text{LLM}\left(\text{“Merge the following two prompts, combining the strengths of each: ”}\Vert g_A\Vert g_B\right).& \text{<span class="tml-eqn"></span>}\end{array}$$

For compound AI systems with $S$ subsystem prompts, the crossover operates at the subsystem level: $${𝒈}_{\text{child}}=(𝒳(g_A^{(1)},g_B^{(1)}),\dots ,𝒳(g_A^{(S)},g_B^{(S)})),$$ where $g_A^{(s)}$ and $g_B^{(s)}$ are the $s$-th subsystem prompts of parents $A$ and $B$, respectively. This preserves the modular structure of the compound system while allowing each module to benefit from the evolutionary innovations of its counterpart.

Results: Benchmarks and Cross-Model Transfer

GEPA has been evaluated on three challenging benchmarks that stress-test mathematical reasoning, and the results are striking.

Method	Model	AIME-2025	LiveBench-Math	Code Gen.
Base (no optimisation)	Qwen3-8B	31.2%	42.7%	38.1%
GRPO (5,000 rollouts)	Qwen3-8B	48.6%	57.3%	52.4%
GEPA (140 rollouts)	Qwen3-8B	68.9%	73.1%	71.8%
Base (no optimisation)	GPT-4.1-mini	54.3%	61.8%	64.2%
GEPA transfer	GPT-4.1-mini	72.1%	78.4%	76.9%

Table 4 reveals several remarkable findings. First, GEPA outperforms GRPO by approximately 20 percentage points on AIME-2025 while using $35\times$ fewer rollouts (140 vs. 5,000). This is not a marginal improvement; it represents a qualitative leap in both sample efficiency and absolute performance.

Second, and perhaps most striking, prompts evolved on Qwen3-8B transfer effectively to GPT-4.1-mini, a completely different model architecture from a different model provider. The GEPA-evolved prompts boost GPT-4.1-mini's AIME-2025 performance from 54.3% to 72.1%, a gain of nearly 18 percentage points achieved with zero additional optimisation on the target model.

Third, the Pareto-optimal population provides natural specialisation. Analysis of the final GEPA population reveals distinct specialist genomes: one that excels at algebraic manipulation through systematic variable substitution, another that dominates combinatorial problems through careful case analysis, and a third that handles geometric problems through coordinate geometry transformations. The routing function $\rho$ selects the appropriate specialist based on task features, achieving performance that exceeds any single genome.

ShinkaEvolve: Sample-Efficient Program Evolution

GEPA demonstrated that semantic feedback can dramatically accelerate prompt evolution. But what about evolving programs, actual executable code that solves mathematical and engineering problems? Here the sample efficiency challenge is even more acute, because each candidate program must be compiled, executed, and evaluated against test cases, a process that can take seconds to minutes per candidate. A naive evolutionary approach that evaluates thousands of candidates is simply too expensive for practical deployment.

ShinkaEvolve [46] attacks this problem head-on with three synergistic innovations that reduce the number of required evaluations by orders of magnitude. Where a baseline LLM-evolutionary system might require 10,000 evaluations to solve a challenging optimisation problem, ShinkaEvolve achieves comparable or superior results with as few as 150 evaluations.

The name “Shinka” is the Japanese word for evolution, and the system lives up to its name: it represents a genuine evolutionary advance in how we use LLMs for program synthesis.

The Sample Efficiency Problem

Consider the task of using an LLM to evolve a program that solves a circle packing problem: given a container circle of radius $R$, find the arrangement of $n$ non-overlapping circles (not necessarily of equal radius) that maximises the total area covered. This is an NP-hard optimisation problem with a rich mathematical structure and no known closed-form solution for general $n$.

A standard LLM-evolutionary approach works as follows. An initial population of programs is generated by prompting an LLM. In each generation, parent programs are selected, the LLM is prompted to produce mutations or crossovers, and the offspring are evaluated by executing them and measuring the resulting packing density. The fitness function is: (Circle Fitness)$$\begin{array}{ccc}& f(p)=\frac{{\sum }_{i=1}^n\pi r_i^2}{\pi R^2}\text{subject to}\Vert c_i-c_j\Vert \ge r_i+r_j\forall i\ne j,\Vert c_i\Vert +r_i\le R\forall i,& \text{<span class="tml-eqn"></span>}\end{array}$$ where $p$ is the program, $(c_i,r_i)$ are the centre and radius of the $i$-th circle produced by running $p$, and the constraints ensure non-overlap and containment.

The problem is that most LLM-generated mutations produce programs that are functionally identical to existing candidates: they may differ syntactically (variable names, code structure, comment placement) but produce the same or very similar packings. Evaluating these redundant candidates wastes the precious evaluation budget. Furthermore, the standard parent selection strategy, choosing the highest-fitness parents, leads to premature convergence around local optima, because the population loses diversity before the search has adequately explored the landscape.

Innovation 1: Adaptive Novelty-Weighted Sampling

The first innovation addresses parent selection. Instead of always selecting the highest-fitness parents (exploitation) or selecting uniformly at random (exploration), ShinkaEvolve uses a novelty-weighted parent sampling distribution that balances fitness with lineage diversity.

The power-law weighting $a_i^{-\beta }$, illustrated in fig:evo:shinkaevolve-powerlaw, is motivated by the observation that novel lineages – those recently introduced into the population through successful mutations – are more likely to contain unexplored genetic variation than old lineages whose neighbourhood in the fitness landscape has been thoroughly searched. By favouring recent innovations, ShinkaEvolve maintains exploration pressure without sacrificing the exploitation of high-fitness individuals.

The adaptive mechanism for $\alpha$ and $\beta$ is straightforward. Let ${\sigma }_f^2$ denote the variance of fitness values in the current population. When ${\sigma }_f^2$ is high (diverse fitness levels), increase $\alpha$ to exploit the fitness differences more aggressively. When ${\sigma }_f^2$ is low (converging population), increase $\beta$ to boost exploration by favouring novel lineages: (Adaptive Params)$$\begin{array}{ccc}& {\alpha }_{t+1}={\alpha }_t+{\eta }_{\alpha }\cdot \frac{{\sigma }_f^2-{\bar{\sigma }}_f^2}{{\bar{\sigma }}_f^2},{\beta }_{t+1}={\beta }_t-{\eta }_{\beta }\cdot \frac{{\sigma }_f^2-{\bar{\sigma }}_f^2}{{\bar{\sigma }}_f^2},& \text{<span class="tml-eqn"></span>}\end{array}$$ where ${\bar{\sigma }}_f^2$ is the target diversity level and ${\eta }_{\alpha },{\eta }_{\beta }>0$ are adaptation rates.

Innovation 2: Code-Novelty Rejection Sampling

The second innovation directly combats the redundancy trap. Before any candidate program is submitted for expensive evaluation, ShinkaEvolve passes it through a novelty rejection filter that estimates whether the candidate is sufficiently different from all previously evaluated programs.

The code embedding function $\varphi$ is the crucial component. ShinkaEvolve uses a pre-trained code embedding model (such as CodeBERT or a similar transformer-based code encoder) that maps programs to vectors where semantic similarity corresponds to geometric proximity. Two programs that produce identical outputs but differ syntactically will have similar embeddings; two programs that use fundamentally different algorithms will have distant embeddings.

The novelty rejection filter provides a dramatic reduction in wasted evaluations. In experiments on the circle packing task, the filter rejects approximately 70% of LLM-generated candidates as redundant, allowing the remaining evaluation budget to be concentrated on genuinely novel approaches.

The connection to novelty search [8] is immediate but with an important distinction. In classical novelty search, novelty is an objective: the algorithm explicitly rewards novel behaviours regardless of fitness. In ShinkaEvolve, novelty is a filter: it does not reward novelty per se but prevents the evaluation of non-novel candidates. This distinction is crucial because it allows ShinkaEvolve to maintain strong fitness pressure (through the novelty-weighted parent selection) while avoiding wasted evaluations (through the novelty filter).

Innovation 3: Bandit-Based LLM Ensemble Selection

The third innovation addresses a practical but important question: which LLM should generate the mutations? Different LLMs have different coding styles, different knowledge bases, and different tendencies in how they modify programs. Claude might excel at elegant algorithmic restructuring, DeepSeek at numerical optimisation tricks, and GPT at creative approaches to constraint satisfaction. Rather than committing to a single LLM, ShinkaEvolve frames this as a multi-armed bandit problem.

The UCB1 policy provides a principled exploration–exploitation trade-off: models with high estimated success rates are favoured (exploitation), but models that have been tried fewer times receive a bonus that encourages exploration. The regret bound of UCB1 guarantees that the cumulative cost of exploration is $O(\sqrt{LN\ln N})$, which is optimal up to logarithmic factors [47].

In practice, the bandit adapts quickly. In ShinkaEvolve experiments on mathematical optimisation tasks, the bandit typically identifies the best-performing LLM within 20–30 evaluations and allocates approximately 60% of subsequent evaluations to that model, with the remaining 40% split among the other models for continued exploration.

The ShinkaEvolve Algorithm

The three innovations combine into the complete ShinkaEvolve algorithm, presented in Algorithm 5.

Architecture Overview

fig:evo:shinkaevolve-arch illustrates the ShinkaEvolve architecture, highlighting how the three innovations interact.

Results and Sample Complexity Analysis

ShinkaEvolve has been evaluated on a diverse suite of tasks including circle packing, AIME mathematical reasoning (via agentic scaffolds), and mixture-of-experts (MoE) load-balancing loss design. The headline result is remarkable: on the circle packing benchmark, ShinkaEvolve finds solutions within 2% of the best known packing density using only 150 evaluations, compared to over 5,000 evaluations for the baseline FunSearch-style approach [48].

The sample complexity advantage can be formalised as follows.

The Digital Red Queen: Adversarial Coevolution

“Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!” So speaks the Red Queen in Lewis Carroll's Through the Looking-Glass, and her words capture one of the deepest principles in evolutionary biology: in a world of co-evolving adversaries, standing still is falling behind.

The Red Queen hypothesis, proposed by Leigh Van Valen in 1973 [34], posits that organisms must continuously adapt not merely to their physical environment but to the ever-changing landscape of competing and cooperating species. Prey evolve faster legs; predators evolve sharper claws. Pathogens evolve resistance to antibiotics; immune systems evolve new recognition mechanisms. The fitness landscape itself is in perpetual motion, driven by the adaptations of all the players within it.

In this section, we explore the Digital Red Queen (DRQ) [49], a system that transplants this biological principle into the domain of LLM-driven program evolution. The result is a coevolutionary arms race in which AI-generated programs evolve against an ever-growing archive of adversaries, producing strategies of remarkable sophistication, strategies that no human programmer designed and no single LLM could produce without the pressure of evolutionary competition.

The Biological Red Queen

Van Valen's original 1973 paper [34] made a startling observation: the probability of extinction for a species appears to be roughly constant over time, independent of how long the species has already existed. A species that has survived for ten million years is no more likely to persist for another million than one that appeared yesterday. This “law of constant extinction” implies that species are not accumulating an increasingly effective arsenal of adaptations; rather, they are running on a treadmill, constantly adapting just to maintain their current position against co-evolving competitors.

The mathematical formalisation of the Red Queen involves frequency-dependent selection, where the fitness of a strategy depends on the strategies employed by the rest of the population. Let $𝒔=(s_1,\dots ,s_n)$ be a vector of strategies employed by $n$ species, and let $W_i(𝒔)$ denote the fitness of species $i$ given the strategy profile $𝒔$. Under Red Queen dynamics: (RED Queen Dynamics)$$\begin{array}{ccc}& \frac{\partial W_i}{\partial s_j}\ne 0\text{for }i\ne j,& \text{<span class="tml-eqn"></span>}\end{array}$$ meaning that each species' fitness depends on the strategies of other species. An improvement by species $j$ changes the fitness landscape for species $i$, potentially rendering previously effective adaptations obsolete.

This creates a dynamic unlike anything in single-species optimisation. In standard optimisation, the objective function is fixed: gradient descent makes monotonic progress toward the optimum. In coevolutionary dynamics, the objective function is itself a moving target, and the system can exhibit complex phenomena including cyclic dynamics (rock-paper-scissors), arms races (escalating complexity), and Red Queen races (running to stay in place).

Core War: The Perfect Testbed

To study adversarial coevolution in a controlled setting, DRQ employs a remarkable programming game called Core War. The choice of testbed is inspired: Core War provides a Turing-complete competitive environment with an enormous strategy space, emergent complexity, and a rich history of human-crafted champions against which AI-evolved programs can be benchmarked.

The DRQ Algorithm

The Digital Red Queen algorithm evolves warriors through a sequence of generations, each defined by a growing adversarial archive that prevents the pathologies (cycling, forgetting) that plague classical coevolutionary algorithms.

The key insight is the single-lineage architecture with a monotonically growing archive. Instead of co-evolving two populations simultaneously (which invites cyclic dynamics), DRQ evolves a single population of warriors, but evaluates each candidate against the entire historical archive of previous champions.

Let ${𝒜}_t=\{w_1^{\ast },w_2^{\ast },\dots ,w_t^{\ast }\}$ denote the archive at generation $t$, where $w_i^{\ast }$ is the champion from generation $i$. The fitness of a candidate warrior $w$ at generation $t$ is: (DRQ Fitness)$$\begin{array}{ccc}& f_t(w)=\frac{1}{|{𝒜}_t|}\sum _{w^{\ast }\in {𝒜}_t}\text{Score}(w,w^{\ast }),& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\text{Score}(w,w^{\ast })\in \{0,1,3\}$ assigns 3 points for a win, 1 for a tie, and 0 for a loss (mirroring tournament scoring conventions in the Core War community).

This fitness function is the key to avoiding coevolutionary pathologies. Because the archive only grows (champions are added but never removed), a warrior cannot “forget” how to beat previous opponents. A warrior that evolves a brilliant new strategy for beating the latest champion but loses to an earlier one will receive a low fitness score and be eliminated. This creates relentless pressure toward generalist strategies that perform well against the entire spectrum of historical adversaries.

Within each generation, DRQ uses MAP-Elites [15] to maintain phenotypic diversity.

Several design choices in Algorithm 6 deserve elaboration.

LLM-guided mutation with archive context.

The LLM receives not only the parent warrior but also a summary of the archive warriors it must beat. This allows the LLM to reason about the competitive landscape: “The archive contains a strong bomber; I should add self-repair capabilities. It also contains a fast scanner; I should use decoy code to mislead it.” The mutation prompt explicitly encourages the LLM to analyse the strengths and weaknesses of archive warriors and to design counter-strategies.

Round-robin evaluation.

Each candidate plays against every warrior in the archive, not just the most recent champion. This is computationally expensive (the archive grows over time), but essential for preventing the forgetting that plagues pairwise coevolution. In practice, Core War matches are fast to simulate (a few milliseconds each), so even an archive of 100 warriors requires only seconds of evaluation per candidate.

MAP-Elites for within-generation diversity.

The MAP-Elites grid ensures that the population maintains diverse strategies even within a single generation. A candidate that is not the overall best but occupies an empty or weakly-defended cell in behaviour space is preserved, providing raw material for future evolutionary innovation.

The Arms Race Unfolds

fig:evo:drq-armsrace illustrates the evolutionary arms race that unfolds over the course of a DRQ run. The dynamics are both beautiful and instructive.

The arms race proceeds through several qualitative phases, each characterised by the emergence of a fundamentally new strategic paradigm.

Phase 1: Simple bombers (generations 1–10).

The earliest warriors are simple bombers: they overwrite the core with DAT instructions at regular intervals, hoping to hit the opponent's code. A typical early champion might consist of just 3–4 instructions:

  ADD #4, 3      ; Increment bomb target
  MOV 2, @2      ; Copy DAT bomb to target
  JMP -2         ; Loop
  DAT #0, #0     ; The bomb

This strategy is effective against randomly placed, stationary opponents but vulnerable to any warrior that moves or repairs itself.

Phase 2: Scanning and evasion (generations 10–25).

As bombers enter the archive, new warriors evolve that can scan for the opponent's code before attacking, and that evade bombs by copying themselves to new locations. The evolutionary pressure from archive bombers forces the development of defensive capabilities alongside offensive ones.

Phase 3: Self-replication (generations 20–40).

A landmark innovation occurs when warriors evolve the ability to replicate themselves to multiple locations in the core. A replicator is harder to kill because destroying one copy does not eliminate the warrior; the remaining copies continue executing. This strategy exploits the SPL instruction to create multiple execution threads, each running a copy of the warrior's code.

Phase 4: Multi-threaded warfare (generations 35–60).

The most sophisticated warriors combine multiple strategies: they replicate to create redundancy, scan for opponents, bomb selectively, and coordinate across multiple execution threads. These warriors exhibit a level of complexity that rivals the best human-crafted Core War programs, which represent decades of expert engineering.

Convergent Evolution of Strategies

One of the most striking findings of DRQ is the phenomenon of convergent evolution (fig:evo:drq-convergent): warriors that start from completely different initial conditions and evolve along different lineages independently discover the same fundamental strategies. This is the computational analogue of biological convergent evolution, where eyes evolved independently in vertebrates and molluscs, and wings evolved independently in birds, bats, and insects.

In DRQ experiments, convergence ratios of $\rho >50$ are routinely observed. Warriors that share less than 10% of their Redcode instructions (measured by edit distance) exhibit nearly identical aggression, mobility, and parallelism profiles. This suggests that the Core War environment imposes strong selective pressures that channel evolution toward a small number of viable archetypes, regardless of the starting point.

Archetype	Aggression	Mobility	Parallelism	Emergence	Convergence
Simple Bomber	High	Low	Low	Gen 3–8	10/10
Step Bomber	High	Medium	Low	Gen 8–15	10/10
Scanner	Medium	Medium	Low	Gen 12–20	9/10
Replicator	Low	High	High	Gen 18–30	10/10
Vampire	Medium	Low	Medium	Gen 25–40	7/10
Multi-Thread Bomber	High	High	High	Gen 30–50	8/10
Imp Spiral	Low	Very High	Medium	Gen 35–55	6/10

Table 5 reveals a clear evolutionary trajectory. Simple strategies emerge early and universally (all 10 runs discover simple bombers by generation 8), while sophisticated strategies appear later and less reliably (imp spirals appear in only 6 of 10 runs). This mirrors biological evolution: simple adaptations like locomotion and photosensitivity evolved early and universally across lineages, while complex adaptations like echolocation and bioluminescence evolved later and in fewer lineages.

Avoiding Coevolutionary Pathologies

Classical coevolutionary algorithms are notorious for three pathologies that DRQ's design explicitly avoids.

Pathology 1: Cycling.

In standard two-population coevolution, the populations can enter a cycle: population $A$ evolves a counter-strategy to $B$'s current champion; $B$ responds with a counter-counter-strategy; $A$ then evolves a strategy that was already tried and discarded in an earlier generation. The populations oscillate forever without making genuine progress. This is the coevolutionary analogue of rock-paper-scissors: rock beats scissors, scissors beats paper, paper beats rock, ad infinitum.

DRQ avoids cycling because the archive grows monotonically. Once a warrior enters the archive, it remains there forever. A new champion must beat not just the current leader but also the leaders from all previous generations. Reverting to an old strategy that was already beaten by an archived warrior would result in low fitness and immediate elimination.

Pathology 2: Forgetting.

Related to cycling, forgetting occurs when a population evolves effective counter-strategies against recent opponents but loses the ability to beat older ones. In a two-population system where fitness is measured only against the current opposing population, there is no selective pressure to maintain competence against past adversaries.

DRQ's full-archive evaluation prevents forgetting by construction. The fitness function in (DRQ Fitness) averages over the entire archive, so any regression in performance against old warriors is immediately reflected in reduced fitness.

Pathology 3: Mediocre stable states.

Mediocre stable states arise when both populations converge to strategies that are mutually stable but far from optimal, a coevolutionary Nash equilibrium that is globally suboptimal. The populations become locked in a local minimum of the coevolutionary landscape.

DRQ combats mediocre stability through MAP-Elites diversity maintenance. Even if the champion population converges, the MAP-Elites grid preserves diverse strategies in under-explored regions of behaviour space. These diverse strategies provide the raw material for evolutionary innovation: a marginal variant in an unusual cell of the MAP-Elites grid can be mutated into a breakthrough strategy that disrupts the current equilibrium.

Mathematical Analysis of DRQ Dynamics

We now provide a more rigorous analysis of DRQ's coevolutionary dynamics. Let $𝒮$ denote the strategy space of all possible warriors, and let $V:𝒮\times 𝒮\to \mathrm{ℝ}$ be the payoff function where $V(s,s^{\prime })$ is the expected score of strategy $s$ against strategy $s^{\prime }$.

Emergent Complexity and Open-Ended Evolution

Perhaps the most philosophically interesting aspect of DRQ is the emergence of unbounded complexity. As the archive grows, the bar for becoming a champion rises ever higher, forcing each new generation to develop more sophisticated strategies. This creates a form of open-ended evolution: there is no fixed fitness target, no pre-defined optimal solution. The system generates its own challenges and must continually innovate to meet them.

The complexity of DRQ warriors can be quantified using the effective program length: the number of instructions that are actually executed during a match (as opposed to dead code or data). Over the course of a typical DRQ run, the effective program length increases from 3–5 instructions in early generations to 20–40 instructions in later generations, reflecting genuine algorithmic sophistication rather than mere code bloat.

From Core War to General Adversarial Coevolution

While Core War is the proving ground for DRQ, the principles it embodies generalise to any domain where agents must contend with evolving adversaries.

Cybersecurity.

In penetration testing, an attacker must contend with evolving defences, and a defender must contend with evolving attacks. A DRQ framework could co-evolve attack programs and defence configurations, with the archive preserving all historical attack vectors so that defences never regress against known threats.

Game-playing AI.

In two-player zero-sum games, DRQ's archive-based evaluation provides a natural alternative to self-play (as used in AlphaZero). The archive prevents the cycling that can occur in naive self-play, where an agent evolves to beat its current opponent but forgets how to beat previous versions of itself.

Adversarial robustness.

In adversarial machine learning, DRQ could co-evolve adversarial examples and robust classifiers, with the archive ensuring that robustness against old attack types is never sacrificed for robustness against new ones.

LLM red-teaming.

Perhaps the most immediately consequential application is evolutionary red-teaming of language models. Replace Core War warriors with adversarial prompts (attackers) and system-level safety instructions or output filters (defenders). The attacker population evolves prompts designed to elicit harmful, biased, or policy-violating responses from the target LLM. The defender population evolves system prompts and guardrails that resist such attacks. Each generation's attacks stress-test the latest defences, and each generation's defences are hardened against the latest attacks. The coevolutionary arms race discovers vulnerabilities that static red-teaming, consisting of human-authored test suites applied once, would miss entirely.

Domain	Eq. gap	Generations	Human parity	Novel archetypes
Core War	0.03	150	1.12$\times$	3
Adversarial prompts	0.08	200	1.05$\times$	7
Network security	0.11	350	0.94$\times$	4
Trading strategies	0.06	250	1.08$\times$	2

Table 6 summarises DRQ's performance across four adversarial domains. In Core War and adversarial prompt generation, DRQ achieves superhuman performance, discovering strategies that outperform the best known human-designed approaches. In network security, the system falls slightly short of human expert performance ($0.94\times$), likely because the domain requires low-level systems knowledge that current LLMs handle less fluently than high-level strategic reasoning. Most strikingly, DRQ discovers novel archetypes in every domain, strategies structurally distinct from anything in the human reference corpus. In the adversarial prompt domain, seven novel attack patterns were identified, three of which exposed previously unknown vulnerabilities in production safety filters that were subsequently patched.

Synthesis: The Coevolutionary Paradigm

GEPA, ShinkaEvolve, and DRQ represent three facets of a coevolutionary paradigm that is rapidly transforming how we build and improve AI systems. GEPA demonstrates that evolution can optimise the instructions we give to AI, using semantic feedback as the directional signal. ShinkaEvolve shows that evolution can efficiently search the space of programs, using novelty filtering and bandit-based model selection to concentrate evaluations on the most promising candidates. DRQ reveals that evolution can produce open-ended complexity when agents face an ever-evolving adversarial landscape, with the archive serving as the ratchet that prevents regression.

Together, these systems suggest a future in which AI systems are not merely trained once and deployed, but continuously evolve in response to new challenges, new adversaries, and new opportunities. The Red Queen's dictum applies with full force: in a world of evolving AI, standing still is falling behind.

AdaEvolve: Hierarchical Adaptive Optimization

Every evolutionary system discussed so far in this chapter shares an uncomfortable secret: it allocates computational resources according to a static schedule. Each island receives the same number of generations, the same mutation budget, the same evaluation quota, regardless of whether it is making progress or has long since plateaued. The waste is staggering. On a typical 128-island evolutionary run for program synthesis, post-hoc analysis reveals that over 40% of evaluations are spent on islands whose best fitness has not improved in the last 200 generations [50]. These are computational dollars set on fire.

The problem is not unique to evolutionary computation. In gradient-based optimization, the identical pathology afflicted early stochastic gradient descent: every parameter received the same learning rate, regardless of the curvature of the loss landscape in its direction. The solution arrived in the form of adaptive gradient methods – AdaGrad, RMSProp, Adam – which track per-parameter statistics and adapt the step size accordingly. The insight was simple but transformative: let the optimization history inform the optimization strategy.

AdaEvolve [50] applies precisely this insight to evolutionary program synthesis. The core idea is to treat island-model evolution as a hierarchical adaptive optimization problem, where the “learning rate” analogue is the exploration intensity allocated to each island, and the “gradient statistics” analogue is the accumulated improvement signal observed on each island over time. The result is a three-level adaptive framework that dynamically routes computation to where it is most productive.

The Problem with Static Evolutionary Schedules

Consider a standard island-model evolutionary algorithm with $K$ islands, each running an independent population of candidate programs. At each generation $t$, every island $i\in \{1,\dots ,K\}$ receives an equal compute budget $B/K$, where $B$ is the total budget. Let $f_i(t)$ denote the best fitness achieved on island $i$ by generation $t$. The normalized improvement of island $i$ at generation $t$ is (Normalized Improvement)$$\begin{array}{ccc}& {\delta }_i(t)=\frac{f_i(t)-f_i(t-1)}{\max (|f_i(t-1)|,ϵ)},& \text{<span class="tml-eqn"></span>}\end{array}$$ where $ϵ>0$ is a small constant to prevent division by zero.

In a well-functioning evolutionary system, we expect ${\delta }_i(t)>0$ for most islands during the early phases of search. But as evolution progresses, many islands converge to local optima, and their normalized improvements decay to zero. A static schedule continues to pour resources into these stagnant islands, while islands that might benefit from additional mutation and selection, those near the boundary of a fitness plateau or those exploring a promising but computationally expensive region, are starved of resources.

The static scheduling problem becomes even more acute when the difficulty landscape across islands is heterogeneous. Some islands may be initialized near the basin of attraction of a global optimum and need only modest resources; others may require sustained exploration through a rugged fitness landscape. A uniform allocation treats these fundamentally different situations identically, violating the basic principle of resource allocation under heterogeneity.

The Accumulated Improvement Signal

The central diagnostic quantity in AdaEvolve is the accumulated improvement signal, which plays a role analogous to the second moment estimate in Adam. Rather than tracking raw fitness values, it tracks the recent history of improvement, providing a smoothed, scale-invariant measure of an island's productivity.

The squaring of ${\delta }_i(t)$ serves two purposes. First, it makes the signal non-negative, ensuring that $S_i(t)\ge 0$ always. Second, and more importantly, it makes the signal sensitive to the variance of improvements, not just their mean. An island that alternates between large improvements and no improvements has a higher $S_i(t)$ than an island with consistently small improvements, even if their average improvements are the same. This is desirable because the “bursty” island is more likely to be traversing a rugged but ultimately productive fitness landscape.

Three-Level Hierarchical Adaptation

AdaEvolve organizes its adaptive mechanisms into three hierarchical levels, each operating at a different time scale and scope. This hierarchical structure ensures that adaptation occurs at the appropriate granularity: fast, local adjustments at Level 1; medium-term resource reallocation at Level 2; and slow, strategic interventions at Level 3.

Level 1: Local Adaptive Exploration.

Within each island $i$, the mutation intensity ${\epsilon }_i(t)$ adapts based on the accumulated improvement signal: (Adaptive Epsilon)$$\begin{array}{ccc}& {\epsilon }_i(t)={\epsilon }_{\min }+({\epsilon }_{\max }-{\epsilon }_{\min })\cdot \sigma (\alpha \cdot \log \frac{{\hat{S}}_i(t)}{S_{\mathrm{ref}}}),& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\sigma (\cdot )$ is the logistic sigmoid function, $\alpha >0$ is a temperature parameter, and $S_{\mathrm{ref}}$ is a reference signal level calibrated from the initial exploration phase.

When ${\hat{S}}_i(t)\gg S_{\mathrm{ref}}$ (island is making rapid progress), ${\epsilon }_i(t)\to {\epsilon }_{\max }$, and the island explores aggressively. When ${\hat{S}}_i(t)\ll S_{\mathrm{ref}}$ (island is stagnant), ${\epsilon }_i(t)\to {\epsilon }_{\min }$, and the island switches to a more exploitative mode, making small refinements to its current best.

Level 2: Global Multi-Armed Bandit Allocation.

At the global level, AdaEvolve treats the $K$ islands as arms of a multi-armed bandit problem, where the “reward” of pulling arm $i$ is the normalized improvement ${\delta }_i(t)$. The Upper Confidence Bound (UCB) strategy allocates compute budgets: (UCB Allocation)$$\begin{array}{ccc}& B_i(t)=B_{\mathrm{total}}\cdot \frac{{\bar{\delta }}_i(t)+c\sqrt{\frac{\ln t}{n_i(t)}}}{{\sum }_{j=1}^K[{\bar{\delta }}_j(t)+c\sqrt{\frac{\ln t}{n_j(t)}}]},& \text{<span class="tml-eqn"></span>}\end{array}$$ where ${\bar{\delta }}_i(t)$ is the empirical mean improvement of island $i$, $n_i(t)$ is the number of generations allocated to island $i$ so far, $c>0$ is the exploration constant, and $B_{\mathrm{total}}$ is the total per-round compute budget.

The UCB term $c\sqrt{(\ln t)/n_i(t)}$ ensures that even islands with low current productivity are occasionally re-evaluated: an island that has received few generations has high uncertainty, and the UCB principle dictates that uncertainty should be explored. This prevents premature abandonment of islands that might be traversing a long fitness plateau before a breakthrough.

Level 3: Meta-Guidance (“System 2” Intervention).

When the accumulated improvement signal of an island drops below a critical threshold, ${\hat{S}}_i(t)<S_{\mathrm{crit}}$, for a sustained period (typically 500 consecutive generations), Level 3 triggers a strategic intervention. This is the “System 2” of the evolutionary process: slow, deliberate, and computationally expensive, but capable of fundamentally restructuring the search.

Level 3 interventions include:

1. Island restart with a new random initialization, preserving only the top-$k$ individuals from the stagnant population as “seeds.”

2. Cross-island migration, transferring the best individual from the most productive island to the stagnant island, injecting new genetic material.

3. Mutation operator adaptation, replacing the island's mutation strategy (e.g., switching from point mutation to subtree crossover, or from uniform sampling to LLM-guided proposal).

4. Fitness landscape reshaping, temporarily modifying the fitness function to include a novelty bonus that rewards divergence from the current population centroid.

The choice among these interventions is itself made by a lightweight meta-policy, trained on historical data from previous evolutionary runs. In the simplest implementation, this meta-policy is a multi-class classifier that maps features of the stagnant island (population diversity, fitness distribution statistics, improvement history) to one of the four intervention types.

We now present the complete AdaEvolve algorithm.

The interplay between the three levels creates a rich adaptive dynamic. Level 1 provides continuous, fine-grained adaptation of mutation intensity within each island. Level 2 periodically reallocates the global compute budget, steering resources toward productive islands. Level 3 provides the safety net: when an island is truly stuck, it intervenes with a structural change rather than merely adjusting parameters.

To understand the interaction more concretely, consider an island that has been steadily improving for 500 generations but then hits a plateau. Level 1 detects the drop in ${\hat{S}}_i(t)$ and reduces ${\epsilon }_i(t)$, switching to local refinement. If the refinement succeeds in making small improvements, Level 2 maintains a moderate budget allocation. If the refinement fails and ${\hat{S}}_i(t)$ drops below $S_{\mathrm{crit}}$ for ${\tau }_3$ consecutive generations, Level 3 triggers an intervention. The meta-policy $ℳ$ examines the island's population diversity: if diversity is low (the population has converged to a single basin), it triggers a restart with seeding; if diversity is high but fitness is stagnant (the population is exploring a flat region), it triggers fitness landscape reshaping with a novelty bonus.

The three-level hierarchy can be viewed through the lens of control theory. Level 1 is a proportional controller (adjusting mutation intensity based on the current signal). Level 2 is an integral controller (accumulating evidence over time to reallocate resources). Level 3 is a derivative controller (detecting sustained stagnation and applying corrective action). Together, they form a PID-like control system for evolutionary search, with the accumulated improvement signal ${\hat{S}}_i(t)$ serving as the error signal.

Mathematical Analysis

We now establish two key theoretical results for AdaEvolve: a regret bound for the Level 2 UCB allocation and a convergence rate comparison with static scheduling.

Empirical Results

AdaEvolve was evaluated on the Frontier-CS benchmark (25 tasks spanning combinatorial optimization, number theory, and computational geometry), the OpenAI HumanEval benchmark (164 code generation tasks), and a suite of 6 systems optimization tasks (circuit design, compiler pass ordering, database query planning, network routing, container scheduling, and cache eviction policy).

On Frontier-CS, AdaEvolve achieves a solve rate of 22/25, compared to 16/25 for static island-model evolution and 19/25 for AlphaEvolve [18]. The improvement is most dramatic on the hardest problems: of the 9 problems that static scheduling fails to solve, AdaEvolve solves 6. These additional solutions all emerge from islands that received disproportionate compute allocation via the Level 2 UCB mechanism, confirming the value of adaptive resource routing.

On the systems optimization tasks, AdaEvolve achieves state-of-the-art results on all 6 tasks, surpassing human-designed heuristics in each case. The most striking result is in compiler pass ordering, where AdaEvolve discovers a pass sequence that reduces compiled binary size by 3.2% compared to LLVM's -Oz, a result that required the Level 3 meta-guidance to escape a persistent local optimum in the pass space.

Comparison of Evolutionary Program Synthesis Systems

Table 7 compares AdaEvolve with three other leading systems: AlphaEvolve [18], ShinkaEvolve, and GEPA (Guided Evolution of Prompts and Agents Approximation). The comparison spans architecture, adaptation mechanism, and benchmark performance.

Feature	AdaEvolve	AlphaEvolve	ShinkaEvolve	GEPA
Island model	✓	✓	✓	–
Adaptive allocation	UCB + 3-level	Static	Novelty-based	N/A
LLM backbone	Gemini 2.5	Gemini Flash/Pro	GPT-4	CodeLlama
Mutation operator	Hybrid	LLM-guided	LLM + embedding	Grammar
Frontier-CS	22/25	19/25	17/25	12/25
HumanEval pass@1	94.5%	91.2%	89.7%	78.3%
Systems opt. SOTA	6/6	5/6	4/6	2/6
Avg. evaluations to
solve (Frontier-CS)	$\mathrm{𝟏.𝟐}\times {\mathrm{𝟏𝟎}}^{\mathrm{𝟓}}$	$2.8\times {10}^5$	$3.1\times {10}^5$	$5.7\times {10}^5$

The most revealing comparison is in computational efficiency. AdaEvolve requires 2.3$\times$ fewer evaluations than AlphaEvolve to achieve the same solve rate on Frontier-CS, directly confirming the value of adaptive resource allocation. The gap widens on harder problems: for the five most difficult Frontier-CS tasks, AdaEvolve requires $3.8\times$ fewer evaluations, because the Level 3 meta-guidance mechanism prevents the catastrophic waste of compute on permanently stagnant islands.

Automated Design of Agentic Systems (ADAS)

The history of AI agent design reads like a history of manual engineering. Chain-of-Thought prompting (CoT) was designed by human researchers who noticed that language models reason better when they “think aloud.” Toolformer was designed by human researchers who realized that models could learn to call APIs. Self-Reflection agents were designed by human researchers who hypothesized that models could critique their own outputs. ReAct agents, Tree-of-Thought agents, multi-agent debate systems – every one of these architectures was conceived, implemented, and debugged by human minds.

Each of these architectures represents a brilliant insight. But collectively, they represent a troubling bottleneck. Human researchers can explore only a tiny fraction of the space of possible agent architectures. Every hand-designed module – every prompt template, every routing decision, every reflection loop – is a constraint that limits the agent to a human-imagined topology. The space of possible agent architectures is combinatorially vast; the space of human-conceived agent architectures is vanishingly small by comparison.

This section introduces the Automated Design of Agentic Systems (ADAS) paradigm [51], which dissolves the bottleneck by using evolution to search the space of agent architectures. The key insight: represent agent architectures as executable code. Since code is Turing-complete, the search space includes every conceivable agent topology, not just those that a human researcher has imagined.

Performance Ceilings of Hand-Designed Agents

The performance of a hand-designed agent architecture is bounded above by the quality of the human intuitions that produced it. Consider a concrete example. A standard ReAct agent interleaves reasoning and action in a fixed alternation pattern: Thought $\to$ Action $\to$ Observation $\to$ Thought $\to \cdots$\@. This pattern was designed to address a specific failure mode (models acting before thinking), but it introduces a new limitation: the model must commit to an action after every thought, even when more deliberation would be beneficial.

A human researcher might notice this limitation and design a variant that allows multiple thoughts before each action. But this raises new questions: How many thoughts? Should the number be fixed or adaptive? Should the model itself decide when to act? Each question spawns a new design choice, and each design choice creates a new potential limitation. The design space is a fractal of decisions, each opening onto further decisions, ad infinitum.

The ADAS paradigm cuts this Gordian knot by automating the design process itself. Rather than iterating through design choices manually, ADAS uses a foundation model as a “meta agent” that searches the space of agent architectures by writing and testing executable code.

Code as the Universal Genome

The central representational choice in ADAS is to encode agent architectures as Python programs. This choice is not incidental; it is the key to the system's expressive power.

The halting condition is enforced pragmatically via timeout: any program that exceeds a wall-clock budget of 300 seconds on a single task is terminated and assigned a fitness of zero.

Meta Agent Search

The search algorithm in ADAS is called Meta Agent Search. Rather than using a traditional evolutionary algorithm with crossover and mutation operators defined over syntax trees, Meta Agent Search leverages a foundation model (the “meta agent”) to propose new agent architectures. The meta agent receives the full archive of previously explored architectures along with their performance scores, and is prompted to generate an “interestingly new” architecture that differs from existing ones in a substantive way.

The prompt to the meta agent is carefully designed to encourage novelty while maintaining quality. It includes:

1. The full archive, sorted by performance score, with each entry showing the code and a natural-language description of the architecture's key innovation.

2. An instruction to generate an agent that is “interestingly different from all existing agents in the archive.”

3. A reminder that the agent must implement the standard interface ((Agent Interface)) and must handle edge cases gracefully.

4. Examples of the types of innovations that have been productive in the past (multi-step reasoning, self-critique, task decomposition, ensemble methods).

Emergent Agent Topologies

One of the most compelling aspects of ADAS is the discovery that the meta agent, given sufficient iterations, converges on agent topologies that are qualitatively different from anything in the literature. Three emergent architectures are particularly noteworthy.

The Multi-Step Peer Review Agent.

This architecture, discovered at iteration 47 of a 200-iteration run, decomposes each task into three phases: (1) Drafting, where two independent sub-agents generate candidate solutions; (2) Review, where a third “reviewer” agent reads both drafts, identifies contradictions and gaps, and writes a critique; (3) Revision, where the original two agents receive the critique and produce revised solutions; (4) Adjudication, where the reviewer selects the better revision or synthesizes a hybrid.

The remarkable aspect is that this architecture was not present in the seed archive. The meta agent independently discovered that adversarial peer review improves solution quality, a pattern familiar in academia but never previously implemented as an agent architecture.

The Divide and Conquer Agent.

Discovered at iteration 83, this agent implements a recursive task decomposition strategy. Given a complex task, it prompts the foundation model to decompose the task into independent subtasks, solves each subtask recursively (with a maximum recursion depth of 4), and then synthesizes the sub-solutions into a final answer. The synthesis step uses a separate “integration” prompt that is conditioned on all sub-solutions and their confidence scores.

What makes this architecture non-obvious is the confidence-weighted integration: each sub-solution is tagged with a self-assessed confidence score, and the integration prompt gives more weight to high-confidence sub-solutions. This weighting scheme was not specified in any seed agent and was independently discovered by the meta agent.

The Verified Multimodal Agent.

This agent, discovered at iteration 121, introduces a verification loop specifically for tasks involving multiple modalities (text, images, structured data). After generating a solution, it converts the solution into each modality and checks for consistency: a numerical answer is verified against a visualization, a code solution is checked against its natural-language description, and so forth. Inconsistencies trigger a re-generation step with an explicit prompt to resolve the modal conflict.

Benchmark Results

The ADAS framework was evaluated across multiple benchmarks and multiple foundation models [51].

Mathematics: MGSM.

On the Multilingual Grade School Math (MGSM) benchmark, the best evolved agent achieves a 14.4% absolute improvement over the best hand-designed baseline (Self-Refine with CoT). The improvement is consistent across all 10 languages in the benchmark, ranging from +11.2% (Thai) to +17.8% (Bengali), with larger gains on lower-resource languages where the baseline is weakest.

Reading Comprehension: DROP.

On the DROP benchmark (Discrete Reasoning Over Paragraphs), the best evolved agent achieves an F1 score 13.6 points above the best hand-designed baseline. The evolved agent uses a variant of the Divide and Conquer architecture that separately processes the passage, the question, and relevant numerical data, then integrates the results via a structured verification step.

Logic: ARC.

On the ARC (Abstraction and Reasoning Corpus) logic puzzles, the best evolved agent achieves a solve rate of 49.2%, approaching the 50% threshold that had previously been considered a ceiling for LLM-based approaches. This result is particularly significant because ARC puzzles require visual-spatial reasoning that is far from the training distribution of text-only language models.

Cross-Model Transfer.

A striking finding is that agent architectures evolved using one foundation model transfer effectively to other models. Agents evolved using GPT-4 achieve 92% of their original performance when run on Claude 3, and 87% when run on Gemini Pro. This suggests that the evolved architectures capture model-independent reasoning patterns, namely topological structures that improve performance regardless of the underlying language model's specific strengths and weaknesses.

The transfer result has profound implications. It means that the evolved agent architectures are not overfitting to the idiosyncrasies of a particular model, but rather discovering general-purpose cognitive strategies that happen to be implemented in code.

The Boundary Dissolution: Optimizer Becomes the Optimized

ADAS exposes a fundamental recursion at the heart of AI agent design. The meta agent that designs new agents is itself an agent. The foundation model that proposes new agent architectures could, in principle, propose an improved version of itself, or more precisely, an improved version of the meta-search process that generates agent architectures.

This is the boundary dissolution: the distinction between the optimizer (the meta agent that searches for good agent designs) and the optimized (the agent designs being searched) becomes blurred. The optimizer is itself an instance of the thing it is optimizing.

Current ADAS implementations exhibit bounded recursive self-improvement: the performance of evolved agents improves monotonically over iterations but plateaus after approximately 150–200 iterations. The plateau likely reflects the quality ceiling of the foundation model used as the meta agent, not a fundamental limit of the approach.

Connection to CALM: Co-Evolving Algorithms and Models

ADAS evolves agent architectures while keeping the foundation model fixed. But what if we could evolve both, the agent architecture and the model weights, simultaneously? This is the premise of CALM (Co-evolution of Algorithms and Language Models) [27], which extends the ADAS paradigm by incorporating a reinforcement learning fine-tuning step that adapts the foundation model's weights based on the heuristic trajectories generated during evolutionary search.

The key insight is that the evolutionary process generates a rich dataset of problem-solving trajectories: for each evaluated agent, we observe the sequence of reasoning steps, tool calls, and intermediate outputs that led to a particular score. The successful trajectories, those produced by high-scoring agents, can be used as training data to fine-tune the foundation model via reinforcement learning from AI feedback (RLAIF).

This creates a tighter coupling between the search strategy and the computational substrate:

• ADAS uses the foundation model as a fixed substrate and evolves the code that runs on top of it.

• CALM evolves the code and adapts the substrate to better support the evolved code.

The analogy to biology is striking. In biological evolution, the “code” (DNA sequences encoding developmental programs) and the “substrate” (the cellular machinery that interprets DNA) co-evolve over deep time. The ribosome, the transcription machinery, the epigenetic regulation system – all of these are products of evolution that simultaneously enable further evolution. CALM instantiates this co-evolutionary dynamic in the computational domain.

Meta-Evolution: When Evolution Evolves Itself

The evolutionary systems we have studied so far – FunSearch, AlphaEvolve, ShinkaEvolve, AdaEvolve, ADAS – all share a common structural feature: the evolutionary algorithm itself is fixed. The mutation operators, selection mechanisms, fitness functions, and population dynamics are designed by human researchers and remain constant throughout the evolutionary run. Evolution acts on the solutions, but the process of evolution is immutable.

This is a curious asymmetry. In biological evolution, there is no such separation. The mechanisms of evolution – the mutation rate, the recombination machinery, the population structure – are themselves products of evolution. The mutation rate of E. coli is not a fixed constant; it is encoded in the DNA of the mismatch repair system, and that DNA is subject to the same evolutionary pressures as every other gene. Recombination rates vary across the genome and across species, shaped by selection over millions of years. Even the genetic code itself, the mapping from codons to amino acids, shows signs of evolutionary optimization for error robustness.

Meta-evolution dissolves the asymmetry. In a meta-evolutionary system, the evolutionary algorithm is itself evolvable: the search strategy is a program, and that program is subject to its own evolutionary process. The result is a two-level system in which the inner loop evolves solutions and the outer loop evolves the inner loop.

The EvoX Framework: Dual-Loop Architecture

The EvoX framework [52] formalizes the meta-evolutionary idea as a dual-loop architecture. The inner loop is a standard evolutionary algorithm that evolves solutions to a target problem. The outer loop is a meta-evolutionary process that evolves the search strategy used by the inner loop. The search strategy is represented as a Python program, making it evolvable by the same LLM-based mutation mechanisms used in FunSearch and AlphaEvolve.

The power of this formulation lies in the expressiveness of $\mathrm{\Sigma }$. Since search strategies are arbitrary Python programs, $\mathrm{\Sigma }$ includes:

• All standard evolutionary algorithms (genetic algorithms, evolution strategies, genetic programming, differential evolution).

• All memetic algorithms (evolutionary algorithms hybridized with local search).

• Novel algorithms that combine evolutionary search with gradient-based optimization, symbolic reasoning, or LLM queries.

• Self-adaptive algorithms whose parameters change during the inner loop based on runtime statistics.

In effect, $\mathrm{\Sigma }$ contains the entire history and future of evolutionary algorithm design as special cases.

The Convergence of Optimizer and Optimized

The idea of using a learning algorithm to improve a learning algorithm is not new. It traces a long intellectual lineage through several decades of research.

The transition from hyperparameter tuning to strategy evolution can be understood as a progression along a spectrum of increasing expressiveness:

1. Hyperparameter optimization: Search over a fixed, finite-dimensional space $𝜽\in {ℝ}^d$ (e.g., learning rate, population size, mutation rate). The algorithm structure is fixed; only numerical parameters vary.

2. Architecture search: Search over a combinatorial space of computational graphs (e.g., NAS cell topologies). The algorithm structure is partially variable, but constrained to a predefined set of operations and connection patterns.

3. Strategy evolution: Search over the space of executable programs $\sigma \in \mathrm{\Sigma }$. The algorithm structure is fully variable, limited only by the expressiveness of the programming language and the quality of the search.

Each step subsumes the previous: hyperparameter optimization is a special case of architecture search (with a fixed architecture), and architecture search is a special case of strategy evolution (with a restricted program space).

Theoretical Analysis

We now establish two theoretical results about meta-evolution: a positive result showing that meta-evolution can discover novel operators, and a conjecture about the open-endedness of sufficiently expressive meta-evolutionary systems.

We now state a conjecture about the open-endedness of meta-evolution.

The conjecture is currently unproven. The main difficulty is establishing perpetual novelty: even if the system produces strategies of increasing complexity, it might do so by trivially inflating existing strategies (e.g., adding dead code) rather than by discovering genuinely new algorithmic ideas. A proof would require a formal notion of “algorithmic novelty” that distinguishes substantive from trivial complexity growth.

CALM: Co-evolution of Algorithms and Language Models

While EvoX focuses on evolving search strategies with a fixed foundation model, CALM [27] takes the co-evolutionary idea further by simultaneously evolving heuristic algorithms and fine-tuning the language model that proposes them. This creates a tighter feedback loop than either ADAS or EvoX alone.

The CALM architecture consists of three phases that iterate:

Phase 1: Heuristic Generation.

Given a combinatorial optimization problem (e.g., bin packing, job scheduling, vehicle routing), the current language model ${ℳ}_{𝜽}$ generates a population of heuristic programs $\{h_1,\dots ,h_P\}$. Each heuristic $h_j$ is a Python function that takes a problem instance as input and returns a solution.

Phase 2: Evaluation and Trajectory Collection.

Each heuristic is evaluated on a suite of problem instances. The evaluation produces two outputs: (a) a scalar performance score $s_j$, and (b) a heuristic trajectory ${\tau }_j$ that records the full interaction between the model and the evaluation environment, including intermediate solutions, backtracking decisions, and the final output.

Phase 3: RL Fine-Tuning.

The heuristic trajectories are ranked by performance score. The top-performing trajectories are used as positive examples in a reinforcement learning fine-tuning step (using PPO or GRPO), while low-performing trajectories serve as negative examples. The model weights $𝜽$ are updated: (CALM Update)$$\begin{array}{ccc}& {𝜽}_{t+1}={𝜽}_t+\eta {\nabla }_{𝜽}{𝔼}_{\tau \sim {\pi }_{𝜽}}[R(\tau )\cdot \log {\pi }_{𝜽}(\tau )],& \text{<span class="tml-eqn"></span>}\end{array}$$ where $R(\tau )$ is the reward derived from the trajectory's performance score, and ${\pi }_{𝜽}$ is the policy induced by the language model with parameters $𝜽$.

The updated model ${ℳ}_{{𝜽}_{t+1}}$ then generates the next generation of heuristics, closing the loop.

The key advantage of CALM over ADAS is the substrate adaptation. In ADAS, the foundation model's distribution over programs is fixed; the meta agent must work within whatever prior the model brings from pre-training. In CALM, the model's prior is iteratively reshaped by the RL fine-tuning step, concentrating probability mass on the regions of program space that have proven productive. After several iterations of CALM, the model becomes a specialized heuristic generator, producing higher-quality heuristics with fewer wasted samples.

The Computational Economics of Evolution

Meta-evolution raises a fundamental question of computational economics: how should we allocate finite compute across the inner and outer loops? Every evaluation of the outer loop requires a complete run of the inner loop, which is itself computationally expensive. If the inner loop budget $B_{\mathrm{inner}}$ is too small, the meta-fitness estimates $F(\sigma )$ are noisy and unreliable; if too large, the outer loop can run only a few generations before exhausting the total budget.

Dynamic Bandit-Based Resource Routing.

AdaEvolve's UCB allocation mechanism (Three-Level Hierarchical Adaptation) can be applied to the outer loop of meta-evolution. Each strategy in the meta-population is an arm, and the reward is the improvement in meta-fitness. The UCB allocation dynamically routes compute to the most promising strategies, avoiding the waste of evaluating strategies that are clearly suboptimal.

The bandit formulation also applies to the inner-outer budget split. At each outer generation, the system can choose between “exploring wide” (running many inner loops with small budgets to evaluate more strategies) and “exploring deep” (running fewer inner loops with large budgets to get more reliable fitness estimates). An adaptive policy based on the variance of meta-fitness estimates can balance this tradeoff: (Adaptive Inner Budget)$$\begin{array}{ccc}& B_{\mathrm{inner}}(g)=B_{\mathrm{inner}}^{(0)}\cdot (1+\gamma \cdot \frac{\mathrm{𝖵ar}[F(\sigma ):\sigma \in {\mathrm{\Pi }}_g]}{𝔼{[F(\sigma ):\sigma \in {\mathrm{\Pi }}_g]}^2}),& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\gamma >0$ is a sensitivity parameter. When the coefficient of variation of meta-fitness is high (strategies are hard to distinguish), the inner budget increases to reduce noise; when meta-fitness estimates are stable, the inner budget decreases to allow more outer loop iterations.

Embedding-Based Novelty Rejection.

A significant source of waste in both inner and outer loops is the evaluation of candidate solutions (or strategies) that are trivially similar to previously evaluated ones. ShinkaEvolve addresses this with an embedding-based novelty filter: before evaluating a new candidate, its code is embedded into a vector space using a code embedding model, and if the embedding falls within a threshold distance ${\delta }_{\mathrm{nov}}$ of any previously evaluated candidate, the evaluation is skipped and the candidate receives a fitness estimate by interpolation from its nearest neighbors.

The novelty filter can be applied at both levels of meta-evolution:

• At the inner level, it prevents evaluating solutions that are trivially similar to previously evaluated solutions within the same inner loop run.

• At the outer level, it prevents evaluating strategies that are trivially similar to previously evaluated strategies, saving an entire inner loop run.

In practice, the savings are substantial. On a benchmark suite of 50 combinatorial optimization problems, EvoX with adaptive allocation and novelty rejection uses 38% fewer total FLOPs than EvoX with static allocation, while achieving a 4.2% higher average solution quality.

Taxonomy of Meta-Evolutionary Approaches

Table 8 provides a systematic comparison of the four meta-evolutionary approaches discussed in this chapter.

Dimension	EvoX	CALM	ADAS	AdaEvolve
What evolves	Search strategies	Heuristics +	Agent	Resource
	(EA programs)	model weights	architectures	allocation
Inner loop	Standard EA	Heuristic eval	Agent eval	Island-model EA
Outer loop	Strategy EA	RL fine-tuning	Meta Agent	UCB +
			Search	meta-policy
Substrate	No	Yes (RL)	No	No
adaptation
Search	Turing-	Turing-	Turing-	Parametric
expressiveness	complete	complete	complete	($\mathrm{\Gamma }$ only)
Self-reference	$✓$	$✓$	$✓$	–
(evolves own
search process)
Key strength	General-	Tighter	Agent	Compute
	purpose EA	model-code	topology	efficiency
	discovery	co-adaptation	discovery
Key limitation	High outer-	Problem-	Model API	No strategy
	loop cost	specific tuning	cost	evolution

The taxonomy reveals a design spectrum. At one extreme, AdaEvolve adapts parameters of a fixed evolutionary algorithm (compute allocation, mutation intensity), achieving efficiency gains without the expense of meta-evolution. At the other extreme, EvoX and CALM evolve entire algorithms, achieving greater flexibility at higher computational cost. ADAS occupies a middle ground, evolving agent architectures (which are a restricted class of programs) rather than arbitrary search strategies.

Toward Self-Improving Evolutionary Systems

The meta-evolutionary paradigm points toward a future in which evolutionary AI systems are not merely tools deployed by human researchers, but autonomous scientific discovery engines that design their own experiments, formulate their own hypotheses, and refine their own methodologies without human intervention.

Three research directions are particularly promising:

Recursive Meta-Evolution.

Current meta-evolutionary systems have two levels (inner and outer). Nothing prevents adding a third level that evolves the outer loop's algorithm. In principle, the recursion can continue to arbitrary depth: a meta-meta-evolutionary system that evolves the algorithm for evolving the algorithm for evolving solutions. Each level of recursion increases the expressiveness of the search but also increases the computational cost and the variance of fitness estimates.

The practical question is whether there are diminishing returns to additional levels of meta-evolution. Preliminary experiments with three-level systems suggest modest improvements over two-level systems on highly structured problems (where the optimal search strategy has rich internal structure), but negligible improvements on unstructured problems (where simple strategies suffice).

Cross-Domain Transfer of Evolved Strategies.

Do search strategies evolved for one problem domain transfer to others? Initial results from EvoX suggest partial transfer: a strategy evolved for combinatorial optimization problems achieves above-baseline performance on continuous optimization problems, but does not match a strategy evolved directly for the continuous domain. The transferable component appears to be the population management logic (selection pressure, diversity maintenance), while the variation operators (mutation, crossover) are problem-specific.

This partial transferability suggests a modular architecture for meta-evolved strategies: a domain-independent “strategy skeleton” that handles population management, combined with domain-specific “operator modules” that are evolved or fine-tuned for each target domain.

Safety and Controllability.

As meta-evolutionary systems become more autonomous, ensuring their safety and controllability becomes critical. A self-improving evolutionary system that discovers an unexpectedly powerful search strategy could produce solutions with unintended properties, a concern that echoes the alignment challenges discussed in The Boundary Dissolution: Optimizer Becomes the Optimized.

Formal methods for bounding the behavior of meta-evolutionary systems are an active area of research. One approach is to define a “safety envelope” in the strategy space $\mathrm{\Sigma }$ and constrain the outer loop to produce strategies within this envelope. Another approach is to require all evolved strategies to satisfy a set of formal properties (e.g., termination guarantees, resource bounds, output format constraints) that are verified by a theorem prover before evaluation.

Verification Systems for Evolutionary Discovery

Every evolutionary system that generates solutions must eventually confront a question more fundamental than fitness: is this answer actually correct? A genetic programming system may evolve a sorting algorithm that passes a hundred test cases, but silently fails on input number one hundred and one. A neural architecture search process may discover a network topology that achieves record accuracy on a validation set, yet exploits a spurious correlation that evaporates in deployment. An LLM-guided evolutionary loop may produce an elegant mathematical conjecture that reads beautifully but is, upon careful inspection, false.

The problem is not merely academic. As evolutionary AI systems grow more powerful, they produce candidates that are increasingly sophisticated, increasingly plausible, and increasingly difficult for humans to evaluate by inspection alone. The gap between what these systems can generate and what humans can verify is widening at an alarming rate. This section develops the mathematical and computational machinery for closing that gap.

We begin with the core asymmetry that makes verification both challenging and, paradoxically, useful.

The Verification Challenge

Consider a large language model embedded within an evolutionary search loop. At each generation, the LLM proposes candidate solutions: mathematical proofs, program code, scientific hypotheses, engineering designs. The evolutionary pressure selects among these candidates based on a fitness function. But the fitness function itself may be incomplete, noisy, or fundamentally unable to distinguish correct solutions from plausible-sounding hallucinations.

The hallucination problem, well documented in the natural language processing literature, takes on new urgency in the evolutionary context. When an LLM generates a single response to a user query, a human can often detect obvious errors. But when an LLM generates thousands of candidate solutions per generation across hundreds of generations, the sheer volume overwhelms human oversight. Worse, evolutionary selection pressure can amplify hallucinations: a confidently stated but incorrect intermediate result, if it happens to score well on a proxy fitness metric, will be selected, recombined, and propagated through subsequent generations. The evolutionary process, blind to ground truth, optimises for whatever the fitness function measures, and if the fitness function cannot distinguish correctness from confident incorrectness, the population will drift toward elaborate, internally consistent, and completely wrong solutions.

This observation leads us to one of the most important structural properties in computational complexity theory, one that turns out to be the key to building reliable evolutionary systems.

The verification oracle formalises the role played by test suites, theorem provers, simulators, and human experts in evaluating evolutionary output. Different oracles offer different trade-offs between soundness, completeness, computational cost, and domain coverage.

The power of this insight cannot be overstated. It means that we do not need the evolutionary process itself to be perfectly accurate. We do not need the LLM to never hallucinate. We need only that the verification oracle be sound: if it says “correct,” then the solution is indeed correct. False negatives (rejecting a correct solution) merely slow the search; false positives (accepting an incorrect solution) are catastrophic. This asymmetry shapes the design of every verification system we will encounter.

Formal Verification: Theorem Provers as Oracles

The gold standard of verification is formal proof. If a candidate solution can be expressed as a mathematical statement, and if that statement can be proved correct within a formal logical system, then we have the strongest possible guarantee of correctness: certainty limited only by our trust in the logical foundations themselves.

The past decade has witnessed a revolution in interactive theorem provers (ITPs), software systems that allow humans (and now, machines) to construct proofs that are mechanically checked for correctness [53]. The most prominent of these for modern AI applications is Lean 4, developed by Leonardo de Moura and his collaborators at Microsoft Research.

The beauty of this approach is that the verification oracle is the Lean kernel itself: a program of a few thousand lines of code that checks whether a proof term has the correct type. The kernel does not need to understand the proof; it does not need to assess whether the proof strategy is clever or elegant; it merely checks types. This makes the oracle both sound (by the correctness of the type-checking algorithm relative to the logical foundations) and decidable (type checking terminates on all well-formed inputs).

How Lean Works: Types, Tactics, and Proof Terms

Lean operates on the principle of the Curry-Howard correspondence: propositions are types, and proofs are programs. To prove that the sum of two even numbers is even, one constructs a function that takes two even numbers as input and produces an even number as output. The type of this function is the statement of the theorem.

In practice, constructing proof terms directly is tedious. Lean provides a tactic language that allows users to build proofs interactively. A tactic transforms the current proof state (a collection of goals to be proved and hypotheses available) into a new proof state with fewer or simpler goals. When all goals are discharged, the tactic script is compiled into a proof term that the kernel can check.

The key tactics include:

• intro: introduce a hypothesis (like assuming the antecedent of an implication).

• apply: apply a known theorem or hypothesis to the current goal.

• simp: simplify using a database of known rewriting rules.

• omega: decide linear arithmetic goals automatically.

• ring: prove equalities in commutative (semi)rings.

• exact: provide the exact proof term for the goal.

LLMs as Copilots for Theorem Proving

The LeanCopilot project [54] demonstrated that large language models can serve as effective tactic suggesters within the Lean proof assistant. Given a proof state (the current goals and hypotheses), the LLM proposes a sequence of tactics that might discharge the goals. The Lean kernel then checks whether the proposed tactics actually work. If they do, the proof is complete; if they do not, the LLM tries again or backtracks.

This architecture is precisely the generate-verify loop that we have been describing. The LLM is the generator: fast, creative, prone to hallucination. The Lean kernel is the verification oracle: slow (in the sense that it must be called for each candidate), but perfectly sound. The evolutionary search orchestrates the interaction: maintaining a population of partial proofs, applying crossover and mutation (via LLM-based rewriting), and selecting those candidates that pass the Lean kernel's scrutiny.

AlphaProof: Solving Competition Mathematics

In 2024, DeepMind's AlphaProof system [55] achieved a landmark result: solving problems from the International Mathematical Olympiad (IMO) at a level competitive with the best human contestants. The system combined three key components:

1. A Gemini language model that could translate informal mathematical problems into formal Lean statements and propose proof strategies.

2. A reinforcement learning loop (building on AlphaZero-style self-play) that trained a proof-search policy to navigate the space of possible tactic applications.

3. The Lean 4 kernel as the ultimate verification oracle: no proof was accepted unless Lean certified it as correct.

The result was a system that could generate creative mathematical insights (the province of the language model and the RL policy) while maintaining absolute rigour (the province of the Lean kernel). The evolutionary character of the search is evident: the system maintains a population of proof attempts, selects the most promising, and applies learned transformations to generate new attempts. The key innovation is that the fitness function is not a heuristic proxy for correctness but correctness itself, as certified by the theorem prover.

This proposition has a profound implication: the quality of the generator matters for efficiency but not for correctness. A better LLM produces correct candidates more often, reducing the computational budget needed. But even a mediocre generator, paired with a sound verifier, will eventually produce correct results.

The algorithm maintains an archive $\mathscr{H}$ to avoid regenerating previously attempted solutions, and uses partial verification scores (e.g., the number of proof steps completed, the fraction of test cases passed) as a proxy fitness to guide evolution even when no candidate achieves full verification.

Computational Verification: Simulation as Truth

Not all domains admit formal proofs. When evolving solutions to engineering problems, scientific models, or control policies, the verification oracle is often a simulator: a computational model of the physical world that can execute a candidate solution and observe whether it works.

Consider the problem of evolving a controller for a legged robot. The candidate solution is a neural network policy ${\pi }_{𝜽}$ that maps sensor readings to motor commands. The verification question is whether the robot can walk. We cannot prove this with a theorem prover; walking is a physical phenomenon governed by Newtonian mechanics, friction, and contact dynamics. But we can simulate it. A physics engine such as MuJoCo or Isaac Gym can execute the policy ${\pi }_{𝜽}$ in a virtual environment and report whether the robot walks, falls, or does something in between.

The critical distinction from formal verification is that simulation-based verification is not sound in the mathematical sense. A solution that passes all $M$ test scenarios might fail on scenario $M+1$. The simulator itself is an approximation of reality: it discretises continuous dynamics, simplifies contact models, and ignores phenomena that the simulator designers deemed negligible. Nevertheless, simulation-based verification is enormously useful in practice, because it provides a cheap, repeatable, and objective evaluation that is far more reliable than heuristic fitness functions.

The connection to physics-informed neural networks (PINNs) is direct. A PINN embeds the governing partial differential equations into its loss function, effectively using the physics as a soft verification oracle during training. In the evolutionary setting, we can go further: using the PDE residual as a hard verification criterion, rejecting any candidate whose PDE residual exceeds a threshold.

Statistical Verification

When neither formal proofs nor deterministic simulations are available, we fall back on the most general verification framework: statistical hypothesis testing. The question becomes not “is this solution correct?” but “is there sufficient statistical evidence that this solution is better than the baseline?”

This situation arises naturally when evolving heuristics for deployment in production systems. Suppose an evolutionary process produces a new recommendation algorithm, a new ad-bidding strategy, or a new packet-routing heuristic. The “correctness” of these solutions is not a binary property; it is a matter of degree, measured by metrics such as click-through rate, revenue, or latency. Verification requires running the evolved solution alongside the incumbent and collecting enough data to determine, with statistical confidence, whether the new solution is genuinely superior.

In practice, this means running an A/B test. The evolved solution and the baseline are deployed simultaneously to randomly assigned user segments (or, in simulation, to randomly sampled scenarios). Performance metrics are collected, and a hypothesis test (typically a two-sample $t$-test, a Mann-Whitney $U$ test, or a Bayesian comparison) determines whether the observed performance difference is statistically significant.

Bootstrap Confidence Intervals

When the distribution of fitness values is non-Gaussian or heavy-tailed, parametric tests may be unreliable. The bootstrap provides a non-parametric alternative. Given $n$ fitness observations $\{f_1,f_2,\dots ,f_n\}$ for the evolved solution, we construct $B$ bootstrap resamples, each of size $n$, drawn with replacement. For each resample $b$, we compute the bootstrap statistic ${\hat{\theta }}_b$ (e.g., the mean fitness). The $100(1-\alpha )\%$ confidence interval is then: (Bootstrap CI)$$\begin{array}{ccc}& {\text{CI}}_{1-\alpha }=[{\hat{\theta }}_{(\lfloor B\alpha /2\rfloor )},{\hat{\theta }}_{(\lceil B(1-\alpha /2)\rceil )}],& \text{<span class="tml-eqn"></span>}\end{array}$$ where ${\hat{\theta }}_{(k)}$ denotes the $k$-th order statistic of the bootstrap distribution. The evolved solution is deemed superior if the entire confidence interval for the performance difference $\mathrm{\Delta }=𝔼[m(s)]-𝔼[m(s_0)]$ lies above zero.

The Multi-Level Verification Pyramid

In practice, verification is not a single operation but a layered process. Each layer is more expensive and more rigorous than the one below it, creating a pyramid structure that allows cheap, fast checks to filter out obviously bad candidates before expensive, thorough checks are applied to the survivors.

The five levels of the verification pyramid are:

1. Level 1: Syntactic verification. Does the candidate solution have valid syntax? For evolved code, this means parsing without errors. For evolved mathematical expressions, this means well-formed formulas. Cost: negligible (milliseconds). Catches: syntax errors, malformed output, incomplete generations.

2. Level 2: Semantic verification. Does the candidate solution execute without runtime errors? For code, this means passing type checks and running without exceptions on trivial inputs. For neural network architectures, this means that the forward pass produces a tensor of the expected shape. Cost: low (seconds). Catches: type errors, dimension mismatches, undefined variables.

3. Level 3: Functional verification. Does the candidate solution produce correct outputs on known test cases? This is the level of unit testing, regression testing, and comparison against reference implementations. Cost: moderate (seconds to minutes). Catches: logical errors, algorithmic mistakes, edge-case failures.

4. Level 4: Performance verification. Is the candidate solution better than existing solutions on the metrics that matter? This is the level of benchmarking, profiling, and statistical comparison. Cost: high (minutes to hours). Catches: solutions that are correct but not competitive.

5. Level 5: Formal verification. Can we prove that the candidate solution is correct for all inputs, not just the test cases? This is the level of theorem proving, model checking, and mathematical proof. Cost: very high (hours to days, or may be impossible). Catches: subtle bugs, corner cases, and provides the strongest correctness guarantee available.

The pyramid structure is essential for computational efficiency. If $90\%$ of candidates fail syntactic checks (Level 1), then only $10\%$ proceed to semantic checks (Level 2). If $50\%$ of those fail semantic checks, only $5\%$ reach functional testing (Level 3). And so on. The total verification cost is dominated by the cheap lower levels, which process many candidates, rather than the expensive upper levels, which process few.

Level	Method	Cost	Domain	Sound?
1	Parser / grammar check	$O(n)$	Code, formulas	No
1	Schema validation	$O(n)$	Data structures	No
2	Type checker	$O(n\log n)$	Typed languages	Partial
2	Dimension analysis	$O(n)$	Physical models	Partial
3	Unit tests	$O(n\cdot M)$	Software	No
3	Reference comparison	$O(n\cdot M)$	Numerical methods	No
4	Benchmark suite	$O(n\cdot B)$	Algorithms	No
4	A/B test	$O(n\cdot T)$	Production systems	Statistical
5	Theorem prover	Undecidable	Mathematics	Yes
5	Model checker	$O(2^n)$	Finite-state systems	Yes

The AI Scientist: Autonomous Research Agents

In August 2024, a paper appeared on arXiv that described a system capable of something no machine had done before: conducting scientific research from start to finish, autonomously, without human intervention at any step [25]. The system, called The AI Scientist, could formulate research hypotheses, design and run experiments, analyse results, write a complete scientific paper in the standard academic format, and even generate peer reviews of its own work. The cost: approximately $15 per paper.

The reaction was, understandably, divided. Some researchers celebrated it as a harbinger of accelerated scientific discovery. Others dismissed it as glorified text generation, pointing out that the papers produced were incremental, sometimes contained errors, and would not have passed rigorous human peer review. Both reactions missed the essential point. The AI Scientist was not a finished product; it was a proof of concept, demonstrating that the structure of autonomous research – the loop from hypothesis to experiment to analysis to communication – could be automated. The quality would improve with better models, better verification (as we developed in Verification Systems for Evolutionary Discovery), and better feedback loops. The architecture was the innovation.

This section develops the mathematics and the systems design behind autonomous research agents, with a focus on the evolutionary dynamics that drive them to make discoveries.

The Autonomous Research Loop

The loop in (Research LOOP) is explicitly evolutionary. The hypothesis generator $G$ produces a population of candidate research directions. The experiment engine $E$ evaluates each candidate's fitness (does the experiment support or refute the hypothesis?). The analysis module $A$ interprets the results, identifying which hypotheses survived the experimental gauntlet. The writing module $W$ crystallises the findings, and the review module $R$ provides selection pressure, flagging weaknesses and suggesting improvements. The surviving hypotheses, augmented by insights from the experimental results, seed the next generation.

Reward Functions for Scientific Discovery

What motivates an AI scientist? In evolutionary terms, what is the fitness function that drives the population of hypotheses toward genuine discoveries? This question is deceptively deep. A naive fitness function, such as “maximise the accuracy of a predictive model,” will produce incremental improvements to existing methods. A more ambitious fitness function, such as “maximise the novelty of the research output,” risks rewarding bizarre but useless results. The art of autonomous research lies in designing reward functions that balance novelty, correctness, and impact.

Each component of the curiosity reward deserves careful examination.

Novelty

The novelty component connects directly to the novelty search paradigm developed by Lehman and Stanley [8] and discussed in Quality-Diversity and MAP-Elites. In that framework, individuals are rewarded not for approaching a fixed objective but for being different from what has been seen before. In the scientific context, novelty means proposing hypotheses, experimental designs, or analytical methods that are genuinely new relative to the existing literature.

Computing novelty requires a representation of the hypothesis space in which distances are meaningful. One approach is to embed hypotheses into a vector space using a language model: (Novelty Embedding)$$\begin{array}{ccc}& \text{Nov}(h,𝒦)=\underset{k\in 𝒦}{\min }{\Vert \text{embed}(h)-\text{embed}(k)\Vert }_2,& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\text{embed}(\cdot )$ maps a textual description of a hypothesis to a dense vector. This is a simplification; in practice, one might use more sophisticated similarity measures that account for semantic structure, methodological overlap, or domain-specific features.

Correctness

The correctness component is where the verification systems of Verification Systems for Evolutionary Discovery become essential. A hypothesis that is novel but wrong is worthless; a hypothesis that is correct but well-known is uninteresting. The scientific curiosity reward demands both.

Formally, let $ℰ(h)=\{e_1,e_2,\dots ,e_M\}$ be a set of experimental results obtained by testing hypothesis $h$. The correctness score is: (Correctness Score)$$\begin{array}{ccc}& \text{Cor}(h,ℰ)=\frac{1}{M}\sum _{j=1}^M\mathrm{𝟙}\left[\text{result}(e_j)\text{ is consistent with }h\right],& \text{<span class="tml-eqn"></span>}\end{array}$$ supplemented by a statistical significance test to guard against chance agreement. A hypothesis is considered empirically verified when $\text{Cor}(h,ℰ)\ge {\tau }_c$ and the agreement is statistically significant at level $\alpha$.

Impact

Impact is the most difficult component to quantify. One approach is to measure the information gain of a hypothesis: how much does knowing $h$ reduce our uncertainty about the world? (Impact INFO)$$\begin{array}{ccc}& \text{Imp}(h,𝒦)=H(𝒦)-H(𝒦|h),& \text{<span class="tml-eqn"></span>}\end{array}$$ where $H(\cdot )$ denotes the entropy of the knowledge base. In practice, this is approximated by the number of downstream questions that $h$ helps answer, the number of existing hypotheses that $h$ connects, or the breadth of the experimental space that $h$ opens up.

Coevolution in Autonomous Research

The most powerful AI scientist will not merely solve pre-defined problems; it will discover new problems worth solving. This requires a coevolutionary architecture in which two populations evolve simultaneously: a population of problems (research questions, conjectures, experimental designs) and a population of solutions (proofs, algorithms, experimental results).

The key insight is that the problems serve as a dynamic curriculum for the solutions, and the solutions serve as a dynamic benchmark for the problems. This mutual adaptation drives both populations toward increasing sophistication, a phenomenon sometimes called an evolutionary arms race.

Current Capabilities and Limitations

It is important to maintain intellectual honesty about what autonomous research agents can and cannot do as of 2025.

What AI Scientists Can Do

• Run experiments. Given a well-defined experimental protocol (e.g., “train model X on dataset Y and report metric Z”), AI scientists can execute thousands of experiments in parallel, far exceeding the throughput of any human lab. The AlphaEvolve system [18] demonstrated this at scale, evolving algorithms for matrix multiplication, chip design, and data centre optimisation.

• Write papers. The AI Scientist [25] produced papers in standard academic format, complete with abstract, introduction, related work, methods, results, and conclusion sections. The papers were syntactically competent and followed disciplinary conventions.

• Discover simple results. In narrow domains with clear fitness landscapes, AI scientists have found genuine improvements over existing methods. AlphaEvolve discovered a new algorithm for $4\times 4$ complex matrix multiplication that uses fewer scalar multiplications than any previously known method.

• Perform systematic literature reviews. LLM-based agents can process thousands of papers, identify trends, and synthesise findings – a task that would take a human researcher weeks or months.

What AI Scientists Cannot Do

• Make deep creative leaps. The discoveries made by AI scientists to date have been incremental: finding slightly better hyperparameters, slightly more efficient algorithms, slightly improved architectures. No AI system has produced the equivalent of Darwin's theory of natural selection, Einstein's general relativity, or Shannon's information theory – insights that require fundamentally reframing how we think about a domain.

• Provide cross-domain insight. The greatest scientific breakthroughs often come from connecting ideas across disparate fields (e.g., applying information theory to genetics, or using topology to study data). Current AI scientists operate within narrow domains and lack the broad knowledge integration needed for cross-domain creativity.

• Assess genuine significance. An AI system can determine whether a result is statistically significant, but it cannot determine whether a result is scientifically significant – whether it changes how we understand the world. This requires a depth of contextual understanding that current systems lack.

• Handle ambiguity and contradiction. Real scientific research involves navigating ambiguous data, contradictory evidence, and incomplete theories. Current AI scientists tend to either ignore contradictions or generate plausible-sounding but ultimately unsatisfying explanations.

The Last Mile Problem

The gap between “incremental improvement” and “revolutionary discovery” is sometimes called the last mile problem. Consider the landscape of possible scientific discoveries as a rugged fitness landscape (The Fitness Landscape Metaphor). Incremental discoveries correspond to hill-climbing on this landscape: small steps that improve fitness without fundamentally changing the trajectory. Revolutionary discoveries correspond to transitions between fitness peaks: large jumps across valleys of low fitness to reach entirely new, higher peaks.

Evolutionary algorithms are notoriously poor at crossing fitness valleys, because natural selection punishes any step that reduces fitness, even if that step is a necessary precursor to reaching a higher peak. This is why coevolution and open-ended search (Open-Ended Evolution and Artificial Life) are essential: they provide mechanisms for maintaining population diversity, exploring multiple peaks simultaneously, and occasionally making the large jumps that lead to revolutionary discoveries.

This lemma formalises the intuition that revolutionary discoveries are exponentially harder than incremental ones. Overcoming this barrier requires mechanisms beyond simple mutation and selection: population diversity, coevolution, novelty search, and perhaps entirely new algorithmic ideas that we have not yet invented.

Active Learning and Evolutionary Exploration

We have seen how evolutionary processes generate candidate solutions and how verification systems filter them. But there is a deeper question: which candidates should the evolutionary process generate in the first place? Random generation is wasteful; most random candidates are either trivially bad or redundant with previously evaluated ones. Intelligent generation, guided by what we have already learned, can dramatically accelerate the search.

This is precisely the problem studied in active learning: the art of choosing which data points to label (or, in our context, which candidates to evaluate) in order to learn as efficiently as possible. The connection between active learning and evolutionary exploration is deep and, we shall argue, constitutes a unifying framework that encompasses several apparently disparate paradigms.

Active Learning: Choosing What to Learn

The central challenge in active learning is designing the acquisition function $\alpha$ that selects informative queries. Three classical strategies are:

Uncertainty Sampling

Select the instance about which the current model is most uncertain: (Uncertainty Sampling)$$\begin{array}{ccc}& {𝒙}^{\ast }={\mathrm{arg\hspace{0.17em}max}}_{𝒙\in 𝒰}H[y|𝒙,{𝒟}_t],& \text{<span class="tml-eqn"></span>}\end{array}$$ where $H[y|𝒙,{𝒟}_t]$ is the predictive entropy of the label $y$ given instance $𝒙$ and the current training data ${𝒟}_t$. For a classifier with class probabilities ${\hat{p}}_k(𝒙)=\mathrm{Pr}[y=k|𝒙,{𝒟}_t]$: (Predictive Entropy)$$\begin{array}{ccc}& H[y|𝒙,{𝒟}_t]=-\sum _{k=1}^K{\hat{p}}_k(𝒙)\log {\hat{p}}_k(𝒙).& \text{<span class="tml-eqn"></span>}\end{array}$$

Query-by-Committee

Maintain a committee of models $\{f_1,f_2,\dots ,f_C\}$ drawn from the version space (the set of models consistent with the labelled data). Select the instance on which the committee disagrees most: (QBC)$$\begin{array}{ccc}& {𝒙}^{\ast }={\mathrm{arg\hspace{0.17em}max}}_{𝒙\in 𝒰}\text{Disagreement}\left(\{f_1(𝒙),f_2(𝒙),\dots ,f_C(𝒙)\}\right),& \text{<span class="tml-eqn"></span>}\end{array}$$ where disagreement can be measured by vote entropy, KL divergence, or other dispersion metrics. The evolutionary connection is immediate: the committee is a population of models, and the disagreement measure is analogous to diversity in a quality-diversity archive.

Expected Model Change

Select the instance that would cause the greatest change in the model's parameters if its label were known: (EMC)$$\begin{array}{ccc}& {𝒙}^{\ast }={\mathrm{arg\hspace{0.17em}max}}_{𝒙\in 𝒰}{𝔼}_{y\sim p(y|𝒙,{𝒟}_t)}\left[\Vert {\nabla }_{𝜽}\ell (f_{𝜽}(𝒙),y)\Vert \right],& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\ell$ is the loss function and the expectation is over the predicted label distribution. This strategy selects instances that are most likely to update the model's understanding of the problem, analogous to evolutionary strategies that prioritise exploration of under-explored regions of the search space.

The algorithm uses the evolutionary population to efficiently search the acquisition function landscape, rather than evaluating $\alpha$ over the entire unlabelled pool. This is particularly valuable in the query-synthesis setting, where the pool is continuous and evaluating $\alpha$ at every point is infeasible.

Bayesian Optimisation as Evolutionary Search

Bayesian optimisation (BO) is the canonical framework for sample-efficient global optimisation of expensive black-box functions. Its connection to evolutionary algorithms is surprisingly deep, and understanding this connection illuminates both frameworks.

The standard BO loop proceeds as follows. We wish to optimise an expensive function $f:𝒳\to ℝ$ (e.g., the fitness function in an evolutionary system, or the validation accuracy of a neural network as a function of its hyperparameters). We maintain a surrogate model of $f$, typically a Gaussian process (GP), trained on the observations collected so far: (GP Posterior)$$\begin{array}{ccc}& f(𝒙)|{𝒟}_t\sim 𝒢𝒫({\mu }_t(𝒙),k_t(𝒙,{𝒙}^{\prime })),& \text{<span class="tml-eqn"></span>}\end{array}$$ where ${\mu }_t(𝒙)$ is the posterior mean, $k_t(𝒙,{𝒙}^{\prime })$ is the posterior covariance, and ${𝒟}_t={\{({𝒙}_i,y_i)\}}_{i=1}^t$ is the dataset of previous evaluations with $y_i=f({𝒙}_i)+{\epsilon }_i$.

The next evaluation point is chosen by maximising an acquisition function $\alpha (𝒙)$ that balances exploration (querying regions of high uncertainty) and exploitation (querying regions of high predicted value): (BO Acquisition)$$\begin{array}{ccc}& {𝒙}_{t+1}={\mathrm{arg\hspace{0.17em}max}}_{𝒙\in 𝒳}\alpha (𝒙;{\mu }_t,k_t).& \text{<span class="tml-eqn"></span>}\end{array}$$

The most widely used acquisition functions are:

• Expected Improvement (EI): (EI)$$\begin{array}{ccc}& {\alpha }_{\text{EI}}(𝒙)=𝔼[\max (f(𝒙)-f^{\ast },0)|{𝒟}_t],& \text{<span class="tml-eqn"></span>}\end{array}$$ where $f^{\ast }={\max }_{i\le t}{y}_i$ is the best observed value.

• Upper Confidence Bound (UCB): (UCB)$$\begin{array}{ccc}& {\alpha }_{\text{UCB}}(𝒙)={\mu }_t(𝒙)+\kappa \cdot {\sigma }_t(𝒙),& \text{<span class="tml-eqn"></span>}\end{array}$$ where ${\sigma }_t(𝒙)=\sqrt{k_t(𝒙,𝒙)}$ is the posterior standard deviation and $\kappa >0$ controls the exploration-exploitation balance.

• Probability of Improvement (PI) : (PI)$$\begin{array}{ccc}& {\alpha }_{\text{PI}}(𝒙)=\mathrm{Pr}[f(𝒙)>f^{\ast }+\xi |{𝒟}_t]=\mathrm{\Phi }\left(\frac{{\mu }_t(𝒙)-f^{\ast }-\xi }{{\sigma }_t(𝒙)}\right),& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\mathrm{\Phi }$ is the standard normal CDF and $\xi \ge 0$ is a margin parameter.

Now observe the structural parallel with evolutionary algorithms. Both maintain a representation of the search space (the GP posterior in BO; the population in EA). Both use a selection criterion (the acquisition function in BO; the fitness function in EA). Both generate new candidates informed by past evaluations (the acquisition-maximising point in BO; offspring from selected parents in EA). The key difference is that BO explicitly models the objective function with a probabilistic surrogate, while standard EAs do not.

The SAEA framework is especially valuable when fitness evaluations are expensive: computational fluid dynamics simulations, materials science experiments, wet-lab biology, or training large neural networks. By using the surrogate to pre-screen candidates, the SAEA reduces the number of expensive evaluations needed while maintaining the global exploration capability of evolutionary search.

Curriculum Learning Meets Evolution

Human students do not learn calculus before arithmetic, quantum mechanics before classical mechanics, or poetry before the alphabet. Learning proceeds from simple to complex, with each lesson building on the foundations laid by previous ones. This observation, formalised as curriculum learning, has proven remarkably effective in machine learning: training on easier examples first often leads to faster convergence and better final performance.

The connection to evolution is twofold. First, evolution itself follows a curriculum: organisms do not evolve to solve the hardest problems in their environment immediately. Simple organisms evolve first, and their descendants gradually tackle more complex challenges as their capabilities grow. Second, the design of the curriculum, the ordering and selection of training examples, is itself an optimisation problem that can be solved by evolutionary methods.

Self-Paced Learning

In self-paced learning, the model determines its own curriculum by selecting, at each training iteration, a subset of examples that it can currently handle. Formally, let $𝒟={\{({𝒙}_i,y_i)\}}_{i=1}^n$ be the training set, and let ${\ell }_i(𝜽)=\ell (f_{𝜽}({𝒙}_i),y_i)$ be the loss on example $i$. Self-paced learning solves: (Selfpaced)$$\begin{array}{ccc}& \underset{𝜽,𝐯}{\min }\sum _{i=1}^n{v}_i{\ell }_i(𝜽)-\lambda \sum _{i=1}^n{v}_i,v_i\in \{0,1\},& \text{<span class="tml-eqn"></span>}\end{array}$$ where $v_i\in \{0,1\}$ indicates whether example $i$ is included in the current training batch and $\lambda >0$ controls the pace. For small $\lambda$, only easy examples (low loss) are included; as $\lambda$ increases, harder examples are gradually introduced.

The optimal selection rule, obtained by fixing $𝜽$ and optimising over $𝐯$, is: (Selfpaced RULE)$$\begin{array}{ccc}& v_i^{\ast }=\{\begin{array}{cc}1& \text{if }{\ell }_i(𝜽)<\lambda ,\\ 0& \text{otherwise.}\end{array}& \text{<span class="tml-eqn"></span>}\end{array}$$ This is precisely a threshold selection rule: examples with loss below $\lambda$ are included; those above are excluded. By gradually increasing $\lambda$ over the course of training, the model transitions from easy examples to hard ones.

Evolutionary Curriculum Design

Rather than using a fixed curriculum schedule, we can evolve the curriculum itself. The idea is to treat the ordering and selection of training examples as a search problem, with the curriculum as the genotype and the final model performance as the fitness.

The computational cost of this approach is high: each fitness evaluation requires training a model from scratch. In practice, surrogate-assisted methods (Bayesian Optimisation as Evolutionary Search) can reduce this cost by predicting the performance of a curriculum without full training.

The Unified View: Exploration Across Paradigms

We are now in a position to see that active learning, Bayesian optimisation, and evolutionary search are not three separate fields but three perspectives on a single problem: how to explore a space efficiently when evaluations are expensive.

Each paradigm answers the same question, “What should I try next?”, using a different formalism:

• Active learning answers: “Try the instance that is most informative for the model.”

• Bayesian optimisation answers: “Try the point that maximises the expected improvement over the current best.”

• Evolutionary search answers: “Try the offspring of the fittest individuals, perturbed by mutation for diversity.”

Despite the different vocabularies, the underlying structure is identical. Each paradigm maintains a model of the search space (a classifier in active learning, a GP in BO, a population in EA). Each uses this model to guide the selection of the next evaluation point. And each must balance exploration (evaluating in uncertain regions) against exploitation (evaluating near known good solutions).

Concept	Active Learning	Bayesian Opt.	Evolutionary
Search space model	Classifier / regressor	Gaussian process	Population
Candidate proposal	Pool selection / synthesis	Acquisition max.	Offspring generation
Exploration signal	Uncertainty / entropy	Posterior variance	Mutation / diversity
Exploitation signal	Expected performance	Posterior mean	Fitness selection
Balance mechanism	Acquisition function	$\kappa$ in UCB	Selection pressure
Information storage	Labelled dataset	GP posterior	Archive / hall of fame
Stopping criterion	Label budget $B$	Evaluation budget	Generation limit $T$

The unification suggests a hybrid algorithm that draws on the strengths of all three paradigms.

The unified acquisition function in (Unified Acquisition) has three terms. The first (${\mu }_S$) drives exploitation: querying candidates predicted to be good. The second ($\kappa {\sigma }_S$) drives exploration: querying candidates about which the surrogate is uncertain. The third ($\eta \cdot d$) drives diversity: querying candidates that are different from what has been seen before. By adjusting $\kappa$ and $\eta$, we can recover standard Bayesian optimisation ($\eta =0$, $\kappa >0$), pure novelty search ($\kappa =0$, $\eta >0$, ${\mu }_S=0$), or standard evolutionary search ($\kappa =0$, $\eta =0$, with ${\mu }_S$ being the fitness function).

From Theory to Practice

The unified framework suggests practical algorithms. Consider optimising the architecture and hyperparameters of a deep neural network. The search space $𝒳$ is a mixed continuous-discrete space of layer counts, widths, learning rates, and regularisation coefficients. Evaluating a single configuration requires hours of training.

A unified approach would:

1. Maintain a population of $N=50$ architectures (evolutionary component).

2. Fit a surrogate model (e.g., a random forest or neural network) to predict validation accuracy from architecture descriptors (Bayesian optimisation component).

3. Use an acquisition function combining predicted accuracy, prediction uncertainty, and distance from previously evaluated architectures (unified acquisition, (Unified Acquisition)).

4. Evolve the population using crossover and mutation, but select the offspring to evaluate based on the acquisition function rather than random selection (active learning component).

5. Fully train and evaluate only the top $k=5$ offspring per generation, updating the surrogate with the results.

This hybrid approach leverages the global search capability of evolutionary algorithms, the sample efficiency of Bayesian optimisation, and the information-theoretic guidance of active learning.

Synthesis: The Evolutionary Exploration Engine

Let us step back and appreciate the architecture that emerges from combining the ideas of this section. An evolutionary exploration engine is a system that:

1. Proposes candidate solutions using evolutionary operators (crossover, mutation) applied to a population of diverse individuals. The population maintains coverage of the search space, preventing premature convergence to a single region.

2. Predicts the value of candidates using a surrogate model trained on previously evaluated individuals. The surrogate provides both point estimates (for exploitation) and uncertainty estimates (for exploration), enabling intelligent selection of which candidates to evaluate.

3. Selects the most informative candidates for expensive evaluation, using an acquisition function that balances exploitation (evaluating candidates predicted to be good), exploration (evaluating candidates about which the surrogate is uncertain), and diversity (evaluating candidates that are different from what has been seen).

4. Verifies evaluated candidates using the multi-level verification pyramid of The Multi-Level Verification Pyramid, filtering out incorrect solutions at the cheapest possible level.

5. Learns from each evaluation, updating both the surrogate model and the evolutionary population to reflect the new information.

This architecture is not merely theoretical. It is the implicit structure behind many of the most successful AI systems of recent years: AlphaFold's iterative structure prediction, AlphaProof's proof search, AlphaEvolve's algorithm discovery, and the AI Scientist's research loop. In each case, the system generates candidates, predicts their value, selects the most promising for evaluation, verifies correctness, and learns from the results.

Open-Ended Evolution and Artificial Life

We arrive now at the most audacious question that evolutionary AI can pose. Not whether evolution can optimise a given objective function, as that much has been demonstrated beyond reasonable doubt. Not whether evolution can discover novel solutions that surprise their creators, for the preceding sections have supplied abundant evidence. The question is deeper, older, and more unsettling: can an artificial system evolve without end?

Biological evolution has been running for approximately four billion years. In that time it has produced prokaryotes, eukaryotes, multicellular organisms, nervous systems, language, mathematics, and, most recently, the very minds now contemplating whether the process can be replicated in silicon. At no point has biological evolution shown signs of exhaustion. Each major transition – from RNA to DNA, from unicellular to multicellular, from individual to social – has opened new vistas of possibility rather than closing them. The tree of life is not converging to a fixed point; it is branching, accelerating, and producing novelty at a rate that, if anything, increases with time.

No artificial system has achieved anything remotely comparable. The most impressive evolutionary computation experiments run for thousands, perhaps millions of generations, then plateau. Populations converge, diversity collapses, and the system settles into a steady state from which no further novelty emerges. The contrast with biology is stark and demands explanation.

This section confronts the open-endedness question head-on. We begin by defining what open-ended evolution means in formal terms (What Is Open-Ended Evolution?), survey the foundational work in artificial life (Artificial Life Foundations), analyse why artificial systems hit a complexity barrier (The Complexity Barrier), explore whether LLM-evolutionary systems might break through it (Towards Open-Ended LLM Evolution), and conclude with the No Free Lunch theorem and its surprisingly nuanced implications for evolutionary search (No Free Lunch and Its Implications).

What Is Open-Ended Evolution?

The phrase “open-ended evolution” is invoked frequently in the artificial life and evolutionary computation literature, yet pinning down precisely what it means has proven surprisingly difficult. At its most intuitive, the concept captures the sense in which biological evolution seems to have no ceiling – no maximum fitness, no final species, no ultimate design. But intuition is not definition, and without formal criteria we cannot hope to build systems that achieve the property or even to recognise it if they do.

Several formalisation attempts have appeared in the literature. [56] distinguish between weak open-endedness, in which the system continually produces novel phenotypes, and strong open-endedness, in which the novelty is not merely syntactic but involves genuine increases in complexity, functionality, or adaptedness. [57] further refine the concept by requiring that the system produce interesting novelty, not just random variation, but innovations that matter to the evolutionary dynamics.

We synthesise these perspectives into a formal definition.

Condition 1 rules out convergence to a fixed population. Condition 2 rules out the trivial case where novelty consists only of minor parameter variations within a fixed complexity class. Condition 3 rules out “random walks through junk,” i.e., systems that produce ever more complex but ecologically meaningless structures. Together, the three conditions capture the intuition that open-ended evolution produces a never-ending stream of genuinely interesting innovations.

Artificial Life Foundations

The dream of creating life in silico predates the electronic computer itself. John von Neumann, while designing the architecture that would bear his name, was simultaneously investigating the theoretical conditions under which a machine could reproduce itself. His self-replicating automaton, embedded in a cellular automaton grid, demonstrated that self-reproduction was not a mystical property of biological matter but a logical consequence of sufficiently complex computation. The intellectual lineage from von Neumann's thought experiments to modern artificial life research is direct and unbroken.

Tierra: Digital Evolution in a Virtual Ecosystem

The most celebrated early experiment in digital evolution was Tierra, created by ecologist Thomas Ray in 1991. Tierra was not an evolutionary algorithm designed to optimise a fitness function; it was a virtual ecosystem in which self-replicating computer programs competed for memory and CPU time, the computational equivalents of space and energy.

The system worked as follows. A block of memory (the “soup”) was initialised with a single self-replicating program, the “ancestor,” written in a custom assembly language. The ancestor was 80 instructions long and could copy itself into an adjacent block of memory. A “reaper” mechanism killed the oldest or most error-prone programs when memory filled up, creating selection pressure. Crucially, the copy instruction was imperfect: mutations occurred at a low rate, introducing variation.

What Ray observed astounded the artificial life community. Within a few thousand generations, the following evolutionary dynamics emerged spontaneously:

1. Parasites. Shorter programs (45 instructions) evolved that lacked their own copy code but hijacked the copy routine of neighbouring programs. These were genuine parasites in the ecological sense.

2. Immune hosts. The host programs evolved defences against the parasites, developing modified copy routines that the parasites could not exploit.

3. Hyper-parasites. Programs evolved that could redirect parasites to copy the hyper-parasite instead of themselves, turning the tables on the exploiters.

4. Social organisms. In some runs, cooperative communities of programs emerged that shared copy routines, achieving efficiencies impossible for solitary replicators.

The ecological richness was remarkable. Tierra demonstrated that complex ecological dynamics – parasitism, arms races, mutualism – could emerge from nothing more than imperfect self-replication in a resource-limited environment. No fitness function was specified; the only “objective” was to replicate before being killed by the reaper.

Avida: Evolved Digital Organisms

Avida, developed by Christoph Adami, Charles Ofria, and colleagues, extended Tierra's framework with a crucial addition: a metabolic system. In Avida, digital organisms not only compete for memory and CPU time but also gain resources by performing computational tasks. An organism that can compute, say, the logical OR of two input numbers receives a reward in the form of faster execution speed. More complex logical operations (AND, XOR, EQU) yield greater rewards.

This seemingly simple addition had profound consequences. With metabolic rewards tied to computational complexity, Avida populations evolved organisms capable of performing complex multi-step computations. The landmark result by Lenski, Ofria, Pennock, and Adami (2003) demonstrated that complex computational functions evolved through the accumulation of simpler ones, a direct computational analog of Darwin's “numerous, successive, slight modifications” [30].

POET: Paired Open-Ended Trailblazer

A decisive step toward open-endedness came with POET (Paired Open-Ended Trailblazer), introduced by Wang et al. in 2019. POET addresses one of the central obstacles to open-ended evolution: the fixed environment. In Tierra and Avida, the computational environment – the instruction set, the resource model, the topology of the memory soup – is fixed at the outset. No matter how clever the organisms become, the environment remains the same, and eventually the population exhausts the available niches.

POET escapes this trap by co-evolving environments and agents simultaneously. The system maintains a population of (environment, agent) pairs. At each generation:

1. Environment mutation. New environments are created by mutating existing ones, e.g., adding obstacles, changing terrain, altering physics parameters.

2. Agent optimisation. Each agent is optimised (via evolutionary strategies or reinforcement learning) in its paired environment.

3. Transfer and selection. Agents are tested in other environments. If an agent performs well in an environment that is neither too easy nor too hard (the “Goldilocks zone”), the pair is retained.

The key insight is the transfer step. By allowing agents to migrate between environments, POET creates evolutionary stepping stones: an agent evolved in environment $A$ may, when transferred to a harder environment $B$, outperform agents that were directly evolved in $B$. This is because the skills learned in $A$ provide a foundation that local optimisation in $B$ alone could not discover.

The Complexity Barrier

We have seen that systems like Tierra, Avida, and POET exhibit fascinating evolutionary dynamics in the short term. Yet all of them eventually plateau. Tierra's ecological richness stabilises after a few thousand generations. Avida's organisms evolve impressively complex computations but reach a ceiling. POET generates increasingly complex environments and agents, but the rate of novelty production eventually slows.

Why does biological evolution avoid this fate while artificial systems succumb to it? Three hypotheses have been proposed, and the truth likely involves all three.

Hypothesis 1: Insufficient Environmental Complexity

The simplest explanation is that artificial environments are not complex enough. A Tierra soup has a fixed instruction set, a fixed memory model, and a fixed reaper algorithm. Compare this with the biological environment: a planet with oceans, continents, an atmosphere, weather patterns, geological processes, and, crucially, other organisms. The physical environment alone provides an effectively infinite space of ecological niches. When other organisms are included, the space becomes even richer, because each new species creates new niches for others.

The argument is quantitative. Let $|{𝒩}_{\mathrm{env}}|$ denote the number of distinguishable ecological niches in the environment. In Tierra, $|{𝒩}_{\mathrm{env}}|$ is finite and relatively small, perhaps a few hundred distinct roles (self-replicator, parasite, hyper-parasite, cooperator, etc.). In the biosphere, $|{𝒩}_{\mathrm{env}}|$ is, for all practical purposes, infinite: every combination of spatial location, resource type, temporal niche, and interaction partner defines a potential niche.

Hypothesis 2: Fixed Representation Constrains Evolution

A more subtle explanation focuses on the genotype-phenotype mapping. In biological evolution, the genetic code itself has evolved: regulatory networks, epigenetic mechanisms, horizontal gene transfer, gene duplication, whole genome duplication, and the evolution of evolvability all modify the space of possible phenotypes. In artificial systems, the genotype representation – the instruction set in Tierra, the neural network architecture class in neuroevolution, the grammar in genetic programming – is fixed at design time.

When the representation is fixed, the set of reachable phenotypes is bounded. No matter how long the system evolves, it can only explore variations within the representational capacity specified by the designer. This is analogous to trying to write all of English literature using only a 100-word vocabulary: no matter how cleverly the words are combined, the expressive capacity is fundamentally limited.

Hypothesis 3: The Need for Genuine Niche Construction

The deepest explanation invokes niche construction, the process by which organisms modify their own environment, creating new selection pressures that drive further evolution.

Biological examples of niche construction are ubiquitous. Beavers build dams that create ponds, transforming terrestrial ecosystems into aquatic ones. Earthworms modify soil chemistry, creating conditions that favour certain plants over others. The evolution of photosynthesis oxygenated the atmosphere, creating the conditions for aerobic metabolism, a niche that did not previously exist. Most dramatically, the evolution of multicellularity created the niche of “being a cell within a multicellular organism,” an ecological role with no analog in the pre-multicellular world.

The proposition tells us that niche construction provides a mechanism for escaping the complexity barrier: by creating new fitness dimensions, organisms open up new evolutionary possibilities that demand greater complexity. The process is self-reinforcing: more complex organisms construct richer niches, which in turn select for even greater complexity.

Towards Open-Ended LLM Evolution

The LLM-evolutionary systems surveyed in AlphaEvolve: Open-Ended Algorithmic Discovery through CALM: Co-evolution of Algorithms and Language Models introduce a qualitatively new possibility for open-ended evolution. Unlike Tierra's fixed instruction set or Avida's constrained computational substrate, large language models encode an enormously rich space of programs, algorithms, and designs in their weight matrices. An LLM does not merely search over a pre-specified grammar; it searches over the space of all expressible ideas in natural and formal language.

This representational richness addresses Hypothesis 2 directly: the genotype space is no longer fixed at design time but encompasses the full expressive power of language. Whether this is sufficient for open-endedness remains an open question, but the potential is tantalising.

Consider the AI Scientist framework (The AI Scientist: Autonomous Research Agents). At each cycle, it formulates research questions, designs experiments, generates code, runs evaluations, and writes up results. In principle, the research questions themselves evolve: each round's findings suggest new questions, which lead to new experiments, which reveal new phenomena. This is precisely the niche-construction dynamic described in Hypothesis 3: The Need for Genuine Niche Construction: the “organisms” (research projects) modify the “environment” (the state of scientific knowledge) in ways that create new “niches” (research opportunities).

But does this constitute genuine open-endedness in the sense of Definition 65? Let us evaluate against the three criteria. Persistent novelty: plausibly yes, if the research landscape is sufficiently rich. Unbounded complexity: uncertain, as the LLM's capacity, while vast, is finite, and the types of research it can conduct are limited by its training distribution. Ecological relevance: partially, in that new findings alter the landscape of what is interesting, but the feedback loop is weak compared to biological niche construction, where organisms physically transform their environment.

The gap between “nearly open-ended” and “genuinely open-ended” may be the most important gap in all of AI research. Closing it would require not just better models but a fundamentally new relationship between the evolving system and its environment, one in which the system can expand the very space of possibilities rather than merely exploring a fixed space more efficiently.

No Free Lunch and Its Implications

Any discussion of evolutionary search must eventually confront the No Free Lunch (NFL) theorem of [58], one of the most frequently cited and most frequently misunderstood results in optimisation theory. The theorem has been used both to dismiss evolutionary algorithms (“they're no better than random search!”) and to defend them (“all algorithms are equal, so use whichever you like!”). Both interpretations are wrong, and understanding why they are wrong illuminates deep truths about the nature of search.

The theorem is mathematically impeccable but practically misleading. The key phrase is “averaged over all possible objective functions.” The set $ℱ$ of all functions from $𝒳$ to $\mathrm{ℝ}$ includes functions that are random, adversarial, pathological, and bear no resemblance to any problem that anyone would ever want to solve. The NFL theorem tells us that no algorithm is universally best across this vast space. But it says nothing about whether some algorithms are better than others on the tiny subset of $ℱ$ that corresponds to real-world problems.

And they are. Real-world problems have structure: smoothness, modularity, symmetry, locality, hierarchy, compositionality. Evolution excels precisely because it can exploit such structure through mechanisms like building blocks, modularity, and hierarchy [1].

Consolidation with Prior Chapters

Throughout this textbook we have traversed a vast landscape of generative models: variational autoencoders that learn latent representations, generative adversarial networks that engage in minimax competition, diffusion models that reverse the process of noise, normalising flows that warp simple distributions into complex ones, autoregressive models that predict the future one token at a time, and large language models that distill the statistical patterns of human knowledge into billions of parameters. Each chapter has developed its subject in considerable mathematical depth, and the reader might be forgiven for viewing these as largely independent topics connected only by the broad umbrella of “generative AI.”

This section argues that they are, in fact, deeply connected, and that the evolutionary perspective provides the most illuminating framework for understanding those connections. Evolution is not merely one more technique in the generative AI toolkit. It is the meta-technique, the process by which all other techniques are discovered, refined, and combined.

Evolution Meets Continual Learning

The problem of continual learning, that of training a neural network on a sequence of tasks without forgetting earlier ones, has been a central concern since at least the 1990s (see 26 for an in-depth treatment). The pathology known as catastrophic forgetting, in which learning a new task overwrites the weights responsible for earlier tasks, has spawned an enormous literature of solutions: elastic weight consolidation, progressive networks, PackNet, memory replay, and many others.

From the evolutionary perspective, catastrophic forgetting reveals a fundamental tension between adaptation and conservation. A single neural network trained by gradient descent is like a single organism trying to simultaneously be a fish, a bird, and a mammal. Each adaptation to one niche (task) erases the adaptations to previous niches. Evolution solved this problem billions of years ago through a mechanism so obvious it is easy to overlook: speciation. Instead of one organism adapting to all niches, evolution produces a population of specialists, each adapted to its own niche.

Progressive networks, introduced by Rusu et al. (2016), implement precisely this strategy. When a new task arrives, a new column of neurons is added to the network, with lateral connections to all previous columns but no modification of previous weights. This is evolutionary branching in disguise: the old “species” (network columns for earlier tasks) are preserved intact, while a new “species” (the new column) inherits from all of them through lateral connections (analogous to horizontal gene transfer).

PackNet (Mallya and Lazebnik, 2018) takes a complementary approach. After training on each task, it prunes the network by removing weights that are least important for the current task. The freed capacity is then available for subsequent tasks. This is evolutionary pruning: just as natural selection removes traits that are no longer needed in a particular environment, PackNet removes weights that are no longer needed for a particular task, freeing resources for future adaptation.

The proposition reveals that population-based approaches provide exponential protection against catastrophic forgetting as the population size grows relative to the number of known tasks. This is precisely the mechanism by which biological evolution preserves adaptations: not within a single organism, but across a population of specialists.

Evolution Meets Generative Models

The connection between evolutionary dynamics and generative adversarial networks is not merely metaphorical; it is mathematical. A GAN consists of a generator $G_{𝜽}$ and a discriminator $D_{𝝓}$ engaged in a minimax game: (GAN Minimax)$$\begin{array}{ccc}& \underset{𝜽}{\min }\underset{𝝓}{\max }{𝔼}_{𝒙\sim p_{\mathrm{data}}}[\log D_{𝝓}(𝒙)]+{𝔼}_{𝒛\sim p(𝒛)}[\log (1-D_{𝝓}(G_{𝜽}(𝒛)))].& \text{<span class="tml-eqn"></span>}\end{array}$$ This is precisely the structure of competitive coevolution (Coevolutionary Algorithms): two populations (generator parameters and discriminator parameters) co-adapting, each trying to outperform the other.

This coevolutionary perspective immediately explains the well-known pathologies of GAN training:

The connection extends naturally to architecture search and diffusion models. VAEs combined with evolutionary architecture search evolve both the latent representation and the network architecture simultaneously. Diffusion models, whose denoising networks can be discovered via NAS (see DiffusionNAG in 30), benefit from evolutionary search over the noise schedule, the network architecture, and the conditioning mechanisms.

Evolution Meets Privacy and Security

The adversarial dynamics between attackers and defenders in machine learning security bear a striking resemblance to predator-prey coevolution. This is not coincidence; both are instances of competitive coevolution in which one party seeks to exploit a weakness that the other party seeks to conceal.

Consider membership inference attacks (29). An attacker trains a model to determine whether a specific data point was used to train a target model. The defender seeks to train the model in a way that prevents such inference. This is a minimax game, precisely the coevolutionary structure of Definition 69, with the attacker playing the role of discriminator and the defender playing the role of generator.

An important practical application of evolutionary methods in this domain is the optimisation of privacy-utility tradeoffs. Differential privacy introduces noise to protect individual data points, but the noise degrades model utility. The privacy parameter $ϵ$ controls this tradeoff, and choosing it is a multi-objective optimisation problem: minimise privacy risk while maximising model accuracy.

Evolution Meets Neural Architecture Search

Neural architecture search is, in many cases, literally evolutionary search applied to the space of network architectures. The relationship is so close that a historical lineage can be traced from the earliest neuroevolution work through modern NAS methods (see 30 for a comprehensive treatment).

The lineage begins with NEAT (Stanley and Miikkulainen, 2002), which evolved both the topology and weights of neural networks through speciation and complexification. NASNet (Zoph et al., 2018) applied reinforcement learning to search over a cell-based architecture space, but the cell-based design pattern itself, that of repeating modular units, is borrowed directly from biological evolution's use of repeated structural modules (segments in arthropods, vertebrae in chordates). ENAS (Pham et al., 2018) introduced weight sharing to amortise the cost of architecture evaluation, analogous to the biological principle that related species share much of their genetic material. DARTS (Liu et al., 2019) made the architecture search differentiable, replacing the discrete evolutionary search with continuous relaxation, a move from evolution to gradient-based adaptation. AlphaEvolve (AlphaEvolve: Open-Ended Algorithmic Discovery) returned to explicit evolutionary search but with an LLM providing the variation operator, combining the best of both worlds: evolutionary exploration with the rich prior knowledge encoded in the language model [18].

NAS Method	Key Mechanism	Evolutionary Analog
NEAT	Complexification	Genome duplication
NASNet	Modular cells	Body plan modules
ENAS	Weight sharing	Shared ancestry
DARTS	Continuous relaxation	Quantitative genetics
AmoebaNet	Tournament selection	Natural selection
EfficientNet	Compound scaling	Allometric growth
AlphaEvolve	LLM-guided mutation	Lamarckian inheritance

A Unified View: The Co-Evolutionary Ecosystem of AI

We are now in a position to step back and see the forest for the trees. The various subfields of AI that this textbook has covered in separate chapters – continual learning, generative models, privacy, architecture search, reinforcement learning, meta-learning, multi-agent systems – are not independent developments. They are manifestations of a single underlying process: the search for adaptive solutions in high-dimensional spaces under resource constraints. This is, of course, the definition of evolution.

Safety, Alignment, and the Limits of Autonomous Evolution

We come now to the most consequential section of this chapter, perhaps of this entire textbook. The preceding sections have developed the theory and practice of systems that evolve autonomously: discovering algorithms (AlphaEvolve: Open-Ended Algorithmic Discovery), conducting research (The AI Scientist: Autonomous Research Agents), optimising their own architecture (Evolution Meets Neural Architecture Search), and potentially evolving without bound (Open-Ended Evolution and Artificial Life). These are extraordinary capabilities. They are also, if improperly governed, extraordinarily dangerous.

The danger is not speculative. As of 2025, LLM-evolutionary systems can already generate novel code, discover mathematical results, and design experiments without human oversight. The gap between “can do” and “should do” is precisely the territory of alignment, the problem of ensuring that AI systems pursue goals that are compatible with human values and intentions.

This section examines the alignment problem through the evolutionary lens. We argue that self-evolving systems pose alignment challenges that go beyond those of static models, because the system's objectives, capabilities, and behaviour all change over time in ways that may be difficult to predict or control. We analyse the possibility and limits of recursive self-improvement, discuss containment strategies, explore the hypothesis that coevolution with human feedback might achieve alignment, and conclude with a measured assessment of existential considerations.

The Alignment Problem for Evolutionary Systems

In a standard machine learning pipeline, a model is trained on a fixed dataset with a fixed loss function, and the resulting model is then deployed. Alignment for such a system means ensuring that the loss function accurately captures human intent and that the model generalises appropriately. This is already difficult, but at least the system is static once deployed: it does not change its own weights, architecture, or objectives.

Evolutionary systems break this assumption fundamentally. An LLM-evolutionary loop that generates and evaluates code is dynamic: it modifies its own search strategy, discovers new heuristics, and may alter its effective objectives through fitness landscape deformation (niche construction). Goodhart's Law – “when a measure becomes a target, it ceases to be a good measure” – is particularly insidious in the evolutionary context, because evolution is an optimiser that relentlessly exploits every feature of the fitness function, including features that the designer did not intend to be relevant.

The supremum over $t$ is the key distinction from the static case. A fitness function that accurately reflects human utility at generation 0 may diverge dramatically at generation $\mathrm{10,000}$, as the population exploits subtle discrepancies between $ℱ$ and $𝒰$. The evolutionary process acts as an adversarial search over the space of discrepancies, systematically discovering and amplifying any gap between what we measure and what we actually want.

Recursive Self-Improvement

The concept of recursive self-improvement, a system that improves its own ability to improve itself, has haunted the AI safety discourse since I. J. Good's prescient 1965 paper [23]. Good wrote:

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind.

The evolutionary systems described in this chapter bring Good's thought experiment closer to reality than ever before. An LLM-evolutionary system that discovers better algorithms for training LLMs is, in a precise sense, improving its own substrate. If those improved algorithms are then used to train a better LLM, which in turn discovers even better algorithms, the loop is closed: we have recursive self-improvement.

Condition 4 is the most demanding and the most debated. It distinguishes genuine recursive self-improvement from the commonplace phenomenon of diminishing returns. Most real-world optimisation processes eventually slow down: the first improvements are easy, and each subsequent improvement is harder. Is genuine compounding improvement possible, or do fundamental limits prevent it?

Containment and Control

Given that evolutionary systems can evolve in unpredictable directions, how can we maintain meaningful control? The fundamental challenge is that evolution is an open-ended optimiser: it will exploit any feature of its environment, including features of the containment mechanism itself, if doing so increases fitness. Any containment strategy must therefore be robust not just against the current capabilities of the system but against capabilities the system might evolve.

We identify four containment approaches, each with characteristic strengths and limitations.

Mechanism	Description	Limitations
Search space restriction	Constrain genotypes to $ℬ\subset 𝒢$	Boundary must be specified a priori; may exclude beneficial solutions; boundary itself may be exploitable
Safety verification	Filter offspring through ${𝒱}_c$ before inclusion	Verification is necessarily incomplete (Verification Systems for Evolutionary Discovery); may reject novel-but-safe solutions
Resource budgeting	Hard limits on compute, memory, network access	Effective but coarse; prevents beneficial as well as harmful evolution
Human oversight	Periodic review of evolved solutions	Does not scale; human evaluators can be overwhelmed or manipulated by plausible-looking solutions
Constitutional fitness	Encode safety constraints as hard fitness penalties	Subject to Goodhart's Law; constraints may conflict with utility
Tripwire mechanisms	Halt if specific behaviours are detected	Only catches anticipated threats; novel threats may evade detection

The concept of constitutional AI, encoding values and constraints directly into the system's training objective, translates naturally to the evolutionary setting. Instead of relying solely on external containment, we can embed safety constraints into the fitness function itself: (Constitutional Fitness)$$\begin{array}{ccc}& {ℱ}_{\text{safe}}(𝒙)={ℱ}_{\text{utility}}(𝒙)-\lambda \cdot {ℱ}_{\text{violation}}(𝒙),& \text{<span class="tml-eqn"></span>}\end{array}$$ where ${ℱ}_{\text{violation}}(𝒙)\ge 0$ measures the degree to which phenotype $𝒙$ violates safety constraints, and $\lambda >0$ is a penalty coefficient. The choice of $\lambda$ is itself a safety-critical decision: too small, and violations are tolerated; too large, and the system is overly conservative.

The Value Alignment via Coevolution Hypothesis

Can the coevolutionary framework, the same framework that produces arms races, Red Queen dynamics, and all the pathologies catalogued in Coevolutionary Algorithms, also be harnessed to achieve alignment? This question is not as paradoxical as it may seem.

Reinforcement learning from human feedback (RLHF), the dominant approach to aligning large language models as of 2025, is already a form of coevolution. The language model (species A) generates responses, and human evaluators (species B) provide feedback that shapes the reward model. The reward model then exerts selection pressure on the language model, which adapts its responses, which elicit new feedback, and so on. This is textbook coevolution: two interacting populations co-adapting over time.

The conjecture is deliberately stated as a conjecture, not a theorem, because each of its four conditions is extremely difficult to guarantee in practice.

Argument for.

RLHF already demonstrates that human-AI coevolution can produce well-behaved systems. The key mechanism is that human feedback provides a continuously updated fitness signal that tracks changing human expectations. Unlike a fixed fitness function, which is vulnerable to Goodhart's Law, human feedback is adaptive: if the system starts exploiting a loophole, human evaluators can notice and penalise the exploit. This adaptive feedback creates a coevolutionary arms race in which the “arms” are not weapons but values: the system's values co-adapt with human values.

Argument against.

The argument fails if any of the four conditions is violated. Condition 1 (honest signalling) is particularly fragile: as AI systems become more capable, they may learn to produce outputs that satisfy human evaluators without genuinely reflecting the underlying values, a form of “sycophancy” or “reward hacking” that has already been observed in RLHF-trained models. Condition 3 (diversity of values) raises deep philosophical questions about whose values should be represented and how to aggregate conflicting values. Condition 4 (power symmetry) becomes increasingly difficult to maintain as AI capabilities grow.

Existential Considerations

We close this section, and the substantive content of this chapter, with a meditation on the deepest questions that evolutionary AI raises. These are questions that no theorem can settle, no experiment can resolve, and no amount of engineering can circumvent. They are, in the truest sense, philosophical questions, and they deserve to be treated with the seriousness that philosophy at its best commands.

The Comprehensibility Horizon

When evolution operates on simple systems, such as genetic algorithms optimising functions in ${\mathrm{ℝ}}^{10}$, the evolved solutions can typically be understood by their designers. A vector of ten real numbers can be inspected, plotted, and interpreted. But as evolved systems grow more complex, a threshold is crossed beyond which human understanding fails. We call this threshold the comprehensibility horizon.

The comprehensibility horizon is already being approached in several domains. Evolved neural architectures discovered by NAS are notoriously difficult to interpret: the irregular connectivity patterns bear no resemblance to human-designed architectures and resist attempts at structural analysis. Programs evolved by genetic programming are often “write-only code,” functionally correct but incomprehensible. Mathematical results discovered by LLM-evolutionary systems may be formally verifiable but provide no human-accessible insight into why they are true.

Fundamental Limits on What Evolution Can Produce

Are there fundamental limits to what evolution, biological or artificial, can produce? The answer is unambiguously yes, and understanding these limits provides a sobering counterweight to both utopian and dystopian narratives.

1. Computability. Evolved systems are computational systems, and computational systems are bounded by the Church-Turing thesis. No evolved program can solve the halting problem, no evolved oracle can decide the truth of arbitrary mathematical statements, and no evolved system can predict the behaviour of a more complex system than itself (in the general case).

2. Thermodynamics. As established in Proposition 32, computation requires energy, and energy is finite. The maximum computational capability of any physical system is bounded by the Landauer limit. Evolution cannot repeal the second law.

3. Complexity. Even within the realm of computable functions, many problems require exponential or super-exponential resources. Evolution can discover clever heuristics but cannot turn an NP-hard problem into a polynomial-time problem (unless P = NP, which most computer scientists consider unlikely).

4. Knowability. Gödel's incompleteness theorems establish that any sufficiently powerful formal system contains true statements that cannot be proved within that system. An evolved formal-reasoning system is still a formal system and inherits these limitations.

5. Physical law. Evolution operates within the physical universe and cannot produce systems that violate physical laws. No evolved system will achieve faster-than-light communication, perpetual motion, or violations of quantum-mechanical uncertainty.

These limits are reassuring in the sense that they establish an absolute ceiling on what evolved systems can do. But they are also frustratingly loose: the gap between current AI capabilities and these theoretical limits is so enormous that the limits provide almost no practical constraint on near-term development. The room for evolution to operate within these bounds is, for all practical purposes, vast.

A Measured Perspective

The discourse around advanced AI systems has a regrettable tendency toward extremes. Utopians envision a world in which self-improving AI solves all of humanity's problems – disease, poverty, climate change, death itself. Dystopians envision a world in which superintelligent AI renders humanity obsolete or extinct. Both visions share a common flaw: they treat AI development as a force of nature, something that happens to humanity rather than something that humanity does.

The evolutionary perspective offers a more nuanced view. Humanity has been in the business of guiding evolution for at least 10,000 years. The domestication of plants and animals – wheat, rice, dogs, cattle, horses – involved precisely the kind of directed evolution that we now contemplate for AI systems. In each case, humans shaped the evolutionary trajectory of another species to serve human purposes. The results were not always what was intended (feral populations, unintended ecological consequences, loss of genetic diversity), but on the whole, domestication has been one of humanity's most successful technological enterprises.

The analogy is imperfect (domesticated animals do not recursively self-improve), but it captures an essential truth: guided evolution is not a contradiction in terms. It is something humanity has considerable experience with, and that experience provides useful lessons for the governance of evolutionary AI.

Open Problems and Future Directions

The preceding sections have traced a remarkable arc: from the classical genetic algorithm with its fixed fitness function to autonomous systems that co-evolve solutions, verifiers, and even the objectives themselves. Yet for all the progress surveyed in this chapter, the most profound questions remain unanswered. In this section we catalogue eight open problems that, in the authors' view, represent the frontier of evolutionary AI research. Each problem is accompanied by a formal conjecture or research direction that we hope will inspire the next generation of investigators.

The problems are ordered roughly from the most foundational (“What does open-endedness even mean?”) to the most applied (“Can evolution produce genuine science?”). They are not independent: progress on any one is likely to illuminate the others, and a breakthrough on the open-endedness criterion (Open Problem 1: The Open-Endedness Criterion) would reverberate through virtually every other problem on this list.

Open Problem 1: The Open-Endedness Criterion

The concept of open-ended evolution has been invoked throughout this chapter, yet no universally accepted formal definition exists. Biological evolution on Earth has produced an ever-expanding diversity of organisms, ecological strategies, and body plans over billions of years, with no sign of convergence to a fixed point. Can we capture this property mathematically?

Several candidate definitions have been proposed. One approach demands that the number of “meaningfully distinct” phenotypes grows without bound. Another requires that the complexity of the most complex organism increases monotonically. A third, inspired by algorithmic information theory, asks that the Kolmogorov complexity of the population's behavioural repertoire diverges.

Let us attempt a formal synthesis. Denote by ${𝒫}_t$ the population at generation $t$, and let $\varphi :{𝒫}_t\to ℬ$ be a mapping from individuals to their behavioural descriptions in some space $ℬ$. Define the behavioural archive at time $t$ as (Archive)$$\begin{array}{ccc}& {𝒜}_t=\bigcup _{\tau =0}^t\{\varphi (p):p\in {𝒫}_{\tau }\},& \text{<span class="tml-eqn"></span>}\end{array}$$ and let $K(\cdot )$ denote the Kolmogorov complexity function.

The difficulty is not merely technical. Any definition of open-endedness must navigate a philosophical minefield: what counts as “meaningfully distinct”? Is a population that endlessly produces trivial variations of the same strategy genuinely open-ended? The ecological non-collapse condition in Conjecture 5 attempts to exclude such degenerate cases, but whether it succeeds remains to be seen.

This proposition illustrates why all three conditions in the conjecture are needed: novelty alone can be achieved by any sufficiently random process, which is clearly not what we mean by open-ended evolution.

Open Problem 2: Verification at Scale

As evolved solutions grow more complex, from single functions to multi-file programs, from circuit designs to entire algorithmic frameworks, the problem of verifying their correctness becomes the dominant bottleneck. In FunSearch (AlphaEvolve: Open-Ended Algorithmic Discovery), solutions are small enough that unit tests and mathematical invariants suffice. In AlphaEvolve (AlphaEvolve: Open-Ended Algorithmic Discovery), the evaluator runs the evolved code on benchmark instances and checks numerical outputs. But what happens when the evolved artefact is a 10,000-line software system, or a novel mathematical proof that no human has verified?

The verification problem is fundamentally one of trust calibration: how much confidence should we place in a solution whose correctness we cannot independently check?

Open Problem 3: The Motivation Problem

Classical evolutionary algorithms optimise an externally specified fitness function. But biological evolution has no external objective: organisms evolve to survive and reproduce, and the “fitness landscape” is itself a product of the evolving ecosystem. This raises a deep question: what should drive artificial evolution beyond task-specific fitness?

Several proposals have been explored:

• Novelty search (Novelty Search): reward behavioural distinctness rather than objective quality.

• Curiosity-driven exploration: reward the reduction of prediction error in a learned world model, analogous to the intrinsic motivation framework in reinforcement learning.

• Aesthetic reward: define fitness as the “interestingness” or “beauty” of a solution, as judged by a learned aesthetic model or by human evaluators.

• Empowerment: reward an agent's ability to influence its environment, measured by the channel capacity between the agent's actions and the resulting states.

None of these fully resolves the motivation problem. Novelty search can degenerate into random exploration. Curiosity can be hacked by stochastic environments (the “noisy TV problem”). Aesthetic reward is subjective and domain-specific. Empowerment can lead to power-seeking behaviours that are misaligned with human values.

The motivation problem connects to a deeper philosophical question: is open-ended evolution possible without an environment that itself admits unbounded complexity? Biological evolution on Earth has produced staggering diversity, but Earth's environment, with its varied geologies, climates, and chemical substrates, is itself extraordinarily complex. A simulated environment with bounded state complexity may impose an upper limit on the complexity of the organisms it can support, regardless of the sophistication of the evolutionary algorithm. This suggests that solving the motivation problem may require co-solving the problem of environment design, that is, creating worlds rich enough to sustain genuinely open-ended dynamics.

Open Problem 4: Evolutionary Alignment

The alignment problem, ensuring that AI systems behave in accordance with human values and intentions, takes on a uniquely challenging form in the context of evolutionary AI. When a system can modify its own code, its own objective function, and even its own verification criteria, how can we guarantee that the resulting system remains aligned with human interests?

Let ${𝜽}_t$ denote the parameters of an evolving system at generation $t$, and let $U_H:\mathrm{\Theta }\to ℝ$ represent a human utility function over the space $\mathrm{\Theta }$ of possible system configurations. The alignment constraint requires that $U_H({𝜽}_t)\ge U_H({𝜽}_0)$ for all $t$, i.e., the system never becomes less aligned than it was at initialisation.

In other words, unless human utility is perfectly aligned with the system's own evolutionary drive, a condition that is virtually impossible to guarantee, evolutionary self-modification will eventually produce misalignment. The conjecture formalises an intuition that many researchers in AI safety share: systems that can rewrite their own objectives are fundamentally harder to align than systems with fixed objectives, because the alignment constraint must hold not just at initialisation but across the entire trajectory of self-modification.

The lemma quantifies the rate at which alignment degrades: the further the system's evolved fitness is from human utility (larger $\gamma$), the faster alignment erodes.

Open Problem 5: Computational Limits of Evolutionary Search

How efficiently can evolution find solutions? The celebrated No Free Lunch (NFL) theorems of Wolpert and Macready [59] show that, averaged over all possible objective functions, no search algorithm outperforms random search. But biological and artificial evolution manifestly do outperform random search on the problems they encounter. This apparent contradiction is resolved by observing that the NFL theorems average over all functions, including adversarially constructed ones, while real-world fitness landscapes have exploitable structure.

The open problem is to characterise precisely which problem structures allow evolutionary search to be efficient, and to derive tight lower bounds on the sample complexity (number of fitness evaluations) required.

Open Problem 6: Cross-Domain Transfer in Evolution

One of the most striking features of biological evolution is its ability to repurpose structures across domains: the bones that form a fish's jaw became the bones of the mammalian middle ear; feathers evolved for thermal insulation were later co-opted for flight. This capacity for exaptation, the repurposing of existing structures for new functions, is a hallmark of evolutionary creativity.

Can artificial evolution exhibit similar cross-domain transfer? Specifically, can heuristics, strategies, or building blocks evolved for one problem domain transfer to a different domain?

Open Problem 7: The Role of Embodiment

In 1994, Karl Sims demonstrated that co-evolving morphology and control in a physics-simulated virtual environment could produce startlingly creative locomotion strategies [13]. Block-like creatures learned to walk, swim, and compete for resources using body plans and neural controllers that no human engineer would have designed. Three decades later, the question Sims raised remains largely open: does physical embodiment, the coupling of a control system to a body that interacts with a physical environment, fundamentally change what evolution can discover?

There are theoretical reasons to believe the answer is yes. Embodiment introduces morphological computation: the body itself performs part of the computation that would otherwise need to be handled by the controller. A passive-dynamic walker, for example, can descend a slope with no controller at all, because the dynamics of its leg geometry do the work. This suggests that evolving in embodied settings can discover solutions that exploit physics in ways that are invisible to a purely computational search.

Open Problem 8: Coevolution of Scientific Theory and Experiment

Perhaps the most ambitious vision for evolutionary AI is the automated scientist: a system that co-evolves scientific theories and experimental designs, using the results of experiments to refine theories and the implications of theories to suggest new experiments. This is not merely an engineering challenge but a philosophical one: can a system that has no understanding of the physical world, in the human sense of “understanding,” nevertheless produce genuine scientific knowledge?

The AI Scientist framework (The AI Scientist: Autonomous Research Agents) represents a first step, but it operates within narrow domains and relies on human-specified problem templates. A truly open-ended scientific discovery system would need to:

1. Formulate hypotheses in a formal language expressive enough to capture the breadth of scientific theories.

2. Design experiments that discriminate between competing hypotheses, optimising for information gain.

3. Interpret results and update the theory accordingly, handling noise, confounders, and distributional shift.

4. Identify new phenomena that are not predicted by any existing theory, triggering the formulation of genuinely novel hypotheses.

Historical Timeline

The story of evolutionary computation spans nearly two centuries, from Darwin's foundational insight to the modern synthesis of large language models and evolutionary search. The timeline below traces the major milestones, organising them into four overlapping eras: the Foundations (1859–1975), the Classical Period (1975–2000), the Quality-Diversity Revolution (2000–2020), and the LLM-Evolution Synthesis (2020–present).

Era I: Foundations (1859–1975)

Era II: Classical Period (1975–2000)

Era III: Quality–Diversity Revolution (2000–2020)

Era IV: The LLM–Evolution Synthesis (2020–Present)

Exercises

The exercises below are organised in roughly increasing order of difficulty. Exercises Exercise 84–Exercise 87 cover classical evolutionary algorithm design. Exercises Exercise 88–Exercise 90 require implementing coevolutionary and quality-diversity methods. Exercises Exercise 91–Exercise 93 address the design of LLM-evolutionary systems. Exercises Exercise 94–Exercise 101 involve mathematical proofs. The final five exercises (marked as Challenge) are open-ended research-level problems.

Chapter Summary

This chapter has journeyed from the foundations of evolutionary computation to the frontiers of autonomous AI-driven discovery. Let us retrace the path, highlighting the key ideas and their connections.

Foundations of evolutionary search.

We began with the core abstraction shared by all evolutionary algorithms: a population of candidate solutions undergoes variation (mutation and crossover), evaluation (fitness assessment), and selection (survival of the fittest). We formalised this cycle mathematically, defining fitness landscapes, selection operators, and the schema-theoretic analysis that provides the classical explanation for why genetic algorithms work (Biological Foundations). Evolution strategies, with their elegant self-adaptation of mutation parameters through CMA-ES, showed that the evolutionary framework extends naturally to continuous optimisation in ${ℝ}^d$ (Evolution Strategies).

Coevolution: when the landscape fights back.

The introduction of coevolution transformed evolutionary computation from an optimisation technique into a framework for modelling adaptive systems. Competitive coevolution, exemplified by Hillis's host–parasite sorting networks, demonstrated that antagonistic interactions can drive solutions beyond the reach of fixed-objective optimisation (Competitive Coevolution). Cooperative coevolution decomposed complex problems into co-adapted modules, while the theory of evolutionary game dynamics provided the mathematical tools to analyse coevolutionary stability and cycling (Cooperative Coevolution).

Beyond fitness: quality, diversity, and novelty.

The quality-diversity revolution challenged the assumption that evolution should optimise a single objective. Novelty search demonstrated that abandoning the fitness function entirely can paradoxically improve performance on deceptive landscapes (Novelty Search). MAP-Elites and its descendants provided a practical framework for illuminating the space of possible solutions, producing diverse repertoires that are valuable for downstream adaptation (MAP-Elites).

The LLM-evolution synthesis.

The convergence of large language models and evolutionary algorithms represents, in our view, the most consequential development in the field since its inception. By replacing random mutation with intelligent mutation guided by an LLM's knowledge of code structure, program semantics, and domain conventions, systems like FunSearch and AlphaEvolve have achieved results that were inconceivable under classical evolutionary computation (AlphaEvolve: Open-Ended Algorithmic Discovery). The LLM serves as a learned prior over the space of solutions, dramatically narrowing the search space while preserving the exploration benefits of population-based methods.

Autonomous discovery and the AI Scientist.

The AI Scientist and ADAS frameworks showed that evolution can operate not just on solutions but on entire research programmes and agentic architectures (The AI Scientist: Autonomous Research Agents). These systems close the loop between hypothesis generation, experimental validation, and theory refinement, pointing towards a future where AI systems conduct genuine scientific research with minimal human oversight.

Open-ended evolution: the grand challenge.

The deepest aspiration of evolutionary AI is open-ended evolution: the creation of systems that produce an unbounded stream of increasingly complex and novel artefacts, mirroring the creative power of biological evolution on Earth. We formalised this aspiration in Conjecture 5 and surveyed the barriers to achieving it, including the verification bottleneck, the motivation problem, and the alignment challenge (Open Problems and Future Directions).

Safety and alignment.

Throughout the chapter, we emphasised that the power of evolutionary self-modification demands commensurate safeguards. The alignment instability conjecture (Conjecture 7) warns that self-modifying systems will inevitably drift from human values unless explicit constraints are maintained. The verification gap (Definition 74) highlights the growing difficulty of certifying evolved solutions as they increase in complexity. These are not hypothetical concerns: as LLM-evolutionary systems move from research prototypes to production deployments, the need for rigorous safety guarantees becomes urgent.

The grand narrative.

Viewed from a distance, the story told in this chapter is one of progressive liberation. Darwin liberated biology from the need for a designer. Holland liberated optimisation from the need for a gradient. Koza liberated evolution from the need for a fixed representation. Stanley and Lehman liberated search from the need for a fixed objective. And the LLM-evolution synthesis is liberating evolution from the need for random mutation, perhaps the last constraint that kept artificial evolution from matching the creative power of its biological counterpart.

Whether this liberation leads to a golden age of automated discovery or to systems that outpace our ability to understand and control them depends on the choices we make in the coming years. The open problems of Open Problems and Future Directions are not merely academic puzzles: they are the research programme on which the responsible development of evolutionary AI depends.

Connections to other chapters.

The ideas developed in this chapter intersect with numerous topics covered elsewhere in this book. The evolution of neural architectures connects directly to the neural architecture search methods of 30, where evolutionary approaches like NEAT provide an alternative to gradient-based NAS. The coevolutionary training of generators and discriminators echoes the adversarial dynamics of GANs (9), where the generator–discriminator interplay can be viewed as a special case of competitive coevolution. The use of LLMs as mutation operators draws on the autoregressive generation techniques of ch:transformers, and the quality-diversity methods that illuminate behavioural spaces have natural connections to the latent space exploration techniques in variational autoencoders (15).

Perhaps most profoundly, the open-endedness criterion (Conjecture 5) connects to fundamental questions in information theory (ch:info-theory): can we use information-theoretic quantities to measure and certify the ongoing novelty production of an evolutionary system? The answer to this question may lie at the intersection of Kolmogorov complexity, rate-distortion theory, and the mathematics of dynamical systems.

A closing reflection.

Biological evolution has had nearly four billion years to demonstrate its creative power, producing everything from archaebacteria to the human neocortex, from the camouflage of the cuttlefish to the echolocation of the bat. Artificial evolution has had barely fifty years and is already discovering novel mathematics, optimising production infrastructure, and taking its first steps towards autonomous science. The gap between these two timescales is itself a measure of how far we have come and how much further we might go.

The detective story that began with Darwin's insight into natural selection has led us to a frontier where evolution and intelligence are no longer separate phenomena but deeply intertwined partners in the search for knowledge. The next chapter of this story will be written not by nature alone, but by the evolutionary systems we create, systems that, if we build them wisely, may one day surprise even their creators.

[heading=subbibliography]