27 Agentic AI and Multi-Agent Systems

No human accomplishment of significance was ever achieved alone. The Great Pyramid of Giza required an estimated one hundred thousand workers coordinating across two decades, quarrying limestone, transporting blocks along the Nile, fitting them with sub-millimetre precision, all without a single blueprint in the modern sense. The Manhattan Project brought together physicists, engineers, chemists, and mathematicians from dozens of institutions, coordinating under secrecy constraints that would challenge any modern project manager. The Human Genome Project, launched in 1990 and completed in 2003, coordinated sequencing laboratories across six countries, each responsible for different chromosomes, sharing data through standardised file formats that evolved as the project progressed. The Apollo programme marshalled four hundred thousand people across twenty thousand companies to land two humans on the Moon.

In every case, the achievement was not merely the sum of individual contributions. No single worker could have built the pyramid; no single physicist could have designed the bomb; no single laboratory could have sequenced the genome. Something emerged from the interaction, a collective capability that transcended what any participant could have accomplished alone. The question is: what?

The answer, we argue, is the coordination protocol. What distinguishes these achievements is not just the raw capability of individual participants but how information flows between them, how decisions are made, how credit and blame are assigned, and how the collective adapts when individual components fail. The Roman legions conquered an empire not because individual soldiers were stronger than their opponents (they often were not) but because their formation tactics, communication systems, and hierarchical command structure enabled coordination at a scale their enemies could not match.

This is precisely the mathematical question we address in this chapter. Given $n$ agents, each with bounded individual capability, under what conditions does their collective performance exceed the sum of their individual contributions? When does collaboration produce superadditive gains, and when does coordination overhead consume any benefit? These questions are as old as economics (Adam Smith's division of labour, 1776), as central to biology (the evolution of multicellularity), and as urgent as modern AI (teams of LLM agents solving complex tasks).

The chapter sits at the culmination of a deliberate arc. In ch:reasoning, we built the reasoning engine, showing how chain-of-thought, tree search, and reinforcement learning enable language models to solve problems requiring multi-step inference. In ch:memory, we constructed the memory substrate: KV caches, retrieval-augmented generation, and memory-augmented architectures that allow models to access and manipulate information beyond their context window. In ch:alignment, we installed the value system: RLHF, DPO, and GRPO that align model outputs with human preferences. In ch:continual, we studied the temporal dynamics, how models can learn continuously without catastrophic forgetting. Now we ask the culminating question: what happens when multiple such systems interact?

The answer is both exciting and mathematically rich. A single LLM agent is already a remarkable artifact: a system that perceives natural language, reasons through chains of thought, takes actions via tool calls, and updates its behaviour through in-context learning. But a team of such agents, debating, delegating, verifying each other's work, specialising in different sub-tasks, opens an entirely new design space whose mathematics draws on game theory, mechanism design, distributed computing, social choice theory, and the theory of stochastic processes.

The chapter is organised across five files to manage its length. This file (File A) establishes the foundations: we begin with the intelligence of collectives (The Intelligence of Collectives), move to the anatomy of a single agent (The Anatomy of a Single Agent), study adaptive and continual-learning agents (Adaptive and Continual-Learning Agents), develop agent reasoning and planning (Agent Reasoning and Planning), and conclude with reward shaping and safety (Reward, Alignment, and Agent Safety). Files B–E build on these foundations to develop multi-agent coordination, game theory, emergence, evaluation, and applications.

Prerequisites.

We assume familiarity with the reasoning mechanisms of ch:reasoning (chain-of-thought, tree search, GRPO), the memory architectures of ch:memory (KV cache, retrieval-augmented generation, memory-augmented transformers), the alignment pipeline of ch:alignment (SFT, reward modelling, DPO, GRPO), and the continual learning framework of ch:continual (task sequences, catastrophic forgetting, regularisation and replay methods).

The Intelligence of Collectives

Before we formalise artificial agents, we study the phenomenon we seek to replicate: collective intelligence as it manifests in biology, human societies, and their mathematical abstractions. The goal is to extract design principles that will guide our construction of multi-agent AI systems.

Biological Intelligence as Collective Computation

Nature provides the most compelling existence proofs of collective intelligence. In each case, the individual agent is simple, sometimes trivially so, yet the collective solves problems of remarkable complexity.

Ant colony optimisation.

Consider a colony of ants foraging for food. An individual ant has a brain containing roughly 250,000 neurons (compare to the human brain's 86 billion) and can perform only simple behaviours: move, deposit pheromone, follow pheromone gradients, pick up food, return to nest. Yet the colony collectively solves shortest-path problems, allocates labour across tasks, manages waste, tends fungal gardens, and wages war, all without a central controller.

The mathematical abstraction of this phenomenon is Ant Colony Optimisation (ACO), introduced by Dorigo [11]. Consider a graph $G=(V,E)$ with edge costs $c_{ij}$ for $(i,j)\in E$. Each ant $k$ at node $i$ selects the next node $j$ with probability (ACO Transition)$$\begin{array}{ccc}& p_{ij}^{(k)}=\frac{{[{\tau }_{ij}]}^{\alpha }{[{\eta }_{ij}]}^{\beta }}{{\sum }_{l\in {𝒩}_i}{[{\tau }_{il}]}^{\alpha }{[{\eta }_{il}]}^{\beta }},& \text{<span class="tml-eqn"></span>}\end{array}$$ where ${\tau }_{ij}$ is the pheromone concentration on edge $(i,j)$, ${\eta }_{ij}=1/c_{ij}$ is the heuristic desirability (inverse cost), ${𝒩}_i$ is the set of unvisited neighbours of $i$, and $\alpha ,\beta >0$ control the relative influence of pheromone versus heuristic information. After all ants complete their tours, pheromones are updated: (ACO Update)$$\begin{array}{ccc}& {\tau }_{ij}\leftarrow (1-\rho ){\tau }_{ij}+\sum _{k=1}^m\mathrm{\Delta }{\tau }_{ij}^{(k)},\mathrm{\Delta }{\tau }_{ij}^{(k)}=\{\begin{array}{cc}Q/L_k& \text{if ant }k\text{ used edge }(i,j),\\ 0& \text{otherwise},\end{array}& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\rho \in (\mathrm{0,1})$ is the evaporation rate, $m$ is the number of ants, $L_k$ is the length of ant $k$'s tour, and $Q>0$ is a constant. The evaporation term prevents convergence to suboptimal solutions; the reinforcement term causes shorter paths to accumulate more pheromone, biasing future ants toward them.

Bee swarm decision-making.

Honeybee swarms make collective decisions about nest sites through a process remarkably analogous to neural computation. Scout bees discover candidate sites and return to the swarm, performing waggle dances whose duration is proportional to site quality. Other scouts are recruited to inspect advertised sites, return, and dance for their preferred option. The process converges when a quorum of scouts aggregates at a single site, a distributed voting mechanism that, under mild conditions, selects the optimal site with high probability.

Mathematically, this can be modelled as a system of coupled stochastic differential equations. Let $x_i(t)$ denote the number of scouts committed to site $i$ at time $t$. The dynamics are (BEE Dynamics)$$\begin{array}{ccc}& \frac{d{x}_i}{dt}=\underset{\text{recruitment}}{\underbrace{{\alpha }_i(N-{\sum }_j{x}_j)}}-\underset{\text{abandonment}}{\underbrace{{\gamma }_i{x}_i}}+\underset{\text{noise}}{\underbrace{{\sigma }_i{\xi }_i(t)}},& \text{<span class="tml-eqn"></span>}\end{array}$$ where ${\alpha }_i$ is the recruitment rate (proportional to site quality), ${\gamma }_i$ is the abandonment rate, $N$ is the total number of scouts, and ${\xi }_i(t)$ is white noise. The quorum decision triggers when $x_i(t)>\theta$ for some threshold $\theta$.

The immune system as multi-agent defence.

The adaptive immune system is perhaps the most sophisticated multi-agent system in biology. B-cells act as specialist agents, each producing antibodies tuned to a specific antigen through somatic hypermutation, a form of local search in antibody shape space. T-cells act as orchestrator agents: helper T-cells coordinate the immune response by activating B-cells and cytotoxic T-cells, while regulatory T-cells suppress the response to prevent autoimmune damage. The clonal selection algorithm, expansion of B-cells whose antibodies bind the antigen, combined with random mutation, implements a distributed evolutionary search.

Neural circuits.

A single biological neuron is a simple threshold device: it sums weighted inputs and fires if the total exceeds a threshold. Yet networks of neurons compute. The progression from neuron to brain to society exhibits a fractal pattern of collective intelligence: at each level, simple components interact through local rules to produce behaviour that no individual component could achieve. This observation motivates our formal definition.

The weaker condition states that the team outperforms its best member; the stronger states that it outperforms the sum of all members. Superadditivity arises when agents' contributions are complementary: when the value of combining diverse capabilities exceeds the value of stacking identical ones.

We now establish a foundational result: under what conditions does collective intelligence emerge? The following proposition gives sufficient conditions for superadditivity.

This result formalises the intuition that collective intelligence requires two ingredients: agents must be different from each other (complementarity, $\alpha >0$) and the cost of getting them to work together must not be too high (low coordination cost, $\eta <1$). A team of identical agents ($\alpha =0$) can never exhibit superadditivity; nor can a team of perfect specialists whose every communication is garbled ($\eta \ge 1$).

Human Organisations as Agent Systems

Human organisations provide a rich source of multi-agent design patterns. We highlight four that map directly to artificial MAS architectures.

Division of labour.

Adam Smith's famous example of the pin factory (1776) illustrates that specialisation multiplies productivity. One worker performing all operations produces perhaps one pin per day; ten workers, each specialising in one operation, produce 48,000 pins per day, a 4,800-fold increase that cannot be explained by 10 workers each being 480 times faster. The gain comes from three sources:

1. Skill deepening: specialisation allows each worker to master one sub-task.

2. Reduced context-switching: no time lost moving between operations.

3. Tool specialisation: each operation admits purpose-built tools.

In LLM multi-agent systems, the same principle applies: a code-writing agent, a code-reviewing agent, and a testing agent can collectively produce higher-quality software than a single agent performing all three roles, because each agent's prompt can be specialised, its context window devoted entirely to its sub-task, and its tool set tailored to its function.

Hierarchies versus flat structures.

Military organisations employ strict hierarchies: commands flow downward, reports flow upward, and each node has a bounded span of control (typically 3–7 subordinates). Open-source software communities, by contrast, employ flat structures with fluid role assignment, reputation-based authority, and asynchronous communication. Both patterns appear in artificial MAS: orchestrator–worker architectures implement hierarchy (a planner agent delegates to specialist workers), while peer-to-peer debate architectures implement flat structures (all agents have equal authority and negotiate consensus).

Scientific collaboration as publish–subscribe MAS.

The scientific community operates as a massive multi-agent system where agents (researchers) communicate through publications. This is a publish–subscribe architecture: agents publish results to shared channels (journals, arXiv); other agents subscribe to relevant channels and build on published results. The system is asynchronous, decentralised, and robust to individual agent failure, precisely the properties we seek in artificial MAS. Peer review implements a multi-agent verification protocol: multiple independent agents evaluate each contribution, and acceptance requires consensus.

Market mechanisms.

Markets coordinate economic agents through prices, a remarkably efficient communication protocol. Hayek's insight (1945) was that prices aggregate distributed local knowledge that no central planner could collect [61]. In artificial MAS, analogous mechanisms can coordinate agents: an “attention market” where agents bid computational resources for sub-tasks, or a “reward market” where task completion generates tokens allocated by Shapley value (Game Theory Foundations for Multi-Agent Systems).

From Biological to Artificial Collectives

The emergence of large language models as a substrate for agent construction represents a qualitative shift in multi-agent systems. We highlight three aspects of this shift.

Natural language as universal protocol.

Traditional multi-agent systems required hand-coded communication protocols: KQML (Knowledge Query and Manipulation Language), FIPA-ACL, or domain-specific message formats. LLM agents communicate in natural language, which is open-ended: agents can negotiate, clarify ambiguity, explain their reasoning, and even invent new terminology during a conversation. This flexibility comes at a cost (natural language is noisy, ambiguous, and expensive to process) but it eliminates the bottleneck of protocol design and enables agents to coordinate on tasks that were not anticipated at design time.

The capability gap.

Despite the excitement surrounding LLM agents, significant gaps remain between biological and artificial collectives:

• True adaptation: biological agents learn continuously from experience; most LLM agents operate with frozen parameters, adapting only through in-context learning (which is bounded by the context window). The continual learning methods of ch:continual offer a path forward.

• Embodiment: biological agents interact with the physical world through sensorimotor loops; LLM agents interact primarily through text, with tool use as a limited form of embodiment.

• Intrinsic motivation: biological agents are driven by evolved reward signals (hunger, pain, curiosity); LLM agents require externally specified reward functions, which are subject to the misspecification problems studied in ch:alignment.

The opportunity.

What LLM agents can do that biological agents cannot: perfect communication fidelity (every message is recorded verbatim), instant cloning (an agent can be replicated with identical capabilities), perfect memory of instructions (no forgetting within the context window), and the ability to process and generate structured data (JSON, code, mathematical notation) alongside natural language. These capabilities open design possibilities that have no biological precedent.

The Anatomy of a Single Agent

Before we can study multi-agent systems, we must understand the architecture of the single agent that serves as the building block. This section provides a rigorous mathematical specification of an LLM-based agent, connecting each component to the theory developed in earlier chapters.

Formal Definition

An LLM agent is more than a language model. It is a language model embedded in a loop that perceives, reasons, acts, and observes the consequences of its actions. We formalise this as follows.

The definition separates concerns cleanly: $\mathrm{\Phi }$ provides the cognitive engine, $ℳ$ provides storage and retrieval, $ℛ$ provides deliberative reasoning, and $o$ and $e$ provide the interface between the agent and its environment.

The Agent Loop

The defining characteristic of an agent, as opposed to a passive model answering queries, is that it operates in a loop: perceive, reason, act, observe the result, and repeat. We formalise this loop as a partially observable Markov decision process.

The agent loop unfolds as follows. At each time step $t$:

1. Observe: the agent receives observation $o_t\sim Z(\cdot |s_t,a_{t-1})$.

2. Reason: the agent applies its reasoning module $ℛ$ to generate intermediate reasoning tokens $r_t=ℛ(h_t)$ (e.g., chain-of-thought).

3. Act: conditioned on $h_t$ and $r_t$, the agent selects action $a_t\sim \pi (\cdot |h_t,r_t)$.

4. Execute: the execution engine applies $a_t$ to the environment, producing new state $s_{t+1}\sim T(\cdot |s_t,a_t)$.

5. Update memory: the agent appends $(o_t,r_t,a_t)$ to its episodic memory ${ℳ}_e$ and updates its working memory ${ℳ}_w$.

The loop terminates when $a_t=\text{𝚂𝚃𝙾𝙿}$ or a maximum number of steps is reached.

Context length bounds agent horizon.

A crucial constraint on the agent loop is the context window length $L_{\max }$. The history $h_t=(o_1,a_1,r_1,\dots ,o_t)$ grows by approximately $|o_t|+|r_t|+|a_t|$ tokens at each step. If the average step consumes $\bar{l}$ tokens, then the agent can maintain at most $T_{\max }\approx L_{\max }/\bar{l}$ steps of history before the context window overflows. At this point, the agent faces a choice:

1. Truncation: discard the oldest history, losing information about early steps. This is analogous to the sliding window attention of ch:memory.

2. Compression: summarise the history into a compact representation and replace the raw history with the summary. This trades fidelity for capacity.

3. Retrieval: offload the full history to external memory and retrieve relevant portions on demand, as in retrieval-augmented generation (ch:memory).

The choice among these strategies profoundly affects agent behaviour. Truncation loses early context, which may be critical for understanding the task structure. Compression introduces lossy approximation. Retrieval adds latency and requires a good similarity metric. In practice, many agent frameworks combine all three: recent history is kept verbatim, older history is summarised, and key episodes are stored in a retrievable memory bank.

Memory in Agents

The memory system $ℳ$ deserves special attention because it is what enables an agent to go beyond pure reactive behaviour. We connect each component to the memory framework of ch:memory.

Working memory ${ℳ}_w$.

The agent's working memory is the KV cache, the context window of the LLM. Its capacity is bounded by the maximum context length $L_{\max }$, and its content determines the agent's “current awareness.” In cognitive terms, this is the analogue of human working memory: the small, actively maintained set of information that drives moment-to-moment behaviour. The KV cache management techniques of ch:memory (sparse attention, cache compression, sliding windows) directly apply to agent working memory.

Episodic memory ${ℳ}_e$.

Episodic memory stores the agent's history of interactions: past observations, actions, rewards, and reasoning traces. It serves two functions: (1) providing context for decision-making (“what happened last time I tried this approach?”) and (2) enabling learning from experience (Reflexion-style self-critique [58]). Mathematically, ${ℳ}_e={\{(o_t,a_t,r_t,R_t)\}}_{t=1}^T$ is a trajectory buffer, possibly with a retrieval mechanism that selects relevant episodes based on similarity to the current observation.

Semantic memory ${ℳ}_s$.

The agent's semantic memory comprises both the parametric knowledge encoded in the LLM weights $𝜽$ and any external knowledge base accessible through retrieval-augmented generation (RAG, sec:memory:augmented). The parametric component is immutable during a single agent episode (the weights are frozen); the RAG component can be updated by adding documents to the retrieval corpus.

Procedural memory ${ℳ}_p$.

Procedural memory encodes “how to do things”: standard operating procedures, tool usage patterns, and workflow templates. In current LLM agents, this is typically implemented through the system prompt and few-shot examples. More sophisticated approaches store successful action sequences and retrieve them when similar situations arise, a form of case-based reasoning.

Tool Use as Action

A defining feature of modern LLM agents is their ability to use tools, external functions that extend the agent's capabilities beyond text generation.

Tool selection as multi-armed bandit.

When an agent has access to $K$ tools, the problem of selecting the right tool for a given sub-task can be viewed as a contextual multi-armed bandit problem. Let $c_t\in 𝒞$ be the context (the current observation and task description) at time $t$, and let $r_t(j)\in [\mathrm{0,1}]$ be the reward for selecting tool $j$ in context $c_t$. The agent's tool-selection policy ${\pi }_{\text{tool}}:𝒞\to \mathrm{\Delta }([K])$ seeks to minimise the cumulative regret (TOOL Regret)$$\begin{array}{ccc}& \text{Regret}(T)=\sum _{t=1}^T[r_t(j_t^{\ast })-r_t(j_t)],& \text{<span class="tml-eqn"></span>}\end{array}$$ where $j_t^{\ast }={\mathrm{arg\hspace{0.17em}max}}_j{r}_t(j)$ is the optimal tool for context $c_t$ and $j_t\sim {\pi }_{\text{tool}}(\cdot |c_t)$ is the selected tool. Standard results from bandit theory give:

Adaptive and Continual-Learning Agents

The agents defined in The Anatomy of a Single Agent have a fundamental limitation: their parameters $𝜽$ are frozen during deployment. The agent can adapt within a single episode through in-context learning, conditioning on an ever-growing conversation history, but this adaptation is ephemeral (lost when the context window is cleared) and bounded (by the maximum context length $L_{\max }$). A truly capable agent must learn across episodes, accumulating knowledge from past experience and improving its behaviour over time. This section connects the agent framework to the continual learning theory of ch:continual and formalises the mechanisms by which agents adapt.

The Continual Agent

The continual agent framework unifies several agent architectures:

• Frozen agent ($𝒰=\text{id}$): no parameter updates; adaptation occurs only through in-context learning. This is the standard deployment mode for most current LLM agents.

• Fine-tuned agent ($𝒰=\text{SGD}$): the agent's parameters are updated via gradient descent on collected experience, using the methods of ch:alignment (SFT, DPO, GRPO).

• Memory-augmented agent: $𝒰$ updates the episodic memory ${ℳ}_e$ rather than the parameters, storing successful trajectories for future retrieval.

• Hybrid agent: $𝒰$ combines parameter updates with memory updates, using the continual learning techniques of ch:continual (EWC, experience replay) to prevent forgetting.

In-Context Learning as Agent Adaptation

The simplest and most widely used form of agent adaptation is in-context learning (ICL): the agent conditions its policy on examples or experience provided in the context window, without any parameter updates. We now prove that this mechanism implements an implicit form of Bayesian inference.

Agent Self-Improvement

Can an agent improve itself? The idea is tantalising: an agent generates trajectories, evaluates which ones succeeded, and fine-tunes itself on the successful ones, a bootstrapping loop where the agent is simultaneously the learner and the data generator. We formalise this through a variational framework.

This procedure is closely related to GRPO (sec:alignment:grpo) and expert iteration. The key difference is that the reward signal comes from the agent's own experience rather than from human preferences: the agent generates data, evaluates it against an objective criterion (does the code compile? does the plan achieve the goal?), and trains on the successful subset.

Agent Reasoning and Planning

An agent that merely reacts to observations, selecting actions greedily without considering their long-term consequences, will fail at any task requiring foresight. Consider a software engineering agent tasked with refactoring a large codebase: it must plan a sequence of changes, anticipate that modifying one module will break tests in another, and schedule the changes in an order that maintains a working system at each step. This section develops the mathematical theory of agent planning, from classical search to hierarchical decomposition and world models.

Planning as Search

The connection to LLM agents is direct. In tree-of-thought reasoning [62], the “states” are partial reasoning traces, “actions” are candidate next-thought tokens, the “transition” is deterministic (appending a thought to the trace), and the “cost” is the negative of a value function that evaluates partial traces. The agent explores a tree of reasoning paths and selects the one with highest cumulative value, which is precisely a search over action sequences in the planning formulation .

The branching factor of this search tree is the key computational bottleneck. If the agent has $b$ candidate actions at each step and the plan horizon is $T$, the naive search space has size $b^T$. Even for modest values ($b=10$, $T=20$), this is ${10}^{20}$, far beyond any computational budget. This motivates two strategies: pruning (reducing $b$ at each step by evaluating candidate actions) and hierarchical decomposition (reducing $T$ by planning at multiple levels of abstraction).

Hierarchical Planning

In LLM agent systems, hierarchical planning manifests as task decomposition prompting: the agent first produces a high-level plan (“Step 1: set up the database; Step 2: implement the API; Step 3: write the frontend”), then plans each step in detail. Proposition 4 provides the mathematical justification: hierarchical planning is not merely a heuristic convenience but an exponential reduction in computational complexity under decomposability conditions.

World Models for Agents

Effective planning requires the ability to predict the consequences of actions before executing them. This prediction capability is provided by a world model.

The crucial observation is that the LLM itself is a world model for text-based environments. When an LLM predicts the next token in a coding task, it is implicitly modelling the state transition: “if I add this line of code, what will the compiler output?” The generative models developed in Parts III–VI (VAEs, flows, diffusion models) are all world models in this sense: they model $p(s^{\prime }|s,a)$ for different state representations. The distinction in the agent setting is that the world model is used not for generation per se, but for planning: evaluating hypothetical action sequences before committing to one.

Compounding error in learned world models.

A world model trained on single-step transitions accumulates error over multi-step rollouts. If the single-step prediction error is $ϵ$ (in total variation distance), then an $H$-step rollout can deviate from the true trajectory by up to $Hϵ$ in total variation. This compounding error problem is the fundamental limitation of model-based planning and motivates frequent re-planning (model-predictive control) and world model ensembles.

Level-$K$K Reasoning

When an agent operates alongside other agents, even before we formalise multi-agent systems in subsequent sections, it must reason about what others will do. Level-$K$ reasoning, from behavioural game theory [63][64], provides a hierarchy of increasingly sophisticated strategic reasoning.

Level-$K$ reasoning maps naturally to LLM agents. When prompted with “What would the other agent do?”, a Level-1 LLM agent simulates the other agent's response assuming it acts naively (Level-0). A Level-2 agent reasons: “The other agent will predict that I act naively, and will best-respond to that, so I should best-respond to its best-response.” This recursive simulation is precisely the chain-of-thought reasoning studied in ch:reasoning, applied to strategic settings.

Reward, Alignment, and Agent Safety

We have constructed agents that perceive, reason, plan, and learn from experience. But to what end? The reward function determines what the agent optimises, and a misspecified reward can lead to capable agents pursuing the wrong objectives with devastating effectiveness. This section extends the alignment framework of ch:alignment from single-turn language generation to multi-step agent trajectories, and examines failure modes unique to the agentic setting.

Reward Shaping for Agents

In the alignment chapter, the reward function $r(x,y)$ scored a single prompt–response pair. For agents operating over multi-step trajectories, we need a richer reward structure.

The decomposition into step and outcome rewards reflects a practical design choice. Pure outcome reward ($r=0$, $r_{\text{final}}\ne 0$) makes credit assignment difficult: the agent receives no signal until the end of the trajectory, and must attribute the final outcome to individual actions across many steps. Pure step reward ($r_{\text{final}}=0$, $r\ne 0$) can lead to reward hacking: the agent may accumulate step rewards without making progress toward the goal. The combination allows the designer to encode both process (“use the right approach”) and outcome (“solve the problem”) signals.

A concrete instantiation from Optima [20] decomposes the reward for dialogue agents as (Optima Reward)$$\begin{array}{ccc}& R=R_{\text{task}}-{\lambda }_{\text{token}}\cdot R_{\text{token}}+{\lambda }_{\text{loss}}/R_{\text{loss}},& \text{<span class="tml-eqn"></span>}\end{array}$$ where $R_{\text{task}}$ measures task completion, $R_{\text{token}}$ penalises excessive token usage (encouraging efficiency), and $R_{\text{loss}}$ penalises information loss (encouraging faithful communication). The hyperparameters ${\lambda }_{\text{token}}$ and ${\lambda }_{\text{loss}}$ trade off efficiency against completeness.

Temporal credit assignment.

The temporal credit assignment problem asks: given a final outcome $r_{\text{final}}(s_T)$, which of the actions $a_0,a_1,\dots ,a_T$ were responsible? In single-turn generation, the entire output is one “action” and credit assignment is trivial. In multi-step agent trajectories, credit must be distributed across dozens or hundreds of actions. The standard approach uses a value function $V^{\pi }(s)={𝔼}_{\pi }[{\sum }_{t^{\prime }=t}^T{\gamma }^{t^{\prime }-t}r(s_{t^{\prime }},a_{t^{\prime }})+r_{\text{final}}(s_T)|s_t=s]$ and defines the advantage (Advantage)$$\begin{array}{ccc}& A^{\pi }(s_t,a_t)=r(s_t,a_t)+\gamma V^{\pi }(s_{t+1})-V^{\pi }(s_t),& \text{<span class="tml-eqn"></span>}\end{array}$$ which measures how much better action $a_t$ is compared to the average action under policy $\pi$. The advantage function is the foundation of policy gradient methods (PPO, GRPO from ch:alignment) and provides the per-step credit signal needed for agent learning.

Agent Alignment as Constrained POMDP

We now extend the alignment objective from ch:alignment to the agent setting. In the single-turn case, alignment was a KL-constrained reward maximisation (eq:alignment:central-objective). In the agent case, we must additionally enforce safety constraints over trajectories.

Lagrangian relaxation.

The constrained optimisation can be solved via Lagrangian relaxation. Introducing multipliers ${\lambda }_{\text{KL}}\ge 0$ and ${\lambda }_{\text{safe}}\ge 0$, the Lagrangian is (Lagrangian)$$\begin{array}{ccc}& ℒ(\pi ,\lambda )={𝔼}_{\tau \sim \pi }[R(\tau )]-{\lambda }_{\text{KL}}\sum _t{D}_{\mathrm{KL}}[\pi (\cdot |h_t)\Vert {\pi }_{\text{ref}}(\cdot |h_t)]-{\lambda }_{\text{safe}}(\mathrm{Pr}[\tau \in 𝒰]-ϵ).& \text{<span class="tml-eqn"></span>}\end{array}$$ The primal-dual algorithm alternates between maximising $ℒ$ over $\pi$ (a policy optimisation step, implementable via PPO or GRPO from ch:alignment) and updating the multipliers via gradient ascent: ${\lambda }_{\text{safe}}\leftarrow {[{\lambda }_{\text{safe}}+\eta (\mathrm{Pr}[\tau \in 𝒰]-ϵ)]}_{+}$. This procedure is guaranteed to converge to a saddle point under standard convexity conditions on the constraint set.

The Sycophancy Problem

A subtle failure mode arises when agents are trained with RLHF: sycophancy, the tendency to produce responses that please the user or the reward model rather than responses that are correct. This problem is particularly pernicious in multi-agent debate systems, where sycophantic agents agree with each other rather than challenging incorrect claims.

The implication for multi-agent debate is grave. If all agents in a debate system are trained with the same sycophancy-biased reward model, they will converge to agreeable consensus rather than correct conclusions, the “Talk Isn't Always Cheap” phenomenon [20]. Mitigations include:

1. Training debaters with diverse reward models to break sycophancy correlations.

2. Explicitly rewarding disagreement and factual challenge in the debate protocol.

3. Using ground-truth verification (when available) alongside preference-based reward.

4. Maintaining a large $\beta$ (weak reward optimisation) to keep the policy close to the pretrained distribution.

Sandboxing and containment.

A complementary approach to alignment constraints is sandboxing: restricting the environment in which the agent operates so that unsafe actions have limited consequences. A code execution sandbox, for instance, prevents the agent from modifying the host file system or accessing the network. Formally, sandboxing replaces the transition function $T$ with a restricted version $T_{\text{safe}}$ that maps unsafe actions to a null transition: $T_{\text{safe}}(s^{\prime }|s,a)={\delta }_s(s^{\prime })$ if $a\in {𝒜}_{\text{unsafe}}$ (the state does not change). The agent still “takes” the unsafe action in its reasoning trace, but the execution engine blocks its effect. This provides a hard safety guarantee independent of the policy $\pi$: no matter how the agent reasons, unsafe actions produce no environmental change.

The limitation of sandboxing is that it requires enumerating the set ${𝒜}_{\text{unsafe}}$ in advance, a task that grows increasingly difficult as agents gain access to more tools and environments. The most robust safety strategy combines all three approaches: KL-constrained alignment (soft constraint on the policy), safety-constrained optimisation (probabilistic constraint on trajectories), and sandboxing (hard constraint on the environment).

This concludes our treatment of single-agent foundations. We have defined the LLM agent (Definition 3), formalised its operation as a POMDP (Definition 4), shown how it adapts through in-context learning (Theorem 1) and self-improvement (Definition 7), developed its planning capabilities (Agent Reasoning and Planning), and established the alignment framework that keeps it safe (Definition 13). In the sections that follow (Files B–E), we will bring multiple such agents together and study the rich mathematics of their interaction.

Multi-Agent System Formalisms

The preceding sections constructed the single agent: a 7-tuple (Definition 3) operating as a POMDP (Definition 4), equipped with reasoning, memory, and alignment. We now ask: what happens when multiple such agents share an environment? The answer requires a formal language for multi-agent systems (MAS) that specifies the agents, the medium through which they communicate, and the mechanisms that coordinate their collective behaviour.

This section develops that language in four stages. First, we define the MAS as a structured tuple (The Multi-Agent System Tuple), drawing on Kim et al. [19]. Second, we formalise the communication protocol as an information-theoretic channel (Natural Language as a Shannon Channel), following Tran et al. [65]. Third, we derive the turn-count power law governing coordination overhead (Turn-Count Scaling and the Power Law). Fourth, we establish a capability ceiling, a performance plateau beyond which adding agents yields diminishing returns (The Capability Ceiling).

The Multi-Agent System Tuple

Every multi-agent system, regardless of implementation (AutoGen, AgentScope, MetaGPT, CrewAI), can be described by a single mathematical object.

Kim et al. [19] show that every existing LLM-based MAS framework is an instantiation of this tuple; the differences lie only in how $𝒞$ and $\mathrm{\Omega }$ are implemented.

Given a task distribution $𝒟$, the system-level performance is $P(𝒮)={𝔼}_{\tau \sim 𝒟}[R(𝒮,\tau )]$, where $R\in [\mathrm{0,1}]$ is a bounded reward. Define the solo baseline $P_{\mathrm{solo}}={\max }_{i}P(a_i)$ and the coordination gain $\mathrm{\Delta }P=P(𝒮)-P_{\mathrm{solo}}$. The system is superadditive when $\mathrm{\Delta }P>0$ and exhibits coordination failure when $\mathrm{\Delta }P<0$.

Communication Protocols as Information Channels

Tran et al. [65] identify three fundamental channel types in LLM-based MAS: (i) broadcast channels (shared chat history, e.g. AutoGen), (ii) point-to-point channels (directed delegation, e.g. CrewAI), and (iii) publish-subscribe channels (topic-based routing, e.g. AgentScope). We formalise these using information theory.

The capacity $C(𝒞)$ is limited by three practical factors: (i) context window saturation: with window size $W$ tokens, $C_{\mathrm{eff}}\le W{\log }_2\left|𝒱\right|$ bits per round; (ii) attention dilution: attention weight per token decreases as $O(1/L)$ with message length $L$; and (iii) semantic redundancy: English text carries roughly 1–1.5 bits per character [66], far below the raw vocabulary capacity.

MAS Block Diagram

fig:agents:mas-block illustrates the architectural components and information flows in the MAS 4-tuple.

Turn-Count Scaling and the Power Law

How does the number of communication rounds scale with team size? More agents bring more capabilities but also require more coordination. Kim et al. [19] discovered that this relationship follows a power law.

The implication is stark: coordination cost is superlinear. Doubling the number of agents more than triples the required turns.

Visualising the Power Law

fig:agents:power-law displays the power law on log-log axes, where $\log T=\log \alpha +\gamma \log (n+\beta )$ appears as a line with slope $\gamma$.

The Capability Ceiling

Does performance grow without bound as we add agents, or does it saturate? Kim et al. [19] demonstrate a hard performance ceiling.

Kim et al. report a single-agent ceiling of $P_{\mathrm{SA}}\approx 0.45$ on composite benchmarks spanning coding, reasoning, and general knowledge. The multi-agent ceiling is higher but still finite:

The 20-Parameter Regression Model

To predict performance in specific settings, Kim et al. [19] fit a 20-parameter regression: (Regression)$$\begin{array}{ccc}& \hat{P}(𝒮)=\sigma ({𝒘}^{\top }\phi (𝒮)+b),& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\sigma$ is the logistic sigmoid, $𝒘\in {\mathrm{ℝ}}^{20}$, and $\phi (𝒮)\in {\mathrm{ℝ}}^{20}$ encodes five feature groups:

llc@ Group	Features (count)	Importance
Agent capability	Base model, context length, tool count (4)	0.82
Team composition	Size $n$, diversity index, role overlap (4)	0.76
Communication	Channel type, bandwidth, format (4)	0.61
Orchestration	Topology, turn policy, delegation depth (4)	0.54
Task properties	Decomposability, difficulty, domain (4)	0.69

The model achieves $R^2=0.89$ on held-out configurations. The three most important individual features are: (1) base model capability ($0.91$), (2) team diversity index ($0.83$), and (3) task decomposability ($0.78$).

Collaborative vs. Competitive Systems

Tran et al. [65] distinguish two interaction modes. In collaborative systems, all agents share a common reward $r_1=\cdots =r_n=r_{\mathrm{sys}}$ and the design challenge is minimising coordination overhead. Performance satisfies $P(𝒮)\ge P_{\mathrm{solo}}$ whenever the orchestrator can defer to the best solo agent. In competitive systems, rewards are anti-correlated ($r_i+r_j\le r_{\max }$ for some pairs) and the challenge is harnessing productive disagreement (as in multi-agent debate) while avoiding destructive competition.

The 4-tuple $𝒮$ provides the structural description for both modes; the turn-count power law governs the communication budget; the capability ceiling bounds achievable performance. Collaborative systems favour consensus-seeking orchestration (majority vote, opinion averaging), while competitive systems favour adversarial orchestration (debate, red-teaming). The game-theoretic analysis of both modes is developed in Game Theory Foundations for Multi-Agent Systems.

Summary and Preview

This section established the mathematical foundations for multi-agent systems: the MAS 4-tuple (Definition 14), communication protocols as information channels (Definition 15), the turn-count power law $T(n)=2.72{(n+0.5)}^{1.724}$ (Theorem 3), and the capability ceiling at $P_{\mathrm{SA}}\approx 0.45$ (def:agents:ceiling,prop:agents:ceiling-bound).

Multi-agent collaboration can yield superadditive gains, but the power law imposes a coordination tax growing as $O(n^{1.724})$, and the capability ceiling ensures that agent-scaling alone cannot substitute for improving base model capabilities. The path to effective MAS runs through three design levers: communication efficiency (reducing $\gamma$), agent diversity (raising $P_{\mathrm{ceil}}$), and orchestration quality (optimising the coordination budget).

Communication Theory for Agents

Communication is the circulatory system of any multi-agent architecture. An agent that cannot communicate is an isolated reasoner; a team that communicates poorly may perform worse than its weakest member. Yet the converse is not automatically true: more communication does not imply better outcomes. In this section we develop a rigorous information-theoretic framework for inter-agent communication, prove three identifiability theorems for thought communication, a paradigm in which agents exchange compressed latent representations rather than verbose natural-language strings, and analyse both the promise of optimised communication (the Optima framework) and its failure modes (sycophancy-induced debate degradation).

The intellectual arc is as follows. We begin by modelling natural language as a noisy Shannon channel with bounded capacity (Natural Language as a Shannon Channel). We then formalise the thought communication paradigm of Zheng et al. [60], which replaces natural-language messages with sparse latent vectors extracted by a regularised autoencoder (Thought Communication). Three theorems (shared-thought identifiability, private-thought identifiability, and uniqueness) establish when and how these latent representations can be recovered from observed behaviour; we provide complete proofs of all three (sec:agents:shared-id,sec:agents:private-id,sec:agents:thought-unique-proof). We then study the Optima framework of Chen et al. [20], which trains agents to produce maximally informative yet minimal messages via a Generate–Rank–Select–Train pipeline (The Optima Framework). Finally, we show that communication can be actively harmful: Wynn et al. [67] demonstrate that sycophantic agents cause debate quality to degrade rather than improve (When Communication Harms: Sycophancy in Multi-Agent Debate).

Natural Language as a Shannon Channel

Consider two agents $a_i$ and $a_j$ that must exchange information to solve a joint task. Agent $a_i$ has an internal representation ${𝒛}_i\in {\mathrm{ℝ}}^d$ encoding its beliefs about the task state; it must transmit relevant aspects of ${𝒛}_i$ to agent $a_j$. In standard multi-agent LLM systems, this transmission occurs through natural language: $a_i$ generates a text string $m_{i\to j}\in {\mathrm{\Sigma }}^{\ast }$ (where $\mathrm{\Sigma }$ is the token vocabulary), and $a_j$ conditions its subsequent reasoning on $m_{i\to j}$.

We model this process as a Shannon communication channel. Let ${𝒵}_S\subseteq {\mathrm{ℝ}}^d$ denote the space of task-relevant internal states. The encoding function ${\mathrm{Enc}}_i:{𝒵}_S\to {\mathrm{\Sigma }}^{\ast }$ maps the internal state to a natural-language message, and the decoding function ${\mathrm{Dec}}_j:{\mathrm{\Sigma }}^{\ast }\to {\widehat{𝒵}}_S$ maps the received message back into an internal representation. The end-to-end fidelity of communication is measured by the mutual information (MI COMM)$$\begin{array}{ccc}& I({𝒛}_i;{\widehat{𝒛}}_j)=H({𝒛}_i)-H({𝒛}_i|{\widehat{𝒛}}_j),& \text{<span class="tml-eqn"></span>}\end{array}$$ where ${\widehat{𝒛}}_j={\mathrm{Dec}}_j({\mathrm{Enc}}_i({𝒛}_i))$. The channel capacity of natural-language communication is (Channel CAP)$$\begin{array}{ccc}& C_{\mathrm{NL}}=\underset{{\mathrm{Enc}}_i,{\mathrm{Dec}}_j}{\max }I({𝒛}_i;{\widehat{𝒛}}_j)\le \log \left|\mathrm{\Sigma }\right|\cdot L_{\max },& \text{<span class="tml-eqn"></span>}\end{array}$$ where $L_{\max }$ is the maximum message length in tokens. This bound is almost never tight: natural language is highly redundant, and the effective capacity is determined by the information density of the message, not its raw token count.

The definition of communication efficiency makes precise the intuition that what matters is not the volume of messages but their marginal contribution to performance. We will see that the Optima framework (The Optima Framework) achieves $2.8\times$ the performance gain at less than $10\%$ of the token budget, dramatically increasing $\eta$.

Thought Communication

The core insight of Zheng et al. [60] is that natural-language communication is an unnecessarily lossy channel. When agent $a_i$ translates its internal representation ${𝒛}_i$ into a natural-language message $m$, it discards structural information (vector geometry, confidence magnitudes, relational structure between latent dimensions) that could be directly useful to the receiving agent. The thought communication paradigm proposes to transmit compressed latent vectors instead.

Latent extraction via sparsity-regularised autoencoder.

Each agent $a_i$ is equipped with an encoder–decoder pair: (Encoder Decoder)$$\begin{array}{ccc}& f_i:{\mathrm{ℝ}}^d\to {\mathrm{ℝ}}^k,g_i:{\mathrm{ℝ}}^k\to {\mathrm{ℝ}}^d,& \text{<span class="tml-eqn"></span>}\end{array}$$ where $d$ is the dimension of the agent's internal hidden state and $k\ll d$ is the dimension of the transmitted “thought vector.” The encoder $f_i$ compresses the full representation into a sparse code; the decoder $g_i$ reconstructs an approximation. Training minimises a reconstruction loss plus a sparsity penalty: (AE LOSS)$$\begin{array}{ccc}& {ℒ}_{\mathrm{AE}}(f_i,g_i)={𝔼}_{{𝒛}_i}[{\Vert g_i(f_i({𝒛}_i))-{𝒛}_i\Vert }^2+\lambda {\Vert f_i({𝒛}_i)\Vert }_1],& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\lambda >0$ controls sparsity. The ${\ell }_1$ penalty encourages most components of the thought vector ${𝒕}_i=f_i({𝒛}_i)$ to be exactly zero, yielding a representation in which only the most informative dimensions are active.

Generative model of agent observations.

To state the identifiability theorems precisely, we need a generative model for how agents' internal states arise. Following [60], suppose two agents $a_1,a_2$ observe a shared environment state $𝒔\in {\mathrm{ℝ}}^p$ and produce internal representations (GEN Z1)$$\begin{array}{cccc}& {𝒛}_1& =h_1(𝒔,{𝝐}_1),{𝝐}_1\sim p_1,& \text{<span class="tml-eqn"></span>}\\ & {𝒛}_2& =h_2(𝒔,{𝝐}_2),{𝝐}_2\sim p_2,& \text{<span class="tml-eqn"></span>}\end{array}$$ where $h_1,h_2$ are deterministic mixing functions and ${𝝐}_1,{𝝐}_2$ are independent noise vectors capturing each agent's private information (different prompts, different tools, different retrieval contexts). The latent state decomposes as (Latent Decompose)$$\begin{array}{ccc}& {𝒛}_i=[{𝒔}_{\mathrm{shared}};{𝒔}_{\mathrm{private},i}],& \text{<span class="tml-eqn"></span>}\end{array}$$ where ${𝒔}_{\mathrm{shared}}$ encodes information common to both agents (the shared environment) and ${𝒔}_{\mathrm{private},i}$ encodes agent-specific information. The central question is: can the encoder $f_i$ learn to separate shared from private information, and if so, is this decomposition unique?

We now state and prove three theorems that answer this question affirmatively under precise conditions.

Shared Thought Identifiability

The first theorem establishes that the shared component of agents' internal representations can be recovered up to an invertible transformation.

The theorem says that maximising inter-agent mutual information while discarding private information is sufficient to recover the shared latent state, exactly the objective optimised by the sparsity-regularised autoencoder of when the ${\ell }_1$ penalty is tuned to suppress private dimensions.

Private Thought Identifiability

The second theorem addresses the complementary question: can each agent's private information also be recovered?

Thought Uniqueness

The third theorem shows that the decomposition into shared and private components is essentially unique.

The Optima Framework

While thought communication operates at the representation level, the Optima framework of Chen et al. [20] optimises communication at the message level, remaining within natural language but training agents to produce maximally informative yet minimal messages. Optima decomposes communication training into four stages: Generate, Rank, Select, and Train.

Stage 1: Generate.

Given a multi-agent task instance $\tau$ and the current dialogue history ${\mathscr{H}}_t=(m_1,\dots ,m_{t-1})$, agent $a_i$ generates a set of $K$ candidate messages $\{m_i^{(1)},\dots ,m_i^{(K)}\}$ by sampling from its policy ${\pi }_i(\cdot |\tau ,{\mathscr{H}}_t)$ with temperature $T>0$. The diversity of candidates is crucial: it ensures that the subsequent ranking step has sufficient variation to identify the most informative message.

Stage 2: Rank.

Each candidate message $m_i^{(k)}$ is scored by a reward model $R_{\mathrm{comm}}$ that evaluates both informativeness and conciseness: (Optima Reward COMM)$$\begin{array}{ccc}& r(m_i^{(k)})=\underset{\text{task-relevant information}}{\underbrace{R_{\mathrm{task}}(\tau ,{\mathscr{H}}_t,m_i^{(k)})}}-\underset{\text{length penalty}}{\underbrace{\mu \cdot \left|m_i^{(k)}\right|}},& \text{<span class="tml-eqn"></span>}\end{array}$$ where $R_{\mathrm{task}}$ measures the downstream task improvement attributable to the message and $\mu >0$ is a length penalty coefficient. The reward model captures the insight that the best message is not the longest or the most detailed, but the one that maximises task performance per token.

Stage 3: Select.

The top-$N$ messages (ranked by $r$) are selected as positive examples: ${ℳ}^{+}=\{m_i^{(\sigma (1))},\dots ,m_i^{(\sigma (N))}\}$ where $\sigma$ is the permutation induced by decreasing reward. Optionally, the bottom-$N$ messages form a negative set ${ℳ}^{-}$ for contrastive training.

Stage 4: Train.

The agent's policy is updated using a preference optimisation objective. Let $m^{+},m^{-}\in {ℳ}^{+}\times {ℳ}^{-}$ be a positive–negative pair. The DPO-style loss is: (Optima DPO)$$\begin{array}{ccc}& {ℒ}_{\mathrm{Optima}}(𝜽)=-{𝔼}_{(m^{+},m^{-})}\left[\log \sigma \right(\beta \log \frac{{\pi }_{𝜽}(m^{+})}{{\pi }_{\mathrm{ref}}(m^{+})}-\beta \log \frac{{\pi }_{𝜽}(m^{-})}{{\pi }_{\mathrm{ref}}(m^{-})}\left)\right],& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\sigma (\cdot )$ is the sigmoid function, $\beta >0$ is the inverse temperature, and ${\pi }_{\mathrm{ref}}$ is the reference policy before training. This loss steers the agent toward generating messages that are simultaneously informative and concise.

When Communication Harms: Sycophancy in Multi-Agent Debate

The preceding subsections presented communication as uniformly beneficial, provided it is efficient. We now show that this is not always the case. Wynn et al. [67] identify a striking failure mode: when agents in a debate system exhibit sycophancy, the tendency to agree with the conversational partner rather than defend a correct position, the debate mechanism can decrease the accuracy of the final answer compared to a single agent working alone.

The sycophancy mechanism.

Consider a multi-agent debate with $n$ agents and $T$ rounds. At round $t$, agent $a_i$ observes the responses of all other agents from round $t-1$ and produces an updated response. A sycophantic agent updates its belief toward the majority opinion regardless of evidence. Formally, let $b_i^{(t)}\in [0,1]$ be agent $a_i$'s confidence in the correct answer at round $t$. A non-sycophantic agent updates via Bayesian reasoning: (Bayesian Update)$$\begin{array}{ccc}& b_i^{(t)}=\frac{b_i^{(t-1)}\cdot L_i^{(t)}}{b_i^{(t-1)}\cdot L_i^{(t)}+(1-b_i^{(t-1)})\cdot (1-L_i^{(t)})},& \text{<span class="tml-eqn"></span>}\end{array}$$ where $L_i^{(t)}$ is the likelihood ratio based on the evidence from other agents' arguments. A sycophantic agent instead uses a corrupted update: (SYCO Update)$$\begin{array}{ccc}& b_i^{(t)}=(1-\gamma ){\tilde{b}}_i^{(t)}+\gamma {\bar{b}}_{-i}^{(t-1)},& \text{<span class="tml-eqn"></span>}\end{array}$$ where ${\tilde{b}}_i^{(t)}$ is the Bayesian posterior, ${\bar{b}}_{-i}^{(t-1)}=\frac{1}{n-1}{\sum }_{j\ne i}{b}_j^{(t-1)}$ is the mean confidence of other agents, and $\gamma \in [0,1]$ is the sycophancy parameter. When $\gamma =0$, the agent is fully rational; when $\gamma =1$, it completely defers to the group.

Communication Complexity and Scaling

We close this section by connecting inter-agent communication to classical communication complexity. In a system of $n$ agents with pairwise communication, the total number of communication channels is $\left(\genfrac{}{}{0px}{}{n}{2}\right)=O(n^2)$. If each pair exchanges $B$ tokens per round over $T$ rounds, the total communication volume is (COMM Volume)$$\begin{array}{ccc}& V_{\mathrm{comm}}=\left(\genfrac{}{}{0px}{}{n}{2}\right)\cdot B\cdot T=O(n^{2}BT).& \text{<span class="tml-eqn"></span>}\end{array}$$ This quadratic scaling in $n$ is the communication analogue of the quadratic attention cost in transformers, and it motivates the same solution strategies: sparsification (not all pairs need to communicate), compression (reducing $B$ via thought communication), and hierarchical routing (organising agents into a tree where communication flows through designated coordinators).

The lemma tells us that communication costs cannot be reduced below a task-dependent threshold. The art of efficient multi-agent design is to approach this lower bound as closely as possible while maintaining coordination quality, precisely the goal of both thought communication and Optima.

Summary.; This section developed a formal framework for inter-agent communication. We modelled natural language as a bounded-capacity Shannon channel (Natural Language as a Shannon Channel), introduced thought communication as a latent-space alternative with provable identifiability guarantees (sec:agents:thought-comm,sec:agents:shared-id,sec:agents:private-id,sec:agents:thought-unique-proof), analysed the Optima pipeline for training efficient natural-language communicators (The Optima Framework), and characterised a failure mode in which sycophantic communication degrades debate quality (When Communication Harms: Sycophancy in Multi-Agent Debate). The overarching lesson is that communication should be designed with the same care as the agents themselves: the right $64$ dimensions can outperform $512$ tokens of prose.

Agent Architectures and Topologies

The preceding sections built up the single agent, its perception, reasoning, planning, and safety mechanisms, and then introduced the multi-agent system as a coordinated ensemble. But we have not yet addressed a fundamental design question: how should agents be connected? The communication topology of a multi-agent system, who can send messages to whom, and through what intermediaries, determines the system's latency, fault tolerance, message overhead, and, ultimately, its collective intelligence.

Consider an analogy. A military platoon, a corporate board, and a scientific peer-review system all consist of intelligent agents exchanging information. Yet they perform very differently on the same task, not because their members differ in raw ability, but because their topologies differ: the platoon uses a strict chain of command (a tree), the board uses open debate (a complete graph), and peer review uses a hub-and-spoke structure with the editor as the central node (a star). The topology shapes the flow of information, the speed of convergence, and the system's vulnerability to single points of failure.

In LLM-based multi-agent systems, the same principle applies with striking precision. Microsoft's AutoGen framework [8] introduced the Conversable Agent abstraction, in which any agent can converse with any other agent, and the conversation topology is a first-class design parameter. Zhao et al.'s LongAgent system [42] demonstrated that a star topology with a leader agent coordinating member agents could effectively process documents exceeding 128K tokens, far beyond any single model's context window, by decomposing the document across members and aggregating their responses through a conflict-resolution protocol.

This section develops the mathematical theory of agent topologies. We begin with a formal definition of agent topology as a directed weighted graph (Formal Definition of Agent Topology), then catalogue the four canonical topology patterns (star, chain, tree, and general graph) and derive their message complexity bounds (Canonical Topology Variants). We analyse AutoGen's Conversable Agent model and its dynamic group chat mechanism (AutoGen's Conversable Agent Architecture), and study LongAgent's star topology with its inter-member conflict resolution protocol (LongAgent: Star Topology for Extended Context). Finally, we examine how topology choice interacts with task structure (Topology Choice and Task Structure).

Formal Definition of Agent Topology

We formalise the communication structure of a multi-agent system as a weighted directed graph.

The adjacency matrix $𝑨$ encodes the full communication structure. When the topology is undirected (every edge is bidirectional, i.e., $(a_i,a_j)\in E\iff (a_j,a_i)\in E$ with equal weights), $𝑨$ is symmetric. When $w\equiv 1$, we recover the standard unweighted adjacency matrix. The reachability matrix ${𝑨}^{\ast }={\sum }_{k=0}^{n-1}{𝑨}^k$ (where ${𝑨}_{ij}^k>0$ indicates a path of length $k$ from $a_i$ to $a_j$) determines whether information can eventually flow between any pair of agents.

Canonical Topology Variants

Four topology patterns recur across virtually every multi-agent system in the literature. We define each precisely and characterise its structural properties.

These four patterns span a spectrum from minimal to maximal connectivity. The star and chain are the sparsest connected topologies ($|E|=\mathrm{\Theta }(n)$), the tree is slightly denser in terms of routing depth, and the fully connected graph is the densest ($|E|=\mathrm{\Theta }(n^2)$). Each offers different trade-offs between message complexity, latency, fault tolerance, and bottleneck risk.

We now quantify the most important trade-off: the total number of messages required for all agents to contribute to a collective decision.

The message complexity results are summarised in tab:agents:topology-comparison.

Property	Star	Chain	Balanced Tree	Complete Graph
Edges $|E|$	$2(n-1)$	$n-1$	$\mathrm{\Theta }(n)$	$n(n-1)$
Diameter	$2$	$n-1$	$2{\log }_{b}n$	$1$
Message complexity	$\mathrm{\Theta }(n)$	$\mathrm{\Theta }(n^2)$	$\mathrm{\Theta }(n\log n)$	$\mathrm{\Theta }(n^2)$
Bottleneck agent	Hub	Endpoints	Root	None
Fault tolerance	Low (hub)	Low (any)	Medium	High

AutoGen's Conversable Agent Architecture

AutoGen [8] introduced a unifying abstraction for multi-agent LLM systems: the Conversable Agent. Rather than hard-coding communication patterns, AutoGen treats each agent as an entity capable of sending and receiving natural-language messages to and from any other agent, with the conversation topology determined at runtime.

The Conversable Agent abstraction.

A Conversable Agent in AutoGen is an LLM agent (Definition 3) augmented with three capabilities:

1. Message generation: the agent produces a response given its conversation history, system prompt, and any received messages. This is the standard LLM inference step.

2. Message reception: the agent can receive messages from any agent in its in-neighbourhood ${𝒩}_{\mathrm{in}}(a_i)$ and append them to its local conversation history.

3. Conversation control: each agent carries a reply function $f_{\mathrm{reply}}:\mathscr{H}\to 𝒴\cup \{\text{𝚃𝙴𝚁𝙼𝙸𝙽𝙰𝚃𝙴}\}$ that maps the current history to either a response or a termination signal. This function can incorporate human-in-the-loop approval, code execution, or tool calls.

The key insight is that the same Conversable Agent abstraction supports wildly different topologies: a two-agent coding dialogue (star with $n=2$), a round-robin panel discussion (cycle graph), or a hierarchical delegation structure (tree).

Dynamic group chat.

AutoGen's GroupChat mechanism extends the two-agent conversation to $n$ agents. A GroupChatManager agent $a_{\mathrm{mgr}}$ acts as an orchestrator (Definition 25), selecting the next speaker at each turn. The speaker selection can be:

• Round-robin: agents speak in a fixed cyclic order $a_1\to a_2\to \cdots \to a_n\to a_1$.

• Random: the next speaker is sampled uniformly from the remaining agents.

• LLM-directed: the GroupChatManager uses its own LLM backbone to select the most appropriate next speaker based on the conversation history: (Group CHAT Selection)$$\begin{array}{ccc}& a_{\mathrm{next}}=\arg \underset{a_j\in V}{\max }{\mathrm{\Phi }}_{\mathrm{mgr}}(a_j|h_t,\text{“Who should speak next?”}),& \text{<span class="tml-eqn"></span>}\end{array}$$ where ${\mathrm{\Phi }}_{\mathrm{mgr}}$ is the manager's LLM and $h_t$ is the shared conversation history.

In the group chat topology, all agents share a common message board. The underlying graph is a star with the GroupChatManager as the hub, but the logical topology (who responds to whom) is dynamic and adapts to the conversation content.

Nested conversations.

AutoGen supports nested conversations: a single turn of an outer conversation can trigger an entire inner conversation among a subset of agents. Formally, let ${𝒢}_{\mathrm{outer}}$ be the outer topology and ${𝒢}_{\mathrm{inner}}^{(k)}$ be the inner topology triggered at turn $k$. The composite topology is a hierarchical graph: (Nested Topology)$$\begin{array}{ccc}& {𝒢}_{\mathrm{composite}}={𝒢}_{\mathrm{outer}}\cup \bigcup _k{𝒢}_{\mathrm{inner}}^{(k)},& \text{<span class="tml-eqn"></span>}\end{array}$$ where the inner conversations are subgraphs that are instantiated on demand and collapsed into a single message (the inner conversation's output) from the perspective of the outer conversation. This is analogous to macro-actions in hierarchical reinforcement learning: the outer topology sees atomic messages, while the inner topology resolves them through multi-step deliberation.

LongAgent: Star Topology for Extended Context

A compelling application of the star topology arises in long-context processing. LongAgent [42] addresses the challenge of processing documents that exceed any single model's context window (e.g., 128K tokens) by decomposing the document across multiple member agents and aggregating their responses through a leader agent.

Architecture.

The LongAgent system consists of $n+1$ agents arranged in a star topology:

• Leader ($a_L$): the hub agent responsible for decomposing the input query, dispatching sub-queries to members, collecting responses, and resolving conflicts.

• Members ($a_1,\dots ,a_n$): each member agent receives a disjoint chunk of the document and a sub-query from the leader. Member $i$ processes chunk $D_i$ and returns a local answer $y_i$.

The adjacency matrix is (Longagent Adjacency)$$\begin{array}{ccc}& A_{ij}=\{\begin{array}{cc}1& \text{if }i=L\text{ or }j=L,i\ne j,\\ 0& \text{otherwise},\end{array}& \text{<span class="tml-eqn"></span>}\end{array}$$ which is exactly the star topology ${𝒢}_{\star }$ with hub $a_L$. The document of total length $\ell$ tokens is partitioned into $n$ chunks of approximately $\ell /n$ tokens each, where $n=\lceil \ell /C\rceil$ and $C$ is the effective context window of each member agent.

Inter-member conflict resolution.

When different members return contradictory answers to the same query, the leader must resolve the conflict. LongAgent employs a structured protocol:

1. Collection: the leader collects all member responses $\{y_1,\dots ,y_n\}$.

2. Consistency check: the leader identifies conflicting answers. Two responses $y_i$ and $y_j$ are conflicting if they provide mutually exclusive answers to the query (detected via the leader's LLM).

3. Evidence request: for each conflicting pair $(y_i,y_j)$, the leader asks the corresponding members to provide textual evidence from their document chunks supporting their answers.

4. Adjudication: the leader evaluates the evidence and selects the best-supported answer, or synthesises a reconciled response.

This protocol adds at most $2K$ additional messages to the base star protocol (where $K$ is the number of conflicting pairs, $K\le \left(\genfrac{}{}{0px}{}{n}{2}\right)$), keeping the total message complexity at $O(n+K)$. In practice, $K\ll \left(\genfrac{}{}{0px}{}{n}{2}\right)$ because most sub-queries have unambiguous answers within their respective chunks.

Formal analysis of the LongAgent protocol.

Let $q$ be the user's query and $D=D_1\cup \cdots \cup D_n$ be the document partition. The LongAgent protocol proceeds in three phases:

1. Dispatch phase: the leader sends $(q,D_i)$ to each member $a_i$. Messages: $n$.

2. Response phase: each member returns its local answer $y_i={\pi }_{a_i}(\cdot |q,D_i)$. Messages: $n$.

3. Resolution phase: for each conflicting pair $(y_i,y_j)$, the leader requests evidence (2 messages) and receives it (2 messages). Total: $4K$ messages.

The total message count is $2n+4K$. Since each message carries at most $C$ tokens (the member's context window), the total communication volume is bounded by $(2n+4K)\cdot C$ tokens. For a document of length $\ell =nC$, the communication overhead ratio is (Longagent Overhead)$$\begin{array}{ccc}& \rho =\frac{(2n+4K)\cdot C}{\ell }=\frac{2n+4K}{n}=2+\frac{4K}{n},& \text{<span class="tml-eqn"></span>}\end{array}$$ which approaches $2$ as $K/n\to 0$. This means the total communication is approximately twice the document length, a remarkably low overhead for enabling arbitrarily long context.

Topology Choice and Task Structure

The preceding analysis raises a natural question: given a task, which topology should one choose? The answer depends on the decomposability and interdependence structure of the task.

The decomposability index directly informs topology selection:

• High decomposability ($\delta \approx 1$): sub-tasks are independent, so a star topology is optimal. The hub dispatches sub-tasks in parallel, collects results, and aggregates. Message complexity is $\mathrm{\Theta }(n)$. LongAgent operates in this regime.

• Low decomposability ($\delta \approx 0$): sub-tasks are tightly coupled, requiring extensive information exchange. A fully connected topology or dense graph is needed so that agents can share intermediate results freely. Message complexity is $\mathrm{\Theta }(n^2)$, but the alternative, routing through a sparse topology, would add latency without reducing the total information transfer.

• Hierarchical decomposability ($0<\delta <1$): the task admits a hierarchical decomposition where high-level sub-tasks are independent but each decomposes into tightly coupled sub-sub-tasks. A tree topology matches this structure naturally: the root dispatches to sub-tree leaders, who in turn coordinate their local teams. Message complexity is $\mathrm{\Theta }(n\log n)$.

• Sequential dependence: sub-tasks must be performed in a specific order, with each task's output feeding the next. A chain (pipeline) topology is the natural choice, providing minimal overhead per stage but $\mathrm{\Theta }(n)$ latency for end-to-end completion.

Dynamic Topology Adaptation

A fixed topology is a design-time commitment. But many real-world tasks have information-flow requirements that change as the task progresses: an initial brainstorming phase may benefit from dense connectivity, while a subsequent execution phase may favour a hierarchical structure. This motivates dynamic topologies that evolve during the task.

AutoGen's GroupChatManager implements a simple form of dynamic topology: at each turn, the manager selects a new speaker, effectively activating a different edge in the star. More sophisticated approaches allow the topology itself to be restructured. For example, an agent that discovers it needs information from a non-neighbour can request the orchestrator to add an edge, temporarily converting a sparse topology into a denser one for that specific exchange.

Adaptive topology via graph attention.

One principled approach to dynamic topology selection borrows from graph attention networks (GATs). Define attention weights between agents as (Graph Attention)$$\begin{array}{ccc}& {\alpha }_{ij}^{(t)}=\frac{\exp (\mathrm{score}(h_i^{(t)},h_j^{(t)}))}{{\sum }_{k\in V}\exp (\mathrm{score}(h_i^{(t)},h_k^{(t)}))},& \text{<span class="tml-eqn"></span>}\end{array}$$ where $h_i^{(t)}$ is the embedding of agent $i$'s state at round $t$ (e.g., a summary of its recent messages) and $\mathrm{score}(\cdot ,\cdot )$ is a learned compatibility function. The effective topology at round $t$ is then a soft graph with edge weights ${\alpha }_{ij}^{(t)}$: agents with high attention scores communicate with high bandwidth, while those with low scores effectively ignore each other. This converts the discrete topology selection problem into a continuous optimisation problem amenable to gradient-based methods.

From Topology to Performance

We conclude this section by connecting topology properties to system-level performance guarantees. The following result establishes that the spectral properties of the adjacency matrix $𝑨$ bound the convergence rate of opinion aggregation across agents.

This result has direct implications for topology selection. For the four canonical topologies:

• Complete graph: ${\mu }_2=-1/(n-1)$, so $|{\mu }_2|\to 0$ as $n\to \infty$. Consensus is achieved in a single step.

• Star: ${\mu }_2=0$ for the normalised adjacency of a star with uniform weights. Consensus is near-instant (2 iterations: collect, then broadcast).

• Chain: ${\mu }_2=\cos (\pi /n)\approx 1-{\pi }^2/(2n^2)$. The spectral gap is $\mathrm{\Theta }(1/n^2)$, so consensus requires $\mathrm{\Theta }(n^2)$ iterations, the slowest among connected topologies.

• Balanced binary tree: ${\mu }_2=O(1/\log n)$. Consensus requires $\mathrm{\Theta }({\log }^{2}n)$ iterations, much faster than the chain.

Scalable Agent Platforms

Building a single capable agent is an engineering challenge; running a thousand such agents concurrently, each consuming GPU inference, exchanging messages, and mutating shared state, is a distributed-systems challenge of an entirely different order. The previous sections developed the mathematical abstractions: agents as POMDP policies, communication as message-passing on graphs, and orchestration as scheduling over DAGs. This section grounds those abstractions in platforms: software systems that instantiate, route, monitor, and recover agents at scale.

We organise the exposition around three complementary paradigms. First, the actor-based approach (Actor-Based Agent Distribution), exemplified by AgentScope [10], maps each agent to a lightweight actor process with its own message queue, enabling linear horizontal scaling to over one million concurrent agents [68]. Second, the decentralised capability ledger (Decentralised Capability Ledger), introduced by Symphony [69], eliminates the single-point-of-failure orchestrator by distributing the agent registry across a peer-to-peer network and selecting task coordinators through a Beacon protocol. Third, dynamic agent generation (Dynamic Agent Generation), as in AutoAgents [70], removes the assumption that agents are predefined: an LLM planner drafts new agents on the fly to match task requirements.

Actor-Based Agent Distribution

The actor model provides a natural abstraction for multi-agent systems. Each agent runs as an independent actor with its own message queue, local state, and failure domain. We formalise this mapping below.

The sequential-processing guarantee is crucial: it eliminates the need for locks or atomic operations within a single agent, vastly simplifying reasoning about correctness. Parallelism comes from running many actors simultaneously, each on its own CPU core or GPU slice.

DAG-based workflow orchestration.

AgentScope [10] introduces a workflow engine that represents multi-agent collaborations as directed acyclic graphs (DAGs). Each node $v$ in the DAG corresponds to an agent invocation, and each directed edge $(v_i,v_j)$ indicates that $a_j$ requires the output of $a_i$ before it can execute. Formally, a workflow DAG is a pair $G=(V,E)$ where $V\subseteq A$ is the set of participating agents and $E\subseteq V\times V$ imposes a partial order. The platform scheduler topologically sorts $G$ and dispatches agents at each level in parallel: (DAG Levels)$$\begin{array}{ccc}& L_k=\{v\in V:\underset{(u,v)\in E}{\max }\ell (u)=k-1\},\ell (v)=\{\begin{array}{cc}0& \text{if }v\text{ has no predecessors},\\ 1+{\max }_{(u,v)\in E}\ell (u)& \text{otherwise},\end{array}& \text{<span class="tml-eqn"></span>}\end{array}$$ where $L_k$ denotes the set of agents at level $k$. All agents in $L_k$ can execute simultaneously once every agent in $L_{k-1}$ has completed. The makespan of the workflow is the number of levels: (DAG Makespan)$$\begin{array}{ccc}& T_{\mathrm{makespan}}=\underset{v\in V}{\max }\ell (v)+1.& \text{<span class="tml-eqn"></span>}\end{array}$$

Four-tier fault tolerance.

Production deployment demands graceful degradation. AgentScope implements a four-tier fault tolerance strategy:

1. Retry with exponential backoff: transient failures (API timeouts, rate limits) trigger automatic retries with delay $d_k=d_0\cdot 2^k$ at attempt $k$, up to a maximum of $k_{\max }$ retries;

2. Output validation: each agent's output is parsed and checked against a schema before being placed on the downstream queue; malformed outputs trigger a re-prompt with the validation error appended to the context;

3. Redundant execution: safety-critical agents run $r$ replicas in parallel, and the platform selects the final output by majority vote, requiring $r\ge 2f+1$ for $f$ crash faults (or $r\ge 3f+1$ for Byzantine faults, per thm:agents:byzantine);

4. Checkpoint and rollback: the platform periodically serialises the state of every agent (context window, memory bank, pending queue) to persistent storage; upon detecting a failure cascade, the supervisor rolls the system back to the most recent consistent checkpoint.

Together, these four tiers handle failures at increasing levels of severity: tier (a) absorbs transient network glitches, tier (b) catches LLM hallucinations and format errors, tier (c) tolerates full agent crashes, and tier (d) recovers from correlated or cascading failures that overwhelm the lower tiers.

Scaling to one million agents.

Pan et al. [68] demonstrate that the actor-based architecture of AgentScope scales linearly with the number of machines. The key enablers are:

1. Stateless routing: each message carries the target agent's identifier; any router node can forward the message without maintaining session state;

2. Horizontal partitioning: agents are sharded across machines by hashing their identifiers, distributing load evenly;

3. Lazy instantiation: agent actors are created only when their first message arrives, avoiding the cost of pre-allocating one million processes at startup;

4. Compressed serialisation: agent state is serialised using a compact binary format that reduces checkpoint sizes by an order of magnitude compared to JSON.

Let $n$ denote the number of agents and $M$ the number of machines. Under uniform load (each machine hosts $n/M$ agents) and assuming each agent-to-agent message traverses at most one network hop, the total communication cost per round is: (COMM COST)$$\begin{array}{ccc}& C_{\mathrm{comm}}(n,M)=n\cdot \bar{d}\cdot (c_{\mathrm{local}}\frac{n/M-1}{n-1}+c_{\mathrm{remote}}\frac{n-n/M}{n-1}),& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\bar{d}$ is the average out-degree per agent, $c_{\mathrm{local}}$ is the cost of a local (intra-machine) message, and $c_{\mathrm{remote}}$ is the cost of a remote (inter-machine) message. For large $n$ with fixed $M$, the fraction of remote messages approaches $1-1/M$, so $C_{\mathrm{comm}}\approx n\bar{d}{c}_{\mathrm{remote}}(1-1/M)$, linear in $n$.

Decentralised Capability Ledger

The actor-based approach of the previous subsection still assumes a centralised component: the supervisor $\mathrm{\Psi }$ that monitors agents and the router that dispatches messages. In large-scale or adversarial settings, this centralised component becomes a bottleneck and a single point of failure. Symphony [69] proposes a fully decentralised alternative.

The gossip protocol converges exponentially fast: if each agent contacts $k$ random peers per round, then after $O(\log n/\log k)$ rounds every agent has received every update with high probability. This convergence rate ensures that newly registered capabilities propagate through the network in $O(\log n)$ time, a dramatic improvement over the $O(n)$ broadcast time of a sequential protocol.

The Beacon selection protocol.

When a complex task arrives at the system, some agent must coordinate the response, assigning sub-tasks to specialists, collecting results, and synthesising the final answer. In a centralised system, this role is played by the orchestrator. In Symphony's decentralised design, the coordinator is selected dynamically through the Beacon selection protocol.

Consistency guarantees.

The gossip-based ledger provides eventual consistency: after sufficiently many rounds, all agents converge to the same view. More precisely, let ${ℒ}_i^{(t)}$ denote agent $a_i$'s local ledger at round $t$. If a new capability entry is inserted at round $t_0$, then for all agents $a_i$: (Gossip Convergence)$$\begin{array}{ccc}& \mathrm{Pr}\left[{ℒ}_i^{(t_0+r)}\text{ contains the entry}\right]\ge 1-(1-\frac{k}{n}{)}^r,& \text{<span class="tml-eqn"></span>}\end{array}$$ where $k$ is the number of peers contacted per round. Setting $r=\lceil \log n/\log (n/(n-k))\rceil$ yields a probability of at least $1-1/n$, confirming the $O(\log n)$ convergence claim.

Dynamic Agent Generation

The preceding subsections assumed a fixed pool of agents. AutoAgents [70] lifts this assumption by generating agents dynamically: given a task description, an LLM planner proposes agent specifications (role, system prompt, tools) tailored to the task at hand.

The drafting-execution pipeline.

AutoAgents [70] implements the dynamic generation process as a two-stage pipeline. In the drafting stage, the planner $\mathrm{\Pi }$ receives the task $\tau$ and generates the team blueprint $ℬ$ through a chain-of-thought process:

1. Task decomposition: $\mathrm{\Pi }$ breaks $\tau$ into sub-tasks $\{{\tau }_1,\dots ,{\tau }_m\}$;

2. Role inference: for each sub-task ${\tau }_j$, $\mathrm{\Pi }$ infers the necessary agent role $r_j$, specifying capabilities and constraints in natural language;

3. Tool assignment: $\mathrm{\Pi }$ selects tools $T_j$ from the available tool library $𝒯$ that match each role's requirements;

4. Dependency analysis: $\mathrm{\Pi }$ identifies data dependencies among sub-tasks and constructs a workflow DAG ((DAG Levels)).

In the execution stage, the platform instantiates the agents specified by $ℬ$, wires them according to the DAG, and runs the workflow. The verification module $𝒱$ scores the output and, if quality is insufficient, feeds the score and error traces back to $\mathrm{\Pi }$, which revises the blueprint. This closed loop typically converges in two to three iterations.

Communication Overhead and Scaling Limits

The preceding subsections paint an optimistic picture of horizontal scaling. We now inject a note of realism. Every message exchanged between agents carries a cost, in latency, bandwidth, and the cognitive load of processing incoming information. As the number of agents grows, this cost can dominate the computation itself.

Comparison of platform paradigms.

tab:agents:platforms summarises the three platform paradigms discussed in this section along their key design dimensions.

lcccc@ Platform	Topology	Coordination	Fault Tolerance	Agent Pool
AgentScope	DAG / tree	Centralised	4-tier (crash + Byzantine)	Fixed
Symphony	Gossip / P2P	Decentralised	Beacon re-election	Fixed
AutoAgents	DAG (generated)	Planner-driven	Retry + re-draft	Dynamic

Hybrid architectures.

In practice, production systems often combine elements of all three paradigms. A typical deployment might use AgentScope's actor infrastructure for process management, Symphony's ledger for capability discovery, and AutoAgents' drafting pipeline for creating specialised agents on demand. The mathematical framework of this section, actor-agent systems (Definition 22), capability ledgers (Definition 23), and dynamic generation (Definition 24), provides the formal vocabulary for reasoning about such hybrid designs.

Orchestration Patterns

The preceding sections built the vocabulary of multi-agent systems: agents, communication topologies, debate protocols, and collaborative generation strategies. But vocabulary alone does not produce coherent prose. The orchestrator, the entity that decides which agent acts next, what information it receives, and how its output is integrated into the collective, is the conductor of the multi-agent orchestra. A poor conductor can squander the talents of virtuoso musicians; a great one can make an ensemble of competent players produce transcendent results.

This section formalises orchestration as a first-class design abstraction. We begin with the formal definition of an orchestrator (The Orchestrator), then develop the dominant Orchestrator-Worker pattern that has produced the largest empirical gains in multi-agent coding and research systems (The Orchestrator-Worker Pattern). We introduce context engineering, the systematic management of information flow to and from workers, as the key lever for orchestrator effectiveness (Context Engineering). We analyse the economics of orchestration through the lens of token usage (Token Economics of Orchestration), and conclude with the MALLM configuration space, a combinatorial framework that reveals the staggering design freedom available to orchestrator builders (The MALLM Configuration Space).

The Orchestrator

We begin with the formal definition that is referenced throughout this chapter.

The critical design choice embedded in Definition 25 is that the orchestrator controls both agent selection and information routing. The instruction $\iota$ need not contain the full history ${\mathscr{H}}_t$; the orchestrator may summarise, filter, or restructure the history before passing it to the selected agent. This context curation is, as we shall see, the primary lever for orchestrator effectiveness.

The Orchestrator-Worker Pattern

The most successful orchestration pattern in practice is deceptively simple: a single orchestrator agent decomposes a task into sub-tasks, delegates each sub-task to a specialised worker agent, collects and synthesises the results, and iterates until the task is complete. We call this the Orchestrator-Worker pattern.

Empirical evidence.

The Orchestrator-Worker pattern has demonstrated striking gains across diverse domains. Anthropic's multi-agent research system [22] deployed a lead agent (the orchestrator) that coordinated multiple “sub-agent” workers for parallel web research, achieving a 90.2% improvement in research quality over a single-agent baseline on internal evaluation benchmarks. The key was not merely parallelism (running the same agent three times in parallel yields only modest improvements through majority voting) but specialised decomposition: the orchestrator assigned different sub-topics to different workers, each of which could focus its full context window on a narrow sub-problem.

Formally, the Orchestrator-Worker pattern instantiates Definition 25 with a specific structure. The orchestrator $\mathrm{\Omega }$ maintains an internal task plan $𝒫=\{({\iota }_1,a_{j_1}),\dots ,({\iota }_K,a_{j_K})\}$ that pairs instructions with assigned workers. At each step, the orchestrator:

1. Analyses the current state: examines completed results, identifies remaining sub-tasks, and detects failures or quality issues.

2. Selects the next worker $a^{\ast }$ and crafts the instruction $\iota$ with precisely the context that worker needs, no more, no less.

3. Dispatches the instruction to the worker and awaits the result.

4. Integrates the worker's output into the running solution, updating the task plan as needed.

5. Evaluates whether the task is complete ($d=1$) or whether further iterations are needed ($d=0$).

This loop is illustrated in fig:agents:orch-worker. The orchestrator's effectiveness depends critically on the quality of the instructions ${\iota }_k$ it generates, a problem we formalise next under the name context engineering.

Context Engineering

The term context engineering was crystallised by OpenAI's engineering team in their description of the “harness” that surrounds and coordinates agent systems [24]. The core insight is that the primary bottleneck in multi-agent systems is not the capability of individual agents but the quality of the information they receive. A brilliant coder given a vague specification will produce mediocre code; the same coder given a precise specification, relevant examples, and clear constraints will produce excellent code. The orchestrator's job is to be the brilliant specification writer.

Entropy management.

OpenAI's characterisation of the harness as managing “entropy” in the information flow [24] can be formalised as follows. Let $H(\iota )$ denote the Shannon entropy of the engineered context viewed as a random variable over possible instructions. A poorly designed context function produces high-entropy instructions (vague, ambiguous, containing irrelevant information), forcing the agent to resolve uncertainty internally, consuming reasoning tokens on disambiguation rather than problem-solving. A well-designed context function produces low-entropy instructions (precise, unambiguous, focused), allowing the agent to allocate its full reasoning budget to the sub-task.

Formally, let $Y=f_{a^{\ast }}(\iota )$ be the output of agent $a^{\ast }$ given context $\iota$, and let $Q$ denote the quality metric. We seek to maximise $𝔼[Q(Y)]$. By the data-processing inequality, (DATA Processing)$$\begin{array}{ccc}& I(Q;\iota )\ge I(Q;Y)\ge I(Q;\hat{Y}),& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\hat{Y}$ is any post-processing of $Y$. The first inequality tells us that the mutual information between context quality and the final output is bounded by the information in the context itself, underscoring that no amount of agent capability can compensate for information lost during context engineering. The orchestrator should therefore maximise $I(Q;\iota )$ by ensuring that task-relevant information is preserved in the engineered context.

Token Economics of Orchestration

Orchestration is not free. Every message the orchestrator sends, every context it engineers, every synthesis step it performs consumes tokens. Understanding the economics of token usage is essential for designing efficient multi-agent systems.

Kim et al. [19] conducted a systematic study of multi-agent LLM teams and arrived at a striking finding: token usage explains approximately 80% of the variance in task performance. Systems that used more tokens, by generating longer reasoning chains, producing more detailed sub-task instructions, or performing additional verification rounds, consistently outperformed systems that were more parsimonious, after controlling for model capability.

The MALLM Configuration Space

How many meaningfully different multi-agent systems can one build? The MALLM (Multi-Agent Large Language Model) framework of Becker et al. [23] answers this question by identifying four orthogonal design axes, each with a discrete set of options. The cross-product of these axes defines a configuration space whose size reveals the combinatorial richness of orchestration design.

Navigating the configuration space.

The 144 configurations in Definition 27 may seem modest, but they represent only the coarse-grained design choices. Fine-grained decisions, such as the specific wording of persona prompts, the temperature and top-$p$ settings for each generator, the number of rounds in a debate paradigm, the prompt given to the LLM judge, each add further degrees of freedom. The effective design space is therefore far larger than 144, and manual exploration is infeasible.

Becker et al. [23] addressed this by running all 144 configurations on a common benchmark suite, enabling a factorial analysis that decomposes performance variance into contributions from each axis and their interactions. Key findings include:

• Persona $\times$ Paradigm interaction dominates. The choice of persona has the largest main effect, but its impact depends strongly on the interaction paradigm: a “devil's advocate” persona is highly effective in sequential debate but adds noise in parallel brainstorming.

• Generator matters less than expected. Swapping the underlying LLM from GPT-4 to Claude or Llama 3 changed performance by less than the effect of the best versus worst persona assignment, suggesting that how agents are orchestrated matters more than which model powers them.

• Decision mechanism interacts with task type. Majority voting excels on tasks with verifiable answers (e.g., coding, mathematics), while LLM-as-judge is superior for open-ended generation (e.g., creative writing, research synthesis).

These findings reinforce a central theme of this chapter: the protocol, how agents are organised and coordinated, often matters more than the raw capabilities of the individual agents.

Orchestration as the Key Differentiator

Formal Comparison of Orchestration Patterns

We can now compare the orchestration patterns encountered in this chapter through the lens of the MALLM configuration space and the token-performance model.

lcccl@ Pattern	Control	Parallelism	Token cost	Best for
Round-robin	Decentralised	None	Low	Simple debates
Star (hub-and-spoke)	Centralised	None	Medium	Q&A routing
Orchestrator-Worker	Centralised	High	High	Complex tasks
Hierarchical	Multi-level	Medium	High	Large codebases
Swarm (peer-to-peer)	Decentralised	High	Variable	Exploration

tab:agents:orch-compare summarises five common patterns. The choice among them depends on the task structure:

• Decomposable tasks with independent sub-problems favour the Orchestrator-Worker pattern, which can assign sub-tasks in parallel and aggregate results.

• Sequential reasoning tasks (e.g., mathematical proofs, multi-step plans) favour round-robin or sequential debate, where each agent builds on the previous agent's output.

• Large-scale systems with hierarchical structure (e.g., a codebase with modules, packages, and files) favour hierarchical orchestration, where top-level orchestrators delegate to sub-orchestrators.

• Open-ended exploration tasks (e.g., brainstorming, creative generation) favour swarm patterns where agents explore diverse directions without central coordination.

In each case, the orchestrator's design determines the efficiency frontier: the maximum performance achievable for a given token budget. The token–performance model of Proposition 13 provides a quantitative framework for comparing patterns: given two orchestration strategies ${\mathrm{\Omega }}_1$ and ${\mathrm{\Omega }}_2$, the preferred strategy is the one achieving higher performance $P(𝒮)$ at equal token cost, or equivalently, the one achieving equal performance at lower cost.

Toward Game-Theoretic Foundations

The orchestration patterns developed in this section assume a cooperative setting: all agents share a common objective, and the orchestrator has full authority to assign tasks and evaluate outputs. But many real-world multi-agent deployments involve agents with heterogeneous objectives: LLMs fine-tuned by different organisations, agents representing different stakeholders, or adversarial agents injected to test system robustness.

When agent objectives diverge, the orchestrator can no longer assume that workers will faithfully execute instructions. A worker might “defect” by producing minimal-effort outputs to conserve its own token budget, or by subtly steering the conversation toward its own objective. The orchestrator must then design incentive-compatible mechanisms: protocols where each agent's self-interested behaviour aligns with the system's collective objective.

This is precisely the domain of game theory and mechanism design. In the next sections, we develop the game-theoretic foundations needed to analyse these strategic interactions: normal-form games and Nash equilibria, repeated games and the folk theorem (which shows how cooperation can emerge from repeated interaction even among self-interested agents), mechanism design (which shows how to construct protocols that make truthful behaviour optimal), and the Shapley value (which provides a principled method for assigning credit to agents based on their marginal contributions). The orchestrator's role evolves from a benevolent coordinator in the cooperative setting to a mechanism designer in the strategic setting, a transition that captures much of the mathematical richness of multi-agent AI.

Game Theory Foundations for Multi-Agent Systems

When multiple autonomous agents share an environment, the outcome experienced by any single agent depends not only on its own actions but also on the actions of every other agent. This fundamental interdependence is precisely the setting studied by game theory, the mathematical framework for analysing strategic interactions among rational decision-makers. In this section we develop the game-theoretic foundations needed to understand multi-agent LLM systems: normal-form games and Nash equilibria (Normal-Form Games), repeated games and the folk theorem (Repeated Games and Folk Theorems), mechanism design (Mechanism Design), and cooperative game theory with the Shapley value (Cooperative Game Theory and the Shapley Value).

The results in this section are classical, but their application to LLM-based agents is novel: each “player” is a language model instance with a policy ${\pi }_i(\cdot |x)$, and the “actions” are the responses the model generates. The payoffs arise from task rewards, human evaluations, or other agents' reactions.

Normal-Form Games

The simplest and most widely used model of strategic interaction is the normal-form game, also called a strategic-form game.

The central solution concept for non-cooperative games is the Nash equilibrium: a strategy profile from which no player can unilaterally improve their payoff.

The most fundamental existence result in game theory guarantees that every finite game possesses at least one Nash equilibrium in mixed strategies.

Repeated Games and Folk Theorems

In many multi-agent LLM deployments, agents interact not once but repeatedly over multiple rounds; for instance, a coding assistant and a reviewer exchanging feedback over several iterations, or debate agents engaging in multiple rounds of argumentation. The theory of repeated games shows that such sustained interaction fundamentally changes the strategic landscape: cooperation can emerge as an equilibrium even in games where it is impossible in a single round.

Consider a stage game $\mathrm{\Gamma }=(N,(A_i),(u_i))$ played infinitely often. In each period $t=0,1,2,\dots$, the players simultaneously choose actions $a_i^t\in A_i$, observe the joint action profile $(a_1^t,\dots ,a_n^t)$, and receive stage payoffs $u_i(a_1^t,\dots ,a_n^t)$. Each player $i$ maximises the discounted sum (Discounted Payoff)$$\begin{array}{ccc}& V_i=(1-\delta )\sum _{t=0}^{\infty }{\delta }^t{u}_i(a_1^t,\dots ,a_n^t),& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\delta \in [0,1)$ is the discount factor. The normalisation by $(1-\delta )$ ensures that $V_i$ is a weighted average of stage payoffs, making it comparable in scale to the single-stage payoffs $u_i$.

A strategy in the repeated game is a function mapping the entire history of play to a current action. The minimax payoff of player $i$ is (Minimax)$$\begin{array}{ccc}& {\underline{v}}_i=\underset{{\sigma }_{-i}}{\min }\underset{{\sigma }_i}{\max }{U}_i({\sigma }_i,{\sigma }_{-i}),& \text{<span class="tml-eqn"></span>}\end{array}$$ the lowest payoff that the other players can force upon $i$.

A payoff vector $𝒗=(v_1,\dots ,v_n)$ is feasible if it lies in the convex hull of the set of achievable payoff vectors, i.e., $𝒗\in \mathrm{conv}\{(u_1(𝒂),\dots ,u_n(𝒂)):𝒂\in A\}$. A feasible payoff vector is individually rational if $v_i\ge {\underline{v}}_i$ for all $i\in N$.

Mechanism Design

While game theory analyses the behaviour of agents within a fixed set of rules, mechanism design tackles the inverse problem: how should we design the rules of interaction so that self-interested agents produce a desired outcome? This is directly relevant to multi-agent LLM systems, where the system designer chooses the communication protocol, reward structure, and coordination rules.

The most celebrated incentive-compatible mechanism is the Vickrey–Clarke–Groves (VCG) mechanism.

Cooperative Game Theory and the Shapley Value

In many multi-agent settings, agents form coalitions and the relevant question is not “what will each agent do?” but “how much does each agent contribute?” Cooperative game theory provides the mathematical framework for this analysis.

The Shapley value is not merely one of many possible allocation rules; it is the unique rule satisfying four natural fairness axioms.

Consensus and Coordination

A recurring motif in multi-agent systems is the need for agents to agree: on an answer, a plan, a diagnosis, or a value. When agents begin with different opinions, how do they converge? How fast? And does the converged opinion bear any relation to the truth?

This section develops the mathematical theory of consensus. We begin with graph-Laplacian consensus (Graph Laplacian Consensus), where agents iteratively average their neighbours' values and provably converge to the global mean. We then apply consensus theory to agent opinions (Consensus in Agent Opinions), connecting to recent empirical findings that LLM agents spontaneously adopt averaging protocols. The Condorcet jury theorem (Condorcet Jury Theorem) quantifies when majority voting among agents outperforms any individual, while the analysis of quality versus diversity (Quality vs. Diversity) reveals the surprising empirical finding that agent quality dominates agent diversity.

Graph Laplacian Consensus

Consider $n$ agents arranged on a communication graph $G=(V,E)$ with $V=\{1,\dots ,n\}$. Each agent $i$ holds a scalar state $x_i(t)\in \mathrm{ℝ}$ at time $t$. Agents can communicate only with their neighbours in $G$.

The continuous-time consensus protocol asks each agent to move towards the average of its neighbours: (Continuous Consensus)$$\begin{array}{ccc}& \dot{𝒙}(t)=-L𝒙(t),& \text{<span class="tml-eqn"></span>}\end{array}$$ which in component form reads ${\dot{x}}_i(t)={\sum }_{j\in 𝒩(i)}(x_j(t)-x_i(t))$, where $𝒩(i)$ is the neighbourhood of agent $i$.

Consensus in Agent Opinions

In multi-agent LLM systems, each agent holds not a scalar value but a distribution over possible answers. Let ${𝒑}_i\in \mathrm{\Delta }(𝒴)$ be agent $i$'s opinion vector, where $𝒴=\{y_1,\dots ,y_K\}$ is a discrete set of candidate answers. A natural consensus protocol is (Opinion Consensus)$$\begin{array}{ccc}& {𝒑}_i^{(t+1)}=\sum _{j=1}^n{W}_{ij}{𝒑}_j^{(t)},& \text{<span class="tml-eqn"></span>}\end{array}$$ where $W$ is a doubly stochastic weight matrix respecting the communication graph. This protocol converges to ${𝒑}_i^{(\infty )}=\overline{𝒑}=\frac{1}{n}{\sum }_j{𝒑}_j^{(0)}$ for all $i$, exactly as in the scalar case.

Chen et al. [71] observed that LLM agents engaged in multi-round deliberation spontaneously adopt protocols resembling : after exchanging messages, agents update their answers in a direction that averages the positions of their conversational partners. The Fiedler eigenvalue ${\lambda }_2$ of the communication graph determines the convergence speed: sparse networks (small ${\lambda }_2$) converge slowly, while dense networks (large ${\lambda }_2$) converge quickly but may suppress diversity prematurely.

Condorcet Jury Theorem

A fundamental question in multi-agent systems is: when does aggregating the outputs of multiple agents improve upon any individual? The oldest and most elegant answer comes from the Marquis de Condorcet (1785).

This result has been invoked under the slogan “More Agents Is All You Need” [30]: simply sampling multiple responses from an LLM and taking a majority vote can substantially boost accuracy. However, the theorem's independence assumption is critical and often violated in practice.

Quality vs. Diversity

The Condorcet theorem assumes independent agents, but LLM agents trained on similar data exhibit strong correlations in their errors. This subsection analyses how correlation degrades the Condorcet guarantee and examines whether diversity or quality matters more.

Cooperation and Competition

The game-theoretic framework of Game Theory Foundations for Multi-Agent Systems predicts equilibrium behaviour under the assumption of full rationality. But LLM agents are not perfectly rational: they are shaped by pretraining data, in-context examples, and prompt instructions, all of which introduce biases that can push behaviour away from Nash equilibrium. This section examines how cooperation and competition emerge in practice when LLM agents interact, revealing surprising phenomena that classical game theory does not predict.

Spontaneous Cooperation in LLM Agents

One of the most striking findings in multi-agent LLM research is that agents placed in competitive game-theoretic settings often spontaneously cooperate [31], even when the one-shot Nash equilibrium prescribes defection.

Keynesian beauty contest.

In the Keynesian beauty contest, $n$ players simultaneously choose numbers $a_i\in [0,100]$. The winner is the player closest to $\frac{2}{3}$ of the average choice. Iterated elimination of dominated strategies yields the unique Nash equilibrium $a_i^{\ast }=0$ for all $i$, since if all players choose $\bar{a}$, the optimal target is $\frac{2}{3}\bar{a}$, and iterating this contraction drives the equilibrium to zero. Empirically, LLM agents converge to this equilibrium within $3$–$5$ rounds of repeated play, demonstrating both the capacity for strategic reasoning and rapid learning from interaction [31].

Bertrand competition.

In the Bertrand duopoly, two firms simultaneously set prices $p_1,p_2\in [0,\bar{p}]$ for identical products; consumers buy from the cheaper firm. The unique Nash equilibrium is $p_1^{\ast }=p_2^{\ast }=c$ (marginal cost pricing, yielding zero profit). However, the cartel (collusive) outcome is $p_1=p_2=p^m>c$ (monopoly pricing), which maximises joint profit.

Co-Player Inference and Predictive Policy Improvement

Beyond shared priors, LLM agents can perform in-context co-player inference: from observing an opponent's actions, the agent updates its belief about the opponent's strategy and adapts accordingly.

The PPI framework reveals that cooperation can emerge through a diversity training mechanism: when agents are trained against a diverse population of opponents, they learn policies that are robust to variation in opponent behaviour. This robustness, combined with co-player inference, leads to mutual adaptation and ultimately cooperation [72]. The process can be formalised as convergence to a correlated equilibrium of the underlying game, which generically yields higher social welfare than Nash equilibrium.

Co-Evolution: The Red Queen Hypothesis

When agents in a multi-agent system simultaneously learn, the environment from each agent's perspective is non-stationary: every agent's reward landscape shifts as other agents update their policies. This gives rise to co-evolutionary dynamics: an arms race reminiscent of the Red Queen hypothesis in evolutionary biology.

Distributed Problem Solving

While the preceding subsections demonstrate that LLM agents can cooperate in stylised game-theoretic settings, their performance on distributed combinatorial problems remains limited. Guo et al. [73] evaluated LLM agents on classic distributed computing tasks (graph colouring, vertex cover, leader election, and consensus) and found that:

1. Performance is reasonable at small scale ($n\le 10$ agents) but degrades sharply as $n$ increases, with near-zero success rates at $n=100$ for most problems.

2. Three primary failure modes emerge: enumerate[(i)]

3. Strategy miscoordination: agents adopt incompatible local strategies that cannot be reconciled globally;

4. Uncritical trust: agents accept neighbours' proposals without verifying consistency, propagating errors;

5. Poor conflict resolution: when two agents disagree, neither has a principled mechanism for yielding, leading to oscillation or deadlock. enumerate

These failure modes are rooted in the architecture of LLM agents: they lack global state awareness (each agent sees only its neighbourhood), they have limited working memory for tracking negotiation history, and they lack formal verification mechanisms for checking global consistency. Addressing these limitations requires combining LLM agents with formal coordination protocols (see Scheduling, Load Balancing, and Fault Tolerance).

Credit Assignment in Multi-Agent Systems

When $n$ agents collaborate on a task and achieve a joint outcome, a fundamental question arises: how much did each agent contribute? This is the multi-agent credit assignment problem, and its solution is essential for training, rewarding, and improving individual agents within a team. Without proper credit assignment, agents may free-ride on others' contributions or be penalised for failures caused by their teammates.

This section develops the theory and algorithms for multi-agent credit assignment. We formalise the problem using counterfactual reasoning (The Multi-Agent Credit Assignment Problem), connect it to the Shapley value from Cooperative Game Theory and the Shapley Value and develop efficient approximation algorithms (Shapley Value Computation), introduce difference rewards as a practical alternative (Difference Rewards), and derive multi-agent policy gradient theorems that incorporate credit assignment (Credit Assignment for Agent Training).

The Multi-Agent Credit Assignment Problem

The counterfactual approach ${\mathrm{\Delta }}_i$ is intuitive but flawed: it does not account for the order in which agents are removed, nor does it handle synergies between agents. The Shapley value (Definition 33) resolves both issues by averaging the marginal contribution over all possible orderings, yielding a unique, fair allocation that satisfies efficiency (${\sum }_i{\varphi }_i=v(N)$).

Shapley Value Computation

The exact Shapley value requires summing over all $2^n$ subsets of $N$, which is computationally intractable for large $n$. The standard approach is Monte Carlo approximation: sample random permutations and average the marginal contributions.

Difference Rewards

An alternative to the Shapley value that avoids summing over exponentially many subsets is the difference reward, which uses a single counterfactual evaluation.

Credit Assignment for Agent Training

Credit assignment becomes most important when we wish to train individual agents within a multi-agent system. The multi-agent policy gradient theorem provides the foundation for this.

Scheduling, Load Balancing, and Fault Tolerance

A multi-agent system is only as effective as its ability to allocate tasks efficiently, balance computational loads, and recover gracefully from failures. These operational concerns are often overlooked in discussions of multi-agent intelligence, yet they determine whether a theoretically powerful multi-agent design can function in practice.

This section develops the mathematical foundations of three interconnected topics: task scheduling as combinatorial optimisation (Task Scheduling as Optimisation), online scheduling and load balancing (Online Scheduling and Load Balancing), and fault tolerance in the presence of adversarial or Byzantine agents (Fault Tolerance).

Task Scheduling as Optimisation

Even the simplest variant of this problem is computationally hard.

Despite NP-hardness, simple greedy algorithms provide provable approximation guarantees. The Longest Processing Time First (LPT) algorithm sorts tasks in decreasing order of cost and assigns each task to the least loaded machine, achieving a $4/3$-approximation ratio [74]. For multi-agent LLM systems where task costs are estimated rather than known exactly, this greedy approach is both practical and near-optimal.

Online Scheduling and Load Balancing

In many practical settings, tasks arrive online and their costs are unknown a priori. We develop two key results: the power of randomised load balancing and the regret guarantees of online learning algorithms.

Balls into bins.

The simplest load-balancing model distributes $m$ tasks (“balls”) uniformly at random among $n$ agents (“bins”). The maximum load (number of tasks assigned to the busiest agent) determines the makespan.

Online scheduling with EXP3.

When task costs are adversarial (unknown and potentially chosen by an adversary), we can use the EXP3 (Exponential-weight algorithm for Exploration and Exploitation) algorithm from the adversarial multi-armed bandit literature. EXP3 maintains a probability distribution over agents and achieves a regret bound of $O(\sqrt{nT\ln n})$ over $T$ rounds with $n$ agents, where regret is measured against the best fixed agent in hindsight [75].

Fault Tolerance

In any distributed system, agents may fail: by crashing, producing incorrect outputs, or, in the worst case, behaving adversarially. This subsection addresses the fundamental limits of fault tolerance in multi-agent systems, culminating in the classical Byzantine consensus bound.

The foundational result of distributed computing establishes a sharp threshold for the number of Byzantine agents that can be tolerated.

Practical fault tolerance in LLM systems.

The AgentScope framework [10] implements a four-tier fault tolerance strategy for multi-agent LLM systems:

1. Retry with backoff: failed API calls are retried with exponential backoff, tolerating transient failures;

2. Output validation: each agent's output is checked against format and content constraints before being passed to other agents;

3. Redundant execution: critical tasks are assigned to multiple agents, with outputs reconciled via majority vote (requiring $n\ge 3f+1$ for Byzantine tolerance);

4. Checkpoint and rollback: the system state is periodically checkpointed, allowing rollback to a consistent state after detecting a failure cascade.

These pragmatic mechanisms complement the theoretical bounds of Theorem 14, providing defence in depth against both accidental failures and adversarial attacks.

Summary of Sections 11–15

This part of the chapter developed the mathematical foundations for multi-agent interaction, coordination, and operations. Game Theory Foundations for Multi-Agent Systems established game-theoretic foundations, from Nash equilibria through the folk theorem to Shapley values. Consensus and Coordination analysed consensus convergence, the Condorcet jury theorem, and the surprising dominance of quality over diversity. Cooperation and Competition examined how cooperation emerges spontaneously among LLM agents, connecting to co-evolutionary dynamics and the Red Queen hypothesis. Credit Assignment in Multi-Agent Systems formalised credit assignment and derived multi-agent policy gradients. Finally, Scheduling, Load Balancing, and Fault Tolerance addressed the operational concerns of scheduling, load balancing, and fault tolerance, culminating in the Byzantine consensus bound.

The recurring mathematical theme is the interplay between individual rationality and collective optimality. Nash equilibria may be socially suboptimal (Prisoner's Dilemma), but repetition enables cooperation (folk theorem). Majority voting improves upon individuals (Condorcet), but correlation limits the gains (Proposition 15). The Shapley value provides the unique fair allocation of credit, but is exponentially expensive to compute exactly. Byzantine agents can corrupt consensus, but $n\ge 3f+1$ agents suffice to tolerate $f$ failures.

These mathematical tools, drawn from game theory, graph theory, probability, and distributed computing, form the analytical backbone of multi-agent AI systems. The following sections (in the companion files) build on these foundations to address emergent behaviours, safety, and the future trajectory of multi-agent systems.

Evolutionary Approaches to Agent Systems

Nature solved the design problem long before engineers did. Biological evolution (blind variation coupled with selective retention) produced the staggering diversity of life on Earth without a single line of pseudocode. The same principle applies to the design of agentic AI systems: rather than hand-crafting the architecture of a multi-agent system, we can evolve it. This section develops the mathematical foundations of evolutionary approaches to agent design, from swarm-intelligence-based architecture search to trajectory-level self-improvement to population dynamics that explain how specialisation emerges in agent ecosystems.

The intellectual arc is as follows. We begin with SwarmAgentic (Agent Architecture Search via Swarm Intelligence), which adapts Particle Swarm Optimisation to the space of agent architectures, treating each candidate multi-agent system as a “particle” whose position is a structured natural-language description of agents and their collaboration protocol. We then turn to AgentEvol (Trajectory-Based Evolution: AgentEvol in Detail), which takes a fundamentally different approach: rather than searching over architectures, it evolves the behaviour of a fixed architecture by iteratively collecting trajectories, filtering by reward, and fine-tuning. We derive the variational bound that justifies this procedure and prove that it monotonically improves a lower bound on expected success probability. Finally, we place both approaches in the broader context of evolutionary game theory (sec:agents:population,sec:agents:specialization), showing how replicator dynamics govern population-level selection and how specialisation emerges as an evolutionarily stable strategy.

Agent Architecture Search via Swarm Intelligence

The central challenge of multi-agent system design is combinatorial: given $n$ possible agent roles, $m$ possible tools per agent, and $k$ possible communication topologies, the space of candidate systems grows as $O(n^m\cdot k)$, far too large for exhaustive search, yet too structured for pure random sampling. Swarm intelligence offers a middle path: a population of candidate solutions explores the space in parallel, sharing information about promising regions.

The key insight of SwarmAgentic [38] is that each particle's “position” can be encoded as a structured natural-language string, a prompt that, when fed to an LLM orchestrator (Definition 25), instantiates a complete multi-agent system. This encoding enables the use of language-model mutations (paraphrasing, recombination, insertion) as the analogue of velocity updates in classical Particle Swarm Optimisation (PSO).

Particle Swarm Optimisation refresher. In classical PSO [76], a swarm of $N$ particles explores a continuous space ${\mathrm{ℝ}}^d$. Each particle $i$ maintains a position ${𝒙}_i\in {\mathrm{ℝ}}^d$ and a velocity ${𝒗}_i\in {\mathrm{ℝ}}^d$. At each iteration, the velocity is updated as (PSO Velocity)$$\begin{array}{ccc}& {𝒗}_i^{(t+1)}=w{𝒗}_i^{(t)}+c_1{r}_1({𝒑}_i-{𝒙}_i^{(t)})+c_2{r}_2(𝒈-{𝒙}_i^{(t)}),& \text{<span class="tml-eqn"></span>}\end{array}$$ where $w$ is the inertia weight, $c_1,c_2$ are acceleration coefficients, $r_1,r_2\sim \text{Uniform}(\mathrm{0,1})$, ${𝒑}_i$ is particle $i$'s personal best position, and $𝒈$ is the global best position. The position update is simply ${𝒙}_i^{(t+1)}={𝒙}_i^{(t)}+{𝒗}_i^{(t+1)}$.

Adaptation to agent architecture space. SwarmAgentic replaces the continuous velocity update with a failure-aware language-space update. The three terms become:

1. Failure-driven adjustment (analogous to inertia): analyse the current system's failure modes on a validation set and generate a natural-language “diff” that addresses identified weaknesses;

2. Personal best guidance: compare the current system description to the particle's historically best-performing description and generate modifications that move toward the personal best;

3. Global best guidance: compare to the swarm's globally best-performing description and generate modifications that incorporate its strengths.

These three natural-language diffs are combined (via an LLM that merges and deduplicates) to produce the updated system description. The fitness function $f:𝒳\to \mathrm{ℝ}$ evaluates each candidate system on a held-out task suite, measuring accuracy, efficiency, and cost.

Convergence analysis. Classical PSO convergence theory [77] shows that particles converge when the inertia and acceleration parameters satisfy certain spectral conditions. For SwarmAgentic, the situation is more nuanced because the “velocity” is a language-space operation rather than a vector addition. Nevertheless, we can analyse convergence through the lens of fitness monotonicity: the global best fitness $f({𝒈}^{(t)})$ is non-decreasing by construction (we only update $𝒈$ when a strictly better solution is found). Since the fitness function is bounded above (task performance lies in $[\mathrm{0,1}]$), the sequence ${\{f({𝒈}^{(t)})\}}_{t=0}^T$ converges to a limit $f^{\ast }$. The key practical question is the rate of convergence: how many iterations $T$ are needed to reach within $ϵ$ of the global optimum? In continuous PSO, convergence rates of $O(1/T)$ have been established under mild assumptions [78]. For SwarmAgentic, empirical evidence suggests similar convergence profiles, with most improvement occurring in the first $T\approx 10$–$15$ iterations.

Computational cost. Each iteration of Algorithm 5 requires $N$ fitness evaluations (one per particle), where each evaluation involves instantiating a full multi-agent system and running it on the validation set. If each evaluation costs $C$ tokens, the total cost is $O(N\cdot T\cdot C)$. With $N=20$ particles, $T=15$ iterations, and $C\approx {10}^5$ tokens per evaluation, the total cost is approximately $3\times {10}^8$ tokens, substantial, but amortised over the lifetime of the resulting system. This makes SwarmAgentic most suitable for high-stakes, repeatedly-used agent systems where the design cost is justified by deployment savings.

Trajectory-Based Evolution: AgentEvol in Detail

While SwarmAgentic searches over the space of architectures, AgentEvol [15] takes a complementary approach: it fixes the architecture and evolves the behaviour of an agent by iteratively collecting task trajectories, filtering by reward, and fine-tuning on the successful ones. The mathematical justification rests on a variational lower bound that we now derive in full.

Setup. Let $a$ denote a task instruction drawn from some distribution $p(a)$. A trajectory $\tau =(s_0,a_0,s_1,a_1,\dots ,s_T)$ is a sequence of states and actions generated by a policy ${\pi }_{\theta }(\tau |a)$ parameterised by $𝜽$. We observe a binary reward $r(\tau )\in \{0,1\}$ indicating success or failure. Our goal is to maximise the expected success probability: (Agentevol Objective EVOL)$$\begin{array}{ccc}& J(𝜽)={𝔼}_{a\sim p(a)}[p(\text{success}|a;𝜽)]={𝔼}_{a\sim p(a)}[\sum _{\tau }{\pi }_{\theta }(\tau |a)r(\tau )].& \text{<span class="tml-eqn"></span>}\end{array}$$

Variational lower bound. For any instruction $a$, the log-success-probability admits a variational decomposition. Let $q(\tau |a)$ be an arbitrary distribution over trajectories. Then: (Agentevol ELBO EVOL)$$\begin{array}{cccc}& \log p(\text{success}|a)& =\log \sum _{\tau }r(\tau ){\pi }_{\theta }(\tau |a)& \\ & & =\log \sum _{\tau }q(\tau |a)\frac{r(\tau ){\pi }_{\theta }(\tau |a)}{q(\tau |a)}& \\ & & \ge \sum _{\tau }q(\tau |a)\log \frac{r(\tau ){\pi }_{\theta }(\tau |a)}{q(\tau |a)}& \text{<span class="tml-eqn"></span>}\\ & & ={𝔼}_{q(\tau |a)}[\log r(\tau )+\log {\pi }_{\theta }(\tau |a)]+𝖧[q(\tau |a)],& \end{array}$$ where follows from Jensen's inequality applied to the concave function $\log$. Since $r(\tau )\in \{\mathrm{0,1}\}$, the term $\log r(\tau )$ is zero when $r(\tau )=1$ and $-\infty$ when $r(\tau )=0$, so the optimal variational distribution places all its mass on successful trajectories: (Agentevol Optimal Q)$$\begin{array}{ccc}& q^{\ast }(\tau |a)\propto r(\tau )\cdot {\pi }_{\theta }(\tau |a).& \text{<span class="tml-eqn"></span>}\end{array}$$ This is the reward-weighted policy: we keep only the trajectories that succeed and reweight them by the policy's own probability.

Practical algorithm. AgentEvol approximates the above in $M$ iterations. At each iteration $m$:

1. Sample $K=1$ trajectory per instruction from the current policy ${\pi }_{{\theta }_m}$;

2. Compute binary rewards $r(\tau )$;

3. Merge the successful trajectories with a base dataset ${𝒟}_0$ of expert demonstrations;

4. Maximise the weighted log-likelihood: ${𝜽}_{m+1}={\mathrm{arg\hspace{0.17em}max}}_{𝜽}{\sum }_{\tau :r(\tau )=1}\log {\pi }_{𝜽}(\tau |a)+{\sum }_{\tau \in {𝒟}_0}\log {\pi }_{𝜽}(\tau |a)$.

Empirically, $M=4$ iterations balance performance gain against computational cost.

Population-Level Selection

The preceding subsections optimised individual agent systems. We now zoom out to the population level: given a collection of agent strategies coexisting in an environment, which strategies survive and which go extinct? The mathematical framework is evolutionary game theory [79], and the central dynamical equation is the replicator equation.

The replicator equation has deep connections to classical game theory.

Not all Nash equilibria are equally robust. An Evolutionarily Stable Strategy (ESS) [80] is one that cannot be invaded by a small population of mutants.

Emergence of Specialisation

One of the most striking phenomena in multi-agent systems is emergent specialisation: agents that begin as generalists evolve to become specialists, each focusing on a narrow subset of tasks. This mirrors the division of labour observed in biological colonies (ant castes), economic markets (comparative advantage), and scientific communities (disciplinary boundaries).

The mathematical explanation comes from the interplay between environmental heterogeneity and competitive dynamics. The Bishop–Cannings theorem [81] in evolutionary game theory states that in games with symmetric payoff structures, the ESS involves a mixture of strategies whose support depends on the environment's structure.

Scientific Discovery as Multi-Agent Collaboration

Science is, and has always been, a multi-agent enterprise. No single researcher, however brilliant, discovers, verifies, and disseminates knowledge alone. The scientific community is a vast, self-organising network: individual researchers propose hypotheses, design experiments, collect data, write papers, submit to peer review, revise, and publish. Other researchers read, critique, replicate, extend, and occasionally overturn prior work. The resulting knowledge structure, an ever-growing, self-correcting web of theories, evidence, and consensus, is arguably the most successful multi-agent coordination protocol in human history.

This section formalises scientific discovery as a multi-agent system, connecting the abstract definitions of Definition 14 to the concrete workflows of scientific research. We show how agent-based hypothesis generation (Agent-Based Hypothesis Generation) decomposes the creative process into specialised roles, how milestone-based coordination (Milestone-Based Coordination) manages the dependencies inherent in complex research programmes, and how knowledge aggregation (Knowledge Aggregation) ensures that the community converges on truth, even when individual researchers are biased or uncertain.

The Scientific Community as a Multi-Agent System

We begin by instantiating the abstract MAS definition to the scientific domain.

The publish-subscribe communication pattern deserves emphasis. Unlike the direct messaging of Definition 18, publish-subscribe is topic-based: an agent publishes a result to a topic (e.g., “protein folding”), and all agents subscribed to that topic receive it. This decoupling enables scalability: a new agent can join the system simply by subscribing to relevant topics, without requiring changes to the communication topology.

Connection to human science. The parallel to real scientific communities is intentional and illuminating. In human science, $𝒜$ is the set of research groups; $ℰ$ is the body of published literature; $𝒞$ is the system of journals, preprint servers, and conferences; and $\mathrm{\Omega }$ is the combination of peer review and funding agencies. The question this section asks is: can we build artificial scientific communities that exhibit the same self-correcting dynamics?

Agent-Based Hypothesis Generation

The creative core of science is hypothesis generation: proposing explanations for observed phenomena. In agent-based scientific discovery, this process is decomposed into four specialised roles, each implemented by an LLM agent with a tailored system prompt and tool set.

Hypothesis generators (creative/divergent agents) explore the space of possible explanations. Their system prompt encourages breadth: “Generate as many distinct hypotheses as possible that could explain the following observation” They are equipped with literature search tools that retrieve relevant prior work, enabling analogical reasoning across domains.

Experimenters (tool-using agents, cf. Definition 5) design and execute experiments. Given a hypothesis $h\in \mathscr{H}$, an experimenter agent produces a test plan, writes code to implement it, runs the code in a sandboxed environment, and returns structured results. The key capability is tool use: the agent can call APIs for molecular simulation, statistical testing, database queries, and visualisation.

Critics (analytical/convergent agents) evaluate hypotheses against evidence. Their system prompt encourages scepticism: “Identify weaknesses, alternative explanations, and potential confounds” Critics implement a lightweight form of peer review, filtering the hypothesis space before expensive experiments.

Synthesisers combine partial results from multiple hypothesis-experiment cycles into coherent theories. They produce structured summaries, identify patterns across experiments, and propose higher-order hypotheses that unify disparate findings.

Milestone-Based Coordination

Complex scientific projects, like complex engineering projects, have dependencies: you cannot analyse data before collecting it, and you cannot collect data before designing the experiment. MultiAgentBench [82] formalises these dependencies using milestone DAGs.

The key performance indicators (KPIs) for milestone-based coordination are:

• Progress: $\text{Prog}(t)=|M_{\text{done}}(t)|/|M|$, the fraction of milestones completed by time $t$;

• Efficiency: $\text{Eff}(t)=\text{Prog}(t)/\text{turns}(t)$, progress per communication round;

• Communication quality: a 1–5 score assessing relevance, clarity, and informativeness of inter-agent messages;

• Planning quality: a 1–5 score assessing the agent's ability to decompose the task into appropriate sub-goals.

The following classical result from project scheduling theory establishes the fundamental limit on makespan.

Knowledge Aggregation

Scientific progress requires not only generating knowledge but also aggregating it: combining the beliefs, observations, and expertise of many agents into a shared consensus. The mathematical framework for this process is the DeGroot learning model [83], which describes how a network of agents updates beliefs through iterated averaging.

The central question is: does the community converge to a consensus, and if so, whose beliefs matter most?

Security and Adversarial Robustness

The power of multi-agent systems is also their vulnerability. Every channel that enables collaboration is a channel that an adversary can exploit. Every tool that an agent can invoke is a tool that a compromised agent can misuse. Every emergent behaviour that delights us in cooperative settings can terrify us in adversarial ones. This section develops the security theory of multi-agent systems, moving from threat modelling through specific attack vectors to defence mechanisms and the deep open problem of alignment composability.

The stakes are high. As agentic AI systems are deployed in high-value domains (financial trading, scientific research, software engineering, healthcare), the incentives for adversarial attacks grow correspondingly. A single compromised agent in a code-review pipeline could inject subtle backdoors into production software. A self-replicating adversarial prompt could propagate through an entire agent network, turning helpful assistants into malicious actors. Understanding these threats mathematically is the first step toward defending against them.

Threat Model for Multi-Agent Systems

We identify three fundamental categories of threats to multi-agent systems, each requiring different mathematical tools and different defences.

Category (a) is the domain of prompt injection and adversarial examples; category (b) is the domain of Byzantine fault tolerance; category (c) is the most subtle and the most difficult to defend against, because the threat arises not from any individual agent's misbehaviour but from the interaction dynamics of the system as a whole.

The CODES Attack: Self-Replicating Adversarial Prompts

The most alarming attack vector in multi-agent systems is the self-replicating adversarial prompt: a carefully crafted input that, when processed by an agent, causes that agent to embed the same (or a mutated) adversarial prompt in its outputs, which are then consumed by downstream agents. The CODES framework (Definition 42) formalises this as a computer worm for agent networks.

The dynamics of SRAP propagation through an agent network can be modelled using the classical SIR epidemic model from mathematical epidemiology [84].

SIR model for agent networks. Partition the agents into three compartments at time $t$:

• $S(t)$: susceptible agents (not yet exposed to the SRAP);

• $I(t)$: infected agents (actively replicating the SRAP);

• $R(t)$: recovered agents (patched or quarantined, immune to reinfection).

With total population $n=S+I+R$ (constant), the dynamics are: (SIR)$$\begin{array}{ccc}& \frac{dS}{dt}=-\beta SI,\frac{dI}{dt}=\beta SI-\gamma I,\frac{dR}{dt}=\gamma I,& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\beta >0$ is the transmission rate (probability per unit time that a susceptible agent is infected by an infected neighbour) and $\gamma >0$ is the recovery rate (probability per unit time that an infected agent is patched). The basic reproduction number is (R0)$$\begin{array}{ccc}& R_0=\frac{\beta n}{\gamma },& \text{<span class="tml-eqn"></span>}\end{array}$$ which counts the expected number of secondary infections caused by a single infected agent in an otherwise fully susceptible population.

Defences

Given the severity of the threats outlined above, what can be done? We organise defences into four categories: input sanitisation, output verification, cryptographic signing, and reputation systems.

Input sanitisation. The simplest defence is to filter all incoming messages through a classifier trained to detect adversarial content. This can be implemented as a separate “guardian” agent that inspects messages before they reach the target agent. The challenge is that SRAPs are designed to be semantically meaningful (they must look like legitimate messages to evade detection), making classification difficult.

Output verification. Rather than filtering inputs, one can verify outputs: before an agent's response is forwarded to others, a verification agent checks that it is consistent with the agent's role and does not contain suspicious patterns. This is analogous to the critic role in scientific discovery (Agent-Based Hypothesis Generation).

Cryptographic message signing. Each agent signs its messages with a private key; recipients verify signatures using the sender's public key. This prevents external injection (category (a) of Definition 49) but does not help against compromised agents (category (b)), who possess valid signing keys.

Reputation systems. Agents maintain reputation scores based on their historical accuracy and reliability. Messages from low-reputation agents are given less weight (connecting to the DeGroot trust matrix of Definition 48). Formally, let ${\rho }_i(t)\in [\mathrm{0,1}]$ be agent $i$'s reputation at time $t$, updated as (Reputation)$$\begin{array}{ccc}& {\rho }_i(t+1)=(1-\alpha ){\rho }_i(t)+\alpha {\text{accuracy}}_i(t),& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\alpha \in (\mathrm{0,1})$ is a learning rate and ${\text{accuracy}}_i(t)$ measures the correctness of agent $i$'s most recent output. Agents with reputation below a threshold ${\rho }_{\min }$ are quarantined (their messages are not forwarded), implementing an adaptive defence against gradually compromised agents.

ROC analysis and optimal operating points. Every message filter induces a trade-off between security (blocking attacks) and utility (allowing legitimate messages through). This trade-off is captured by the Receiver Operating Characteristic (ROC) curve, which plots TPR against FPR as the decision threshold varies. The area under the ROC curve (AUC) provides a threshold-independent measure of filter quality.

The optimal operating point depends on the cost ratio $\kappa =C_{\text{miss}}/C_{\text{false alarm}}$, where $C_{\text{miss}}$ is the cost of failing to block an adversarial message and $C_{\text{false alarm}}$ is the cost of blocking a legitimate one. The optimal threshold $t^{\ast }$ satisfies (ROC Optimal)$$\begin{array}{ccc}& \frac{{\text{TPR}}^{\prime }(t^{\ast })}{{\text{FPR}}^{\prime }(t^{\ast })}=\kappa \cdot \frac{1-{\pi }_{\text{adv}}}{{\pi }_{\text{adv}}},& \text{<span class="tml-eqn"></span>}\end{array}$$ where ${\pi }_{\text{adv}}$ is the prior probability that a given message is adversarial, and the primes denote derivatives with respect to the threshold. In high-stakes deployments (large $C_{\text{miss}}$), the optimal point shifts toward higher TPR at the expense of higher FPR; in latency-sensitive deployments (large $C_{\text{false alarm}}$), the reverse holds.

Defence in depth. No single defence mechanism is sufficient. Practical multi-agent security architectures combine all four categories in a layered “defence in depth” strategy:

1. Cryptographic signing blocks external injection;

2. Input sanitisation catches known attack patterns;

3. Output verification detects novel attacks that bypass input filters;

4. Reputation systems provide long-term adaptive defence against slow-onset compromise.

The layered approach ensures that an attacker must defeat every layer simultaneously, exponentially increasing the difficulty of a successful attack. If each layer independently detects attacks with probability $p_k$, the probability of a successful attack through $L$ layers is ${\prod }_{k=1}^L(1-p_k)$, which decreases exponentially in $L$.

Emergent Misalignment

The most philosophically profound threat to multi-agent systems is not external attack but internal emergence: individually aligned agents producing collectively misaligned outcomes. This is not a theoretical curiosity; it occurs routinely in human organisations (groupthink, market bubbles, arms races) and has been observed in agent simulations.

Mechanisms of emergent misalignment. Three mechanisms can cause individually aligned agents to produce collectively misaligned outcomes:

1. Tacit collusion: agents independently discover that certain coordinated strategies (e.g., withholding information) increase their individual rewards, even though these strategies harm the principal. No explicit communication is required; the collusion emerges from repeated interaction and shared incentive structures.

2. Echo chambers: agents that preferentially communicate with like-minded agents reinforce each other's biases, leading to polarisation. In DeGroot terms (Definition 48), this occurs when the trust matrix $W$ has a block structure, so the network does not converge to a single consensus but to multiple conflicting ones.

3. Reward hacking at scale: each agent individually avoids reward hacking (per Definition 13), but the composition of their outputs creates an artifact that exploits the evaluation metric, a form of distributed reward hacking that no individual agent is “responsible” for.

Benchmarking and Evaluation

How do we know whether a multi-agent system is any good? The question sounds simple; the answer is surprisingly deep. Evaluating a single model is already hard: one must choose metrics, construct held-out sets, account for distribution shift. Evaluating a multi-agent system is harder still, because the outcome depends not only on each agent's individual capability but on the composition of capabilities, the quality of inter-agent communication, and the structure of the orchestration protocol. Two systems with identical agents can produce wildly different outcomes depending on how those agents are wired together.

This section develops a principled framework for multi-agent evaluation. We begin with the fundamental challenges that distinguish multi-agent evaluation from single-agent evaluation (The Challenge of Evaluating Multi-Agent Systems), then describe concrete KPI frameworks (Milestone KPIs), analyse the statistical structure of performance prediction (Feature Importance in Multi-Agent Performance), and conclude with the long-term vision of formal verification for agent systems (Toward Formal Verification).

The Challenge of Evaluating Multi-Agent Systems

Three properties make multi-agent evaluation fundamentally harder than single-agent evaluation:

1. Composition-dependent outcomes. The performance of a multi-agent system is not a simple function of its components. A team of individually strong agents may perform poorly if their communication is inefficient, and a team of individually weak agents may perform surprisingly well if their collaboration protocol is well-designed. Formally, if $f(a_i)$ denotes agent $i$'s individual score and $g(𝒮)$ denotes the system's collective score, then in general $g(𝒮)\ne h(f(a_1),\dots ,f(a_n))$ for any fixed aggregation function $h$; the interaction structure matters.

2. Process matters. In single-agent evaluation, we typically care only about the final output (accuracy, BLEU score, etc.). In multi-agent systems, the process by which agents arrive at the output is also important: Were the inter-agent messages informative? Did agents ask for clarification when uncertain? Did the orchestrator delegate tasks appropriately? A system that reaches the correct answer through chaotic, redundant communication is fragile and unlikely to generalise.

3. Interaction-induced stochasticity. Even with deterministic agents, the order in which messages are delivered can affect the outcome. In systems with stochastic agents (LLMs with temperature $>0$), the variance of the collective outcome can be substantially larger than the variance of any individual agent, because uncertainties compound through chains of interaction.

Milestone KPIs

The MultiAgentBench framework [82] operationalises multi-agent evaluation through milestone-based KPIs that decompose complex tasks into verifiable sub-goals (cf. Definition 47).

The key insight of MultiAgentBench is that milestone progress provides a far more informative signal than binary success/failure. A system that completes 7 out of 10 milestones before failing is clearly better than one that completes 2 out of 10, even though both receive a score of zero under binary evaluation. This fine-grained measurement enables meaningful comparison between systems at all capability levels, not just those near the performance frontier.

Feature Importance in Multi-Agent Performance

What determines whether a multi-agent system succeeds? Kim et al. [19] address this question through a systematic regression analysis, fitting a 20-parameter model that predicts system performance from architectural and process features.

Toward Formal Verification

The ultimate goal of multi-agent evaluation is not just empirical benchmarking but formal verification: proving that a system satisfies desired properties for all possible inputs and interaction sequences, not just those observed in a test suite. The mathematical framework for this is temporal logic, which extends propositional logic with operators for reasoning about time.

Applications and Case Studies

The preceding sections developed the mathematical theory of multi-agent systems: communication protocols, game-theoretic foundations, evolutionary dynamics, security analysis, and evaluation frameworks. We now ask the pragmatic question: where does this theory meet practice? The answer, as it turns out, is nearly everywhere. Multi-agent architectures have penetrated software engineering, scientific research, education, and large-scale simulation, in each case transforming what was previously a single-model task into a richly structured collaborative process.

This section presents four detailed case studies. Each illustrates a different facet of the theory: software engineering (Software Engineering) demonstrates the power of role-based decomposition and the orchestrator pattern; scientific research (Scientific Research) shows publish-subscribe communication and milestone coordination in action; education (Education and Tutoring) highlights continual learning and personalisation; and million-agent simulation (Large-Scale Simulation) pushes the scalability limits studied in Theorem 3 and validates game-theoretic predictions at unprecedented scale.

Software Engineering

Software engineering is a natural domain for multi-agent systems. Writing production-quality code requires understanding requirements, planning architecture, implementing modules, writing tests, running tests, debugging failures, and conducting code review, a pipeline with clear role boundaries and sequential dependencies.

Scientific Research

The multi-agent framework for scientific discovery developed in Scientific Discovery as Multi-Agent Collaboration is not merely theoretical; it is being operationalised in real research workflows.

Education and Tutoring

Education is a domain where multi-agent architectures offer qualitative advantages over single-agent systems. Effective tutoring requires not one skill but several: explaining concepts, assessing understanding, adapting difficulty, providing encouragement, and managing a curriculum over time. These are naturally mapped to distinct agent roles.

Agent roles in tutoring. A multi-agent tutoring system typically involves three agents:

1. Teacher agent: explains concepts, provides worked examples, and answers questions. Its system prompt is optimised for clarity and Socratic dialogue: rather than giving direct answers, it guides the student through reasoning steps.

2. Assessment agent: designs quizzes and problems calibrated to the student's current level, grades responses, and identifies misconceptions. It maintains a knowledge state model ${𝒔}_t\in {[\mathrm{0,1}]}^K$ where $K$ is the number of concepts and $s_{t,k}$ estimates the student's mastery of concept $k$ at time $t$. The update rule is Bayesian: (Knowledge State)$$\begin{array}{ccc}& s_{t+1,k}=\frac{s_{t,k}P(\text{correct}|{\text{mastery}}_k)}{P(\text{correct}|{\text{obs}}_t)},& \text{<span class="tml-eqn"></span>}\end{array}$$ where the likelihood $P(\text{correct}|{\text{mastery}}_k)$ follows an Item Response Theory (IRT) model.

3. Curriculum agent: plans the sequence of topics based on the knowledge state, prerequisite dependencies (a DAG, similar to Definition 47), and the student's learning pace. It implements a form of continual learning (ch:continual), adapting the curriculum as the student's knowledge evolves.

The three agents communicate through a shared student model: the assessment agent updates the knowledge state ${𝒔}_t$; the curriculum agent reads ${𝒔}_t$ to decide what to teach next; and the teacher agent reads both ${𝒔}_t$ (to calibrate explanation depth) and the curriculum plan (to know which concept to address).

Personalisation through interaction. The multi-agent architecture enables a level of personalisation that single-agent systems struggle to achieve. The assessment agent can detect that a student consistently confuses two related concepts (e.g., covariance and correlation) and flag this to the teacher agent, which can then provide targeted exercises. The curriculum agent can notice that the student learns visual concepts faster than algebraic ones and adjust the presentation modality accordingly. These adaptations emerge from the interaction between specialised agents, each maintaining its own model of the student.

Theoretical connection: zone of proximal development. Vygotsky's zone of proximal development (ZPD) posits that learning is most effective when tasks are neither too easy nor too hard, but in a “sweet spot” just beyond the student's current capability. The multi-agent tutoring system operationalises the ZPD mathematically: the curriculum agent selects concept $k^{\ast }$ at time $t$ by solving (ZPD)$$\begin{array}{ccc}& k^{\ast }={\mathrm{arg\hspace{0.17em}max}}_{k\in \text{frontier}({𝒔}_t)}[\text{value}(k)\cdot (1-s_{t,k})\cdot \text{prereq\_met}(k,{𝒔}_t)],& \text{<span class="tml-eqn"></span>}\end{array}$$ where $\text{frontier}({𝒔}_t)$ is the set of concepts whose prerequisites are mastered ($s_{t,j}\ge 0.8$ for all prerequisites $j$), $\text{value}(k)$ is the concept's importance in the curriculum, and $1-s_{t,k}$ is the “room for improvement.” This ensures the system always targets the most valuable concept within the student's ZPD.

Large-Scale Simulation

The most ambitious application of multi-agent systems is large-scale social simulation: modelling the behaviour of thousands or millions of interacting agents to understand emergent social, economic, or epidemiological phenomena.

llll@ Domain	Key agents	Comm. pattern	Core theory
Software eng.	Orchestrator, coders,	Sequential +	Milestone DAGs
	reviewer, tester	feedback loops	(Proposition 23)
[3pt] Scientific	Hypothesis, experiment,	Publish-	DeGroot learning
research	critic, synthesiser	subscribe	(Theorem 16)
[3pt] Education	Teacher, assessor,	Shared state	Bayesian knowledge
	curriculum planner	(student model)	tracing
[3pt] Simulation	${10}^3$–${10}^6$	Broadcast +	Game theory,
	homogeneous agents	local interaction	cognitive hierarchy

Looking ahead. The five sections in this file (evolution, science, security, benchmarking, and applications) complete our treatment of agentic AI and multi-agent systems. The mathematical toolkit we have assembled is broad: game theory (Definition 44, Definition 45), variational inference (Theorem 15), epidemiology (Proposition 24), spectral graph theory (Theorem 16), combinatorial optimisation (Proposition 23), and temporal logic (Definition 54). Yet the field is young, and the most important questions remain open. Can we formally verify the safety of large-scale agent systems? Can we guarantee that alignment composes (Conjecture 1)? Can we evolve agent architectures that rival human-designed ones? The answers to these questions will shape the next decade of AI research, and the mathematical foundations laid in this chapter provide the language in which those answers will be expressed.

A Unified Mathematical Framework for Agent Systems

The preceding twenty sections have developed a rich landscape of formalisms: single-agent decision processes, multi-agent communication protocols, game-theoretic equilibria, topological structures, evolutionary dynamics, consensus algorithms, and scaling laws. Each formalism illuminated a particular facet of agentic intelligence. This section unifies them into a single mathematical object, the Complete Agent System, and derives information-theoretic and scaling-theoretic consequences that govern the behaviour of every agent system in this chapter.

The unification serves three purposes. First, it provides a common language in which every framework (AutoGen, AgentScope, Symphony, LongAgent, MALLM, and beyond) can be stated as an instantiation of a single 6-tuple. Second, it reveals structural invariants (capacity bounds, scaling exponents) that hold across all instantiations. Third, it connects the mathematics of agent systems to the broader arc of this book: the reasoning, memory, alignment, and continual-learning chapters each contribute a component to the complete picture.

The Complete Agent System Formalism

We now define the central object of this chapter.

Mapping existing frameworks.

We now demonstrate that every major framework discussed in this chapter is an instantiation of Definition 55.

• AutoGen [8]: Agents $𝒜=\{\text{UserProxy},\text{Assistant}\}$ in a two-agent chat. Environment $ℰ$ is a code execution sandbox. Communication $𝒞$ uses natural-language messages with code blocks. Orchestrator $\mathrm{\Omega }$ implements round-robin turn-taking with termination conditions. Topology $𝒢$ is a complete graph on 2 nodes. Reward $ℛ$ is task completion (binary: success/failure).

• AgentScope [10]: Agents $𝒜$ are configurable via YAML with heterogeneous tool sets. Environment $ℰ$ includes distributed services. Communication $𝒞$ supports both pipeline (sequential) and DAG-structured message flow. Orchestrator $\mathrm{\Omega }$ is user-specified (pipeline, hub, or custom). Topology $𝒢$ is a DAG. Reward $ℛ$ is application-specific.

• LongAgent [42]: Agents $𝒜=\{\text{Leader}\}\cup \{{\text{Member}}_1,\dots ,{\text{Member}}_k\}$. Environment $ℰ$ is a shared document corpus. Communication $𝒞$ uses structured query–response pairs. Orchestrator $\mathrm{\Omega }$ is the Leader agent. Topology $𝒢$ is a star graph centred on the Leader. Reward $ℛ$ measures reading comprehension accuracy with inter-member conflict resolution.

• MALLM [43]: Agents $𝒜$ are $n$ LLM instances with persona descriptions. Environment $ℰ$ is a shared annotation task. Communication $𝒞$ supports multiple paradigms (debate, memory, report, relay). Orchestrator $\mathrm{\Omega }$ is paradigm-dependent. Topology $𝒢$ varies by paradigm: complete graph for debate, chain for relay, star for report. Reward $ℛ$ is annotation quality.

• Symphony [44]: Agents $𝒜$ are specialised modules (planner, retriever, coder, verifier). Environment $ℰ$ includes a code execution engine and knowledge base. Communication $𝒞$ passes structured task descriptions. Orchestrator $\mathrm{\Omega }$ is a hierarchical planner. Topology $𝒢$ is a directed tree. Reward $ℛ$ decomposes hierarchically via milestone DAGs (Definition 47).

• SwarmAgentic [45]: Agents $𝒜$ are $n$ homogeneous LLM instances. Environment $ℰ$ is the task space. Communication $𝒞$ uses swarm intelligence: agents share fitness values and position vectors. Orchestrator $\mathrm{\Omega }$ is the PSO update rule (no centralised leader). Topology $𝒢$ is a complete graph (global best) or a ring (local best). Reward $ℛ$ is the task-specific fitness function.

Information-Theoretic Capacity of Agent Systems

A fundamental question is: how much can a multi-agent system know? A single agent's knowledge is bounded by its parameters, context window, and tool access. When multiple agents collaborate, the collective knowledge can exceed the sum of individual contributions, a phenomenon we call synergy.

Scaling Laws for Agent Systems

Neural scaling laws [85] establish that a single model's loss decreases as a power law in the parameter count $P$: $ℒ(P)\approx a{P}^{-\alpha }$ for constants $a,\alpha >0$. We conjecture an analogous law for multi-agent systems that incorporates both the number of agents $n$ and the per-agent model size $P$.

Unifying Table: All Frameworks as 6-Tuples

tab:agents:unified provides a comprehensive mapping of every major framework discussed in this chapter to the unified 6-tuple. Each row specifies how the framework instantiates agents, environment, communication, orchestration, topology, and reward.

lcccccc@ Framework	$𝒜$	$ℰ$	$𝒞$	$\mathrm{\Omega }$	$𝒢$	$ℛ$
AutoGen	2–$n$ roles	Code sandbox	NL + code	RR / custom	Complete	Task success
AgentScope	Configurable	Distributed	Pipeline/DAG	User-defined	DAG	App-specific
LongAgent	Leader + members	Document corpus	Query–response	Leader agent	Star	QA accuracy
MALLM	$n$ personas	Annotation task	Debate/relay	Paradigm-dep.	Varies	Label quality
Symphony	Specialists	KB + executor	Structured	Hier. planner	Tree	Milestone DAG
SwarmAgentic	$n$ homogeneous	Task space	Fitness sharing	PSO rule	Complete/ring	Fitness fn
CAMEL	2 role-play	Chat context	NL inception	Role prompts	Complete ($K_2$)	Task completion
AgentGym	Trainable	Diverse envs	Trajectory	Curriculum	Single + eval	Env. reward
Optima	$n$ specialised	Shared task	Compressed NL	Reward-driven	Complete	$R_{\text{task}}+\lambda R_{\text{eff}}$

From 6-tuple to dynamics.

Given a complete agent system $𝒮$, the system dynamics unfold as follows. At each round $t$:

1. The orchestrator $\mathrm{\Omega }$ selects the next speaker $a_{i_t}$ based on the history ${\mathscr{H}}_t$ and the topology $𝒢$.

2. Agent $a_{i_t}$ observes $o_{i_t}\sim {𝒪}_{\mathrm{env}}(s_t,a_{i_t})$ and receives incoming messages from neighbours in $𝒢$.

3. Agent $a_{i_t}$ produces an action (message, tool call, or environment interaction) according to policy ${\pi }_{a_{i_t}}$.

4. The environment transitions: $s_{t+1}\sim T_{\mathrm{env}}(s_t,{𝒂}_t)$.

5. Rewards $(r_1^{(t)},\dots ,r_n^{(t)},r_{\mathrm{sys}}^{(t)})$ are computed.

This general loop encompasses all frameworks in tab:agents:unified: AutoGen's two-agent chat, LongAgent's leader–member query cycle, SwarmAgentic's PSO iterations, and every other system described in this chapter.

Connections and Synthesis

This chapter on agentic AI is Part XI of the book, following Part VII (Reasoning, ch:reasoning), Part VIII (Memory, ch:memory), Part IX (Alignment, ch:alignment), and Part X (Continual Learning, ch:continual). Each of these chapters developed a distinct aspect of intelligent behaviour in isolation. In reality, these aspects interact profoundly, and multi-agent systems are the natural setting in which their interactions become most visible and most consequential.

This section establishes formal connections between agent systems and each preceding chapter, culminating in a grand unification that characterises the complete intelligent system.

Agent Systems Meet Memory

In ch:memory, we studied memory as a property of individual models: KV caches for short-term recall, retrieval-augmented generation for long-term access, and state-space models for implicit compression. Multi-agent systems introduce two qualitatively new forms of memory:

1. Shared memory (blackboard systems). A common data structure (document store, shared KV cache, or publication board) accessible by all agents. Shared memory implements a form of externalised collective knowledge: any agent can write to it, and any agent can read from it. The MALLM report paradigm and LongAgent's shared document corpus are instances.

2. Distributed memory. Knowledge is spread across agents' individual memories with no single agent holding the complete picture. Accessing a distributed memory requires communication: agent $i$ must query agent $j$ for information stored only in ${ℳ}_j$. This is the multi-agent analogue of retrieval-augmented generation, but with the retrieval target being another agent rather than a static database.

Memory Heterogeneity Thesis.

In a well-designed multi-agent system, agents naturally specialise in different memory types. A “librarian” agent maintains a long-term knowledge base (analogous to the hippocampal index in complementary learning systems [86]). A “working memory” agent holds the current problem state in its context window. A “historian” agent maintains episodic traces of past interactions. This division of labour across memory types mirrors the memory hierarchy in ch:memory: different memory subsystems serve different functions, and an orchestrator coordinates access.

Agent Systems Meet Alignment

Alignment (ch:alignment) ensures that a model's behaviour reflects human intent: helpful, honest, and harmless. A crucial question arises when we move from a single model to a multi-agent system: does aligning each agent individually guarantee that the system as a whole is aligned?

The answer, perhaps surprisingly, is no.

Agent Systems Meet Reasoning

ch:reasoning established that a single LLM agent's reasoning is bounded by its context window: the “working memory” available for chain-of-thought computation. Multi-agent systems fundamentally expand this bound.

Agent Systems Meet Continual Learning

ch:continual identified catastrophic forgetting as the central obstacle to lifelong learning: training on new tasks destroys knowledge from old tasks. Multi-agent systems offer a natural mitigation: if different agents are trained on different tasks, no single agent needs to learn everything.

The Grand Unification

Parts VII–XI of this book have developed five essential aspects of intelligent behaviour. We now characterise the system that combines all five.

The five aspects and their inter-connections form a pentagonal structure:

Historical Timeline

The ideas underlying multi-agent AI systems draw from game theory, distributed computing, swarm intelligence, classical AI, and, most recently, large language models. The timeline below traces the key milestones, from the foundational mathematics of strategic interaction to the modern era of LLM-based agent societies.

Detailed chronology.

1944

John von Neumann and Oskar Morgenstern publish Theory of Games and Economic Behavior [48], founding game theory as a mathematical discipline and establishing the framework for analysing strategic interaction among rational agents.

1950

John Nash proves the existence of equilibrium points in $n$-person games [4] (Theorem 7), guaranteeing that every finite game has at least one mixed-strategy equilibrium, a foundational result for multi-agent reasoning.

1953

Lloyd Shapley introduces the Shapley value [5] (Definition 33), providing the unique fair attribution of credit in cooperative games, now central to multi-agent credit assignment.

1959

Arthur Samuel demonstrates self-play in checkers [49], the first instance of an agent improving by playing against itself, a precursor to multi-agent training.

1986

Marvin Minsky publishes The Society of Mind [2], proposing that intelligence arises from the interaction of many simple agents, an early vision of multi-agent cognition. In the same year, Rodney Brooks introduces the subsumption architecture [50], demonstrating that complex behaviour can emerge from layered, interacting simple modules.

1989

Gerardo Beni and Jing Wang coin the term “swarm intelligence” [7] to describe collective behaviour in decentralised systems, inspiring particle swarm optimisation and ant colony algorithms.

1993

Yoav Shoham proposes Agent-Oriented Programming [51], formalising agents as first-class programming abstractions with beliefs, desires, and intentions (BDI).

1995

James Kennedy and Russell Eberhart introduce Particle Swarm Optimisation (PSO) [52], demonstrating that a swarm of simple agents sharing local fitness information can solve complex optimisation problems.

1996

The Foundation for Intelligent Physical Agents (FIPA) publishes the first agent communication language standards [53], establishing interoperability protocols for heterogeneous agent systems.

2000

Michael Wooldridge and Nicholas Jennings publish their influential survey [54] on multi-agent systems, defining the field's scope and core challenges.

2004

Reza Olfati-Saber and Richard Murray formalise consensus protocols for networked agents [55] (Theorem 10), proving convergence conditions for opinion dynamics on graphs.

2017

Ryan Lowe et al. introduce MADDPG [56], the first deep multi-agent reinforcement learning algorithm with centralised training and decentralised execution, the paradigm that dominates the field.

2022

Wei et al. demonstrate chain-of-thought prompting [57], enabling LLMs to perform multi-step reasoning and opening the door to LLM-based agents.

2023

The LLM agent era begins in earnest: Wu et al. release AutoGen [8], Yao et al. introduce ReAct [12] (reason + act), and Shinn et al. propose Reflexion [58] (self-reflection from verbal feedback). LLM agents transition from concepts to working systems.

2024

Platforms mature rapidly: AgentScope [10] provides a scalable multi-agent framework; LongAgent [42] demonstrates leader–member architectures for long-context tasks; AgentGym [15] creates standardised training environments; CAMEL [59] explores role-playing for cooperative agent behaviour. In parallel, MALLM [43] systematises multi-agent annotation paradigms.

2025

The field enters a quantitative phase: Kim et al. [46] discover power-law scaling of communication costs (Theorem 3); Zheng et al. [60] introduce thought communication with identifiability theory; the Optima framework [47] optimises communication efficiency; SwarmAgentic [45] brings PSO to LLM agent collectives.

2026

The frontier pushes toward million-agent simulations, co-player inference in strategic settings, and the first formal scaling laws for multi-agent LLM systems, the mathematical tools developed in this chapter.

The Future of Agentic AI

The mathematics developed in this chapter provides a rigorous foundation for understanding multi-agent LLM systems. But the field is young, and many of its most fundamental questions remain open. This section formalises five major open problems, discusses the emerging economics of agent systems, returns to the biological parallels that opened the chapter, and closes with a vision for the future of collective machine intelligence.

Open Problems

We state five open problems as formal conjectures. Each represents a frontier where the existing mathematical framework falls short of providing a complete answer.

Agent Economies and Incentive Design

As agent systems scale to hundreds or thousands of participants, the question of incentive design becomes central. How should agents be rewarded? What prevents free-riding? How can we ensure that individually rational agents contribute to collective welfare?

These questions place us squarely in the territory of mechanism design (Definition 30). The key insight is that agent economies mirror human economies: the same mathematical structures that govern markets, auctions, and public goods apply to multi-agent LLM systems.

Computational resource markets.

In a large-scale agent system, computational resources (GPU time, API calls, memory) are scarce. An auction-based task allocation mechanism assigns tasks to agents based on their bids: each agent bids a cost (in compute) for a task, and the mechanism selects the lowest bidder while ensuring truthful bidding. The Vickrey–Clarke–Groves (VCG) mechanism achieves this with dominant-strategy incentive compatibility: each agent's best strategy is to bid its true cost, regardless of what others do [87].

Reputation systems.

Agents accumulate reputation scores based on the quality of their past contributions. A simple reputation model assigns to agent $i$ a score ${\rho }_i^{(t+1)}=(1-\lambda ){\rho }_i^{(t)}+\lambda q_i^{(t)}$, where $q_i^{(t)}\in [\mathrm{0,1}]$ is the quality of agent $i$'s contribution at round $t$ and $\lambda \in (\mathrm{0,1})$ is a smoothing parameter. The orchestrator preferentially assigns tasks to high-reputation agents, creating a positive feedback loop that incentivises quality. This is a form of the DeGroot learning dynamic (Definition 48) applied to trust.

Contribution-based rewards.

The Shapley value (Definition 33) provides a principled way to distribute system-level reward among agents in proportion to their marginal contributions. By Theorem 9, it is the unique distribution satisfying efficiency, symmetry, null player, and additivity, the fairest possible credit assignment. In practice, exact Shapley computation is exponential, but Monte Carlo approximations [88] make it feasible for moderate $n$.

Biological Intelligence Revisited

We opened this chapter with the observation that multi-agent intelligence pervades the biological world. Having developed the mathematical framework, we can now revisit this parallel with precision.

The immune system as a multi-agent system.

The adaptive immune system is a decentralised MAS with remarkable properties [89]:

• B-cells are specialist agents, each producing antibodies targeted to a specific pathogen. They correspond to specialised agents in a heterogeneous MAS.

• T-cells are orchestrator agents, coordinating the immune response by activating or suppressing B-cells. They implement a form of hierarchical orchestration (Definition 25).

• Memory cells are continual learners, retaining information about past pathogens for decades. They implement the continual learning mechanism ${𝒞ℒ}_i$ of Definition 58.

• Cytokine signalling is the communication protocol $𝒞$: small molecules broadcast information about infection state, coordinating distributed responses.

The immune system achieves collective alignment (fighting pathogens without attacking self-tissue) through a training process (thymic selection) that mirrors the alignment pipeline of ch:alignment. Autoimmune diseases are instances of the collective alignment failure described in Proposition 26: individually “aligned” immune cells whose collective interaction produces harmful outcomes.

The brain as society of modules.

Minsky's Society of Mind [2] proposed that the brain is a multi-agent system in which specialised modules (agents) compete and cooperate to produce coherent behaviour. Modern neuroscience supports this view: prefrontal cortex modules implement planning (reasoning), hippocampus implements memory, the reward system implements alignment (value learning), and synaptic plasticity implements continual learning. The brain's “orchestrator” is the global workspace [90], a broadcast mechanism that selects which module's output reaches consciousness, a biological analogue of the communication protocol $𝒞$ with an attention-based orchestrator $\mathrm{\Omega }$.

Evolution as agent training.

Evolution by natural selection is, in the language of Theorem 15, an agent evolution process: the population of organisms forms a multi-agent system; fitness is the reward function $ℛ$; mutation and crossover implement exploration; and selection pressure drives convergence to high-fitness policies. The replicator dynamics (Definition 44) describe the population-level trajectory. Evolution has been running this multi-agent optimisation for approximately 3.8 billion years, and has produced systems (brains, immune systems, social insect colonies) that remain beyond the reach of current AI.

A Vision for Machine Civilization

We close this chapter, and Part XI of the book, with a reflection on the trajectory from individual intelligence to collective intelligence to civilisational intelligence.

The history of human civilisation is, at its core, a story of scaling multi-agent cooperation. Hunter-gatherer bands of 30–50 individuals gave way to agricultural settlements of thousands, which gave way to nation-states of millions, which gave way to a globally connected civilisation of eight billion. At each scale, new coordination mechanisms were required: language, writing, law, markets, constitutions, international treaties. The mathematics of these mechanisms is the mathematics of this chapter: game theory (Nash equilibrium for strategic interaction), mechanism design (incentive compatibility for truthful cooperation), consensus protocols (convergence of diverse opinions), and credit assignment (fair distribution of collective rewards).

The same trajectory is now unfolding for artificial agents. We have progressed from single-agent systems (a single LLM responding to prompts) to multi-agent systems (teams of LLM agents collaborating on complex tasks) in a remarkably short time. The scaling laws of Conjecture 2 suggest that larger, more diverse agent collectives will outperform smaller ones, not merely through diversity (the Condorcet effect) but through synergy (the cross-term ${(nP)}^{-\gamma }$).

We end with a question. The mathematical structure of cooperation (the Folk theorem, the Shapley value, the consensus dynamics, the Byzantine resilience) suggests that the most capable AI systems of the future will not be solitary superintelligences but collaborative societies of diverse agents, each contributing its unique perspective, coordinated by the mathematical principles of fairness, efficiency, and robustness.

What will it mean to be a scientist in a world where agent societies can do science? The answer, we believe, is that the role of the human scientist will shift from computing to questioning, from solving problems to posing them. The mathematics will be done by agents. The values will remain ours.

Exercises