Agent context grows unbounded on long runs (no compaction)

[lfnothias]

Problem

On a multi-step reproduction run the execution agent's context grew to ~866k input tokens per step. The run crawled (1h15m+ without converging) and was expensive. There is currently no context management for the agent: smolagent_factory runs with max_steps=128 and no compaction or token budget, so a long run accumulates its full step history and re-sends it every step.

Root cause

The agent (smolagents) keeps every step in memory.steps and rebuilds the full message list on each step (standard ReAct). Over dozens of steps the live context grows roughly linearly, dominated by accumulated tool outputs and generated code. This is intra-run accumulation, distinct from the cross-run load_agent_memory replay (which only prepends one matched prior step).

Evidence

• Haffner reproduction run: per-step input tokens climbed past 800k around step ~33; stopped at 1h15m without converging.

Why it matters

• Cost: input tokens dominate the bill on long agentic runs.
• Latency: every step carries the whole history.
• Reliability: approaches model context limits.

Proposed direction (for discussion)

Add lightweight memory management / context compaction to the agent loop, inspired by coding agents (opencode / Claude Code): once the context crosses a threshold, summarize or drop the oldest steps while keeping recent steps verbatim plus a running summary, then continue. Start minimal (e.g. truncate old tool outputs first, then compact older steps), measure, iterate.

This is the alternative to the hard max_steps / context-window guard that was dropped in the #158 review: rather than capping or aborting, compact and continue.

Open questions

• Hook point: a smolagents step callback vs a wrapper around memory.steps?
• Strategy: deterministic truncation of old tool outputs first, then an LLM summary of older steps?
• Threshold, and how many recent steps to keep verbatim?
• Which model summarizes (cheap tier)?

[Fosowl]

I'd push back on compaction, the problem is real but this is the wrong fix for Mimosa.
Compaction is a boundary: Summarize-and-continue is a lossy operation, exactly what the framework is designed to avoid. Except this boundary is placed by a token counter (so almost certainly mid-basin, the worst spot) and it's invisible: the workflow graph says "one agent" but the effective topology has hidden cuts the judge and meta-orchestrator can't see.
It's also worse for us as scientific reasoning are full of hypotheses under revision, hedged conclusions, negative results etc...
Most importantly, a context ballooning to 866k over 33 steps is a signal: the task needed a boundary the workflow doesn't have. Compaction hides that signal.
One possible alternative would be to have a penalty that grow proportional to the dispersity between agents final context lenght as that would create a pressure that reward workflows that balance context between agents

[lfnothias]

Your objection is right and I withdraw the compaction proposal.

The point that convinces me is the third one: a context ballooning to 866k over 33 steps is a signal that the task needed a boundary the workflow does not have. Compaction would remove the symptom and leave the cause, and it would do so by placing a lossy cut where a token counter happens to fire rather than where the reasoning permits one. For scientific work, where hypotheses are under revision and negative results carry information, a summarise-and-continue step also silently discards exactly the material a judge would need. And the invisibility argument is the strongest of all: an effective topology with hidden cuts that neither the judge nor the meta-orchestrator can see is worse than a workflow that is honestly too large.

Your alternative is a better fit for the framework, because it turns the problem into selection pressure rather than a mechanism. If the penalty grows with the dispersity of the agents' final context lengths, workflows that pile everything onto one agent lose ranking points against workflows that distribute the work, and the search is free to discover the boundary itself instead of having one imposed.

There is already a natural place for it. selection.py composes qd_score as (1 - w)·quality + w·novelty − λ·length_penalty, where _length_penalty is a clipped relative excess in [0, 1]. A dispersity term has the same shape and can be subtracted alongside it with its own lambda, so the existing structure does not need to change.

Two things I would rather agree with you before sending a pull request:

First, the measure. A coefficient of variation over the per-agent context lengths is scale free and behaves well when agents are many; a simpler (max − min) / max is easier to read and reacts strongly to a single dominant agent, which is the case you care about. I lean towards the coefficient of variation, normalised and clipped to [0, 1] like the length penalty, but I do not have a strong view.

Second, the signal. The per-agent context length is available from the agent memory that cost accounting already walks. I would attach the per-agent lengths to the run and read them in selection the same way _genotype_chars reads run.code, rather than have the selector touch the filesystem.

I would default the new lambda to zero so nothing changes until it is set deliberately, then we can look at whether the pressure actually moves the population before making it the default. If that plan sounds right to you I will implement it that way.

[lfnothias]

Sent as #176. It follows the plan above, with one change I would flag: a plain clipped coefficient of variation saturates before the extreme, so [10,10,10,900] and [1,1,1,900] clip to the same value and leave no gradient in the severe regime. Dividing by sqrt(n-1), the coefficient's maximum for n agents, keeps the range [0,1] and restores the gradient.

Two limitations are stated in the pull request and asserted in tests rather than hidden: the measure is scale free, so equally large contexts are not penalised, and the pressure can in principle be satisfied by raising the floor rather than lowering the ceiling. Whether to add a term sensitive to absolute context is your call, and I did not assume it.

Lambda defaults to zero, and the dispersity is reported in the validation result and the log line even when the penalty is off, so the signal is visible before anyone decides to act on it.