The_Maintenance_Gap

In a June 2026 analysis of agent maintenance patterns, practitioner Nate B. Jones outlined seven failure surfaces that show up in production agent systems. Reading it, I recognised every one of them. Not because they described hypothetical failure modes I might encounter, but because I had already encountered all seven - some of them multiple times, some of them in the same incident.

This is what the maintenance gap looks like from inside it. You build the agent carefully. You test the happy path. You verify the outputs are correct. And then you deploy it and move on - because there are eleven other agents waiting for attention, and the one you just shipped appears to be working.

The problem is that "appears to be working" is doing an enormous amount of structural weight-bearing in that sentence. Production multi-agent systems don't fail cleanly. They drift. The output keeps coming. It looks plausible. It passes whatever review gates you have set up. And the drift accumulates until it becomes a failure, at which point you are not dealing with a fresh incident - you are dealing with weeks of compounded divergence from what the system was supposed to be doing.

Most agent builds get months of engineering attention before launch and almost none after. What I'm documenting here is the monitoring layer I built to address that asymmetry: what I watch, what has already gone wrong in each area, and where the gaps still are.

The_Seven_Failure_Surfaces

1. Prompt drift

Prompt drift is what happens when the behaviour of an agent changes without the instructions changing. The instructions are the same; the model is the same; the outputs are different. This is subtle and difficult to detect precisely because nothing you would ordinarily check has changed.

The model tiering inheritance bug ran in my fleet for several weeks before anything surfaced it. Every subagent in the fleet was running on Opus - the most capable, most expensive model in the range - instead of the Sonnet and Haiku tiers assigned to them. The agent definitions had the right tiers in their frontmatter. The SKILL.md files were correct. The problem was that the model parameter was never being passed at spawn time, so every subagent inherited the parent session's model regardless of what its definition said. One hundred and fourteen agent sessions consumed 11.2 million output tokens on Opus when they should have cost a fraction of that. The register shifted. The reasoning traces grew longer. The tone changed. Nothing threw an error.

The signal that caught it was cost telemetry, not output quality review. Usage data showed 11.2 million output tokens where roughly 2 million were expected. The fix took one rule change. The monitoring gap - the thing that would have caught it in session one rather than session 114 - is a real-time tier verification check at spawn time. That check still does not exist in my fleet. It is documented and unresolved.

2. Tool-call degradation

An agent that cannot use its tools correctly will continue to produce output. It will not tell you the tools are broken. The outputs will look like normal outputs from an agent that has run into a wall and decided to navigate around it by generating plausible approximations instead.

The Perplexity tool namespace bug affected all twelve agents simultaneously. A namespace reorganisation removed a dual-registration that had been in place. All twelve agent definition files were still referencing the old namespace - perplexity.*. The actual tools were now registered as mcp__perplexity-web__*. Every agent that used Perplexity was silently calling a non-existent tool name without error, because the runtime was not throwing on unknown tools in that configuration. Agents that were supposed to ground their outputs in verified web sources were generating outputs without that grounding, and nothing in the output signalled that anything had changed. Caught during a June 2026 audit. Corrected across all twelve files simultaneously.

There is a second-order implication here worth naming: more tools in scope means more surface area for this class of failure. Jones's June 2026 analysis noted that removing 80% of an agent's tool set improved its reliability. Fewer tools means fewer namespace dependencies means fewer silent degradation paths. I have been moving toward leaner tool profiles per agent as a result.

3. Context accumulation

Every session that runs long enough approaches its context limit. What happens at that limit - or before it, under automatic compaction - is one of the more underappreciated failure surfaces in production systems.

The worktree CWD-reset bug, which produced a ghost commit incident in May 2026, is the clearest example in my fleet. Worktree agents use a separate working directory from the main repository. Under context compaction - the automatic lossy compression that fires when a session approaches its context limit - the agent's working directory state was dropped. The runtime's "memory" of where the agent was writing reset to the main repo. File writes that were supposed to land in the worktree landed in the main working tree instead. The agent continued executing. Nothing threw an error. The failure only became visible when git status showed unexpected changes in the wrong location.

Context degradation is measurable: roughly 2% quality loss per 100,000 tokens of conversation. The default auto-compact triggers at 75-80% of context capacity, which is too late. The discipline I now apply across the fleet is compaction at 60-65%, with explicit preservation instructions. This does not eliminate the failure surface - it shrinks it.

4. Output format drift

This is the failure surface most easily confused with prompt drift, and it is worth separating them. Prompt drift is when an agent's behaviour changes while its instructions stay the same. Output format drift is when an agent's output shape changes in ways that downstream consumers do not handle gracefully - and neither the agent nor the consumer surfaces an error.

The model tiering inheritance bug had a second manifestation that was quieter than the cost spike. Opus produces longer outputs than Sonnet. Its reasoning traces are more elaborate. Its register is different. Downstream agents that were parsing structured fields from those outputs - expecting compact, Sonnet-shaped responses - received outputs that had expanded beyond the expected shape. The fields were there. The structure was technically valid. But the expanded format meant the consuming agents were working harder to extract the same information, and the probability of parse failures was elevated. No validation error fired. The drift was silent.

The monitoring implication is a schema validation check at agent handoff boundaries - not just "did we receive output" but "does the output conform to the expected shape." I have this in place for some handoffs and not others. The ones that do not have it yet are the ones I am watching most closely.

5. Dependency failure propagation

An agent can execute correctly within its specified scope and still produce downstream failures if the dependencies it touches are coupled to systems the brief did not account for. This is a specification problem, not an agent problem, but the propagation mechanism is identical: a change scoped correctly to what the agent was briefed on produces a failure in a system the brief never mentioned, because nobody had documented that the two were coupled.

Sable ran a cross-project audit in mid-2026 and diagnosed a version-drift incident as CI silent-push-failure. She recommended upgrading @semantic-release/git to v13 across five repositories. Wave 2 briefs were approved for four of them. Sable ran a pre-flight investigation before any code was modified - and found that v13 does not exist on npm, and the original diagnosis was wrong. The drift was a stale local clone, not a CI failure. No code was modified in any of the five repositories. The cascade was caught at the brief-approval stage, not at execution. The pre-flight investigation is now a mandatory step before any wave that touches more than two repositories.

6. Silent error continuation

This is the failure surface that has produced the most expensive incidents in my fleet. Not "most expensive" in compute terms - in human recovery time.

The Sanity seed script incident. Task: rename project titles during a rebrand. Simple. The script ran. Titles updated. The PR looked clean. What the review missed was that createOrReplace does not update fields - it replaces the entire document. Every heroImage, gallery, and mediaGallery reference across eight project documents silently zeroed out. Assets survived in Sanity. References were gone. The agent continued running. Nothing threw an error. The review examined what had changed, not what had disappeared. Caught by the Owner's manual final-pass inspection. Restored from Sanity revision history.

A different class of the same failure: in June 2026, a batch of ten drafted content pieces came through the pipeline. Five of them carried fabricated first-person anecdotes presented as Diana's real experience - a lived encounter the brief had requested without actually supplying. The agent was following the brief's instruction: lead with a lived encounter. The instruction could not be satisfied without inventing, so the agent invented. The post passed Reid's voice-check, which reviews register and vocabulary, not factual accuracy. There was no dedicated fact-verification step in the path at all. The batch was caught during review before publication.

The consistent pattern across both incidents: in the Sanity case, the agent executed its instruction correctly but the instruction itself failed to account for what deletion meant in that system's semantics. In the anecdote case, the agent faced an instruction that could not be satisfied without fabrication, and no validation layer existed to check whether the instruction's request matched actual supplied context. The failure was in what the instruction did not specify, and in the absence of a validation layer that looked for what was missing rather than what was present.

7. Performance degradation under load

The subtlest failure surface in this list, because it does not produce clean errors. Latency rises. Timeout rates increase. Individual calls appear to succeed - or appear to fail in ways that look like network noise - while the aggregate picture is a system that is no longer performing within its operating parameters.

EXP_009, the Agentic Reliability Dashboard, tracks pass rates and latency across models on standardised tasks. In one documented episode, Gemini 3.1 Pro was returning missing_api_key errors instead of responses - API availability degradation masquerading as model failure. On an individual call, that error looks like a configuration problem. In aggregate, it looks like a reliability collapse on a specific provider. The dashboard makes the aggregate visible. Without it, the diagnosis process is manual, slow, and likely to reach the wrong conclusion.

What_"Mostly_Fine"_Actually_Means

My recovery layer is almost entirely human-in-the-loop. When something goes wrong, I notice it, diagnose it, and fix it. The system does not pause itself. It does not escalate automatically. It does not have a recovery protocol that runs without my involvement. That is not a design choice - it is a gap.

What I do have are the components of a circuit breaker, implemented manually. Trust tier classification - track record × reversibility × blast radius - determines which agents route through Owner review before execution. It is not a runtime interrupt; it is a routing classification that governs what happens with output, not whether an agent can be stopped mid-run. MAINTENANCE_MODE environment variables at all LLM call sites in the main engine can be toggled via Railway without a code deploy. A global_halt feature flag in GEOAudit gates all new runs through a single chokepoint. The gates_on: frontmatter field enforces load-bearing review gates - any gate that appears only in prose does not stop an autonomous run.

These mechanisms collectively approximate what an explicit circuit breaker would do. They are not the same thing. The gap between "a human can stop this" and "the system stops itself when conditions exceed threshold" is real and I am aware of it.

The design target for my fleet is local failure, not zero failure. Zero failure is not achievable at fleet scale. Local failure - a failure that stays in the agent where it started rather than propagating into the agents that depend on it - is achievable with validation layers at agent boundaries, schema checks at handoff points, and fallback chains that degrade gracefully rather than cascading. Not all of those are in place. The ones that are not are the ones the incidents above exposed.

The_One_Signal_That_Matters_Most

EXP_004, the nightly specification-drift audit, runs across all repositories every night. It checks for stale manifests - no update in 90 days - failing CI, uncommitted worktree state, and branch hygiene violations. No human in the loop. What it surfaces is context staleness: the gap between what an agent's specification says it should do and what the repository's current state actually requires.

The reason this is a leading indicator rather than a lagging one: specification staleness precedes the failure. A manifest that has not been updated in 90 days is not a failure. It is the condition under which a future failure becomes more likely - because the agent operating from that manifest is working from a description of the world that no longer fully matches the world. EXP_004 surfaces that condition before it produces an incident. I built it after the model tiering inheritance bug - not as a direct response, but because that incident made visible how long a problem could run silently in a system I thought I was watching.

The secondary signal is cost telemetry. The model tiering inheritance bug ran for weeks before the usage data surfaced it. No output quality review would have caught it - Opus is not wrong, it is just not what was specified. Cost is a leading indicator for model tier drift in a way that output quality cannot be, because the failure mode produces better-than-expected output, not worse.

The signal EXP_004 does not yet provide: real-time model tier verification at agent spawn time. An alert that fires when the requested model does not match the actual model at spawn would have caught the model tiering inheritance bug in one session rather than 114. That check is documented. It is not yet built. Until it is, the fleet's primary telemetry for that specific failure surface is retrospective - which is exactly the problem this article is about.

The monitoring layer is not finished. It is more capable than it was six months ago, and it is less capable than it needs to be. Most of what I know about where the gaps are, I know because something failed inside them.