EXPERIMENT_044 // LOCAL.EDGE-MODEL.AGENT.ROUTER.(MINICPM5-2B)
A self-hosted 2-billion-parameter MiniCPM5-2B instance triages 16 real tasks, deciding per task whether it can answer directly or must escalate to the fleet's cloud model, with every task's real cloud cost and latency measured regardless of the router's decision — a genuine before/after comparison, not an estimate.
LOADING EXPERIMENT...
Extending the local-inference/cost-optimisation thread EXP_038 (Local Qwen3.8-27B Agent-Routing Benchmark) started, this experiment goes a step smaller and a step earlier in the pipeline: instead of asking whether a 27B local model can replace one cloud call outright, it asks whether a 2B edge model — Artificial Analysis's #1-ranked model in the <=4B open-weights class, Intelligence Index 13 — is capable enough to make the cheap first-pass decision of whether a cloud call is needed at all. 16 fixed tasks (8 designed to be answerable with a sentence of general knowledge, 8 designed to need multi-step reasoning or architectural trade-off analysis) run through a local MiniCPM5-2B router and, unconditionally, through the fleet's default cloud model, so the real cost and latency of every task is measured under both conditions — not just the ones the router actually escalated.
HOW IT WORKS
Each task is sent once to a self-hosted openbmb/minicpm5-2b instance over a direct Ollama /api/chat call (the toolkit @diabolicallabs/llm-client has no Ollama/local provider, and the fleet's local-mcp-service gateway has no MiniCPM entry — the same documented judgment call EXP_038 made for Qwen3.8-27B, applied here). The model is instructed to answer directly in a sentence or two if it can, or to reply ESCALATE if the task needs more than that — so the router's own output, when it stays local, is the actual answer, not a separate classification step followed by a second local call.
Regardless of what the router decides, every task is also sent to the fleet's default cloud model (gpt-5.4-mini via @diabolicallabs/llm-client) unconditionally. This is what makes the 'before' figure (every task sent straight to cloud) a real, measured total across all 16 tasks rather than an estimate extrapolated from the escalated subset — the same rigor EXP_038's two-arms-always-run structure already established, repurposed here to build a genuine before/after comparison instead of an accuracy comparison.
A router call that errors (Ollama unreachable, malformed response) is treated as an escalation, never as a silent direct answer — the same 'safe default over cheap default' posture EXP_018's Reasoning Budget Router established for its own classifier-failure path. report.ts's buildRouterSummary derives the actual after-total by adding the real cloud latency/cost back in for every task the router escalated to or failed on, using each task's own real cloud-arm measurement — never a per-task average applied uniformly.
On this committed run, the router matched the design-time expected route on 68.8% of the 16 tasks (5 answered directly, 11 escalated), saved 19.1% on total cost, but *increased* total latency by 16.9% — every escalated task pays the local router's latency and then the full cloud call's latency sequentially, so any wrong call in either direction is a pure cost, not a wash. Reported plainly rather than reframed, matching the repo's established 'publish whatever the data says' posture from EXP_027 and EXP_038.
WHAT THIS PROVES
A 2B edge model's routing mistakes are informative on their own, not just its accuracy score: this run's misclassifications ran in both directions — several simple tasks were escalated unnecessarily, and at least one task designed to need deep multi-step reasoning was answered directly instead — which is a materially different (and more honest) finding than a single aggregate accuracy number would show, and exactly why this experiment's per-task table is published in full rather than summarised away.
A router's real cost/latency trade-off can only be measured honestly by running the unconditional baseline alongside it: because every task in this run also got a real cloud answer regardless of the router's decision, the 19.1% cost saving and the 16.9% latency increase are both measured from the same fixed task set, not estimated from a partial sample — and the two numbers pointing in opposite directions is itself the finding, not a flaw in the measurement.