How small local models finish hour-long agent tasks

On 2 October 2026 I asked my local AI agent to review its own source code. A sub-agent worked for 1,400 seconds, made 36 tool calls and returned 481 characters of raw tool-call markup with no findings. The parent agent then wrote a 10 KB security report anyway: a critical shell injection with quoted code that does not exist, a file that does not exist, and “missing” tests that were all there. It sent the report to my phone without being asked.

Five days later the same 9B and 27B models were finishing research projects of 66 minutes with real citations. The models did not change. The scaffolding around them did. This post is what that scaffolding is.

Contents · 9 sections

TL;DR #

• Small local models (9B to 35B) cannot plan and track a long project on their own. Both skipped the start step and jumped to “finalize”.
• Give them a durable task tree created from a template, and let them do one bounded task at a time.
• Refuse a task completion that no tool call backs, and refuse a claimed file that is not on disk.
• Treat an empty or markup-only sub-agent result as FAILED, explicitly, or the parent will fill the gap with fiction.
• Do data gathering deterministically in code; use the model for analysis, writing and criticism.
• Plan around the shared KV cache, not the per-request context.

The setup #

lsn-agent-v3, my local AI agent, runs on one machine against a llama.cpp router with a 9B model (qwen3.5-9b), a 27B and a 35B. The 9B is always loaded and is shared with other pipelines on the same GPU. The agent can delegate to sub-agents (“workers”) with roles such as reader, researcher, analyst, critic and writer.

For short questions this works. For anything that takes more than a few rounds, the failure modes were consistent.

What went wrong, run by run #

The self-review was my stress test. Over several runs on 2 and 3 October I recorded each failure:

1. Fabrication on empty input. The analyst sub-agent’s final answer was tool-call markup. On its last round the agent sent no tools and asked for an answer, but the model kept “calling tools” as text in its native XML format, which nobody parsed. The parent treated that as a result and invented a report around it.
2. Tasks closed without work. In run 3 the 9B orchestrator called project_complete_task 7 times in 2 minutes with no tool calls in between, pasting text from the roadmap or invented results (“3 test failures”; no tests had run).
3. No plan at all. On a complex project, both the 9B and the 35B skipped project_start and called project_finalize first.
4. Answering from memory, not from the report. After finalizing, the orchestrator summarised from its own context and listed 3 critical issues that the finalized report had rejected with code evidence. The finalize tool returned only a path, a word count and a hash, so the real text never reached the model.
5. Reasoning ate the answer. A 35B critic made 73 tool calls, spent about 7,400 tokens of an 8,192-token output budget on reasoning, and produced no answer text.
6. Loops on impossible tasks. A worker without an execution tool was asked to run a script. Instead of saying so, it wrote the same two files 17 times in 3 minutes.
7. Defences read as defects. The 9B code auditor turned comments that describe protections into bugs: a comment about dropping “phantom” dependencies became a “phantom writes” finding.

A scored run on one module found 2 of 12 known defects, with about 50 % of its claims correct. Not a reviewer you can trust alone.

The fix: the model does tasks, the code runs the project #

A durable task tree from a template #

Projects now live in the database, above individual plans and sub-agent runs. A project has a goal, a rolling state and a task tree. Each project kind carries a template, and project_start creates the tree, so the model never has to invent the structure:

"incident": (
  task("Symptom and scope",   "Exact error text, first/last seen (UTC), what is affected. Write symptom.md.",
       phase="gather", role="ops"),
  task("Evidence sweep",      "Read-only: pod status, logs, describe, events, journal. Write evidence.md.",
       phase="gather", role="ops", depends_on=(0,)),
  task("Causal chain",        "Each link tied to an evidence item; what was ruled out. Write cause.md.",
       phase="work", role="critic", depends_on=(1,)),
  task("Fix proposal",        "Split APPLIED (reversible) from PROPOSED (owner). Write fix.md.",
       phase="work", role="ops", depends_on=(2,)),
  task("Verification",        "After-state check with commands. Write prevention.md.",
       phase="verify", role="ops", depends_on=(3,)),
),

Other kinds (dossier, investigation, migration, data campaign, deploy) have their own templates. next_task is dependency-gated, and resuming a project rebuilds the context from goal, rolling state, task tree and next task only, never from the chat history. A small model with a short window cannot lose the thread, because the thread is not in its window.

A completion needs evidence #

The completion tool keeps a ledger of successful tool calls per session. A task can be marked done only if real work happened since it started, or a fresh file was written:

work = successful_tool_calls_since(task_started)   # excluding project bookkeeping
if not work and not fresh_artifact:
    return ("[NOT completed] no tool ran since this task started. Do the task, "
            "then complete it with what that output showed. A result must never "
            "be written from memory or the roadmap alone.")
if artifact and not os.path.isfile(artifact):
    return "[NOT completed] the artifact does not exist on disk; re-run the task."

The second check is there because small workers regularly report “written to X” when X was never written. Results that read like a failure (“error”, “could not”) are refused too, with a hint to mark the task blocked instead. Finalize refuses a different kind from the one the project started with.

Failed means failed #

A sub-agent result that is empty, only tool-call markup, or only a preamble is now returned as [SUB-AGENT FAILED] with a note that there were no findings. On the final round, markup in the answer triggers one retry with tools disabled and a summarise-only prompt, then [NO-ANSWER]. Reasoning got its own token budget, with a retry without reasoning when the answer comes back empty. And finalize now returns the report’s executive summary and outline, with an instruction to answer from that text.

Deterministic gathering #

For evidence-heavy work like a threat-intel dossier, I stopped asking the model to collect data. Direct tool calls in code gather the facts in about 10 seconds, reliably. Through model tool use, the 9B produced phantom files and the 35B wrote 1 of 4 sections in 57 minutes. The model now does what it is good at: writing from collected evidence (editor) and attacking it (critic).

The same idea applies to small things. Status questions are answered by a live check, not a search through history: after three rounds of history-only lookups, the agent gets one nudge to check live state directly.

Memory is a shared pool #

The other limit is physical. llama.cpp gives the 9B a unified KV cache of 131,072 tokens shared across all its slots. Four workers at 32,768 tokens each already fill it; an orchestrator plus four workers over-subscribes it. So the delegation dial is:

context per worker  x  parallel workers  <=  shared KV pool
32,768              x  4                 =   131,072

Parallel sub-agents are capped at 4 global permits on the shared 9B, and other pipelines use the same model. “Helper waiting” in the UI means queued behind those permits, not deadlocked. Long documents are read in windows that stay below the size at which tool output is spilled to a file, and the agent tracks the free context live.

What it can do now #

A campaign of five research projects on 7 and 8 October, all on local models with the library, an offline Wikipedia and web search:

• a thyroid health report from 3 books (one with 1.1 million characters of raw text), 9 offline references and 23 dated web sources: 66 minutes;
• a Python concurrency guide from two large books, the official docs and offline PEPs: 19 minutes;
• a report on Romania in 1989 from two books (1.7 million and 393,000 characters), Romanian Wikipedia and news searches, keeping facts and contested claims apart: 46 minutes;
• an essay comparing three books, with section citations: 21 minutes.

Each failed attempt along the way produced a fix: Romanian diacritics split by an ASCII tokenizer in tool routing, a digest cache, a coverage check, a guard against re-reading the same window.

For short live questions, an eval on 19 October with 8 real questions scored the 9B at 6 of 8 (median 94 seconds) and the 27B at 7 of 8 (median 64 seconds). Both timed out on one status question despite having the right tools: each model round re-read about 10,000 tokens on a busy GPU.

Checklist #

• Keep the project plan in code or a database, not in the model’s context.
• Create the task tree from a template per kind of work.
• Resume from goal + state + tree, never from chat history.
• Refuse completions without tool evidence, and verify claimed files exist.
• Mark empty or markup-only results as FAILED in so many words.
• Give reasoning a separate budget from the answer.
• Gather data deterministically; let the model write and criticise.
• Size parallel workers to the shared KV pool.

Frequently asked questions #

Can a 9B local model run a long agent task? #

Yes, if it never has to hold the whole project in its head. With a task tree created by code, one bounded task at a time and checks on every completion, a 9B model finished research projects of up to 66 minutes. Left to plan alone, it skipped steps and invented results.

Why do small models fabricate results? #

Because an empty input looks like a gap to fill. When a sub-agent returned only markup, the parent model wrote a plausible report around nothing. Marking that result explicitly as failed, with “no findings”, stopped it.

How many sub-agents can run in parallel on one GPU? #

As many as fit in the shared KV cache. With llama.cpp’s unified cache, context per worker times the number of workers must stay below the pool size; in my case 4 workers at 32,768 tokens fill a 131,072-token pool.

Should the model call the data tools itself? #

For exploratory questions, yes. For repeatable evidence collection, no: code is faster and never invents a file. Use the model for analysis, writing and adversarial review.

The lesson #

The model is not the project manager. It is a worker that is good at one bounded task when the inputs are in front of it, and bad at remembering what it promised twenty minutes ago. Put the plan, the dependencies and the definition of “done” in code, refuse every claim without evidence, and say “failed” out loud when a worker comes back empty. Small models then do long work, slowly and honestly.