Research / Study · November 2026 · 7 min read

Teaching an agentto manage sub-agents

Training on our environment's decision data cut a 27B open model's project cost by 59% and took it past a frontier model, on projects it had never seen. On an outside benchmark built by someone else, the same model got worse. Both results are here, and so is the reason.

Summary

We built an environment where a lead agent runs a project through a crew of sub-agents, trained Qwen 3.6 27B on 2,932 of its decisions, and scored it once on 24 held-out projects: $370 a project against $902 for the untrained model, ahead of Claude Sonnet 5.5 used with the same prompt. A control trained on the same situations with random decisions got worse than untrained, so the gain comes from the decisions, not the format. Then we ran ClawArena-Team, a third-party benchmark of the same skill: the trained model scored 36.0 against 42.3 for the untrained one.

Why this matters

Agents increasingly work as managers: one model starts others, hands them files and tools, and stitches their answers together. The hard part is not perception, it is judgment. Which model can read this file? Which tools does this job need? Which files should this helper see, and which should it never touch? Most models answer the last question badly: they hand a helper the whole folder, just in case. That habit costs tokens, invites accidents and, in an outside benchmark, is the single weakest axis for every model tested.

01Build the jobA lead agent runs a project through a crew of sub-agents. For each one it picks a model (text, vision or audio), the tools it may use and the files it may see; it writes the brief, checks the work and reruns what came back wrong, blocked or out of date. Everything is priced in dollars.
02Measure firstUntrained, Qwen 3.6 27B handed sub-agents whole folders 76% of the time; only half of what it granted was needed. That is the same habit an outside benchmark flags in this model. Frontier models did better but not well.
03Train on decisions2,932 lead-agent turns from practice projects, each labelled by the environment's best scripted manager: which model, which tools, which files, what to check. No human labels and no output from another model.
04Score once24 fresh projects, frozen before training and scored a single time, then a benchmark we did not write: ClawArena-Team, 41 scenarios with real LLM sub-agents and execution-checked answers.

Score, 0 = no plan · 100 = hindsight optimum

Same 27B, untrained After Scripted controller · 94.3 Claude Sonnet 5.5, prompted · 95% interval

Same 27B · trained on Forge decisionsSame 27B · three worked examples in the promptClaude Sonnet 5.5, prompted · 95% interval

40.3 →78.640.3 →59.450.7–69.9

24 fresh held-out projects · scored once

0255075100

Score: 0 = a careless generalist manager; 100 = the hindsight optimum. Same model and server for both Qwen rows; only the weights or the prompt differ. The scripted line is the manager that wrote the training labels.

The habits

What changed,per project.

$902 → $370

Project cost

49% → 96%

Granted files the sub-agent needed

76% → 6%

Sub-agents given a whole folder

1.3 → 0

Incidents (a sub-agent changed files outside its brief)

2.9 → 1.4

Wrong deliverables shipped

762 → 145

Tokens the lead writes per turn

On a benchmark we did not write

ClawArena-Team (MIT licence, 41 scenarios, answers checked by running code) gives a text-only lead agent a real workspace and a fixed pool of real sub-agents. We ran the untrained and trained model on the same server and the same sub-agent pool; 38 scenarios finished in both (one failed in both for a harness error). The headline Subagent-Management Score fell from 42.3 to 36.0, and the drop holds up per scenario (6.2 points, 95% interval 1.0 to 11.3).

The habit we trained was too narrow for the new setting. In our environment every work item names its files, so the best manager grants exactly those files and only the tools the job uses. In ClawArena the files are not named: a sub-agent has to search for them. The trained model kept cutting search tools and granting narrow file lists, so its sub-agents hit walls almost twice as often (2,571 blocked accesses against 1,362), it created 36% more sub-agents to work around them, and it finished fewer tasks.

That is a result we can act on. The next version of the environment hides some files behind search, so the right grant is sometimes a folder plus search tools, and makes under-granting cost what it costs in real work. Then we score it on ClawArena again, preregistered the same way.

ClawArena-Team

Outside our world,it got worse.

42.3 → 36.0

Subagent-Management Score (higher is better)

150 → 132

Rounds of tasks completed (of 239)

35.0% → 25.4%

Workspace permission precision

1,362 → 2,571

Blocked file accesses by sub-agents

286 → 390

Sub-agents created

Cost

What it cost,mistakes included.

$8.45

Training two adapters (the real one and a random-label control)

$24.06

Serving, held-out scoring and the outside benchmark on rented GPUs

$25.16

Frontier and base-model runs, sub-agent pool calls (OpenRouter)

$57.67

Total, including about $2.40 of GPU time lost to two serving faults

Limitations

One base model (Qwen 3.6 27B), one training run, one environment. The in-environment result is on 24 held-out projects scored once; the outside result is one run per model on 38 ClawArena-Team scenarios, with the benchmark's sub-agent pool served partly through OpenRouter and an audio-model fault in the first part of the run that affected both models equally. Our untrained reproduction scored 42.3 against the published 46.4. Results do not show that the trained model is a better manager outside this environment; on the outside test it was worse.

Data: crew-chief episodes, labels, protocol and adapters · under a data agreementRequest the episodes →

Train your agentson your operation.

We build the environment to your task, verify it by replay, and train a model against it. You get the model, the decision data and the report.