Research / Study · October 2026 · 6 min read

Training a small modelto run the dock

Two rounds of training on plans the simulator had verified took a 9B open model from 60.8 to 79.8 on fresh loading-dock shifts, past the scripted controller that already runs the dock. About $11 of compute.

Shift replay · cost of delaysEnv · loading-dock / held-outHeld-out scenario, median savingTick 24 / 24 · lower is better

Real episode · Oct 24, 2026

Same 9B, untrained · $6,97078.4Trained 9B · $3,620Scripted controller · $3,800 · 76.5

7 of 24 ticksadded cost · trained 9B

Summary

Our last study showed a 9B open model could plan a shift once a harness carried out the moves, but it still trailed the scripted controller. This time we trained it. On 24 fresh shifts it had never seen, the trained model saved $490 a shift against the controller planning alone, and $727 with the harness picking its moves.

Why this matters

Every label came from the environment: a plan was good because the simulated shift cost less, not because a person or a bigger model said so. That is what a verifiable environment buys you. You can train against it cheaply, and you can check the result on shifts the model has never seen.

01Measure firstA fixed bank of hard shifts the model never trains on, scored by the simulator: dollars of delay, not a judge's opinion. The untrained 9B scored 60.8.
02Round 1: learn from the simulator3,549 plans chosen by searching each practice shift in hindsight. The model learned the format and stopped writing broken plans, but only reached 67.5: hindsight plans assume knowledge it can't have.
03Round 2: learn from its own winsThe model played 240 practice shifts six times each. Where its best try beat the scripted controller, those plans became the training set: 1,309 examples it could actually produce.
04Score once, on fresh shiftsA new bank of 24 shifts, generated after the plan was frozen and scored a single time. 77.9 on its own, 79.8 with the harness picking moves.

Score, 0 = no plan · 100 = hindsight optimum

Same 9B, untrained After training Scripted controller · 73.7

Round 1 · trained on simulator plansRound 2 · trained on its own best shiftsRound 2 · harness picks the moves

60.8 →67.560.8 →77.960.8 →79.8

24 fresh held-out scenarios · scored once

0255075100

Score: 0 = first come, first served; 100 = the hindsight optimum. The same model and server for every row; only the trained weights differ.

Cost

What it cost,mistakes included.

$2.30

Training, two rounds (3.9 GPU-hours on a rented 48 GB card)

$2.20

Self-play, scoring and setup

$6.25

A first attempt, lost when an idle GPU ran out of credit

$10.75

Total

What it means

The first round taught the model the job's format: it stopped writing plans the harness couldn't use. The second round is what crossed the controller. Learning from hindsight plans wasn't enough, because those plans assume knowledge of trucks that haven't arrived. Learning from its own best shifts was, because those are plans it can actually produce. Running the trained model costs about a tenth of a cent a shift.

Limitations

24 held-out shifts; the plan-only gain over the controller is small (95% interval $10 to $991 a shift). On the 12-scenario bank from the last study, the trained model (77.4) is ahead of the controller but not significantly, and about ten points below larger models planning in the same harness (87–88). The base model is Qwen 3.5 9B. Results are specific to this environment.

Data: lora-v1 and lora-v2 episodes, protocols and adapters · under a data agreementRequest the episodes →

Train a modelon your operation.

We build the environment to your task, verify it by replay, and train a small model against it. You get the model, the episodes and the report.