Evidence¶
What deputy saves and what it costs in quality, measured on 2026-11-18 with the current version. Savings use the yardstick described in configuration: bytes divided by four on both sides, so a difference is an order of magnitude, not an invoice.
- Worker:
Jackrong/Qwen3.8-27B-MTP-GGUF:Q4_K_M, served locally by llama.cpp on a laptop. - Frontier references: Claude Opus 5.5 (
claude-opus-5-5) and Claude Sonnet 5.5 (claude-sonnet-5-5), each recorded separately.
Everyday tasks on two real repositories¶
21 tasks of the kind an agent hands off during ordinary work, each with checks: 12 on a private multi-app TypeScript and Python repository (some of its documents are in Italian), 9 on this repository. Inventories, summaries with cited lines, a section list translated from Italian, git history, code questions, counts, searches and one-line lookups. One correction round allowed. The set was run twice, with the current worker profile and tools on the second run.
Quality. First run: all 21 answers passed their checks, and Claude Opus 5.5 compared every
answer with the files by hand, line numbers included: all 21 were correct, one cited range off by
one line. Second run: 20 passed; the 21st was correct but failed a check written with a line limit
that was out of date (the file had grown). cites_lines now takes the file's path and reads its
current length, so that check cannot go stale.
Saving, net of what the orchestrator paid for the request and the result:
| Kind of task | Tasks | Saved, run 1 | Saved, run 2 | Worker time per task |
|---|---|---|---|---|
| Reading: summaries, inventories, translated sections, history, code questions | 11 | 52,751 | 61,767 | about 2 minutes |
| Lookups and searches: one line, a commit subject, which files mention X | 10 | 2,400 | 10,833 | 30 to 60 s |
A reading task saves about 4,800 to 5,600 frontier tokens. The lookups' figure varies because it counts what the worker happened to read: on the second run it read more around a few short answers. What an orchestrator would read for a lookup is a line or two either way.
Reading pays; lookups and searches do not, because the orchestrator would see the same short
result itself. That is the rule in the delegation guide, and why a task that names only small files
is flagged small_input.
The worker profile¶
The 10 fast tasks of the built-in suite, 3 runs each (30 runs per profile), on the local worker:
| Profile | Passed | Citations correct | Mean time | Output tokens |
|---|---|---|---|---|
| Every tool named, one phrase each (the bundled profile) | 100 % | 100 % | 23.5 s | 265 |
| Same, with three tools named | 100 % | 100 % | 28.3 s | 320 |
| Same as the first, plus seven rules from public prompt collections | 96.7 % | 83 % | 27.0 s | 304 |
Telling a small model which tools it has makes it faster at equal quality. Adding rules made this 27B model worse, so the profile stays short; those ideas went into the orchestrator's guide instead.
On the same tasks the frontier models, one pass each by hand: Claude Opus 5.5 passed 10 of 10 (about 9,380 tokens to read and answer), Claude Sonnet 5.5 passed the first 8 of 8 (about 9,270). Delegating them costs the orchestrator only the request and the result.
Worker tools that pay for themselves¶
Each measured on the local worker, with and without the tool:
| Tool | Task | Without | With |
|---|---|---|---|
list_files |
how many note files, and which is the largest | 1 of 3 passed, 127 s | 3 of 3, 14 s |
summarise_log |
which worker logged most errors in a 2,400-line log | 2 of 2, 69 s | 2 of 2, 29 s |
read-only git |
which commit changed a rule, the old value, and why | not possible | 3 of 3, 49 s |
summarise_log keeps every distinct kind of line: a 2,400-line log of about 34,500 tokens becomes
about 120, and a real 215-line CI log keeps all 176 kinds of line in about 3,000 tokens instead of
7,700. A search with many hits lists every matching file with its count: 66 hits in 29 files cost
about 490 tokens.
Where an agent's reading goes, and rtk¶
deputy discover on 11 Claude Code sessions of one real repository: about 5.2 million tokens of
tool results read by the orchestrator. Reads (files, searches, listings) were 62 %, and large reads,
over about 1,000 tokens each, 30 % of everything: that is what delegation can take. Command output
(git, gh, test runners) was 15 %.
That 15 % is what rtk (1.2.0, run unmodified) compresses. On read-only commands of two real repositories:
| Command | Raw | With rtk | Saved |
|---|---|---|---|
git diff over 5 commits |
139,031 | 8,182 | 94 % |
git log -30 |
2,451 | 843 | 65 % |
ls -la of a source folder |
661 | 148 | 77 % |
gh run list |
583 | 219 | 62 % |
grep -rn, pytest -q, ruff check |
4 to 2,305 | same | 0 % |
The two tools cut different parts of the bill and do not interfere: rtk shortens what shell commands print; deputy takes long reads off the orchestrator, and its worker never uses the shell.
What this does not show¶
- Savings are estimates with one yardstick on both sides; real tokenizers differ by up to about a quarter, and prompt caching changes what a re-read costs.
- 21 tasks on two repositories, two runs; the profile and tool comparisons have 2 or 3 runs. The numbers show direction and order of magnitude.
- The frontier references are one pass by hand, not repeated measurements.
- Worker time is not free for you: a local 27B model needs one to three minutes for a long document.
- Answers are only as checked as their checks; Claude Opus 5.5 reviewed every answer of both runs against the files they were given.
A nightly benchmark runs on the maintainer's machine (local serving);
deputy bench --compare shows what changed between two nights.