Measured, then published.
Limits included.
Controlled studies, benchmarks, papers and writeups. Each one names its limits and links its data.
When does a pilot need a crew?
Puppetmaster against Codex’s own subagents. Same model, same effort, only the crew changed. Ran it at 16 regions, then at 64. Follow-up 71.6% and 85.1% cheaper. Blind judges 9 to 0 and 42 to 2. Cold wall time: a wash both times.
Open the study →
64-region world · Puppetmaster · pair 4 follow-upStudies and benchmarks 4
Hard Proof: no repro, no bug
A release gate for code that AI agents write. A review finding counts only with a repro that a script runs, and a release waits until each confirmed bug is closed. On one release range hpf confirmed 8 bugs and two AI review tools confirmed 2 each. A proof is the second tool, for a small rule that the owner states. Draft for review.
PaperDataArtifactsStrongOrc
Which models keep delegated work intact? Measured orchestration reliability against total spend, as orchestrator and as worker.
RepoDocsGrok 4.6 independent bench
Independent DeepSWE v1.1 and Terminal-Bench v3.0 runs of Grok 4.6 High, checked against the vendor card.
Puppetmaster on SWE-bench Lite
Three arms, same model ceiling. CodeGraph plus routing came in about 47% cheaper than the frontier-only arm.
Papers 6
Same model, different crew
The 16-region study in full: frozen rules, a calibrated blind judge pair, four pairs, every disclosure, and what it says about when to fan out.
StudyDataExtractive Residuals After Compaction
What a harness should keep after compaction: a bounded handle catalog, a last-wins story and an FTS vault, scored without an LLM judge.
PDFFable 5.1 noteCodeSubtraction is conditional
Deletion-first prompting in agentic refactors. 240 cells: an interaction between task and model, not a universal law.
PDFCodeDurable Autoresearch
Durable research memory on medium GPUs (24 GB and under): fingerprints, verification and negative memory. About 20% GPU time saved.
Durable state makes a workday cheap
Query completed work before inference. When 1 in 20 turns is novel, the day costs 5% of the model calls.
State, Not Tokens
Repository-scale agent reasoning is bound by state architecture, not nominal context length. A controlled study.
PDFCode
