culprit run · Culprit docs

Step 2. Find the first commit between a good ref and a bad ref that made your agent worse.

Terminal

culprit run --good v1.0 --bad HEAD --budget-usd 3

--good is any ref (tag, branch or commit) where the agent was good; --bad is one where it's worse. Everything else comes from culprit.toml.

What you'll see#

Output

run 77f60c9fccf8: 14 commits in v1.0..HEAD, 10 tasks × 3
  screen ×1: good 9/10, bad 5/10; full k=3 on 5 tasks
  good 19/20 CI [0.76, 0.99]  bad 6/20 CI [0.15, 0.52]
  regression gate: one-sided Fisher p=0.0000 (alpha 0.05)
  discriminating tasks: t02, t04, t05, t01, t03
  4ebe03f827 install failed → SKIP
  464e05fa11 'revert requests pin': 14/15 CI [0.70, 0.99] vs t=0.50 → GOOD
  462db36ee1 'docs: maintenance note 11': 1/15 CI [0.01, 0.30] vs t=0.50 → BAD
  a6b1a67aaf 'docs: maintenance note 9': 14/15 CI [0.70, 0.99] vs t=0.50 → GOOD
  e38a2b4ed8 'speed up tool handling': 1/15 CI [0.01, 0.30] vs t=0.50 → BAD
Culprit: e38a2b4ed865a45a70d0ac149aa88204610620a8 'speed up tool handling' (confidence: 95% Wilson)

Reading it from the top:

The run ID (77f60c9fccf8) is what localise, fix, report and resume take.
The ends are measured first: good passed 19/20, bad 6/20.
The regression gate checks the drop is real. Here p ≈ 0, so the bisect goes ahead.
Each bisect step shows the commit, its passes, its 95% interval and the threshold, then GOOD, BAD or SKIP.
The last line names the culprit.

How bisect works explains each part.

Why the commit was made#

When a run finds its culprit, it records the commit's intent, which then appears in the report and the PR comment:

From Entire, when the commit has a checkpoint (a recorded coding-agent session). That gives the developer's actual request, for example "make it faster: run the tool calls in parallel and drop the per-result validation step, because it adds latency". This needs the entire CLI on your PATH.
Otherwise from the commit message and diff.

Only the culprit commit's context is read, never its neighbours'. culprit fix uses the intent to keep what the commit was for.

How it can end#

Status	Meaning	Next
`found`	The culprit is named.	`culprit localise`
`no_regression`	No significant drop. Nothing is blamed.	Check for drift
`ambiguous`	Skipped commits hide the exact boundary.	Read the suspects in the report
`budget_exceeded`	The cost cap was reached.	`culprit resume` with a higher cap
`infra_error`	Your provider kept failing. Nothing was counted.	Fix the cause, then `culprit resume`

Exit code 0 for found and no_regression, 1 otherwise.

Common variations#

More certainty on a noisy agent, at a higher cost:

Terminal

culprit run --good v1.0 --bad HEAD --k 5 --budget-usd 6

Faster, if your provider's rate limits allow:

Terminal

culprit run --good v1.0 --bad HEAD --workers 8

Options#

Option	Default	Description
`--good REF`	required	A ref where the agent was good.
`--bad REF`	required	A ref where it's worse.
`--budget-usd X`	3.00	Hard cost cap for this run.
`--k N`	3	Repetitions per task.
`--workers N`	4	Agent runs in parallel.
`--repo PATH`	from `culprit.toml`	The agent's repo.
`--eval-module MODULE`	from `culprit.toml`	The module that runs one task.
`--suite DIR`	from `culprit.toml`, else `evals`	The suite directory.
`--suite-ref REF`	from `culprit.toml`, else `main`	Ref whose suite judges every commit.
`--reuse / --no-reuse`	reuse	Reuse matching baseline results from earlier runs.
`--staged / --no-staged`	staged	Screen every task once before running the full `k` on the ends.

Next#

Find the responsible lines with culprit localise <run_id>.