Docs
Workflow, in order

culprit init

Step 1. Save your keys once, and set up an agent's repo so every other command works without flags.

Terminal
culprit init

Run it anywhere inside your agent's repo. It's safe to run again: it updates culprit.toml and only asks for keys you haven't saved.

What it does#

  1. Finds the repo root and warns if there's no uv.lock, since Culprit installs each commit with uv sync --locked.
  2. Sets up your keys. The first time, it asks for your model key and an optional neatlogs key, and saves them once for all your repos. See Keys and configuration.
  3. Finds your eval module (a file named eval.py, evals.py or eval/__main__.py) and counts the tasks in evals/.
  4. Writes culprit.toml and adds .culprit/ to .gitignore.
  5. Offers to run one task exactly as a bisect step would, and shows the parsed result. This calls your model once.
Output
Setting up /home/you/refund-agent
  OPENROUTER_API_KEY: from culprit init
  your agent will run on z-ai/glm-5.3 via openrouter during a bisect
  eval modules found: refund_agent.eval
  26 tasks in evals/: refund_small_order, refund_timeout_once, …
  added .culprit/ to .gitignore
  wrote /home/you/refund-agent/culprit.toml
  check passed: task refund_small_order: passed=True (cost=0.0026, latency_ms=7086)

If it can't find an eval module, see Prepare your agent.

Common variations#

Change your saved keys:

Terminal
culprit init --keys

Skip the questions and the check, for scripts and CI:

Terminal
culprit init --yes

Name the eval module and suite yourself:

Terminal
culprit init --eval-module myagent.eval --suite evals --suite-ref main

Options#

Option Default Description
--repo PATH . Any directory inside your agent's repo.
--eval-module MODULE detected The module that runs one task.
--suite DIR evals Directory with one file per task.
--suite-ref REF current branch Ref whose suite judges every commit.
--check / --no-check ask; no with --yes Run one task to check its output.
--keys off Ask for the keys again, even if they're already set.
--yes, -y off Accept the defaults. Fails if no model key is set anywhere.

Exit code 0 when set up, 1 if --check failed, 2 if it couldn't set up (not a git repo, no key with --yes, no eval module).

Next#

Commit culprit.toml so your team and CI share it, then find the culprit.

Culprit 0.1.0 · Apache-2.0 · Team PiedPiper