benzi

A compiler-backed AI coding agent

82.6% on SWE-bench Verified · 1.85¢ a task

Benzi compiles your codebase into a resolved, queryable map of calls, data flow, control flow, and class hierarchy first; then it reads, explains, runs, edits and verifies your code using it. Includes a runtime tracer that settles what static analysis can’t.

github.com/oooscoos/Benzi
0SWE-bench Verified
1.5–2×faster
2–6×less source code read
0languages
PythonTypeScriptJavaScriptGoRustJavaCC++C#RubyHTML / CSS / DOM-JS PythonTypeScriptJavaScriptGoRustJavaCC++C#RubyHTML / CSS / DOM-JS
what Benzi actually does

What is Benzi? What can it do?

Benzi is a compiler-first agentic harness. It can autonomously investigate bugs, modify an entire project, build features, execute tests, and repair failures. The difference? Benzi compiles your codebase into a single, uniform representation first, and then works on every task using that. Quick facts:

CodeIndex + FrontendIndex

CodeIndex compiles ten languages: Python, JavaScript, TypeScript, Java, C#, C++, C, Go, Rust and Ruby. FrontendIndex covers the web layer: HTML, CSS and DOM-JS. Want another language? Please open a feature request on GitHub.

Compiles locally. Brings your own model.

The compiler and index run on your machine, so your codebase is never uploaded. Benzi is model-agnostic: plug in your own API key for the provider you prefer.

The full agent, or just the compiler

Run the Benzi agent as a VS Code extension or from the command line. Or take the compiler on its own, without Benzi's agentic loop, and plug it into the agent you already use over MCP. Benzi is free to use during beta.

the three lenses

Call flow, data flow, control flow.

A body tells you what it does. The edges tell you what it means. Switch lenses:

Who calls whom

The skeleton. Every proven edge the compiler resolved, plus the candidate edges it couldn't — dynamic dispatch, tagged with the set of overrides it might land on. Walk it forward to see what a route reaches, reverse for the full blast radius.

get_callerscall_treetrace_pathget_hierarchy
reverse blast radius
        main
         └─ run_capture
              └─ Sensor.read   ← you are here
                   ├─ calibrate
                   └─ db.insert   (external)

3 proven callers · 1 possible via dispatch

Where a value comes from

Trace a parameter backward through the caller's assignments and up the call graph to the literal it originates in — and walk a return forward to where it finally lands.

backflowforwardflowtrace_path
backflow  Sensor.read(sensor_id)
  sensor_id ← req.query.id   @ api.py:41
  sensor_id ← route.params   @ main.py:12
  sensor_id ← "T-42"        origin: fixture

2 call sites · 1 defaulted arg

Under what condition

The tool that answers "when does this happen". One-level outline of a body: every branch header verbatim, every loop exit with its label, every early return, interleaved with the calls in source order — indented to real nesting.

skim_source
def tick(self):
  if self.game_over or self.paused: return
  if self.pending:                       # queued turn
     self.direction = pending.pop(0)
  if self._off_board(head) or head in body:
     return self.end_game()          # ← the guard
  ...
live demo

Click a node. Feel the map.

Not a mock-up — this is the shape of the real map. Select a symbol and its callers, its callees, the values that move through it, and the tier behind every edge all come back at once.

$ click a node to inspect it

A call that resolved (solid), a dispatch candidate (dashed — a traced run is what turns it observed), and the value crossing an edge (seafoam), because the argument expression is what binds the parameter. Pick any node — the panel reports the same three.

honest by construction

Every edge is labelled with how much to trust it.

Precision over recall — a coin-flip would undermine the whole map. So nothing is ever guessed silently; it's tagged.

RESOLVED

In-repo edge, on hard compiler evidence.

CANDIDATE

Ambiguous: the bounded set of possible targets.

OBSERVED

Confirmed by an actual run, with real values.

EXTERNAL

Into a library, with the import evidence.

UNRESOLVED

Seen but not settled, and it says why.

UNINDEXED

Never parsed, with the reason.

the edit loop

Every edit gated, receipted, reversible.

Benzi runs every write through the same compiler that built the map, then keeps the map in step with the code. Nothing lands unchecked, and any write can be undone.

  1. 01Write The agent changes a line span or a whole symbol, and every write is logged as a step.
  2. 02Parse-gated The real language parser checks the result first. An edit that breaks the parse reverts itself.
  3. 03Receipt A landed edit reports the changed symbol, its callers, how the call graph moved, and how far the lines below shifted.
  4. 04Reindex Only the changed files are recompiled, so every later query sees the code as it is now.
  5. 05Rollbackoptional Undo the last N writes from a snapshot, files and map together, with no partial states.
  6. 06Runtime traceroptional Run it, then ask Benzi what actually happened: the real arguments, the return values, and which dispatch target won.
it remembers you

Persistent, per-repo memory.

The index is always in context; facts Benzi learns are saved as durable memories, so a future session opens already knowing them — the convention, the gotcha, the bug's real origin.

  • remember()Save a non-obvious fact the moment it's learned — with the why, which is the valuable half.
  • recall()Pull any memory back in full when it becomes relevant.
  • forget()Drop a fact that turned out stale, so no future session is misled by it.
# a memory that pays for itself later
type    gotcha
name    "config-load-order"
body    "env.py must import before settings,"
        "or SECRET_KEY reads empty. Load-bearing."

# → every future session opens knowing this
the toolset

Every tool answers one question.

Every lookup is O(1): the compiler has already resolved the answers into hashmaps, so the tools read them instead of searching. This list shows 16 of Benzi's 36 tools. Full tool list here.

and it builds, too

StallionSwipe — a dating app for horses, written in one session.

Every horse portrait is drawn in code — procedural SVG, no image files — and matched horses flirt back through a live AI model. Frontend, backend and prompts were all written by Benzi.

StallionSwipe swipe deck StallionSwipe profile detail StallionSwipe live AI chat StallionSwipe profile creation
0 image files procedural SVG portraits live AI chat one session
Meet the horses →
evidence

Measured, not asserted.

One configuration, one attempt per instance, no ensembling, no test-time selection — graded by the official SWE-bench harness.

0SWE-bench Verified · pass@1
DeepSeek V4.1-Flash · one run
413 / 500
Real GitHub issues resolved
$0.0185
Generation cost per task
$9.26
Total generation cost, all 500
97.6%
Input tokens served from prompt cache

On a separate bug-fixing benchmark: 24 real bugs across 10 languages, each fixed once. The models stay the same — Sonnet and DeepSeek — so the only thing that changes is the harness: Benzi, Claude Code, or a plain DeepSeek harness. Below is how much source each one had to read, bug by bug, easiest to hardest. Hover any point for the bug and its count.

Benzi · Sonnet
0
Benzi · DeepSeek
0
Claude Code · Sonnet
0
DeepSeek harness · DeepSeek
0

Source lines read (lower is better) across the same 24 bugs — 9,125 and 16,407 for the two Benzi series, against 20,704 for Claude Code and 43,598 for a plain harness. It's the one metric whose measurement is identical across harnesses, and the gap widens with difficulty.

On CodeGraph's own benchmark — their repos, their questions, their methodology — all four answers were scored blind by frontier models, from a fresh account:

1 Benzi Product 9.6

Unmatched technical depth, architectural insights, edge-case coverage, and structural visualizations (flowcharts/tables).

2 Bare CC 8.8

Exceptionally precise, concise line-by-line tracing, high citation accuracy, and immediate zero-fluff code mechanics.

3 Benzi MCP 8.4

Strong overall structure, clear stage-by-stage breakdowns, and reliable coverage of core mechanisms.

4 CodeGraph 8.1

Highly readable, great code snippets and summaries, but slightly less granular on deep runtime invariants.

Benzi ranked first with every reviewer. Full results →

faq & comparisons

FAQ & comparisons

Nine questions, straight answers — what runs where, what it costs, and how Benzi differs from the tools you already know.

Does my code leave my machine?
No. In VS Code, the CLI, or MCP, the compiler and index run locally — nothing uploaded, no copy kept. Only the snippets the agent actually reads go to your model provider, same as any AI assistant, and less of them: 9,125 lines read vs Claude Code’s 20,704 on the same 24 bugs. The browser demo differs — it fetches a public repo server-side, read-only, deletes it after your session.
Is it actually free?
Yes, Benzi doesn’t charge. The CLI and MCP are BYOK, so you pay your own model provider. Early and in development — that’s the trade, not a paywall.
Can it run on a local model?
In principle, yes: Benzi talks to models through standard APIs, so a local model (Ollama, LM Studio and the like) could plug in. It isn’t supported yet, though. Today Benzi runs on hosted providers with your own key.
Can I point it at a private repo?
Not the web demo (public GitHub API only). Everywhere else, yes — the compiler runs locally on whatever path you give it.
How large a repo can it handle?
VS Code handles real codebases — microsoft/vscode, 923k lines, indexes in ~2 minutes, then caches. The browser demo caps at 2,000 files, 2 MB each.
How is this different from Cursor, Copilot, or Claude Code?
They search — grep or embeddings. Benzi resolves first: a real index of symbols, calls, inheritance and data flow, queried instead of guessed. Same 24 bugs, 2.3× less source read than Claude Code. Details: what the index actually changes.
How is this different from an LSP-backed MCP server?
An LSP answers at a cursor, in one open file: go-to-definition or find-references, one position and one hop at a time. Benzi compiles the whole repo up front into one index, so the questions are whole-codebase ones: transitive call trees, the path between two functions, and data flow — where a bad value came from or where a return value lands. Every answer also carries a confidence tier: resolved, candidate, unresolved (with the reason), or observed. An LSP gives an answer or nothing. The runtime tracer then settles what static analysis can’t, by watching what actually fires. And the compiler itself is language agnostic: ten languages run through one pipeline into one index format. An LSP setup needs a separate server per language, each installed, configured and kept running.
How is this different from CodeQL or Sourcegraph’s SCIP indexers?
They need a working build and index in batch: dependencies installed, the project compiling, a CI job measured in minutes. Benzi needs no build, works on half-finished code, and re-parses only what changed on every turn. That’s what makes gated writes possible: each edit is checked against a fresh index before the next step, not after a rebuild. One pipeline covers all ten languages instead of one indexer per language. Where that costs precision, Benzi flags the call site as candidate or unresolved instead of guessing.
How is this different from CodeGraph?
Both index instead of search, but CodeGraph retrieves — ranked candidates from a queried database. Benzi resolves — settles what a name binds to before answering, and refuses rather than guesses when a call site is ambiguous. It also models code the way an engineer reads it — file → scopes → call flow → data/control flow — not a flat symbol graph.
On CodeGraph’s own benchmark (their repos, their questions, their methodology), Gemini, ChatGPT, Claude and DeepSeek each reviewed the answers from a fresh account, and all four ranked Benzi’s first. Full results →
get started

Two ways in.

Run Benzi's own agentic loop — in the editor or from a terminal — or hand the compiled index to the agent you already use. Bring your own key; free during beta.

Use case 1 · the agentic loop

Benzi reads, plans, edits, runs, and checks its own change.

The full product, with the loop closed: model, key and account live in one settings panel, and the CLI shares the same login.

In VS Code
  1. 1. Install

    Install Benzi from the VS Code Marketplace.

  2. 2. Open the settings panel

    Click the Benzi icon in the left activity bar to open the Benzi: Settings panel.

  3. 3. Pick a model, paste a key

    Pick your model and paste your API key: an Anthropic key, or a non-Anthropic one (DeepSeek, Groq, Kimi and others).

  4. 4. Log in

    Enter your email and press Send, then type in the one-time code it emails you.

  5. 5. Run

    Press Save model & keys, then Run Benzi.

Get it on the Marketplace →
From the terminal

The same loop, headless — for scripts and CI. Python 3.11–3.14 on Windows, macOS and Linux.

$ pip install benzi

Installs the benzi-login, benzi-headless and benzi-mcp commands.

$ benzi-login --login you@example.com
$ benzi-headless . "fix the failing test in parser.py"

The login is the one VS Code uses: --anthropic-key YOUR_KEY, or --nonanthropic-key YOUR_KEY for DeepSeek, Groq, Kimi and others.

View on PyPI →
Use case 2 · the compiler over MCP

Use it with the agent you already have.

No model, no key, none of Benzi's loop: benzi-mcp serves the compiled index to Claude Code, Cursor or any MCP client, and your own agent makes every model call.

Same install, same login
  1. 1. pip install benzi

    The same package as above — Python 3.11–3.14 on Windows, macOS and Linux.

  2. 2. benzi-login

    Log in exactly as you would for headless. You can skip the model and key, because your own agent does the model calls.

Register it instead of benzi-headless

Then register benzi-mcp with your agent. For example, in Claude Code, once for all your projects:

$ claude mcp add --scope user benzi -- benzi-mcp

benzi-mcp then serves whichever folder you start Claude Code in, so switching projects needs nothing extra. To pin it to one repo no matter where you start, pass that repo's path:

$ claude mcp add --scope user benzi -- benzi-mcp /path/to/your/repo
Public repository · docs · releases · issues

github.com/oooscoos/Benzi

Docs, releases, discussions and the issue tracker live there.

Open the repo →