benzi

A compiler-backed AI coding agent

78.2% on SWE-bench Verified · 9.5¢ a fix

Benzi compiles your codebase into a resolved, queryable map of calls, data flow, and control flow first; then it explains, runs, edits and verifies your code using it. Includes a runtime tracer that settles what static analysis can’t.

github.com/oooscoos/Benzi
0SWE-bench Verified
1.5–2×faster
2–6×less source code read
0languages
PythonTypeScriptJavaScriptGoRustJavaCC++C#RubyHTML / CSS / DOM-JS PythonTypeScriptJavaScriptGoRustJavaCC++C#RubyHTML / CSS / DOM-JS
what Benzi actually does

Three questions. One compiled map.

Most codebase questions reduce to three: where is it, what does it touch, and what actually happens when it runs? Benzi compiles the repository first, then gives the agent purpose-built tools to interrogate that same map—using runtime evidence when static analysis can’t settle the answer.

Compile before the LLM thinks

Turn entities into exact symbols and record their relationships, including provenance, before the LLM is loaded. The agent then cuts through the codebase with 30+ targeted tools, relying on compiler-derived structure and avoiding raw source reads where possible.

Prove what a change touches

Trace callers, aliases, callbacks, registry references, and other indirect relationships—not just direct calls. Every edit is parse-checked immediately, and the finished change is statically checked for new errors.

Resolve what static analysis can’t

Whatever can only be settled by execution is settled by execution. Dynamic relations start as CANDIDATE; when observed at runtime, they become OBSERVED, with actual arguments and return values recorded by the tracer.

the three lenses

Call flow, data flow, control flow.

A body tells you what it does. The edges tell you what it means. Switch lenses:

Who calls whom

The skeleton. Every proven edge the compiler resolved, plus the candidate edges it couldn't — dynamic dispatch, tagged with the set of overrides it might land on. Walk it forward to see what a route reaches, reverse for the full blast radius.

get_callerscall_treetrace_pathget_hierarchy
reverse blast radius
        main
         └─ run_capture
              └─ Sensor.read   ← you are here
                   ├─ calibrate
                   └─ db.insert   (external)

3 proven callers · 1 possible via dispatch

Where a value comes from

Trace a parameter backward through the caller's assignments and up the call graph to the literal it originates in — and walk a return forward to where it finally lands.

backflowforwardflowtrace_path
backflow  Sensor.read(sensor_id)
  sensor_id ← req.query.id   @ api.py:41
  sensor_id ← route.params   @ main.py:12
  sensor_id ← "T-42"        origin: fixture

2 call sites · 1 defaulted arg

Under what condition

The tool that answers "when does this happen". One-level outline of a body: every branch header verbatim, every loop exit with its label, every early return, interleaved with the calls in source order — indented to real nesting.

skim_source
def tick(self):
  if self.game_over or self.paused: return
  if self.pending:                       # queued turn
     self.direction = pending.pop(0)
  if self._off_board(head) or head in body:
     return self.end_game()          # ← the guard
  ...
live demo

Click a node. Feel the map.

This approximates what Benzi does on a real repo: select a symbol, and its callers, callees and tier come back at once.

$ click a node to inspect it

Nodes are symbols; edges are resolved calls. Green edges are proven; the tier line names any dispatch candidate.

honest by construction

Every edge is labelled with how much to trust it.

Precision over recall — a coin-flip would undermine the whole map. So nothing is ever guessed silently; it's tagged.

PROVEN

The compiler resolved the call on hard evidence. This is fact — a caller you can rely on.

CANDIDATE

Dynamic dispatch and only that. Carries the bounded set of overrides it might land on. Running it is what resolves it.

UNRESOLVED

Seen but not settled — and it says why: opaque receiver, untyped local, or a real name collision with candidates attached.

OBSERVED

What actually ran under the tracer, with real argument and return values. The only truth for which dispatch target won.

it remembers you

Persistent, per-repo memory.

The index is always in context; facts Benzi learns are saved as durable memories, so a future session opens already knowing them — the convention, the gotcha, the bug's real origin.

  • remember()Save a non-obvious fact the moment it's learned — with the why, which is the valuable half.
  • recall()Pull any memory back in full when it becomes relevant.
  • forget()Drop a fact that turned out stale, so no future session is misled by it.
# a memory that pays for itself later
type    gotcha
name    "config-load-order"
body    "env.py must import before settings,"
        "or SECRET_KEY reads empty. Load-bearing."

# → every future session opens knowing this
the toolset

Every tool answers one question.

Every lookup is O(1): the compiler has already resolved the answers into hashmaps, so the tools read them instead of searching. This list shows 16 of Benzi's 36 tools. Full tool list here.

evidence

Measured, not asserted.

One configuration, one attempt per instance, no ensembling, no test-time selection — graded by the official SWE-bench harness.

0SWE-bench Verified · pass@1
DeepSeek v4-flash · one run
391 / 500
Real GitHub issues resolved
$0.095
Generation cost per fix
379
Median source lines read per instance
97%
Input tokens served from prompt cache

On a separate bug-fixing benchmark: 24 real bugs across 10 languages, each fixed once. The models stay the same — Sonnet and DeepSeek — so the only thing that changes is the harness: Benzi, Claude Code, or a plain DeepSeek harness. Below is how much source each one had to read, bug by bug, easiest to hardest. Hover any point for the bug and its count.

Benzi · Sonnet
0
Benzi · DeepSeek
0
Claude Code · Sonnet
0
DeepSeek harness · DeepSeek
0

Source lines read (lower is better) across the same 24 bugs — 9,125 and 16,407 for the two Benzi series, against 20,704 for Claude Code and 43,598 for a plain harness. It's the one metric whose measurement is identical across harnesses, and the gap widens with difficulty.

On CodeGraph's own benchmark — their repos, their questions, their methodology — all four answers were scored blind by frontier models, from a fresh account:

1 Benzi Product 9.6

Unmatched technical depth, architectural insights, edge-case coverage, and structural visualizations (flowcharts/tables).

2 Bare CC 8.8

Exceptionally precise, concise line-by-line tracing, high citation accuracy, and immediate zero-fluff code mechanics.

3 Benzi MCP 8.4

Strong overall structure, clear stage-by-stage breakdowns, and reliable coverage of core mechanisms.

4 CodeGraph 8.1

Highly readable, great code snippets and summaries, but slightly less granular on deep runtime invariants.

Benzi ranked first with every reviewer. Full results →

and it builds, too

StallionSwipe — a dating app for horses, written in one session.

Every horse portrait is drawn in code — procedural SVG, no image files — and matched horses flirt back through a live AI model. Frontend, backend and prompts were all written by Benzi.

StallionSwipe swipe deck StallionSwipe profile detail StallionSwipe live AI chat StallionSwipe profile creation
0 image files procedural SVG portraits live AI chat one session
Meet the horses →
faq & comparisons

FAQ & comparisons

Nine questions, straight answers — what runs where, what it costs, and how Benzi differs from the tools you already know.

Does my code leave my machine?
No. In VS Code, the CLI, or MCP, the compiler and index run locally — nothing uploaded, no copy kept. Only the snippets the agent actually reads go to your model provider, same as any AI assistant, and less of them: 9,125 lines read vs Claude Code’s 20,704 on the same 24 bugs. The browser demo differs — it fetches a public repo server-side, read-only, deletes it after your session.
Is it actually free?
Yes, Benzi doesn’t charge. The CLI and MCP are BYOK, so you pay your own model provider. Early and in development — that’s the trade, not a paywall.
Can it run on a local model?
In principle, yes: Benzi talks to models through standard APIs, so a local model (Ollama, LM Studio and the like) could plug in. It isn’t supported yet, though. Today Benzi runs on hosted providers with your own key.
Can I point it at a private repo?
Not the web demo (public GitHub API only). Everywhere else, yes — the compiler runs locally on whatever path you give it.
How large a repo can it handle?
VS Code handles real codebases — microsoft/vscode, 923k lines, indexes in ~2 minutes, then caches. The browser demo caps at 2,000 files, 2 MB each.
How is this different from Cursor, Copilot, or Claude Code?
They search — grep or embeddings. Benzi resolves first: a real index of symbols, calls, inheritance and data flow, queried instead of guessed. Same 24 bugs, 2.3× less source read than Claude Code. Details: what the index actually changes.
How is this different from an LSP-backed MCP server?
An LSP answers at a cursor, in one open file: go-to-definition or find-references, one position and one hop at a time. Benzi compiles the whole repo up front into one index, so the questions are whole-codebase ones: transitive call trees, the path between two functions, and data flow — where a bad value came from or where a return value lands. Every answer also carries a confidence tier: resolved, candidate, unresolved (with the reason), or observed. An LSP gives an answer or nothing. The runtime tracer then settles what static analysis can’t, by watching what actually fires. And the compiler itself is language agnostic: ten languages run through one pipeline into one index format. An LSP setup needs a separate server per language, each installed, configured and kept running.
How is this different from CodeQL or Sourcegraph’s SCIP indexers?
They need a working build and index in batch: dependencies installed, the project compiling, a CI job measured in minutes. Benzi needs no build, works on half-finished code, and re-parses only what changed on every turn. That’s what makes gated writes possible: each edit is checked against a fresh index before the next step, not after a rebuild. One pipeline covers all ten languages instead of one indexer per language. Where that costs precision, Benzi flags the call site as candidate or unresolved instead of guessing.
How is this different from CodeGraph?
Both index instead of search, but CodeGraph retrieves — ranked candidates from a queried database. Benzi resolves — settles what a name binds to before answering, and refuses rather than guesses when a call site is ambiguous. It also models code the way an engineer reads it — file → scopes → call flow → data/control flow — not a flat symbol graph.
On CodeGraph’s own benchmark (their repos, their questions, their methodology), Gemini, ChatGPT, Claude and DeepSeek each reviewed the answers from a fresh account, and all four ranked Benzi’s first. Full results →
get started

Four ways in.

Zero setup in a browser, the full product on your own machine, or the index handed to the agent you already use.

Live demo · no install

Live demo

Paste a GitHub URL; Benzi compiles it into a map of calls, data flow and class hierarchy, then answers in a chat panel beside an interactive graph. Read-only — nothing is written to the repo, and there's no signup.

varianttech.net/demo →
VS Code extension · full product

VS Code extension

Benzi reads, explains, runs, edits and checks its own changes. Setup lives in a Benzi settings panel in the sidebar (model, API key, email login). Bring your own key; free during beta.

Get it on the Marketplace →
Python package · outside the editor

pip MCP/CLI

Three commands, one login config shared with VS Code.

$ pip install benzi

Python 3.11–3.14 on Windows, macOS and Linux. Installs the benzi-login, benzi-mcp and benzi-headless commands.

  1. 1. Log in

    Emails you a one-time code, then saves your key. The same login works in VS Code.

    $ benzi-login --login you@example.com
    $ benzi-login --anthropic-key YOUR_KEY      # or: benzi-login --nonanthropic-key YOUR_KEY
  2. 2. Give your agent Benzi's map

    Serves the compiled index as tools to Claude Code, Cursor or any MCP client.

    $ benzi-mcp
  3. 3. Or run the full agent

    Point it at a repo from the terminal, for scripts and CI.

    $ benzi-headless . "fix the failing test in parser.py"
View on PyPI →
Public repository · docs · releases · issues

github.com/oooscoos/Benzi

this repo is docs, releases, discussions and the issue tracker. Open the repo →