Hamza Ouhssaine
Contact

Code Auditor

A LangGraph agent that treats a repository as hostile input

Code Auditor takes a Git URL or a local folder and assumes the worst about it. It scans for malware and secrets before anything else touches the code, maps and analyses it without ever running it on the host, builds a risk-ranked roadmap, then fixes one item at a time in a sandbox. Every change waits for a person to approve it.

Type
Personal project, 2026, designed and built end to end
Agent
LangGraph state machine, 11 nodes, SQLite checkpoints
Isolation
Network-less Docker containers with memory, CPU and process caps
Models
Claude Sonnet for the roadmap and the fixes, both as structured output
Tests
478 backend test functions, 18 frontend tests

Prototype. Every stage is tested against fakes of the model and the containers, and the scanners against real tools. The fix loop has not yet run end to end against a real model inside Docker.

From repository to roadmap

The first half of a run reads the repository and never executes it. Each stage is a node of a fixed LangGraph state machine rather than a free agent loop, so the order of the checks can’t be talked around.

One repository through the audit
Repository Git URL or folder Preflight scan malware and secrets Quarantine abort the run Report why it stopped Explore ast map, no imports Diagnose ruff, mypy, radon Security audit bandit, semgrep, osv Approval gate critical findings Plan risk-ranked roadmap Roadmap items by priority Repository Git URL or folder Preflight scan malware and secrets Quarantine abort the run Report why it stopped Explore ast map, no imports Diagnose ruff, mypy, radon Security audit bandit, semgrep, osv Approval gate critical findings Plan risk-ranked roadmap Roadmap items by priority
  • Input or output
  • LLM call
  • Code
  • Check or gate
  1. Repository A Git URL, cloned with full history into a throwaway workspace (600 s timeout). A local folder is copied without following symlinks.
  2. Preflight scan ClamAV, YARA rules and TruffleHog run in a container with no network. The gate fails closed and trusts the hits, not the scanner’s own “passed” flag. Here: no hits.
  3. Explore Maps modules, dependency graph, entry points and tests by parsing with ast. Nothing is imported or executed. Caps: 2 MB per file, 20,000 files.
  4. Diagnose ruff, mypy and radon run in a 4 GB container with no network; every finding comes back with a severity.
  5. Security audit bandit, semgrep, osv-scanner and gitleaks, scored with CVSS. Critical is reserved for security tools; a Critical finding would stop here for a human.
  6. Plan Findings are compacted (150 listed one by one, the rest grouped) and Claude Sonnet returns a typed roadmap. Code then re-derives every severity instead of trusting the model.
  7. Roadmap Items with severity, priority, rationale and the files they touch, ready for the fix loop. With no model, or on a model error, a rule-based roadmap is built instead.

Other routes through the graph

  • Any preflight hit, or no preflight result at all: quarantine, abort, and a report that says why.
  • A Critical security finding: the run halts at the approval gate before any planning.
  • No findings, or an empty roadmap: straight to the report.
  • Cost ceiling reached while planning: a rule-based roadmap, and the run is marked aborted.

The fix loop, with a human in it

The second half changes code, one roadmap item at a time. The model writes files; code decides whether they are safe to write, a sandbox decides whether they work, and a person decides whether they stay.

One roadmap item through the fix loop
Roadmap item next by priority Act returns whole files Verify pytest, read-only Human approval approve or reject Rollback byte-exact snapshots Report Markdown, no model Roadmap item next by priority Act returns whole files Verify pytest, read-only Human approval approve or reject Rollback byte-exact snapshots Report Markdown, no model
  • Input or output
  • LLM call
  • Code
  • Check or gate
  1. Roadmap item The highest-priority item. Scope and the $10 cost ceiling are checked before any model call.
  2. Act Up to 4 whole files go to the model, which returns complete new files, never diffs or shell commands. Each file is validated, snapshotted and written atomically; the diff comes from difflib.
  3. Verify pytest runs in a sandbox with no network, on a read-only mount of the workspace. A test fails.
  4. Act Second attempt. Each item gets 3 in all, counting the first.
  5. Verify Tests pass. “Unverifiable” is kept apart from “failed” and is never retried.
  6. Human approval The graph pauses with interrupt() until a reviewer approves or rejects. The decision must also exist as a database row, and the row is the authority.
  7. Report After the last item, a Markdown report rendered by code, not by a model. “Fixed” means approved, verified and not rolled back.

Other routes through the graph

  • Still failing after 3 attempts, or the iteration cap reached: byte-exact rollback, then the gate.
  • Rejected without feedback: the change is reverted. Rejected with feedback: back to Act with a fresh retry budget.
  • Nothing written to disk: the gate lets the run continue without stopping.

Containment

The repository is untrusted, and so is the model’s output. Both are kept on a short leash.

  • Four kinds of container

    Preflight (2 GB), analysis (4 GB), security (6 GB) and the test sandbox (2 GB), each with no network, 2 CPUs, 512 processes and a timeout between 10 and 30 minutes.

  • Write on the host, run in the box

    Edits are written to the workspace on the host; code only ever runs in the sandbox, against a read-only mount. Snapshots live outside the mount.

  • Hard limits on a run

    50 iterations, one hour of wall clock, a $10 cost ceiling checked before each model call, and a LangGraph recursion limit of 200.

  • Pinned tools

    ruff 0.9.10, mypy 1.15.0, radon 6.0.1, bandit 1.8.6, semgrep 1.172.0, gitleaks 8.24.0, osv-scanner 2.4.0, TruffleHog 3.88.0, pytest 8.3.4.

What a real repository showed

A first manual run on a real 56-file repository, without Docker or a model, is recorded in the project notes. 134 of its 137 security findings were bandit’s B101, a plain assert statement, and none of the 17 roadmap items fell inside the fixer’s default allowlist.

Noise and scope, not model quality, are what a tool like this has to get right first.

Engineering choices

  • A state machine, not a free agent

    Fixed nodes with narrow jobs. The model plans and writes files; it never chooses which check runs next.

  • Fail closed

    A missing preflight result counts as a failed one. The scanners’ hits decide, not their exit flags.

  • Severity in code

    The model may suggest a severity; code re-derives it from where the finding came from. Critical belongs to security tools only.

  • Whole files, not patches

    The model returns complete files. Diffs are computed by difflib, and unsafe output is refused before anything is written.

  • Approval needs a record

    A LangGraph interrupt pauses the run, and resuming requires an approval row in the database. No row, no resume.

  • A report without a model

    The final report is rendered by code from recorded events, so it can’t overstate what was fixed.

Stack

Agent
LangGraph, SQLite checkpoints, Anthropic SDK (structured output)
Analysis
ruff, mypy, radon, bandit, semgrep, osv-scanner, gitleaks
Preflight
ClamAV, YARA, TruffleHog
Backend
Python 3.12, FastAPI, WebSockets, SQLModel, Alembic, Docker SDK
Frontend
React 19, TypeScript, Vite, Tailwind CSS 4
Quality
pytest, Vitest, parser tests on captured real tool output

Building something similar?

I’m happy to walk through the design, the trade-offs, or what I would change next.

Email me All projects