Skip to content

tigrankh/code-guardian

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Code Guardian

A command-line tool that wraps the Trivy security scanner. You give it one or more Git repositories; for each it scans the code, fetches the repository's GitHub popularity, prints a short summary, and writes a per-repository result bundle — including a Graphviz dependency graph with CRITICAL-bearing components highlighted.


TL;DR — install & run

Where to run from:

  • Docker run (pulled image): any directory you like — results are written to ./results under your current directory (via the $(pwd)/results volume mount).
  • Building the image or the local (Poetry) run: from the project root — the folder containing pyproject.toml, the Dockerfile, and .env (cd /path/to/code-guardian) — since the build needs the source and the local run loads .env and the default output path from there.

Option A: Docker (recommended — bundles Trivy, git, and Graphviz)

Pull the published image:

docker pull ghcr.io/tigrankh/code-guardian:v1.0.0

Run it (results land in ./results under your current directory via the volume mount):

docker run --rm \
  -v "$(pwd)/results:/app/results" \
  ghcr.io/tigrankh/code-guardian:v1.0.0 -r https://github.com/OWASP/NodeGoat.git

Useful variations:

# multiple repos
docker run --rm -v "$(pwd)/results:/app/results" ghcr.io/tigrankh/code-guardian:v1.0.0 \
  -r https://github.com/OWASP/NodeGoat.git \
  -r https://github.com/OWASP/railsgoat.git

# with a GitHub token (5000 req/hr instead of 60) — passed as env, never baked in
docker run --rm -e GH_TOKEN=ghp_xxx -v "$(pwd)/results:/app/results" \
  ghcr.io/tigrankh/code-guardian:v1.0.0 -r https://github.com/OWASP/NodeGoat.git

# from a file of repos (one per line; blank lines and '#' comments ignored)
docker run --rm -v "$(pwd)/repos.txt:/app/repos.txt:ro" -v "$(pwd)/results:/app/results" \
  ghcr.io/tigrankh/code-guardian:v1.0.0 -f /app/repos.txt

# persist Trivy's vulnerability DB between runs (avoids re-downloading it)
docker run --rm -v trivy-cache:/root/.cache -v "$(pwd)/results:/app/results" \
  ghcr.io/tigrankh/code-guardian:v1.0.0 -r https://github.com/OWASP/NodeGoat.git

Tip: for brevity, give it a short local name — docker tag ghcr.io/tigrankh/code-guardian:v1.0.0 code-guardian — then use code-guardian in place of the full reference above.

Building the image yourself

If you'd rather build from source instead of pulling, run this from the project root:

docker build -t code-guardian .

That produces a local image named code-guardian; substitute it for ghcr.io/tigrankh/code-guardian:v1.0.0 in any of the run commands above.

Option B: Local (Poetry)

Prerequisites on your PATH: trivy, git, and the Graphviz dot binary.

  • macOS: brew install trivy graphviz
  • Linux: apt-get install graphviz and install Trivy via its official instructions (it isn't in the default apt repositories).
poetry install
poetry run code-guardian -r https://github.com/OWASP/NodeGoat.git

Run from the project root (the .env file and relative output paths are resolved from the current directory).

Configuration (.env or environment variables)

Variable Required Default Purpose
DEFAULT_OUTPUT_DIR no ./results Where result bundles are written (used when -o is omitted)
GH_TOKEN no GitHub token; if set, requests are authenticated (higher rate limit)

CLI

code-guardian [OPTIONS]

  -r, --repositories       TEXT  Repo URL (repeatable)
  -f, --repositories-file  FILE  Text file of repos, one per line
  -o, --output-dir         PATH  Output directory (default: DEFAULT_OUTPUT_DIR)
  -w, --workers            INT   Max concurrent heavy scans (default: 4)
  -l, --log-level          TEXT  DEBUG | INFO | WARNING | ERROR | CRITICAL
  -v, --verbose                  On error, print the full stack trace

-r and -f can be combined; duplicate repositories are scanned only once.

Output

For each repository, <output-dir>/<repo-name>/:

File Contents
report.json machine-readable: repository identity + popularity, severity statistics, dependency-graph components
report.html self-contained human report: summary, the inlined dependency graph, and the full vulnerability table
vulnerabilities.jsonl the full vulnerability list, one JSON object per line
dependency-graph.svg the Graphviz dependency graph (CRITICAL components highlighted in red)

Architecture: a per-repository pipeline

Each repository is processed by an ordered pipeline of stages. A stage does one thing, reading its inputs from and writing its outputs onto a shared PipelineContext that flows through the stages — so later stages consume what earlier ones produced.

            ┌────────┐   ┌────────┐   ┌─────────┐   ┌────────────┐   ┌────────┐   ┌─────────┐
  repo URL ─► Fetch  ├──►│  Scan  ├──►│  Parse  ├──►│ RenderSvg  ├──►│ Write  ├──►│ Summary │─► stdout + files
            └────────┘   └────────┘   └─────────┘   └────────────┘   └────────┘   └─────────┘
              GitHub        Trivy        ijson           dot          json/html      print
Stage Responsibility Produces on the context
FetchStage Build the repository and fetch its GitHub popularity (stars/forks) repository
ScanStage Run trivy repo <url> as a subprocess → raw Trivy JSON trivy_json_path
ParseStage Stream-parse the Trivy JSON: tally severities, build the component graph, and stream every vulnerability to vulnerabilities.jsonl report
RenderSvgStage Build the Graphviz DOT and render it to SVG via dot (CRITICAL nodes highlighted) svg_path
WriteStage Write report.json and the self-contained report.html (inlines the SVG, streams the vuln table from the .jsonl) result files
SummaryStage Print the per-repository summary to stdout

Supporting layers:

  • clients/ — external boundaries: GithubClient (REST popularity) and TrivyClient (subprocess wrapper for the scanner). Each is the place subprocess / HTTP details live, so stages stay thin.
  • models/ — typed Pydantic data: RepositoryData, ReportData, ComponentData, VulnerabilityData, Severity.
  • pipeline/BaseStage, the six stages, the PipelineContext, and PipelineManager (runs an ordered list of stages for one repository).

Why trivy repo (and not a manual git clone)

ScanStage uses trivy repo <url>, which clones the repository internally with Trivy's built-in Git implementation. That means no git binary is required for cloning and no temporary working tree to manage. The dependency graph is reconstructed entirely from Trivy's JSON (Packages + DependsOn + which packages carry a CRITICAL finding), so the ephemeral checkout Trivy discards is never needed afterwards.


Parallelization & resource model

Concurrency lives above the pipeline. The orchestrator builds one PipelineContext per repository and launches every repository's pipeline as an independent task:

await asyncio.gather(
    *(pipeline_manager.run(ctx) for ctx in contexts),
    return_exceptions=True,
)

So all repositories run simultaneously, and the work is bounded per resource, not per repository.

Per-resource concurrency limits

The expensive resources differ, so each gets its own limit — a single asyncio.Semaphore that is shared across all repositories (the stages are constructed once and reused for every repo, so a stage's semaphore is one global object). Each stage acquires its own semaphore around its work; the manager does the acquiring.

Resource (semaphore) Limit Stages Why
Network max(16, workers) Fetch GitHub calls are cheap I/O — fan out wide
Subprocess workers (default 4) Scan + Render Trivy and dot are heavy processes — cap the combined count to avoid oversubscribing the machine
CPU os.cpu_count() Parse, Write Match CPU/file work to cores

Because the semaphores are shared, --workers 4 means at most 4 heavy subprocesses across the entire run — not 4 per repository. With 100 repos and workers=4, four scans run while the rest wait their turn on the shared semaphore.

Keeping the event loop unblocked

asyncio is single-threaded: a synchronous call would freeze every repository's progress. Each stage is therefore non-blocking:

  • Scan & Render run their external processes with asyncio.create_subprocess_exec (truly async — the loop is free while Trivy / dot run).
  • Parse & Write do blocking CPU/file work, so they are offloaded with asyncio.to_thread (the file I/O and subprocess waits release the GIL, so they overlap).
  • Fetch uses async httpx.

The result is genuine overlap: while one repo renders its graph in a worker thread, another is scanning and a third is parsing.

Streaming large outputs (bounded memory)

Real repositories produce large Trivy JSON. The vulnerability list — the only unbounded part — is never held in memory:

  • ParseStage streams the Trivy JSON with ijson and appends each vulnerability straight to vulnerabilities.jsonl, keeping only the bounded aggregates (severity counts and the component graph) in the ReportData model.
  • WriteStage streams that .jsonl row-by-row into the HTML table.

So memory stays flat regardless of how many vulnerabilities a repository has.

Resilience & streaming results

  • Failure isolationgather(return_exceptions=True) means one bad/unreachable repository is logged and skipped; the rest of the run is unaffected.
  • Streamed reportingSummaryStage is the last stage of each repository's own pipeline, so a small/fast repo prints its results immediately rather than waiting for a slow one.

Notable design decisions

  • trivy repo to avoid a git dependency for cloning and any temp-tree management.
  • Streaming the vulnerability list to .jsonl to keep memory bounded on large scans.
  • Per-resource semaphores (network / subprocess / cpu) instead of one global limit, so cheap fetches fan out while heavy subprocesses are capped.
  • GitHub token applied per request, read at send time and never stored on the client object (kept out of the client's state / repr).
  • Two report formatsreport.json (machine-readable) alongside report.html (human-readable, self-contained with the graph inlined).

Development

poetry install        # installs runtime + dev (pytest) dependencies
poetry run pytest     # run the test suite

Tests are offline and fast — external boundaries (GitHub, Trivy) are faked, so no network or Trivy binary is needed. The suite mirrors the package layout under tests/ and covers the Trivy-output parser, URL/popularity handling, the per-stage pipeline data-flow, and the concurrency model (e.g. that the shared semaphore caps concurrency globally rather than per repository).

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors