Add Gazelle Clojure plugin (babashka-based parser)

[miridius]

Background

Bazel reads BUILD.bazel files to figure out what to build. In a large Clojure repo those files describe every clojure_library / clojure_test / clojure_binary target plus the deps between them. Hand-maintaining them at scale is impractical, so we generate them from the source tree.

We've been doing that with gen_srcs (a standalone bazel run target). It walks the repo, parses every Clojure namespace, computes deps, writes BUILD files. It works, but it's outside the normal Bazel lifecycle: no incremental updates, no path-scoped runs, no per-package directives, no interop with other languages also generating BUILD files in the same repo.

This PR adds an experimental Gazelle plugin as an alternative. gen_srcs continues to exist until the Gazelle plugin is deemed stable (we'd eventually drop it).

bazel run @rules_clojure//gazelle:gazelle_bin                  # whole-repo update
bazel run @rules_clojure//gazelle:gazelle_bin -- src/foo bar/  # path-scoped

What's Gazelle?

Gazelle is a BUILD-file generator that runs as a Bazel build target. You write a language plugin (in Go, Gazelle's own implementation language) that tells it how to find your source files and what rules to emit; Gazelle handles the rest (directory walk, BUILD-file parsing, rule merging, cross-package dep resolution). Most modern Bazel rule sets ship a Gazelle plugin (Go, Java, Python, Rust, …) so users can keep their BUILD files in sync with one command.

Why a subprocess?

Gazelle plugins are Go code. But namespace parsing belongs in Clojure (edamame already handles reader conditionals, (ns ...) forms, splice forms, etc.). There aren't good Clojure parsers for Go, plus it's MUCH easier for us to maintain Clojure code.

The plugin follows the Java Gazelle plugin architecture: the Go side is glue, the actual parsing lives in a long-running Clojure subprocess it talks to over stdio.

Why babashka, not JVM Clojure?

bb's ~30ms cold start (vs ~1s for the JVM) makes path-scoped runs viable: a per-file-save gazelle invocation completing in sub-second time vs. multi-second for the JVM variant.

The cost is that bb can't load tools.deps (Java reflection blocks AOT), so the bb side has its own rule-construction code (gazelle_server.bb) rather than reusing rules-clojure.gen-build. The two implementations are kept in sync via a shared parity fixture (test/rules_clojure/rollup_rules_fixtures.edn) loaded by both sides' tests.

flowchart LR
    Bazel[bazel run //gazelle:gazelle_bin] --> Plugin[Go plugin<br/>gazelle/]
    Plugin <-->|JSON lines<br/>on stdio| Server[bb subprocess<br/>gazelle_server.bb]
    Plugin --> Builds[BUILD.bazel files]

Loading

Life of a Gazelle run

Gazelle walks the repo top-down, calling hook points on every package. The plugin implements all four:

Hook	Job
Configure	Read per-package # gazelle: directives. On the root call, auto-discover deps.edn and boot the bb subprocess.
GenerateRules	RPC the bb server with the package's file list. Translate returned {kind, attrs} specs into *rule.Rule.
Resolve	For each clojure_library, walk its :requires and fill in :deps against Gazelle's cross-package index.
AfterResolvingDeps	Shut the subprocess down.

Wire protocol

The bb server speaks newline-delimited JSON on stdio. One request per line, one response per line.

Request	When	Response
init	Once at startup	Resolved dep graph (dep_ns_labels per platform), deps_bazel overrides, source_paths, ignore_paths
parse	Once per package	NamespaceInfo per basename group + the __clj_lib / __clj_files rollup rules

Stdio (not gRPC / sockets / etc.) because zero setup, crash-safe (subprocess dies, Go side sees EOF and log.Fatalfs), easy to debug (bb gazelle_server.bb and paste JSON at it).

What lives where

Rule construction is bb-side (gazelle_server.bb's ns-rules): AOT-vs-plain decisions (:bazel/clojure_library ns-meta), test attr passthrough (:bazel/clojure_test for size/tags/timeout), :require → dep label mapping, clojure_binary for :bazel/clojure_binary, java_library for .js siblings, and rollup composition. ns-rules returns [{:type :clojure_library :attrs {...}} ...]; Go translates verbatim into *rule.Rule.

The Go side does three things:

1. Subprocess plumbing. Start the bb server, marshal requests, parse responses, mark the runner dead on any I/O error so subsequent calls short-circuit instead of racing a corpse. Tees the subprocess stderr through an in-memory ring buffer so a crash's final lines surface in processExitInfo.
2. Wire-format translation. {:type :clojure_library :attrs {:name "core" …}} → rule.NewRule("clojure_library", "core") + r.SetAttr(...).
3. Dep resolution. Per clojure_library: intra-repo index first (matches (:require [my.foo]) to //src/my:foo when another package generated it), then init's dep_ns_labels per platform, plus per-target overrides from deps_bazel.

Static deps that don't need Gazelle's index (org_clojure_clojure, import-deps, gen-class-deps, ns-library-meta extras) are pre-merged bb-side and seeded into the dep set from the rule's existing :deps; Resolve only adds what genuinely needs the cross-package index.

Configuration

Per-package directives in BUILD-file comments:

# gazelle:clojure_enabled false        # skip this directory tree entirely
# gazelle:clojure_deps_edn deps.edn    # which deps.edn to use (default: root)
# gazelle:clojure_deps_repo @other     # use a different repo tag for external labels
# gazelle:clojure_aliases :dev,:test   # deps.edn aliases to activate at init time

Important

Migrating from gen_srcs? gen_srcs takes aliases via CLI args. The plugin reads them from # gazelle:clojure_aliases :a,:b,... in the root BUILD.bazel. If unset, the plugin defaults to every alias in deps.edn (matching gen_srcs's typical deps.install(aliases = [...]) invocation pattern).

Failure semantics

An empty GenerateResult for a previously-rule-bearing package looks to Gazelle like "delete every rule". A green run that wipes the build graph is worse than a noisy exit, so the plugin fails loud on:

• Parser startup / transport / shutdown errors (subprocess stderr tail included in the message)
• Walk errors under subdirHasClojureFiles (permission denied, broken symlink, etc.)
• Unknown rule kinds from the bb server (closed RuleKind enum)
• Malformed wire shapes: missing dep_ns_labels.clj / .cljs, or NamespaceInfo mixing the Clojure-group and JS-only shapes
• Missing rules_clojure bazel_dep in MODULE.bazel

bb side: the request loop catches Throwable (not just Exception) so OOM / VirtualMachineError can't silently kill the subprocess; non-fatal errors return a {type:"error", message: <full cause chain>} envelope so the actionable root cause surfaces.

Subdir rollup

Each package emits a __clj_lib / __clj_files rollup that aggregates its own rules plus the rollups of any Clojure-bearing subdirectories. Naively that's an O(n) WalkDir per package — quadratic across the tree. Gazelle's bottom-up walk lets the plugin record hasClojureContent[rel] for each visited package and consult it as an O(1) lookup when the parent gets generated (falls back to the on-disk walk for any subdir we haven't visited yet — defensive, shouldn't happen in normal Gazelle ordering).

Intermediate-only directories (no direct .clj/.cljs/.cljc/.js files, but Clojure-bearing subdirs) still emit rollup rules so consumers of //foo:__clj_lib keep working when foo/ is an aggregator with code only in foo/bar/.

Tests

• //test/rules_clojure:gazelle-server-bb-test wraps bb-side unit tests covering ns parsing (reader conditionals, splice forms), libspec shapes, @deps/BUILD.bazel multi-line parsing, rule rollup, cache invariants (corrupt sha, corrupt transit), find-output-base cache validation, scan-jar .cljc / .cljs paths, parse-group failure modes, the -main request loop, and handle-init source-path tiebreaker.
• //gazelle:gazelle_test is the Go-side Configure / GenerateRules / Resolve suite, including an end-to-end GenerateRules test against a stubbed bb script that exercises the full Configure → GenerateRules → Resolve pipeline.
• //gazelle/clojureparser:clojureparser_test is the wire-protocol round-trip tests using a self-contained tempdir fixture (tiny deps.edn + empty @deps/BUILD.bazel set via GAZELLE_DEPS_BUILD).

Cross-process parity between gazelle_server.bb's rollup-rules and gen_build.clj's rollup-rules is pinned by test/rules_clojure/rollup_rules_fixtures.edn, loaded by tests on both sides.

Relation to #84

Supersedes #84 (the JVM-Clojure parser variant). That PR can be closed once this one is reviewed.