Skip to content

stergiotis/boxer

Repository files navigation

boxer

Go Reference Go Report Card

Maturity

Alpha, incomplete test coverage, unstable, API may still change heavily.

Why

Boxer reimplements and vendors much of what mainstream practice imports or rents. The premises behind that trade — dependencies as owned liabilities with known incentives, problem-oriented description languages on a boring host, one machine-readable data model projected across memory / wire / storage, a toolkit that observes itself (runtime and code), mechanical sympathy as the efficiency posture, one machine-checked architect, and interfaces split by task complexity up to agentic operation — are stated, with their costs and failure modes, in doc/explanation/why-boxer.md.

Installation

go get github.com/stergiotis/boxer

What's inside

Boxer is a collection of packages under public/. The larger subsystems:

  • algebraicarch/pushout — algebraic three-way merge for line-graphs via categorical pushouts: graggle/store for the line-graph data structure, graggle/patch for the patch DAG, envelope for transmittable patches; the graggle and pseudo-edge constructs follow Joe Neeman's ojo design (see pushout/graggle/NOTICE); Includes a BackendI/RepoI seam under algebraicarch/pushout/pijul with two realisations — a native pushout backend (no external binary) and a text backend that shells out to pijul.
  • semistructured/leeway — code-driven entity-attribute-value data model with a staged codegen pipeline (DDL / DML / read-access / streaming read-access / marshalling).
  • semistructured/markdown/obsidian — goldmark-based parser for Obsidian-flavored markdown (callouts, wikilinks, embeds, tags, highlights, frontmatter).
  • db/clickhouse/dsl — typed ClickHouse SQL DSL with an AST, marshalling, and nanopass rewrite passes (ADR-0002, ADR-0006).
  • streaming/persisted/kafka — embedded Kafka producer/consumer derived from Redpanda Connect's franz-go integration (ADR-0005).
  • caching — read-through batch cache aimed at ETL / build / graph-traversal pipelines: latency-hidden via dependency accumulation and partition-aware bulk fetches, with optional disk-backed L2.
  • analytics/similarity/compression — compression-based similarity metrics (NCD, CCC) over any Reset-able compressor.
  • math/numerical/finddivisions and math/numerical/timeticks — axis-tick layout: Heckbert / Wilkinson / Talbot for numeric and log axes; a uPlot-derived calendar ladder with locale-aware boundary snapping for time axes.
  • science/geo/h3 — H3 geospatial indexing via a Rust→WASM→wazero bridge (ADR-0003); Rust source under rust/h3bridge.
  • thestack/imzero2 — ImZero v2: an egui-based immediate-mode UI stack (egui2 FFI bindings, widgets, the IDS design system, demo apps), rendered by the Rust backend under rust/imzero2.
  • thestack/fffi2 — Framed Foreign Function Interface: the typed FFI / IR layer imzero2 builds on (ADR-0049).
  • keelson — application runtime for imzero2 apps: an AppI registry with dock/CLI hosts, an in-process bus, a bus-codec, a facts store, background-task supervision, and a help system.
  • observability/sysmetrics — Linux system-metrics collectors (cpu, mem, disk, net, proc, sensors, battery, container, opt-in GPU backends) (ADR-0019).
  • science/geo/swisstopo — Swiss LV95 ⇄ WGS84 coordinate transforms, GeoTIFF elevation sampling, and line-of-sight queries.
  • fec — forward error correction (e.g. fec/ea/golay24).
  • eb, eh — structured error building and error handling.
  • batching, containers, hashing, identity, logical, observability, parsing, slices, statespace, unsafeperf, … — utility packages.

internal/ carries vendored third-party ports.

ImZero v1 (Dear ImGui-based) lives in imzero_imgui. ImZero v2 (egui-based) is part of this module: the Go side under public/thestack/{imzero2,fffi2}, its Rust egui renderer under rust/imzero2, and runnable demo applications under apps/ (play, imztop, capdemo, capinspector, taskdemo).

Building

Boxer uses Go build tags to gate optional features and Go experiments (GPU backends, the goexperiment.jsonv2 experiment, opt-in profiling). The canonical tag set lives in ./tags; pass it to every go build, go test, and go vet invocation:

go build -tags="$(cat ./tags)" ./...
go test  -tags="$(cat ./tags)" ./...
go vet   -tags="$(cat ./tags)" ./...

Without these tags, packages that opt into one of those features fail to compile with misleading undefined identifier errors.

Documentation

Boxer follows the Diátaxis framework (ADR-0001). Docs live next to the code they describe:

Style Conventions

File Extensions

Boxer uses chained file extensions (e.g. file.docx.pdf.txt):

.out.<ext>
Generated source code checked into the repository, e.g. myfile.out.go.
.gen.<ext>
Source code generated during the regular build (part of the binary distribution, not the source distribution), e.g. myfile.gen.go.
.idl.go
A (Framed) Foreign Function Interface (FFI) Interface Definition Language file — a subset of the Go language.

Folders

Path specificity increases with depth. Example: ./fec/ea/golay24fec is forward error correction (a well-known technical term); ea is Eingabe-Ausgabe (German for input/output, chosen to avoid clashing with stdlib io); golay24 is the specific algorithm.

Ideally the leaf package name is discriminative enough to drive IDE autocompletion.

Glossary

e2e
End-to-end.
ea
Input-output (German abbreviation, to distinguish from core packages).
fec
Forward error correction.
inst
Instance (similar to self / this).
vcs
Version control system (git, svn, hg, perforce, …).

Compliance

Third-party licenses are vetted by a CI gate that builds a CycloneDX SBOM with cyclonedx-gomod and enforces the project policy (ADR-0004). Inline ports of third-party code, the bundled h3.wasm artifact's license chain, and the gate's policy are documented in THIRD_PARTY_NOTICES.md. NOTICE carries the project's own attribution.

Contributing

Currently, no third-party contributions are accepted.

AI Codegen Declaration

Code and documentation up to commit aa78183 is 100% human-generated; subsequent work includes substantial LLM contributions. Per-commit authorship is recorded in the git history via Co-Authored-By trailers — the provenance source of record — and summarised over time by boxer gov repo authorship. Earlier revisions additionally gated LLM-authored files behind llm_generated_* build tags so an AI-free build stayed possible; that gate was retired once it no longer described a useful build (see ADR-0083). The provenance it cached remains derivable from the trailers.

License

The MIT License (MIT) 2023-2026 — Panos Stergiotis. See LICENSE for full terms.

About

Data Engineering Toolbelt - DB-to-Glass

Topics

Resources

License

Security policy

Stars

7 stars

Watchers

1 watching

Forks

Contributors