Skip to content

refactor: split glm.c → colibri.c + 4 header modules (−18%) - #391

Merged
JustVugg merged 6 commits into
JustVugg:devfrom
ZacharyZcR:refactor/split-colibri
Jul 19, 2026
Merged

refactor: split glm.c → colibri.c + 4 header modules (−18%)#391
JustVugg merged 6 commits into
JustVugg:devfrom
ZacharyZcR:refactor/split-colibri

Conversation

@ZacharyZcR

@ZacharyZcR ZacharyZcR commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Rename glm.ccolibri.c (the project is called colibrì, not glm)
  • Extract four self-contained modules into header-only files, following the existing st.h/tier.h/grammar.h pattern:
file lines content
quant.h 672 SIMD matmul kernels (AVX2/AVX-512/VNNI/NEON/i8mm/VSX), quantization — pure compute, zero Model dependency
sample.h 143 RNG (xorshift64*), top-p distribution, stop-set management
kv_persist.h 121 .coli_kv on-disk KV cache persistence
telemetry.h 189 Dashboard protocol lines (TIERS/EMAP/HITS), stats dump, usage save/load, hardware probe
  • Main engine: 6588 → 5396 lines (−18%)
  • make colibri is the new primary target; make glm kept as a phony alias
  • CI, setup.sh, coli CLI, and all 10 test files updated
  • make check passes: C + Python, 73 tests, zero warnings

Motivation

The 6.6k-line single file was the #1 barrier to review and community contribution — every PR touching the engine conflicted with every other. This is the first split pass, targeting the four cleanest modules (lowest coupling). The remaining ~5.4k lines (MoE dispatch, attention, expert I/O, serve protocol, main) are candidates for a follow-up once this lands.

Test plan

  • make colibri — zero errors, zero warnings (Linux, CPU)
  • make check — 73 tests pass (C unit + Python)
  • make glm — backward-compat alias works
  • CI (Linux / Windows / macOS) — automated on push

@ZacharyZcR
ZacharyZcR changed the base branch from dev to main July 18, 2026 20:05
@JustVugg JustVugg mentioned this pull request Jul 19, 2026
@JustVugg

Copy link
Copy Markdown
Owner

Sequencing note after v1.0.0 shipped: this refactor is wanted, but it's a 78-file rename+split that conflicts with everything in flight — merging it now would invalidate the rebase we just requested on #399 (the KV-quant series, which predates today's dev movement in the exact code this PR relocates). Proposed order: #399 lands first (it's semantically hard, this PR is mechanically hard — rebasing mechanics over semantics is the cheaper direction), then this gets a fresh rebase in a quiet window and goes in as the only open change touching the core. We'll ping you when the runway is clear — apologies for the wait, and thanks for the patience.

Rename glm.c → colibri.c and extract four self-contained modules
into header-only files (same pattern as st.h/tier.h/grammar.h):

  quant.h      (672 lines) — SIMD matmul kernels, quantization
  sample.h     (143 lines) — RNG, top-p sampling, stop-set
  kv_persist.h (121 lines) — .coli_kv disk persistence
  telemetry.h  (189 lines) — dashboard protocol, stats, usage

Main engine file shrinks from 6588 to 5396 lines (−18%).

Build system: primary target is now colibri$(EXE); `make glm`
remains as a phony alias for backward compat. CI, setup.sh,
coli CLI, and all 10 test files that include the engine are
updated. make check passes (C + Python, 73 tests, zero warnings).
New files:
  README.zh-CN.md — simplified Chinese (大陆用词)
  README.it.md    — Italian (the project's "mother tongue")

All four READMEs now link to each other in a consistent nav bar.
Updated zh-TW to reflect glm.c → colibri.c rename and new headers.
Lightweight i18n without react-i18next: a LocaleProvider context +
useLocale() hook with {{var}} interpolation, auto-detect from
navigator.language, persisted to localStorage.

98 translation keys across 4 locale files (en/zh-CN/zh-TW/it).
All user-visible strings in App/Brain/Profiling/ErrorBoundary are
now t() calls. Language switcher added to the sidebar footer.

Build clean (tsc + vite), 18 tests pass.
@ZacharyZcR
ZacharyZcR force-pushed the refactor/split-colibri branch from a1badb0 to 93b4a8e Compare July 19, 2026 13:12
@ZacharyZcR
ZacharyZcR changed the base branch from main to dev July 19, 2026 13:12
@JustVugg
JustVugg merged commit 61004dc into JustVugg:dev Jul 19, 2026
8 checks passed
JustVugg pushed a commit that referenced this pull request Jul 19, 2026
…d onto #391 split)

Re-derivation of e7/disk-class-instr @ de6dd6d (base caa49f7) onto
origin/dev @ 61004dc, after PR #391 split c/glm.c into c/colibri.c plus
quant.h/sample.h/kv_persist.h/telemetry.h/grammar.h. Semantic equivalence,
not a copy: same events, same accounting, re-sited onto the new tree.

Placement: colibri.c, unchanged from before the split. telemetry.h (#391)
is the dashboard/stats module (HWINFO/TIERS/EMAP/HITS protocol lines +
usage persistence) — prof_report(), expert_load_impl(), moe(), g_prof_io
and g_edisk_ns all stayed in colibri.c, so DISK-CLASS's aggregation and
printing follow them there. No relocation needed, no DEVIATION.

Read path: #362 (prefetcher-v3, 22509fc) turned out to be testbed-only
scope per its own merge message ("glm.c untouched") — confirmed zero
c/glm.c changes in that merge's diff. The seven expert_load() call sites
this patch touches (pipe_worker, expert_host_ensure, moe()'s OMP miss
loop, pilot_realload, repin_pass_limit, pin_load x2) are structurally
identical to the pre-split tree; the demand flag re-attaches at the same
sites with the same semantics (1 only at moe()'s own PIPE/OMP miss path,
0 everywhere else), so DISK-CLASS still counts demand loads only.

One real drift, unrelated to #362: #417 (cfcc742) fixed the exact "Metal
pre-routed FASE A never bumps the real elast/eaccess_clock" defect this
feature's comments described as a documented, deliberately-unfixed
upstream issue — the real clock now ticks in FASE A too. The private
elast_dc/eaccess_clock_dc clock is kept anyway: its job was never only
to route around that freeze, it also snapshots pre-bump state so a
call's own routing bump can't contaminate its own classification, and
keeping DISK-CLASS's bookkeeping fully separate from stock elast state
is what makes "byte-identical with PROF=0" provable by construction
instead of by argument. Code comments referencing the old defect are
updated to reflect the fix (historical note + #417/cfcc742 pointer)
rather than describing a bug that no longer exists.

Gates: make glm METAL=0 and METAL=1 both clean, zero warnings (matches
stock 61004dc, also built clean with zero warnings for comparison).
make test-c: 0 failures. make test-python: 77 tests, OK.

Authored by Fable 5 in Claude Code, analysis in partnership with
@monotophic.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
NeuralNotwerk added a commit to NeuralNotwerk/colibri that referenced this pull request Jul 21, 2026
…— rebased onto colibri.c

Rebases the JustVugg#399 series onto current dev (colibri.c + extracted headers, post-JustVugg#391),
which also picks up JustVugg#445's CUDA_RELEASE_HOST pin-budget fix for free. The KV-quant work
is additive — dev had no KV8/TQ code — so the split just relocated scaffolding:

  * colibri.c    — KV8/TQ globals + KVState fp8/packed byte caches, kv_alloc, the
                   attention consumers (CPU rotated-int4 orthogonality path; Metal/CUDA
                   native fused kernels dispatch), env parsing, the pin_load KV8-shadow
                   VRAM projection (merged with JustVugg#445's additive prefix budget).
  * kv_fp8.h / kv_tq.h  — the fp8 e4m3 and rotated-int4/PolarQuant codecs (new headers).
  * kv_persist.h — .coli_kv disk format v2 (fp8) / v3 (TQ): magic-tagged, on-load format
                   detection, v1->v2/v3 in-RAM upgrade + self-heal, fsync durability.
  * backend_metal.mm — two-library split (fixes the f32 fast-math regression: fast-math for
                   f32/consumer kernels, safe-math only for the RNE encoders) + native
                   codec-1 TQ4 fused attention (Hadamard-orthogonality: rotate the query
                   once, dot packed nibbles, unrotate the context; no f32 restage).
  * backend_cuda.cu/.h — native codec-1 TQ4 kernel (attention_absorb_kernel_tq) + host entry.
  * openai_server.py — dispatcher skips unrecognized engine telemetry frames instead of
                   tearing down (unified single/multi-slot interface, version-skew tolerant).

Note: the KV-tier flag definitions (g_kv8/g_tq/g_tq_bits/g_tq_codec/g_kv_shadow) moved above
the kv_persist.h include so the disk format can see them.

Validated on this rebase: `make colibri` clean; `make test-c` green (incl. kv_fp8, kv_tq
identities + MLA-consumer equivalence, and kv_disk v1/v2/v3 round-trip + upgrade + reject +
self-heal); `make metal-test` green (f32 byte-exact, native TQ4 1.3-1.5x faster than staging,
layer decode ok). CUDA compiles clean on the fleet box (sm_89+sm_120, cuda-test incl.
attention_absorb_tq) and ran coherent end-to-end on GLM-5.2 744B.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
NeuralNotwerk added a commit to NeuralNotwerk/colibri that referenced this pull request Jul 21, 2026
…— rebased onto colibri.c

Rebases the JustVugg#399 series onto current dev (colibri.c + extracted headers, post-JustVugg#391),
which also picks up JustVugg#445's CUDA_RELEASE_HOST pin-budget fix for free. The KV-quant work
is additive — dev had no KV8/TQ code — so the split just relocated scaffolding:

  * colibri.c    — KV8/TQ globals + KVState fp8/packed byte caches, kv_alloc, the
                   attention consumers (CPU rotated-int4 orthogonality path; Metal/CUDA
                   native fused kernels dispatch), env parsing, the pin_load KV8-shadow
                   VRAM projection (merged with JustVugg#445's additive prefix budget).
  * kv_fp8.h / kv_tq.h  — the fp8 e4m3 and rotated-int4/PolarQuant codecs (new headers).
  * kv_persist.h — .coli_kv disk format v2 (fp8) / v3 (TQ): magic-tagged, on-load format
                   detection, v1->v2/v3 in-RAM upgrade + self-heal, fsync durability.
  * backend_metal.mm — two-library split (fixes the f32 fast-math regression: fast-math for
                   f32/consumer kernels, safe-math only for the RNE encoders) + native
                   codec-1 TQ4 fused attention (Hadamard-orthogonality: rotate the query
                   once, dot packed nibbles, unrotate the context; no f32 restage).
  * backend_cuda.cu/.h — native codec-1 TQ4 kernel (attention_absorb_kernel_tq) + host entry.
  * openai_server.py — dispatcher skips unrecognized engine telemetry frames instead of
                   tearing down (unified single/multi-slot interface, version-skew tolerant).

Note: the KV-tier flag definitions (g_kv8/g_tq/g_tq_bits/g_tq_codec/g_kv_shadow) moved above
the kv_persist.h include so the disk format can see them.

Validated on this rebase: `make colibri` clean; `make test-c` green (incl. kv_fp8, kv_tq
identities + MLA-consumer equivalence, and kv_disk v1/v2/v3 round-trip + upgrade + reject +
self-heal); `make metal-test` green (f32 byte-exact, native TQ4 1.3-1.5x faster than staging,
layer decode ok). CUDA compiles clean on the fleet box (sm_89+sm_120, cuda-test incl.
attention_absorb_tq) and ran coherent end-to-end on GLM-5.2 744B.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
NeuralNotwerk added a commit to NeuralNotwerk/colibri that referenced this pull request Jul 21, 2026
…— rebased onto colibri.c

Rebases the JustVugg#399 series onto current dev (colibri.c + extracted headers, post-JustVugg#391),
which also picks up JustVugg#445's CUDA_RELEASE_HOST pin-budget fix for free. The KV-quant work
is additive — dev had no KV8/TQ code — so the split just relocated scaffolding:

  * colibri.c    — KV8/TQ globals + KVState fp8/packed byte caches, kv_alloc, the
                   attention consumers (CPU rotated-int4 orthogonality path; Metal/CUDA
                   native fused kernels dispatch), env parsing, the pin_load KV8-shadow
                   VRAM projection (merged with JustVugg#445's additive prefix budget).
  * kv_fp8.h / kv_tq.h  — the fp8 e4m3 and rotated-int4/PolarQuant codecs (new headers).
  * kv_persist.h — .coli_kv disk format v2 (fp8) / v3 (TQ): magic-tagged, on-load format
                   detection, v1->v2/v3 in-RAM upgrade + self-heal, fsync durability.
  * backend_metal.mm — two-library split (fixes the f32 fast-math regression: fast-math for
                   f32/consumer kernels, safe-math only for the RNE encoders) + native
                   codec-1 TQ4 fused attention (Hadamard-orthogonality: rotate the query
                   once, dot packed nibbles, unrotate the context; no f32 restage).
  * backend_cuda.cu/.h — native codec-1 TQ4 kernel (attention_absorb_kernel_tq) + host entry.
  * openai_server.py — dispatcher skips unrecognized engine telemetry frames instead of
                   tearing down (unified single/multi-slot interface, version-skew tolerant).

Note: the KV-tier flag definitions (g_kv8/g_tq/g_tq_bits/g_tq_codec/g_kv_shadow) moved above
the kv_persist.h include so the disk format can see them.

Validated on this rebase: `make colibri` clean; `make test-c` green (incl. kv_fp8, kv_tq
identities + MLA-consumer equivalence, and kv_disk v1/v2/v3 round-trip + upgrade + reject +
self-heal); `make metal-test` green (f32 byte-exact, native TQ4 1.3-1.5x faster than staging,
layer decode ok). CUDA compiles clean on the fleet box (sm_89+sm_120, cuda-test incl.
attention_absorb_tq) and ran coherent end-to-end on GLM-5.2 744B.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
BColsey added a commit to BColsey/colibri that referenced this pull request Jul 21, 2026
Retarget PR JustVugg#377 onto current dev (origin/dev @ 4aca059) after JustVugg#391 split
glm.c into colibri.c + quant.h/sample.h/kv_persist.h/telemetry.h. Reconciles
four collisions:

- JustVugg#391 split: all glm.c hunks resited -- prof_*/g_prof_io/ProfBase live in
  colibri.c (NOT telemetry.h); tiers_emit/emap_emit -> telemetry.h (with a
  rammap_slot forward-decl); st_fd_is_tmpfs/st_fd_fs_magic -> st.h; .coli_kv
  state-dir -> serve_ctx_init/run_serve/run_serve_mux + main.
- DISK-CLASS (1f00142): prof_physical_read_bytes/ProfPhysicalWire merged ON
  TOP of dev's dc_* fields; PROF protocol line extended 9->17 fields
  additively; expert_load_impl 6-arg `demand` signature preserved; g_prof_io
  routed through prof_ssd_tensor_bytes (tmpfs-excluded).
- DUAL-SSD mirror (JustVugg#298/JustVugg#469): ESlot.backing hand-merged with dev's
  aslab/afslab; map_of_fd exact-length + MADV_HUGEPAGE composes with rep_bfd.
- int3 fmt=5 (JustVugg#168): rammap_bind_one reuses dev's qt_resolve_fmt/detect_group_size
  (inline fmt-detection dropped; duplicate detect_group_size not re-added).

Verified: make colibri 0 warnings; make check 187 tests pass (test_uring skips
in sandboxes via the PR's helpers; test_rammap builds against colibri.c and
passes). Default path byte-identical -- oracle-safe by construction.

Co-Authored-By: Claude <noreply@anthropic.com>
JustVugg added a commit to KingIcyCreamProjects/colibri that referenced this pull request Jul 21, 2026
… conflicts + fix fmt=5 scale cap

Conflict resolution (13 hunks, 6 files):
- colibri.c / st.h: dev already carries an equal-or-stronger guard for the same
  threats (qt_resolve_fmt resolves AND validates the quant layout, refusing
  unknown byte counts instead of falling through to int2; the shape-product
  overflow guard is present). Took dev's side — no check is lost.
- olmoe.c: the load_expert_w hardening targeted a function the olmoe refactor
  replaced; dev now validates the equivalent path in load_expert_merged.
- web/*: i18n churn only, took dev's strings.

This PR's unique value is preserved and is the point of the merge: the server
hardening in openai_server.py (+121), plus json.h, tok.h, Makefile and the
downloader/requirements pinning.

Bug found and fixed while validating: st_read_f32_cap's call site bounded the
scale read with the PER-ROW cardinality (O), which is correct for int8/int4/int2
but rejects grouped formats — int3-g64 (fmt=5) keeps O*i3_groups(I) scales, so a
legitimate container was refused (tests/test_int3_load failed). The cap now
matches the cardinality qt_resolve_fmt already validates and falloc reserved.

Verified: clean build, token-exact unchanged (fp 32/32, int4 21/32),
tests/test_int3_load ok, full make check OK.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
pull Bot pushed a commit to bryanwills/colibri that referenced this pull request Jul 22, 2026
The JustVugg#391 rename left release.yml building 'make glm' (an alias that now
produces 'colibri') and then asserting/copying 'c/glm', which no longer
exists. CI never catches this: only a tag push runs this workflow, so the
v1.1.0 tag build failed on macOS at 'ls -lh glm' with the other jobs
cancelled and the release skipped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
pull Bot pushed a commit to danielabelski/colibri that referenced this pull request Jul 22, 2026
python3 openai_server.py --model <dir> looked for a binary named 'glm' next
to itself. Since JustVugg#391 the build produces 'colibri', so direct invocation was
broken on any clean checkout -- it only appeared to work in trees that still
had a stale glm from an older build. Spotted by @RDouglasSharp while working
on JustVugg#488.

Resolve the engine by probing colibri / colibri.exe first and falling back to
glm / glm.exe, the same order the coli launcher uses, so old trees keep
starting. Verified by removing the stale glm and running the gateway: it
resolves to colibri and the engine launches.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
BColsey added a commit to BColsey/colibri that referenced this pull request Jul 24, 2026
Retarget PR JustVugg#377 onto current dev (origin/dev @ 4aca059) after JustVugg#391 split
glm.c into colibri.c + quant.h/sample.h/kv_persist.h/telemetry.h. Reconciles
four collisions:

- JustVugg#391 split: all glm.c hunks resited -- prof_*/g_prof_io/ProfBase live in
  colibri.c (NOT telemetry.h); tiers_emit/emap_emit -> telemetry.h (with a
  rammap_slot forward-decl); st_fd_is_tmpfs/st_fd_fs_magic -> st.h; .coli_kv
  state-dir -> serve_ctx_init/run_serve/run_serve_mux + main.
- DISK-CLASS (1f00142): prof_physical_read_bytes/ProfPhysicalWire merged ON
  TOP of dev's dc_* fields; PROF protocol line extended 9->17 fields
  additively; expert_load_impl 6-arg `demand` signature preserved; g_prof_io
  routed through prof_ssd_tensor_bytes (tmpfs-excluded).
- DUAL-SSD mirror (JustVugg#298/JustVugg#469): ESlot.backing hand-merged with dev's
  aslab/afslab; map_of_fd exact-length + MADV_HUGEPAGE composes with rep_bfd.
- int3 fmt=5 (JustVugg#168): rammap_bind_one reuses dev's qt_resolve_fmt/detect_group_size
  (inline fmt-detection dropped; duplicate detect_group_size not re-added).

Verified: make colibri 0 warnings; make check 187 tests pass (test_uring skips
in sandboxes via the PR's helpers; test_rammap builds against colibri.c and
passes). Default path byte-identical -- oracle-safe by construction.

Co-Authored-By: Claude <noreply@anthropic.com>
NeuralNotwerk added a commit to NeuralNotwerk/colibri that referenced this pull request Jul 25, 2026
…— rebased onto colibri.c

Rebases the JustVugg#399 series onto current dev (colibri.c + extracted headers, post-JustVugg#391),
which also picks up JustVugg#445's CUDA_RELEASE_HOST pin-budget fix for free. The KV-quant work
is additive — dev had no KV8/TQ code — so the split just relocated scaffolding:

  * colibri.c    — KV8/TQ globals + KVState fp8/packed byte caches, kv_alloc, the
                   attention consumers (CPU rotated-int4 orthogonality path; Metal/CUDA
                   native fused kernels dispatch), env parsing, the pin_load KV8-shadow
                   VRAM projection (merged with JustVugg#445's additive prefix budget).
  * kv_fp8.h / kv_tq.h  — the fp8 e4m3 and rotated-int4/PolarQuant codecs (new headers).
  * kv_persist.h — .coli_kv disk format v2 (fp8) / v3 (TQ): magic-tagged, on-load format
                   detection, v1->v2/v3 in-RAM upgrade + self-heal, fsync durability.
  * backend_metal.mm — two-library split (fixes the f32 fast-math regression: fast-math for
                   f32/consumer kernels, safe-math only for the RNE encoders) + native
                   codec-1 TQ4 fused attention (Hadamard-orthogonality: rotate the query
                   once, dot packed nibbles, unrotate the context; no f32 restage).
  * backend_cuda.cu/.h — native codec-1 TQ4 kernel (attention_absorb_kernel_tq) + host entry.
  * openai_server.py — dispatcher skips unrecognized engine telemetry frames instead of
                   tearing down (unified single/multi-slot interface, version-skew tolerant).

Note: the KV-tier flag definitions (g_kv8/g_tq/g_tq_bits/g_tq_codec/g_kv_shadow) moved above
the kv_persist.h include so the disk format can see them.

Validated on this rebase: `make colibri` clean; `make test-c` green (incl. kv_fp8, kv_tq
identities + MLA-consumer equivalence, and kv_disk v1/v2/v3 round-trip + upgrade + reject +
self-heal); `make metal-test` green (f32 byte-exact, native TQ4 1.3-1.5x faster than staging,
layer decode ok). CUDA compiles clean on the fleet box (sm_89+sm_120, cuda-test incl.
attention_absorb_tq) and ran coherent end-to-end on GLM-5.2 744B.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
NeuralNotwerk added a commit to NeuralNotwerk/colibri that referenced this pull request Jul 25, 2026
…— rebased onto colibri.c

Rebases the JustVugg#399 series onto current dev (colibri.c + extracted headers, post-JustVugg#391),
which also picks up JustVugg#445's CUDA_RELEASE_HOST pin-budget fix for free. The KV-quant work
is additive — dev had no KV8/TQ code — so the split just relocated scaffolding:

  * colibri.c    — KV8/TQ globals + KVState fp8/packed byte caches, kv_alloc, the
                   attention consumers (CPU rotated-int4 orthogonality path; Metal/CUDA
                   native fused kernels dispatch), env parsing, the pin_load KV8-shadow
                   VRAM projection (merged with JustVugg#445's additive prefix budget).
  * kv_fp8.h / kv_tq.h  — the fp8 e4m3 and rotated-int4/PolarQuant codecs (new headers).
  * kv_persist.h — .coli_kv disk format v2 (fp8) / v3 (TQ): magic-tagged, on-load format
                   detection, v1->v2/v3 in-RAM upgrade + self-heal, fsync durability.
  * backend_metal.mm — two-library split (fixes the f32 fast-math regression: fast-math for
                   f32/consumer kernels, safe-math only for the RNE encoders) + native
                   codec-1 TQ4 fused attention (Hadamard-orthogonality: rotate the query
                   once, dot packed nibbles, unrotate the context; no f32 restage).
  * backend_cuda.cu/.h — native codec-1 TQ4 kernel (attention_absorb_kernel_tq) + host entry.
  * openai_server.py — dispatcher skips unrecognized engine telemetry frames instead of
                   tearing down (unified single/multi-slot interface, version-skew tolerant).

Note: the KV-tier flag definitions (g_kv8/g_tq/g_tq_bits/g_tq_codec/g_kv_shadow) moved above
the kv_persist.h include so the disk format can see them.

Validated on this rebase: `make colibri` clean; `make test-c` green (incl. kv_fp8, kv_tq
identities + MLA-consumer equivalence, and kv_disk v1/v2/v3 round-trip + upgrade + reject +
self-heal); `make metal-test` green (f32 byte-exact, native TQ4 1.3-1.5x faster than staging,
layer decode ok). CUDA compiles clean on the fleet box (sm_89+sm_120, cuda-test incl.
attention_absorb_tq) and ran coherent end-to-end on GLM-5.2 744B.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
NeuralNotwerk added a commit to NeuralNotwerk/colibri that referenced this pull request Jul 26, 2026
…— rebased onto colibri.c

Rebases the JustVugg#399 series onto current dev (colibri.c + extracted headers, post-JustVugg#391),
which also picks up JustVugg#445's CUDA_RELEASE_HOST pin-budget fix for free. The KV-quant work
is additive — dev had no KV8/TQ code — so the split just relocated scaffolding:

  * colibri.c    — KV8/TQ globals + KVState fp8/packed byte caches, kv_alloc, the
                   attention consumers (CPU rotated-int4 orthogonality path; Metal/CUDA
                   native fused kernels dispatch), env parsing, the pin_load KV8-shadow
                   VRAM projection (merged with JustVugg#445's additive prefix budget).
  * kv_fp8.h / kv_tq.h  — the fp8 e4m3 and rotated-int4/PolarQuant codecs (new headers).
  * kv_persist.h — .coli_kv disk format v2 (fp8) / v3 (TQ): magic-tagged, on-load format
                   detection, v1->v2/v3 in-RAM upgrade + self-heal, fsync durability.
  * backend_metal.mm — two-library split (fixes the f32 fast-math regression: fast-math for
                   f32/consumer kernels, safe-math only for the RNE encoders) + native
                   codec-1 TQ4 fused attention (Hadamard-orthogonality: rotate the query
                   once, dot packed nibbles, unrotate the context; no f32 restage).
  * backend_cuda.cu/.h — native codec-1 TQ4 kernel (attention_absorb_kernel_tq) + host entry.
  * openai_server.py — dispatcher skips unrecognized engine telemetry frames instead of
                   tearing down (unified single/multi-slot interface, version-skew tolerant).

Note: the KV-tier flag definitions (g_kv8/g_tq/g_tq_bits/g_tq_codec/g_kv_shadow) moved above
the kv_persist.h include so the disk format can see them.

Validated on this rebase: `make colibri` clean; `make test-c` green (incl. kv_fp8, kv_tq
identities + MLA-consumer equivalence, and kv_disk v1/v2/v3 round-trip + upgrade + reject +
self-heal); `make metal-test` green (f32 byte-exact, native TQ4 1.3-1.5x faster than staging,
layer decode ok). CUDA compiles clean on the fleet box (sm_89+sm_120, cuda-test incl.
attention_absorb_tq) and ran coherent end-to-end on GLM-5.2 744B.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
gohlerdev pushed a commit to gohlerdev/BetterColibri that referenced this pull request Jul 28, 2026
The tiny-model efficiency/regression suite silently skipped on every
tree since the JustVugg#391 rename: it looked for glm.exe (test_inefficiency
ENGINE, efficiency.run_engine default, test_cuda_env GLM) while the
build produces colibri — the exact bug class JustVugg#526 fixed in the gateway.
All three now use the gateway's resolution order (colibri, colibri.exe,
glm, glm.exe) with actionable skip messages.

Executing the suite for the first time exposed a flaky assertion:
test_cpu_vs_cpu_determinism bounded run-to-run tok/s drift at 25%, but
a tiny-model decode is ~20 ms of wall clock — scheduler noise, not an
engine property (measured >2x variance, ~1 failure in 3 runs). The
determinism contract keeps its exact greedy hit-rate equality (the
part that catches real non-determinism) and timing gets a sanity bound
(positive, finite) instead of a cross-run tolerance.

Validation: with the glm_tiny fixture present the suite now RUNS
(5 tests executed, was 8/8 skipped) and passes 5/5 consecutive runs;
full test-python 143 OK. Teacher-forcing oracle validated alongside:
TF=1 vs transformers reference = 32/32 at f32.
NeuralNotwerk added a commit to NeuralNotwerk/colibri that referenced this pull request Jul 28, 2026
…— rebased onto colibri.c

Rebases the JustVugg#399 series onto current dev (colibri.c + extracted headers, post-JustVugg#391),
which also picks up JustVugg#445's CUDA_RELEASE_HOST pin-budget fix for free. The KV-quant work
is additive — dev had no KV8/TQ code — so the split just relocated scaffolding:

  * colibri.c    — KV8/TQ globals + KVState fp8/packed byte caches, kv_alloc, the
                   attention consumers (CPU rotated-int4 orthogonality path; Metal/CUDA
                   native fused kernels dispatch), env parsing, the pin_load KV8-shadow
                   VRAM projection (merged with JustVugg#445's additive prefix budget).
  * kv_fp8.h / kv_tq.h  — the fp8 e4m3 and rotated-int4/PolarQuant codecs (new headers).
  * kv_persist.h — .coli_kv disk format v2 (fp8) / v3 (TQ): magic-tagged, on-load format
                   detection, v1->v2/v3 in-RAM upgrade + self-heal, fsync durability.
  * backend_metal.mm — two-library split (fixes the f32 fast-math regression: fast-math for
                   f32/consumer kernels, safe-math only for the RNE encoders) + native
                   codec-1 TQ4 fused attention (Hadamard-orthogonality: rotate the query
                   once, dot packed nibbles, unrotate the context; no f32 restage).
  * backend_cuda.cu/.h — native codec-1 TQ4 kernel (attention_absorb_kernel_tq) + host entry.
  * openai_server.py — dispatcher skips unrecognized engine telemetry frames instead of
                   tearing down (unified single/multi-slot interface, version-skew tolerant).

Note: the KV-tier flag definitions (g_kv8/g_tq/g_tq_bits/g_tq_codec/g_kv_shadow) moved above
the kv_persist.h include so the disk format can see them.

Validated on this rebase: `make colibri` clean; `make test-c` green (incl. kv_fp8, kv_tq
identities + MLA-consumer equivalence, and kv_disk v1/v2/v3 round-trip + upgrade + reject +
self-heal); `make metal-test` green (f32 byte-exact, native TQ4 1.3-1.5x faster than staging,
layer decode ok). CUDA compiles clean on the fleet box (sm_89+sm_120, cuda-test incl.
attention_absorb_tq) and ran coherent end-to-end on GLM-5.2 744B.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
JustVugg added a commit to CooperSheroy/colibri that referenced this pull request Jul 31, 2026
…y name

Both sides were partly right. This branch softens the tiny-oracle expectation
from 32/32 to ~30-32/32 (the point of the PR, per JustVugg#482), but writes it against
./glm.exe, which no longer exists -- glm.c became colibri.c in JustVugg#391. dev has the
current binary name and the old, too-strict number.

Merged rather than picked: dev's ./colibri.exe with this branch's ~30-32/32.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
BColsey added a commit to BColsey/colibri that referenced this pull request Aug 1, 2026
Retarget PR JustVugg#377 onto current dev (origin/dev @ 4aca059) after JustVugg#391 split
glm.c into colibri.c + quant.h/sample.h/kv_persist.h/telemetry.h. Reconciles
four collisions:

- JustVugg#391 split: all glm.c hunks resited -- prof_*/g_prof_io/ProfBase live in
  colibri.c (NOT telemetry.h); tiers_emit/emap_emit -> telemetry.h (with a
  rammap_slot forward-decl); st_fd_is_tmpfs/st_fd_fs_magic -> st.h; .coli_kv
  state-dir -> serve_ctx_init/run_serve/run_serve_mux + main.
- DISK-CLASS (1f00142): prof_physical_read_bytes/ProfPhysicalWire merged ON
  TOP of dev's dc_* fields; PROF protocol line extended 9->17 fields
  additively; expert_load_impl 6-arg `demand` signature preserved; g_prof_io
  routed through prof_ssd_tensor_bytes (tmpfs-excluded).
- DUAL-SSD mirror (JustVugg#298/JustVugg#469): ESlot.backing hand-merged with dev's
  aslab/afslab; map_of_fd exact-length + MADV_HUGEPAGE composes with rep_bfd.
- int3 fmt=5 (JustVugg#168): rammap_bind_one reuses dev's qt_resolve_fmt/detect_group_size
  (inline fmt-detection dropped; duplicate detect_group_size not re-added).

Verified: make colibri 0 warnings; make check 187 tests pass (test_uring skips
in sandboxes via the PR's helpers; test_rammap builds against colibri.c and
passes). Default path byte-identical -- oracle-safe by construction.

Co-Authored-By: Claude <noreply@anthropic.com>
mcollinswisc pushed a commit to mcollinswisc/colibri that referenced this pull request Aug 5, 2026
## The roster said four families; there are five

DeepSeek V4 Flash landed in JustVugg#165 and was tuned in JustVugg#839, but the README still
opened with "Four families run today" and its table stopped at OLMoE. Someone
scanning the front page had no way to learn the engine exists.

It is now in the opening line, in the roster table, and in the hardware table
above it -- ~167 GB on disk, 16 GB of RAM minimum and 22 comfortable, measured
on the reference box rather than estimated.

## The DeepSeek section described a version that no longer exists

It called the path "experimental" and said "DSpark is intentionally kept for a
separate stacked follow-up". DSpark is in, and the honest state is more
interesting than either claim:

  - the checkpoint streams with no conversion -- routed experts stay native fp4,
    dense stays fp8-e4m3 with UE8M0 block scales
  - greedy, one KV slot, no tools or grammar yet: said plainly, because finding
    that out from a rejected request is worse
  - --ram is the knob that matters. 43 x 256 routed experts are ~137 GiB and a
    token touches 301 of them, so the cache hit rate sets tok/s. It changes
    speed only, never output.
  - speculative drafting is implemented, verified, and OFF, with the numbers
    that made that call: 1 accepted in 15 for the markov drafter, 10 in 24 for
    full MTP, and a 14-token answer that took 495 seconds to replay its
    rejected suffixes

That last one is the point of documenting it at all. The code stays, the
measurement stays beside it, and whoever retries this on faster storage starts
from evidence instead of from scratch.

## Repo layout described a tree that has not existed since July

It listed `glm.c`, renamed to `colibri.c` in JustVugg#391 three weeks ago, and no other
engine -- so the file that runs GLM was wrong and the four files that run
everything else were missing. Also absent: quant.h, compat.h, expert_store.h,
route_trace.h, kv_prefix.h, the Metal and Vulkan backends, resource_plan.py,
and docker/.

Every path and every make target in the new listing was checked to exist on
this branch before it was written down.

The rule behind the layout is now stated, because it is the one that keeps
being violated: one .c per model family, over shared single headers. An engine
owns its architecture and nothing else. The recurring defects in this tree --
the OpenMP thread count, the KV prefix reuse, the NaN router guard -- are all
the same shape: a mechanism that landed in one engine and never reached its
siblings.

## Also

`#### Other supported models` now sits where the roster table is, so
`[Full roster ↓](#other-supported-models)` in the opening paragraph resolves to
the table instead of to prose four sections earlier.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants