Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
58ead39
ci: GitHub Actions for engine, web UI, Python tests, and CUDA syntax …
ZacharyZcR Jul 13, 2026
f7c7ab6
ci: run make check on ubuntu / windows (MSYS2 UCRT64) / macos (#140)
lEWFkRAD Jul 13, 2026
6728029
fix: set stdout to O_BINARY on Windows to prevent READY sentinel corr…
Jul 16, 2026
9cf97a4
Fix Windows cuda-dll build: quote NVCC, gate MSVC warnings, add confi…
mohamedmastouri2000-boop Jul 16, 2026
2b182ae
glm: COLI_NUMA=1 interleaves resident weights across NUMA nodes — +13…
ZacharyZcR Jul 16, 2026
09bf17f
PROF=1: opt-in performance profile (latency percentiles, expert I/O, …
claude Jul 14, 2026
63a6824
PROF=1: per-request reports in mux serve mode too (the web-dashboard …
claude Jul 14, 2026
6afffbc
Profiling page: per-turn phase timings, live in the web dashboard
claude Jul 14, 2026
36394bc
Profiling: fix the phase accounting so 'Other' stops swallowing the d…
claude Jul 15, 2026
deafa72
docs: serve protocol reference — mux wire format, telemetry lines, HT…
ZacharyZcR Jul 16, 2026
a3e9425
fix(oracle): create glm_tiny/ before saving so a fresh checkout works
tt1203 Jul 16, 2026
2a5961a
tests: exactness oracle for the grouped-int4 kernel (fmt=4)
JustVugg Jul 16, 2026
05f9bd4
docs: add winget python one-liner to Windows install block (#310)
Magnet-js Jul 16, 2026
6429123
README: restructure around the technical path — parameters and full r…
ZacharyZcR Jul 16, 2026
ac7103f
tests: extend the fmt=4 oracle to the fused gate+up kernel (#298)
JustVugg Jul 16, 2026
6de32c5
Don't trust a converted checkpoint's config for the stop set
JustVugg Jul 16, 2026
fa821a1
oracle: hard-fail on transformers < 5.11.0 (interleaved-RoPE floor, #…
woolcoxm Jul 16, 2026
c70d943
Merge branch 'main' into fix/windows-cuda-dll-build
mohamedmastouri2000-boop Jul 16, 2026
c5f2027
olmoe: PPL=1 teacher-forced perplexity mode (loss meter for throughpu…
lornecodes Jul 16, 2026
7bc9f6e
docs/media: refreshed dashboard shots (full-residency 4 tok/s, disk 0…
ZacharyZcR Jul 16, 2026
5272d19
Merge pull request #144 from ZacharyZcR/ci/github-actions
JustVugg Jul 16, 2026
78c77bf
ci: make the CUDA job able to fail, and able to run (#144)
JustVugg Jul 16, 2026
95b059b
Merge pull request #318 from tt1203/fix/oracle-mkdir-glm-tiny
JustVugg Jul 16, 2026
c686f6b
Merge pull request #320 from Magnet-js/docs/winget-python-readme
JustVugg Jul 16, 2026
b04c2ca
Merge pull request #322 from woolcoxm/fix/oracle-transformers-pin
JustVugg Jul 16, 2026
545d9d0
Merge pull request #317 from ZacharyZcR/docs/serve-protocol
JustVugg Jul 16, 2026
59988c4
Merge pull request #143 from lEWFkRAD/ci/make-check-matrix
JustVugg Jul 16, 2026
644e0d1
Merge pull request #302 from alekseysorokin68/main
JustVugg Jul 16, 2026
2dca36a
Merge pull request #314 from mohamedmastouri2000-boop/fix/windows-cud…
JustVugg Jul 16, 2026
3cec90c
Merge pull request #313 from ZacharyZcR/feat/numa-expert-interleave
JustVugg Jul 16, 2026
98f7c88
numa: parenthesise the page-align arithmetic (#313)
JustVugg Jul 16, 2026
ee5d273
Merge pull request #232 from nbeerbower/profiling-upstream
JustVugg Jul 16, 2026
e7188df
tests: test_stops must not assume /tmp exists (fixes the windows job)
JustVugg Jul 16, 2026
b75422d
ci: build the Windows CUDA path (nvcc + MSVC host) — the one toolchai…
lEWFkRAD Jul 16, 2026
57030ab
Merge pull request #327 from lEWFkRAD/ci/windows-cuda-build
JustVugg Jul 16, 2026
cead3d8
Merge pull request #324 from dawnfield-institute/feat/olmoe-ppl-mode
JustVugg Jul 16, 2026
80f886c
convert_olmoe.py: wire --ebits through to quantization (#323)
mohamedmastouri2000-boop Jul 16, 2026
69284fa
docs: Windows 11 native install walkthrough - toolchain, SAC, CUDA DL…
mohamedmastouri2000-boop Jul 16, 2026
d45cfa2
Merge branch 't321' into trial321b
JustVugg Jul 16, 2026
6507817
st: chunked pread with EINTR retry and honest short-read errors
nbeerbower Jul 16, 2026
ed8dab4
test_st_pread: Windows-compatible — relative tmpdir, fork subtest gat…
nbeerbower Jul 16, 2026
81a5677
glm.c: detect 9p by statfs f_type, not the "/mnt/" path prefix
tonuonu Jul 17, 2026
721bfcb
docs: add Traditional Chinese README
bm1016bm-svg Jul 17, 2026
3f239db
tools: rate-scale the E8 lattice ball by bit-width — int3-e8 is now a…
ZacharyZcR Jul 17, 2026
528b3cf
fix(flake): package engine, support modules, and python for Nix build…
woolcoxm Jul 17, 2026
8bf4cb9
coli: chat attaches to a running serve — the engine survives the chat…
JustVugg Jul 17, 2026
7eb2393
coli: serve pidfile + coli stop — one command to shut engine and serv…
JustVugg Jul 17, 2026
617ae51
Merge pull request #348 from woolcoxm/fix/nix-flake-packaging-345
JustVugg Jul 17, 2026
d5327e2
omp: cap LLVM libomp idle spin — a parked engine must not burn 3000% …
JustVugg Jul 17, 2026
03e643f
fix(flake): create flake.lock, use --set-default for COLI_ENGINE
LordMZTE Jul 17, 2026
411f237
fix(warnings): silence two -Wall/-Wextra warnings on the MinGW build
woolcoxm Jul 17, 2026
5e2be61
fix(test): make test_stops build on Windows (mkdtemp compat shim)
woolcoxm Jul 17, 2026
099e0c1
docs(env): refresh ENVIRONMENT.md — document all 34 drifted env vars
woolcoxm Jul 17, 2026
ba00889
sampling: partial top-p select in dist_build — O(V) heapify + k pops,…
woolcoxm Jul 17, 2026
37c96ee
bench: add bench_topp -- head-to-head old-qsort vs new-heap timing on…
woolcoxm Jul 17, 2026
5d16368
sampling: partial top-keep select in attention_rows DSA — O(nk) quick…
woolcoxm Jul 17, 2026
211d448
win32: stop erasing the real ReadFile error — pread failures name the…
JustVugg Jul 17, 2026
1552db2
Merge pull request #347 from ZacharyZcR/tools/e8-rate-scaled
JustVugg Jul 17, 2026
1223f9c
cuda: batch ragged attention across independent streams
ZacharyZcR Jul 17, 2026
1af9435
Merge pull request #328 from mohamedmastouri2000-boop/fix/olmoe-ebits
JustVugg Jul 17, 2026
59d74ae
Merge pull request #340 from tonuonu/fix/fs-detect-9p-statfs
JustVugg Jul 17, 2026
3d0a28b
Merge pull request #350 from woolcoxm/fix/build-warnings-dev
JustVugg Jul 17, 2026
c1e709e
Merge pull request #351 from LordMZTE/nix-flake
JustVugg Jul 17, 2026
d6d28f0
Merge pull request #353 from woolcoxm/docs/environment-refresh-dev
JustVugg Jul 17, 2026
d2226e2
Merge pull request #346 from bm1016bm-svg/docs/readme-zh-tw
JustVugg Jul 17, 2026
19f2677
Merge pull request #329 from mohamedmastouri2000-boop/docs/windows-na…
JustVugg Jul 17, 2026
1a956b0
Merge pull request #331 from nbeerbower/st-pread-full
JustVugg Jul 17, 2026
70a5879
Merge branch 'p354' into trialsamp
JustVugg Jul 17, 2026
8b36736
Merge branch 'p357' into trial357
JustVugg Jul 17, 2026
ef97f3c
Windows: fix non-ASCII chat prompt corruption (ANSI codepage vs UTF-8)
vologn4mor Jul 17, 2026
7f70a8d
sampling: guard against non-finite logits (was silent token-0 spew)
KingIcyCreamProjects Jul 17, 2026
e2d39ab
sampling: NaN-skip the mx scan too (review follow-up on #369)
KingIcyCreamProjects Jul 17, 2026
786f98d
cuda: load ragged attention entry point on Windows
ZacharyZcR Jul 17, 2026
c3a90ec
profile CPU GPU tier execution costs
ZacharyZcR Jul 17, 2026
946fcd4
glm: fmt=4 support in qt_addrow/qt_matvec_rows — CPU absorb path deco…
bokiko Jul 17, 2026
73e0e5e
docs(api): add "Connect a coding CLI or editor" recipe (#373)
KingIcyCreamProjects Jul 17, 2026
570d738
profile effective CPU expert bandwidth
ZacharyZcR Jul 17, 2026
153ee16
bench: multi-seed bench_dsa_select — kill single-input pivot-luck spi…
woolcoxm Jul 17, 2026
1deaa25
Merge pull request #375 from bokiko/fix/cpu-absorb-fmt4
JustVugg Jul 17, 2026
d5dac9b
Merge pull request #376 from KingIcyCreamProjects/docs/connect-coding…
JustVugg Jul 17, 2026
6ce6cc4
Merge pull request #374 from ZacharyZcR/perf/p0-execution-profile
JustVugg Jul 17, 2026
4fb5b4e
sampling: survive non-finite logits instead of emitting token 0 forev…
JustVugg Jul 17, 2026
eb6d432
docs: collapse WINDOWS.md/windows.md case collision into one file (#371)
JustVugg Jul 17, 2026
db09466
convert(fp8->int4): --mtp/--indexer respected on the --indir path, no…
JustVugg Jul 17, 2026
679c074
telemetry: split expert hits into pin-tier vs LRU ecache (#336)
JustVugg Jul 17, 2026
b09273f
serve: MTP status message tells the truth in multiplexed mode (#358)
JustVugg Jul 17, 2026
605b0e2
Merge pull request #365 from ZacharyZcR/feat/ragged-batch-attention-u…
JustVugg Jul 18, 2026
8a1dcca
docs(api): tell coding-CLI users about the prefill cost before they h…
JustVugg Jul 18, 2026
f8e4dc9
serve: honor --ngen when the client omits max_tokens; convert: print …
JustVugg Jul 18, 2026
6d3a616
cuda: keep ragged KV resident across decode steps
ZacharyZcR Jul 17, 2026
9c17dc6
plan NUMA interleave on multi-socket Linux
ZacharyZcR Jul 18, 2026
4fcbea9
Merge branch 'p370' into t370
JustVugg Jul 18, 2026
3fed654
Merge pull request #372 from ZacharyZcR/feat/paged-ragged-kv
JustVugg Jul 18, 2026
caa49f7
Merge pull request #384 from ZacharyZcR/perf/auto-numa
JustVugg Jul 18, 2026
6a2ab47
convert: resolve --mtp/--indexer shards from the local index instead …
bokiko Jul 18, 2026
799fe41
coli: honor THINK=1 in run mode
Jul 18, 2026
308e269
convert(fp8->int4): --indir copies the full metadata set and resumes …
Jul 18, 2026
d6676c1
tools: repair_mtp_int8.py — fix int4-converted MTP heads in place; wa…
NeuralNotwerk Jul 18, 2026
05bba79
release: version infrastructure + GitHub Release workflow
ZacharyZcR Jul 18, 2026
d2d3a7b
Fix CUDA detection on Windows: cuda_binary()/cuda_linkage() always re…
Stonki13 Jul 18, 2026
4d780e2
Merge branch 'p378' into trialsafe
JustVugg Jul 18, 2026
e5b1e19
Merge branch 'p366' into trialsafe
JustVugg Jul 18, 2026
6690c4f
Merge branch 'p394' into trialsafe
JustVugg Jul 18, 2026
36d389b
Merge pull request #402 from anrasi/fix/coli-run-think
JustVugg Jul 19, 2026
19bab42
Merge pull request #404 from anrasi/fix/indir-meta-resume
JustVugg Jul 19, 2026
3f99d2b
Merge pull request #395 from Stonki13/fix/windows-cuda-detection
JustVugg Jul 19, 2026
66b5e57
Merge pull request #397 from NeuralNotwerk/fix/mtp-int8-repair
JustVugg Jul 19, 2026
d94c681
Merge pull request #352 from woolcoxm/fix/test-stops-windows
JustVugg Jul 19, 2026
2122c00
serve: end-to-end tool-calling regression test + unparsed-marker diag…
JustVugg Jul 19, 2026
0f606bc
test: skip the tools e2e suite on Windows (shebang mock engine)
JustVugg Jul 19, 2026
c90e2cc
glm: measured-RSS guard — the RAM budget enforces itself at the safe …
JustVugg Jul 19, 2026
2f8fefd
Merge pull request #408 from JustVugg/p401-tools-e2e
JustVugg Jul 19, 2026
e25aeec
Merge pull request #410 from JustVugg/p403-rss-guard
JustVugg Jul 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions .github/workflows/check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# CI: run the repo's own dependency-free gate (`make check` = clean + portable
# CPU build + C unit suites + Python stdlib tests) on the three claimed
# platforms. No model downloads, no CUDA, no external deps — by design (#140).
name: check

on:
push:
branches: [main, dev]
pull_request:
branches: [main, dev]

jobs:
linux:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: make check
run: make -C c check

windows:
# The job that would have caught #68/#137 pre-merge: native MinGW-w64
# (MSYS2/UCRT64), the exact toolchain the README's Windows port targets.
runs-on: windows-latest
defaults:
run:
shell: msys2 {0}
steps:
- uses: actions/checkout@v4
- uses: msys2/setup-msys2@v2
with:
msystem: UCRT64
update: false
install: >-
make
mingw-w64-ucrt-x86_64-gcc
mingw-w64-ucrt-x86_64-python
- name: make check
run: make -C c check

macos:
# clang; libomp for the threaded path (Makefile falls back to
# single-threaded automatically if it's ever missing).
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
- name: install libomp
run: brew install libomp
- name: make check
run: make -C c check
72 changes: 72 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,78 @@ jobs:
-Xcompiler=-Wall,-Wextra
echo "CUDA syntax check passed"

windows-cuda-build:
# The Windows CUDA path has no coverage anywhere: `engine-cuda-syntax` above
# compiles with nvcc's *GCC* host on Linux, and check.yml's windows job is
# MinGW/UCRT64 CPU-only by design (#140). But nvcc on Windows requires MSVC
# as its host compiler — it does not accept MinGW — so `make cuda-dll` runs a
# toolchain nothing else in CI touches. That gap is not theoretical: every
# bug in #158 was a *build* failure on this path (MSVC rejects the GCC-style
# -Xcompiler=-Wall,-Wextra with "D8021 invalid numeric argument '/Wextra'",
# unresolvable CUDA_HOME/NVCC defaults, POSIX setenv in the kernel test), and
# #314 was CUDA_HOME with spaces — the layout the CUDA installer ships by
# default. Both classes are compile-time and need no GPU to catch.
#
# Build-only ON PURPOSE: GitHub's hosted runners have no NVIDIA device, so
# this job proves the Windows+MSVC CUDA build stays buildable, NOT that the
# kernels or the DLL loader behave on real silicon. That still needs hardware
# (see #157). Claiming otherwise would be the false confidence the
# engine-cuda-syntax comment above already warns about.
name: CUDA build (Windows, MSVC host)
# windows-2022, NOT windows-latest: the latest image now ships Visual Studio
# 18 (MSVC 14.5x), and CUDA's crt/host_config.h hard-errors on any host newer
# than VS 2022 ("Only the versions between 2017 and 2022 (inclusive) are
# supported"). That is a real constraint for every CUDA user on Windows, not
# a CI quirk — pinning tracks what the toolkit actually supports. Revisit when
# a CUDA release accepts VS 18; -allow-unsupported-compiler would only mask it.
runs-on: windows-2022
steps:
- uses: actions/checkout@v4
- name: MSVC environment (puts cl.exe on PATH for nvcc -ccbin)
uses: ilammy/msvc-dev-cmd@v1
- name: Install CUDA toolkit (compiler only)
uses: Jimver/cuda-toolkit@v0.2.19
with:
# Same pin as engine-cuda-syntax: v0.2.19's version table stops at
# 12.6.2. This installs to the default "C:\Program Files\NVIDIA GPU
# Computing Toolkit\..." path, so CUDA_HOME is space-bearing here —
# which is exactly the #314 regression this job would have caught.
cuda: '12.6.2'
method: network
# cudart as well as nvcc: unlike the Linux job (which only needs to
# *compile* backend_cuda.cu), cuda-dll links it, and on Windows the
# runtime headers/import lib ship as a separate installer component —
# with '["nvcc"]' alone this fails at `#include <cuda_runtime.h>`.
sub-packages: '["nvcc", "cudart"]'
- uses: msys2/setup-msys2@v2
with:
msystem: UCRT64
update: false
# inherit: cl.exe (msvc-dev-cmd) and nvcc (cuda-toolkit) are added to
# the *Windows* PATH by the steps above; without inheriting it the
# recipe's `command -v` guards fail inside the MSYS2 shell.
path-type: inherit
install: >-
make
mingw-w64-ucrt-x86_64-gcc
- name: make cuda-dll (nvcc + MSVC host)
shell: msys2 {0}
run: |
cd c
# CUDA_ARCH is pinned: the default is `native`, which asks the driver
# what card is present — there is none here, so it must be explicit.
# sm_80 matches engine-cuda-syntax and is supported by the 12.6 pin.
make cuda-dll CUDA_ARCH=sm_80
test -f coli_cuda.dll || { echo "cuda-dll reported success but produced no DLL" >&2; exit 1; }
echo "coli_cuda.dll built (MSVC host)"
- name: make glm CUDA_DLL=1 (host links backend_loader, not cudart)
shell: msys2 {0}
run: |
cd c
make glm CUDA_DLL=1
test -f glm.exe || { echo "glm CUDA_DLL=1 reported success but produced no exe" >&2; exit 1; }
echo "glm.exe built against the DLL loader"

web:
name: Web UI
runs-on: ubuntu-latest
Expand Down
99 changes: 99 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
name: Release

on:
push:
tags: ['v*']

permissions:
contents: write

jobs:
build:
strategy:
matrix:
include:
- os: ubuntu-latest
name: linux-x86_64
ext: ""
make_args: "ARCH=x86-64-v3"
- os: macos-latest
name: macos-arm64
ext: ""
make_args: ""
- os: windows-latest
name: windows-x86_64
ext: ".exe"
make_args: ""
shell: msys2
runs-on: ${{ matrix.os }}
defaults:
run:
shell: ${{ matrix.shell || 'bash' }}
steps:
- uses: actions/checkout@v4

- if: matrix.shell == 'msys2'
uses: msys2/setup-msys2@v2
with:
msystem: UCRT64
update: false
install: >-
make
mingw-w64-ucrt-x86_64-gcc

- if: matrix.os == 'macos-latest'
run: brew install libomp

- name: Build engine
run: |
cd c
make glm ${{ matrix.make_args }}
ls -lh glm${{ matrix.ext }}

- name: Package
run: |
TAG=${GITHUB_REF#refs/tags/}
mkdir -p dist
cp c/glm${{ matrix.ext }} dist/colibri-${TAG}-${{ matrix.name }}${{ matrix.ext }}
cp c/coli dist/
cp c/version.py dist/
cp c/openai_server.py dist/
cp c/resource_plan.py dist/
cp c/doctor.py dist/
cp LICENSE dist/
cd dist
if [ "${{ matrix.ext }}" = ".exe" ]; then
7z a colibri-${TAG}-${{ matrix.name }}.zip *
else
tar czf colibri-${TAG}-${{ matrix.name }}.tar.gz *
fi

- uses: actions/upload-artifact@v4
with:
name: colibri-${{ matrix.name }}
path: dist/colibri-*.*

release:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: actions/download-artifact@v4
with:
path: artifacts
merge-multiple: true

- name: Create GitHub Release
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG=${GITHUB_REF#refs/tags/}
# Extract the latest version section from CHANGELOG
NOTES=$(awk "/^## \\[${TAG#v}\\]/{found=1;next} /^## \\[/{if(found)exit} found{print}" CHANGELOG.md)
if [ -z "$NOTES" ]; then
NOTES="Release ${TAG}"
fi
gh release create "$TAG" artifacts/* \
--title "colibrì ${TAG}" \
--notes "$NOTES"
53 changes: 53 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Changelog

All notable changes to colibrì are documented here.
Format follows [Keep a Changelog](https://keepachangelog.com/).

## [1.0.0] — 2026-07-19

First tagged release. The engine has been running in production since late June
2026; this tag marks the baseline for semantic versioning going forward.

### Highlights

- **GLM-5.2 (744B MoE)** runs on ~25 GB RAM in pure C, streaming experts from disk
- **Three-tier placement**: VRAM (hot) / RAM (warm) / NVMe (cold), with a learning
cache that pins your workload's hottest experts automatically
- **CUDA backend**: multi-GPU expert tier, dense tensor distribution, batched
ragged attention, resident pipeline (`COLI_CUDA_PIPE=2`)
- **Metal backend** (Apple Silicon): batched expert SwiGLU + fused decode attention
on unified memory GPU
- **MTP speculation**: native GLM-5.2 draft heads, grammar-forced drafts, kernel-
pinned verification (`SPEC_PIN=1`)
- **OpenAI-compatible API**: `coli serve` with SSE streaming, KV slots, bounded
queue, web dashboard (`coli web`)
- **Web UI**: chat with live metrics, expert cortex brain page, profiling breakdown,
expert atlas 3-D galaxy
- **Cross-platform**: Linux, macOS, Windows 11 (native MinGW), PowerPC; CI on all three
- **Auto-tune**: `coli plan --auto-tier` classifies the bottleneck and derives
MTP/PIPE/NUMA/PIN settings with explanations

### Engine

- Token-exact validation against `transformers` oracle (teacher-forcing 32/32)
- Compressed MLA KV cache (576 floats/token, 57× smaller), persisted across
restarts (`.coli_kv`, zero re-prefill)
- DSA sparse attention (lightning indexer), faithfully implemented
- Router-lookahead prefetch (`PILOT=1`, 71.6% predictive)
- Async expert I/O pool (`PIPE=1`), io_uring batching (`URING=1`)
- NUMA-aware expert placement (`COLI_NUMA=1`, +13–40% on multi-socket)
- AVX2 / AVX-512 / AVX-VNNI / ARM NEON / NEON-i8mm / POWER VSX kernels
- int4 / int8 / int2 / grouped-int4 (fmt=4) quantization formats

### Tools

- `coli convert` — FP8→int4 one-shard-at-a-time converter
- `coli doctor` — read-only setup diagnostics
- `coli plan` — resource planner with auto-tune prescription
- `coli bench` — MMLU / HellaSwag / ARC quality benchmarks
- Expert atlas (`tools/analyze.py --web`) — measured topic affinity for 19,456 experts

### Community

- 30+ hardware datapoints in the benchmark tracker
- Contributions from 20+ authors across engine, docs, tooling, and ports
Loading
Loading