Skip to content

Document the 79 undocumented environment variables (and fix the scan that missed them) - #799

Open
ZacharyZcR wants to merge 1 commit into
JustVugg:devfrom
ZacharyZcR:docs/environment-coverage
Open

Document the 79 undocumented environment variables (and fix the scan that missed them)#799
ZacharyZcR wants to merge 1 commit into
JustVugg:devfrom
ZacharyZcR:docs/environment-coverage

Conversation

@ZacharyZcR

Copy link
Copy Markdown
Contributor

Draft. Docs only β€” no code touched.

The engine reads 212 environment variables. ENVIRONMENT.md covered 133.

The 79 missing ones are not all internal debug knobs. They include CTX_MAX, MAX_NEW, REP_PEN, CHAT, WARMUP, NOGPU, GPU_DEV β€” and essentially the entire configuration surface of the Kimi K3 engine (K3_BITS, K3_EXPERT_GB, K3_LAYERS, K3_MAXT, K3_VK_GB, …, 22 of them).

The doc still described a single engine

"The C engine binary (c/glm, built from c/glm.c) reads all of these."

That has not been true for a while. There are four binaries and they do not share a knob set β€” K3_* is kimi_k3-only, INK_* is inkling-only, and a variable exported for the wrong engine is silently ignored with nothing to tell you. The intro now carries an ownership table, and three new sections cover the sister engines:

  • Kimi K3 β€” 22 variables
  • Inkling β€” CTX_MAX, PIN_N, REP_PEN, INK_*, GPU_DEV, NOGPU
  • OLMoE β€” HOT, WIDE, SMOOTH, CONF_LIMIT, MAX_NEW, CHAT, EXPERT_DROP, WARMUP

Plus the genuinely shared ones that live in headers every engine includes (COLI_USAGE, USAGE_SAVE, COLI_USAGE_DECAY, RANS_*, OMP_NUM_THREADS).

Why the drift went unnoticed β€” MAINTAINING-DOCS.md, fixed here too

The maintenance procedure is what let this accumulate, so patching only the output would guarantee the same gap next time:

  1. It scanned c/*.c only. route_trace.h, rans.h, sample.h and omp_tune.h own knobs every engine inherits, and the CUDA/Metal/Vulkan backends own theirs β€” 25 variables live in files the scan never opened.
  2. It grepped for getenv(. COLI_PROMPT is read through getenv_utf8(), the Windows-safe wrapper, so the procedure reported an existing variable as removed.
  3. Its "documented but not in code" list is a trap as written. It flags COLI_API_KEY, COLI_DEBUG, COLI_TOOL_SALVAGE β€” all real, just read by openai_server.py/coli β€” plus prose like O_DIRECT and F_NOCACHE. Following it literally would have deleted correct rows. A note now says to check the Python side first.
  4. It did not exclude c/tests/, whose fixtures read their own variables (EXPERT_RAW, TMPDIR) that are not part of the user-facing surface.

Every command in the updated procedure was run as written before committing.

On the content

Each added row's default and behaviour comes from the call site, not from guesswork β€” including the measured numbers already recorded in the source comments:

  • XEXP β€” +11.6% on a 2-socket 48-core Ice Lake, neutral-to-negative on a 24-core box, hence opt-in
  • PIN_N β€” 83.6% hit / 0.32 tok/s at cap/4 vs 95.6% / 0.80 tok/s at 40/layer on the 975B
  • COLI_KV_SHARE β€” slot TTFT 50.1s β†’ 1.7s on 6x5090 with a 675-token shared prefix
  • COLI_VK_TEST_BALLAST β€” 7.9s @2.6k buffer objects β†’ 15.6s @4.3k with 2.9 GB VRAM still free

Ones that are genuinely internal are documented as internal rather than dressed up: COLI_GPU_FAIL_AFTER (fault injection), COLI_VK_TEST_BALLAST (reproducing a VRAM-pressure effect), I3_AVX512_TEST, DEBUG_LOGITS, COLI_LOGIT_DUMP.

Verification

212 variables, 212 documented β€” the code-vs-doc diff is now empty in the undocumented direction, checked with the corrected procedure from MAINTAINING-DOCS.md.

Nothing was deleted: every name the old procedure would have flagged for removal was traced to a real reader first (Python side or prose).

Not addressed here

colibri.c alone accounts for 187 getenv call sites and 111 global switches β€” 5-8x the other engines. Documenting them does not make that surface smaller, and consolidating it would change behaviour for existing users and scripts, so it is deliberately out of scope for a docs PR.

The engine reads 212 environment variables. ENVIRONMENT.md covered 133 of
them. The 79 missing ones are not all internal debug knobs -- they include
`CTX_MAX`, `MAX_NEW`, `REP_PEN`, `CHAT`, `WARMUP`, `NOGPU`, `GPU_DEV` and
essentially the entire configuration surface of the Kimi K3 engine.

Why the drift went unnoticed is in MAINTAINING-DOCS.md's own procedure,
which is fixed here too:

  It scanned `c/*.c` only. route_trace.h, rans.h, sample.h and omp_tune.h
  own knobs that every engine inherits, and the CUDA/Metal/Vulkan backends
  own theirs -- 25 variables live in files the scan never opened.

  It grepped for `getenv(`. COLI_PROMPT is read through getenv_utf8(), the
  Windows-safe wrapper, so the procedure reported an existing variable as
  removed.

  Its "documented but not in code" list is a trap as written: it flags
  COLI_API_KEY, COLI_DEBUG and friends, which are real but read by
  openai_server.py/coli, plus prose like O_DIRECT and F_NOCACHE. A note now
  says to check the Python side before deleting a row.

  It did not exclude c/tests/, whose fixtures read their own variables
  (EXPERT_RAW, TMPDIR) that are not user-facing.

The doc also still described a single engine: "The C engine binary (c/glm,
built from c/glm.c) reads **all** of these." That has not been true for
some time. There are four binaries and they do not share a knob set --
`K3_*` is kimi_k3-only, `INK_*` is inkling-only, and a variable exported
for the wrong engine is silently ignored with nothing to tell you. The
intro now carries an engine/ownership table, three new sections cover the
sister engines' own variables, and the maintenance procedure records how to
find an owner.

Every added row's default and behaviour comes from the call site, including
the measured numbers already in the source comments (XEXP's +11.6% on a
2-socket Ice Lake and neutral-to-negative on 24 cores; PIN_N's 83.6% vs
95.6% hit rate; COLI_KV_SHARE's 50.1s -> 1.7s TTFT).

After this the diff between code and doc is empty in the "undocumented"
direction: 212 variables, 212 documented.
@ZacharyZcR
ZacharyZcR marked this pull request as ready for review August 3, 2026 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant