v3.14.0

Triton KV cache kernels (V9B), progressive HTTP download (V10), reshard tool (V11).

• V9B: Fused Triton pack/unpack kernels for GPUCompressedKVCache (2.94x faster), Triton rANS encode kernel
• V10: stream_from_hub() — decompress directly from HuggingFace CDN, zero .bs bytes written to disk
• V11: bigsmall reshard CLI — split, join, or rebalance .bs shards by layer boundary

See CHANGELOG.md for full details.

v3.13.1

Polish release on top of v3.13.0: BF16-native F32 auto-routing for models like Whisper, populated base fingerprints for auto-delta matching.

See CHANGELOG.md for full details.

v3.13.0

User-experience release: delta compression, new CLI commands (scan, apply, repair, info), resume on interrupted compresses, ECC mode, V8 codec, streaming improvements.

See CHANGELOG.md for full details.

v3.12.0

v3.12.0 ships binary index encoding for fast tensor lookup on large models, plus a dedup-regression test that pins the existing v2.2.0 tied-tensor feature against future refactors.

Step 0 storage analysis

Two of the three improvements in the spec turned out to be either already shipped or not applicable to the current architecture:

Spec item	Status	Why
Tensor deduplication	Already shipped (v2.2.0)	tied_ref codec + duplicate_map. Step 0 confirmed zero models in the local set have tied tensors — modern LLMs don't tie embed/lm_head. Now has a regression test on a synthetic tied model.
Layer-aligned shard splitting	Not implemented	BigSmall inherits the safetensors shard layout from the source HF model. Re-sharding is a separate "reshard" tool — out of immediate scope.
Binary index encoding	Shipped	New bigsmall.index.bin written alongside the JSON for models with ≥ 100 tensors.

Added

• bigsmall.hub_index.write_binary_index(directory, shard_paths) — 30-byte fixed-width records per tensor + shared name/codec tables. Magic BSIX, version 1.
• bigsmall.hub_index.read_binary_index(path) — same shape as read_index() plus a binary field with full per-tensor offset/codec records.
• bigsmall.hub_index.maybe_read_binary_index(directory) — graceful None fallback.
• Auto-write of .bin in compress_for_hub when tensor count ≥ 100.

Tests

• 6 new tests in tests/test_opt_step9.py: binary index roundtrip, threshold behaviour, missing-file fallback, bad-magic rejection, synthetic-tied-model dedup regression. 142 passed / 0 skipped / 4 deselected (up from 136).

Compatibility

• Zero changes to existing file formats.
• Older .bs shards work unchanged; binary index is purely optional.
• read_index() still defaults to JSON; binary path is opt-in via maybe_read_binary_index().

Install: pip install bigsmall==3.12.0

v3.11.0

v3.11.0 ships testing infrastructure — property-based tests, multi-platform CI, integration-marker discipline, and two new real-model integration tests.

Added

• Hypothesis property tests in tests/test_property_based.py (5 properties on random bf16/fp16/fp32 tensors of 1-3D shapes): roundtrip, valid codec name, fast/full verify, streaming-vs-standard md5 match. ~16 s for all five.
• @pytest.mark.integration marker registered via pytest.ini. Default pytest tests/ excludes via addopts = -m "not integration". Opt in with pytest -m integration tests/.
• The 2 historically-skipped GPT-2 tests are now @pytest.mark.integration — properly catalogued instead of silently skipped.
• tests/test_integration.py — 2 new real-model integration tests: compress_from_hub(gpt2) round-trip + full verify() on a real model.
• .github/workflows/ci.yml — ubuntu / windows / macos × Python 3.10 / 3.11 / 3.12. BIGSMALL_FORCE_CPU=1 so GPU/Triton tests skip themselves cleanly.

Test count

Snapshot	Passed	Skipped	Deselected	Integration
v3.10.0	131	2	0	—
v3.11.0	136	0	4	4

Skip count went 2 → 0. The 2 persistent skips are now properly tagged.

Deliberate scope notes

• No decode-speed regression: speed varies on CI hardware (Numba JIT cold-start), flaky on shared runners. Ratio regression already covered by test_codec_regression.py (locked baseline, 0.05pp tolerance).
• No GPU tests in CI: GitHub Actions runners have no GPU.

Install: pip install bigsmall==3.11.0

v3.10.0

v3.10.0 ships CLI improvements: new stat and diff subcommands, a fast verify --fast mode that runs in seconds even on multi-GB shards, a richer benchmark with per-layer-type breakdown, and --no-progress on the IO-heavy commands.

Added

• bigsmall verify --fast — header-only integrity check (offsets, codec names, index consistency). Runs in seconds on multi-GB shards. --verbose adds counters on the pass path. Also importable programmatically as bigsmall.verify.verify_fast(path).

• bigsmall stat <file.bs> — detailed per-tensor table (name, shape, dtype, codec, raw, cmp, ratio) plus summary footer with codec breakdown. Flags: --tensor <substring> filters, --sort {ratio,size,name}, --reverse.

• bigsmall diff <a.bs> <b.bs> — three-way structural diff (identical / changed / only-in-A / only-in-B). Header-only — does not decompress. Exit 0 identical, 1 if any difference.

• bigsmall benchmark enhancements — per-layer-type breakdown table (attn_qkv / attn_out / mlp_gate_up / mlp_down / norm / embedding / lm_head / other), peak-RSS measurement (via psutil), -o/--output flag, --no-detail to skip the breakdown.

• --no-progress flag on compress, decompress, and benchmark. Forwarded to the existing progress= argument. Default unchanged.

Tests

• 7 new tests in tests/test_opt_step7.py covering all four CLI features. 131 passed / 2 skipped (up from 124).

Compatibility

• Zero format changes. All existing .bs files work with all new commands.
• Existing CLI commands and verify (without --fast) unchanged.

Install: pip install bigsmall==3.10.0

v3.9.0

v3.9.0 ships streaming-compression infrastructure plus several ergonomics improvements.

Added

• bigsmall.compress_streaming(src, dst) — encodes one tensor at a time via safetensors lazy loading. Output bit-identical to compress() on models without tied weights (md5-verified). Trade-offs: no cross-tensor tied-weight dedup (most modern LLMs don't tie anyway), serial encode (no worker pool).
• bigsmall.compress_from_hub(repo_id, output_dir) — downloads each shard via huggingface_hub and runs compress_streaming. Peak RAM stays at one tensor regardless of model size.
• bigsmall.decompress_layers(bs_path, layer_indices, …) — decompress only the requested transformer layers. Useful for partial fine-tuning, layer analysis, early-exit inference.
• BigSmallStreamingModel(prefetch=N) — optional async prefetch worker (lazy init). Default disabled.
• Better error messages in from_pretrained(): missing path / missing config.json now surface actionable suggestions.

Memory measurement on Phi-3.5-mini shard 1 (4.97 GB raw)

Path	RSS growth	Python heap peak
compress(workers=1)	11.5 GB	11.79 GB
compress_streaming	8.19 GB	3.37 GB
Reduction	1.41x	3.50x

For a 70B model that's the difference between "needs 140 GB RAM" and "needs ~5 GB RAM".

Tests

• 5 new tests in tests/test_opt_step6.py. 124 passed / 2 skipped (up from 119).

What did NOT pan out

• Async prefetch doesn't unlock streaming inference throughput: with decode at ~3s/layer and GPU forward at ~85ms total/token, the decoder is the critical path. Prefetch is shipped as infrastructure; real unlock needs the GPU AC kernel (v3.2.0 Triton roadmap).
• compress_from_hub streams from the local HF cache, not the CDN itself. Truly buffered streaming from the CDN (zero local disk) would need re-implementing safetensors lazy loading on HTTP range requests — multi-day project, deferred.

Install: pip install bigsmall==3.9.0

v3.8.0

v3.8.0 is a research-only release. Two compression-ratio improvement ideas from OPT_STEP5_CLAUDE.md were investigated; both decision gates failed. No code changes ship. The findings are documented so future sessions don't re-investigate.

Idea 1: per-tensor custom exponent remapping → REJECTED

Spec predicted 0.2-0.5pp gain. Actual aggregate header saving: 0.00078pp of raw (125x below the 0.1pp gate). On 209 BF16 tensors across Phi-3.5-mini + Qwen3-8B shard 1, mean used exponents = 22.

Why off by ~250x: remapping is a bijection → H(remapped)=H(original). The Categorical AC coder is already at Shannon's optimum regardless of symbol labels. Only header overhead reduction could help (~330 B/tensor = negligible).

Idea 2: cross-model family pooled entropy → REJECTED

Family	Models	KL penalty	Header saving	Net
Qwen	Qwen2.5-32B + Qwen3-8B	27.4 MB	494 B	−27.4 MB
Gemma	gemma-3-{4b,12b,27b}-it	6.6 MB	1558 B	−6.6 MB

Same finding pattern as A2 (cross-tensor shared tables within one model) from the V4 lossless arc.

Why ship anyway

Per spec: "still bump to 3.8.0 with research findings documented. The measurement work is valuable even if nothing ships." The CHANGELOG entry is the documented answer for anyone tempted to try either approach again.

Compatibility

Zero code changes outside __version__ and CHANGELOG. 119 tests pass / 2 skipped. pip install bigsmall==3.8.0 is functionally identical to 3.7.0 with this CHANGELOG entry attached.

Install: pip install bigsmall==3.8.0

v3.7.0

v3.7.0 unlocks parallel tensor encoding on Windows. The historical hard-coded workers=1 default was overly conservative — Windows spawn-context multiprocessing works correctly and produces bit-identical output.

Speedup on Phi-3.5-mini partial shard (876 MB raw, 20 BF16 tensors)

Workers	Wall time	Speedup
1	115.19 s	1.00x
2	79.33 s	1.45x
4	63.30 s	1.82x
8	68.79 s	1.67x (past optimal)

Outputs are md5-identical across all worker counts.

Added

• Default workers = min(cpu_count, 8) on all platforms (was 1 on Windows). Override via BIGSMALL_WORKERS env var still works.
• encoder._safe_workers() — caps worker count by available RAM (psutil) and tensor count. Always returns ≥ 1.
• Explicit mp_context = spawn on ProcessPoolExecutor for cross-platform consistency. Same fix applied to compress_delta().

Tests

• 5 new tests in tests/test_multiprocessing.py. 119 passed / 2 skipped (up from 114).

Compatibility

• Output is deterministic across worker counts — every existing .bs file is reproducible at any workers setting.
• BIGSMALL_WORKERS=1 still selects the serial (no-pool) path.

What did NOT pan out

• Spec target >4x: actual 1.82x. Process-spawn + pickle overhead caps Windows-spawn at this workload. Pushing further needs Numba-warm workers or thread-pool variant.

Install: pip install bigsmall==3.7.0

v3.6.0

v3.6.0 ships bf16_se_single_kernel — the entire BF16 tensor encode and decode collapsed into one Numba @njit function per direction. Eliminates per-bucket Python boundary crossings AND numpy argsort (replaced with O(n) counting sort).

Largest single-session speedup of the v3.x speed arc.

Phi-3.5-mini shard 1 (128 BF16 tensors, 4.97 GB)

Codec	Encode	Decode	Decode vs AC
bf16_se_ac (3.3.0)	43.4 MB/s	25.7 MB/s	1.00x
bf16_se_rans (3.4.0)	45.0 MB/s	27.0 MB/s	1.04x
bf16_se_tans (3.5.0)	48.4 MB/s	58.4 MB/s	2.27x
bf16_se_single_kernel (3.6.0)	98.6 MB/s	117.5 MB/s	4.57x

Added

• bigsmall.codecs.single_kernel — one @njit function each for encode/decode covering the full pipeline (sign/exp/mantissa split, SE freq + rANS encode, O(n) counting-sort, per-bucket mantissa freq + rANS encode, blob assembly). Zero Python orchestration between phases.
• New codec name bf16_se_single_kernel registered.
• compress(prefer_speed=True) now picks bf16_se_single_kernel over bf16_se_tans when within +0.6% size tolerance.
• 6 new tests. 114 passed / 2 skipped (up from 108).

Compatibility

• All existing .bs files (3.0.0-3.5.0) decode bit-identically.
• bf16_se_single_kernel files require bigsmall >= 3.6.0.
• Default compress() behavior unchanged.

What did NOT pan out (honest)

• Spec gate of <0.2pp size cost: actual +0.45pp on Phi shard 1 (per-bucket rANS framing + slightly less-efficient Numba quantisation). Out-of-spec by ~2.25x but trade-off documented and opt-in.
• Streaming inference >1 token/sec: still ~130 s/token. Weight-decode speedup is real but streaming is bottlenecked by HF model setup + per-layer transfers, not entropy decoding.
• KV cache <100ms/pass: ~14 s at seq=2000 (down from 30s baseline). Real progress, not "live".

Install: pip install bigsmall==3.6.0