Comparison to Colibri PR #165 (DS4F support)

[jasalt]

Hi, ds4f has been quite capable and I've been interested in getting it running on AMD 780M iGPU / 64GB DDR5 (Vulkan) with disk streaming.

I found this project and noted also that Colibri has open PR with support at JustVugg/colibri#165 with related issue JustVugg/colibri#727.

While I haven't tried either yet, I'm interested about the practical differences between this project v.s. Colibri when/if it gets the ds4f support, and if the projects could benefit mutually in sharing testing data with different configurations e.t.c.

[jasalt]

Generated summary (w/ Pi & gpt-5.6-luna), facts unverified.

 Compared against PR #165’s current head (f60e3be9, open), this repository is a prototype/structure validation, while the PR   
 is a substantially more complete DeepSeek-V4 engine.                                                                          
                                                                                                                               
 ┌──────────────────┬────────────────────────────────────────┬───────────────────────────────────────────────────────────────┐ 
 │ Area             │ This repo                              │ Colibri PR #165                                               │ 
 ├──────────────────┼────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ 
 │ Inference        │ Hash router + checksums; no real model │ Full target-only V4 inference, attention, mHC, MoE,           │ 
 │                  │ computation                            │ tokenizer, greedy decode                                      │ 
 ├──────────────────┼────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ 
 │ Checkpoint       │ Single safetensors file or custom      │ Official sharded safetensors, native tensor                   │ 
 │ format           │ packed pool/trunk                      │ names/dtypes/shapes                                           │ 
 ├──────────────────┼────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ 
 │ Expert storage   │ Global fixed-slot cache; no lease      │ Per-layer ExpertStore, leases, mutexes, LRU/hot pinning,      │ 
 │                  │ ownership                              │ prefetch                                                      │ 
 ├──────────────────┼────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ 
 │ Dense weights    │ Packed trunk with pin/ring reader      │ Optional resident dense-layer cache; otherwise streamed layer │ 
 │                  │                                        │ tensors                                                       │ 
 ├──────────────────┼────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ 
 │ Routing          │ Deterministic fixture hash             │ Actual gate matmul, sqrt(softplus()), bias-based selection,   │ 
 │                  │                                        │ normalized weights                                            │ 
 ├──────────────────┼────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ 
 │ Quantization     │ Python FP8/I8 → MXFP4 converter        │ Native C quantization and FP4/FP8 matmul paths                │ 
 ├──────────────────┼────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ 
 │ Memory planning  │ Approximate double-based forecast      │ Checked uint64_t arithmetic, system reserve, dense/head/cache │ 
 │                  │                                        │ tier planning                                                 │ 
 ├──────────────────┼────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ 
 │ Validation       │ Minimal config and offset checks       │ Header/file bounds, dtype, rank, shape overflow, tensor-byte  │ 
 │                  │                                        │ consistency                                                   │ 
 ├──────────────────┼────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ 
 │ Tests            │ Synthetic fixtures and trace replay    │ Token-exact oracle, ownership/lifetime tests, native quant    │ 
 │                  │                                        │ tests, hostile-shape tests                                    │ 
 └──────────────────┴────────────────────────────────────────┴───────────────────────────────────────────────────────────────┘ 
                                                                                                                               
 ### PR fixes several issues found here                                                                                        
                                                                                                                               
 - File ownership: PR explicitly owns/closes safetensors indexes and tests cleanup on expert-store failure                     
   (c/tests/test_v4_ownership.c). This repo aliases pool.fd to st.fd, then closes it prematurely (src/main.c:90-91).           
 - Read failures: PR returns errors and invalidates failed expert leases. This repo silently publishes empty slots             
   (src/cache.c:125-131).                                                                                                      
 - Top-k safety: PR allocates routing arrays based on validated topk; this repo has fixed 64-element arrays and overflows when 
   topk > 64 (src/main.c:154-170).                                                                                             
 - Hostile metadata: PR validates safetensors file bounds, shapes, dtypes, and arithmetic in c/st.h:476-558; this repo largely 
   trusts offsets and sizes (src/st.c:28-63).                                                                                  
 - Numerical validation: PR has token-exact tests. This repo’s make test currently fails quantize_selftest because the NumPy   
   quantizer mishandles negative values (tools/convert-ds4f.py:678-685).                                                       
                                                                                                                               
 ### Important caveat                                                                                                          
                                                                                                                               
 The PR is not merely a better version of this implementation; it is a different scope and roughly an order of magnitude       
 larger. Porting it wholesale would also introduce more platform-specific code, OpenMP, native SIMD paths, and a larger        
 dependency/supply-chain surface. Its V4 core has no obvious malicious behavior, but the broader branch includes downloader    
 tooling using Hugging Face/ModelScope/curl.                                                                                   
                                                                                                                               
 Bottom line: this repo captures the cache/trunk design ideas, but PR #165 is the reference for production-grade checkpoint    
 validation, ownership, real routing, and inference.