Releases: hoffresearch/nest
Release list
v0.3.0
additive release within frozen format v1. existing v0.2 files load unchanged in v0.3 readers.
highlights
- int4 block-64 embeddings (encoding 7): the first real sub-int8 size lever; fused dequant+dot simd kernels bit-identical across avx2/neon/scalar; ships as the
nanopreset (size_ratio 0.209, recall@10 0.913 on the published ruler). - published preset ladder + honest net-of-fp reconciliation:
dat/measure/ladder.jsoncarries the whole curve (compressed, tiny, micro, nano, hybrid plus the mrl points); every sub-int8 row is machine-checkably disclosed as stored-precision, never a bare-slab claim. - g1 graph pillar: optional chunk-to-chunk csr (section 0x0C graph_adjacency, typed edges NEXT_CHUNK/SEMANTIC/CITATION), runtime CsrIndex + bounded bfs, new
search-graphsubcommand feeding the same mandatory exact rerank. content_hash-excluded, so citations stay stable. - matryoshka prefix truncation:
nest.build(mrl_dim=K)slices and re-l2-normalizes before quantization; additive manifest pair mrl_dim/full_dim; the int4 ladder is valid at {256, 192, 128}; recall cost on the non-mrl baseline published as an honest curve.micro(mrl256-int8) is the named preset. - forge-core FORGE-0a: the ingestion layer's frozen .fci canonical-intermediate schema in a separate cargo workspace, versioned independently of the nest format.
test surface
288 rust tests in the sovereign workspace (was 134 in v0.2.0) plus 6 forge-core tests on its own manifest.
ruler caveat
every recall@10 figure above is measured on a self-perturbation ruler: it reports rank-stability under quantization, not real-query retrieval, and is likely inflated. the real-query (mteb-style) ruler is gate-zero (pending).
compatibility
no format break. new optional sections (0x0A dictionary, 0x0B dedup_map, 0x0C graph_adjacency), wire encodings (4 intpack, 5 zstd_dict, 7 int4, 9 fsst, 10 txt_streams), and additive manifest fields (mrl_dim/full_dim, capabilities_ext) all live within v1. v0.2 readers skip the unknown optional sections and reject the new encodings with a typed error, never silently. files built without the new features stay byte-identical.
full deltas: doc/changelog.md.