Serving or rebuilding a 21M x 768d store needs a ~128-256 GB box today; the 4 vCPU / 16 GB serving-node parity target from #79 is unreachable regardless of scan speed.
Measured/verified contributors at 21M:
- Reopen repopulates every ChunkRecord.vector from dequantized codes (~64 GB fp32 resident in the document map) even for vector-only stores that never read them on the query path.
- Quantized codes are resident twice at serve time: the mutable bit-plane form (~8 GB) plus the derived SIMD-blocked scan cache (~8 GB).
- Chunk-id strings are held in several maps (documents, chunk_owner_by_id, stable-id bridges).
Directions, roughly in order of leverage:
- Skip vector materialization at reopen for vector-only stores; reconstruct on demand for the few paths that need rows (cluster build already reads reconstruction straight from the index).
- Drop or lazily discard the mutable code buffer once the blocked scan cache is built on read-only handles.
- Intern or shorten per-chunk id storage.
Goal: a 21M store serves within ~16 GB. The wiki_dpr benchmark (#85) has the store and pipeline to measure against.
Serving or rebuilding a 21M x 768d store needs a ~128-256 GB box today; the 4 vCPU / 16 GB serving-node parity target from #79 is unreachable regardless of scan speed.
Measured/verified contributors at 21M:
Directions, roughly in order of leverage:
Goal: a 21M store serves within ~16 GB. The wiki_dpr benchmark (#85) has the store and pipeline to measure against.