Random access term dictionary - #12688

Open

Tony-X wants to merge 61 commits into

apache:mainfrom

Tony-X:ramdon_access_term_dict

Commits on Sep 19, 2023

Setup no-op Lucene90RandomAcessDictionaryPostingsFormat
Tony Xu
committed

Commits on Oct 16, 2023

Commits on Oct 26, 2023

Move the code to be under sandbox
Tony Xu
committed

Commits on Oct 27, 2023

Add interfaces for encoding/decoding TermStates
Tony Xu
committed
Make the concrete TermStateCodecComponents singletons
Tony Xu
committed

Commits on Oct 30, 2023

Fix the expected export module check
Tony Xu
committed
Implment TermStateCodecComponent.getBitWidth for monotonically increasing values
Tony Xu
committed

Commits on Nov 2, 2023

Implement a codec (not Lucene Codec) for IntBlockTermState
Tony Xu
committed
Add more javadoc and minor re-naming
Tony Xu
committed
TestTermStateCodecImpl to decode at non-block starting positions
Tony Xu
committed

Commits on Nov 6, 2023

Commits on Nov 7, 2023

Implement compact generic byte-oriented BitPacker
Tony Xu
committed
Fix issues identfied by precommit checks
Tony Xu
committed
Remove unused member field `totalNumBytesWritten`
Tony Xu
committed
Test TermStateCodecImpl with real compact bit-packer
Tony Xu
committed
Implement TermStateCodecImpl.getCodec for (TermType, IndexOptions)
Tony Xu
committed

Commits on Nov 9, 2023

Implement term (type, ord) lookup in TermsIndex
Tony Xu
committed

Commits on Nov 13, 2023

Commits on Nov 16, 2023

Setup sketch implementations for RandomAccessTermsDict
Tony Xu
committed
remove unneeded initialization of int to 0.
Tony Xu
committed
Support serialize/deserialize for TermsStats
Tony Xu
committed
Explictlty mark the generic type arugment of FSTCompiler<Long> in TermsIndexBuilder
Tony Xu
committed

Commits on Nov 18, 2023

Commits on Nov 19, 2023

Test serailize/deserialize multiple fields' term dictionary
Tony Xu
committed

Commits on Nov 20, 2023

Remove unused member in RandomAccessTermsDictWriter
Tony Xu
committed
Implement Lucene99RandomAccessTermsReader/Writer
Tony Xu
committed
Move the package from sandbox to codecs
Tony Xu
committed
Fix bugs in writing Lucene99RandomAccessTermsW
Tony Xu
committed

Commits on Nov 21, 2023

Implement TermsEnum for Lucene99RandomAccess
Tony Xu
committed

Commits on Nov 22, 2023

Commits on Nov 25, 2023

Optimize for less allocation
Tony Xu
committed
Make decode TermState allocation-free
Tony Xu
committed

Commits on Nov 26, 2023

Use ThreadLocal to reuse TermDataReader data objects
Tony Xu
committed
Forked FST.java to work with primtive long
Tony Xu
committed
Allocate only one set of buffers in TermDataReader instead of one set per type
Tony Xu
committed
Make TermDataReader lazily init its buffer and clone IndexInput
Tony Xu
committed

Commits on Nov 27, 2023

Implement BytesRefPrimitiveLongFSTEnum that works with a primitive long FST
Tony Xu
committed
Fix getFirstArc() bug in PrimitiveLongFST.
Tony Xu
committed
Reuse single IntBlockTermState in TermDataReader
Tony Xu
committed

Commits on Dec 15, 2023

Don't create slice description when requesting random-access input slice
Tony Xu
committed
Use primitive long FST for term lookup to avoid allocation from boxing-unboxing
Tony Xu
committed
Make RAFDirectory resilient to `null` description when slicing
Tony Xu
committed
Implement interesect
Tony Xu
committed
Lazy decode termstate in IntersectEnum
Tony Xu
committed
Minor non-functionarly change for TermsIndexBuilder
Tony Xu
committed
implement FST + FSA intersection that leverages fast addressing of arc/transitions
Tony Xu
committed
Reuse stack frames to avoid allocating too many Arc and Transitions
Tony Xu
committed