Skip to content

feat(memory): add a rebuildable local lexical index for Mate topics #111

Description

@robinbraemer

Outcome

Add a local rebuildable lexical/BM25 index for bounded Mate memory topics without changing Markdown authority.

Scope

  • Chunk topic bodies with metadata and stable source hashes.
  • Evaluate Bun/SQLite FTS5 and simpler in-process alternatives before selecting the smallest reliable derivative format.
  • Build and update the index atomically after validated topic mutations.
  • Support deterministic top-k search, type/scope filtering, and bounded recency signals.
  • Fall back to the current selector when the index is missing, stale, corrupt, or incompatible.

Acceptance criteria

  • Deleting the index and restarting rebuilds equivalent results from Markdown.
  • Partial writes and Pod interruption cannot make stale results appear current.
  • Index files are private to the Mate PVC and never copied to PostgreSQL or another Mate.
  • Search and rebuild stay within declared CPU, memory, disk, and latency budgets.
  • Lexical retrieval is evaluated against the baseline before becoming a default candidate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions