Skip to content

Restore lightrag-hku 1.4.x compatibility and drop hardcoded 1536-dim embeddings - #8

Open
mmavka wants to merge 1 commit into
olafgeibig:mainfrom
mmavka:fix/lightrag-hku-1-4-x-compat
Open

Restore lightrag-hku 1.4.x compatibility and drop hardcoded 1536-dim embeddings#8
mmavka wants to merge 1 commit into
olafgeibig:mainfrom
mmavka:fix/lightrag-hku-1-4-x-compat

Conversation

@mmavka

@mmavka mmavka commented May 21, 2026

Copy link
Copy Markdown

Trying to run knowledge-mcp 0.4.2 against the latest lightrag-hku (1.4.16) with a local Ollama setup hit three errors in a row. This PR fixes all of them.

What's broken

1. llm_model_max_token_size removed in lightrag-hku 1.4.x

LightRAG.__init__() no longer accepts llm_model_max_token_size. Creating a KB fails with:

TypeError: LightRAG.__init__() got an unexpected keyword argument 'llm_model_max_token_size'

Dropped the kwarg from the constructor call in rag.py. The behaviour it used to control is now handled elsewhere in lightrag-hku and doesn't need to be set explicitly here.

2. Several QueryParam fields removed in lightrag-hku 1.4.x

The per-KB config template populates QueryParam with ids, model_func, chunk_top_k, and enable_rerank. None of those exist in the current QueryParam dataclass, so any query crashes with:

TypeError: QueryParam.__init__() got an unexpected keyword argument 'ids'

(or one of the other three, depending on dict iteration order).

Filtered those four keys out of final_query_params before the QueryParam construction. The existing description filter right above is doing the same dance, so I slotted the rest in next to it.

This is the conservative fix: the per-KB config can keep emitting those fields without breaking, and if any come back upstream we don't have to touch anything else.

3. openai_embed is hardcoded to 1536 dims

openai_embedding_func in openai_func.py delegates to lightrag.llm.openai.openai_embed, which is wrapped with @wrap_embedding_func_with_attrs(embedding_dim=1536) (i.e. OpenAI text-embedding-3-* dimensions baked in via decorator).

Fine for OpenAI, but anyone pointing the embedding endpoint at an OpenAI-compatible server with a different model (Ollama bge-m3 at 1024 dims, mxbai-embed-large also 1024, etc.) gets:

total elements (1024) cannot be evenly divided by expected dimension (1536)

embedding_dim is already configurable in config.yaml (lightrag.embedding.embedding_dim), but never honoured because the decorator on openai_embed overrides it.

Rewrote the inner function to call the OpenAI-compatible endpoint directly with AsyncOpenAI instead of going through openai_embed. Six lines, respects whatever dimension the endpoint actually returns. The EmbeddingFunc wrapper just below (lines 90–93) still reads embedding_dim from config, so dimensions are validated correctly downstream; they're just no longer pinned to 1536 in the inner function.

How tested

Local setup:

  • knowledge-mcp 0.4.2 with these patches
  • lightrag-hku 1.4.16
  • OpenAI-compatible LLM endpoint (small Go proxy fronting OpenRouter, gpt-oss-120b free)
  • Ollama serving bge-m3 (1024-dim) at http://localhost:11434/v1

After the patches:

  • create <kb> and add_text <kb> <file> succeed; entity/relation extraction runs cleanly through OpenRouter
  • Queries via the MCP tools (answer, query_hybrid, query_local, query_global, retrieve) all return synthesised answers with citations
  • KB persists across server restarts

Before the patches each of the three issues blocks at a different stage (create / first query / first embedding call).

Notes

  • Didn't touch the LightRAG dependency in pyproject.toml. Pinning to a known-good range would be more solid long-term, but it's outside what I needed today.
  • There's a cosmetic RuntimeError: no running event loop at shell exit. Unrelated, doesn't affect ingest or query results. Can look at it separately if useful.
  • The QueryParam kwarg filter is the minimal one: only keys I actually saw blowing up. There may be more dead fields in the default per-KB config; happy to do a follow-up sweep if you want.

Let me know if you'd rather split this into separate commits (compat fixes vs the embed change). They're independent, and I can rebase.

…embeddings

Three small fixes that together let knowledge-mcp work against the latest
lightrag-hku (1.4.16) plus arbitrary embedding dimensions.

1. Drop llm_model_max_token_size from the LightRAG constructor call in
   rag.py. The kwarg was removed upstream in 1.4.x, so KB creation was
   failing with TypeError before this.

2. Filter ids, model_func, chunk_top_k, and enable_rerank out of
   QueryParam kwargs before construction. Those fields were removed
   from QueryParam upstream but still ship in the default per-KB config
   template, so every query was crashing.

3. Replace the openai_embed wrapper in openai_embedding_func with a
   direct AsyncOpenAI client call. openai_embed is decorated with
   wrap_embedding_func_with_attrs(embedding_dim=1536), which silently
   pinned the embedding dimension to OpenAI text-embedding-3-* and broke
   any non-1536-dim endpoint (Ollama bge-m3, mxbai-embed-large, etc.).

Tested locally against lightrag-hku 1.4.16, OpenRouter as the LLM
endpoint, and Ollama bge-m3 (1024-dim) for embeddings. KB create,
add_text, and all five query MCP tools work end-to-end after the
patches; before, each issue blocks at a different stage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant