GML-1997 token calculator, GML-1998 Extra Chat model

[chengbiao-jin]

PR Type

Enhancement, Other

---

Description

• Add token-aware truncation across services

  • Introduce TokenCalculator utility
  • Propagate token_limit via config
  • Truncate context/retrieved text for LLMs

• Migrate to vector search queries

  • New GSQL queries return edges
  • Add chunk content streaming

• UI/WS improvements and RAG selection

  • Pass rag_pattern via query param/context
  • Better markdown tables, graph rendering

• Dependency and prompt upgrades

---

Diagram Walkthrough

flowchart LR
  cfg["llm_config with token_limit"] --> tc["TokenCalculator"]
  tc --> emb["Embeddings (truncate inputs)"]
  tc --> gen["Agent generation (truncate context)"]
  tc --> ret["Retrievers (truncate retrieved)"]
  ret --> vq["Vector Search GSQL (Hybrid/Community/Sibling)"]
  gen --> llm["LLM Answer"]
  vq --> edges["Edges + Results"]
  edges --> ui["UI Graph + Chat"]

Loading

File Walkthrough

	Relevant files
Enhancement	16 files token_calculator.py Introduce token counting and truncation utility                    +224/-0  embedding_services.py Enforce embedding input token limits                                          +26/-5    agent_generation.py Truncate context and support dict input                                    +21/-2    agent_graph.py Vector-search integration and configurable retrieval          +20/-16  BaseRetriever.py Truncate retrieved sources before response generation        +15/-0    tg_proxy.py Add sizeLimit and stabilize polling                                            +7/-5      graph_rag.py Stream chunk contents and coordinated shutdown                      +59/-9    ui.py Add rag_pattern param and WS auth handling                              +22/-10  ActionProvider.tsx WS connects with rag_pattern and robust events                      +17/-4    Bot.tsx Persist defaults and provide RagPattern context                    +28/-12  CustomChatMessage.tsx Enable GFM markdown for better tables                                        +6/-5      KnowledgeGraphPro.tsx Parse edges/nodes and render graph robustly                            +29/-14  StreamChunkContent.gsql Add query to stream chunk content                                                +8/-0      GraphRAG_Hybrid_Vector_Search.gsql Vector hybrid search returning edges and context                  +11/-7    GraphRAG_Community_Vector_Search.gsql Vector community search with edges output                                +5/-3      Chunk_Sibling_Vector_Search.gsql Sibling windowing search with edges                                            +5/-1
Configuration changes	2 files config.py Propagate token_limit to services                                                +8/-0      server_config.json Add global token_limit configuration                                          +1/-0
Documentation	1 files chatbot_response.txt Tighten output format and markdown guidance                            +3/-0
Dependencies	1 files requirements.txt Upgrade FastAPI and Starlette versions                                      +2/-2
Additional files	17 files Dockerfile +1/-1      tigergraph_embedding_store.py +1/-1      chatbot_response.txt +25/-7    generate_cypher.txt +1/-0      chatbot_response.txt +1/-0      generate_cypher.txt +1/-0      generate_function.txt +4/-1      util.py +2/-3      workers.py +1/-1      util.py +2/-2      package.json +13/-10  Contexts.tsx +2/-1      Start.tsx +28/-23  agent.py +3/-0      CommunityRetriever.py +1/-0      HybridRetriever.py +1/-0      supportai.py +51/-30

---