fix: set model id for custom models in llamastack config - #208
Closed
skattoju wants to merge 21 commits into
Closed
Conversation
…/v1.1.13 Release v1.1.13
# Conflicts: # deploy/cluster/helm/Chart.yaml # deploy/cluster/helm/values.yaml
…/v1.1.14 Release v1.1.14
# Conflicts: # deploy/cluster/helm/Chart.yaml # deploy/cluster/helm/values.yaml
…/v1.1.15 Release v1.1.15
# Conflicts: # deploy/cluster/helm/Chart.yaml # deploy/cluster/helm/values.yaml
…/v1.1.16 Release v1.1.16
# Conflicts: # deploy/cluster/helm/Chart.yaml # deploy/cluster/helm/values.yaml
…/v1.1.17 Release v1.1.17
Introduce a pluggable runner architecture so the chat service can dispatch to different agent frameworks (LlamaStack, LangGraph, CrewAI) based on the agent's runner_type. - Add runner_type column to virtual_agents table (default: "llamastack") - Add runner_type to Pydantic schemas (VirtualAgentBase, VirtualAgentUpdate) - Create BaseRunner abstract class defining the streaming interface - Extract all LlamaStack logic into LlamaStackRunner - Rewrite ChatService as a thin dispatcher that delegates to runners - Add Alembic migration for the new column - Add 24 unit tests covering dispatcher, base class, model, and schemas - Fix dev container build: replace dnf nmap-ncat with Python socket check Co-authored-by: Cursor <cursoragent@cursor.com>
The virtual agent GET endpoint now returns runner_type in the response body. Update the tavern integration test to expect this field. Co-authored-by: Cursor <cursoragent@cursor.com>
…from skattoju/langgraph-runner-abstraction feat: runner abstraction layer for multi-framework agent support
…MCP integration (rh-ai-quickstart#201) * feat: implement LangGraph runner for multi-framework agent execution Add a LangGraph-based runner that uses the prebuilt ReAct agent pattern with token-level SSE streaming, MCP tool integration, and session persistence via an in-memory checkpointer. Changes: - Add LangGraphRunner implementing BaseRunner with create_react_agent - Wire langgraph runner into ChatService dispatcher - Add LANGGRAPH_LLM_API_BASE/KEY/DEFAULT_MODEL config settings - Fix runner_type passthrough in virtual agent create/response APIs - Fix model name priority: agent.model_name takes precedence over default - Add LangGraph env vars to local dev compose.yaml - Add unit tests for LangGraphRunner - Add langgraph, langchain-openai, langchain-mcp-adapters dependencies Co-authored-by: Cursor <cursoragent@cursor.com> * feat: add declarative graph engine for LangGraph agents Add support for YAML-driven multi-node graph agents alongside the existing ReAct agent mode. Agents with a graph_config field use a declarative pipeline (LLM, MCP tool, MCP tool map, router nodes); agents without it continue using create_react_agent. Changes: - Add graph_config JSON column to VirtualAgent model + Alembic migration - Create graph_engine.py with async node execution, MCP protocol handling (httpx), template rendering, and SSE event streaming - Add dispatch logic in LangGraphRunner: graph_config -> graph engine, otherwise -> ReAct agent - Add runner_type and graph_config support to agent template system - Add vacation planner template with declarative graph config - Add 37 unit tests for graph engine + fix existing runner tests Co-authored-by: Cursor <cursoragent@cursor.com> * feat: add vacation planner MCP servers and compose integration Port three FastMCP servers from vacation-planner POC for use with the declarative graph engine: - travel_research_mcp (Tavily Search API) on port 7001 - hotel_mcp (SerpApi Google Hotels) on port 7002 - flight_mcp (SerpApi Google Flights) on port 7003 Each server includes Containerfile, requirements.txt, and server.py. Updated compose.yaml with service definitions and MCP URL env vars for the backend. Added SERPAPI_API_KEY to .env.example. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(frontend): add LangGraph UI support for graph agent streaming - Add runner_type and graph_config fields to Agent, AgentTemplate, and related TypeScript types - Add NodeStartedEvent/NodeCompletedEvent stream event types and GraphNodeContentItem content type for tracking graph node execution - Add handleNodeStarted/handleNodeCompleted stream handlers with nodeIdToLabel utility for human-readable node names - Wire node_started/node_completed event dispatch in useChat hook - Add GraphProgressTracker component (PatternFly ProgressStepper) showing per-node execution status with spinner/checkmark indicators - Add GraphNodeOutputSection component for labeled, expandable per-node output with markdown rendering - Update chat separateContent to detect graph nodes and render structured output sections instead of concatenated text - Add runner_type selector and graph_config JSON editor to agent creation form with validation - Show runner_type badge on agent cards and template deploy modal Co-authored-by: Cursor <cursoragent@cursor.com> * fix: incremental template sync so new YAML templates are added without DB wipe The startup template loader previously skipped loading entirely when any suites already existed, causing newly added templates (e.g. Vacation Planner) to never appear in the database. Now performs an incremental sync, inserting only missing suites and templates. Co-authored-by: Cursor <cursoragent@cursor.com> * fix: use inputs.destination for flight search instead of places_list_task output The flight_research_task node was referencing {outputs.places_list_task} for the destination, which produced garbled LLM output. Point it at {inputs.destination} so the user-supplied destination is used directly. Co-authored-by: Cursor <cursoragent@cursor.com> * fix: improve IATA code resolution with city-airport map and strict matching Add a well-known city-to-airport lookup map so cities like "Paris" resolve to CDG without relying on regex or SerpApi value scanning. Restrict _find_iata_in_candidates to only check explicit IATA keys (iata_code, iata, code), preventing false positives like "ILE" from "Ile-de-France". Reorder _resolve_airport_code to check the city map before regex extraction. Co-authored-by: Cursor <cursoragent@cursor.com> * fix: handle 404 in MCP session retry and stream graph via astream Expand the MCP session-invalid retry to also cover HTTP 404 responses containing "session" in the body, not just 400. Rewrite run_streaming to use LangGraph's graph.astream(state, stream_mode="updates") instead of manually iterating nodes. This preserves the full graph topology (conditional edges, routing) while still yielding incremental SSE events (node_started, response, node_completed) for real-time frontend progress updates. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: extract structured input fields from user message via LLM Add _extract_input_fields to LangGraphRunner that uses the LLM to parse structured parameters (destination, interests, dates, etc.) from the user's natural-language message. Extracted values override hardcoded defaults from input_fields in the graph config, so "weekend getaway to Paris" correctly sets destination to Paris instead of the YAML default. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(frontend): consolidate graph progress into inline status indicators Remove the standalone GraphProgressTracker stepper component from chat.tsx to eliminate duplicate checkmark sets. Integrate inline status indicators (checkmark / spinner / circle) directly into each GraphNodeOutputSection toggle, so progress and output appear together. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: parallel graph node execution with auto-dependency analysis Replace the linear node chain with automatic dependency analysis that scans template references ({outputs.node_id}, items_path, depends_on) to build a fan-out/fan-in topology. Independent nodes now execute concurrently via LangGraph's StateGraph with Annotated reducers for safe parallel state merging. For the vacation planner, hotel_research_task and flight_research_task now run in parallel with places_list_task, significantly reducing end-to-end latency. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(frontend): render graph node output as markdown with GFM support Add remark-gfm and apply PatternFly's pf-v6-c-content class to the GraphNodeOutputSection so that headings, links, tables, lists, and code blocks render properly instead of displaying raw markdown text. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(test): update template_startup test for incremental sync The test_skip_when_templates_exist test now properly mocks both the suite and template DB queries plus the YAML loader, matching the incremental sync logic that queries suites and templates separately. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(frontend): resolve eslint/prettier lint errors Auto-fix prettier formatting and prefix unused nodeId param with underscore to satisfy @typescript-eslint/no-unused-vars. Co-authored-by: Cursor <cursoragent@cursor.com> * fix: use python healthchecks for MCP servers (no curl in slim image) The python:3.11-slim base image used by MCP server containers does not include curl, causing healthchecks to always fail. Replace curl with a python urllib one-liner that is guaranteed to be available. Co-authored-by: Cursor <cursoragent@cursor.com> * fix: configure web search tool and improve LlamaStack error handling - Register builtin::websearch tool group with Tavily search provider in llamastack-run.yaml so the web_search tool type is recognized - Add env_file directive in compose.yaml to inject TAVILY_API_KEY from project root .env into the LlamaStack container - Handle error chunks in StreamAggregator where type is None but an error dict is present in the chunk payload - Add retry logic in the stream loop to gracefully exclude tools that are not found on the server, preventing 400 errors from breaking the entire response Co-authored-by: Cursor <cursoragent@cursor.com> * fix: use TCP socket healthchecks for MCP servers The streamable-http MCP transport returns 406 Not Acceptable on plain GET requests to /mcp, causing urllib-based healthchecks to always fail. Switch to TCP socket connection checks which reliably verify the server is listening without depending on HTTP response codes. Made-with: Cursor --------- Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: dev mode agent templates and local setup Co-authored-by: Cursor <cursoragent@cursor.com> * test: align integration tests with dev model llama3.2:1b * fix: align integration tests with LangGraph template changes - Allow extra keys (runner_type, graph_config) in template detail response - Add tavily-search provider to expected providers list - Drop question-mark requirement for demo questions (LangGraph templates use imperative prompts) Made-with: Cursor * fix: add graph_config to virtual agent endpoint test The VirtualAgent response now includes a graph_config field (from the LangGraph feature). Add it to the expected response so the strict key check passes. Made-with: Cursor --------- Co-authored-by: Cursor <cursoragent@cursor.com>
* working * abstract stream logic to make more readbale * refactor * working * working * working * fix: replace SQLite source compilation with pysqlite3-binary The backend container build was failing because `sqlite-devel` is unavailable in UBI9 default repos and compiling SQLite from source is fragile under QEMU emulation on Apple Silicon. Use the `pysqlite3-binary` pip package instead, which ships a pre-compiled SQLite >= 3.45 wheel. A startup monkey-patch in main.py swaps it in before ChromaDB (via CrewAI) imports sqlite3. Made-with: Cursor * fix: route CrewAI LLM through LlamaStack instead of hardcoded gpt-4o Add OPENAI_API_KEY and OPENAI_API_URL env vars to compose so CrewAI can initialize without errors, and replace the hardcoded gpt-4o model with llama3.2:1b which is actually registered in LlamaStack. Made-with: Cursor * fix: prevent duplicate task output in CrewAI multi-agent streaming Task content was emitted twice: once via real-time TEXT streaming and again when _drain_task_events re-emitted the full task output on completion. Track which tasks already received streamed content and skip re-emission in the drain, fixing doubled hotel results and repeated final questions. Made-with: Cursor * feat: add CrewAI runner config, switch to MaaS endpoints, and add CrewAI vacation planner template - Add CREWAI_LLM_API_BASE, CREWAI_LLM_API_KEY, CREWAI_DEFAULT_MODEL settings in backend config, mirroring the LangGraph runner pattern - Switch both LangGraph and CrewAI runners from local LlamaStack OpenAI-compat endpoint to configurable MaaS (Model-as-a-Service) endpoints with env var overrides and sensible defaults - Pass root .env file to all podman compose commands via --env-file so secrets and config flow through consistently - Add env_file directive to backend service in compose.yaml - Add CrewAI multi-agent vacation planner template with destination, hotel, flight researchers and itinerary planner using MCP tool servers - Strip transient reasoning content items from assistant messages on stream completion in the frontend useChat hook - Add LangGraph & CrewAI feature plan documentation - Remove unused vulture_whitelist.py Made-with: Cursor * feat: add env-configured default model option to model dropdown When LANGGRAPH_DEFAULT_MODEL or CREWAI_DEFAULT_MODEL env vars are set, a "Default Model (Environment Configured)" option appears in the model selector so MaaS models work without needing to be registered in LlamaStack. Each runner resolves the sentinel to its own env-configured default. The /llms endpoint also gracefully returns this entry when LlamaStack is unavailable. Made-with: Cursor * fix: resolve lint errors in frontend and backend Fix ESLint/Prettier formatting issues in useChat.ts and agents.tsx. Align .flake8 max-line-length (120) with pre-commit and CI config. Made-with: Cursor * fix: add noqa E402 to imports in main.py for pysqlite3 swap The pysqlite3/sqlite3 module swap must run before any other imports, which causes flake8 to flag all subsequent imports as E402. Made-with: Cursor * fix: CrewAI crew stops early — flight tool unused, noisy output, bad formatting - Remove async_execution from hotel/flight tasks for predictable sequential flow - Simplify flight agent prompt (overly prescriptive params confused the model) - Set max_iter=2 for tool-less agents to avoid 15 wasted ReAct cycles - Import and use Process enum instead of raw string - Track per-task streamed char count so fallback fires for low-output tasks - Improve hotel/flight tool output to return clean markdown - Strengthen itinerary planner prompt to prevent Thought/Action noise - Add agent/task assignment logging for easier debugging Made-with: Cursor * fix: pass dict instead of set for streamed_tasks in test The test_skips_reemit_for_streamed_tasks test was passing a set for streamed_tasks, but _drain_task_events expects a Dict[str, int] and calls .get() on it, causing an AttributeError. Made-with: Cursor --------- Co-authored-by: Ryan Johnson <johnson2500@live.com>
The llama-stack subchart requires each enabled model to have an `id` field for `registered_resources.models.model_id`. When using a custom model not predefined in the subchart, this field was never set, causing a pydantic ValidationError on startup. Default the id to the model key name, with an optional LLM_ID/SAFETY_ID override. Made-with: Cursor
Collaborator
|
Closing this PR in favor of new PR - #223 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
registered_resources.models.model_idto be a valid string, but when deploying a custom model (not predefined in the subchart), theidfield was never set by the install script, resulting in a pydanticValidationErroron LlamaStack startup.global.models.$LLM.id, defaulting to the model key name. An optionalLLM_ID/SAFETY_IDenv var allows overriding the identifier if needed.Test plan
llama-4-scout-17b-16e-w4a16) not predefined in the subchart — LlamaStack starts successfullyllama-3-1-8b-instruct) to verify no regression — the subchart's defaultidshould take precedence via Helm mergeMade with Cursor