feat(ai-orchestrator): improve mesh router classification for inferred/generic queries#143
Merged
Merged
Conversation
Add 6 test cases for mesh router generic query inference: - Status/requirement queries route to po - Architecture/tooling queries route to architect - Accessibility/compliance queries route to a11y
Validate that MESH_SYSTEM_PROMPT includes all 6 persona domains and contains required routing instruction keywords. This prevents accidental prompt regression during future refactors.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the AI Orchestrator’s Ambiguity Mesh Router prompt so generic /query-style messages (prefixed with @isolate-) can be classified to the most appropriate specialist persona instead of defaulting to null, and adds unit tests to validate the new routing behavior and prevent prompt regressions.
Changes:
- Expanded
MESH_SYSTEM_PROMPTwith persona domain descriptions and inference guidance for@isolate-generic queries. - Exported
MESH_SYSTEM_PROMPTto enable direct prompt-constant testing. - Added new unit tests covering inferred routing for generic queries plus basic prompt-content validations.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| libs/ai-orchestrator/src/orchestrator/mesh-router.ts | Exports and updates the mesh router system prompt to support best-fit persona inference for @isolate- generic queries. |
| libs/ai-orchestrator/src/tests/mesh-router.spec.ts | Adds inference-focused test cases and prompt-constant validation tests. |
Address review feedback:
1. Update MESH_SYSTEM_PROMPT to explicitly state that inference only
applies to messages starting with @isolate-, preventing unintended
mesh jumps for ordinary in-thread questions without explicit persona tags.
2. Change 'best-fit' to 'ONLY' in the keyword validation test to clarify
the restricted scope.
3. Strengthen domain description test to check for actual bullet prefixes
('- po:', etc.) instead of loose substring matching.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #142
Update the mesh router to enable LLM-based inference of the best-fit specialist persona for generic
/querycommands, rather than returningnullwhen no explicit persona tag is present. The webhook-listener already prefixes all/querycommands with@isolate-, ensuring every manual query reaches the LLM for classification.Key improvement: Generic queries like "What is the status of this issue?" now route to the
po(Product Owner) persona instead of silently failing or being routed to an unrelated specialist (e.g.,docsora11y).Changes
Core Changes
MESH_SYSTEM_PROMPTto include:po, architecture questions toarchitect, accessibility questions toa11y@isolate-<persona>tags override inferenceTest Changes
MESH_SYSTEM_PROMPTconstant (all personas present, required keywords present)Configuration Changes
MESH_SYSTEM_PROMPTconstant so it can be tested directlyCopilot Review Triage
Deferred follow-ups
None. Issue #142 is fully complete in this PR.
Testing: All 496 tests pass (289 ai-orchestrator tests including 8 new mesh-router tests). Full test suite validates no regressions across all projects.