Skip to content

feat: add human-in-the-loop middleware for tool approval process#12

Open
NoOne7135 wants to merge 3 commits into
mainfrom
tool-approval
Open

feat: add human-in-the-loop middleware for tool approval process#12
NoOne7135 wants to merge 3 commits into
mainfrom
tool-approval

Conversation

@NoOne7135

Copy link
Copy Markdown
Contributor
  • Integrated human-in-the-loop middleware to handle dangerous tool executions.
  • Updated AgentRuntime to include new middleware and handle approval decisions.
  • Modified system prompts to clarify the approval process for dangerous actions.
  • Enhanced TurnLifecycleService to support resuming turns with approval decisions.
  • Updated TurnStreamConsumer to manage interrupt events for tool approvals.
  • Added new types and structures to support approval decisions in turn types.
  • Implemented agent event handling for tool approval requests and responses.
  • Created a new ToolApprovalRenderer component for displaying approval messages in the UI.
  • Updated agent chat manager to handle tool approval requests and submissions.
  • Enhanced API endpoints to process tool approval decisions and stream responses.
  • Refactored skills documentation to clarify mutation confirmation requirements.

- Integrated human-in-the-loop middleware to handle dangerous tool executions.
- Updated AgentRuntime to include new middleware and handle approval decisions.
- Modified system prompts to clarify the approval process for dangerous actions.
- Enhanced TurnLifecycleService to support resuming turns with approval decisions.
- Updated TurnStreamConsumer to manage interrupt events for tool approvals.
- Added new types and structures to support approval decisions in turn types.
- Implemented agent event handling for tool approval requests and responses.
- Created a new ToolApprovalRenderer component for displaying approval messages in the UI.
- Updated agent chat manager to handle tool approval requests and submissions.
- Enhanced API endpoints to process tool approval decisions and stream responses.
- Refactored skills documentation to clarify mutation confirmation requirements.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a human-in-the-loop (HITL) approval flow for “dangerous” tool executions across the agent runtime, SSE streaming layer, and the web UI, enabling turns to interrupt for approvals and then resume based on an approve/reject decision.

Changes:

  • Introduces HITL middleware + interrupt/resume handling in the agent runtime/turn lifecycle, including streaming “interrupt” events and resuming via LangGraph Command.
  • Adds a new /agent/approval SSE endpoint and frontend chat/UI support to render approval prompts and submit decisions.
  • Updates prompts/docs/types to reflect that dangerous mutations are approved via the runtime UI rather than textual confirmation.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
surfaces/web-sse/createSseEventEmitter.ts Adds SSE emission for interrupt events (HITL approval requests).
sessionStore.ts Adds helper to fetch the latest turn for resume support.
index.ts Wires plugin options into TurnLifecycleService for resume support.
endpoints/core.ts Adds /agent/approval endpoint to submit approve/reject and stream resumed output.
custom/types.ts Extends message part types/data to represent tool approvals in UI.
custom/skills/mutate_data/SKILL.md Updates mutation instructions to rely on runtime approval UI instead of textual confirmation.
custom/conversation_area/ToolApprovalRenderer.vue New UI component to display approval-required cards and actions.
custom/conversation_area/MessageRenderer.vue Renders the new approval UI part type in chat messages.
custom/composables/useAgentStore.ts Handles interrupt-to-approval message insertion and decision submission state updates.
custom/composables/agentStore/useAgentChat.ts Adds client call to /agent/approval and consumes SSE stream on approval submission.
apiBasedTools.ts Adds agent.isDangerous metadata to tool schemas for HITL gating.
agentTurnService.ts Implements interrupt tracking, resume via Command, and emits interrupt events to clients.
agentEvents.ts Adds new interrupt event type to the agent event model.
agent/turn/turnTypes.ts Updates turn input types to support approvals/resume and runtime stream input changes.
agent/turn/TurnStreamConsumer.ts Consumes LangGraph “updates” stream to detect interrupts and forward them via callback.
agent/turn/TurnLifecycleService.ts Adds resume() to continue the latest turn and seed initial response state.
agent/systemPrompt.ts Updates system prompt guidance to describe dangerous actions and rely on approval UI.
agent/runtime/AgentRuntime.ts Adds humanInTheLoopMiddleware and streams both “messages” and “updates”.
agent/middleware/apiBasedTools.ts Adjusts tool wrapping logic and error handling; integrates tool call tracking changes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread agent/middleware/apiBasedTools.ts Outdated
Comment thread agentTurnService.ts
Comment on lines +151 to +159
if (approvalPart?.data) {
approvalPart.data.status = 'processing';
}

await submitToolApprovalResponse(sessionId, decision);

if (approvalPart?.data) {
approvalPart.data.status = decision === 'approve' ? 'approved' : 'rejected';
}
Comment thread endpoints/core.ts
Comment thread custom/skills/mutate_data/SKILL.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants