feat: add MCP server, McpChannel, eval primitives, and OTel tracing

[sajeerzeji]

feat(toolpack-sdk, toolpack-agents): MCP server, eval primitives, OTel tracing, and McpChannel

Summary

This PR ships four major capabilities across toolpack-sdk and toolpack-agents:

1. MCP Server (toolpack-sdk) — expose Toolpack as an MCP server over HTTP or stdio
2. Eval Primitives (toolpack-agents) — dataset management, test runner, scoring, and regression reports
3. OTel Tracing (toolpack-agents) — OpenTelemetry interceptor for distributed traces
4. McpChannel (toolpack-agents) — bridge Toolpack agents into an MCP server as callable tools

---

1. MCP Server (sdk.startMcpServer())

Toolpack can now act as a fully spec-compliant MCP server (Streamable HTTP transport), exposing its 100+ built-in tools to any MCP client — Claude Desktop, Cursor, or custom agents.

Key features:

• HTTP and stdio transports — HTTP for remote/cloud use, stdio for Claude Desktop / Cursor
• port: 0 support — OS-assigned free port; actual port returned on McpServerHandle.port
• Filtering — expose: { categories } or expose: { tools } to restrict what's visible
• Auth — three modes: static (pre-shared tokens), jwt (JWKS/Auth0/Supabase/Clerk), custom (bring your own verifier)
• Search mode — searchMode: true exposes only tool.search; clients discover tools on demand, drastically reducing context token usage for large tool registries
• Agent exposure — agents registered via McpAgentDefinition appear as agent.<name> in tools/list
• OAuth Protected Resource Metadata — /.well-known/oauth-protected-resource mounted automatically in JWT mode

New files:

• packages/toolpack-sdk/src/mcp/server.ts — core server implementation
• packages/toolpack-sdk/src/mcp/server-types.ts — McpServerHandle, McpAgentDefinition, all auth config types
• packages/toolpack-sdk/src/mcp/server-auth.ts — static / JWT / custom auth middleware
• packages/toolpack-sdk/docs/examples/mcp-server-example.ts — full working example

Modified:

• packages/toolpack-sdk/src/toolpack.ts — Toolpack.startMcpServer() public API added
• packages/toolpack-sdk/src/mcp/index.ts — new exports wired up
• packages/toolpack-sdk/src/tools/registry.ts — k8sToolsProject added to loadBuiltIn() (fixed tool count: 95 → 106)

Tests:

• packages/toolpack-sdk/tests/integration/mcp-server.test.ts — 15 HTTP integration tests (real server, no mocking); opt-in via RUN_INTEGRATION_TESTS=1
• packages/toolpack-sdk/tests/unit/mcp-server.test.ts — unit tests
• packages/toolpack-sdk/tests/unit/mcp-server-auth.test.ts — auth unit tests

---

2. Eval Primitives (@toolpack-sdk/agents)

A lightweight eval framework for tracking agent answer quality over time.

New files:

• eval-types.ts — EvalCase, EvalRun, EvalVerdict, EvalReport types
• eval-dataset.ts — EvalDataset: holds cases, supports filter/add
• eval-runner.ts — EvalRunner: runs each case through an agent, collects results
• eval-scorer.ts — ExactMatchScorer, ContainsScorer, LLMJudgeScorer, CustomScorer
• eval-report.ts — compareEvalRuns(), formatEvalReport() for regression detection

Usage pattern:

const runner = new EvalRunner({ agent, dataset, scorers: [new ContainsScorer()] });
const run = await runner.run();
const report = compareEvalRuns(baselineRun, run);
expect(report.regressions).toHaveLength(0); // CI gate

All primitives exported from @toolpack-sdk/agents main entry point.

---

3. OTel Tracing (createOTelTracerInterceptor)

An OpenTelemetry interceptor that creates a span per agent invocation, compatible with any OTel-compliant backend (Jaeger, Honeycomb, Datadog, etc.).

New file: packages/toolpack-agents/src/interceptors/builtins/otel-tracer.ts

Exported as createOTelTracerInterceptor from @toolpack-sdk/agents.

---

4. McpChannel

Bridges a Toolpack agent into an MCP server. When an MCP client calls tools/call for agent.<name>, McpChannel delivers the input to the agent's invokeAgent() and returns the output as the MCP tool result.

New files:

• packages/toolpack-agents/src/channels/mcp-channel.ts
• packages/toolpack-agents/src/channels/mcp-channel.test.ts

Usage:

const ch = new McpChannel();
const agent = new PrReviewerAgent({ channels: [ch] });
await agent.start();
await sdk.startMcpServer({ transport: 'stdio', agents: [ch.asAgentDefinition(agent)] });

Exported from @toolpack-sdk/agents as McpChannel and McpChannelConfig.

---

Bug fixes

• k8sToolsProject missing from loadBuiltIn() — Kubernetes tools existed in the codebase but were never registered. Fixed; tool count goes 95 → 106.
• Removed manual test files — test-mcp-server.ts and test-mcp-server-http.ts from repo root deleted; replaced by proper integration tests.

---

Documentation

All affected documentation updated:

File	Changes
README.md (root + packages)	Tool count, channel count, MCP server section added
documentation/docs/guides/mcp-server.md	Full MCP Server section (auth modes, search mode, agent exposure, handle API)
documentation/docs/agents/channels.md	McpChannel section added
documentation/docs/agents/testing.md	Evals section added
packages/toolpack-agents/README.md	McpChannel, Evals, OTel in features; McpChannel channel section; Evals section
packages/toolpack-agents/docs/channels.md	McpChannel section (mirror)
packages/toolpack-agents/docs/testing.md	Evals section (mirror)
packages/toolpack-agents/package.json	npm description updated
documentation/docusaurus.config.ts	Tagline and JSON-LD description updated
documentation/docs/intro.md	Tool/channel counts updated
documentation/docs/tools/overview.md	Category count updated
development/docs-staged/README.md	Channel count updated

---

Test plan

• npm run test passes in toolpack-sdk and toolpack-agents
• RUN_INTEGRATION_TESTS=1 npx vitest run tests/integration/mcp-server.test.ts — all 15 tests pass
• MCP server connects from Claude Desktop via stdio config
• Static auth rejects requests with missing/wrong token (HTTP 401)
• searchMode: true — tools/list returns only tool.search
• Agent exposed via McpChannel is callable from an MCP client as agent.<name>
• compareEvalRuns returns empty regressions array on identical runs