Skip to content

testing.md

Latest commit

 

History

History
202 lines (152 loc) · 7.64 KB

testing.md

File metadata and controls

202 lines (152 loc) · 7.64 KB
Raw

Testing Guide

This document describes the full test stack for logiscape/mcp-sdk-php and how to run each layer. The overlap between layers is intentional: unit tests catch regressions in isolated components, PHPStan catches type and control-flow mistakes, conformance tests catch spec deviations, the MCP Inspector catches subtle protocol behaviour drifts, and real-world AI applications catch integration surprises that none of the above would.

Before opening a pull request that touches protocol or transport code, please run at least the first four layers. See CONTRIBUTING.md for when each is expected.

1. Unit tests (PHPUnit)

composer test
# narrower examples:
./vendor/bin/phpunit tests/Server/ServerSessionTest.php
./vendor/bin/phpunit --filter testInitializeHandshake tests/Server/ServerSessionTest.php

Conventions:

  • Test classes are final and extend PHPUnit\Framework\TestCase.
  • Test layout mirrors src/ — tests/Server/ServerSessionTest.php tests src/Server/ServerSession.php.
  • Mcp\Shared\MemoryStream and InMemoryTransport let tests exercise sessions without real I/O.

2. Static analysis (PHPStan)

composer analyse

Configuration lives in phpstan.neon. If you intentionally introduce a baseline violation, explain why in your PR description.

3. Combined regression check

composer check

Runs PHPUnit followed by PHPStan. This should pass on every PR.

4. MCP Conformance suite

Not part of composer check because it requires Node.js. During v2 development the suite runs on two independently pinned tool versions (both pinned in package.json so results are reproducible — each baseline file is tied to its installed version):

  • Stable track (composer conformance): the pinned stable tool with the published-spec scenarios, gated by conformance/conformance-baseline.yml. This is the legacy regression gate; both of its baseline lists are currently empty (100% pass).
  • Draft track (composer conformance-draft): the pinned 0.2.0-alpha tool line — the upstream RC-validation track carrying the 2026-07-28 draft-spec scenarios — installed under the npm alias conformance-draft and gated by conformance/conformance-draft-baseline.yml. Draft baseline entries carry a documented root cause and only shrink.

Install both pinned tools once:

npm install

Then:

# Stable track (published-spec scenarios)
composer conformance              # server + client
composer conformance-server
composer conformance-client

# Draft track (2026-07-28 draft-spec scenarios)
composer conformance-draft        # server + client
composer conformance-draft-server
composer conformance-draft-client

# single scenario:
php conformance/run-conformance.php server tools-list
php conformance/run-conformance.php server-draft server-stateless

How it works: the runner starts conformance/everything-server.php via PHP's built-in server for server scenarios, and the conformance framework spawns conformance/everything-client.php for client scenarios. The full mechanics, baseline-curation rules, and result interpretation are in conformance/README.md. Do not edit either baseline file to paper over a regression — that is explicitly against the project's no-shortcut rule.

Run the stable track when your change touches (add the draft track when the change touches 2026-07-28 behavior):

  • src/Shared/BaseSession.php or anything in src/Shared/
  • src/Server/ session, handler, or transport code
  • src/Client/ClientSession.php or transports
  • src/Types/ protocol types
  • src/Server/McpServer.php
  • authorization code under src/Client/Auth/ or src/Server/Auth/

5. Official MCP Inspector

The Model Context Protocol Inspector is a reference tool from the MCP project itself. It speaks the protocol honestly and is invaluable for catching differences between what the spec says a server should do and what our server actually does.

Against a local server

npx @modelcontextprotocol/inspector php conformance/everything-server.php

Then open the Inspector UI in the browser. Try:

  • Tools tab: list tools and call each one; check argument validation, structured output, and any icons or rich metadata render correctly.
  • Prompts tab: list and fetch each prompt; confirm completion suggestions fire on prompt-argument inputs.
  • Resources tab: list and read resources and resource templates.
  • Elicitation and sampling (if the example exercises them): confirm the Inspector's elicitation card renders and responses flow back through the SDK correctly.

For OAuth-protected HTTP servers, follow examples/server_auth/README.md to set up tokens, then supply the token in the Inspector's Authorization header field.

What to look for

  • Is the negotiated protocol version what you expect?
  • Do advertised capabilities match what handlers you actually registered?
  • Do notifications (progress, resource updates, elicitation) round-trip cleanly in the Inspector's event log?
  • Does the server fail cleanly when the client does something unexpected (e.g., cancels a request mid-flight)?

Inspector smoke-tests are encouraged for any protocol-touching PR. A one-line note in the PR description ("Verified tools + prompts in Inspector against conformance/everything-server.php") is enough.

6. Real-world AI applications

Real clients find bugs automated suites can't, because they exercise the SDK the way end users will. Contributors are encouraged, not required to run at least one of the following before submitting user-visible changes.

Claude Code

Claude Code can connect to local and remote MCP servers.

Then start a Claude Code session and ask the model to use one of your tools. Watch for:

  • Tool registration showing up (usually via /mcp or similar diagnostic).
  • Tool calls succeeding with realistic argument shapes the LLM produces.
  • Structured output rendering sensibly in the conversation.

Consult the official Claude Code documentation for the latest MCP details.

OpenAI API

The OpenAI API Platform playground allows users to test remote HTTP MCP servers, both public and OAuth-protected with an access token. Consult the OpenAI documentation for the latest details.

What to report

If either Claude Code or the OpenAI API behaves unexpectedly (but the Inspector and conformance suite look fine), that usually indicates either a bug in the SDK or a spec ambiguity. Open a bug report and include:

  • The client you used and its version.
  • The request/response sequence (from the SDK's logger if you can).
  • What you expected vs. what happened.

Troubleshooting

  • PHPUnit cannot find tests: run composer dump-autoload -o and retry.
  • PHPStan reports a type error you disagree with: please don't add a baseline entry silently — open an issue or discuss it on the PR.
  • Conformance shows a new failure: first confirm against a clean main. If the failure reproduces on main, it's an existing regression and worth its own issue. If it's caused by your branch, fix it or investigate whether it's a fair failure of an optional extension (in which case it may belong in the baseline with a documented reason, not silently suppressed).
  • Inspector can't connect: check stdio vs HTTP transport mismatch, then check that ext-curl and ext-json are enabled on the CLI PHP.

See also