[Demo] Add Panther end-to-end test suite for use cases and profiler#2300
Open
chr-hertel wants to merge 1 commit into
Open
[Demo] Add Panther end-to-end test suite for use cases and profiler#2300chr-hertel wants to merge 1 commit into
chr-hertel wants to merge 1 commit into
Conversation
carsonbot
added
Bug
Adds a browser-driven test suite covering the ten demo use cases and the Symfony AI panel of the web profiler. It runs locally only, excluded from the default test suite, since it drives a real browser and calls the AI platforms for real. Panther boots the app in the dev environment, because the AI data collector only exists with kernel.debug enabled - which also makes the real API keys from .env.local and the environment of the Docker services available.
chr-hertel
force-pushed
the
panther-demo-tests
branch
from
fe93d82 to
911b7f7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wires up Symfony Panther in the demo and covers the ten use
cases plus the Symfony AI panel of the profiler - the manual click-through this issue asked to
automate. Tests only, no production code is touched.
Local only, as discussed in the issue: the suite is not part of the default test suite (
phpunit.xmlgets
defaultTestSuite="default", so CI is unaffected), it needs a browser, and every test calls theAI platforms for real. Run it with
vendor/bin/phpunit --testsuite e2e- 15 tests, ~2 minutes. Setupis documented in
demo/tests/E2E/README.md.Each use case is one test, driving the flow once and then asserting both the UI and what the AI
bundle collected for that very request: the platform calls with their model, input, options and
result, the registered tools, the tool calls, and the token usage.
ProfilerTestcovers the toolbarblock and the panel sections themselves. Tests skip themselves when their API key is missing
(
OPENAI_API_KEYfor eight of them,HUGGINGFACE_API_KEYfor the cropping,MISTRAL_API_KEYforthe OCR), and
BlogTestskips without an indexed store.Two notes on how it runs:
kernel.debugenabled - so the profiler has something to show. That also means the web serverreads the real keys from
.env.local, and the environment of the Docker services is passed on fromsymfony var:export, the waysymfony servedoes it.of the screen: the webcam shows Chrome's test pattern, the microphone plays back
fixtures/audio.mp3.While writing the suite it caught the SSE session bug on a first-time visitor - a browser without a
session cookie, which is what every test starts from. That is fixed in #2302 (and #2304), so this
branch is rebased on top of them and needs no demo changes of its own.