feat: add error_context to v2 result execution and report schema - #208
Merged
Conversation
Playwright >= 1.51 writes an error-context.md for every failed test (test info, error details, page snapshot). Reporters could only upload it as an opaque file attachment, so there was no way to get the raw text onto the clipboard. Adds a nullable error_context string to v2 ResultExecution, alongside stacktrace, and the same field to the report file format so reporters and the report validator stay in sync. Capped at 262144 characters. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The endpoint rejects a whole batch with 413 on two guards, neither of which was in the spec: more than 2000 results (pre-existing), and the aggregate execution.error_context budget added alongside this field. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Over-cap values are truncated silently and the request still succeeds, which the previous wording left open to reading as a rejection. Also note that the captured context can include rendered page content, and that the field is not returned by the result read endpoints. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
abelharisov
approved these changes
Jul 31, 2026
sklmx
approved these changes
Jul 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Playwright ≥ 1.51 writes an
error-context.mdfor each failed test (error details, ARIA page snapshot, source frame) — meant to be pasted into an AI agent or bug report. Today reporters can only send it as an opaque file attachment, so reading it means downloading the file from storage. Noerror_contextconcept existed in the specs.What
Adds a nullable, write-only
error_contextstring next tostacktrace:testops-api/v2/schemas/ResultExecution.yaml— public API v2 (single + bulk)report/schemas/result.yaml— report file format, keepsreporters-validatorin syncValues over 262144 chars are truncated server-side (request still succeeds). Also documents the bulk endpoint's
413, which was previously undocumented.Notes
additionalProperties: false, so existing reports keep validating.npm run validate(spectral) passes.qase-tms/app#3640is queued to deploy, otherwise the API silently drops the field.Rollout (AICP-31)
qase-tms/app#3640— storage, validation, internal endpoint, Copy buttonqase-tms/qase-javascript#993— regenerated v2 client, commons model, Playwright reporterApp must reach production before a reporter release: its own e2e suite reports to production Qase.