-
Notifications
You must be signed in to change notification settings - Fork 9
[DO NOT MERGE!] docs: document PWCS cancellation feature [RED-511] #308
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
db94324
82da719
82c0fb6
ffa7046
3574b3d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| --- | ||
| openapi: post /v1/cancel | ||
| title: Cancel a check or test session | ||
| sidebarTitle: Cancel a session | ||
| --- | ||
|
|
||
| Stops an in-progress check session or test session by ID. The request body discriminates by which ID you pass: | ||
|
|
||
| - Pass a `checkSessionId` to cancel a [check session](/api-reference/check-sessions/retrieve-a-check-session-v2) (UI "Schedule now" or `checkly deploy`). | ||
| - Pass a `testSessionId` to cancel a [test session](/detect/testing/overview) (`checkly test`, `checkly trigger`, or `checkly pw-test`). | ||
|
|
||
| Optionally pass `sequenceId` (a single UUID or an array) to cancel only specific runs within the session. Omit it to cancel every in-progress run. | ||
|
|
||
| Only Playwright Check Suite runs are cancellable. If a test session also contains URL, API, Browser, Multistep, Heartbeat, TCP, DNS, or ICMP runs, those continue executing until they finish normally. See [Cancellation](/concepts/cancellation) for the full flow, the effect on alerts and metrics, and how the `CANCELLED` status and `isCancelled` flag surface in responses. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| --- | ||
| openapi: get /v2/check-sessions/{checkSessionId}/completion | ||
| --- | ||
|
|
||
| Call this endpoint to await the completion of a check session. A successful response will be returned once the check session reaches its final state (i.e. when it passes, fails, or is cancelled). | ||
|
|
||
| If the check session takes a long time to complete, the endpoint will return a timeout error code. You should keep calling the endpoint until you receive a successful response, or a non-timeout related error code. If using *curl*, its `--retry` option is suitable. | ||
|
|
||
| The successful response of this endpoint is equivalent to the [retrieve a check session](/api-reference/check-sessions/retrieve-a-check-session-v2) endpoint's response for a completed check session. | ||
|
|
||
| <Note> | ||
| **v2 vs v1:** v2 returns the new `CANCELLED` value in `status` for sessions that were [cancelled](/concepts/cancellation). v1 reports the same sessions as `TIMED_OUT` for backward compatibility. The response shape is otherwise identical, including the `isCancelled` boolean on per-result objects. | ||
| </Note> |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,4 @@ | ||
| --- | ||
| openapi: get /v1/check-sessions/{checkSessionId}/completion | ||
| --- | ||
| deprecated: true | ||
| --- |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| --- | ||
| openapi: get /v2/check-sessions/{checkSessionId} | ||
| --- | ||
|
|
||
| Retrieves a check session. Results may be incomplete if the check session is still in progress. | ||
|
|
||
| Once a check session has finished, results will include at least one check result for each run location: one result with `resultType` equal to `"FINAL"`, and zero or more results with `resultType` equal to `"ATTEMPT"` (one for each failed attempt, if any). | ||
|
|
||
| Each result contains just enough information to quickly determine whether the check run was successful or not. To dive even deeper into individual results, use the [retrieve a check result](/api-reference/check-results/retrieve-a-check-result) endpoint. | ||
|
|
||
| <Note> | ||
| **v2 vs v1:** v2 returns the new `CANCELLED` value in `status` for sessions that were [cancelled](/concepts/cancellation). v1 reports the same sessions as `TIMED_OUT` for backward compatibility. The response shape is otherwise identical, including the `isCancelled` boolean on per-result objects. | ||
| </Note> |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,4 @@ | ||
| --- | ||
| openapi: get /v1/check-sessions/{checkSessionId} | ||
| --- | ||
| deprecated: true | ||
| --- |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| --- | ||
| openapi: post /v2/check-sessions/trigger | ||
| --- | ||
|
|
||
| Starts a check session for each check that matches the provided target filters. If no filters are given, matches all eligible checks. | ||
|
|
||
| This endpoint does not wait for the check session to complete. Use the [await completion](/api-reference/check-sessions/await-the-completion-of-a-check-session-v2) or [retrieve check session](/api-reference/check-sessions/retrieve-a-check-session-v2) endpoints to track progress if needed. | ||
|
|
||
| Standard alerting rules apply to finished check runs. | ||
|
|
||
| Equivalent to the _Schedule Now_ button in the UI. | ||
|
|
||
| <Note> | ||
| **v2 vs v1:** v2 returns the new `CANCELLED` value in `status` for sessions that were [cancelled](/concepts/cancellation). v1 reports the same sessions as `TIMED_OUT` for backward compatibility. The response shape is otherwise identical, including the `isCancelled` boolean on per-result objects. | ||
| </Note> |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,4 @@ | ||
| --- | ||
| openapi: post /v1/check-sessions/trigger | ||
| --- | ||
| deprecated: true | ||
| --- |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -178,6 +178,15 @@ The Checkly CLI evaluates Git information from your local or CI environment on a | |
|
|
||
| <CliEnvVars /> | ||
|
|
||
| ## Cancelling a deploy | ||
|
|
||
| {/* TODO: confirm CLI version that ships deploy Ctrl+C handling and fill in the Note below. */} | ||
| <Note>Available in CLI vX.X+.</Note> | ||
|
|
||
| `checkly deploy` may run [Playwright Check Suites](/detect/synthetic-monitoring/playwright-checks/overview) as part of post-deploy verification. Pressing <kbd>Ctrl</kbd>+<kbd>C</kbd> during that phase prompts you to either **cancel** the recorded check session or **detach** and let it finish in the background. Detach is the default. | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Confirm: (1) |
||
|
|
||
| See [Cancellation](/concepts/cancellation) for the full flow. | ||
|
|
||
| ## Related Commands | ||
|
|
||
| - [`checkly login`](/cli/checkly-login) - Log in to your Checkly account | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -432,6 +432,15 @@ npx checkly pw-test -- test.spec.ts | |
| - Test sessions are recorded by default with full logs, traces, and videos | ||
| - View all artifacts in Checkly's UI | ||
|
|
||
| ## Cancelling a run | ||
|
|
||
| {/* TODO: confirm CLI version that ships pw-test Ctrl+C handling and fill in the Note below. */} | ||
| <Note>Available in CLI vX.X+.</Note> | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fill in the CLI version that ships pw-test Ctrl+C handling (RED-375), or remove this Note if Ferran prefers a generic call-out. The Linear ticket is still in In Review. |
||
|
|
||
| Press <kbd>Ctrl</kbd>+<kbd>C</kbd> during a `checkly pw-test` run to cancel the test session. The CLI sends a cancel request and waits for the runner to stop. Pressing <kbd>Ctrl</kbd>+<kbd>C</kbd> a second time exits the CLI without waiting. | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Verify the Ctrl+C UX matches what shipped: "no detach prompt; second Ctrl+C exits without waiting." I took this from the project spec (RED-375), not from reading the CLI source. |
||
|
|
||
| See [Cancellation](/concepts/cancellation) for the full flow. | ||
|
|
||
| ## Related Commands | ||
|
|
||
| - [`checkly test`](/cli/checkly-test) - Test your setup before deployment | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -380,6 +380,15 @@ Specify [environment variables](/cli/environment-variables) to dry run checks wi | |
| npx checkly test --env ENVIRONMENT_URL="https://preview.acme.com" --env PASSWORD=doremiabc123 | ||
| ``` | ||
|
|
||
| ## Cancelling a run | ||
|
|
||
| {/* TODO: confirm CLI version that ships test Ctrl+C handling and fill in the Note below. */} | ||
| <Note>Available in CLI vX.X+.</Note> | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fill in the CLI version that ships |
||
|
|
||
| When you started the run with `--record`, pressing <kbd>Ctrl</kbd>+<kbd>C</kbd> prompts you to either **cancel** the recorded test session in Checkly or **detach** and let it finish in the background. Detach is the default. Cancellation only applies to Playwright Check Suite runs inside the session. | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Verify "cancel or detach, detach is default" matches what shipped. RED-373 was marked Duplicate so I'm not 100% sure this is the final UX. |
||
|
|
||
| See [Cancellation](/concepts/cancellation) for the full flow. | ||
|
|
||
| ## Related Commands | ||
|
|
||
| - [`checkly pw-test`](/cli/checkly-pw-test) - Run Playwright tests in the Checkly cloud | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -372,6 +372,15 @@ npx checkly trigger --tags staging --record --test-session-name "Pre-prod valida | |
|
|
||
| <Tip>If your production deployment includes monitoring changes and updates, [use `npx checkly test`](/cli/checkly-test) to validate your preview environment with the updated monitoring configuration.</Tip> | ||
|
|
||
| ## Cancelling a run | ||
|
|
||
| {/* TODO: confirm CLI version that ships trigger Ctrl+C handling and fill in the Note below. */} | ||
| <Note>Available in CLI vX.X+.</Note> | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fill in the CLI version that ships |
||
|
|
||
| When you started the run with `--record`, pressing <kbd>Ctrl</kbd>+<kbd>C</kbd> prompts you to either **cancel** the recorded test session in Checkly or **detach** and let it finish in the background. Detach is the default. Cancellation only applies to Playwright Check Suite runs inside the session. | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same UX question as in checkly-test.mdx — confirm "detach is the default" is accurate. |
||
|
|
||
| See [Cancellation](/concepts/cancellation) for the full flow. | ||
|
|
||
| ## Related Commands | ||
|
|
||
| - [`checkly deploy`](/cli/checkly-deploy) - Deploy your Checkly configuration | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| --- | ||
| title: 'Cancellation' | ||
| description: 'Stop in-flight Playwright Check Suite runs from the UI, API, or CLI. Other check types cannot be cancelled.' | ||
| sidebarTitle: 'Cancellation' | ||
| --- | ||
|
|
||
| Cancellation lets you stop an in-progress [Playwright Check Suite](/detect/synthetic-monitoring/playwright-checks/overview) run. Cancelled results are flagged with `isCancelled: true`, do not trigger alerts, and are excluded from availability and performance metrics. A session that contains at least one cancelled run ends in a terminal `CANCELLED` status. | ||
|
|
||
| ## What you can cancel | ||
|
|
||
| You can cancel two kinds of sessions, and the behaviour differs: | ||
|
|
||
| **Check sessions** — created by the UI "Schedule now" button and `checkly deploy`. A check session is one execution of a single Playwright Check Suite, fanned out across the locations you configured. Cancelling the session stops every in-progress run; cancelling a single run leaves the rest of the session going. | ||
|
|
||
| **Test sessions** — created by `checkly test`, `checkly trigger`, and `checkly pw-test`. A test session can contain multiple checks of different types. Cancelling the test session stops only the Playwright Check Suite runs inside it; any URL, API, Browser, Multistep, Heartbeat, TCP, DNS, or ICMP runs in the same session continue until they finish normally. You can also cancel a single Playwright run within the session. | ||
|
|
||
| <Warning> | ||
| **Only Playwright Check Suite runs are cancellable.** Other check types cannot be cancelled, even when they appear inside a test session alongside Playwright runs. Scheduled runs are also excluded; cancellation is for user-initiated runs only. | ||
| </Warning> | ||
|
|
||
| ## How to cancel | ||
|
|
||
| <Tabs> | ||
| <Tab title="UI"> | ||
| On a check session or test session page, click **Cancel session** in the header to stop the whole session. To stop just a single run, use the cancel button on its row. | ||
| </Tab> | ||
|
|
||
| <Tab title="Public API"> | ||
| Send a `POST` request to [`/v1/cancel`](/api-reference/cancel/cancel-a-session) with either a `checkSessionId` or a `testSessionId`. Pass an optional `sequenceId` (or array) to cancel only specific runs within the session. | ||
|
|
||
| ```bash Cancel a whole check session | ||
| curl -X POST https://api.checklyhq.com/v1/cancel \ | ||
| -H "Authorization: Bearer $CHECKLY_API_KEY" \ | ||
| -H "X-Checkly-Account: $CHECKLY_ACCOUNT_ID" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{"checkSessionId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"}' | ||
| ``` | ||
|
|
||
| ```bash Cancel specific runs in a test session | ||
| curl -X POST https://api.checklyhq.com/v1/cancel \ | ||
| -H "Authorization: Bearer $CHECKLY_API_KEY" \ | ||
| -H "X-Checkly-Account: $CHECKLY_ACCOUNT_ID" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{"testSessionId": "a1b2c3d4-...", "sequenceId": ["b2c3d4e5-..."]}' | ||
| ``` | ||
| </Tab> | ||
|
|
||
| <Tab title="CLI"> | ||
| Press <kbd>Ctrl</kbd>+<kbd>C</kbd> during `checkly pw-test`, `checkly test`, `checkly trigger`, or `checkly deploy`. The CLI prompts you to either cancel the running session or detach (let it finish in the background). | ||
|
|
||
| See the [CLI reference](/cli/overview) for command-specific behavior. | ||
| </Tab> | ||
| </Tabs> | ||
|
|
||
| ## What happens after a run is cancelled | ||
|
|
||
| <Steps> | ||
| <Step title="The runner stops execution"> | ||
| The runner receives the cancellation signal, aborts the in-flight Playwright run, and reports the result back to Checkly. | ||
| </Step> | ||
|
|
||
| <Step title="The result is marked cancelled"> | ||
| Each cancelled run's result has `isCancelled: true`. Once the session has stopped (any remaining non-Playwright checks finish normally), the session's status becomes `CANCELLED` because at least one of its runs was cancelled. | ||
| </Step> | ||
|
|
||
| <Step title="No alert is sent"> | ||
| Cancelled runs never produce failure or recovery alerts, regardless of your alert settings. See [Alert configuration](/communicate/alerts/configuration). | ||
| </Step> | ||
|
|
||
| <Step title="Metrics ignore the run"> | ||
| Availability, response time percentiles, and success ratios exclude cancelled runs. Your check status does not change. Public and private dashboards omit cancelled runs from their summary counts. | ||
| </Step> | ||
|
|
||
| <Step title="Automated incidents are not triggered"> | ||
| Because no alert is sent and the check status does not change, [automated incidents](/communicate/incidents/overview) are not opened from a cancelled run. | ||
| </Step> | ||
| </Steps> | ||
|
|
||
| ## API versioning | ||
|
|
||
| Cancellation introduces two API additions: a new `CANCELLED` value on the check-session status enum, and a new `isCancelled` boolean on individual check results. The `isCancelled` flag is additive and ships on every check-result and check-session response without a version bump. The status enum change is breaking for clients that switch on status values exhaustively, so the new `CANCELLED` value is only returned by the v2 check-session endpoints; v1 maps cancelled sessions to `TIMED_OUT` in `status` so existing clients continue to work unchanged. | ||
|
|
||
| <ResponseField name="v1 check sessions" type="deprecated"> | ||
| [`POST /v1/check-sessions/trigger`](/api-reference/check-sessions/trigger-a-new-check-session), [`GET /v1/check-sessions/{checkSessionId}`](/api-reference/check-sessions/retrieve-a-check-session), and [`GET /v1/check-sessions/{checkSessionId}/completion`](/api-reference/check-sessions/await-the-completion-of-a-check-session) report cancelled sessions as `TIMED_OUT` in `status` rather than `CANCELLED`. Per-result objects still include `isCancelled`, so individual cancelled runs remain identifiable. | ||
| </ResponseField> | ||
|
|
||
| <ResponseField name="v2 check sessions" type="recommended"> | ||
| [`POST /v2/check-sessions/trigger`](/api-reference/check-sessions/trigger-a-new-check-session-v2), [`GET /v2/check-sessions/{checkSessionId}`](/api-reference/check-sessions/retrieve-a-check-session-v2), and [`GET /v2/check-sessions/{checkSessionId}/completion`](/api-reference/check-sessions/await-the-completion-of-a-check-session-v2) return `CANCELLED` in `status` for cancelled sessions. Response shape is otherwise the same as v1. | ||
| </ResponseField> | ||
|
|
||
| <ResponseField name="check results" type="additive change"> | ||
| [`GET /v1/check-results/{checkId}/{checkResultId}`](/api-reference/check-results/retrieve-a-check-result) and the list endpoints include `isCancelled` on every result. | ||
| </ResponseField> | ||
|
|
||
| <ResponseField name="cancel action" type="new endpoint"> | ||
| [`POST /v1/cancel`](/api-reference/cancel/cancel-a-session) is the action endpoint itself. It accepts either a `checkSessionId` or a `testSessionId` in the body, so a single endpoint covers both. | ||
| </ResponseField> | ||
|
|
||
| If you build automation against check sessions and need to distinguish cancelled sessions from genuine timeouts at the session level, switch to the v2 endpoints. Otherwise, the `isCancelled` flag on per-result objects is available in both versions. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fill in the CLI version that ships
checkly deployCtrl+C handling (RED-374 dup), or remove this Note.