plugins: preserve qveris runtime surfaces

.agents/skills/openclaw-qa-testing/SKILL.md

@@ -0,0 +1,148 @@
+---
+name: openclaw-qa-testing
+description: Run, watch, debug, and extend OpenClaw QA testing with qa-lab and qa-channel. Use when Codex needs to execute the repo-backed QA suite, inspect live QA artifacts, debug failing scenarios, add new QA scenarios, or explain the OpenClaw QA workflow. Prefer the live OpenAI lane with regular openai/gpt-5.4 in fast mode; do not use gpt-5.4-pro or gpt-5.4-mini unless the user explicitly overrides that policy.
+---
+
+# OpenClaw QA Testing
+
+Use this skill for `qa-lab` / `qa-channel` work. Repo-local QA only.
+
+## Read first
+
+- `docs/concepts/qa-e2e-automation.md`
+- `docs/help/testing.md`
+- `docs/channels/qa-channel.md`
+- `qa/README.md`
+- `qa/scenarios/index.md`
+- `extensions/qa-lab/src/suite.ts`
+- `extensions/qa-lab/src/character-eval.ts`
+
+## Model policy
+
+- Live OpenAI lane: `openai/gpt-5.4`
+- Fast mode: on
+- Do not use:
+  - `openai/gpt-5.4-pro`
+  - `openai/gpt-5.4-mini`
+- Only change model policy if the user explicitly asks.
+
+## Default workflow
+
+1. Read the scenario pack and current suite implementation.
+2. Decide lane:
+   - mock/dev: `mock-openai`
+   - real validation: `live-frontier`
+3. For live OpenAI, use:
+
+```bash
+OPENCLAW_LIVE_OPENAI_KEY="${OPENAI_API_KEY}" \
+pnpm openclaw qa suite \
+  --provider-mode live-frontier \
+  --model openai/gpt-5.4 \
+  --alt-model openai/gpt-5.4 \
+  --output-dir .artifacts/qa-e2e/run-all-live-frontier-<tag>
+```
+
+4. Watch outputs:
+   - summary: `.artifacts/qa-e2e/run-all-live-frontier-<tag>/qa-suite-summary.json`
+   - report: `.artifacts/qa-e2e/run-all-live-frontier-<tag>/qa-suite-report.md`
+5. If the user wants to watch the live UI, find the current `openclaw-qa` listen port and report `http://127.0.0.1:<port>`.
+6. If a scenario fails, fix the product or harness root cause, then rerun the full lane.
+
+## Character evals
+
+Use `qa character-eval` for style/persona/vibe checks across multiple live models.
+
+```bash
+pnpm openclaw qa character-eval \
+  --model openai/gpt-5.4,thinking=xhigh \
+  --model openai/gpt-5.2,thinking=xhigh \
+  --model openai/gpt-5,thinking=xhigh \
+  --model anthropic/claude-opus-4-6,thinking=high \
+  --model anthropic/claude-sonnet-4-6,thinking=high \
+  --model zai/glm-5.1,thinking=high \
+  --model moonshot/kimi-k2.5,thinking=high \
+  --model google/gemini-3.1-pro-preview,thinking=high \
+  --judge-model openai/gpt-5.4,thinking=xhigh,fast \
+  --judge-model anthropic/claude-opus-4-6,thinking=high \
+  --concurrency 16 \
+  --judge-concurrency 16 \
+  --output-dir .artifacts/qa-e2e/character-eval-<tag>
+```
+
+- Runs local QA gateway child processes, not Docker.
+- Preferred model spec syntax is `provider/model,thinking=<level>[,fast|,no-fast|,fast=<bool>]` for both `--model` and `--judge-model`.
+- Do not add new examples with separate `--model-thinking`; keep that flag as legacy compatibility only.
+- Defaults to candidate models `openai/gpt-5.4`, `openai/gpt-5.2`, `openai/gpt-5`, `anthropic/claude-opus-4-6`, `anthropic/claude-sonnet-4-6`, `zai/glm-5.1`, `moonshot/kimi-k2.5`, and `google/gemini-3.1-pro-preview` when no `--model` is passed.
+- Candidate thinking defaults to `high`, with `xhigh` for OpenAI models that support it. Prefer inline `--model provider/model,thinking=<level>`; `--thinking <level>` and `--model-thinking <provider/model=level>` remain compatibility shims.
+- OpenAI candidate refs default to fast mode so priority processing is used where supported. Use inline `,fast`, `,no-fast`, or `,fast=false` for one model; use `--fast` only to force fast mode for every candidate.
+- Judges default to `openai/gpt-5.4,thinking=xhigh,fast` and `anthropic/claude-opus-4-6,thinking=high`.
+- Report includes judge ranking, run stats, durations, and full transcripts; do not include raw judge replies. Duration is benchmark context, not a grading signal.
+- Candidate and judge concurrency default to 16. Use `--concurrency <n>` and `--judge-concurrency <n>` to override when local gateways or provider limits need a gentler lane.
+- Scenario source should stay markdown-driven under `qa/scenarios/`.
+- For isolated character/persona evals, write the persona into `SOUL.md` and blank `IDENTITY.md` in the scenario flow. Use `SOUL.md + IDENTITY.md` only when intentionally testing how the normal OpenClaw identity combines with the character.
+- Keep prompts natural and task-shaped. The candidate model should receive character setup through `SOUL.md`, then normal user turns such as chat, workspace help, and small file tasks; do not ask "how would you react?" or tell the model it is in an eval.
+- Prefer at least one real task, such as creating or editing a tiny workspace artifact, so the transcript captures character under normal tool use instead of pure roleplay.
+
+## Codex CLI model lane
+
+Use model refs shaped like `codex-cli/<codex-model>` whenever QA should exercise Codex as a model backend.
+
+Examples:
+
+```bash
+pnpm openclaw qa suite \
+  --provider-mode live-frontier \
+  --model codex-cli/<codex-model> \
+  --alt-model codex-cli/<codex-model> \
+  --scenario <scenario-id> \
+  --output-dir .artifacts/qa-e2e/codex-<tag>
+```
+
+```bash
+pnpm openclaw qa manual \
+  --model codex-cli/<codex-model> \
+  --message "Reply exactly: CODEX_OK"
+```
+
+- Treat the concrete Codex model name as user/config input; do not hardcode it in source, docs examples, or scenarios.
+- Live QA preserves `CODEX_HOME` so Codex CLI auth/config works while keeping `HOME` and `OPENCLAW_HOME` sandboxed.
+- Mock QA should scrub `CODEX_HOME`.
+- If Codex returns fallback/auth text every turn, first check `CODEX_HOME`, `~/.profile`, and gateway child logs before changing scenario assertions.
+- For model comparison, include `codex-cli/<codex-model>` as another candidate in `qa character-eval`; the report should label it as an opaque model name.
+
+## Repo facts
+
+- Seed scenarios live in `qa/`.
+- Main live runner: `extensions/qa-lab/src/suite.ts`
+- QA lab server: `extensions/qa-lab/src/lab-server.ts`
+- Child gateway harness: `extensions/qa-lab/src/gateway-child.ts`
+- Synthetic channel: `extensions/qa-channel/`
+
+## What “done” looks like
+
+- Full suite green for the requested lane.
+- User gets:
+  - watch URL if applicable
+  - pass/fail counts
+  - artifact paths
+  - concise note on what was fixed
+
+## Common failure patterns
+
+- Live timeout too short:
+  - widen live waits in `extensions/qa-lab/src/suite.ts`
+- Discovery cannot find repo files:
+  - point prompts at `repo/...` inside seeded workspace
+- Subagent proof too brittle:
+  - prefer stable final reply evidence over transient child-session listing
+- Harness “rebuild” delay:
+  - dirty tree can trigger a pre-run build; expect that before ports appear
+
+## When adding scenarios
+
+- Add or update scenario markdown under `qa/scenarios/`
+- Keep kickoff expectations in `qa/scenarios/index.md` aligned
+- Add executable coverage in `extensions/qa-lab/src/suite.ts`
+- Prefer end-to-end assertions over mock-only checks
+- Save outputs under `.artifacts/qa-e2e/`

.agents/skills/openclaw-qa-testing/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "QA Test OpenClaw"
+  short_description: "Run and debug qa-lab and qa-channel scenarios"
+  default_prompt: "Use $openclaw-qa-testing to run or extend the OpenClaw QA suite with qa-lab and qa-channel, using regular openai/gpt-5.4 in fast mode for live OpenAI runs."

.agents/skills/openclaw-release-maintainer/SKILL.md

@@ -86,6 +86,22 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts <published-version>
 - For stable correction releases like `YYYY.M.D-N`, it also verifies the
   upgrade path from `YYYY.M.D` to `YYYY.M.D-N` so a correction publish cannot
   silently leave existing global installs on the old base stable payload.
+- Treat install smoke as a pack-budget gate too. `pnpm test:install:smoke`
+  now fails the candidate update tarball when npm reports an oversized
+  `unpackedSize`, so release-time e2e cannot miss pack bloat that would risk
+  low-memory install/startup failures.
+- Keep direct npm global coverage enabled in install smoke. It exercises plain
+  `npm install -g <candidate>` fresh installs and npm-driven update installs,
+  because many users install with npm even when docs prefer pnpm.
+- Use `pnpm test:live:media video` for bounded video-provider smoke when video
+  generation is in release scope. The default video smoke skips `fal`, runs one
+  text-to-video attempt per provider with a one-second lobster prompt, and caps
+  each provider operation with `OPENCLAW_LIVE_VIDEO_GENERATION_TIMEOUT_MS`
+  (`180000` by default).
+- Run `pnpm test:live:media video --video-providers fal` only when FAL-specific
+  proof is required. Its queue latency can dominate release time.
+- Set `OPENCLAW_LIVE_VIDEO_GENERATION_FULL_MODES=1` only when intentionally
+  validating the slower image-to-video and video-to-video transform lanes.
 
 ## Check all relevant release builds
 
@@ -97,6 +113,13 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts <published-version>
   - `pnpm release:check`
   - `OPENCLAW_INSTALL_SMOKE_SKIP_NONROOT=1 pnpm test:install:smoke`
 - Check all release-related build surfaces touched by the release, not only the npm package.
+- For beta-style full e2e batteries, hard-cap top-level long lanes instead of letting them run indefinitely. Use host `timeout --foreground`/`gtimeout --foreground` caps such as:
+  - `45m` for `OPENCLAW_INSTALL_SMOKE_SKIP_NONROOT=1 pnpm test:install:smoke`
+  - `90m` for `pnpm test:docker:all`
+  - Parallels caps from the `openclaw-parallels-smoke` skill
+    If a lane hits its cap, stop and inspect/fix the affected lane before continuing; do not continue to wait on the same process.
+- Actual npm install/update phases are capped at 5 minutes. If `npm install -g`, installer package install, or `openclaw update` takes longer than 300s in release e2e, stop treating the run as healthy progress and debug the installer/updater or harness.
+- Serialize host build/package mutations ahead of VM lanes. Finish `pnpm build`, `pnpm ui:build`, `pnpm release:check`, install smoke, and any Docker/package-prep lanes before starting Parallels `npm pack` lanes; otherwise `dist` can disappear during VM pack prep and produce false failures.
 - Include mac release readiness in preflight by running the public validation
   workflow in `openclaw/openclaw` and the real mac preflight in
   `openclaw/releases-private` for every release.
@@ -120,6 +143,10 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts <published-version>
   `.github/workflows/openclaw-npm-release.yml`, but it still needs a valid
   `NPM_TOKEN` because `npm dist-tag` management is separate from trusted
   publishing.
+- Direct stable publishes can also run the same workflow with
+  `sync_stable_dist_tags=true` to point both `latest` and `beta` at the
+  already-published stable version. This also needs the `npm-release`
+  environment approval and `NPM_TOKEN`.
 - The publish run must be started manually with `workflow_dispatch`.
 - The npm workflow and the private mac publish workflow accept
   `preflight_only=true` to run validation/build/package steps without uploading
@@ -178,7 +205,10 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts <published-version>
   plan does not yet support required reviewers there, do not assume the
   environment alone is the approval boundary; rely on private repo access and
   CODEOWNERS until those settings can be enabled.
-- Do not use `NPM_TOKEN` or the plugin OTP flow for OpenClaw releases.
+- Do not use `NPM_TOKEN` or the plugin OTP flow for the OpenClaw package
+  publish path; package publishing uses trusted publishing.
+- Use `NPM_TOKEN` only for explicit npm dist-tag management modes, because npm
+  does not support trusted publishing for `npm dist-tag add`.
 - `@openclaw/*` plugin publishes use a separate maintainer-only flow.
 - Only publish plugins that already exist on npm; bundled disk-tree-only plugins stay unpublished.
 
@@ -248,19 +278,25 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts <published-version>
     passes with the same stable tag, `promote_beta_to_latest=true`,
     `preflight_only=false`, empty `preflight_run_id`, and `npm_dist_tag=beta`,
     then verify `latest` now points at that version.
-17. Start
+17. If the stable release was published directly to `latest` and `beta` should
+    follow it, start `.github/workflows/openclaw-npm-release.yml` again with
+    the same stable tag, `sync_stable_dist_tags=true`,
+    `promote_beta_to_latest=false`, `preflight_only=false`, empty
+    `preflight_run_id`, and `npm_dist_tag=latest`, then verify both `latest`
+    and `beta` point at that version.
+18. Start
     `openclaw/releases-private/.github/workflows/openclaw-macos-publish.yml`
     for the real publish with the successful private mac `preflight_run_id` and
     wait for success.
-18. Verify the successful real private mac run uploaded the `.zip`, `.dmg`,
+19. Verify the successful real private mac run uploaded the `.zip`, `.dmg`,
     and `.dSYM.zip` artifacts to the existing GitHub release in
     `openclaw/openclaw`.
-19. For stable releases, download `macos-appcast-<tag>` from the successful
+20. For stable releases, download `macos-appcast-<tag>` from the successful
     private mac run, update `appcast.xml` on `main`, and verify the feed.
-20. For beta releases, publish the mac assets but expect no shared production
+21. For beta releases, publish the mac assets but expect no shared production
     `appcast.xml` artifact and do not update the shared production feed unless a
     separate beta feed exists.
-21. After publish, verify npm and the attached release artifacts.
+22. After publish, verify npm and the attached release artifacts.
 
 ## GHSA advisory work