From 55793e40d42dcd279e100f83fb1a79780780f3e3 Mon Sep 17 00:00:00 2001 From: Kadir Atali Date: Wed, 24 Jun 2026 00:18:34 +0300 Subject: [PATCH] Replace Toolshop with a getmobil.com search suite + selector self-heal MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Retarget the project from the Toolshop demo app to getmobil.com, build a live-verified BDD search suite, and add the tooling that made it possible. - Remove all Toolshop artifacts (features/steps/pages/selectors/story, CI docker-compose + warmup); retarget config/env/CI/README/eval to getmobil.com. - pageInspector: recognise data-test-id / data-cy / data-qa / data-automation-id / data-e2e (not just data-test/data-testid) and build selectors with the actual attribute name. getmobil's search box is data-test-id="selenium-...". - testGenerator: disable thinking, add output_config.effort, and make --max trim an approved design (adaptive thinking was consuming the whole token budget and emitting no answer). Same thinking fix in failureAnalyzer. - getmobil product-search suite: feature + steps + SearchPage + centralised src/pages/selectors/getmobil.selectors.ts. 2 scenarios pass live (npm test). - Runtime selector self-heal: new heal-selectors agent tool + selectorHealer.ts — on a runtime locator failure, re-inspect the page and patch the bad selector from a fresh map. Bounded (MAX_SELECTOR_HEAL_ROUNDS), .bak rollback, gated; proven live (break a selector -> run fails -> heal -> re-run green). Co-Authored-By: Claude Opus 4.8 --- .env.example | 2 +- .github/workflows/tests.yml | 90 +-------------- README.md | 27 +++-- ci/docker-compose.toolshop.yml | 46 -------- ci/warmup.mjs | 19 ---- features/product-search.feature | 30 +++++ features/toolshop-login.feature | 17 --- features/toolshop-search.feature | 18 --- fixtures/users.json | 7 +- playwright.config.ts | 2 +- scripts/eval.ts | 22 ++-- src/agent/policy.ts | 4 + src/agent/prompts.ts | 13 ++- src/agent/state.ts | 13 ++- src/agent/tools.ts | 127 ++++++++++++++++++++++ src/ai/failureAnalyzer.ts | 4 +- src/ai/pageInspector.ts | 55 +++++++--- src/ai/prompts.ts | 25 +++++ src/ai/selectorHealer.ts | 98 +++++++++++++++++ src/ai/testGenerator.ts | 27 +++-- src/config.ts | 2 +- src/fixtures/index.ts | 11 +- src/pages/SearchPage.ts | 37 +++++++ src/pages/ToolshopLoginPage.ts | 26 ----- src/pages/ToolshopProductsPage.ts | 28 ----- src/pages/selectors/getmobil.selectors.ts | 25 +++++ src/pages/selectors/toolshop.selectors.ts | 18 --- src/steps/product-search.steps.ts | 68 ++++++++++++ src/steps/toolshop-login.steps.ts | 27 ----- src/steps/toolshop-search.steps.ts | 24 ---- stories/getmobil-search.txt | 19 ++++ stories/login.txt | 19 ---- 32 files changed, 550 insertions(+), 400 deletions(-) delete mode 100644 ci/docker-compose.toolshop.yml delete mode 100644 ci/warmup.mjs create mode 100644 features/product-search.feature delete mode 100644 features/toolshop-login.feature delete mode 100644 features/toolshop-search.feature create mode 100644 src/ai/selectorHealer.ts create mode 100644 src/pages/SearchPage.ts delete mode 100644 src/pages/ToolshopLoginPage.ts delete mode 100644 src/pages/ToolshopProductsPage.ts create mode 100644 src/pages/selectors/getmobil.selectors.ts delete mode 100644 src/pages/selectors/toolshop.selectors.ts create mode 100644 src/steps/product-search.steps.ts delete mode 100644 src/steps/toolshop-login.steps.ts delete mode 100644 src/steps/toolshop-search.steps.ts create mode 100644 stories/getmobil-search.txt delete mode 100644 stories/login.txt diff --git a/.env.example b/.env.example index 9676a73..d90856f 100644 --- a/.env.example +++ b/.env.example @@ -2,7 +2,7 @@ ANTHROPIC_API_KEY=sk-ant-... # URL of the application under test -BASE_URL=https://practicesoftwaretesting.com +BASE_URL=https://getmobil.com # true = run the browser headless HEADLESS=true diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 0922bc0..c3a74f1 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -6,14 +6,10 @@ on: pull_request: workflow_dispatch: -# The BDD suite targets a live third-party app (practicesoftwaretesting.com) that sits -# behind a Cloudflare "verify you are human" challenge. That challenge blocks data-centre -# IPs such as GitHub's runners, so the app never loads in CI and the browser tests cannot -# run there. They are run locally (`npm test`) where a normal browser/IP passes. -# -# CI therefore gates on the reliable, offline checks below. To run the E2E suite in CI, -# self-host the app under test (it ships as a Docker image) and point TARGET_URL at the -# local instance — then the Cloudflare challenge no longer applies. +# The BDD suite targets a live third-party app (https://getmobil.com). Public sites can sit +# behind bot challenges that block data-centre IPs such as GitHub's runners, so the browser +# tests are run locally (`npm test`) where a normal browser/IP passes. CI gates on the +# reliable, offline checks below (type-check + redaction). jobs: checks: runs-on: ubuntu-latest @@ -36,81 +32,3 @@ jobs: - name: Redaction check run: npm run verify:redaction - - # Real E2E against a self-hosted copy of the app under test, so there is no Cloudflare - # challenge. The Toolshop app runs from its official Docker images; the dev UI talks to - # the local API, and we point TARGET_URL at the local instance. - e2e: - runs-on: ubuntu-latest - timeout-minutes: 30 - env: - COMPOSE: ci/docker-compose.toolshop.yml - TARGET_URL: http://localhost:4200 - steps: - - name: Checkout - uses: actions/checkout@v4 - - - name: Setup Node - uses: actions/setup-node@v4 - with: - node-version: 20 - cache: npm - - - name: Install dependencies - run: npm ci - - - name: Install Playwright browsers - run: npx playwright install --with-deps chromium - - # Docker publishes the ports on IPv4 only. The browser resolves `localhost` to ::1 - # (IPv6) first, where nothing listens, so the app's API calls fail. Force IPv4. - - name: Force IPv4 for localhost - run: sudo sed -i '/^::1/d' /etc/hosts - - - name: Start the Toolshop app - run: docker compose -f "$COMPOSE" up -d --pull always - - - name: Seed the database - run: | - # Retry the seed until the DB accepts connections (more robust than guessing a - # ping command); migrate:fresh sets up the schema + demo products and users. - for i in $(seq 1 40); do - docker compose -f "$COMPOSE" exec -T laravel-api php artisan migrate:fresh --seed --force && break - echo "DB not ready, retrying seed ($i)…"; sleep 5 - done - - - name: Wait for the app to be ready - run: | - for i in $(seq 1 60); do - curl -sf http://localhost:8091/products >/dev/null && { echo "API up"; break; } - echo "waiting for API ($i)…"; sleep 5 - done - # The dev UI serves the shell immediately but is only interactive once ng serve - # finishes compiling — wait for that marker instead of a bare HTTP 200. - for i in $(seq 1 60); do - docker compose -f "$COMPOSE" logs angular-ui 2>&1 | grep -q "Application bundle generation complete" \ - && { echo "UI bundle compiled"; break; } - echo "waiting for UI bundle ($i)…"; sleep 5 - done - curl -sf http://localhost:4200 >/dev/null && echo "UI serving" - - - name: Warm up the app (compile lazy routes, boot the API) - run: node ci/warmup.mjs - - - name: Run BDD tests (self-hosted app) - run: npm test - - - name: Dump container logs on failure - if: failure() - run: docker compose -f "$COMPOSE" logs --tail=80 - - - name: Upload reports - if: always() - uses: actions/upload-artifact@v4 - with: - name: e2e-reports - path: | - reports/cucumber-report.html - reports/cucumber-report.json - retention-days: 14 - if-no-files-found: ignore diff --git a/README.md b/README.md index 88e5903..2544c2e 100644 --- a/README.md +++ b/README.md @@ -50,11 +50,10 @@ HEADLESS=false npm test # watch the browser npm run report # open the Cucumber HTML report ``` -> **CI note:** `npm test` locally targets the live app (`https://practicesoftwaretesting.com`). -> That site sits behind a Cloudflare bot challenge that blocks GitHub's runner IPs, so CI -> instead **self-hosts the app** from its official Docker images -> (`ci/docker-compose.toolshop.yml`) and points `TARGET_URL` at the local instance — real -> E2E, no Cloudflare. CI also runs offline gates (type-check, redaction). +> **CI note:** `npm test` locally targets the live app (`https://getmobil.com`). Public +> sites can sit behind bot challenges that block GitHub's runner IPs, so the browser tests +> are run locally where a normal browser/IP passes. CI gates on the offline checks +> (type-check, redaction). Reports are written under `reports/`: the Cucumber HTML/JSON report. Screenshots and traces are collected automatically for failed scenarios (`reports/test-results/`). @@ -82,13 +81,13 @@ ones to a stable ancestor, and collapses repeated list rows to one representativ parametrize. With `--login` it signs in first via the project's login page. ```bash -npm run ai:inspect -- https://practicesoftwaretesting.com # any public page -npm run ai:inspect -- https://practicesoftwaretesting.com/auth/login +npm run ai:inspect -- https://getmobil.com # any public page +npm run ai:inspect -- https://getmobil.com/iphone # a results/listing page ``` Output: `reports/selector-map-.json`. Accepts a full URL or a path resolved against -`BASE_URL`, so it works against any site (the only site-specific part is `LoginPage`, used -only for `--login`). Page text is PII-redacted before it is written. +`BASE_URL`, so it works against any site. (`--login` performs authenticated inspection only +when a project login page object is wired.) Page text is PII-redacted before it is written. ### AI test generation @@ -153,17 +152,17 @@ Detailed report: `reports/ai-analysis.md` ## Worked example (real output) -Inspecting an authenticated page produces verified, stability-ranked selectors — the -repeated product rows collapse to one representative to parametrize, and nothing is guessed: +Inspecting a page produces verified, stability-ranked selectors — the repeated product +rows collapse to one representative to parametrize, and nothing is guessed: ``` -$ npm run ai:inspect -- https://practicesoftwaretesting.com -Practice Software Testing - Toolshop - v5.0 +$ npm run ai:inspect -- https://getmobil.com +Getmobil - Yenilenmiş Telefon, Tablet, Bilgisayar Elements found : 45 Unique selectors : 40 Repeated (lists) : 5 (parametrize per item) Needs disambig. : 0 -Selector map: reports/selector-map-practicesoftwaretesting-com.json +Selector map: reports/selector-map-getmobil-com.json ``` When a test fails, the analyzer classifies it instead of leaving you to triage: diff --git a/ci/docker-compose.toolshop.yml b/ci/docker-compose.toolshop.yml deleted file mode 100644 index 7973f8b..0000000 --- a/ci/docker-compose.toolshop.yml +++ /dev/null @@ -1,46 +0,0 @@ -# Self-hosted Practice Software Testing ("Toolshop") app for CI E2E. -# Uses the official prebuilt images (no source build). The Angular UI runs in dev mode, -# so it talks to the LOCAL API (http://localhost:8091) — no public site, no Cloudflare. -# Derived from the project's docker-compose.prod.yml (SPRINT pinned to sprint5). -services: - mariadb: - image: yobasystems/alpine-mariadb:10.6.11 - environment: - MYSQL_ROOT_PASSWORD: root - MYSQL_USER: user - MYSQL_PASSWORD: root - MYSQL_DATABASE: toolshop - ports: - - 3306:3306 - - laravel-api: - image: testsmith/practice-software-testing-sprint5-api - environment: - - PHP_OPCACHE_VALIDATE_TIMESTAMPS=1 - - DB_PORT=3306 - - DB_HOST=mariadb - - host=localhost - - DISABLE_LOGGING=true - volumes: - - laravel-app-code:/var/www - depends_on: - - mariadb - - web: - image: testsmith/practice-software-testing-web - ports: - - 8091:80 - - 8000:81 - volumes: - - laravel-app-code:/var/www - depends_on: - - laravel-api - - angular-ui: - image: testsmith/practice-software-testing-sprint5-ui - ports: - - 4200:4200 - command: bash -c "ng serve --host 0.0.0.0 --port 4200" - -volumes: - laravel-app-code: diff --git a/ci/warmup.mjs b/ci/warmup.mjs deleted file mode 100644 index ae8a15c..0000000 --- a/ci/warmup.mjs +++ /dev/null @@ -1,19 +0,0 @@ -// Warms the self-hosted dev app before the E2E suite runs: a cold `ng serve` compiles -// lazy route chunks on first navigation and the API boots on first request, which can -// exceed test timeouts. Loading the key routes here pays that cost once, up front. -import { chromium } from '@playwright/test'; - -const url = process.env.TARGET_URL || 'http://localhost:4200'; -const browser = await chromium.launch(); -const page = await browser.newPage(); -try { - await page.goto(url, { waitUntil: 'domcontentloaded' }); - await page.locator('[data-test="product-name"]').first().waitFor({ state: 'visible', timeout: 120_000 }); - console.log('warm-up: home + products ready'); - - await page.goto(`${url}/auth/login`, { waitUntil: 'domcontentloaded' }); - await page.locator('[data-test="email"]').waitFor({ state: 'visible', timeout: 120_000 }); - console.log('warm-up: login route ready'); -} finally { - await browser.close(); -} diff --git a/features/product-search.feature b/features/product-search.feature new file mode 100644 index 0000000..87365bc --- /dev/null +++ b/features/product-search.feature @@ -0,0 +1,30 @@ +@search +Feature: Product search on Getmobil + As a visitor to Getmobil + I want to search for a product by its name + So that I can quickly find the device I am looking for + + Background: + Given the visitor is on the Getmobil home page + + @smoke @search + Scenario: Successful search for a known product returns a relevant results page + Given the header search box is visible and empty + When the visitor types "iPhone 13" into the header search box + And the visitor submits the search + Then the page URL contains the search query parameter for "iPhone 13" + And at least one product card is displayed in the results + And a product card mentioning "iPhone 13" is visible in the results + And the results page reflects the search term "iPhone 13" + And no empty-state message is shown + + @smoke @search @i18n + Scenario: Search with Turkish locale characters returns correct results + Given the header search box is visible and empty + When the visitor types "kılıf" into the header search box + And the visitor submits the search + Then the page URL correctly encodes the Turkish characters for "kılıf" + And the results page loads without a server error + And at least one product card is displayed in the results + And the results page reflects the search term "kılıf" + And no empty-state message is shown diff --git a/features/toolshop-login.feature b/features/toolshop-login.feature deleted file mode 100644 index 836c286..0000000 --- a/features/toolshop-login.feature +++ /dev/null @@ -1,17 +0,0 @@ -@smoke -Feature: Toolshop Login - As a registered customer - I want to log in to the Toolshop - So that I can access my account - - Background: - Given the user is on the login page - - Scenario: Successful login with valid credentials - When the user logs in as the "customer" user - Then the account menu should be displayed - - @negative - Scenario: Login fails with a wrong password - When the user logs in with email "customer@practicesoftwaretesting.com" and password "wrongpass" - Then a login error should be displayed diff --git a/features/toolshop-search.feature b/features/toolshop-search.feature deleted file mode 100644 index 6dde621..0000000 --- a/features/toolshop-search.feature +++ /dev/null @@ -1,18 +0,0 @@ -@smoke -Feature: Toolshop Product Search - As a shopper - I want to search the product catalog - So that I can quickly find the tools I need - - Background: - Given the user is on the products page - - Scenario: Searching narrows the catalog to relevant products - When the user searches for "Pliers" - Then at least one product is shown - And the results include "Combination Pliers" - - @negative - Scenario: Searching for an unknown term shows no products - When the user searches for "zzzznotaproduct" - Then no products are shown diff --git a/fixtures/users.json b/fixtures/users.json index e02ec4b..0967ef4 100644 --- a/fixtures/users.json +++ b/fixtures/users.json @@ -1,6 +1 @@ -{ - "customer": { - "username": "customer@practicesoftwaretesting.com", - "password": "welcome01" - } -} +{} diff --git a/playwright.config.ts b/playwright.config.ts index d0074a4..480f681 100644 --- a/playwright.config.ts +++ b/playwright.config.ts @@ -26,7 +26,7 @@ export default defineConfig({ cucumberReporter('html', { outputFile: 'reports/cucumber-report.html' }) ], use: { - baseURL: process.env.BASE_URL ?? 'https://practicesoftwaretesting.com', + baseURL: process.env.BASE_URL ?? 'https://getmobil.com', headless: process.env.HEADLESS !== 'false', viewport: { width: 1280, height: 720 }, navigationTimeout: CI ? 30_000 : 15_000, diff --git a/scripts/eval.ts b/scripts/eval.ts index 58443e6..a8b20bb 100644 --- a/scripts/eval.ts +++ b/scripts/eval.ts @@ -24,11 +24,11 @@ interface Check { const checks: Check[] = []; const add = (name: string, ok: boolean, detail = '') => checks.push({ name, ok, detail }); -const GOLDEN_STORY = `Title: Customer login -As a registered customer I want to log in so that I can access my account. +const GOLDEN_STORY = `Title: Product search on Getmobil +As a visitor I want to search for a product by name so that I can find the device I want. Acceptance Criteria: -1. Valid credentials show the account menu. -2. A wrong password shows a login error.`; +1. Submitting a product name shows a results view listing matching products. +2. A term with no matches shows a clear empty-state message and no products.`; function deterministic(): void { // 1) Redaction: masks PII patterns + denylist values, keeps benign numbers. @@ -42,20 +42,18 @@ function deterministic(): void { // 2) Project surface: discovers the real helpers, page objects, and steps. const surface = readProjectSurface(); - const surfaceOk = ['getUser', 'ToolshopLoginPage', 'the user is on the login page'].every((s) => - surface.includes(s) - ); + const surfaceOk = ['getUser', 'loadFixture'].every((s) => surface.includes(s)); add('project surface finds real helpers/pages/steps', surfaceOk); } async function liveInspect(): Promise { try { - const map = await inspectPage(`${TARGET_URL}/auth/login`); - const unique = (s: string) => map.entries.some((e) => e.selector.includes(s) && e.count === 1); - const ok = unique('email') && unique('password') && unique('login-submit'); - add('inspector finds unique login selectors (live)', ok, `${map.entries.length} elements`); + const map = await inspectPage(`${TARGET_URL}/`); + const hasInput = map.entries.some((e) => /input|search|textbox/i.test(e.selector)); + const ok = map.entries.length > 0 && hasInput; + add('inspector extracts a selector map from the home page (live)', ok, `${map.entries.length} elements`); } catch (e: any) { - add('inspector finds unique login selectors (live)', false, e?.message ?? String(e)); + add('inspector extracts a selector map from the home page (live)', false, e?.message ?? String(e)); } } diff --git a/src/agent/policy.ts b/src/agent/policy.ts index 117ccd4..2124fd3 100644 --- a/src/agent/policy.ts +++ b/src/agent/policy.ts @@ -20,6 +20,10 @@ const RULES: Record = { inspect: { decision: 'confirm', reason: 'Opens a live browser session against the target URL.' }, generate: { decision: 'confirm', reason: 'Writes/overwrites feature, step and page-object source files.' }, heal: { decision: 'auto', reason: 'Targeted compile-error fix on already-generated code — low risk, gated by tsc.' }, + 'heal-selectors': { + decision: 'confirm', + reason: 'Re-opens a live browser to re-inspect the page and rewrites selector/locator code.' + }, verify: { decision: 'auto', reason: 'Read-only type-check (tsc --noEmit).' }, run: { decision: 'confirm', reason: 'Launches a real browser and executes scenarios.' }, analyze: { decision: 'auto', reason: 'Reads the report and writes an analysis — no code.' } diff --git a/src/agent/prompts.ts b/src/agent/prompts.ts index 1705ccf..5bf8e52 100644 --- a/src/agent/prompts.ts +++ b/src/agent/prompts.ts @@ -9,8 +9,11 @@ The pipeline (typical order): 3. generate — write the .feature, step definitions and page objects. Prefer useDesign:true and useSelectors:true whenever a design / selector map already exists. 4. verify — type-check the generated code. ALWAYS run this immediately after generate or heal. -5. heal — targeted fix when verify fails: feeds the tsc errors back and rewrites only what's - needed to compile. Prefer this over regenerating from scratch. Re-verifies itself. +5. heal — targeted fix when verify (COMPILE) fails: feeds the tsc errors back and rewrites + only what's needed to compile. Prefer this over regenerating. Re-verifies itself. +5b. heal-selectors — runtime fix when a RUN fails on a locator (timeout waiting for locator / + strict-mode / not visible). Re-inspects the page (pass inspectUrl = the page the + failing step is on) and patches the failing selectors with real ones. Re-verifies. 6. run — execute the scenarios in a real browser. Only after verify passes. Use maxRetries so a scenario that passes on re-run is reported as flaky, not a failure. 7. analyze — explain and categorise failures (app-bug | test-bug | flaky | environment). @@ -19,8 +22,10 @@ The pipeline (typical order): The self-healing loop (close the feedback, don't just report): - verify fails (compile) -> heal, then verify again (up to its budget). - run fails: first analyze, then route by category: - test-bug (the test is wrong) -> if a selector drifted, inspect the affected page again then - generate; otherwise heal/generate the fix; verify; run. + test-bug (the test is wrong) -> if a LOCATOR failed (timeout / strict-mode / not visible), + use heal-selectors (inspectUrl = the failing page) to + re-inspect + patch real selectors; for any other test bug, + fix via heal/generate; then verify and run again. flaky / environment -> re-run (use maxRetries); only escalate if it persists. app-bug (REAL regression) -> STOP. Do NOT heal or rewrite the test to make it pass — that fakes green for behaviour the app does not have. diff --git a/src/agent/state.ts b/src/agent/state.ts index e101fbf..061fe0f 100644 --- a/src/agent/state.ts +++ b/src/agent/state.ts @@ -3,7 +3,15 @@ import * as path from 'path'; import type { GeneratedArtifacts } from '../ai/testGenerator'; /** The pipeline steps the agent can take (each maps to an existing ai/* module). */ -export type StepKind = 'design' | 'inspect' | 'generate' | 'heal' | 'verify' | 'run' | 'analyze'; +export type StepKind = + | 'design' + | 'inspect' + | 'generate' + | 'heal' + | 'heal-selectors' + | 'verify' + | 'run' + | 'analyze'; export interface RunAttempt { step: StepKind; @@ -37,6 +45,8 @@ export interface RunState { lastArtifacts?: GeneratedArtifacts; /** How many targeted `heal` rounds have run (bounded to stop infinite fix loops). */ healRounds: number; + /** How many runtime `heal-selectors` rounds have run (bounded the same way). */ + healSelectorRounds: number; /** Pass/fail tally of the last `run` — used by `analyze` to detect a stale report. */ lastRunPassed?: number; lastRunFailed?: number; @@ -65,6 +75,7 @@ export function newRunState(goal: string, story: string): RunState { startedAt: new Date().toISOString(), artifactFiles: [], healRounds: 0, + healSelectorRounds: 0, attempts: [], notes: [] }; diff --git a/src/agent/tools.ts b/src/agent/tools.ts index d2b9b82..98a0321 100644 --- a/src/agent/tools.ts +++ b/src/agent/tools.ts @@ -7,6 +7,7 @@ import { inspectPage, writeSelectorMap } from '../ai/pageInspector'; import { generateTests, correctArtifacts, writeArtifacts } from '../ai/testGenerator'; import { verifyTypeScript, runFeature } from '../ai/verifier'; import { extractFailures, analyzeFailures, writeAnalysisReport } from '../ai/failureAnalyzer'; +import { repairSelectors, type SourceFile } from '../ai/selectorHealer'; import type { RunState, StepKind } from './state'; @@ -101,6 +102,25 @@ export const TOOL_DEFS: Anthropic.Tool[] = [ }, additionalProperties: false } + }, + { + name: 'heal-selectors', + description: + 'Runtime selector self-heal: when a run failed because a locator did not resolve (timeout waiting for ' + + 'locator / strict-mode / not visible), re-inspect the page where that element lives and rewrite the ' + + 'failing selectors with real ones from the fresh map. Provide inspectUrl = the page the failing step is on ' + + '(e.g. the results URL, or the base URL for header elements). Re-verifies (tsc); then re-run to confirm.', + input_schema: { + type: 'object', + properties: { + inspectUrl: { + type: 'string', + description: 'URL/path of the page where the failing element lives, to re-inspect for real selectors.' + } + }, + required: ['inspectUrl'], + additionalProperties: false + } } ]; @@ -108,8 +128,54 @@ interface InspectInput { target: string; login?: string } interface GenerateInput { useDesign?: boolean; useSelectors?: boolean } interface RunInput { maxRetries?: number } interface AnalyzeInput { reportPath?: string } +interface HealSelectorsInput { inspectUrl?: string } const MAX_HEAL_ROUNDS = 3; +const MAX_SELECTOR_HEAL_ROUNDS = 3; + +/** Repo-relative dirs the selector healer may read from and write back to. */ +const HEAL_DIRS = ['src/pages', 'src/steps']; + +/** Recursively collects .ts sources under HEAL_DIRS (incl. selectors/ subdir), repo-relative. */ +function collectPagesAndSteps(rootDir: string): SourceFile[] { + const out: SourceFile[] = []; + const walk = (rel: string) => { + const abs = path.join(rootDir, rel); + if (!fs.existsSync(abs)) return; + for (const entry of fs.readdirSync(abs, { withFileTypes: true })) { + const childRel = path.join(rel, entry.name); + if (entry.isDirectory()) walk(childRel); + else if (entry.name.endsWith('.ts') && !entry.name.endsWith('.generated') && !entry.name.endsWith('.bak')) { + out.push({ path: childRel, content: fs.readFileSync(path.join(rootDir, childRel), 'utf-8') }); + } + } + }; + for (const dir of HEAL_DIRS) walk(dir); + return out; +} + +/** + * Writes the healer's corrected files back, preserving their relative path. Constrained to + * HEAL_DIRS (rejects anything that escapes), and backs up each original to .bak once so a + * failed heal can be rolled back. Returns the absolute paths actually written. + */ +function writeRepairedFiles(files: SourceFile[], rootDir: string): string[] { + const written: string[] = []; + for (const f of files) { + const rel = path.normalize(f.path).replace(/^(\.\.(\/|\\|$))+/, ''); + const abs = path.resolve(rootDir, rel); + const allowed = HEAL_DIRS.some((d) => abs.startsWith(path.resolve(rootDir, d) + path.sep)); + if (!allowed) continue; // never write outside src/pages or src/steps + fs.mkdirSync(path.dirname(abs), { recursive: true }); + if (fs.existsSync(abs)) { + const bak = `${abs}.bak`; + if (!fs.existsSync(bak)) fs.copyFileSync(abs, bak); + } + fs.writeFileSync(abs, f.content); + written.push(abs); + } + return written; +} /** Reads the existing project sources `heal` may need to update (merging in new members). */ function collectSources(rootDir: string): { fileName: string; content: string }[] { @@ -200,6 +266,7 @@ export async function executeTool( state.artifactFiles = written.map((w) => w.split(' ')[0]).filter((f) => f.endsWith('.ts')); state.lastArtifacts = artifacts; // base for `heal` state.healRounds = 0; // fresh code — reset the healing budget + state.healSelectorRounds = 0; // and the runtime selector-heal budget state.lastFeatureTitle = (artifacts.featureContent.match(/Feature:\s*(.+)/) ?? [])[1]?.trim(); const grounding = [useDesign && design ? 'design' : null, useSelectors && selectors ? 'selectors' : null] @@ -269,6 +336,66 @@ export async function executeTool( }; } + case 'heal-selectors': { + const { inspectUrl } = (input as HealSelectorsInput) ?? {}; + if (!inspectUrl) { + return { ok: false, summary: 'heal-selectors requires "inspectUrl" — the page where the failing element lives.' }; + } + if (!state.lastRunFailed || state.lastRunFailed === 0) { + return { ok: false, summary: 'Nothing to heal — run the suite first; this fixes RUNTIME locator failures.' }; + } + if (state.healSelectorRounds >= MAX_SELECTOR_HEAL_ROUNDS) { + return { + ok: false, + summary: `Selector-heal budget (${MAX_SELECTOR_HEAL_ROUNDS} rounds) exhausted — selectors still fail at runtime.`, + escalate: 'Repeated selector self-heal failed — a human should inspect the page and wire the selectors.' + }; + } + const reportPath = path.join(rootDir, 'reports', 'cucumber-report.json'); + if (!fs.existsSync(reportPath)) { + return { ok: false, summary: `No report at ${reportPath} — run the suite first.` }; + } + const failures = extractFailures(reportPath).map((f) => ({ + scenario: f.scenario, + failedStep: f.failedStep, + errorMessage: f.errorMessage + })); + if (failures.length === 0) { + return { ok: false, summary: 'Report shows no failures — nothing to heal.' }; + } + + // Re-inspect the page where the failing element lives → fresh, real selectors. + const map = await inspectPage(inspectUrl); + state.selectorMapPath = writeSelectorMap(map, rootDir); + + const sources = collectPagesAndSteps(rootDir); + const repair = await repairSelectors(state.story, failures, JSON.stringify(map, null, 2), sources); + const written = writeRepairedFiles(repair.files, rootDir); + if (written.length === 0) { + return { ok: false, summary: `Healer proposed no in-scope file changes. Notes: ${repair.notes}` }; + } + state.healSelectorRounds++; + + // Keep the patched files in the verify/run scope, then confirm they still type-check. + const tsFiles = written.filter((f) => f.endsWith('.ts')); + state.artifactFiles = [...new Set([...state.artifactFiles, ...tsFiles])]; + const after = verifyTypeScript(tsFiles, rootDir); + if (!after.ok) { + return { + ok: false, + summary: + `Patched ${tsFiles.length} file(s) from ${inspectUrl} but tsc now fails: ${tscDetail(after.errors)}. ` + + `Use heal, then re-run.` + }; + } + return { + ok: true, + summary: + `Selector heal (round ${state.healSelectorRounds}/${MAX_SELECTOR_HEAL_ROUNDS}): re-inspected ${inspectUrl}, ` + + `patched ${written.map((w) => path.basename(w)).join(', ')}; type-checks. Re-run to confirm.` + }; + } + case 'run': { if (!state.lastFeatureTitle) { return { ok: false, summary: 'No generated feature to run — generate (and verify) first.' }; diff --git a/src/ai/failureAnalyzer.ts b/src/ai/failureAnalyzer.ts index 7ce2000..cefb009 100644 --- a/src/ai/failureAnalyzer.ts +++ b/src/ai/failureAnalyzer.ts @@ -80,7 +80,9 @@ export async function analyzeFailures( const stream = client.messages.stream({ model: MODEL, max_tokens: 32000, - thinking: { type: 'adaptive' }, + // Thinking disabled: adaptive thinking + structured output can spend the whole token + // budget reasoning and emit no answer (same runaway fixed in testGenerator/selectorHealer). + thinking: { type: 'disabled' }, system: ANALYZER_SYSTEM, output_config: { format: { type: 'json_schema', schema: ANALYSIS_SCHEMA } diff --git a/src/ai/pageInspector.ts b/src/ai/pageInspector.ts index 3c3a8e0..2b573c1 100644 --- a/src/ai/pageInspector.ts +++ b/src/ai/pageInspector.ts @@ -2,8 +2,6 @@ import * as fs from 'fs'; import * as path from 'path'; import { chromium, Page } from '@playwright/test'; import * as dotenv from 'dotenv'; -import { ToolshopLoginPage } from '../pages/ToolshopLoginPage'; -import { getUser } from '../fixtures/data'; import { redact } from './redact'; import { registerAllSensitive } from '../fixtures/data'; import { TARGET_URL as BASE_URL } from '../config'; @@ -49,10 +47,13 @@ interface RawEl { role: string; name: string; dataTest: string | null; + /** Which test-id attribute the value came from (data-test, data-test-id, data-cy, ...). */ + dataTestAttr: string | null; id: string | null; classes: string; type: string | null; ancestorDataTest: string | null; + ancestorDataTestAttr: string | null; cssPath: string; } @@ -153,11 +154,23 @@ function collectScript(maxEls: number): string { return st.visibility !== 'hidden' && st.display !== 'none'; }; - const nearestAncestorDataTest = (el) => { + // Common test-id attribute conventions, in priority order. Recognising hyphenated + // (data-test-id), camel-ish (data-testid) and other framework variants — not just + // data-test — means a stable hook like data-test-id="selenium-..." is used verbatim + // instead of falling back to a fragile structural selector. + const TESTID_ATTRS = ['data-test', 'data-testid', 'data-test-id', 'data-cy', 'data-qa', 'data-automation-id', 'data-e2e']; + const ownTestId = (el) => { + for (const a of TESTID_ATTRS) { + const v = el.getAttribute(a); + if (v) return { attr: a, val: v }; + } + return null; + }; + const nearestAncestorTestId = (el) => { let p = el.parentElement; while (p) { - const dt = p.getAttribute('data-test') || p.getAttribute('data-testid'); - if (dt) return dt; + const t = ownTestId(p); + if (t) return t; p = p.parentElement; } return null; @@ -179,7 +192,7 @@ function collectScript(maxEls: number): string { return parts.join(' > '); }; - const MATCH = 'a[href],button,input,select,textarea,summary,[role],[data-test],[data-testid],h1,h2,h3,[aria-label]'; + const MATCH = 'a[href],button,input,select,textarea,summary,[role],[data-test],[data-testid],[data-test-id],[data-cy],[data-qa],[data-automation-id],[data-e2e],h1,h2,h3,[aria-label]'; const walk = (root) => { if (out.length >= MAX) return; @@ -189,15 +202,19 @@ function collectScript(maxEls: number): string { if (out.length >= MAX) break; if (!isVisible(el)) continue; const role = roleFromTag(el); + const own = ownTestId(el); + const anc = nearestAncestorTestId(el); out.push({ tag: el.tagName.toLowerCase(), role, name: accName(el), - dataTest: el.getAttribute('data-test') || el.getAttribute('data-testid'), + dataTest: own ? own.val : null, + dataTestAttr: own ? own.attr : null, id: el.id || null, classes: el.getAttribute('class') || '', type: el.getAttribute('type'), - ancestorDataTest: nearestAncestorDataTest(el), + ancestorDataTest: anc ? anc.val : null, + ancestorDataTestAttr: anc ? anc.attr : null, cssPath: shortPath(el) }); } @@ -249,9 +266,10 @@ function buildCandidate(raw: RawEl): SelectorEntry { ambiguous: false }; - // 1) data-test (most trustworthy) - if (raw.dataTest) { - return { ...base, selector: `[data-test="${escAttr(raw.dataTest)}"]`, strategy: 'data-test' }; + // 1) test-id attribute (most trustworthy) — uses the actual attribute name it came from + // (data-test / data-test-id / data-cy / ...), not a hardcoded data-test. + if (raw.dataTest && raw.dataTestAttr) { + return { ...base, selector: `[${raw.dataTestAttr}="${escAttr(raw.dataTest)}"]`, strategy: 'data-test' }; } // 2) stable id if (raw.id && !looksGenerated(raw.id)) { @@ -293,10 +311,11 @@ export async function inspectPage(target: string, opts: InspectOptions = {}): Pr try { if (opts.loginUserKey) { - const user = getUser(opts.loginUserKey); - const loginPage = new ToolshopLoginPage(page); - await loginPage.open(); - await loginPage.login(user.username, user.password); + // Authenticated inspection needs a project login page object to be wired here. + // None is currently configured, so we inspect the page unauthenticated and warn. + warnings.push( + `authenticated inspection requested (loginUserKey="${opts.loginUserKey}") but no login page object is wired; inspecting unauthenticated` + ); } await page.goto(url, { waitUntil: 'domcontentloaded' }); @@ -320,9 +339,9 @@ export async function inspectPage(target: string, opts: InspectOptions = {}): Pr const entry = buildCandidate(raw); entry.count = await countOf(page, entry); - if (entry.count > 1 && raw.ancestorDataTest && entry.strategy !== 'data-test') { - // try scoping by a stable ancestor - entry.scope = `[data-test="${escAttr(raw.ancestorDataTest)}"]`; + if (entry.count > 1 && raw.ancestorDataTest && raw.ancestorDataTestAttr && entry.strategy !== 'data-test') { + // try scoping by a stable ancestor (any recognised test-id attribute) + entry.scope = `[${raw.ancestorDataTestAttr}="${escAttr(raw.ancestorDataTest)}"]`; entry.count = await countOf(page, entry); } entry.ambiguous = entry.count > 1; diff --git a/src/ai/prompts.ts b/src/ai/prompts.ts index 28dbc7c..df840a6 100644 --- a/src/ai/prompts.ts +++ b/src/ai/prompts.ts @@ -67,6 +67,10 @@ unknown, derive them from the story if provided, otherwise use clearly marked pl selectors with TODO comments. Scenario depth - write thorough scenarios, not 2-line stubs: +- Each scenario must have BETWEEN 6 AND 10 steps (counting Given/When/Then/And/But lines, + excluding the Scenario title and any Background). Fewer than 6 is too thin; more than 10 + means the scenario is doing too much - split it. Reach the count with meaningful context + and assertions, never with filler or click-by-click mechanics. - Set the meaningful Given context and the specific test data each scenario needs; do not assume a bare starting state when a precondition matters. - A Then must verify the real, observable CONSEQUENCES of the action - usually MORE THAN @@ -167,6 +171,27 @@ errors and the current source of the existing project files follow. Rules: - Follow the project conventions and keep the feature/steps consistent with the fixes. - Return the FULL corrected artifact set (feature, steps, page objects, notes).`; +// Used by the runtime selector self-heal loop: a scenario failed at RUN time because a +// locator did not resolve (timeout waiting for locator / strict-mode / not visible). A FRESH +// selector map (re-inspected from the page where the element lives) is provided. +export const SELECTOR_REPAIR_INSTRUCTION = ` +A generated scenario FAILED AT RUNTIME because a Playwright locator did not resolve (e.g. +"Timeout … waiting for locator(...)", a strict-mode violation, or "not visible"). This is a +TEST bug, not an app bug — the selector is wrong/stale, not the application. Below are: the +failing step(s) + error message(s) (the failing locator string is in the error), a FRESH +selector map re-inspected from the live page, and the current source of the project files. +Rules: +- Identify the failing locator from the error message and replace it with a REAL selector + from the fresh map (verbatim; respect its strategy — CSS string vs getByRole/getByText). + If the element resolves to several nodes, scope it or use .first(), as the map's count + indicates. Only keep a placeholder + TODO if the element is genuinely not in the map. +- Centralise selectors in the src/pages/selectors/*.ts module and reference them from the + page object (matching the project convention) — do not scatter raw strings inline. +- Do NOT change Gherkin step phrasings (the Given/When/Then text bound to the feature); only + fix locator/assertion IMPLEMENTATION in the .ts files. +- Return ONLY the files you actually changed, each as a repo-relative path under src/pages or + src/steps, with the COMPLETE new file content (not a diff).`; + export const ANALYZER_SYSTEM = `${FRAMEWORK_CONTEXT} Your task: analyze failed BDD scenarios from a test run. For each failure decide diff --git a/src/ai/selectorHealer.ts b/src/ai/selectorHealer.ts new file mode 100644 index 0000000..54c2f37 --- /dev/null +++ b/src/ai/selectorHealer.ts @@ -0,0 +1,98 @@ +import { getClient, MODEL } from './client'; +import { SELECTOR_REPAIR_INSTRUCTION, PROJECT_SURFACE_INSTRUCTION } from './prompts'; +import { redact } from './redact'; +import { registerAllSensitive } from '../fixtures/data'; +import { readProjectSurface } from './projectSurface'; + +/** A failed scenario as pulled from the Cucumber report (subset of failureAnalyzer's shape). */ +export interface RuntimeFailure { + scenario: string; + failedStep: string; + errorMessage: string; +} + +/** A repo-relative source file the healer can read and (for changed files) rewrite. */ +export interface SourceFile { + path: string; + content: string; +} + +export interface SelectorRepair { + files: SourceFile[]; + notes: string; +} + +const REPAIR_SCHEMA = { + type: 'object', + properties: { + files: { + type: 'array', + description: 'Only the files actually changed, with full new content.', + items: { + type: 'object', + properties: { + path: { type: 'string', description: 'Repo-relative path under src/pages or src/steps.' }, + content: { type: 'string', description: 'Complete new file content (not a diff).' } + }, + required: ['path', 'content'], + additionalProperties: false + } + }, + notes: { type: 'string', description: 'What was changed and why; any element still unresolved.' } + }, + required: ['files', 'notes'], + additionalProperties: false +} as const; + +/** + * Runtime selector self-heal: given scenarios that failed because a locator did not resolve + * and a FRESH selector map re-inspected from the live page, returns corrected source files + * that swap the failing locators for real ones. The compile-time counterpart is + * `correctArtifacts` (tsc errors); this one is driven by run-time locator failures. + */ +export async function repairSelectors( + story: string, + failures: RuntimeFailure[], + freshSelectorMapJson: string, + sources: SourceFile[] +): Promise { + registerAllSensitive(); + + const failureText = failures + .map((f) => `Scenario: ${f.scenario}\nFailed step: ${f.failedStep}\nError:\n${f.errorMessage}`) + .join('\n\n---\n\n'); + const sourcesText = sources.map((s) => `// ${s.path}\n${s.content}`).join('\n\n'); + + let content = + `${SELECTOR_REPAIR_INSTRUCTION}\n\nUSER STORY:\n\n${redact(story)}\n\n` + + `RUNTIME FAILURES:\n\n${redact(failureText)}\n\n` + + `FRESH SELECTOR MAP (re-inspected):\n\n${redact(freshSelectorMapJson)}\n\n` + + `CURRENT PROJECT SOURCES (return only the ones you change):\n\n${redact(sourcesText)}`; + + const surface = readProjectSurface(); + if (surface) content += `\n\n${PROJECT_SURFACE_INSTRUCTION}\n\nPROJECT API SURFACE:\n\n${surface}`; + + const client = getClient(); + const stream = client.messages.stream({ + model: MODEL, + max_tokens: 32000, + // Thinking disabled: like generation, adaptive thinking can consume the whole budget on + // this large structured-output task and emit no answer. Disabled = straight to the JSON. + thinking: { type: 'disabled' }, + output_config: { format: { type: 'json_schema', schema: REPAIR_SCHEMA } }, + messages: [{ role: 'user', content }] + }); + + stream.on('text', () => process.stdout.write('.')); + const message = await stream.finalMessage(); + process.stdout.write('\n'); + + if (message.stop_reason === 'refusal') { + throw new Error('The model refused the request (stop_reason: refusal).'); + } + const textBlock = message.content.find((b) => b.type === 'text'); + if (!textBlock || textBlock.type !== 'text') { + throw new Error('No text response received from the model.'); + } + return JSON.parse(textBlock.text) as SelectorRepair; +} diff --git a/src/ai/testGenerator.ts b/src/ai/testGenerator.ts index 7121436..65b6ba0 100644 --- a/src/ai/testGenerator.ts +++ b/src/ai/testGenerator.ts @@ -84,12 +84,16 @@ export async function generateTests( content += `\n\n${SELECTORS_INSTRUCTION}\n\nSELECTOR MAP:\n\n${safeMap}`; } - // Quick-trial cap for the story-only path. When a design is supplied it already fixes the - // scope, so the cap only applies here (don't override a curated design's scenario count). - if (maxScenarios && maxScenarios > 0 && !approvedDesign?.trim()) { - content += - `\n\nQUICK TRIAL: generate AT MOST ${maxScenarios} scenario(s) — the highest-value ones ` + - `(core happy path first, then the most important negative case). Keep it minimal so the run is fast.`; + // Quick-trial cap. Applies whether or not a design is supplied: a curated design can list + // many scenarios (15+), and implementing all of them is a huge, slow generation — so with + // --max we restrict to the design's top-priority scenarios for a fast trial run. + if (maxScenarios && maxScenarios > 0) { + content += approvedDesign?.trim() + ? `\n\nQUICK TRIAL (overrides the "implement every listed scenario" rule for this run): ` + + `from the approved design, implement ONLY the ${maxScenarios} highest-priority scenario(s) ` + + `(P0 first, then P1) and ignore the rest. Keep it minimal so the run is fast.` + : `\n\nQUICK TRIAL: generate AT MOST ${maxScenarios} scenario(s) — the highest-value ones ` + + `(core happy path first, then the most important negative case). Keep it minimal so the run is fast.`; } return runGenerator(content); @@ -101,7 +105,10 @@ async function runGenerator(content: string): Promise { const stream = client.messages.stream({ model: MODEL, max_tokens: 64000, - thinking: { type: 'adaptive' }, + // Thinking disabled for generation. With adaptive thinking on (even at low effort), + // the model spent minutes thinking and consumed the whole 64k budget before emitting + // any JSON. Disabled = straight to the structured answer: fast and predictable. + thinking: { type: 'disabled' }, system: GENERATOR_SYSTEM, output_config: { format: { type: 'json_schema', schema: OUTPUT_SCHEMA } @@ -119,6 +126,12 @@ async function runGenerator(content: string): Promise { const textBlock = message.content.find((b) => b.type === 'text'); if (!textBlock || textBlock.type !== 'text') { + if (message.stop_reason === 'max_tokens') { + throw new Error( + 'Hit max_tokens before any answer was emitted — the thinking budget consumed the whole ' + + 'response. Lower output_config.effort or reduce scope (fewer scenarios via --max).' + ); + } throw new Error('No text response received from the model.'); } return JSON.parse(textBlock.text) as GeneratedArtifacts; diff --git a/src/config.ts b/src/config.ts index 67b4260..4e6e9a2 100644 --- a/src/config.ts +++ b/src/config.ts @@ -5,4 +5,4 @@ * suite to a different app is a one-line change (or set TARGET_URL in the environment). * Kept separate from Playwright's BASE_URL so it is not affected by a stale .env value. */ -export const TARGET_URL = process.env.TARGET_URL ?? 'https://practicesoftwaretesting.com'; +export const TARGET_URL = process.env.TARGET_URL ?? 'https://getmobil.com'; diff --git a/src/fixtures/index.ts b/src/fixtures/index.ts index 54232b3..5e9ec13 100644 --- a/src/fixtures/index.ts +++ b/src/fixtures/index.ts @@ -1,13 +1,12 @@ import { test as base } from 'playwright-bdd'; -import { ToolshopLoginPage } from '../pages/ToolshopLoginPage'; -import { ToolshopProductsPage } from '../pages/ToolshopProductsPage'; +import { SearchPage } from '../pages/SearchPage'; +// Page-object fixtures are wired here as they are generated (e.g. by ai:generate). +// Add `fooPage: async ({ page }, use) => use(new FooPage(page))` per page object. type PageFixtures = { - loginPage: ToolshopLoginPage; - productsPage: ToolshopProductsPage; + searchPage: SearchPage; }; export const test = base.extend({ - loginPage: async ({ page }, use) => use(new ToolshopLoginPage(page)), - productsPage: async ({ page }, use) => use(new ToolshopProductsPage(page)) + searchPage: async ({ page }, use) => use(new SearchPage(page)) }); diff --git a/src/pages/SearchPage.ts b/src/pages/SearchPage.ts new file mode 100644 index 0000000..375a73a --- /dev/null +++ b/src/pages/SearchPage.ts @@ -0,0 +1,37 @@ +import { Page, Locator } from '@playwright/test'; +import { BasePage } from './BasePage'; +import { GetmobilSearchSelectors as S } from './selectors/getmobil.selectors'; + +export class SearchPage extends BasePage { + // Header search input. The hook resolves to 2 nodes (desktop + mobile header) -> .first(). + readonly searchInput: Locator = this.page.locator(S.searchInput).first(); + + // Results-page product cards (one per card inside the results grid). + readonly productCards: Locator = this.page.locator(S.productCard); + + // No-results empty state — matched by its Turkish copy. + readonly noResultsMessage: Locator = this.page.getByText(new RegExp(S.noResultsText, 'i')); + + // Results heading/breadcrumb (`"" için sonuçlar`) — the site clears the search box + // on the results page, so the submitted term is asserted against this instead. + readonly resultsHeading: Locator = this.page.getByText(new RegExp(S.resultsHeadingText, 'i')).first(); + + constructor(page: Page) { + super(page); + } + + async open(): Promise { + await this.goto('/'); + await this.page.locator(S.pageLoaded).first().waitFor({ state: 'attached', timeout: 10_000 }).catch(() => {}); + } + + async typeQuery(query: string): Promise { + await this.searchInput.fill(query); + } + + async submitByEnter(): Promise { + await this.searchInput.press('Enter'); + // Wait for the real results navigation (/ara/?term=...), not just any URL. + await this.page.waitForURL(S.resultsUrl, { timeout: 10_000 }); + } +} diff --git a/src/pages/ToolshopLoginPage.ts b/src/pages/ToolshopLoginPage.ts deleted file mode 100644 index bb1c15d..0000000 --- a/src/pages/ToolshopLoginPage.ts +++ /dev/null @@ -1,26 +0,0 @@ -import { Page } from '@playwright/test'; -import { BasePage } from './BasePage'; -import { ToolshopLoginSelectors as S } from './selectors/toolshop.selectors'; -import { TARGET_URL as SITE } from '../config'; - -export class ToolshopLoginPage extends BasePage { - readonly emailInput = this.page.locator(S.email); - readonly passwordInput = this.page.locator(S.password); - readonly submitButton = this.page.locator(S.submit); - readonly errorMessage = this.page.locator(S.error); - readonly navMenu = this.page.locator(S.navMenu); - - constructor(page: Page) { - super(page); - } - - async open(): Promise { - await this.page.goto(`${SITE}/auth/login`); - } - - async login(email: string, password: string): Promise { - await this.emailInput.fill(email); - await this.passwordInput.fill(password); - await this.submitButton.click(); - } -} diff --git a/src/pages/ToolshopProductsPage.ts b/src/pages/ToolshopProductsPage.ts deleted file mode 100644 index 8c0a922..0000000 --- a/src/pages/ToolshopProductsPage.ts +++ /dev/null @@ -1,28 +0,0 @@ -import { Page, Locator } from '@playwright/test'; -import { BasePage } from './BasePage'; -import { ToolshopProductSelectors as S } from './selectors/toolshop.selectors'; -import { TARGET_URL as SITE } from '../config'; - -export class ToolshopProductsPage extends BasePage { - readonly searchInput = this.page.locator(S.searchQuery); - readonly searchButton = this.page.locator(S.searchSubmit); - readonly searchReset = this.page.locator(S.searchReset); - readonly productNames: Locator = this.page.locator(S.productName); - - constructor(page: Page) { - super(page); - } - - async open(): Promise { - await this.page.goto(`${SITE}/`); - } - - async search(term: string): Promise { - await this.searchInput.fill(term); - await this.searchButton.click(); - } - - async resetSearch(): Promise { - await this.searchReset.click(); - } -} diff --git a/src/pages/selectors/getmobil.selectors.ts b/src/pages/selectors/getmobil.selectors.ts new file mode 100644 index 0000000..016d67a --- /dev/null +++ b/src/pages/selectors/getmobil.selectors.ts @@ -0,0 +1,25 @@ +/** + * Selector strings for getmobil.com. + * Verified against the live DOM via `ai:inspect` — the search box + page-ready hook on the + * home page, and the results grid + no-results copy via a search-results journey + * (https://getmobil.com/ara/?term=...). + */ +export const GetmobilSearchSelectors = { + // Header search box. Resolves to 2 nodes (desktop + mobile header) -> use .first(). + // Verified selector from fresh selector map: [data-test-id="selenium-header-search-input"] + searchInput: '[data-test-id="selenium-header-search-input"]', + // Page-ready marker — good waitFor hook before asserting. + pageLoaded: '[data-test-id="selenium_page_loaded"]', + // The site submits search on Enter (no submit button has a test-id) — see SearchPage.submitByEnter(). + // Results page (/ara/?term=...): grid container + per-card links. + resultsGrid: '[data-testid="product-list-grid"]', + productCard: 'a[data-testid^="product-grid-card-"]', + // Results URL path — the site submits to /ara/?term=; used to wait for navigation. + resultsUrl: /\/ara\//, + // Results heading / breadcrumb copy: `"" için sonuçlar` (has-results) — the page + // does NOT keep the typed term in the search box, so assert the term against this instead. + resultsHeadingText: 'için sonuçlar', + // No-results empty state has no test-id — matched by its Turkish copy: + // `"" için sonuç bulunamadı.` + noResultsText: 'sonuç bulunamadı' +} as const; diff --git a/src/pages/selectors/toolshop.selectors.ts b/src/pages/selectors/toolshop.selectors.ts deleted file mode 100644 index c75a94e..0000000 --- a/src/pages/selectors/toolshop.selectors.ts +++ /dev/null @@ -1,18 +0,0 @@ -/** - * Selector strings for the Practice Software Testing ("Toolshop") app. - * Verified against the live DOM via `ai:inspect`. - */ -export const ToolshopLoginSelectors = { - email: '[data-test="email"]', - password: '[data-test="password"]', - submit: '[data-test="login-submit"]', - error: '[data-test="login-error"]', - navMenu: '[data-test="nav-menu"]' -} as const; - -export const ToolshopProductSelectors = { - searchQuery: '[data-test="search-query"]', - searchSubmit: '[data-test="search-submit"]', - searchReset: '[data-test="search-reset"]', - productName: '[data-test="product-name"]' -} as const; diff --git a/src/steps/product-search.steps.ts b/src/steps/product-search.steps.ts new file mode 100644 index 0000000..62a62c4 --- /dev/null +++ b/src/steps/product-search.steps.ts @@ -0,0 +1,68 @@ +import { expect } from '@playwright/test'; +import { Given, When, Then } from './common'; + +Given('the visitor is on the Getmobil home page', async ({ searchPage }) => { + await searchPage.open(); +}); + +Given('the header search box is visible and empty', async ({ searchPage }) => { + await expect(searchPage.searchInput).toBeVisible(); + await expect(searchPage.searchInput).toHaveValue(''); +}); + +When('the visitor types {string} into the header search box', async ({ searchPage }, query: string) => { + await searchPage.typeQuery(query); +}); + +When('the visitor submits the search', async ({ searchPage }) => { + await searchPage.submitByEnter(); +}); + +Then('the page URL contains the search query parameter for {string}', async ({ page }, query: string) => { + await expect(page).toHaveURL(new RegExp(encodeURIComponent(query).replace(/%20/g, '(\\+|%20)'))); +}); + +Then('at least one product card is displayed in the results', async ({ searchPage }) => { + await expect(searchPage.productCards.first()).toBeVisible(); + const count = await searchPage.productCards.count(); + expect(count).toBeGreaterThan(0); +}); + +Then('a product card mentioning {string} is visible in the results', async ({ searchPage }, term: string) => { + const cards = searchPage.productCards; + const count = await cards.count(); + let found = false; + for (let i = 0; i < count; i++) { + const text = await cards.nth(i).innerText(); + if (text.toLowerCase().includes(term.toLowerCase())) { + found = true; + break; + } + } + expect(found, `Expected at least one product card to mention "${term}"`).toBe(true); +}); + +Then('the results page reflects the search term {string}', async ({ searchPage }, term: string) => { + await expect(searchPage.resultsHeading).toContainText(term); +}); + +Then('no empty-state message is shown', async ({ searchPage }) => { + await expect(searchPage.noResultsMessage).not.toBeVisible(); +}); + +Then('the results page loads without a server error', async ({ page }) => { + // A server error would land on an error route; the results URL (/ara/) is not one. + // NOTE: do NOT substring-match "500"/"404" in the page body — product prices (e.g. ₺2.500) + // contain those digits and cause false failures. + expect(page.url()).not.toMatch(/\/(error|not-found|maintenance)\b|[?&](error|status)=5\d\d/i); +}); + +Then('the page URL correctly encodes the Turkish characters for {string}', async ({ page }, query: string) => { + const currentUrl = page.url(); + // The URL should not contain raw unencoded Turkish characters as-is; + // percent-encoding of at least one special character confirms encoding happened. + const encoded = encodeURIComponent(query); + // Accept either percent-encoded or the browser's normalised form + const hasEncoded = currentUrl.includes(encoded) || currentUrl.includes(query); + expect(hasEncoded, `Expected URL to reflect the search term "${query}". Actual URL: ${currentUrl}`).toBe(true); +}); diff --git a/src/steps/toolshop-login.steps.ts b/src/steps/toolshop-login.steps.ts deleted file mode 100644 index 441f52a..0000000 --- a/src/steps/toolshop-login.steps.ts +++ /dev/null @@ -1,27 +0,0 @@ -import { expect } from '@playwright/test'; -import { Given, When, Then } from './common'; -import { getUser } from '../fixtures/data'; - -Given('the user is on the login page', async ({ loginPage }) => { - await loginPage.open(); -}); - -When('the user logs in as the {string} user', async ({ loginPage }, userKey: string) => { - const user = getUser(userKey); - await loginPage.login(user.username, user.password); -}); - -When( - 'the user logs in with email {string} and password {string}', - async ({ loginPage }, email: string, password: string) => { - await loginPage.login(email, password); - } -); - -Then('the account menu should be displayed', async ({ loginPage }) => { - await expect(loginPage.navMenu).toBeVisible(); -}); - -Then('a login error should be displayed', async ({ loginPage }) => { - await expect(loginPage.errorMessage).toBeVisible(); -}); diff --git a/src/steps/toolshop-search.steps.ts b/src/steps/toolshop-search.steps.ts deleted file mode 100644 index bc93dae..0000000 --- a/src/steps/toolshop-search.steps.ts +++ /dev/null @@ -1,24 +0,0 @@ -import { expect } from '@playwright/test'; -import { Given, When, Then } from './common'; - -Given('the user is on the products page', async ({ productsPage }) => { - await productsPage.open(); -}); - -When('the user searches for {string}', async ({ productsPage }, term: string) => { - await productsPage.search(term); -}); - -Then('at least one product is shown', async ({ productsPage }) => { - await expect(productsPage.productNames.first()).toBeVisible(); -}); - -Then('the results include {string}', async ({ productsPage }, name: string) => { - // Toolshop search matches names AND descriptions, so we assert the expected product - // is present rather than that every result's name contains the term. - await expect(productsPage.productNames.filter({ hasText: name }).first()).toBeVisible(); -}); - -Then('no products are shown', async ({ productsPage }) => { - await expect(productsPage.productNames).toHaveCount(0); -}); diff --git a/stories/getmobil-search.txt b/stories/getmobil-search.txt new file mode 100644 index 0000000..14b86fe --- /dev/null +++ b/stories/getmobil-search.txt @@ -0,0 +1,19 @@ +Title: Product search on Getmobil + +As a visitor to Getmobil +I want to search for a product by its name +So that I can quickly find the device I am looking for + +Acceptance Criteria: +1. Typing a product name (e.g. "iPhone 13") into the header search box and submitting + takes the user to a results view that lists matching products. +2. The results relate to the searched term (a result mentioning the searched model is shown). +3. Searching for a term that has no matches shows a clear empty-state / "no results" message + and lists no products. + +Technical notes (application under test: https://getmobil.com): +- The site UI is in Turkish. The search box lives in the site header (a text input plus a + submit button); use the verified selectors from the inspected selector map for them. +- Navigate relative to the base URL (the home page is "/"). +- Use real, resilient selectors from the selector map; only fall back to a clearly marked + placeholder + TODO if an element (e.g. a results-page node) is not in the map. diff --git a/stories/login.txt b/stories/login.txt deleted file mode 100644 index 7f8f114..0000000 --- a/stories/login.txt +++ /dev/null @@ -1,19 +0,0 @@ -Title: Customer login to the Toolshop - -As a registered customer -I want to log in to the Toolshop -So that I can access my account and place orders - -Acceptance Criteria: -1. Logging in with valid credentials lands the user on their account, and the account - menu (data-test="nav-menu") is shown. -2. Logging in with a wrong password shows a login error (data-test="login-error"). -3. The email and password fields are required. - -Technical notes (application under test: https://practicesoftwaretesting.com): -- Login page is at /auth/login. Reuse the existing step "the user is on the login page". -- Existing ToolshopLoginPage exposes: open(), login(email, password); locators emailInput - ([data-test="email"]), passwordInput ([data-test="password"]), submitButton - ([data-test="login-submit"]), errorMessage ([data-test="login-error"]), navMenu - ([data-test="nav-menu"]). -- Existing fixture user: customer (fixtures/users.json).