feat(batch): implement batch-runner MVP per PRP-33 (#280)

[w7-mgfcode]

Summary

Implements the Batch Runner MVP per PRPs/PRP-33-batch-runner-mvp.md. One batch_job row fans out into N batch_job_item rows; each item runs sequentially via lazy delegation to JobService.create_job; per-item metrics land in the pinned five-key JSONB {wape, smape, mae, bias, sample_size}; the parent settles to completed | failed | partial.

Unblocks PRP-34 (parallel execution), PRP-35 (priority queue), PRP-36 (export and retry), PRP-37 (champion and heatmap) — the forward-compat columns + the partial picker index ship MVP-owned per PRP-33 § "Cross-Slice Coordination Matrix".

Closes #280. Refs #277.

What changed

• Migration (alembic/versions/c1d2e3f40512_create_batch_tables.py): batch_job + batch_job_item tables, enums (BatchStatus, BatchOperation, BatchItemStatus), CHECK constraints (status / operation / priority band), FK CASCADE on batch_id, partial picker index with predicate exactly WHERE (status = 'pending'), plus 11 b-tree + 1 GIN index.
• Models (app/features/batch/models.py): SQLAlchemy 2.0 Mapped[] columns mirroring the migration; valid-transition maps for parent + child.
• Schemas (app/features/batch/schemas.py): Pydantic v2 with ConfigDict(strict=True) on request bodies + Field(strict=False) on the two date fields (the strict-mode policy linter at app/core/tests/test_strict_mode_policy.py discovers and approves them). BatchScope.kind and BatchSubmitRequest.operation use Literal[...] (not str enums) for strict-mode compatibility; BatchScopeKind is retained for response/internal callers.
• Service (app/features/batch/service.py): submit / _expand_scope (handles manual, region, category, top_revenue, all) / _pick_next (compiles to FOR UPDATE SKIP LOCKED) / _execute_item (lazy-imports JobService per the cycle precedent at forecasting/service.py:786-787) / _shape_metrics (sample_size derived inside the batch slice from fold_metrics per PRP § "Why not 10" — Option (b) explicitly rejected as a vertical-slice violation) / _settle / get / list_items. structlog lifecycle events: batch.{created,item_started,item_completed,item_failed,completed}.
• Routes (app/features/batch/routes.py): POST /batch/forecasting (202), GET /batch/{id}, GET /batch/{id}/items (allow-listed sort_by); 404s and the scope-cap 422 go through app/core/exceptions to RFC 7807 application/problem+json. Router wired into app/main.py between jobs_router and ingest_router.
• Config: Settings.batch_max_scope_expansion = 1000 + matching .env.example row.
• Frontend stub at /visualize/batch: types in frontend/src/types/api.ts, hooks in frontend/src/hooks/use-batches.ts (useSubmitBatch, useBatch with 2s poll on in-flight states, useBatchItems), placeholder page in frontend/src/pages/visualize/batch.tsx. Per PRP narrowing: no slider, no cancel button, no retry, no heatmap.

PRP-33 success criteria

• alembic upgrade head on fresh Postgres creates both tables, every CHECK, and the partial picker index with predicate exactly WHERE (status = 'pending') — verified via pg_indexes query in test_migration_partial_index_present.
• 3-pair manual backtest scope settles completed with completed_items=3; every item's metrics JSONB carries exactly {wape, smape, mae, bias, sample_size} — test_submit_batch_happy_path.
• grep -rn "for_update" app/features/batch/service.py → service.py:245: .with_for_update(skip_locked=True).
• Settings.agent_require_approval unchanged (still ["create_alias", "archive_run", "save_scenario"]).
• app/features/jobs/, app/features/forecasting/, tests/test_e2e_demo.py unmodified (verified via git status).
• All validation gates green: ruff check + ruff format --check + mypy app/ + pyright app/ + pytest -v -m "not integration" (1457 pass) + pytest -v -m integration (7 batch + e2e demo passing) + frontend pnpm tsc --noEmit + pnpm lint + pnpm vitest --run (119 pass).

Test plan

• 25 backend unit tests (no DB) — models, schemas (strict-mode JSON path regression), service (pinned JSONB shape, picker SQL, manual cartesian, frozen params).
• 7 backend integration tests (real Postgres) — happy path, partial failure, scope-cap 422, sort allow-list silent fallback, partial-index predicate, lifecycle event emission.
• Non-regression: tests/test_e2e_demo.py still green.
• Reviewer dogfoods /visualize/batch against a seeded DB.

🤖 Implementation per PRP-33; manual review encouraged on the _expand_scope direct-ORM-query approach (the PRP suggested lazy AnalyticsService/DimensionsService imports; this implementation uses direct data_platform.models queries — the data-platform-shared-orm-layer memory documents this as the de-facto shared ORM layer).

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 129c6476-43d7-43bf-ae7e-482e3585f1e0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/prp-33-batch-runner-mvp

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[sourcery-ai]

Sorry @w7-mgfcode, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery