feat: schedule ingestion engine#31
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Pull request overview
This PR replaces the legacy client-side CSV schedule import with a server-driven ingestion workflow backed by Supabase Edge Functions and a transactional Postgres RPC, plus a new admin import wizard UI (upload → diff/conflicts → commit).
Changes:
- Added
diff-scheduleandcommit-scheduleEdge Functions, plus acommit_schedulePostgres RPC to perform atomic schedule writes. - Implemented a new admin schedule import wizard UI with stage-mismatch and orphan-set resolution flows.
- Added unit tests for diffing logic and integration tests for the commit RPC.
Reviewed changes
Copilot reviewed 25 out of 26 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| vite.config.ts | Adds test runner config exclusions. |
| supabase/migrations/20260509142022_commit_schedule_rpc.sql | Adds constraints and the transactional commit_schedule RPC used by ingestion. |
| supabase/functions/_shared/auth.ts | Shared admin auth + CORS helpers for Edge Functions. |
| supabase/functions/diff-schedule/index.ts | Edge Function endpoint to compute a diff from CSV rows vs DB. |
| supabase/functions/diff-schedule/diff.ts | Core diff/matching logic (artists, stages, sets, orphan detection). |
| supabase/functions/diff-schedule/diff.test.ts | Unit tests covering slugging, time conversion, matching rules, and conflicts. |
| supabase/functions/commit-schedule/index.ts | Edge Function endpoint that calls the commit_schedule RPC. |
| supabase/functions/commit-schedule/commit-schedule.test.ts | Integration tests targeting the RPC behavior against local Supabase. |
| src/services/scheduleImportService.ts | Frontend service layer for parsing CSV + invoking diff/commit + building commit payloads. |
| src/pages/admin/FestivalScheduleImport.tsx | New admin page wrapper for the import wizard route. |
| src/pages/admin/FestivalEdition.tsx | Adds an “Import” tab and routing to the new import page. |
| src/components/router/GlobalRoutes.tsx | Wires the /import sub-route under festival edition admin routes. |
| src/components/Admin/ScheduleImport/ScheduleImportWizard.tsx | Wizard state machine: upload → review → commit result, plus cache invalidation. |
| src/components/Admin/ScheduleImport/CsvUploadStep.tsx | CSV upload + timezone selection + invokes diff. |
| src/components/Admin/ScheduleImport/DiffReviewStep.tsx | Review UI container including conflicts and commit action. |
| src/components/Admin/ScheduleImport/DiffSummaryBanner.tsx | Summary banner for diff results. |
| src/components/Admin/ScheduleImport/StageMismatchResolver.tsx | UI to map mismatched stage names or create new stages. |
| src/components/Admin/ScheduleImport/OrphanedSetsPanel.tsx | UI to archive/keep orphaned sets not present in CSV. |
| src/components/Admin/ScheduleImport/CommitResultCard.tsx | Success UI and “import another file” reset action. |
chiptus
force-pushed
the
feat/schedule-ingestion
branch
from
4f1b288 to
c8110dd
Compare
|
✅ DB Migrate succeeded for |
chiptus
commented
May 9, 2026
Postgres grants EXECUTE on new functions to PUBLIC by default, which would let any authenticated PostgREST client call commit_schedule directly and bypass the commit-schedule Edge Function's admin-only gate. Revoke EXECUTE from PUBLIC and grant it only to service_role for the RPC and its helpers. https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
A B2B cell like "Carl Cox | Carl Cox" produced a duplicated artist list, which changes the diff's roster key (breaking matches against existing sets) and sends duplicate slugs downstream. Normalize each row's artist list to a case-insensitive unique set. https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
The schedule-ingestion branch added an artists.slug dedupe + constraint re-add step to sync-from-prod.sh. Drop it — that belongs in a migration, and re-adding the constraint unconditionally can abort the sync. https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
chiptus
commented
May 20, 2026
commit_schedule__upsert_stages previously suffixed every imported stage slug with a uuid chunk to dodge the (edition, slug) unique constraint. Per review, a stage matching an existing one by name OR slug should be treated as the same stage: unarchive it instead of creating a duplicate. Replaced the single ON CONFLICT upsert with a per-row match-or-insert loop, so the slug stays clean (slugify(name), no suffix). https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
chiptus
commented
May 20, 2026
… (name) The per-row name-or-slug loop guarded a path that can't occur: slugify and the diff's strip() both collapse non-alphanumerics, so any two names that would collide on (edition, slug) also strip-collide and are flagged by the diff as a mismatch -- they never reach upsert_stages as a plain new stage. Back to a single ON CONFLICT (festival_edition_id, name) upsert with a plain slugify(name) slug. https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
commit_schedule had the set update, set create and orphan archive logic inline. Pull each into its own commit_schedule__ helper that returns its row count, matching the upsert_artists/upsert_stages pattern, so the RPC body reads as the workflow. Helpers run in the same explicit order (update, create, archive) and are revoked from PUBLIC like the rest. https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
Add .order("time_start", { nullsFirst: false }).order("id") to the sets
query so the available[0] fallback in findMatchingSet always picks the
same row for the same input regardless of Postgres storage order.
https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
- commit_schedule__parse_ts now treats '' / whitespace as NULL so a malformed caller can't abort the transaction with a cast error - types:generate scripts run under bash -o pipefail so a failed supabase gen types no longer writes empty files with exit 0 - CsvUploadStep resets readFileMutation on file selection so the UI can't show a new filename with the previous file's row count https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
chiptus
commented
May 21, 2026
Replace the two inline types:generate scripts with a single gen-types.sh that takes an optional --local flag. The script runs under set -euo pipefail so a failed generation aborts instead of writing empty files. types:generate:local is dropped — run pnpm types:generate --local instead. https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
Wrap each jsonb_array_elements(p_x) call in COALESCE(p_x, '[]'::jsonb) so a direct RPC caller passing NULL for an array param gets a no-op instead of aborting the whole transaction. https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
- artists/stages dedupe migrations order the window by archived ASC so an active row keeps its slug/name and an archived duplicate is the one rewritten, instead of breaking links to the visible record - csvRowSchema trims artist names, rejects empty-after-trim, and case-insensitively de-duplicates so a direct Edge caller can't skew the diff's roster key https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 80 out of 87 changed files in this pull request and generated 3 comments.
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
Comments suppressed due to low confidence (1)
.github/workflows/_db_migrate.yml:41
- supabase/setup-cli was switched to
@v2without pinning a specific CLI version. This can make migrations brittle/non-reproducible if the Supabase CLI changes behavior. Consider pinning the CLI version in the action configuration (or installing an explicit version) to avoid unexpected breakages.
- uses: actions/checkout@v4
- uses: supabase/setup-cli@v2
- name: Push migrations
env:
SUPABASE_ACCESS_TOKEN: ${{ secrets.SUPABASE_ACCESS_TOKEN }}
PROJECT_REF: ${{ inputs.target == 'prod' && vars.PROD_PROJECT_REF || vars.STAGING_PROJECT_REF }}
DB_PASSWORD: ${{ inputs.target == 'prod' && secrets.PROD_DB_PASSWORD || secrets.STAGING_DB_PASSWORD }}
run: |
supabase link --project-ref "$PROJECT_REF" --password "$DB_PASSWORD"
supabase db push --password "$DB_PASSWORD"
- api.ts routes both calls through an invokeEdgeFunction helper that detects FunctionsHttpError and reads the function's JSON error body, so validation issues / RPC messages reach the UI instead of a generic "non-2xx" string - CsvDropZone clears the file input value after selection so re-picking the same file still fires onChange https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
- update_sets wraps stage_id/time_start/time_end in COALESCE with the existing column value, so importing a CSV without Date/Time columns corrects names/rosters instead of wiping schedule metadata - sync_set_artists raises on a NULL/empty artist roster instead of silently DELETE-ing the set's links and inserting nothing https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
Keep the exported computeDiff at the top so the file reads as the pipeline first; DiffState and the createState/collectNewArtists/ applyStageResolution helpers follow it. https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
- resolveStage only substring-matches when both stripped stage names are >= 3 chars, so a short DB stage name no longer false-positives as a mismatch against unrelated CSV stages - parseScheduleCsv rejects artist/stage names with no [a-z0-9] up front with a clear message, instead of failing opaquely at commit - diff-schedule caps the request at 5000 rows to bound payload/CPU - commit_schedule__parse_ts marked STABLE (text->timestamptz depends on the session TimeZone), not IMMUTABLE https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
main already has 20260520000000_fix_festival_assets_rls_policies, so this branch's 20260509142022/23/24 migrations are out-of-order and a db push rejects them. Renumber to 20260522000000/01/02 so they apply after main's latest migration. https://claude.ai/code/session_01T7UYCNxqTRMB4HJ6pk1nEm
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the old client-side CSV import with a server-side ingestion system.
Two Supabase Edge Functions (
diff-schedule,commit-schedule) handle the diff and atomic commit via a Postgres RPC. The frontend wizard walks admins through upload → conflict resolution → commit.Key design decisions: sets are matched by artist roster — with stage and date used only as tiebreakers when a roster has multiple candidate sets — preserving votes; orphaned sets surfaced as explicit archive/keep conflicts; stage name mismatches resolved via map-to-existing or create-new; all writes wrapped in a single transaction with full rollback on failure.