CI: build/it-postgres run twice on every push to an open PR

[iuliag]

Symptom

build and it-postgres (and, until adobe/spacecat-api-service#2903, type-check) run twice per push whenever the push lands on a branch that already has an open, same-repo PR — once under the push event, once under pull_request: synchronize, for the identical commit. Both show up as separate, independently-"Required" checks in the PR UI.

Root cause

Every consumer of service-ci.yaml triggers on both an unrestricted push and pull_request (opened, synchronize, ready_for_review, reopened, edited). When a branch has an open PR, a push fires both events for the same SHA → two workflow runs. Jobs with no event-name guard (build, it-postgres) execute in both; jobs already guarded with if: github.event_name == 'pull_request' (protect-nyc-config, validate-pr-title) do not duplicate.

This isn't accidental: the unrestricted push trigger is what drives branch-deploy (docs/vpc-config.md:19 — "push to any non-main branch"), which intentionally deploys feature branches to DEV independent of whether a PR exists yet. build/it-postgres are needs: of that job (and of deploy-stage/semantic-release on main), so they can't simply be skipped on push the way the PR-only gates were.

Scope

12 repos currently consume service-ci.yaml, all with the unrestricted push trigger (so all pay the build duplication whenever they push to an open PR): mysticat-projector-service, spacecat-api-service, spacecat-audit-worker, spacecat-auth-service, spacecat-autofix-worker, spacecat-content-processor, spacecat-fulfillment-worker, spacecat-import-job-manager, spacecat-import-worker, spacecat-jobs-dispatcher, spacecat-reporting-worker, spacecat-scrape-job-manager.

Only spacecat-api-service currently opts into it-postgres: true, but any future consumer that opts in inherits the same doubling.

Impact

• Wasted compute, not wasted wall-clock. The two runs start ~simultaneously and execute in parallel, so time-to-"all-checks-green" is roughly bounded by the slower run, not doubled — unless org-level concurrent-runner-slot limits cause one run to queue behind the other, which would serialize them. I don't have queue-time telemetry to confirm this is actually happening at current PR volume; worth checking the Actions usage dashboard before assuming it's a latency problem and not just a cost one.
• 2x compute for zero additional signal, every time. build and it-postgres are each documented as ~5-7 min (service-ci.yaml:198-200 comment) — same commit, same result, run twice. Multiplied across every push-while-PR-open, across up to 12 repos, this is a real, quantifiable but currently untracked chunk of Actions minutes; I'd want the billing/usage dashboard numbers before putting a dollar figure on it rather than guessing.
• Doubled flake exposure on the required path. it-postgres has a documented history of intermittent 15-minute timeouts from module-scope keep-alive socket pools pinning the event loop (service-ci.yaml:226-232). Since the screenshot shows both the push- and pull_request-triggered instances marked "Required," a flake in either blocks merge on a commit that is otherwise fine — running it twice roughly doubles the chance of hitting that failure mode per push.
• Ambiguous for anything monitoring merge-readiness programmatically. The two duplicate checks share the same job name in the UI/API, distinguished only by which workflow run/event produced them — exactly the shape that trips up a script or agent polling gh pr checks / the GraphQL statusCheckRollup and matching by check name. It has to correctly treat them as two distinct required entries rather than one superseding the other, or it can report "ready to merge" off one green instance while the duplicate is still pending or red.

Already fixed

adobe/spacecat-api-service#2903 guards type-check (a repo-local job with zero downstream consumers) to pull_request-only — no trade-off, since nothing needs it to also run on push.

Options for build/it-postgres (the harder half — every option here trades something)

1. Concurrency-cancel: key the concurrency.group on branch name instead of ref so the push- and pull_request-triggered runs share a group and the older one gets cancelled. Cheap, but the loser reports cancelled, which branch protection treats as failing — risks flaky, seemingly-random merge blocks, and can kill the push-triggered run before branch-deploy executes.
2. Skip on push when a PR is already open: add a preflight job that checks gh pr list --head <branch> --state open and gates build/it-postgres off on push when true. Breaks branch-deploy for the common case of pushing while a PR is open (the majority of feature-branch pushes), since needs: treats a skipped dependency as unmet — directly contradicts the documented "push to any non-main branch deploys" guarantee.
3. Restrict push to main only, and move branch-deploy to trigger off pull_request (checking out github.event.pull_request.head.sha). Removes the duplication entirely for feature-branch commits. Trade-off: branches without an open PR no longer get a DEV deploy on push — you'd have to open a PR first (or add workflow_dispatch for a manual pre-PR deploy). This directly changes the "cadence-friendly," "depends on it" guarantee in docs/vpc-config.md:23,135 for all 12 consumers.
4. Share the build/test result across the two runs (e.g. workflow_run-chained artifact reuse, or restructure so one run's success is reused by the other instead of re-executed). This is the only option that removes both the compute waste and the flake-doubling without regressing branch-deploy's push-without-PR guarantee — but it's a real redesign of service-ci.yaml, not a quick patch, and needs coordinated rollout across all 12 consumers.

Ask

Pick a direction (2, 3, or 4 above), or explicitly decide the current 2x cost is an acceptable trade-off for keeping branch-deploy simple and leave it as-is. Happy to scope whichever direction into a plan once there's a call on it.

[iuliag]

@solaris007 thoughts on this?