Skip to content

Move setLastReconciledSpec to after successful transition start#1076

Closed
Yanisdje wants to merge 10 commits into
apache:mainfrom
Shopify:03-20-last_reconciled_bug
Closed

Move setLastReconciledSpec to after successful transition start#1076
Yanisdje wants to merge 10 commits into
apache:mainfrom
Shopify:03-20-last_reconciled_bug

Conversation

@Yanisdje

Copy link
Copy Markdown

This fixes a bug where the last reconciled spec on the flink b/g deployment shows an updated spec even when the update failed.

Summary

  • Move setLastReconciledSpec() to after startTransition() and startSavepointRedeployTransition() succeed, so it's not stamped if the transition fails
  • Previously, lastReconciledSpec was set before the transition call — if the transition failed or was later aborted, lastReconciledSpec reflected the new spec while the active child still ran the old one. Subsequent deploys would see no diff and fall into PATCH_CHILD (in-place upgrade) instead of a proper B/G transition
  • Add assertion in verifySavepointFetchFailureRecovery confirming lastReconciledSpec does not contain the failed spec change

Context

Slack thread: https://shopify.enterprise.slack.com/archives/C0ALAR1TLP8

Test plan

  • verifySavepointFetchFailureRecovery passes with new assertion
  • Full B/G controller test suite passes (72 tests)

Fixes Shopify/streaming-compute#882

@Yanisdje Yanisdje changed the title 03 20 last reconciled bug Move setLastReconciledSpec to after successful transition start Mar 20, 2026
@Yanisdje
Yanisdje marked this pull request as draft March 20, 2026 18:05
@gyfora

gyfora commented Apr 2, 2026

Copy link
Copy Markdown
Contributor

How do these 2 PRs relate to each other? Should we close one?
#1072
#1076

@Yanisdje Yanisdje closed this Apr 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants