Skip to content

feat(archive): export cold monthly partitions#366

Merged
KurodaKayn merged 3 commits into
mainfrom
feat/cold-partition-export
Jun 28, 2026
Merged

feat(archive): export cold monthly partitions#366
KurodaKayn merged 3 commits into
mainfrom
feat/cold-partition-export

Conversation

@KurodaKayn

Copy link
Copy Markdown
Owner

Feature Description

  • Adds a PostgreSQL-only cold monthly partition archive path for managed event/session history tables.
  • Exports whole cold partitions as JSONL to the configured R2/S3 object storage prefix before detaching and dropping them.
  • Keeps the existing row-level archive path for partial-month retention cleanup and non-PostgreSQL runs.
  • Updates the Phase 4 database optimization checklist to mark cold partition export complete while leaving archive recovery as follow-up.

Implementation Approach

  • Introduces monthly partition archive specs for publish events, extension execution events, project/workspace activities, and remote browser sessions.
  • Uses the PostgreSQL partition catalog plus managed partition-name bounds to select only whole partitions older than the retention cutoff.
  • Deletes extension execution event claims before dropping archived extension-event partitions, and skips remote browser session partitions containing non-terminal rows.

Testing

  • cd backend && go test ./... passed.
  • cd backend && golangci-lint run passed with 0 issues.

Risks

  • The partition path detaches and drops partitions only after upload succeeds, but production rollout should still verify object-storage retention and restore procedure before enabling aggressive retention windows.

@KurodaKayn KurodaKayn force-pushed the feat/cold-partition-export branch 2 times, most recently from 8bcabae to d5885f1 Compare June 28, 2026 03:34
Cold monthly event partitions need a fast archival path once they age past retention.

Add a PostgreSQL partition archive flow that uploads JSONL to object storage before detaching and dropping the partition.

The existing row-level archive remains as the fallback for partial months and non-PostgreSQL runs.
Cold partition export relies on deterministic partition names and object keys.

Add archive helper tests for monthly partition bounds, archive object keys, and identifier quoting.

The helper coverage keeps the destructive partition path easier to review without a PostgreSQL fixture.
The Phase 4 checklist should reflect the new cold partition export path.

Mark R2/S3 cold partition export complete and point the progress matrix at the archive worker verification files.

Archive recovery remains the remaining Phase 4 follow-up.
@KurodaKayn KurodaKayn force-pushed the feat/cold-partition-export branch from d5885f1 to a4e3b67 Compare June 28, 2026 08:01
@KurodaKayn KurodaKayn merged commit 747876e into main Jun 28, 2026
14 checks passed
@KurodaKayn KurodaKayn deleted the feat/cold-partition-export branch June 28, 2026 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant