Skip to content

docs: add concurrent data migration procedure guide#2421

Open
ctron wants to merge 1 commit into
guacsec:mainfrom
ctron:docs/concurrent-data-migration
Open

docs: add concurrent data migration procedure guide#2421
ctron wants to merge 1 commit into
guacsec:mainfrom
ctron:docs/concurrent-data-migration

Conversation

@ctron

@ctron ctron commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add a step-by-step procedure for running data migrations with concurrency on a pre-existing database copy
  • Cover environment variables, single-runner and multi-runner examples, and the full skip-then-migrate workflow
  • Include a complete Kubernetes Job manifest (secrets, indexed parallel data migration jobs, schema migration job with MIGRATION_DATA_SKIP)
  • Add navigation entries for data.adoc and the new guide in nav.adoc

Test plan

  • Review rendered AsciiDoc for correctness
  • Verify env var names match migration/src/data/mod.rs (Options struct)
  • Verify CLI invocations match trustd/src/db.rs
  • Verify migration names match migration/src/lib.rs

🤖 Generated with Claude Code

Summary by Sourcery

Add documentation for running concurrent data migrations and link it into the migration docs navigation.

Documentation:

  • Introduce a detailed guide for concurrent data migrations, including environment configuration, workflows, and Kubernetes examples.
  • Add navigation entries to expose the concurrent data migration guide and related data migration docs in the migration book.

@sourcery-ai

sourcery-ai Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Adds a new AsciiDoc guide describing how to run concurrent data migrations against a copied production database, and wires it into the existing migration docs navigation.

File-Level Changes

Change Details Files
Introduce a detailed concurrent data migration procedure guide in the migration docs.
  • Create a new AsciiDoc page explaining concurrent data migration on a pre-existing database copy
  • Document required environment variables, mapping them to existing migration options
  • Provide single-runner and multi-runner CLI invocation examples aligned with current binaries
  • Describe the skip-then-migrate workflow using MIGRATION_DATA_SKIP and related flags
  • Include full Kubernetes Job manifest examples for parallel data migrations and schema migrations
docs/book/modules/migration/pages/concurrent-data-migration.adoc
Update migration documentation navigation to expose the data migration docs and the new concurrent migration guide.
  • Ensure data.adoc is linked from the migration navigation structure
  • Add an entry for the new concurrent-data-migration guide to nav.adoc
  • Position the new guide logically among existing migration docs for discoverability
docs/book/modules/migration/nav.adoc

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • In the new concurrent data migration guide, consider cross-linking more explicitly to the existing data.adoc page so readers can easily understand how the concurrent workflow builds on the baseline data migration concepts.
  • For the Kubernetes Job manifest example, it may help to clearly mark customizable fields (e.g., image name, namespace, resource requests/limits, secret names) so operators can better distinguish between required values and example placeholders.
  • Where you describe single-runner vs multi-runner concurrency, consider calling out any ordering or idempotency assumptions of individual migrations so users understand when parallel execution is safe and when it is not.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In the new concurrent data migration guide, consider cross-linking more explicitly to the existing `data.adoc` page so readers can easily understand how the concurrent workflow builds on the baseline data migration concepts.
- For the Kubernetes Job manifest example, it may help to clearly mark customizable fields (e.g., image name, namespace, resource requests/limits, secret names) so operators can better distinguish between required values and example placeholders.
- Where you describe single-runner vs multi-runner concurrency, consider calling out any ordering or idempotency assumptions of individual migrations so users understand when parallel execution is safe and when it is not.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@codecov

codecov Bot commented Jun 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.33%. Comparing base (0bb3674) to head (9fbd3c1).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2421      +/-   ##
==========================================
+ Coverage   71.30%   71.33%   +0.02%     
==========================================
  Files         450      450              
  Lines       27164    27191      +27     
  Branches    27164    27191      +27     
==========================================
+ Hits        19370    19397      +27     
+ Misses       6663     6656       -7     
- Partials     1131     1138       +7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add a step-by-step guide for running data migrations with concurrency,
including environment variables, single/multi-runner examples, and a
full Kubernetes Job manifest covering the complete workflow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ctron ctron force-pushed the docs/concurrent-data-migration branch from fad5c6d to 9fbd3c1 Compare June 29, 2026 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant