fix(datasync): per-table --force wipe, source identity in checkpoints, schema check for pre-existing targets

[morgo]

This PR fixes three verified safety gaps in the EXPERIMENTAL sync (datasync) runner, one commit per gap.

Gap 1: --force dropped the entire target database, including tables the sync never owned

Problem. When no resumable checkpoint existed, --force executed DROP DATABASE IF EXISTS on the target. But sync's own fresh-run model tolerates foreign tables in the target database (checkTargetEmpty only validates tables named after source tables), and move's equivalent wipe is per-table. A target database shared with, say, an unrelated reporting table meant --force destroyed it.

Fix. The wipe is now per-object, mirroring move's wipeTargets: only the target copies of the source tables plus the sync checkpoint table are dropped, and the database is still ensured to exist (CREATE DATABASE IF NOT EXISTS). The decision moved from ensureTargetDatabase (pre-open, no-database admin connection) into setup() after source-table discovery, which also removes forceTargetResumable's extra connection dance. --force help/doc text updated.

Testing. TestSyncForcePreservesForeignTables: a target database holding a foreign table (with rows), a stale non-resumable checkpoint (present, empty watermark), and junk in a source-named table. --force recovers: the foreign table and its rows survive, the source-named tables are recreated with exactly the source's data, and the checkpoint is recreated. Pre-fix, the foreign table was destroyed. TestSyncForce now pins that a non-source table survives the force wipe.

Gap 2: checkpoints recorded no source identity — resume against a replaced source replayed a colliding file:pos silently

Problem. The checkpoint persisted a bare binlog.NNNNNN:offset, and resume only validated that a binlog file with that name exists (pkg/change/binlog.go). Binlog file names are sequential on every server, so after an Aurora/RDS failover behind a stable endpoint, a replica promotion, or a rebuilt source, StartFromPosition succeeded on a different server's file and silently skipped or mis-replayed events. GTID mode is immune; the default file:pos mode was not.

Fix. The position is now stored wrapped in a small versioned JSON payload inside datasync's own Position field (the shared pkg/checkpoint schema is unchanged): {"v":1,"position":...,"server_uuid":...,"source_addr":...}. @@server_uuid is the identity (it changes across failover/promotion/rebuild); the address is diagnostics only. On resume with the built-in file:pos client, a server_uuid mismatch is a hard error pointing at --force (fresh sync) or --gtid (failover-safe). GTID and injected change.Source positions are unwrapped without verification, and the payload decodes strictly (unknown fields rejected + version check) so an injected source's opaque position can never be misread as ours. hasResumableCheckpoint applies the same gate, so --force wipes and starts fresh instead of resuming into the same error.

Breaking change (deliberate, experimental feature): old checkpoints without identity are unverifiable, so a file:pos resume now refuses them with a clear error unless --force is given (or --gtid is used).

Testing. TestSyncResumeSourceIdentity: a real run records the payload (asserted to carry the source's actual @@server_uuid and a non-empty position); a direct SQL UPDATE swaps in a wrong server_uuid (inner position untouched and still valid on this server — pre-fix this resume succeeded silently) and resume hard-errors; restoring the correct uuid resumes normally with data intact; rewriting the position to the legacy bare form errors with the unverifiable-checkpoint message; --force then recovers with a fresh copy. TestSyncPositionEncodeDecode unit-tests the codec (round-trip, legacy/foreign-JSON/wrong-version fallbacks).

Gap 3: pre-existing empty target tables were accepted with zero schema comparison

Problem. createTargetTables skips tables that already exist on the target, and the fresh-run gate (checkTargetEmpty) validated only emptiness. move closed exactly this hole with schemaDiff (pkg/move/check/target_state.go), whose comment names the danger: a primary-key collation difference makes REPLACE/INSERT IGNORE silently collapse case-distinct rows, and the continuous checksum then never converges.

Fix. move's schemaDiff is lifted into the shared pkg/statement as statement.SchemaDiff (it is pure CREATE TABLE canonicalization/diffing — that package's domain); move's three call sites (target_state, resume_state, source_schema_consistency) now call it there. datasync's fresh-run path compares SHOW CREATE TABLE of every pre-existing empty target table against its source table via the same function and errors — with a runnable ALTER in the message — before any copy starts. Identical pre-created tables (the declarative pre-created-schema workflow) still pass.

Testing. TestSyncFreshTargetSchemaMismatch: a pre-created empty target table with a different column collation is rejected before any row is copied (asserts zero rows and no checkpoint table were produced); an identical pre-created table passes and the copy proceeds. TestSchemaDiff* unit tests moved to pkg/statement with a new collation-mismatch case; move's TestTargetStateCheck*, TestResumeStateCheck* and TestSourceSchemaConsistencyCheck still pass against the relocated function.

Test runs

All against MySQL 8.0.45 (GTID enabled), targeted per the repo's shared-MySQL conventions:

• ./pkg/datasync/ — full package test list (E2E, initial copy, copy-only, status task, resume ×4, force ×2, identity ×2, schema check, legacy default, defer-secondary-indexes ×2, continuous checksum ×2, validate): all pass
• ./pkg/datasync/ with -race on the three new integration tests + codec test: pass
• ./pkg/statement/, ./pkg/move/check/ (target_state/resume_state/source_schema_consistency): pass
• golangci-lint run (uncapped) on pkg/datasync, pkg/statement, pkg/move: 0 issues

🤖 Generated with Claude Code