Skip to content

Migration and seed implementaion#152

Open
Gautam7352 wants to merge 32 commits into
devfrom
migration-implementaion
Open

Migration and seed implementaion#152
Gautam7352 wants to merge 32 commits into
devfrom
migration-implementaion

Conversation

@Gautam7352

@Gautam7352 Gautam7352 commented Apr 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Implements database migrations and seeding using Goose, replacing the previous stub InitSchema() approach. Migrations are now a dedicated CI/CD step, fully decoupled from the app server.

fixes #51
fixes #127
fixes #56
fixes #57
fixes #58
fixes #59


What changed

Migration runner (cmd/migrate)

  • New binary that applies all pending Goose migrations and exits
  • Intended to run as a one-off job (init container or CI/CD step) before the app server starts
  • The app server no longer has any migration logic

Seed runner (cmd/seed)

  • New binary that populates the database with deterministic development data
  • Fully idempotent — uses ON CONFLICT (id) DO NOTHING so re-runs are safe
  • Seeds two users and three todos with fixed UUIDs for stable foreign keys

Embedded migrations (db/embed.go)

  • SQL migration files are embedded at compile time via embed.FS
  • Shared between the migrate binary and tests without duplicating the embed directive
  • Replaced backend/db/init.sql with a proper Goose migration at backend/db/migrations/00001_init_schema.sql

Database package (internal/database/db.go)

  • Removed the InitSchema() stub
  • Minor cleanup: improved error messages, removed stale comments

Dockerfile

  • Added a migrate-builder and migrate stage (scratch-based, statically linked)
  • The migrate target produces a minimal binary-only image

Docker Compose (compose.yml / compose-dev.yml)

  • Added a migrate service that runs the migrate binary
  • backend now depends on migrate completing successfully (service_completed_successfully) instead of directly on the DB being healthy
  • Fixed postgres volume mount path to ./backup/data:/var/lib/postgresql/data

Makefile

  • Added make migrate and make seed targets

air.toml

  • Replaced deprecated bin/full_bin fields with entrypoint

Docs

  • Removed outdated architecture/API/schema/best-practices docs
  • Updated docs/readme.md to reflect the new migration and seed workflow

Tests

Added backend/internal/database/migrate_test.go with property-based tests using rapid and testcontainers:

  • P1 — Migration file structural invariants (naming convention, Goose annotations, CREATE TABLE IF NOT EXISTS)
  • P2 — Migration application round-trip (version recorded in goose_db_version after goose.Up)
  • P3 — Migration idempotency (running goose.Up twice doesn't change row counts)
  • P4 — App server does not migrate (Connect() leaves no goose_db_version table)
  • P5 — Seed runner idempotency (double-seeding produces identical row counts)
  • P6 — Concurrent migration safety (two goroutines migrating simultaneously result in exactly one applied version)

Tests spin up a real Postgres 17 container per test via testcontainers — no mocks.


How to run locally

# Apply migrations
make migrate

# Seed dev data
make seed

# Or via Docker Compose (migrate runs automatically before backend starts)
make dev

Comment thread backend/cmd/server/main.go Outdated
Comment thread docs/backend_schema.md Outdated
@Gautam7352 Gautam7352 changed the title Migration-implementaion Migration and seed implementaion Apr 8, 2026
@Gautam7352 Gautam7352 requested a review from TanishqSingla April 8, 2026 13:23
Comment thread backend/migration/db/versions/00001_init_schema.sql Outdated
Comment thread compose.yml Outdated
Comment thread compose.yml Outdated
Comment thread compose.yml Outdated
Comment thread backend/go.mod Outdated
Comment thread backend/go.mod Outdated
github.com/testcontainers/testcontainers-go v0.41.0
github.com/testcontainers/testcontainers-go/modules/postgres v0.41.0
golang.org/x/crypto v0.48.0
pgregory.net/rapid v1.2.0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this used for?

Comment thread backend/internal/database/migrate_test.go Outdated
Comment thread backend/internal/database/db.go Outdated
Comment thread backend/internal/database/db.go Outdated
Comment thread backend/internal/database/db.go Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants