Skip to content

[DMS-1255] Publish and restore MSSQL database template packages (Minimal/Populated) at parity with PostgreSQL#1100

Open
simpat-jesus wants to merge 1 commit into
mainfrom
DMS-1255
Open

[DMS-1255] Publish and restore MSSQL database template packages (Minimal/Populated) at parity with PostgreSQL#1100
simpat-jesus wants to merge 1 commit into
mainfrom
DMS-1255

Conversation

@simpat-jesus

@simpat-jesus simpat-jesus commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Extends the PostgreSQL-only database-template pipeline to MSSQL — CI can now build, restore-verify, and publish EdFi.Api.{Minimal,Populated}.Template.MsSql.{5.2.0,6.1.0} packages containing a native BACKUP DATABASE .bak — and homologates the local deployment topology: CMS and DMS now share one physical database on both engines (previously PostgreSQL-only behavior). Consuming the published template packages from the bootstrap flows (a wrapper restore phase) is deliberately not in this PR; that is DMS-1271, which builds on the -DbOnly start-script slice introduced here.

Changes

  • Template build/verify tooling: engine dispatch in Template-Management.psm1 (BACKUP DATABASE / RESTORE ... WITH MOVE via sqlcmd against dms-mssql, transient in-container backups cleaned up; PG pg_dump/psql path unchanged); multi-file .bak support (one MOVE per RESTORE FILELISTONLY row); pg_terminate_backend before the PG verify-restore DROP; template settings parameterized by engine token + artifact extension; verify-template-restore.ps1 engine dispatch with T-SQL structural asserts and the serveability probe.
  • CI: build-{minimal,populated}-template.yml take a database_engine input (engine-dispatched credentials, registration, capture, verify; engine-dependent package-name validation; smoke templates stay PG-only); input-validation parity on the minimal workflow; post-load content gates (populated: edfi.School > 0; minimal: descriptors > 0) so an empty load can never be packaged or published; new EdFi.Api.{Minimal,Populated}.Template.MsSql.yml callers mirror the PG callers; provision-e2e-database.ps1 gains the MSSQL lane the template workflows use.
  • Shared-database topology: .env.mssql points the Configuration Service at Database=${MSSQL_DB_NAME} (was a dedicated edfi_configurationservice database), matching PostgreSQL's existing shared default; OpenIddict identity init targets the shared database (ENV:MSSQL_DB_NAME); the contradictory compose fallbacks in local-config.yml/published-config.yml (which named edfi_configurationservice even for PG) now interpolate ${POSTGRES_DB_NAME}. An opt-in separate-CMS-database mode is DMS-1270.
  • Engine parity on the published flow: bootstrap-published-dms.ps1/start-published-dms.ps1 accept -DatabaseEngine (mssql.yml swap, readiness wait, engine-aware OpenIddict init); the wrapper forwards the engine to both start scripts; start-published-dms.ps1's transitional direct-SQL -LoadSeedData switch is removed (wrapper-level API seeding is the seed path); build-dms.ps1 StartEnvironment forwards -DatabaseEngine/-DataStandardVersion (now ValidateSet-guarded).
  • -DbOnly phase slice on both start scripts: database container + readiness wait only (PG pg_isready poll, MSSQL Wait-MssqlReady), mutually exclusive with -InfraOnly/-DmsOnly. Introduced as the primitive the template-restore bootstrap branch (DMS-1271) sequences around.
  • Bootstrap staging hardening: the wrapper revalidates a staged .bootstrap/ workspace against the effective SCHEMA_PACKAGES and re-stages on mismatch — previously a stale workspace was silently reused across Data Standard switches, serving one DS's content under another's label; load-dms-seed-data.ps1 derives its sample-data ref tag from the selected Data Standard instead of hardcoding v5.2.0.
  • Env wiring: composing the .env.mssql overlay rewrites DATABASE_TEMPLATE_PACKAGE's engine token (PostgreSqlMsSql) via the new Convert-TemplatePackageToken, preserving DS-version/template variance.
  • Docs: MSSQL template/CI pipeline and shared-topology documentation in eng/docker-compose/README.md; command-boundaries.md documents the -DbOnly slice and the published-flow engine contract; stale claims about the removed published -LoadSeedData flag corrected in both design docs.

Testing

  • Pester: eng/docker-compose/tests 615 passed / 0 failed (new specs: -DbOnly parameter surface + mutual exclusions on both start scripts, unconditional engine forwarding in the wrapper, workspace-revalidation matrix, seed ref-tag mapping, build-dms.ps1 forwarding, shared-topology connection-string pins); eng/DatabaseTemplates/tests 42 passed (dump/restore command contracts pinned to exact argument sequences incl. the multi-file .bak fixture, package-identity matrix).
  • Live local validation on DS 5.2, three -LoadSeedData legs on the shared topology: MSSQL Minimal (seed exit 0; 19 descriptors served through a real token), PostgreSQL Minimal regression (seed exit 0; 19 descriptors), and MSSQL Populated (bootstrap phases and serving all healthy — 19 descriptors + 3 schools served; the BulkLoadClient itself exited nonzero under a pre-existing local rate-limiter storm unrelated to this change, reported separately as a follow-up). Topology verified on both engines: sys.databases contains no edfi_configurationservice, and the shared DMS database holds the 24 dmscs CMS tables (identical table set on both engines).
  • Template package building was previously validated end to end on this branch's donor across the full 8-scenario matrix (engine × Minimal/Populated × DS 5.2/6.1), including restore verification and effective-schema-hash startup validation; the tooling files are unchanged from that validated state.
  • PSScriptAnalyzer clean on changed files (enforced by the pre-commit hook); all four workflow files lint/parse.

Notes for Reviewers

  • Scope split: this PR was reduced after DMS-1270/DMS-1271 were opened. Consumption of the published templates (wrapper restore branch, restore-before-CMS ordering) is DMS-1271; the opt-in separate-CMS-database mode is DMS-1270. A complete working implementation of the restore flow is preserved on the DMS-1255-restore-donor branch for DMS-1271 to draw from.
  • Rollout order: the Azure Artifacts feed has no packages under the EdFi.Api.*.Template.MsSql.* ids yet. Suggested CI validation post-merge: dispatch the MsSql callers with publish_package=false first, then re-dispatch a PostgreSQL caller to prove the now-shared reusable workflows regressed nothing.
  • Topology flip: MSSQL local stacks previously created a dedicated CMS database; fresh volumes are recommended when switching branches across this change (start-local-dms.ps1 -DatabaseEngine mssql -d -v). E2E env files keep their own explicit topology and are untouched.
  • Removed flag: start-published-dms.ps1 no longer accepts -LoadSeedData (it silently meant direct-SQL template restore, unlike the wrapper flag of the same name); wrapper-level -LoadSeedData API seeding is the supported path.
  • Artifact coupling: the .bak couples to the pinned mssql/server:2022-latest image (documented at the database_engine workflow input), as the PG .sql dump couples to the PG major version.

🤖 Generated with Claude Code

@simpat-jesus simpat-jesus requested a review from a team as a code owner July 8, 2026 05:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant