Add opt-in SQL root-level projected columns for repository stores

[Copilot]

This splits the storage evolution into the first track only: keep the current document model and JSON-based nested semantics, while allowing SQLite/PostgreSQL repositories to switch root-level fields to real columns. The new mode is opt-in per repository or adapter via rootLevelFieldsWhenAvailable.

• What changed

  • Added rootLevelFieldsWhenAvailable?: boolean to repository/store config, with adapter-level default support in SQL storage config.
  • Kept the existing non-relational model intact.
  • When projection is enabled, projected root fields are stored in dedicated columns and are no longer duplicated in data.
  • Corrected the flag semantics so this is an exclusive mode switch:
    • false / omitted: reads, writes, and queries use data
    • true: derived root fields are read, written, and queried via dedicated root columns

• Schema-derived projected columns

  • Derive projected root-level fields from the encoded repository schema.
  • Root scalar fields use typed SQL columns when possible:
    • string
    • number
    • boolean
    • nullable forms of the above
    • encoded date-like fields that serialize to root strings
  • Complex root fields are also projected:
    • nested objects
    • arrays
    • other non-scalar encoded root fields
  • Complex projected fields use JSON / JSONB columns.

• SQLite / PostgreSQL store behavior

  • On fresh table setup, include projected root columns when the mode is enabled.
  • On writes in projected mode, persist:
    • non-projected fields into data
    • projected root fields into dedicated columns
  • On reads in projected mode, reconstruct projected fields from dedicated columns only.
  • Removed runtime fallback/backfill behavior:
    • no COALESCE(...) fallback from projected columns back to data
    • no on-the-fly backfill of projected columns from existing data
    • no runtime ALTER TABLE / migration behavior for existing rows
  • Existing schema/data migration for switching modes is expected to happen offline.

• Query planning

  • In projected mode, root-level filters, ordering, and scalar selection resolve to projected columns directly.
  • Nested paths under projected JSON root fields resolve against the projected JSON/JSONB columns directly.
  • Root-level relation-style array queries continue to work when those arrays are projected into JSON/JSONB columns.
  • When the mode is disabled, queries continue to use the data column.

• Tests

  • Added coverage for:
    • projected-field metadata derivation from schema, including complex root fields
    • SQL generation using projected scalar and projected JSON columns
    • exclusive projected-mode behavior without JSON fallback
    • row parsing from projected columns without falling back to stale JSON payloads
    • omitting projected fields from data

Example:

const repo = yield* makeRepo("User", User, {
  config: {
    rootLevelFieldsWhenAvailable: true
  }
})

With that enabled, root-level fields are stored in dedicated SQL columns when available. Scalars use typed columns where possible, while complex root fields use JSON/JSONB columns. Projected fields are not duplicated into data, and projected-mode reads/writes/queries use the root columns instead of falling back to data.

[pkg-pr-new]

Open in StackBlitz

@effect-app/cli

npm i https://pkg.pr.new/effect-app/libs/@effect-app/cli@763

effect-app

npm i https://pkg.pr.new/effect-app/libs/effect-app@763

@effect-app/eslint-codegen-model

npm i https://pkg.pr.new/effect-app/libs/@effect-app/eslint-codegen-model@763

@effect-app/eslint-shared-config

npm i https://pkg.pr.new/effect-app/libs/@effect-app/eslint-shared-config@763

@effect-app/infra

npm i https://pkg.pr.new/effect-app/libs/@effect-app/infra@763

@effect-app/vue

npm i https://pkg.pr.new/effect-app/libs/@effect-app/vue@763

@effect-app/vue-components

npm i https://pkg.pr.new/effect-app/libs/@effect-app/vue-components@763

commit: 0b3b0b9

[patroza]

@copilot document the plan for next step; relational fields

[Copilot]

@copilot document the plan for next step; relational fields

Added the Phase 2 relational follow-up plan to the config docs so it’s documented next to rootLevelFieldsWhenAvailable and the repository config surface. Addressed in 24782ef.

[patroza]

@copilot don't write the separate fields also to data. and support any field type, including complex types, which should simply be json fields, like data is.

[Copilot]

@copilot don't write the separate fields also to data. and support any field type, including complex types, which should simply be json fields, like data is.

Addressed in 93db22d. Projected root fields are no longer duplicated into data, and root projection now covers any root field type by using JSON/JSONB columns for complex fields while keeping nested filters and root-array relation queries working.

[patroza]

@copilot you misunderstood rootLevelFieldsWhenAvailable setting; the when available is about when database adapters usually use data column, when the setting is enabled you should switch to using root level columns and do migrations when there are schema changes.

you should not support both at the same time, with fallbacks. A migration of data should also happen offline, not on the fly. Queries and writes should either be to data column when setting is omitted/false, and to columns when true

[Copilot]

@copilot you misunderstood rootLevelFieldsWhenAvailable setting; the when available is about when database adapters usually use data column, when the setting is enabled you should switch to using root...

Addressed in 34fb23e, with cleanup in 0b3b0b9. rootLevelFieldsWhenAvailable now switches SQL repos into an exclusive root-column mode: projected fields no longer fall back to data, runtime backfill/ALTER behavior was removed, and the docs/tests now reflect that existing schema/data migration is expected offline.