OXY-17: Add IN / NOT IN value-list predicates to the sql query DSL - #300
OXY-17: Add IN / NOT IN value-list predicates to the sql query DSL#300Kalin-Rudnicki wants to merge 3 commits into
Conversation
💬 Feedback from KalinA core design goal of Is that goal unreasonable here? I don't think so — can we tweak it to keep the SQL string static? The obvious candidate is to bind the whole collection as a single array parameter and emit Note this would converge |
Reworked
|
| DSL | SQL (static, one ?) |
|---|---|
p.id.in(ids) |
p.id = ANY(?) |
p.id.notIn(ids) |
p.id <> ALL(?) |
The ? count is always 1 regardless of list size, so QueryContext.sql is a plain constant again. Removed the sentinel token, the InClause runtime expander, the per-execution ctx.sql rewrite, InputEncoder.SeqEncoder, the GeneratedFragment inClauses channel, and GeneratedInClause. batched now works with IN (it was previously disabled because the ? count varied per input).
Reuses OXY-6 / OXY-18 machinery
Encoding goes through RowRepr.ArrayRepr[A](col).encoder (ArraySeqEncoder → InputWriter.unsafeWriteArray → createArrayOf), the same array-bind path introduced by OXY-18 and surfaced as ids.contains(col) in OXY-6 (#298). The runtime Seq[A] is adapted with ArraySeq.untagged.from. No ::type[] cast — the JDBC typed array carries its element type.
⚠️ Convergence flag: this makesin/notIneffectively sugar over OXY-6's= ANY(?). #298 and this PR now overlap heavily on mechanism and differ only in DSL surface (containsvsin/notIn, plus this adds<> ALL(?)negation). Suggest merging one, then rebasing the other onto a shared generation helper and deciding whether both spellings should coexist.
Truth table / edge cases (verified)
= ANY / <> ALL reproduce the exact 3-valued logic of IN / NOT IN:
- Empty (native, no special-casing):
= ANY('{}')→FALSE(likeIN ()),<> ALL('{}')→TRUE(likeNOT IN ()). NOT IN+ NULL footgun: preserved identically, not worsened — and unreachable here (array columns are non-nullable; optional value-lists are rejected).- Single-column element only; const value-lists rejected.
Verification
@compile(debug=true)confirms static templates:WHERE p.id = ANY(?)andWHERE p.groupId = ? AND p.id <> ALL(?).- New static-SQL assertions in the test:
ctx.sql contains "p.id = ANY(?)",!contains "IN (",contains "p.id <> ALL(?)". CustomQuerySpec13/13 pass against real Postgres (testcontainers) — non-empty / single / empty / large /NOT IN/NOT INempty / composition / delete / multi-input interleaving.
Full rationale + decisions in report/OXY-17.md. Confidence: 9/10.
Adds `col.in(coll)` / `col.notIn(coll)` to the `@compile` query DSL, filtering a single column against a runtime `Seq[A]` with expanded placeholders `col IN (?, ?, ...)` / `NOT IN (...)`. Because the `?` count depends on the runtime list length (which is not known when `QueryContext.sql` is baked at construction), the compiled SQL carries a unique sentinel token where the predicate goes, plus an `InClause` expander that renders the final SQL per-execution (empty list -> `FALSE` for IN, `TRUE` for NOT IN -- never the illegal `IN ()`). Each element binds as its own scalar parameter via a new `InputEncoder.SeqEncoder`, embedded in the normal encoder chain so multi-input interleaving stays correct with no execute-time special-casing. Kept deliberately distinct from OXY-6's single-array `= ANY(?)` / `UNNEST`. - DSL: `Q.in` / `Q.notIn` extension methods. - Parsing: dedicated `RawQueryExpr.InList` + `QueryExpr.InList` (single-column LHS, runtime-input RHS). - Generation: `GeneratedInClause` third channel through `GeneratedFragment`; `FragmentBuilder.inList`; `makeQuery` threads clauses into `QueryI`/`QueryIO`. - Runtime: `InClause` renderer; `QueryI`/`QueryIO.Simple` render SQL per input; `batched` guarded against dynamic IN. - Tests: `CustomQuerySpec` "in / notIn (OXY-17)" (non-empty/empty/single/large, NOT IN, composition, delete) + compiled queries in queries.scala. - Docs: queries.md DSL vocabulary + IN/NOT IN section cross-linked to OXY-6. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…<> ALL(?))
Owner feedback: oxygen-sql aims for 100% STATIC SQL strings, but the first `IN`
cut emitted a runtime-variable number of `?` placeholders (sentinel token +
per-execution `InClause` expander that rewrote `ctx.sql` on every run).
Rework: bind the whole collection as ONE `java.sql.Array` parameter and emit a
constant string -- `col = ANY(?)` for `in`, `col <> ALL(?)` for `notIn`. The `?`
count is always 1, independent of list size, so `ctx.sql` is static again. No
sentinel, no per-exec SQL rewrite; `batched` now works with IN.
Reuses OXY-6/OXY-18 array machinery (`RowRepr.ArrayRepr` / `ArraySeqEncoder` /
`InputWriter.unsafeWriteArray`); the runtime `Seq[A]` is adapted via
`ArraySeq.untagged.from`. `= ANY` / `<> ALL` reproduce the exact 3-valued truth
table of `IN` / `NOT IN`, including empty-collection (`= ANY('{}')` -> FALSE,
`<> ALL('{}')` -> TRUE) and the NOT-IN-with-NULL footgun (unreachable here: array
columns are non-nullable, optional value-lists rejected). This converges `in` /
`notIn` with OXY-6's `= ANY(?)` -- flagged in report for human reconciliation.
Removed: query/InClause.scala, generation/GeneratedInClause.scala,
InputEncoder.SeqEncoder, GeneratedFragment's inClauses channel, QueryI/QueryIO
inClauses+execCtx, ParsedQuery makeQuery inClauses threading.
Tests: same behavioral matrix (non-empty/single/empty/large/NOT IN/composition/
delete/multi-input) + static-SQL assertions (`= ANY(?)`, no `IN (`, `<> ALL(?)`).
Full CustomQuerySpec 13/13 green on real Postgres.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011YxWKdsz97QT9BD7AdpSq6
What
Adds
IN/NOT INvalue-list support to theoxygen-sql@compilequery DSL:Filters a single column against a runtime
Seq[A]with expanded placeholderscol IN (?, ?, ...)/NOT IN (...).How
The
?count depends on the runtime list length, which is unknown whenQueryContext.sqlis baked at construction. So the compiled SQL carries a unique sentinel token where the predicate goes, plus a runtimeInClauseexpander that renders the final SQL per-execution:IN->FALSE, emptyNOT IN->TRUE(never the illegalIN ()).InputEncoder.SeqEncoder, embedded in the normal encoder chain, so multi-input interleaving stays correct with no execute-time special-casing.Kept deliberately distinct from OXY-6's single-array
= ANY(?)/UNNEST.Decisions (see report for full list)
col.in(coll)/col.notIn(coll); collection isSeq[A](Set ->.toSeq).notInfirst-class (clean empty-list rewrite). SubqueryIN (SELECT ...)out of scope.batched+ dynamic IN unsupported (throws); value-list must be a runtimeinput, notconst.Verification
oxygen-sql+sql-it/Testcompile clean.CustomQuerySpec"in / notIn (OXY-17)" runs against a real Postgres (testcontainers) and passes: non-empty / single / empty(FALSE) / large lists, NOT IN, NOT IN empty(TRUE), composition, delete-by-IN, multi-input interleaving.CustomQuerySpec(13 tests) passes — no regressions.docs/docs/sql/queries.md), cross-linked to OXY-6.Confidence: 8.5/10
High confidence in the implemented scope (green integration tests, localized design). Residual risk: IN embedded in subqueries / INSERT-FROM-SELECT not explicitly tested; const-list & Set inputs intentionally unsupported.
Full notes + assumptions:
report/OXY-17.md.🤖 Generated with Claude Code