Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions docs/docs/sql/queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,8 @@ input makes it a `QueryO`/`Query`. Pass `debug = true` (`@compile(debug = true)`
| Form | Purpose |
|------|---------|
| `input[I]` / `input.optional[I]` / `input.const(i)` | bind a runtime / optional / compile-time-constant parameter |
| `input.array[I]` / `input.set[I]` + `ids.contains(col)` | bind a whole `Seq` / `Set` as one array parameter, expanded to `col = ANY(?)` |
| `select.unnest(ids)` | use a collection input as a `UNNEST(?)` join/table source (one row per element) |
| `select[A]` | select all columns of table `A` |
| `join[A] if <cond>` / `leftJoin[A] if <cond>` | inner / left join (`leftJoin` yields `Option[A]`) |
| `where if <cond>` | filter |
Expand All @@ -143,6 +145,75 @@ val personJoinNotes: QueryIO[UUID, (Person, Note)] =
> Custom column types flow through automatically: `input[Email]` and `select[UserRow]` use the
> `RowRepr`/encoder/decoder for `Email` you defined in [Models](models.md).

### Array input (`= ANY(?)`)

To filter against a collection of values, bind the whole collection as a **single** array parameter
with `input.array[I]` (a `Seq[I]`) or `input.set[I]` (a `Set[I]`) and test membership with
`ids.contains(col)`. This generates Postgres `col = ANY(?)` — one bind parameter carrying a
`java.sql.Array`, instead of expanding an `IN (…)` list to N placeholders (which hits the JDBC
~32767-param limit and blows up the plan cache).

```scala
@compile
val selectByIdArray: QueryIO[Seq[UUID], Person] =
for {
ids <- input.array[UUID]
p <- select[Person]
_ <- where if ids.contains(p.id)
} yield p
```

```scala
selectByIdArray.execute(Seq(id1, id2, id3))
```

Notes:

- `input.array[I]` takes a `Seq[I]`, `input.set[I]` takes a `Set[I]`; either is bound as one JDBC
array, so callers pass their collection directly (no `ArraySeq.from(…)` conversion needed).
- An empty collection yields `col = ANY('{}')`, i.e. no matches.
- `I` must be a single-column type (e.g. `UUID`, `Long`, `String`, or a `RowRepr` newtype over one).
Composite/multi-column element types are not supported yet.
- Array inputs compose with other inputs, e.g. `where if ids.contains(p.id) && p.groupId == groupId`.
- No explicit `::type[]` cast is emitted: the JDBC driver builds a typed array via
`createArrayOf(<base type>, …)`, so `= ANY(?)` already knows the element type.

### Array input as a join table (`UNNEST(?)`)

The primary way to use a collection input is as a **table source** you can join against: `select.unnest(ids)`
turns an `input.array[I]` / `input.set[I]` collection into a `UNNEST(?)` from-item that yields one row
per element, aliased so it can be joined/filtered/selected like any other table. The whole collection
still binds as a single `java.sql.Array` parameter.

```scala
@compile
val notesByPersonIdUnnest: QueryIO[Seq[UUID], Note] =
for {
ids <- input.array[UUID]
id <- select.unnest(ids) // FROM UNNEST(?::uuid[]) id(id)
n <- join[Note] if n.personId == id
} yield n
```

generates (roughly):

```sql
SELECT n.id, n.person_id, n.note
FROM UNNEST(?::uuid[]) id(id)
JOIN note n ON n.person_id = id.id
```

Notes:

- The unnested element is a normal query variable — reference it in the join/where (`n.personId == id`)
or select it (`yield (id, n)`). It is emitted as a named, qualified column (`id.id`) so it never
collides with a same-named column of a joined table.
- An empty array produces zero rows (the join matches nothing).
- `I` must be a single-column type, same restriction as `input.array` / `input.set`.
- Composes with other inputs/joins/wheres, e.g. an extra `input[String]` used in a `where`.
- Currently supported as the root `FROM` source; `JOIN UNNEST(?)` (unnest as a non-root join item)
is not yet supported.

## A real repo method

```scala
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,12 @@ final class DecoderBuilder {
case (queryExpr: QueryExpr.ConstValue, Some(parentContext)) => convert.const(queryExpr, parentContext)
case (queryExpr: QueryExpr.InputVariableReferenceLike, Some(parentContext)) => convert.input(queryExpr, parentContext)
case (queryExpr: QueryExpr.QueryVariableReferenceLike, _) => convert.query(queryExpr)
case (queryExpr: QueryExpr.Binary, _) => convert.binary(queryExpr)
case (queryExpr: QueryExpr.BuiltIn, _) => convert.builtIn(queryExpr)
case (queryExpr: QueryExpr.Composite, _) => convert.composite(queryExpr, parentContext)
case (queryExpr: QueryExpr.ConstValue, None) => ParseResult.error(queryExpr.fullTerm, "No RowRepr to compare with")
case (queryExpr: QueryExpr.InputVariableReferenceLike, None) => ParseResult.error(queryExpr.fullTerm, "No RowRepr to compare with")
case (_: QueryExpr.ArrayContains, _) => ParseResult.success(GeneratedResultDecoder.single(TypeclassExpr.RowRepr.boolean.resultDecoder, TypeRepr.of[Boolean]))
case (queryExpr: QueryExpr.Binary, _) => convert.binary(queryExpr)
case (queryExpr: QueryExpr.BuiltIn, _) => convert.builtIn(queryExpr)
case (queryExpr: QueryExpr.Composite, _) => convert.composite(queryExpr, parentContext)
case (queryExpr: QueryExpr.ConstValue, None) => ParseResult.error(queryExpr.fullTerm, "No RowRepr to compare with")
case (queryExpr: QueryExpr.InputVariableReferenceLike, None) => ParseResult.error(queryExpr.fullTerm, "No RowRepr to compare with")

def const(queryExpr: QueryExpr.ConstValue, parentContext: TypeclassExpr.RowRepr): ParseResult[GeneratedResultDecoder] =
ParseResult.success(GeneratedResultDecoder.single(parentContext.resultDecoder, queryExpr.fullTerm.tpe.widen))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ final case class FragmentBuilder(inputs: List[InputPart])(using Quotes) {
private val nonConstInputParams: List[VariableReference.NonConstInput] =
inputs.map(_.mapQueryRef).flatMap {
case p: VariableReference.FromInput => p.some
case p: VariableReference.ArrayFromInput => p.some
case p: VariableReference.OptionalFromInput => p.some
case _: VariableReference.FromConstInput => None
}
Expand Down Expand Up @@ -60,6 +61,11 @@ final case class FragmentBuilder(inputs: List[InputPart])(using Quotes) {
case (Some(inputTpeTupGen), VariableReference.FromInput(param)) =>
val idx: Int = symIdxMap.getOrElse(param.sym, report.errorAndAbort("sym not found?", param.tree.pos))
InputRepr.NonConst(TermTransformer.FromProductGenericField(inputTpeTupGen, inputTpeTupGen.fields(idx)), false)
case (None, VariableReference.ArrayFromInput(_, _)) =>
InputRepr.NonConst(TermTransformer.Id, false)
case (Some(inputTpeTupGen), VariableReference.ArrayFromInput(param, _)) =>
val idx: Int = symIdxMap.getOrElse(param.sym, report.errorAndAbort("sym not found?", param.tree.pos))
InputRepr.NonConst(TermTransformer.FromProductGenericField(inputTpeTupGen, inputTpeTupGen.fields(idx)), false)
case (None, VariableReference.OptionalFromInput(_)) =>
InputRepr.NonConst(TermTransformer.Id, true)
case (Some(inputTpeTupGen), VariableReference.OptionalFromInput(param)) =>
Expand All @@ -83,6 +89,7 @@ final case class FragmentBuilder(inputs: List[InputPart])(using Quotes) {
case (queryExpr: QueryExpr.ConstValue, Some(parentContext)) => queryExprToFragment.constOrInput(queryExpr, parentContext)
case (queryExpr: QueryExpr.InputVariableReferenceLike, Some(parentContext)) => queryExprToFragment.constOrInput(queryExpr, parentContext)
case (queryExpr: QueryExpr.QueryVariableReferenceLike, _) => queryExprToFragment.query(queryExpr)
case (queryExpr: QueryExpr.ArrayContains, _) => queryExprToFragment.arrayContains(queryExpr)
case (queryExpr: QueryExpr.Binary, _) => queryExprToFragment.binary(queryExpr)
case (queryExpr: QueryExpr.BuiltIn, _) => queryExprToFragment.builtIn(queryExpr)
case (queryExpr: QueryExpr.Composite, _) => queryExprToFragment.composite(queryExpr, parentContext)
Expand Down Expand Up @@ -220,6 +227,17 @@ final case class FragmentBuilder(inputs: List[InputPart])(using Quotes) {
} yield GeneratedFragment.of(lhsFrag, op.sqlPadded, rhsFrag)
}

def arrayContains(queryExpr: QueryExpr.ArrayContains)(using ParseContext, GenerationContext, Quotes): ParseResult[GeneratedFragment] =
for {
colFrag <- queryExprToFragment.query(queryExpr.queryCol)
// wrap the compared column's `RowRepr[A]` into `RowRepr[Seq[A]]` / `RowRepr[Set[A]]`, so the
// whole collection binds as a single `?` param (via `ArraySeqEncoder` -> `java.sql.Array`).
arrRowRepr <- collectionAsArrayRepr(queryExpr.arrInput, queryExpr.queryCol.rowRepr)
inputFrag <- GenerationContext.updated(input = GenerationContext.Parens.Never) {
queryExprToFragment.input(queryExpr.arrInput, arrRowRepr)
}
} yield GeneratedFragment.of(colFrag, " = ANY(", inputFrag, ")")

def builtIn(queryExpr: QueryExpr.BuiltIn)(using ParseContext, GenerationContext, Quotes): ParseResult[GeneratedFragment] =
queryExpr match
case QueryExpr.Static(_, out, _) => ParseResult.Success(GeneratedFragment.sql(out))
Expand Down Expand Up @@ -306,6 +324,33 @@ final case class FragmentBuilder(inputs: List[InputPart])(using Quotes) {
s"\n ) AS ${s.subQueryTableName}",
)

/** Wrap an element `RowRepr[A]` into the collection `RowRepr` matching how `input` was declared (`array` -> `Seq`, `set` -> `Set`). */
private def collectionAsArrayRepr(
input: QueryExpr.InputVariableReferenceLike,
elemRowRepr: TypeclassExpr.RowRepr,
)(using ParseContext): ParseResult[TypeclassExpr.RowRepr] =
input.queryRef match
case VariableReference.ArrayFromInput(_, kind) => ParseResult.success(elemRowRepr.collectionAsArray(kind))
case _ => ParseResult.error(input.fullTerm, "expected an `input.array` / `input.set` collection")

def select(s: SelectPart.FromUnnest)(using ParseContext, GenerationContext, Quotes): ParseResult[GeneratedFragment] =
for {
// bind the referenced collection input as a single `?` param via `RowRepr[Seq[A]]` / `RowRepr[Set[A]]` (`ArraySeqEncoder`)
arrRowRepr <- collectionAsArrayRepr(s.arrInput, s.elemRowRepr)
inputFrag <- GenerationContext.updated(input = GenerationContext.Parens.Never) {
queryExprToFragment.input(s.arrInput, arrRowRepr)
}
// cast `?::<baseType>[]` so postgres knows the array/element type of the table source, then
// `<alias>(<alias>)` names the single output column so it can be referenced as `<alias>.<alias>`.
alias = s.mapQueryRef.sqlString
castType: Expr[String] = '{ "::" + ${ s.elemRowRepr.expr }.columns.columns.head.columnType.baseType + "[]" }
} yield GeneratedFragment.of(
"\n FROM UNNEST(",
inputFrag,
GeneratedFragment.sql(castType),
s") $alias($alias)",
)

def update(u: UpdatePart)(using Quotes): ParseResult[GeneratedFragment] =
ParseResult.success(
GeneratedFragment.of(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,7 @@ private[sql] object ParsedQuery extends Parser[Term, ParsedQuery] {
select match {
case select: SelectPart.FromTable => Growable(select.mapQueryRef)
case select: SelectPart.FromSubQuery => Growable.many(select.mapQueryRefs)
case select: SelectPart.FromUnnest => select.queryRefs
},
Growable.many(joins).flatMap(_.queryRefs),
Growable.option(where).flatMap(_.filterExpr.queryRefs),
Expand All @@ -201,6 +202,7 @@ private[sql] object ParsedQuery extends Parser[Term, ParsedQuery] {
selectFrag <- select match
case select: SelectPart.FromTable => fragmentBuilder.select(select)
case select: SelectPart.FromSubQuery => fragmentBuilder.select(select)
case select: SelectPart.FromUnnest => fragmentBuilder.select(select)
joinFrag <- joins.traverse(fragmentBuilder.join).map(GeneratedFragment.flatten(_))
whereFrag <- where.traverse(fragmentBuilder.where).map(GeneratedFragment.option)
orderByFrag <- orderBy.traverse(fragmentBuilder.orderBy).map(GeneratedFragment.option)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ private[generic] sealed trait QueryExpr {
this match {
case unary: QueryExpr.QueryVariableReferenceLike => ParseResult.success(unary.rowRepr)
case _: QueryExpr.Binary => ParseResult.success(TypeclassExpr.RowRepr.boolean)
case _: QueryExpr.ArrayContains => ParseResult.success(TypeclassExpr.RowRepr.boolean)
case _ => ParseResult.error(fullTerm, "Unable to extract RowRepr")
}

Expand Down Expand Up @@ -297,6 +298,22 @@ private[generic] object QueryExpr extends Parser[RawQueryExpr, QueryExpr] {
rhs: QueryExpr,
) extends Binary

//////////////////////////////////////////////////////////////////////////////////////////////////////
// ArrayContains
//////////////////////////////////////////////////////////////////////////////////////////////////////

/**
* Represents `arr.contains(col)`, generated as `col = ANY(?)` where `?` binds the whole array.
*/
final case class ArrayContains(
fullTerm: Term,
queryCol: QueryExpr.QueryVariableReferenceLike,
arrInput: QueryExpr.InputVariableReferenceLike,
) extends QueryExpr {
override def queryRefs: Growable[VariableReference] = queryCol.queryRefs ++ arrInput.queryRefs
override def show(using Quotes): String = s"${arrInput.show}.contains(${queryCol.show})"
}

//////////////////////////////////////////////////////////////////////////////////////////////////////
// BuiltIn
//////////////////////////////////////////////////////////////////////////////////////////////////////
Expand Down Expand Up @@ -368,8 +385,13 @@ private[generic] object QueryExpr extends Parser[RawQueryExpr, QueryExpr] {

override def parse(expr: RawQueryExpr)(using ParseContext, Quotes): ParseResult[QueryExpr] =
expr match {
case expr: RawQueryExpr.VariableReferenceLike => VariableReferenceLike.parse(expr)
case expr: RawQueryExpr.Binary => Binary.parse(expr)
case expr: RawQueryExpr.VariableReferenceLike => VariableReferenceLike.parse(expr)
case expr: RawQueryExpr.Binary => Binary.parse(expr)
case RawQueryExpr.ArrayContains(fullTerm, arr, elem) =>
for {
arrInput <- QueryExpr.InputVariableReferenceLike.parse(arr)
queryCol <- QueryExpr.QueryVariableReferenceLike.parse(elem)
} yield QueryExpr.ArrayContains(fullTerm, queryCol, arrInput)
case RawQueryExpr.InstantiateTable(fullTerm, gen, givenTableRepr, args) => args.traverse(parse).map(QueryExpr.InstantiateTable(fullTerm, gen, givenTableRepr, _))
case RawQueryExpr.RandomUUID(fullTerm) => ParseResult.success(QueryExpr.Static(fullTerm, "gen_random_uuid()", TypeclassExpr.RowRepr.uuid))
case RawQueryExpr.InstantNow(fullTerm) => ParseResult.success(QueryExpr.Static(fullTerm, "NOW()", TypeclassExpr.RowRepr.instant))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,22 @@ object TypeclassExpr {
def optional: TypeclassExpr.RowRepr =
TypeclassExpr.RowRepr { '{ $expr.optional } }

/** Names this schema's (single) column, so an otherwise-unnamed scalar can be referenced as `alias.name`. */
def prefixedInline(prefix: String): TypeclassExpr.RowRepr =
TypeclassExpr.RowRepr { '{ $expr.prefixedInline(${ Expr(prefix) }) } }

/**
* Wraps this element `RowRepr[A]` into a `RowRepr[Seq[A]]` / `RowRepr[Set[A]]` (see
* [[oxygen.sql.schema.RowRepr.seqRepr]] / [[oxygen.sql.schema.RowRepr.setRepr]]), so the whole
* collection binds as one Array-typed `?` param.
*/
def collectionAsArray(kind: VariableReference.ArrayInputKind): TypeclassExpr.RowRepr =
TypeclassExpr.RowRepr {
kind match
case VariableReference.ArrayInputKind.Seq => '{ $expr.seqRepr }
case VariableReference.ArrayInputKind.Set => '{ $expr.setRepr }
}

def productSchemaField(term: Term, field: String): TypeclassExpr.RowRepr =
TypeclassExpr.RowRepr {
type T
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ private[generic] sealed trait VariableReference {

final def show: String = this match
case VariableReference.FromInput(param) => param.name.greenFg.toString
case VariableReference.ArrayFromInput(param, kind) => s"${kind.show}(${param.name.greenFg})"
case VariableReference.OptionalFromInput(param) => s"optional(${param.name.greenFg})"
case VariableReference.FromConstInput(param, term, _) => s"const(${param.name.greenFg} = ${term.showAnsiCode})"
case VariableReference.FromQuery(_, _, _, true, sqlString) => sqlString.hexFg("#7EB77F").toString
Expand All @@ -37,6 +38,27 @@ private[generic] object VariableReference {
override val nonConstInputType: TypeRepr = param.tpe
}

/** The collection kind of an [[ArrayFromInput]] -- both bind as a single `java.sql.Array` param. */
enum ArrayInputKind {
case Seq, Set
def show: String = this match
case ArrayInputKind.Seq => "array"
case ArrayInputKind.Set => "set"
}

/**
* A collection input bound as a single `?` param (`java.sql.Array`).
* `param.tpe` is the user-facing collection type -- `Seq[A]` (`kind = Seq`) or `Set[A]`
* (`kind = Set`) -- so the query input type matches what the user declared; it is converted to a
* JDBC array at bind time.
*/
final case class ArrayFromInput(
param: Function.NamedParam,
kind: ArrayInputKind,
) extends NonConstInput {
override val nonConstInputType: TypeRepr = param.tpe
}

final case class OptionalFromInput(
param: Function.NamedParam,
) extends NonConstInput {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ object InputPart extends MapChainParser[InputPart] {
for {
(mapAAFC, input) <- AppliedAnonFunctCall.parseTyped[T.InputLike](term, "map function").parseLhsAndFunct {
case ('{ Q.input.apply[a] }, mapAAFC) => mapAAFC.funct.parseParam1.map { mapParam => InputPart(VariableReference.FromInput(mapParam)) }
case ('{ Q.input.array[a] }, mapAAFC) => mapAAFC.funct.parseParam1.map { mapParam => InputPart(VariableReference.ArrayFromInput(mapParam, VariableReference.ArrayInputKind.Seq)) }
case ('{ Q.input.set[a] }, mapAAFC) => mapAAFC.funct.parseParam1.map { mapParam => InputPart(VariableReference.ArrayFromInput(mapParam, VariableReference.ArrayInputKind.Set)) }
case ('{ Q.input.optional[a] }, mapAAFC) => mapAAFC.funct.parseParam1.map { mapParam => InputPart(VariableReference.OptionalFromInput(mapParam)) }
case ('{ Q.input.const[a](${ expr }) }, mapAAFC) => mapAAFC.funct.parseParam1.map { mapParam => InputPart(VariableReference.FromConstInput(mapParam, expr.toTerm, TypeRepr.of[Any])) }
}
Expand Down
Loading
Loading