Support temporary functions in stored queries#4291
Open
sergei-pustovykh wants to merge 57 commits into
Open
Conversation
…n catalog on startup
…or and RecordLayerMetricCollector
…emp-funcs # Conflicts: # fdb-record-layer-core/src/main/java/com/apple/foundationdb/record/RecordMetaData.java # fdb-record-layer-core/src/main/java/com/apple/foundationdb/record/RecordMetaDataBuilder.java # fdb-record-layer-core/src/main/proto/record_metadata.proto # fdb-relational-api/src/main/java/com/apple/foundationdb/relational/api/metadata/SchemaTemplate.java # fdb-relational-core/src/main/java/com/apple/foundationdb/relational/recordlayer/metadata/NoOpSchemaTemplate.java # fdb-relational-core/src/main/java/com/apple/foundationdb/relational/recordlayer/query/OfflineStoredQueriesProcessor.java # fdb-relational-core/src/main/java/com/apple/foundationdb/relational/recordlayer/query/PlanGenerator.java # yaml-tests/src/test/resources/schema-template-stored-queries.yamsql
…emp-funcs # Conflicts: # fdb-relational-core/src/main/java/com/apple/foundationdb/relational/recordlayer/query/PlanGenerator.java
hatyo
reviewed
Jul 1, 2026
hatyo
left a comment
Contributor
There was a problem hiding this comment.
Retracting accidental approval, need to review first.
hatyo
requested changes
Jul 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR extends the stored-queries concept introduced previously: a stored query can now declare the temporary functions that its SELECT body depends on. The DDL syntax, the metadata shape, and the offline plan generation path all grow accordingly.
New DDL clause
CREATE QUERYnow accepts zero or moreWITH CREATE TEMPORARY FUNCTION ...clauses between the query name and theAS <select>body. Each clause uses the existingCREATE TEMPORARY FUNCTIONsyntax — same shape as a standalone temp-function DDL, including[OR REPLACE]?and theON COMMIT DROP FUNCTIONkeywords.CREATE QUERY top_orders WITH CREATE TEMPORARY FUNCTION half(in x bigint) ON COMMIT DROP FUNCTION AS SELECT * FROM orders WHERE col1 < x / 2 WITH CREATE OR REPLACE TEMPORARY FUNCTION big(in x bigint) ON COMMIT DROP FUNCTION AS SELECT * FROM half(x) WHERE col2 > 100 AS SELECT id FROM big(200)Each temp-function source is captured verbatim (whitespace and casing preserved) and persisted alongside the SELECT body.
Metadata shape change
The previous PR stored Map<String, String> (name -> SQL). This PR replaces that with Map<String, StoredQuery> where StoredQuery is a small immutable struct holding the SELECT body and an ordered List of CREATE TEMPORARY FUNCTION ... source strings. There are two parallel StoredQuery types - one in fdb-record-layer-core (RecordMetaData.StoredQuery) and one in fdb-relational-api - for the same module-layering reason as View, Index, and other metadata pairs. The relational-core serializer translates between them.
On the wire, temp functions are added as
repeated string temp_functiontoPStoredQuerymessage:Offline plan generation for stored queries with temp functions
OfflineStoredQueriesProcessoriterates through each stored query'sWITH ... CREATE TEMPORARY FUNCTION ...clauses and generates a plan for each one using a customMetadataOperationsFactory. The factory returns an empty (no-op)ConstantAction. Instead, it rebuilds local schema template from theRecordLayerInvokedRoutinethatDdlVisitorhands it during plan generation. We do not execute the resulting procedure plan. All template modifications happen at plan-generation time.Once all temp functions have been processed this way, the processor plans the stored query's SELECT body against the amended template and inserts the resulting plan into
RelationalPlanCache.