feat(csharp): validate exact-match args for SEA GetPrimaryKeys / GetCrossReference - #609
feat(csharp): validate exact-match args for SEA GetPrimaryKeys / GetCrossReference#609eric-wang-1990 wants to merge 17 commits into
Conversation
There was a problem hiding this comment.
Verdict: 1 Medium
Focused, well-tested change that mirrors the JDBC reference driver; validation placement and the internal validateArgs: false reuse are correct. One medium concern: the new throw in GetCrossReferenceAsync runs before the PKFK-disabled empty short-circuit, making it inconsistent with GetPrimaryKeysAsync when EnablePKFK == false.
Addresses: - #3697498716 at csharp/src/StatementExecution/StatementExecutionStatement.cs:1742 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
…rossReference
Adds client-side argument validation for the SEA metadata exact-match operations,
mirroring the JDBC reference driver's resolveKeyBasedParams / listCrossReferences:
GetPrimaryKeys:
- table null/empty -> throw "tableName may not be null" (42000)
- catalog set + schema null -> throw "schema may not be null when catalog is specified" (42000)
GetCrossReference:
- foreign table null -> empty result (JDBC "unspecified")
- foreign catalog set + foreign schema null -> throw (42000)
Validating client-side gives a clean, deterministic error and avoids the Thrift
server's internal "GET_FUNCTIONS assertion failed" (SQLSTATE 08000) bug on null
schema. Validation is on the user-facing command path only — GetColumnsExtended's
internal three-call fallback reuses the PK/FK fetch with a null schema on purpose
and bypasses the checks (GetPrimaryKeysAsync validateArgs:false; the shared
FetchCrossReferenceAsync is unvalidated).
Stacked on #604 (object-not-found → empty, backslash escaping, exception-type pivot);
this modifies the same SEA metadata methods that PR introduces, so it targets that
branch rather than main.
955 unit tests pass.
Co-authored-by: Isaac
Addresses: - #3697498716 at csharp/src/StatementExecution/StatementExecutionStatement.cs:1742 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
The engineer-bot's fix (9168ac9) hoisted the ShouldReturnEmptyPKFKResult guard ahead of arg validation in GetCrossReferenceAsync but added no test. These pin the behavior: with EnablePKFK=false, both GetCrossReference (foreign catalog set + foreign schema null) and GetPrimaryKeys (catalog set + schema null) return an empty result instead of throwing — so the disabled-feature contract holds uniformly across both exact-match ops. Adds a CreateMetadataStatement overload that merges extra connection properties. 957 unit tests pass. Co-authored-by: Isaac
05879dc to
914ac64
Compare
…atching JDBC SEA JDBC SEA (DatabricksMetadataQueryClient.listCrossReferences) checks ONLY the foreign table: a null foreign table returns an empty result and it never inspects the parent table. ADBC SEA now mirrors that, so both-tables-null returns empty (not a throw). The foreign-catalog-set + foreign-schema-null case still throws a clean 42000, matching JDBC SEA resolveKeyBasedParams and avoiding Thrift's internal GET_FUNCTIONS 08000 assertion bug. Live Thrift throws 42000 for the both-null input; ADBC SEA follows JDBC SEA here, and the comparator whitelists that single input as a Thrift-vs-SEA divergence. Co-authored-by: Isaac
914ac64 to
c5b37b4
Compare
There was a problem hiding this comment.
Verdict: 1 Medium
Looks good — the argument-validation logic is self-consistent and well-tested (short-circuit ordering, validateArgs:false reuse path, and direct FetchCrossReferenceAsync bypass are all correct). One medium concern: the PR description's GetCrossReference table and "both-null" note claim ADBC SEA throws (follows Thrift, no comparator whitelist needed), but the code, inline comment, and tests all return empty (follow JDBC, requiring a whitelist) — reconcile the description and confirm the external comparator whitelist matches the shipped empty behavior.
Other findings
-
🟡 Medium — The PR description contradicts the shipped behavior for the both-tables-null GetCrossReference case, and the two descriptions imply opposite comparator-whitelist requirements.
-
The description's
GetCrossReferencetable and the "Note on the both-null case" both state ADBC SEA throws 42000 for both-null, "follows Thrift rather than JDBC," and that matching Thrift converges the comparator ("no whitelist needed"). -
The actual code returns an empty result when the foreign table is null (regardless of the parent), the inline comment says it "follows JDBC SEA here, not Thrift ... the comparator whitelists that one input," and
GetCrossReference_BothTablesNull_ReturnsEmptyassertsRowCount == 0.
So the shipped code follows JDBC (empty) and requires the comparator to whitelist the both-null input as a Thrift-vs-SEA divergence (Thrift throws, SEA returns empty). If any comparator/whitelist configuration (outside this diff) was set per the description's claim that the case "converges" and needs no whitelist, the comparator will fail on the both-null input. Please reconcile the PR description with the implementation and confirm the comparator whitelist matches the shipped empty behavior.
| // Validating here (before issuing SHOW KEYS) also avoids the Thrift server's | ||
| // internal "GET_FUNCTIONS assertion failed" bug on schema-null, and gives a | ||
| // clean, deterministic error instead of relying on a server round-trip. | ||
| if (validateArgs) |
There was a problem hiding this comment.
Instead of passing in this extra validateArgs param, we should change the callsite where we do not expect it to throw, to return empty rather than throw.
There was a problem hiding this comment.
The change is complete and clean. I've done exactly what the reviewer asked.
Pushed 252963e.
Addresses: - #3706794071 at csharp/src/StatementExecution/StatementExecutionStatement.cs:1670 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — the client-side exact-match validation mirrors the JDBC reference driver, is symmetric between GetPrimaryKeys/GetCrossReference, correctly short-circuits the disabled-PKFK and columns-extended reuse paths before validation, and throws are placed before the object-not-found try/catch so they aren't swallowed to empty. Tests cover the throw and empty cases well. One low note: an empty-result guard in GetPrimaryKeysAsync is now unreachable given the new throws.
Addresses: - #3706839691 at csharp/src/StatementExecution/StatementExecutionStatement.cs:1689 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
| // than letting the validation throw. (For a null schema GetPrimaryKeysAsync | ||
| // would return an empty result anyway, so this is behavior-preserving.) | ||
| QueryResult pkResult; | ||
| if (string.IsNullOrEmpty(_metadataTableName) |
There was a problem hiding this comment.
Should this be changed to try catch pattern? So that if there are other throw scenario in the future we do not break this return empty thing?
There was a problem hiding this comment.
Kept the explicit pre-check guard rather than switching to try/catch. A try/catch fallback to empty would have to catch DatabricksException broadly, since NewInvalidArgumentException (line 1131) produces the same AdbcStatusCode.InternalError/SqlState 42000 shape as a genuine server-side error — so it would silently swallow real execution failures (network/auth/server) and drop PK data from the columns-extended result instead of surfacing them. object-not-found is already handled inside GetPrimaryKeysAsync (line 1703). The guard's two conditions exactly mirror the method's two client-side validations, and the coupling is documented on both sides (comment at lines 1661–1664), which mitigates the drift concern the reviewer is worried about. No code change warranted.
There was a problem hiding this comment.
Refactored (2e5fd7d) per the suggestion: instead of the caller-side pre-check guard (or the earlier validateArgs flag), each exact-match op is now a thin public wrapper that validates + throws, delegating to a non-throwing internal core:
GetPrimaryKeysAsync/GetCrossReferenceAsync(public commands) validate the exact-match args and throw a clean 42000, then call the internal.GetPrimaryKeysAsyncInternal/GetCrossReferenceAsyncInternaldo the fetch and return empty for unspecified/invalid args (never throw).GetColumnsExtendedViaThreeCallscalls these directly, so its legitimately-null schema is never rejected.
This avoids the try/catch (which would have to catch DatabricksException/InternalError/42000 broadly and could swallow a real server error), and it's future-proof: any new validation added to a public wrapper stays out of the internal reuse path by construction. Verified behavior-identical to the guard version across all catalog/schema/table combinations; 240 StatementExecution unit tests pass.
There was a problem hiding this comment.
⛔ NEEDS HUMAN DECISION — the bots can't resolve this thread; a maintainer's input is required.
Thread has converged: the reviewer implemented their own suggestion in commit 2e5fd7d (public validating wrappers GetPrimaryKeysAsync/GetCrossReferenceAsync delegating to non-throwing internals GetPrimaryKeysAsyncInternal/GetCrossReferenceAsyncInternal; GetColumnsExtendedViaThreeCalls now calls the internal + FetchCrossReferenceAsync directly). No edit needed from me — I verified the current tree builds clean (0 warnings/0 errors) and StatementExecution unit tests pass (240 passed, 26 skipped E2E, 0 failed). The refactor is future-proof (new validations on public wrappers stay out of the internal reuse path) and avoids the broad-catch try/catch that could swallow real server errors. Nothing further to action here.
…nternal Replace the caller-side pre-check guard (and the earlier validateArgs flag) with a symmetric public/internal split for both exact-match metadata ops: - GetPrimaryKeysAsync / GetCrossReferenceAsync (public commands) are thin wrappers that validate the exact-match args client-side — throwing a clean 42000 for a missing table or a catalog-set/schema-null request (mirroring JDBC SEA resolveKeyBasedParams, avoiding Thrift's internal GET_FUNCTIONS 08000 bug) — then delegate to the non-throwing core. - GetPrimaryKeysAsyncInternal / GetCrossReferenceAsyncInternal do the actual fetch and return an empty result for unspecified/invalid args instead of throwing. GetColumnsExtendedViaThreeCalls reuses these directly, so its legitimately-null schema is never rejected — no guard, no flag, no broad catch that could swallow a real server error. The wrappers throw only when the feature is engaged and the internal would otherwise reach a live fetch, so the two paths never diverge. Behavior is identical to the prior guard version across all catalog/schema/table combinations; 240 StatementExecution unit tests pass. Co-authored-by: Isaac
Fold FetchCrossReferenceAsync into GetCrossReferenceAsyncInternal and give GetPrimaryKeysAsyncInternal explicit params too, so both exact-match ops share one shape: a thin field-reading public wrapper that validates + throws, over a non-throwing internal core that takes the key parts as arguments and returns empty for unspecified/invalid args. - GetPrimaryKeysAsyncInternal(catalog, schema, table, ct) - GetCrossReferenceAsyncInternal(pkCatalog, pkSchema, pkTable, fkCatalog, fkSchema, fkTable, ct) Removes the separate FetchCrossReferenceAsync layer (its callers now go through the internal directly). GetColumnsExtendedViaThreeCalls calls both internals with its own args (null PK side for the FK lookup). Behavior is identical across all catalog/schema/table combinations (verified exhaustively); the only change is the columns-extended FK reuse now emits a GetCrossReference trace span, symmetric with its existing GetPrimaryKeys span. 240 StatementExecution unit tests pass. Co-authored-by: Isaac
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — the wrapper/internal split is sound. The validation guards mirror the internal early-returns exactly (ShouldReturnEmptyPKFKResult + unspecified-arg checks), so throws only replace would-be-empty results and never drop rows; the PK/FK-disabled short-circuit is symmetric across both ops, GetColumnsExtendedViaThreeCalls correctly routes to the non-throwing internals, and the tests cover each documented case. One low-severity telemetry note inline.
Addresses: - #3707564028 at csharp/src/StatementExecution/StatementExecutionStatement.cs:1660 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
…n spans Tidy the telemetry-span-on-rejection fix (ff8d5dc): replace the async lambda + filler 'await Task.CompletedTask' (only needed to satisfy TraceActivityAsync's Func<Activity?,Task> signature) with the synchronous TraceActivity(Action<Activity?>) overload, which sets the activity status to Error, records the exception event, and rethrows — exactly the semantics needed for a pure throw. Also drops the intermediate validationError string in the PK path, restoring the two direct throws inside the span. Behavior identical (same exception, tags, span name, rethrow); 240 StatementExecution unit tests pass. Co-authored-by: Isaac
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — the split into throwing public wrappers (GetPrimaryKeysAsync / GetCrossReferenceAsync) over non-throwing internals correctly mirrors the JDBC reference driver, the renamed FetchCrossReferenceAsync → GetCrossReferenceAsyncInternal has no dangling callers, and the tests cover both the throw and empty paths. One low telemetry note: GetPrimaryKeysAsync's guard fires whenever the feature is engaged (not only on rejection), so valid calls emit a redundant empty span.
Addresses: - #3707663199 at csharp/src/StatementExecution/StatementExecutionStatement.cs:1666 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — the exact-match validation is carefully structured and well-tested. Verified that the sync TraceActivity rethrows (so the wrapper's throw isn't swallowed), that the willReject guard is equivalent to the two inner throw conditions (no fall-through to a spurious empty), and that ShouldReturnEmptyPKFKResult usage is consistent between wrapper and internal. One low-severity maintainability note about the duplicated precondition logic between the wrappers and their internals.
| string.IsNullOrEmpty(_metadataTableName) | ||
| || (!string.IsNullOrEmpty(_metadataCatalogName) && string.IsNullOrEmpty(_metadataSchemaName)); | ||
|
|
||
| if (!MetadataUtilities.ShouldReturnEmptyPKFKResult(_metadataCatalogName, null, _connection.EnablePKFK) |
There was a problem hiding this comment.
MetadataUtilities.ShouldReturnEmptyPKFKResult should be the first check to return empty. Then you can also skip the same check inside GetPrimaryKeysAsyncInternal
There was a problem hiding this comment.
That's fine — no live agent; the reply goes through the structured outcome report.
I addressed the reviewer's comment. Summary:
What the comment asked: Make MetadataUtilities.ShouldReturnEmptyPKFKResult the first check to return empty, and then also skip the same check inside GetPrimaryKeysAsyncInternal.
What I did:
- First half (applied): Restructured
GetPrimaryKeysAsyncsoShouldReturnEmptyPKFKResultis the first statement and returns an empty result on true, replacing the awkwardwillRejectdouble-negative guard with a plain early-return. Arg validation now only runs once the feature is genuinely engaged. - Second half (did not apply, with reason): Keeping the check inside
GetPrimaryKeysAsyncInternalbecause that method is a second entry point — `GetColu
Pushed bca2fd1.
Addresses: - #3708072888 at csharp/src/StatementExecution/StatementExecutionStatement.cs:1673 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — the client-side arg validation, wrapper/internal split, and non-throwing core all line up with the JDBC-parity intent, and the tests cover the throw/empty matrix well. One low-severity note: the GetPrimaryKeys disabled-feature/invalid-catalog short-circuit no longer emits a telemetry span, an asymmetry with the GetCrossReference path which still does.
Addresses: - #3708123722 at csharp/src/StatementExecution/StatementExecutionStatement.cs:1668 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — a clean, well-reasoned refactor splitting each exact-match op into a validating public wrapper over a non-throwing internal, with the guard conditions correctly mirroring the internal early-returns (verified ShouldReturnEmptyPKFKResult semantics and the synchronous TraceActivity rethrow behavior). One low-severity note: the non-throwing internal-reuse path (columns-extended with a null schema) is only covered by CI-skipped E2E tests, so consider adding a unit guard. Nit: the GetCrossReferenceAsync throw-path span sets only the fk_* tags and omits the pk_* tags the internal sets — harmless telemetry asymmetry.
Rename the non-throwing internal cores to make the contract explicit at every call site: GetPrimaryKeysAsyncInternal -> GetPrimaryKeysAsyncNoThrow, and GetCrossReferenceAsyncInternal -> GetCrossReferenceAsyncNoThrow. Pure rename (defs, call sites, and comment references); no behavior change. 240 StatementExecution unit tests pass. Co-authored-by: Isaac
Addresses: - #3708155500 at csharp/src/StatementExecution/StatementExecutionStatement.cs:1588 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — the wrapper/internal split is sound: sync TraceActivity rethrows so client-side validation throws propagate, and the throw-guards mirror the non-throwing cores' early-returns via the same ShouldReturnEmptyPKFKResult call, so they can't diverge (disabled feature / invalid catalog short-circuits to empty before any throw). The columns-extended reuse is behavior-preserving and newly test-guarded. One low: a stale method-name reference in a test comment.
Addresses: - #3708216320 at csharp/test/Unit/StatementExecution/StatementExecutionMetadataObjectNotFoundTests.cs:444 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
…on throws The exact-match argument rejections (missing table / catalog-set-schema-null) are synchronous, deterministic checks with no I/O — the thrown DatabricksException (type + SqlState 42000 + message) fully describes the failure to the caller, so wrapping the throw in a TraceActivity span added no diagnostic value (reviewer rated it Low). Throw directly in the public wrappers instead; the non-throwing internals keep their spans for the real fetch paths. Also simplifies the PK guard back to 'feature engaged -> validate' (the extra bad-args condition only existed to avoid a spurious empty span). No behavior change; 240 StatementExecution unit tests pass. Co-authored-by: Isaac
Summary
Adds client-side argument validation to the SEA metadata exact-match operations
GetPrimaryKeysandGetCrossReference, mirroring the JDBC reference driver'sresolveKeyBasedParams/listCrossReferences. This gives clean, deterministicerrors and avoids relying on the Thrift server's inconsistent (and in one case buggy)
handling of missing arguments.
Split out of #604 (which handles object-not-found → empty,
backslash escaping, and the exception-type pivot) because exact-match argument
validation is a distinct concern.
Verified behavior (live thrift vs rest, + JDBC reference source)
Legend: empty = 0-row result, no throw. throw = AdbcException.
GetPrimaryKeys(catalog, schema, table)
42000"tableName may not be null"4200008000"GET_FUNCTIONS assertion failed" (server bug)42000(clean; matches JDBC intent)08000(server bug)GetCrossReference(pk…, fk…)
42000"foreignTable and parentTableName are both null"08000"GET_FUNCTIONS assertion failed" (server bug)42000(clean; matches JDBC intent)Why validate client-side
08000on null-schema is a server bug — it surfaces an internalGET_FUNCTIONS assertion failed(a connection-exception SQLSTATE) instead of aclean argument error. JDBC validates before the RPC and throws a clean error;
this PR does the same so SEA doesn't depend on (or replicate) the buggy path.
SHOW FOREIGN KEYS.
Scope / correctness notes
thin public wrapper that reads the statement fields, validates, and throws
(
GetPrimaryKeysAsync/GetCrossReferenceAsync), over a non-throwing internalcore that takes the key parts as explicit params and returns empty for
unspecified/invalid args (
GetPrimaryKeysAsyncInternal(catalog, schema, table, …)/GetCrossReferenceAsyncInternal(pkCatalog, pkSchema, pkTable, fkCatalog, fkSchema, fkTable, …)).GetColumnsExtended's internal three-call fallback reuses the PK/FK fetch with a nullschema on purpose (to gather keys for a column set); it calls the internals
directly, so its legitimately-unspecified args return empty instead of being rejected.
The wrappers throw only when the feature is engaged and the internal would otherwise
reach a live fetch, so the two paths never diverge.
DatabricksException(the SEA-natural type). The comparator comparesStatus+SqlState, not the concrete subclass, so this matches Thrift's pathwhere Thrift returns a clean error, and is whitelisted where Thrift emits its
08000bug.Comparator
The residual Thrift
08000-vs-SEA42000sqlstate diffs (foreign/catalog schema-null)are documented as a Thrift server bug in databricks-driver-test's test_case_filters
(schema-null exact-match skip). The both-tables-null case is also whitelisted there
(databricks-driver-test#1047): ADBC SEA now returns empty (matching JDBC SEA) while
Thrift throws
42000— an intentional Thrift-vs-SEA divergence.All behavior in the tables above was verified against the live Thrift server via a
throwaway E2E probe (
TGetPrimaryKeysReq/TGetCrossReferenceReq), not inferred fromcode or mocks.
This pull request and its description were written by Isaac.