feat(mssql): support injectable access_token and azure_credential#4147
Open
sdebruyn wants to merge 3 commits into
Open
feat(mssql): support injectable access_token and azure_credential#4147sdebruyn wants to merge 3 commits into
sdebruyn wants to merge 3 commits into
Conversation
…SQL destinations Add an `authentication` credential option to MsSqlCredentials so the mssql, synapse and fabric destinations can authenticate to Azure-hosted SQL with Microsoft Entra ID instead of a SQL login: - azure-identity token methods (cli, default/auto, environment, interactive, devicecode, msi/managedidentity): dlt acquires an access token and injects it via pyodbc attrs_before (SQL_COPT_SS_ACCESS_TOKEN), so they work cross-platform (including macOS, where the ODBC driver's built-in Entra ID modes are unreliable). - driver-native methods (ActiveDirectoryServicePrincipal, ActiveDirectoryPassword, ActiveDirectoryIntegrated, ActiveDirectoryInteractive) are passed as Authentication=. - empty authentication keeps the existing SQL login (backwards compatible); the default Service Principal method without a secret falls back to DefaultAzureCredential. The shared auth helpers live in the mssql destination; synapse inherits them and fabric reuses them, dropping its previous fabric-only token override. azure-identity is imported lazily (only for token methods). Adds unit tests for every method on mssql and fabric, a Docker-based integration test for the SQL-login connection path, and documentation. Closes dlt-hub#2059 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sdebruyn
added a commit
to sdebruyn/dlt
that referenced
this pull request
Jul 1, 2026
Add dlt-hub#4147 (injectable access token / Azure TokenCredential for the MS SQL destinations) to the bundled pull request list, and note that dlt-hub#4142 now also carries the concurrent-load safety fix for staging-optimized replace. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sdebruyn
changed the title
sdebruyn
added a commit
to sdebruyn/dlt
that referenced
this pull request
Jul 1, 2026
Add dlt-hub#4147 (injectable access token / Azure TokenCredential for the MS SQL destinations) to the bundled pull request list, and note that dlt-hub#4142 now also carries the concurrent-load safety fix for staging-optimized replace.
sdebruyn
force-pushed
the
feat/mssql-access-token-credential
branch
from
21fd54f to
ca11c36
Compare
This was referenced Jul 1, 2026
Open
…n to two methods Redesign the `authentication` credential option to be driver-aware for the pyodbc driver used at this point in the mssql/synapse/fabric destinations: - Driver-native (passed as `Authentication=` in the DSN, poolable): the ODBC driver itself performs the sign-in. Adds `ActiveDirectoryMsi` alongside the existing `ActiveDirectoryServicePrincipal`, `ActiveDirectoryPassword`, `ActiveDirectoryIntegrated`, `ActiveDirectoryInteractive`. - azure-identity token injection (via pyodbc `attrs_before`, not poolable): narrowed to the two methods pyodbc/msodbcsql cannot perform natively, renamed to match ODBC's own naming convention: `ActiveDirectoryDefault` (DefaultAzureCredential) and `ActiveDirectoryDeviceCode` (DeviceCodeCredential). - The thin `default` alias for `ActiveDirectoryDefault` is kept; `auto` is not carried over since it never existed on `devel` and has no backwards-compatibility reason to exist here (PR1 would have been the one introducing it). - Removes the other dlt-custom lowercase names (`cli`, `environment`, `interactive`, `devicecode`, `msi`, `managedidentity`): the native `ActiveDirectory*` names now cover Interactive/DeviceCode/Msi directly, and `cli`/`environment` are covered by `ActiveDirectoryDefault`, whose `DefaultAzureCredential` chain already includes the Azure CLI and environment credentials. Updates the mssql/fabric/synapse destination docs and the offline mssql/fabric configuration unit tests to match.
Add two ways to bypass `authentication` resolution entirely on the shared MsSqlCredentials (mssql/synapse) and on FabricCredentials, in precedence order: - `access_token`: a pre-fetched Entra ID bearer token, injected as-is via attrs_before. - `azure_credential`: an externally constructed azure-identity TokenCredential (e.g. DefaultAzureCredential()), injected at runtime only -- marked NotResolved so dlt never attempts to read it from a config provider. dlt calls its get_token() to acquire the token. When either is set, dlt skips the DefaultAzureCredential fallback and the `authentication` named-method resolution, and the DSN omits AUTHENTICATION/UID/PWD, mirroring the existing token-method behavior. Validation no longer requires an `authentication` method or Service Principal fields in that case. This precedence check is layered on top of the driver-native/azure-identity-injection taxonomy from the auth redesign (access_token/azure_credential are checked first in every touched function; everything below falls through to the unchanged named-`authentication` resolution). Also harden Fabric's `_ensure_fabric_token_initialized` (COPY INTO staging path) to skip proactive token initialization when the staging Service Principal credentials carry no secret, instead of letting ClientSecretCredential raise before any data moves.
sdebruyn
force-pushed
the
feat/mssql-access-token-credential
branch
from
ca11c36 to
e474e7a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Building on #4140, this PR lets you hand a Microsoft Entra ID access token directly to the MS SQL family destinations (mssql, synapse, fabric), instead of having dlt acquire one through a named authentication method. Two new credential fields:
access_token: a pre-acquired AAD bearer token. dlt injects it into the ODBC connection as is and acquires nothing itself.azure_credential: anazure.core.credentials.TokenCredentialobject. dlt callsget_token("https://database.windows.net/.default")on it to obtain and refresh the token. This fits hosts that already expose a credential, such as a Microsoft Fabric notebook running under the user's own identity.When either field is set, dlt skips the named
authenticationresolution and the DefaultAzureCredential fallback, and leaves AUTHENTICATION, UID and PWD out of the connection string.access_tokentakes precedence overazure_credential, which takes precedence over theauthenticationmethod.This PR also hardens the Fabric staging token setup: when no service principal secret is configured it now returns early instead of constructing a credential that raises before any data moves.
Both changes are adapted from #3872 onto the shared
MsSqlCredentialsintroduced in #4140.Related Issues
This PR adds no new behavior that closes an issue on its own; it extends the authentication work in #4140 (issues #2059 and #4145) and ports the access-token and credential-injection ideas from #3872.
Additional Context