Feature description
Enable the staging-optimized replace strategy for the fabric destination, backed by DDL transactions and ALTER SCHEMA ... TRANSFER.
Use case
The fabric destination inherits its capabilities from synapse, which has supports_ddl_transactions = False and does not offer staging-optimized (Synapse dedicated SQL pools support neither). Microsoft Fabric Warehouse does support DDL transactions and ALTER SCHEMA ... TRANSFER.
For a full-table replace, that makes the operation a metadata-only schema swap instead of a truncate-and-insert or insert-from-staging copy. That is what staging-optimized is for, and it is much faster on large tables. Today Fabric users cannot select it.
The Fabric docs already describe staging-optimized as using ALTER SCHEMA ... TRANSFER and state that Fabric supports DDL transactions, so the current behavior also contradicts the documentation.
Proposed solution
Have the fabric destination advertise and implement staging-optimized. Two parts are needed; flipping the capability flags alone is not enough:
- advertise
supports_ddl_transactions = True and add staging-optimized to supported_replace_strategies;
- override
_create_replace_followup_jobs on the Fabric client. SynapseClient deliberately bypasses the mssql ALTER SCHEMA ... TRANSFER replace job and always falls back to the generic truncate-and-insert job. Without the override, staging-optimized would be advertised but silently behave like insert-from-staging.
Related issues
N/A
Feature description
Enable the
staging-optimizedreplace strategy for thefabricdestination, backed by DDL transactions andALTER SCHEMA ... TRANSFER.Use case
The
fabricdestination inherits its capabilities fromsynapse, which hassupports_ddl_transactions = Falseand does not offerstaging-optimized(Synapse dedicated SQL pools support neither). Microsoft Fabric Warehouse does support DDL transactions andALTER SCHEMA ... TRANSFER.For a full-table
replace, that makes the operation a metadata-only schema swap instead of a truncate-and-insert or insert-from-staging copy. That is whatstaging-optimizedis for, and it is much faster on large tables. Today Fabric users cannot select it.The Fabric docs already describe
staging-optimizedas usingALTER SCHEMA ... TRANSFERand state that Fabric supports DDL transactions, so the current behavior also contradicts the documentation.Proposed solution
Have the
fabricdestination advertise and implementstaging-optimized. Two parts are needed; flipping the capability flags alone is not enough:supports_ddl_transactions = Trueand addstaging-optimizedtosupported_replace_strategies;_create_replace_followup_jobson the Fabric client.SynapseClientdeliberately bypasses the mssqlALTER SCHEMA ... TRANSFERreplace job and always falls back to the generic truncate-and-insert job. Without the override,staging-optimizedwould be advertised but silently behave likeinsert-from-staging.Related issues
N/A