You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#67056 decoupled remote logging from the hardcoded branches in airflow_local_settings.py: core and
the Task SDK now resolve the handler via ProvidersManager dispatch on the [logging] remote_base_log_folder URL scheme, instantiating the provider class through a no-arg from_config() classmethod. This issue migrates the wasb scheme.
What
Add WasbRemoteLogIO.from_config() in providers/microsoft/azure/src/airflow/providers/microsoft/azure/log/wasb_task_handler.py,
mirroring the legacy branch:
— including the [logging] remote_task_handler_kwargs IO-kwargs merge, expanduser on base_log_folder, reading [azure_remote_logging] remote_wasb_log_container (fallback airflow-logs) into wasb_container, and stripping the wasb:// prefix from remote_base_log_folder, so behavior is unchanged for existing configs.
Register the wasb scheme under a remote-logging: section in providers/microsoft/azure/provider.yaml and mirror it in providers/microsoft/azure/src/airflow/providers/microsoft/azure/get_provider_info.py.
Add tests mirroring TestS3RemoteLogIOFromConfig in providers/amazon/tests/unit/amazon/aws/log/test_s3_task_handler.py.
Verify end to end with a real system test: set up the actual backend yourself (a real
service or a local equivalent), run a task with remote logging enabled, and confirm logs are
uploaded and read back through the new dispatch path. Include the setup and verification
results in the PR description.
Notes
The legacy branch matches any value starting withwasb — including non-URI values like wasb-logs. Scheme dispatch only fires for proper wasb://... URIs (a value without :// has no
URL scheme), so non-URI values keep resolving through the legacy fallback until the chain is
removed. The wasb:// URI form should be documented as the supported form going forward; migrating
non-URI configs can be handled in the deprecation step tracked by the meta issue.
Reference
Merged examples to follow: #69817 (s3), #69816 (cloudwatch). If from_config raises on a bad
config, the shared factory falls back to the legacy path, so this is not a breaking change.
The legacy branch derives a default connection id from WasbHook.default_conn_name; handle the
connection id the same way the merged amazon PRs do (the hook reads [logging] remote_log_conn_id itself).
Part of #70265 (related: #67056).
Why
#67056 decoupled remote logging from the hardcoded branches in
airflow_local_settings.py: core andthe Task SDK now resolve the handler via
ProvidersManagerdispatch on the[logging] remote_base_log_folderURL scheme, instantiating the provider class through a no-argfrom_config()classmethod. This issue migrates thewasbscheme.What
WasbRemoteLogIO.from_config()inproviders/microsoft/azure/src/airflow/providers/microsoft/azure/log/wasb_task_handler.py,mirroring the legacy branch:
airflow/airflow-core/src/airflow/config_templates/airflow_local_settings.py
Lines 226 to 248 in 104ad12
— including the
[logging] remote_task_handler_kwargsIO-kwargs merge,expanduseronbase_log_folder, reading[azure_remote_logging] remote_wasb_log_container(fallbackairflow-logs) intowasb_container, and stripping thewasb://prefix fromremote_base_log_folder, so behavior is unchanged for existing configs.wasbscheme under aremote-logging:section inproviders/microsoft/azure/provider.yamland mirror it inproviders/microsoft/azure/src/airflow/providers/microsoft/azure/get_provider_info.py.TestS3RemoteLogIOFromConfiginproviders/amazon/tests/unit/amazon/aws/log/test_s3_task_handler.py.service or a local equivalent), run a task with remote logging enabled, and confirm logs are
uploaded and read back through the new dispatch path. Include the setup and verification
results in the PR description.
Notes
The legacy branch matches any value starting with
wasb— including non-URI values likewasb-logs. Scheme dispatch only fires for properwasb://...URIs (a value without://has noURL scheme), so non-URI values keep resolving through the legacy fallback until the chain is
removed. The
wasb://URI form should be documented as the supported form going forward; migratingnon-URI configs can be handled in the deprecation step tracked by the meta issue.
Reference
Merged examples to follow: #69817 (s3), #69816 (cloudwatch). If
from_configraises on a badconfig, the shared factory falls back to the legacy path, so this is not a breaking change.
The legacy branch derives a default connection id from
WasbHook.default_conn_name; handle theconnection id the same way the merged amazon PRs do (the hook reads
[logging] remote_log_conn_iditself).