docs: align dynamic chain example with preceding static example#2
Open
jonathankibg wants to merge 153 commits into
Open
docs: align dynamic chain example with preceding static example#2jonathankibg wants to merge 153 commits into
jonathankibg wants to merge 153 commits into
Conversation
- Introduced new data models: CalendarDeadlineResponse and CalendarDeadlineCollectionResponse for handling aggregated deadline counts.
- Implemented the GET /ui/calendar/{dag_id}/deadlines endpoint to retrieve deadline data, including parameters for granularity and deadline time filters.
- Updated CalendarService to include a method for fetching deadline calendar data, aggregating results by time bucket and missed status.
- Added unit tests for the new endpoint to ensure correct functionality and response structure.
…ache#69427) The Kafka provider already injects an ``oauth_cb`` token callback for Google Managed Kafka based on the bootstrap servers hostname. Amazon MSK offers the same OAUTHBEARER-based IAM authentication (and MSK Serverless supports IAM only), but there was no equivalent branch, forcing users to wire the token signer by hand. This adds a symmetric branch: when the bootstrap servers point at an Amazon MSK endpoint (``*.kafka.<region>.amazonaws.com`` or ``*.kafka-serverless.<region>.amazonaws.com``, including ``.cn`` regions) and ``sasl.mechanism`` is ``OAUTHBEARER``, the hook derives the AWS region from the hostname and installs an ``oauth_cb`` backed by ``aws_msk_iam_sasl_signer.MSKAuthTokenProvider``. The callback converts the signer's millisecond expiry to the seconds expected by confluent-kafka. An explicit user-provided ``oauth_cb`` is never overwritten, and a missing signer library raises ``AirflowOptionalProviderFeatureException`` pointing at the new ``msk`` extra.
…ache#69495) The database session in the FastAPI layer is injected via the SessionDep dependency, which automatically commits the transaction when a request completes successfully. Explicitly calling session.commit() inside the route handler is redundant and circumvents the dependency's intended transaction lifecycle management. This brings the route handlers in line with the codebase guideline that prohibits functions accepting a session parameter from calling commit.
…he#69179) Signed-off-by: PoAn Yang <payang@apache.org>
…pache#69299) Reading a running task's log through an executor materializes the whole log in the API server before the bounded LogStreamAccumulator can bound memory, so large logs spike the API server heap. This adds an interface executors can implement to stream log lines lazily instead.
PR apache#67217 wired version_data into the task execution path but not callbacks, so callbacks for a pinned run initialized their bundle without the manifest tasks used. Populate version_data on the callback producer paths (ExecuteCallback.make and the CallbackRequest creation sites) under the same pin guard as tasks, carry it on BaseCallbackRequest, and forward it through prepare_callback_bundle to get_bundle.
…pache#69583) * Make ts-sdk supervisor schema check a manual prek hook CI's static-checks job runs `prek --all-files`, which runs every default-stage hook against the whole repo tree regardless of what a given PR actually touched. Because check-ts-sdk-supervisor-schema lists task-sdk's schema.json as a trigger file, any task-sdk-only PR that changes the wire schema fails static checks over an unrelated, not-yet-regenerated ts-sdk file it never intended to touch. Move the hook to the manual stage, matching how other on-demand checks in this file (update-notice-year, upgrade-important-versions, update-docker-gpg-keys) are handled: it stops gating unrelated PRs and ts-sdk owners run it themselves (`prek run --hook-stage manual check-ts-sdk-supervisor-schema`) when they pick up a schema change. * Selectively skip ts-sdk supervisor schema check instead of manual stage The first attempt made check-ts-sdk-supervisor-schema a manual-stage hook, but review found that removes it from CI entirely (nothing invokes it at the manual stage), so schema drift would merge undetected. Reuse the existing selective-skip mechanism already used for ktlint and the mypy-* hooks instead: the hook keeps running in CI's default static-checks sweep, but is now skipped when neither ts-sdk/ nor the supervisor wire schema changed, so unrelated task-sdk-only PRs no longer fail on it while real drift is still caught. * Skip ts-sdk supervisor schema check unless ts-sdk itself changes A task-sdk PR that bumps the supervisor wire schema should not be responsible for regenerating the ts-sdk types - that is the ts-sdk follow-up PR's job. Trigger the hook only on ts-sdk/ changes, both in CI (selective skip) and locally (the hook's files pattern), so schema authors are never blocked on a file they do not own. Drift merged via a schema-only PR is still caught by full-tests runs (canary and structural-change PRs run every hook) and by the next ts-sdk PR. * Regenerate ts-sdk supervisor types for asset event extra fields The wire schema gained an extra field on GetAssetEventByAsset and GetAssetEventByAssetAlias in apache#69453 without the ts-sdk types being regenerated. This PR's own changes to dev/breeze and the prek config force a full static-checks run, where no hook is skipped, so the check correctly surfaced that pre-existing drift and blocked CI.
* Fix flaky KubernetesPodOperator log-timestamp test The parametrize data for test_fetch_container_logs_before_current_sec_various_logs built each log line's timestamp and the test's `now` value from separate pendulum.now() calls at collection time. When those calls straddled a second boundary, the "timestamp equal to the current second" cases (3 and 6) would intermittently fail because the values were no longer equal to the second. Build timestamps as offsets from a single fixed reference instant instead. * Sync Apache Kafka provider docs index with its msk extra The msk extra was added to providers/apache/kafka/pyproject.toml without regenerating docs/index.rst, so the extras table was missing the corresponding row. CI's doc-sync check flags this drift on every PR touching this branch, regardless of relevance, so fix it here to unblock.
…e#69519) * Let area:kubernetes-tests label force the Kubernetes tests job Contributors validating a change whose impact selective-checks' file-path heuristics miss previously had to reach for `full tests needed`, which pulls in the entire test matrix. Reusing the existing `area:kubernetes-tests` triage label lets them force just the Kubernetes tests job instead. * Add a test for the kubernetes-tests label with no changed files Covers a label-only trigger (e.g. added after the fact to an existing PR) to confirm the forced job doesn't depend on the file-based heuristics also matching.
…#69272) Add a `federated_token_provider` connection extra: a dotted-path callable that supplies a short-lived OIDC JWT in-process, which the hook exchanges for a Databricks OAuth token via the same RFC 8693 endpoint the `federated_k8s` path uses. Works with any federation-trusted issuer and in any environment, not only Kubernetes. `client_id` is optional (account-wide policy) or supplied (service principal policy). The Kubernetes federation path is behaviourally unchanged.
…on (apache#69490) The detached job runs under `setsid` so it leads its own process group, and `on_kill` signals that group. The wrapper recorded `$!` as the group leader, but `$!` is only the leader when `setsid(1)` runs in place. When job control is on in the launching shell, `setsid(1)` forks (`setsid(2)` cannot create a new session from an existing group leader), so `$!` names the short-lived setsid parent, not the job. Cancellation then signals a dead group, the real command keeps running as an orphan, and the `exit_code` file is never written, so the task stays deferred until the trigger times out. Record the job's own pid instead: right after `setsid(2)`, POSIX guarantees `pid == pgid == sid` for the caller and that identity survives the following `exec`, so `$$` written from inside the job script is always the true PGID, whether or not setsid forked. The launcher no longer records `$!`, and since the pid file is read only at cancellation time (long after submission) the job's own write is always in place first. Make the end-to-end kill test deterministic by forcing job control through a real controlling terminal, exercising the exact fork path that orphaned the job, and remove the `flaky(reruns=5)` marker. Test markers are unique per test and per worker so the suite stays isolated under `pytest -n auto`.
apache#68926) durable=True caching is per-task-instance, per-run, and cleared on success -- exactly the scope of the AIP-103 task state store. On Airflow >= 3.3 the cache now lives there instead of an ObjectStorage JSON file, so durable execution no longer needs the [common.ai] durable_cache_path config, and large step payloads are offloaded automatically through the configured worker state store backend. The ObjectStorage backend is kept as the fallback for Airflow < 3.3, selected at runtime by the operator. Both backends share one storage interface, so the caching model/toolset wrappers are unchanged. * Fix common.ai durable CI failures: compat collection, missing test, spelling Three CI failures on this branch, all in the durable execution code: - Compat 3.0.6/3.1.8/3.2.2: ``test_task_state_store`` used ``pytest.importorskip("airflow.sdk.execution_time.context")`` to skip on pre-3.3 cores, but that module exists on older cores -- only the ``NEVER_EXPIRE`` name it pulls in (via ``task_state_store``) is 3.3+. Collection blew up before the skip could fire. Gate on ``AIRFLOW_V_3_3_PLUS`` instead, matching how ``AgentOperator._build_durable_storage`` guards the import. - Non-DB / Low-dep core: ``test_providers_modules_should_have_tests`` flagged the new ``durable/base.py`` as having no test. Add ``test_base.py`` covering the reserved-key invariants and the ``DurableStorageProtocol`` runtime contract. - Docs spellcheck: "sizeable" -> "sizable" in the agent operator docs. * Address review comments on durable storage tests - task_state_store docstring: "older cores" -> "older airflow versions". - test_agent: skip the task-state-store build test on < 3.3 instead of patching AIRFLOW_V_3_3_PLUS -- the inline 3.3-only import would otherwise fail on the compat runners once collection is fixed. - test_agent: give the accessor a TaskStateStoreAccessor spec and the task instance an attribute spec; hoist the version-agnostic DurableStorage import. - test_base: assert the two shipped backends (ObjectStorage always, task state store on 3.3+) satisfy DurableStorageProtocol, to catch method drift. * Harden durable task-state-store saves from code review - save_tool_result: normalize results through JSON before writing. The store validates against pydantic JsonValue (stricter than json.dumps -- rejects tuples and non-string dict keys), so a tool returning `(value, meta)` or an int-keyed dict previously crashed a step that already succeeded. Normalizing (tuple->list, non-str keys->str) matches the <3.3 ObjectStorage backend and keeps the step durable. - save_model_response / save_tool_result: make the store write best-effort -- a failed write (e.g. an oversize value on the metadata DB) is skipped with a warning instead of failing an already-succeeded step and self-poisoning retry. - cleanup: log delete failures instead of silently suppressing, so a value orphaned in external storage is at least visible. - test: make FakeTaskStateStore mirror the real contract (reject None, validate JsonValue) and add tuple / non-string-key regression tests it now catches. - test_base: trim to the real-backend protocol-conformance tests + key-prefix invariant; drop the tests that only exercised typing.Protocol machinery. - docs: note durable keys carry the reserved DURABLE_KEY_PREFIX.
Airflow 3 hid the conf column even though the API and column definition already existed. Operators triggering Dags with run conf need that context in the list view without toggling columns manually. closes: apache#53382
…gap (apache#69582) * Exclude the full Gradle wrapper from the Java SDK source release Only gradle-wrapper.jar was excluded from the sourceRelease tarball (LEGAL-570), but gradlew, gradlew.bat, and gradle-wrapper.properties still shipped, drawing a binding -1 in the 1.0.0-beta1 RC2 vote. The wrapper stays tracked in git; only the git-archive output used by sourceRelease now omits it. Document both ways to build an extracted source package without the wrapper, add a release-verification checklist, and add a vote-email template with the full 8-artifact staged list so future RCs don't repeat the RC2 feedback. * Add LICENSE/NOTICE and reproducible archives to Java SDK jars Staged convenience-binary jars (main, sources, javadoc, test-fixtures) were missing META-INF/LICENSE and META-INF/NOTICE, violating ASF release policy for distributed binaries. Builds were also not byte-reproducible: jar entries carried wall-clock timestamps, blocking GitHub-driven publishing. Centralizing both fixes in the airflow-jvm-conventions plugin, applied by every jar-producing module (sdk, jul, log4j2, slf4j, jpl, processor, plugin), covers all modules without per-module duplication. * Manage airflow-sdk-jpl version in the Java SDK BOM The BOM constrained sdk, jul, log4j2, processor, and slf4j but not jpl, even though jpl is published to Nexus and the bundled example module depends on it. Consumers pulling in jpl via the BOM alone had no version constraint to resolve against. * Manage airflow-sdk-gradle-plugin version in the Java SDK BOM airflow-sdk-gradle-plugin publishes under the same org.apache.airflow Maven coordinate convention as the other SDK artifacts, but was left out of the BOM's constraints when airflow-sdk-jpl was added — the same gap the jpl fix was closing. Consumers importing airflow-sdk-bom still got no version pin for the plugin artifact. * Use rc<N> placeholder in Java SDK release verification clone example * Keep wrapper properties in the archive * Add verification in BOM to not miss subprojects * Touch up on Java SDK verification process --------- Co-authored-by: TP <uranusjr@apache.org>
Around 3.2/3.3 we refactored how otel traces were emitted, meaning these columns were unused, but we didn't remove them at the time.
…ache#68882) * fix(providers/airbyte): migrate from requests.Session to httpx.Client for airbyte-api>=1.0 airbyte-api 1.0.0rc2 switched its HTTP layer from requests to httpx. The client parameter now requires an httpx-compatible HttpClient protocol and no longer accepts requests.Session. Changes: - Replace requests.Session proxy setup with httpx.Client(mounts=...) - Update pyproject.toml: airbyte-api>=1.0.0rc2, httpx>=0.28.1, drop requests - Update test to assert isinstance(client, httpx.Client) instead of checking .proxies attribute (not available on httpx.Client) Co-Authored-By: AJ Steers <aj@airbyte.io> * fix: move httpx import to top-level in test file Co-Authored-By: AJ Steers <aj@airbyte.io> * fix: strengthen proxy test and bump airbyte-api lower bound to >=1.0.0 - Test now verifies proxy transport mounts are configured per-scheme - Bump airbyte-api lower bound from >=1.0.0rc2 to >=1.0.0 (GA released) Co-Authored-By: AJ Steers <aj@airbyte.io> * Fix test compatibility with airbyte-api 1.0 Pydantic models Co-Authored-By: AJ Steers <aj@airbyte.io> * Regenerate provider docs index and add 1.0 migration changelog note Co-Authored-By: AJ Steers <aj@airbyte.io> * Regenerate airbyte provider docs index after merging main Co-Authored-By: AJ Steers <aj@airbyte.io> * Bump airbyte provider to 6.0.0 for breaking airbyte-api 1.0 / httpx migration Co-Authored-By: AJ Steers <aj@airbyte.io> * Revert manual airbyte version bump and reframe changelog note as non-breaking Per reviewer feedback, this is not an Airflow-code breaking change, so the version stays release-manager-managed (5.5.2) rather than a hardcoded 6.0.0 bump. The manual 6.0.0 entry in provider.yaml also broke the prepare-provider-documentation CI step, which diffs against the previous version tag (providers-airbyte/5.5.2) that is not yet released. Retitle the changelog section from 'Breaking changes' to 'Transitive dependency changes' and drop the 6.0.0 header. Co-Authored-By: AJ Steers <aj@airbyte.io> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* i18n(ko): Add Korean translations for Dags list run state counts * i18n(ko): Add missing Korean translations for preset filters and variable import
* Add Beeline JDBC parameters to Hive CLI * Address Hive jdbc_params review: drop transport_mode special-casing Remove the transport_mode connection extra and form widget so transportMode is supplied like any other parameter via jdbc_params, the single validated injection point. Consolidate the six JDBC-param helpers into one _append_jdbc_params method that validates name and value in a single pass, and tighten the parameter-name regex to reject names ending in a separator. Update the connection docs accordingly and drop the manual changelog entry (provider changelogs are regenerated from git log by the release manager). * Validate Hive JDBC parameter values
Bumps the fab-ui-package-updates group with 1 update in the /providers/fab/src/airflow/providers/fab/www directory: [prettier](https://github.com/prettier/prettier). Updates `prettier` from 3.9.4 to 3.9.5 - [Release notes](https://github.com/prettier/prettier/releases) - [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md) - [Commits](prettier/prettier@3.9.4...3.9.5) --- updated-dependencies: - dependency-name: prettier dependency-version: 3.9.5 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: fab-ui-package-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
The missing f-string prefixes caused Cloud Batch errors to display the
literal `{job_name}`.
…opic provider docs (apache#69709) The Managed Agents docs URL now returns 404, and the connections page note about first-party-only features omitted Managed Agents even though the hook enforces the same restriction on it.
Co-authored-by: Maksim Yermakou <maksimy@google.com>
* Fix airflowctl JSON parsing for Dag run conf
airflowctl dags trigger --conf rejected JSON object strings because the
generated parser validated them with raw dict, which cannot consume a JSON
string. Parse JSON object CLI values into dictionaries before validation so
conf values such as {"my-key": "my-value"} are accepted.
closes: apache#67989
* Expand json_dict_type test coverage for airflowctl conf parsing
Cover dict input early return, invalid JSON, and non-object JSON so all
branches of the conf parser are exercised.
---------
Co-authored-by: Atakan Yenel <atakanyenel@microsoft.com>
…9633) Add a new reserved key to dagrun conf, `airflow/parent_trace_context`. When set with a W3C traceparent, the dag run span will be nested under this trace.
* Add Common Data Quality provider skeleton * Fixup tests * remove hash ref from .gitignore * Fixup docs * Resolve comments
* Fix airflow db clean hang on MySQL when delete fails (apache#66177) * Rename newsfragment to match PR number The check-newsfragment-pr-number CI job requires the newsfragment filename to match the PR number, not the linked issue number. * Remove newsfragment per review * Suppress rollback errors so the original delete error propagates * Explain why the archive table drop reuses the session connection * Force the FK-failure path in backend regression tests via config copy Keeps the hang-regression tests meaningful once dag_version cleanup learns to skip FK-pinned rows (apache#68339). * Guard archive-table cleanup so it can't mask the original delete error When _do_delete unwinds from a failed DELETE, dropping the archive table in the finally block could itself raise (e.g. a dead connection on MySQL). Python makes a finally-raised error the top-level exception, so that cleanup error would replace the original delete failure -- and an OperationalError from the drop could then be downgraded to a warning by _suppress_with_logging, silently masking the real IntegrityError. Guard the cleanup so it only propagates on the success path; on the failure path it is logged and the original error survives. * Update airflow-core/tests/unit/utils/test_db_cleanup.py Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com> * Retry CI after provider compat startup failure The previous run failed because the Mongo provider test fixture could not connect to its temporary mongod container. No source change is needed; this commit retriggers CI so the PR can use a fresh signal. --------- Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com> Co-authored-by: Ramachandra Nalam <nalamvamsi13@gmail.com>
…69863) thrift 0.24.0 (apache/thrift#3584) is incompatible with the databricks-sql-connector thrift<=0.23.0 requirement, so during highest resolution the resolver preferred the newest thrift and downgraded the connector, which in turn dragged the databricks provider back to an old version. Pinning the connector to >=4.0.0 keeps PyPI constraints installable with a current databricks-sql-connector and provider. closes: apache#69603
…e#69866) Co-authored-by: Wei Lee <weilee.rx@gmail.com> Signed-off-by: PoAn Yang <payang@apache.org>
…69738) Signed-off-by: PoAn Yang <payang@apache.org>
Expanding a mapped task creates the N downstream task instances inside `DagRun.update_state`. Two `N+1` query patterns there turned a wide fan-out into a scheduler stall that scaled linearly with map width: a per-index `session.merge()` to persist each new instance, and a per-index `dag_run` `SELECT` when dependency evaluation later called `get_dagrun()` on instances whose dag_run relationship was never loaded. Persist the instances with a batched `session.add()`/`flush()` and prime the shared `dag_run` relationship once during expansion, so both scale with a constant number of queries instead of with map width. We could have also defensively added a `set_committed_value` on the dagrun attr for the index-0 case but it's not necessary as is because the dag run object is cached for that reused TI object.
…69846) * Allow `AthenaOperator` queries without a `database` argument Athena does not require the `Database` field in `QueryExecutionContext`. Queries with fully qualified table names can therefore run without a default database. https://docs.aws.amazon.com/athena/latest/APIReference/API_StartQueryExecution.html#API_StartQueryExecution_RequestSyntax Requiring the argument forces users to provide a default database even when the query does not need one. * Use `Database` in `query_execution_context` for OpenLineage datasets When the `database` argument of `AthenaOperator` is `None`, the `Database` field in `QueryExecutionContext` can still provide the default database for unqualified table names in a query. * Clarify `AthenaOperator` database fallback behaviour in its docstring Suggested in apache#69846 (comment) It is better to explain to users who omit `database` or set it to `None` that `Database` in `query_execution_context` remains effective.
…pache#69890) RedshiftCreateClusterOperator defaults publicly_accessible=True, so the example_sql_to_s3 and example_s3_to_sql system tests created Redshift clusters with public endpoints, triggering recurring AppSec Publicly Accessible Database findings. Set publicly_accessible=False to match example_redshift and example_redshift_s3_transfers, which already set it. Note: these two tests, unlike example_redshift*, open a direct SQL connection to the cluster endpoint (SqlToS3Operator / S3ToSqlOperator via a redshift connection built from describe_clusters). With a non-public endpoint the test worker must reach the cluster from inside the VPC; verify the runner has in-VPC network access before relying on this change. Co-authored-by: Sean Ghaeli <ghaeli@amazon.com>
Signed-off-by: PoAn Yang <payang@apache.org>
* Pin lang-SDK k8s test's Go/Java SDKs to upstream main Release/backport branches often lack go-sdk/java-sdk entirely or carry a stale, branch-cut-frozen copy, so backporting a core/task-sdk fix to such a branch would otherwise run the lang-SDK coordinator test against a missing or outdated SDK snapshot. The Go bundle and Java jar builds now always fetch go-sdk/java-sdk from upstream main regardless of the checked out branch, while airflow-core/task-sdk and the test's own harness fixtures keep tracking whatever branch is checked out. Fix lang-SDK Java build missing task-sdk sibling in native mode java-sdk's sdk/build.gradle.kts reads a sibling ../task-sdk/.../schema.json at build time. The upstream-main extraction only contains go-sdk/java-sdk, so building from it directly (native/CI toolchain mode) failed looking for a task-sdk directory that doesn't exist. Symlink the real, local task-sdk alongside the extraction so the reference resolves, without pulling task-sdk itself from upstream. Restore lang-SDK k8s test's gradle wrapper jar dropped by export-ignore java-sdk/.gitattributes marks gradle/wrapper/gradle-wrapper.jar export-ignore so ASF source-release tarballs stay free of compiled binaries (LEGAL-570). git archive, used to pull go-sdk/java-sdk out of upstream main for this test, respects that attribute and silently drops the jar, leaving the extracted java-sdk's ./gradlew unable to find itself and failing every build. Shorten gradle-wrapper.jar restoration comments * Restore gradle wrapper scripts dropped by export-ignore in lang-SDK k8s test Upstream main's java-sdk/.gitattributes export-ignores gradlew and gradlew.bat (ASF LEGAL-570) in addition to the wrapper jar, so the git-archive extraction of upstream sources no longer contains the wrapper script the Java build invokes, failing CI with FileNotFoundError: './gradlew'. Restore all three wrapper files from the pinned commit instead of only the jar. * Fix breeze k8s setup-lang-sdk-test crash with --dry-run In dry-run mode run_command skips execution and returns stdout="" (a str), so every filesystem step consuming a command's output crashed: write_bytes on the gradle wrapper restore raised TypeError, the Go bundle copytree/copy raised FileNotFoundError, the Java jar glob hit sys.exit(1), and the artifact upload raised StopIteration. Guard those steps so --dry-run prints the full provisioning command sequence and completes cleanly.
apache#69522) When ObjectStoragePath is used with a sharepoint/msgraph protocol and a connection that has client_id, client_secret, and tenant_id but no explicit token_endpoint, MSGDriveFS fails because it receives a non-None oauth2_client_params dict from Airflow and skips its own token_endpoint construction logic, leaving AsyncOAuth2Client.fetch_token() without a target URL. Fix: if token_endpoint is not in oauth2_client_params and tenant_id is set, construct the default Microsoft identity platform v2.0 token endpoint: https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token Co-authored-by: Anmol Mishra <anmol.mishra@hellositi.ai>
…tch (apache#69306) * Prevent malformed Elasticsearch log entries from crashing task log fetch A single stored log entry with a non-string `event` field (for example, a task that logs a list or dict as the sole message argument) currently crashes the entire task-log-fetch request with an unhandled pydantic.ValidationError, instead of degrading gracefully. _read() built StructuredLogMessage objects from stored Elasticsearch hits without catching validation failures. This catches ValidationError per hit and falls back to a stringified event, matching the existing fallback pattern in file_task_handler.py's _log_stream_to_parsed_log_stream. closes: apache#69304 * Fix Elasticsearch compat tests on Airflow 2.11 The new StructuredLogMessage fallback tests exercise an Airflow 3-only type, but the provider compatibility matrix still runs this provider against Airflow 2.11.1. Guarding those tests the same way as the runtime path keeps the compatibility job focused on behavior that actually exists in that release line. * Fix Elasticsearch compat test for finished tasks The malformed log-entry regression test was unintentionally exercising Airflow 3's live-log delegation path because the test task instance was still running. Marking the task instance successful keeps the test on the Elasticsearch-read path it is intended to cover, which matches the provider compatibility and lowest-dependencies jobs. * Lower malformed-log fallback logging to debug level _safe_build_structured_log_message runs once per stored hit inside the _read comprehension, so logging the fallback at warning level fires once per malformed line within a single log-fetch request. Drop it to debug so a request with many malformed entries does not flood the logs.
The java_sdk e2e mode and the lang-SDK k8s test solved the same cold-build problem two different ways (containerized Gradle with a repo-local cache vs host toolchain with salted home-dir caches), which a review follow-up on apache#69239 asked to reconcile. Converge the e2e mode on the apache#69411 native-toolchain convention: CI provisions the JDK with actions/setup-java and caches ~/.gradle, dropping the per-run eclipse-temurin image pull and the --network=host Gradle lock-handover workaround; local runs keep the containerized build so a dev host still needs no JDK.
jonathankibg
force-pushed
the
docs/align-dynamic-chain-example
branch
2 times, most recently
from
July 16, 2026 16:25
eb10b9f to
9e15a2a
Compare
The dynamic example creates five operators whereas the preceding static example chains only four tasks. Update the range so both examples are consistent.
jonathankibg
force-pushed
the
docs/align-dynamic-chain-example
branch
from
July 16, 2026 21:57
9e15a2a to
73752f4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes a small inconsistency in the
chain()documentation example.The preceding example demonstrates:
which chains four tasks.
However, the equivalent dynamic example immediately below creates five tasks:
This change updates the range to create four operators instead, making both examples equivalent and easier to compare.
This is a documentation-only change and does not affect Airflow behavior.
Was generative AI tooling used to co-author this PR?