Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion airflow-core/adr/lang-sdk/0003-pure-java-dags.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@

## Status

Proposed
Accepted

> **Note:** This ADR describes pure Java DAG authoring (entire DAGs written in Java without a
> Python file), which was removed from the scope of
Expand Down
10 changes: 5 additions & 5 deletions airflow-core/adr/lang-sdk/0004-dag-parsing.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@

## Status

Proposed
Accepted

> **Note:** This ADR describes language-native DAG file parsing, which was removed from the
> scope of [AIP-108](https://cwiki.apache.org/confluence/x/pY4mGQ). Per AIP-108, Java tasks are
Expand All @@ -40,7 +40,7 @@ This ADR details the DAG parsing side of the coordinator architecture described

### Extension Point: `BaseCoordinator`

A single abstract base class — `BaseCoordinator` — handles both DAG parsing and task execution. Concrete subclasses ship as standalone distributions (`apache-airflow-coordinators-<lang>`) under the shared `airflow.sdk.coordinators` namespace package; they are **not** Airflow providers and are not registered through `provider.yaml`. For DAG parsing, a subclass must implement two methods:
A single abstract base class — `BaseCoordinator` — handles both DAG parsing and task execution. Concrete subclasses ship as part of the Task SDK (`apache-airflow-task-sdk`) under the shared `airflow.sdk.coordinators` namespace package (see [ADR-0005](0005-coordinator-packaging.md)); they are **not** Airflow providers and are not registered through `provider.yaml`. For DAG parsing, a subclass must implement two methods:

| Method | Signature | Responsibility |
|---|---|---|
Expand Down Expand Up @@ -279,10 +279,10 @@ For DAG parsing, a new language provider needs:

**JavaCoordinator:**

The Java SDK implements all DAG-parsing contracts in a single `BaseCoordinator` subclass shipped as `apache-airflow-coordinators-java`:
The Java SDK implements all DAG-parsing contracts in a single `BaseCoordinator` subclass shipped in the Task SDK (`apache-airflow-task-sdk`):

```python
# Distribution: apache-airflow-coordinators-java
# Distribution: apache-airflow-task-sdk
# Module: airflow.sdk.coordinators.java.coordinator
class JavaCoordinator(BaseCoordinator):
def __init__(self, *, name, java_executable="java", jvm_args=None, jdk_home=None):
Expand Down Expand Up @@ -408,7 +408,7 @@ Both share test cases defined in `test_dags.yaml`, ensuring the Java SDK produce

## Consequences

- The DAG file processor can be extended to any language without modifying Airflow Core — only a `BaseCoordinator` subclass distributed as `apache-airflow-coordinators-<lang>` plus an entry in `[sdk] coordinators` is needed.
- The DAG file processor can be extended to any language without modifying Airflow Core — only a `BaseCoordinator` subclass (packaged per [ADR-0005](0005-coordinator-packaging.md)) plus an entry in `[sdk] coordinators` is needed.
- The language runtime must produce exact DagSerialization v3 JSON, requiring cross-language validation infrastructure (e.g., `test_dags.yaml` + `compare.py`).
- The base class absorbs all TCP/process plumbing, so language providers only implement two methods for DAG parsing.
- The subprocess bridge adds latency and a process boundary; DAG parsing for non-Python files is inherently slower than in-process Python parsing.
12 changes: 6 additions & 6 deletions airflow-core/adr/lang-sdk/0005-coordinator-packaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,13 @@

## Status

Proposed
Accepted

> **Note:** This ADR describes coordinator packaging as a standalone distribution separate
> from the Task SDK. The current plan is to ship coordinators as part of the Task SDK
> (`apache-airflow-task-sdk`); a separate distribution may be introduced later once the
> coordinator interface exits experimental status, but is not committed. This document is
> retained for reference if that split is revisited. Tracked operationally in
> **Note:** An earlier revision of this ADR proposed coordinator packaging as a standalone
> distribution separate from the Task SDK. The accepted decision ships coordinators as part
> of the Task SDK (`apache-airflow-task-sdk`); a separate distribution may be introduced
> later once the coordinator interface exits experimental status, but is not committed.
> Tracked operationally in
> [apache/airflow#66451](https://github.com/apache/airflow/issues/66451).

## Context
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@

## Status

Proposed
Accepted

> **Note:** This ADR records a *negative* decision. It exists so that future feature requests
> or PRs proposing multi-file / multi-language source display for stub-backed Dags can be
Expand Down