Add task instance management commands to airflowctl CLI by Suraj-kumar00 · Pull Request #67947 · apache/airflow

Suraj-kumar00 · 2026-06-03T09:37:59Z

Adds task instance management support to the airflowctl CLI, enabling users to get, list, clear, and update task instances directly from the terminal.

Closes: #61547
Related: #66173

Changes

`airflow-ctl/src/airflowctl/api/operations.py`

Added TasksOperations class (matching Airflow core CLI naming) with get, list, clear, and update methods
Restored PluginsOperations class that was lost during merge with main (Added plugins command to airflowctl #64935)

`airflow-ctl/src/airflowctl/api/client.py`

Added tasks property to Client for auto-generated CLI command wiring
Restored plugins property that was lost during merge with main

`airflow-ctl/src/airflowctl/ctl/cli_config.py`

Added "clear" to output_command_list for correct CLI output handling

`airflow-ctl/src/airflowctl/ctl/help_texts.yaml`

Added tasks: section with help text for all four subcommands

`scripts/in_container/run_capture_airflowctl_help.py`

Added "tasks" to the commands list for SVG/help image generation

`airflow-ctl/tests/airflow_ctl/api/test_operations.py`

Added TestTasksOperations with 7 unit tests covering get, list, clear, and update
Restored TestPluginsOperations tests lost during merge

`airflow-ctl-tests/tests/airflowctl_tests/test_airflowctl_commands.py`

Added end-to-end integration tests for all 4 tasks commands using positional arguments

CLI Commands

airflowctl tasks list   <dag_id> <dag_run_id>
airflowctl tasks get    <dag_id> <dag_run_id> <task_id>
airflowctl tasks clear  <dag_id> [--dry-run] [--only-failed] [--task-ids t1,t2]
airflowctl tasks update <dag_id> <dag_run_id> <task_id> [--new-state <state>] [--note <note>]

Important

🛠️ Maintainer triage note for @Suraj-kumar00 · by @potiuk · 2026-07-08 15:38 UTC

Helpful heads-up from the maintainers — please address before this PR can be reviewed:

Helm tests failing (Helm tests / Unit tests Helm: security (K8S 1.30.13)). See the contributor guide.

Full criteria: Pull Request quality criteria.

The ball is in your court — you've been assigned to this PR. Fix the above, then mark it Ready for review.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

Suraj-kumar00 · 2026-06-03T12:34:10Z

Hello @potiuk,
Could you please review now and the CI Pipeline can be run?

Also could you help me get this that do i have to do the testing manually on local? This is taking so much time locally!!!

bugraoz93

There are some foundational things needs changing, thanks for the PR!

potiuk · 2026-06-09T08:28:14Z

@Suraj-kumar00 A few things need addressing before review — see our Pull Request quality criteria.

❌ Pre-commit / static checks. See docs.

No rush.

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

Suraj-kumar00 · 2026-06-09T08:40:45Z

Hi @potiuk, I'll make the changes tonight!

henry3260 · 2026-06-17T07:10:01Z

+                            if (
+                                expanded_parameter in args_dict.keys()
+                                and args_dict[expanded_parameter] is not None
+                            ):
+                                val = args_dict[expanded_parameter]


If I understand correctly, the None filtering is not strictly required for fields where null and “not provided” are equivalent. More importantly, this new path does not protect boolean fields, because the generated CLI currently defaults bool arguments to False. For ClearTaskInstancesBody, some API defaults are True or None (dry_run, only_failed, reset_dag_runs, run_on_latest_version), so airflowctl tasks clear <dag_id> may send explicit False values and override the API defaults.

Yeah, you're right on both counts.

Fixed by changing the default from False to None for bool fields in _create_arg_for_non_primitive_type, so unset flags are now omitted from the request body and the API defaults apply. This also brings the path in sync with the primitive arg path, which already defaulted to None.

Added a regression test (test_command_factory_body_bool_field_defaults_to_none) using ClearTaskInstancesBody to lock it in.

Suraj-kumar00 · 2026-06-17T15:46:47Z

Hello @henry3260,

I'm looking into this. Thanks for the review!

bugraoz93

I agree with Henry's comments and could you please check CI what is failing on the integration tests and static checks

Suraj-kumar00 · 2026-06-21T14:21:22Z

Hi @henry3260 @potiuk,

I have made the changes as per the feedback, could you please review?

Suraj-kumar00 · 2026-06-23T16:56:04Z

Hi @henry3260 @potiuk,

Let me know the feedback once you get the time to look into this.

Thanks!

aaron-y-chen

Hi, thanks for the PR :)

Copilot

Pull request overview

Adds a new airflowctl tasks command group backed by REST API v2 Task Instance endpoints, plus related CLI generation fixes and documentation artifacts for the new commands.

Changes:

Introduces TasksOperations and wires it into the Client to expose tasks get/list/clear/update.
Updates CLI command generation to avoid forcing unset boolean body fields to False, and adds small output/pagination handling tweaks.
Extends help text, help-capture scripting, and adds unit + end-to-end tests for the new tasks commands.

Reviewed changes

Copilot reviewed 9 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
scripts/in_container/run_capture_airflowctl_help.py	Adds `tasks` to the list of commands captured for help-image generation.
airflow-ctl/tests/airflow_ctl/ctl/test_cli_config.py	Updates/extends CLI factory tests to ensure boolean body fields default to `None` and stay unset unless provided.
airflow-ctl/tests/airflow_ctl/api/test_operations.py	Adds unit tests for `TasksOperations` plus restores/adjusts plugin tests and adds pool PATCH behavior tests.
airflow-ctl/src/airflowctl/ctl/help_texts.yaml	Adds help text entries for the new `tasks` subcommands.
airflow-ctl/src/airflowctl/ctl/cli_config.py	Adjusts CLI arg generation/output rules and improves body-param expansion (bool defaults, list CSV parsing).
airflow-ctl/src/airflowctl/api/operations.py	Adds `TasksOperations` and updates pool PATCH behavior to always send `include_deferred`.
airflow-ctl/src/airflowctl/api/client.py	Adds the `tasks` operations property (and restores `plugins`).
airflow-ctl/docs/images/output_tasks.svg	New generated help output image for `airflowctl tasks`.
airflow-ctl/docs/images/output_main.svg	Regenerated main help output image to include the new `tasks` group.
airflow-ctl/docs/images/command_hashes.txt	Updates command hashes to include `tasks` and reflect regenerated outputs.
airflow-ctl-tests/tests/airflowctl_tests/test_airflowctl_commands.py	Adds end-to-end command invocations covering all `tasks` subcommands.

Suraj-kumar00 · 2026-07-06T15:50:08Z

Hello @nailo2c, thanks for the review I'll be pushing the changes soon, and will keep you posted on this.

Suraj-kumar00 · 2026-07-08T19:39:04Z

Hi @potiuk, I'll be able to push the changes tomorrow.
Thanks

Suraj-kumar00 · 2026-07-10T05:36:20Z

Hi @nailo2c @potiuk,

I have made the changes as per the feedback, could you please review?

aaron-y-chen · 2026-07-10T08:27:43Z

Could you address the requested changes from henry3260 and bugraoz93?

ashb · 2026-07-10T11:28:22Z

                        arg_action=argparse.BooleanOptionalAction if annotation is bool else None,  # type: ignore
                        arg_help=f"{field} for {parameter_key} operation",
-                        arg_default=False if annotation is bool else None,
+                        arg_default=None,


Why this change?

This was addressing henry3260's comment: the generated CLI hard-coded False for every bool body flag, so tasks clear sent dry_run=False and only_failed=False, overriding the API defaults of True and turning a dry run into a real clear. Reworked per your other comments: no more None. Each generated bool flag now defaults to its datamodel field's own default (the generated client models carry the API defaults from the OpenAPI spec), falling back to False. Every flag is a concrete True or False, and existing commands are unaffected since their models' bool defaults are already False.

Ah right, taking the default from the API makes some sense.

For what it's worth then, rather than, rather using None as not set, I'd rather we used an explicit NotSet / ARG_NOT_SET sentinel value. I don't know if we do that elsewhere in the airflowctl, but we do have that elsewhere in airflow codebase.

Suraj-kumar00 · 2026-07-12T15:41:57Z

Could you address the requested changes from henry3260 and bugraoz93?

All addressed in the latest push: henry3260's bool-defaults issue is fixed by sourcing flag defaults from the datamodels (see thread above), his pool test question was resolved by reverting that file, and the help images/hashes are regenerated. bugraoz93's requests (positional required params, tasks naming to match core CLI) are in. The branch is also squashed onto latest main for a clean diff. One open question remains on the operations class naming, see ashb's thread.

Copilot

Pull request overview

Copilot reviewed 9 out of 11 changed files in this pull request and generated 2 comments.

+def _get_bool_flag_default(field_info: Any) -> bool:
+    """Return the CLI default for a generated bool flag: the datamodel field's default (the API default), or False."""
+    return field_info.default if isinstance(field_info.default, bool) else False


+            # datamodel default is None (fallback semantics), so the flag falls back to False
+            "--run-on-latest-version": False,
+        }


BobDu · 2026-07-20T09:30:18Z

cc @ashb @bugraoz93 @potiuk @Suraj-kumar00

Following up on the naming discussion in the review thread (TasksOperations vs TaskInstanceOperations — "these are TaskInstance operations" vs "match core CLI naming"), where Ash suggested taking it to the dev list:

I have started that discussion: https://lists.apache.org/thread/32oqb3j7f22mlj12t2q95f5n4fqzhlcp

It frames the general question — should the user-facing command surface align with REST API resources, or keep parity with the airflow CLI — with the seven open task-command PRs (this one included) as the concrete motivation.

Since the class name here directly determines the user-facing command group, it may be worth reaching consensus there first before merging this PR — that would avoid shipping the commands under a name that might immediately need a breaking rename.

bugraoz93 · 2026-07-21T18:20:51Z

cc @ashb @bugraoz93 @potiuk @Suraj-kumar00

Following up on the naming discussion in the review thread (TasksOperations vs TaskInstanceOperations — "these are TaskInstance operations" vs "match core CLI naming"), where Ash suggested taking it to the dev list:

I have started that discussion: https://lists.apache.org/thread/32oqb3j7f22mlj12t2q95f5n4fqzhlcp

It frames the general question — should the user-facing command surface align with REST API resources, or keep parity with the airflow CLI — with the seven open task-command PRs (this one included) as the concrete motivation.

Since the class name here directly determines the user-facing command group, it may be worth reaching consensus there first before merging this PR — that would avoid shipping the commands under a name that might immediately need a breaking rename.

Awesome, thanks for creating the discussion!

Suraj-kumar00 requested review from amoghrajesh, ashb, bugraoz93, dheerajturaga, gopidesupavan, jason810496 and potiuk as code owners June 3, 2026 09:38

boring-cyborg Bot added area:airflow-ctl backport-to-airflow-ctl/v0-1-test kind:documentation labels Jun 3, 2026

Suraj-kumar00 changed the title ~~Feat/task instance cli support~~ Add task instance management commands to airflowctl CLI Jun 3, 2026

Suraj-kumar00 force-pushed the feat/task-instance-cli-support branch from bfb913a to 193ed6a Compare June 3, 2026 10:57

bugraoz93 added the full tests needed We need to run full set of tests for this PR to merge label Jun 3, 2026

bugraoz93 closed this Jun 3, 2026

bugraoz93 reopened this Jun 3, 2026

bugraoz93 requested changes Jun 3, 2026

View reviewed changes

Comment thread airflow-ctl-tests/tests/airflowctl_tests/test_airflowctl_commands.py Outdated

Comment thread airflow-ctl/src/airflowctl/api/operations.py Outdated

Comment thread airflow-ctl/src/airflowctl/ctl/cli_config.py Outdated

Suraj-kumar00 force-pushed the feat/task-instance-cli-support branch from 0b76de6 to 400f426 Compare June 3, 2026 18:56

Suraj-kumar00 requested a review from jscheffl as a code owner June 3, 2026 19:22

Suraj-kumar00 force-pushed the feat/task-instance-cli-support branch from 4da1910 to a584bd1 Compare June 3, 2026 19:23

Suraj-kumar00 requested a review from bugraoz93 June 3, 2026 20:19

henry3260 requested changes Jun 17, 2026

View reviewed changes

bugraoz93 reviewed Jun 17, 2026

View reviewed changes

Comment thread airflow-ctl/docs/images/output_main.svg

Suraj-kumar00 force-pushed the feat/task-instance-cli-support branch from 81a9ef1 to f602199 Compare June 21, 2026 10:08

Suraj-kumar00 requested a review from bugraoz93 June 21, 2026 19:31

Suraj-kumar00 force-pushed the feat/task-instance-cli-support branch from 67381fe to 93cf5cd Compare July 1, 2026 14:03

Suraj-kumar00 requested a review from henry3260 July 1, 2026 14:03

aaron-y-chen reviewed Jul 3, 2026

View reviewed changes

Comment thread airflow-ctl/src/airflowctl/api/operations.py Outdated

henry3260 requested a review from Copilot July 3, 2026 04:54

Copilot started reviewing on behalf of henry3260 July 3, 2026 04:55 View session

Copilot AI reviewed Jul 3, 2026

View reviewed changes

Comment thread airflow-ctl/src/airflowctl/api/operations.py Outdated

Comment thread airflow-ctl/src/airflowctl/api/operations.py Outdated

potiuk assigned Suraj-kumar00 Jul 8, 2026

ashb reviewed Jul 10, 2026

View reviewed changes

Comment thread airflow-ctl/src/airflowctl/api/operations.py

ashb reviewed Jul 10, 2026

View reviewed changes

Comment thread airflow-ctl/src/airflowctl/api/operations.py Outdated

ashb reviewed Jul 10, 2026

View reviewed changes

Comment thread airflow-ctl/tests/airflow_ctl/ctl/test_cli_config.py Outdated

ashb reviewed Jul 10, 2026

View reviewed changes

Comment thread airflow-ctl/tests/airflow_ctl/ctl/test_cli_config.py Outdated

Add task instance management commands to airflowctl CLI

bfbbdcf

Suraj-kumar00 force-pushed the feat/task-instance-cli-support branch from 64c4bd9 to bfbbdcf Compare July 12, 2026 07:55

Merge branch 'main' into feat/task-instance-cli-support

7b381ab

Merge branch 'main' into feat/task-instance-cli-support

57c0055

Suraj-kumar00 requested review from aaron-y-chen, ashb and Copilot July 13, 2026 16:11

Copilot started reviewing on behalf of Suraj-kumar00 July 13, 2026 16:11 View session

Merge branch 'main' into feat/task-instance-cli-support

cdf6803

Copilot AI reviewed Jul 13, 2026

View reviewed changes

Uh oh!

Conversation

Suraj-kumar00 commented Jun 3, 2026 • edited by potiuk Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

airflow-ctl/src/airflowctl/api/operations.py

airflow-ctl/src/airflowctl/api/client.py

airflow-ctl/src/airflowctl/ctl/cli_config.py

airflow-ctl/src/airflowctl/ctl/help_texts.yaml

scripts/in_container/run_capture_airflowctl_help.py

airflow-ctl/tests/airflow_ctl/api/test_operations.py

airflow-ctl-tests/tests/airflowctl_tests/test_airflowctl_commands.py

CLI Commands

Uh oh!

Suraj-kumar00 commented Jun 3, 2026

Uh oh!

bugraoz93 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

potiuk commented Jun 9, 2026

Uh oh!

Suraj-kumar00 commented Jun 9, 2026

Uh oh!

henry3260 Jun 17, 2026

Choose a reason for hiding this comment

Uh oh!

Suraj-kumar00 Jun 21, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Suraj-kumar00 commented Jun 17, 2026

Uh oh!

bugraoz93 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Suraj-kumar00 commented Jun 21, 2026

Uh oh!

Suraj-kumar00 commented Jun 23, 2026

Uh oh!

aaron-y-chen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Suraj-kumar00 commented Jul 6, 2026

Uh oh!

Suraj-kumar00 commented Jul 8, 2026

Uh oh!

Suraj-kumar00 commented Jul 10, 2026

Uh oh!

aaron-y-chen commented Jul 10, 2026

Uh oh!

Uh oh!

Uh oh!

ashb Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

Suraj-kumar00 Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

ashb Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Suraj-kumar00 commented Jul 12, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Suraj-kumar00 commented Jun 3, 2026 •

edited by potiuk

Loading

`airflow-ctl/src/airflowctl/api/operations.py`

`airflow-ctl/src/airflowctl/api/client.py`

`airflow-ctl/src/airflowctl/ctl/cli_config.py`

`airflow-ctl/src/airflowctl/ctl/help_texts.yaml`

`scripts/in_container/run_capture_airflowctl_help.py`

`airflow-ctl/tests/airflow_ctl/api/test_operations.py`

`airflow-ctl-tests/tests/airflowctl_tests/test_airflowctl_commands.py`