Group and filter Dags by folder on the Dags page by RabahAmrouche05 · Pull Request #68544 · apache/airflow

RabahAmrouche05 · 2026-06-14T18:29:32Z

What

Adds a folder navigation tree to the Dags page. The tree is built from each
Dag's relative_fileloc directory, and selecting a folder filters the Dag list
to the Dags that live in it (and its subfolders).

Why

The Dags page only offered a flat list. In deployments with many Dags spread
across subfolders of the dags folder (e.g. team_a/etl/…, team_b/ml/…) there
was no way to browse or filter by folder.

How

Backend — a server-side relative_fileloc_prefix filter on GET /ui/dags
(strict folder match, so team_a never catches team_alpha) and a new
GET /ui/dags/folders endpoint returning the distinct folders of all readable
Dags. Filtering is done server-side so the result stays correct across
pagination, sorting and the other filters.
Frontend — a collapsible folder tree on the Dags page that drives the
filter via a dag_folder search param, with the tree hierarchy reconstructed
client-side from the folder paths.

Dags located at the bundle root contribute no folder and appear under "All Dags".

Tests: backend (folders endpoint + filter, incl. the team_a / team_alpha
edge case) and frontend (buildFolderTree unit tests + DagFolderTree
component tests).

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Opus 4.8)

Generated-by: Claude Code (Opus 4.8) following the guidelines

Important

🛠️ Maintainer triage note for @RabahAmrouche05 · by @potiuk · 2026-07-02 17:46 UTC

Some review feedback from @pierrejeambrun is waiting on you:

3 unresolved review thread(s) from @pierrejeambrun need a reply or a fix.

The ball is in your court — you've been assigned to this PR. Reply or push a fix in each thread, then mark them resolved.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

boring-cyborg · 2026-06-14T18:29:37Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

jscheffl · 2026-06-14T21:27:58Z

Can you please add a screenshow or small video recording of how it looks like?

RabahAmrouche05 · 2026-06-17T14:13:57Z

Thanks for the review @jscheffl !

Here's what the feature looks like:

Screenshot — the example DAG folder structure on disk (team_a/etl/…, team_b/ml/…).

Video — navigating the new folder tree on the Dags page: expanding/collapsing folders, selecting a folder to filter the list (server-side, so it stays correct across pagination), and going back to "All Dags".

Enregistrement.de.l.ecran.2026-06-17.a.16.03.29.mov

I also removed the newsfragment as you suggested.

One more note: a main merge had left the generated OpenAPI TS client out of sync (the GetDagFolders types were lost during the merge resolution), which was failing the static checks. I've regenerated the client, so that should be green now.

pierrejeambrun · 2026-06-22T16:08:56Z

PR is marked as draft, are you planning additional work before being ready for review?

RabahAmrouche05 · 2026-06-23T19:26:15Z

Hi @pierrejeambrun No further work planned, it's ready for review. I've addressed @jscheffl 's feedback (added a screenshot + a short video of the feature, and removed the newsfragment).

pierrejeambrun

Thanks for the PR, that's cool!

Tested and working as expected, a few suggestions though.

Always-rendered tree — even deployments with all Dags at the bundle root get the 280px sidebar (just "All Dags" + an empty message). Slight clutter for the flat case; consider hiding the panel when there are no folders. Or even better, a visual toggle to hide/display this sidebar.

CI need fixing (conflicts, rebasing should also fix the failing check)

pierrejeambrun

LGTM, this needs a rebase + conflict solving before we can merge.

pierrejeambrun · 2026-07-20T13:45:32Z

Need to fix the conflicts before we can merge.

pierrejeambrun

I just had a chat with Jed, very good comment from him is that this doesn't take into account multi bundle deployment.

Meaning that dags coming from different bundle will be merged under the same tree. All dags will appear as if they were collocated under the same directory. Which probably isn't what the user expect.

Would it be possible to update the PR so when there are multiple bundles (has to be returned in the DagFolderCollectionResponse payload, for each folder basically) the bundle is displayed in the UI as the top level entry and then the folder tree appears under each bundle. When there is only one bundle keep the current UI.

Add a collapsible folder navigation tree to the Dags page, built from each Dag's relative_fileloc directory. Selecting a folder filters the list to the Dags it contains (and its subfolders) via a new server-side filter, so the result stays correct across pagination, sorting and other filters. Because relative_fileloc is relative to each bundle, folders are scoped to their bundle: in multi-bundle deployments the bundle is shown as the top-level entry with its folder tree nested underneath, so folders sharing the same path in different bundles stay separate. Single-bundle deployments keep the flat tree. Backend adds a relative_fileloc_prefix filter on GET /ui/dags and a new GET /ui/dags/folders endpoint returning the distinct (bundle, folder) pairs of all readable Dags. Dags at the bundle root contribute no folder and appear under 'All Dags'. The folder sidebar is hidden for flat deployments with no folders.

RabahAmrouche05 · 2026-07-22T20:02:45Z

Thanks @pierrejeambrun and @jedcunningham, good catch on the multi-bundle case! I updated the PR.

The folders endpoint now returns the bundle with each folder, since a folder path is relative to its bundle and can exist in more than one. When there are several bundles the tree shows the bundle first and the folders under it, so two bundles with the same path don't get mixed together. With a single bundle it stays flat like before.

Selecting a folder now uses its bundle too (with the existing bundle_name filter), so team_a/etl in two bundles points to the right dags.

Also rebased on main to fix the conflicts. Added tests for both (same path in two bundles staying separate, and the bundle grouping in the tree).

bbovenzi · 2026-07-23T16:23:03Z

+        borderRadius="sm"
+        cursor="pointer"
+        gap={1}
+        onClick={() => onSelectFolder({ bundleName: bundle.bundleName, folder: undefined })}


We shouldn't be putting onClick handlers on flex and divs. They should be buttons. But fortunately, we don't have to do this ourselves. Chakra has a Tree View component that we can use!

https://www.chakra-ui.com/docs/components/tree-view

boring-cyborg Bot added area:API Airflow's REST/HTTP API area:translations area:UI Related to UI/UX. For Frontend Developers. translation:default labels Jun 14, 2026

RabahAmrouche05 marked this pull request as ready for review June 14, 2026 18:35

RabahAmrouche05 requested review from bbovenzi, bugraoz93, choo121600, ephraimbuddy, guan404ming, jason810496, pierrejeambrun, rawwar, ryanahamilton, shubhamraj-git and vatsrahul1001 as code owners June 14, 2026 18:35

jscheffl reviewed Jun 14, 2026

View reviewed changes

Comment thread airflow-core/newsfragments/68544.feature.rst Outdated

RabahAmrouche05 requested a review from jscheffl June 17, 2026 17:50

potiuk marked this pull request as draft June 22, 2026 06:35

potiuk assigned RabahAmrouche05 Jun 22, 2026

RabahAmrouche05 marked this pull request as ready for review June 26, 2026 20:29

pierrejeambrun reviewed Jun 29, 2026

View reviewed changes

Comment thread airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/DagFolderTree.tsx Outdated

Comment thread airflow-core/tests/unit/api_fastapi/core_api/routes/ui/test_dags.py

Comment thread airflow-core/src/airflow/api_fastapi/core_api/routes/ui/dags.py Outdated

pierrejeambrun changed the title ~~UI: Group and filter Dags by folder on the Dags page~~ Group and filter Dags by folder on the Dags page Jun 29, 2026

pierrejeambrun added this to the Airflow 3.3.1 milestone Jun 29, 2026

RabahAmrouche05 force-pushed the dags-folder-grouping branch from a8d2151 to ea0797e Compare June 30, 2026 20:32

RabahAmrouche05 force-pushed the dags-folder-grouping branch from ea0797e to 3040cb2 Compare June 30, 2026 21:12

pierrejeambrun approved these changes Jul 7, 2026

View reviewed changes

potiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jul 8, 2026

bbovenzi reviewed Jul 8, 2026

View reviewed changes

Comment thread airflow-core/src/airflow/api_fastapi/common/parameters.py

bbovenzi modified the milestones: Airflow 3.3.1, Airflow 3.4.0 Jul 9, 2026

RabahAmrouche05 force-pushed the dags-folder-grouping branch from 3040cb2 to 9fc1556 Compare July 13, 2026 21:09

RabahAmrouche05 requested a review from henry3260 as a code owner July 13, 2026 21:09

pierrejeambrun removed the ready for maintainer review Set after triaging when all criteria pass. label Jul 20, 2026

pierrejeambrun reviewed Jul 20, 2026

View reviewed changes

Comment thread airflow-core/src/airflow/api_fastapi/common/parameters.py

RabahAmrouche05 force-pushed the dags-folder-grouping branch from 9fc1556 to 6025038 Compare July 22, 2026 19:58

bbovenzi reviewed Jul 23, 2026

View reviewed changes

Uh oh!

Conversation

RabahAmrouche05 commented Jun 14, 2026 • edited by potiuk Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

How

Was generative AI tooling used to co-author this PR?

Uh oh!

boring-cyborg Bot commented Jun 14, 2026

Uh oh!

Uh oh!

jscheffl commented Jun 14, 2026

Uh oh!

RabahAmrouche05 commented Jun 17, 2026

Uh oh!

pierrejeambrun commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RabahAmrouche05 commented Jun 23, 2026

Uh oh!

pierrejeambrun left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pierrejeambrun commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

RabahAmrouche05 commented Jul 22, 2026

Uh oh!

bbovenzi Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

RabahAmrouche05 commented Jun 14, 2026 •

edited by potiuk

Loading

pierrejeambrun commented Jun 22, 2026 •

edited

Loading

pierrejeambrun left a comment •

edited

Loading

pierrejeambrun commented Jul 20, 2026 •

edited

Loading