Validate documented commands against the Breeze and prek registries

.pre-commit-config.yaml

@@ -1014,6 +1014,20 @@ repos:
           ^scripts/ci/prek/generate_agent_skills\.py$
         pass_filenames: false
         additional_dependencies: ['pyyaml', 'rich>=13.6.0']
+      - id: check-documented-commands
+        name: Check documented breeze and prek commands exist
+        entry: ./scripts/ci/prek/check_documented_commands.py
+        language: python
+        files: >
+          (?x)
+          ^AGENTS\.md$|
+          ^contributing-docs/.*\.rst$|
+          ^dev/breeze/src/airflow_breeze/commands/|
+          ^(?:.*/)?\.pre-commit-config\.yaml$|
+          ^scripts/ci/prek/check_documented_commands\.py$
+        pass_filenames: false
+        require_serial: true
+        additional_dependencies: ['pyyaml', 'rich>=13.6.0']
       - id: run-skill-eval
         name: Run skill-eval against AGENTS.md and cases
         entry: ./dev/skill-evals/eval.py

contributing-docs/03_contributors_quick_start.rst

@@ -669,8 +669,8 @@ You can also switch terminal multiplexer via breeze config:
 
 .. code-block:: bash
 
-  breeze setup config --terminal_multiplexer tmux
-  breeze setup config --terminal_multiplexer mprocs
+  breeze setup config --terminal-multiplexer tmux
+  breeze setup config --terminal-multiplexer mprocs
 
 ** Benefits of using tmux:**
 

contributing-docs/27_cli_implementation_guide.rst

@@ -101,7 +101,7 @@ HTTP client and operations: ``airflow-ctl/src/airflowctl/api/`` (``client.py``,
 
    .. code-block:: bash
 
-      breeze testing airflow-ctl-integration-test
+      breeze testing airflow-ctl-integration-tests
 
 Rewiring an Existing ``airflow`` CLI Command
 ---------------------------------------------

contributing-docs/testing/airflow_ctl_tests.rst

@@ -39,7 +39,7 @@ Then, you can run the tests using the following command:
 
 .. code-block:: bash
 
-   breeze testing airflow-ctl-integration-test
+   breeze testing airflow-ctl-integration-tests
 
 .. note::
 

contributing-docs/testing/integration_tests.rst

@@ -188,7 +188,7 @@ Running Integration Tests from the Host
 You can also run integration tests using Breeze from the host. Depending on the type of integration,
 you can rum "providers" or "core" integration tests. You can consult the table above to see which
 integration is "core" and which is "provider" one, also by running the
-``breeze providers-integration-tests --help`` or ``breeze core-integration-tests --help`` command
+``breeze testing providers-integration-tests --help`` or ``breeze testing core-integration-tests --help`` command
 you can see the list of available integrations for each type of test.
 
 Runs all core integration tests:

contributing-docs/testing/task_sdk_integration_tests.rst

@@ -100,7 +100,7 @@ If you use ``breeze`` to run the integration tests and you do not have the image
 if you do not answer ``no``.
 
 This will build the right image ``ghcr.io/apache/airflow/main/prod/python3.10.latest`` (with the right
-Python version) that will be used to run the tests. The ``breeze prod image build`` command by default -
+Python version) that will be used to run the tests. The ``breeze prod-image build`` command by default -
 when run from sources of airflow - will use the local sources and build the image using ``uv``
 to speed up the build process. Also, when building from sources it will check if the assets are built
 and will error if they are not. However it will not check if the assets are up to date - so make sure
@@ -113,7 +113,7 @@ and did not build your assets after that.
     because the docker-compose setup we use in tests will automatically mount the local Python sources into the
     container, so you can iterate quickly without rebuilding the image. However, if you want to test changes
     that require new image (like modifying dependencies, system packages, rebuilding UI etc.) you will need
-    to rebuild the image with the ``breeze prod image build`` command.
+    to rebuild the image with the ``breeze prod-image build`` command.
 
 After you build the image, there are several ways to run Task SDK Integration Tests,
 depending based on your preferences. The ways are listed below.

contributing-docs/testing/unit_tests.rst

@@ -133,7 +133,7 @@ Airflow unit test types
 Airflow tests in the CI environment are split into several test types. You can narrow down which
 test types you want to use in various ``breeze testing`` sub-commands in three ways:
 
-* By specifying the ``--test-type`` when running a single test type in ``breeze testing core-tests``, ``breeze testing providers-tests``, or ``breeze testing integration-tests`` commands.
+* By specifying the ``--test-type`` when running a single test type in ``breeze testing core-tests``, ``breeze testing providers-tests``, or ``breeze testing core-integration-tests`` commands.
 * By specifying a space-separated list of test types via the ``--parallel-test-types`` or ``--excluded-parallel-test-types`` options when running tests in parallel.
 
 The defined test types are:
@@ -154,7 +154,7 @@ We also have types that run "all" tests (ignoring folders, but looking at ``pyte
 * ``All-Quarantined`` - Tests that are flaky and need to be fixed (``quarantined`` marker).
 * ``All`` - All tests are run (this is the default).
 
-We also have ``Integration`` tests that run with external software via the ``--integration`` flag in the ``breeze`` environment (via ``breeze testing integration-tests``).
+We also have ``Integration`` tests that run with external software via the ``--integration`` flag in the ``breeze`` environment (via ``breeze testing core-integration-tests`` and ``breeze testing providers-integration-tests``).
 
 * ``Integration`` - Tests that require external integration images running in docker-compose.
 

scripts/ci/prek/breeze_command_walker.py

@@ -0,0 +1,66 @@
+#!/usr/bin/env python
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Introspect Breeze's click command tree and print it as structured JSON.
+
+Runs inside Breeze's virtualenv (``uv run --project dev/breeze``) because the
+consumer, ``check_documented_commands.py``, runs in the prek hook venv where
+``airflow_breeze`` cannot be imported.
+
+Emits one ``REGISTRY_JSON:`` line per command path with its group / passthrough
+/ options. Walks the live click objects rather than ``Context.to_info_dict()``,
+which drops the ``context_settings`` that ``breeze k8s deploy-cluster`` relies
+on to mark itself passthrough.
+"""
+
+from __future__ import annotations
+
+import json
+
+import click
+from airflow_breeze.breeze import main
+
+
+def is_passthrough(cmd: click.Command) -> bool:
+    """True when trailing words are forwarded verbatim, so options can't be validated."""
+    if (cmd.context_settings or {}).get("ignore_unknown_options"):
+        return True
+    return any(isinstance(param, click.Argument) and param.nargs == -1 for param in cmd.params)
+
+
+def option_strings(cmd: click.Command) -> list[str]:
+    strings: set[str] = set()
+    for param in cmd.params:
+        if isinstance(param, click.Option):
+            strings.update(param.opts)
+            strings.update(param.secondary_opts)
+    return sorted(strings)
+
+
+def walk(cmd: click.Command, path: str) -> dict[str, dict]:
+    is_group = isinstance(cmd, click.Group)
+    tree: dict[str, dict] = {
+        path: {"group": is_group, "passthrough": is_passthrough(cmd), "opts": option_strings(cmd)}
+    }
+    if isinstance(cmd, click.Group):
+        for name, sub in cmd.commands.items():
+            tree.update(walk(sub, f"{path} {name}"))
+    return tree
+
+
+if __name__ == "__main__":
+    print("REGISTRY_JSON:" + json.dumps(walk(main, "breeze")))