From 5267fb06390da60ef7a78f3d29a065e698f24feb Mon Sep 17 00:00:00 2001 From: Sean Mooney Date: Tue, 14 Apr 2026 08:59:16 +0100 Subject: [PATCH] Add agentic coding guidance and docs AI-assisted development is increasingly common in Nova reviews, but without a small amount of project-specific routing automated tools can choose incorrect defaults such as pytest, GitHub-style review workflows, or unsafe git operations. At the same time, project conventions should remain in contributor documentation and deterministic checks rather than large tool-specific instruction files. This change adds a compact AGENTS.md routing index for high-signal local workflow guardrails and links to the canonical Nova documentation. It also adds contributor documentation for maintaining that routing layer, repository orientation, dependency placement, and commit message conventions, including DCO, Gerrit Change-Id, and AI attribution trailers. The testing guide now summarizes Zuul gating and unsquashed commit series expectations, while the threading reference documents the current avoidance of new eventlet-specific primitives. Local AI tool preference files and .tmp scratch output are ignored so local state does not appear in upstream reviews. Assisted-By: claude-code claude-sonnet-4-6 Assisted-By: pi gpt-5.5 Change-Id: I2ed2159a6d4154e7d6f743c78965e95f89625328 Signed-off-by: Sean Mooney --- .gitignore | 10 ++ AGENTS.md | 41 ++++++ doc/source/contributor/agentic-coding.rst | 126 ++++++++++++++++++ doc/source/contributor/commit-messages.rst | 84 ++++++++++++ doc/source/contributor/dependencies.rst | 59 ++++++++ doc/source/contributor/index.rst | 55 +++++++- doc/source/contributor/repo-overview.rst | 86 ++++++++++++ doc/source/contributor/testing.rst | 23 +++- .../testing/eventlet-profiling.rst | 12 +- doc/source/reference/threading.rst | 27 ++++ 10 files changed, 512 insertions(+), 11 deletions(-) create mode 100644 AGENTS.md create mode 100644 doc/source/contributor/agentic-coding.rst create mode 100644 doc/source/contributor/commit-messages.rst create mode 100644 doc/source/contributor/dependencies.rst create mode 100644 doc/source/contributor/repo-overview.rst diff --git a/.gitignore b/.gitignore index 1673767b61b..622880997c8 100644 --- a/.gitignore +++ b/.gitignore @@ -55,3 +55,13 @@ releasenotes/build # Files created by alembic /nova.db /nova_api.db + +# agentic tools and files +.tmp/ +.pi/ +.cursor/ +.cursorrules +.claude/ +.vscode/ +.codex/ +CLAUDE.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000000..2ec54646df3 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,41 @@ +# AGENTS.md — agent routing index + +Agents: explore the repo directly; this file is a routing index, not a contributor guide. + +## Workflow + +**Session memory:** Write plans, notes, and ephemeral files to `.tmp/` +(gitignored) rather than the system temporary directory. + +**For non-trivial planning**, inspect deps and tooling: +`pyproject.toml` · `tox.ini` · `.pre-commit-config.yaml` · +`requirements.txt` · `test-requirements.txt` + +**Tests**: Use `tox` or `stestr`; never use `pytest`. + Invoke them directly, for example `tox -e pep8`. + Assume project tools are installed and available on `$PATH`. + +**Routing:** +- Repo layout: [doc/source/contributor/repo-overview.rst](doc/source/contributor/repo-overview.rst) +- Style, hacking, checks: [HACKING.rst](HACKING.rst) +- RPC: [rpc.rst](doc/source/reference/rpc.rst) +- Conductor: [conductor.rst](doc/source/reference/conductor.rst) +- REST API: [api.rst](doc/source/contributor/api.rst) +- Microversions: [microversions.rst](doc/source/contributor/microversions.rst) / [api-microversion-history.rst](doc/source/reference/api-microversion-history.rst) +- Concurrency/threading: [threading.rst](doc/source/reference/threading.rst) / [concurrency.rst](doc/source/admin/concurrency.rst) +- Test conventions, fixtures: [HACKING.rst](HACKING.rst) / [testing.rst](doc/source/contributor/testing.rst) +- Dependencies and packaging: [dependencies.rst](doc/source/contributor/dependencies.rst) +- Documentation: [documentation.rst](doc/source/contributor/documentation.rst) +- Commit messages: [commit-messages.rst](doc/source/contributor/commit-messages.rst) +- Agentic coding conventions: [agentic-coding.rst](doc/source/contributor/agentic-coding.rst) + +## Guardrails + +- **Tools:** Do not install missing tools with a package manager or `pip` +- **Concurrency**: Do not introduce asyncio or new eventlet usage. Review the + threading and concurrency docs when changing concurrent code. +- **Review**: Nova uses Gerrit, not GitHub PRs. Series are always unsquashed; + each commit must be independently testable and correct. +- **Git**: Read-only operations (`git log`, `git diff`, `git status`) are fine. + Do not run mutating operations (`add`, `commit`, `reset`, `checkout`, `push`, + `stash`, `merge`, `rebase`, etc.) unless explicitly instructed to do so. diff --git a/doc/source/contributor/agentic-coding.rst b/doc/source/contributor/agentic-coding.rst new file mode 100644 index 00000000000..a3269b8b8f5 --- /dev/null +++ b/doc/source/contributor/agentic-coding.rst @@ -0,0 +1,126 @@ +.. _agentic-coding: + +============== +Agentic Coding +============== + +This document covers Nova's conventions for AI-assisted development: how +shared project guidance is structured, what belongs in ``AGENTS.md``, and how +contributors should configure local tooling. + +Philosophy +========== + +Nova's approach to agentic coding is minimal scaffolding and native +discovery: if a convention must be followed it is documented in human-readable +prose that agents can find via normal file exploration. Agent context files +are kept short so they route rather than encode knowledge. + +Principles +========== + +Tool-neutral documentation +-------------------------- + +Nova contributor documentation is written for contributors first. Mention +specific AI coding tools only when documenting tool-specific behavior or local +configuration examples. Prefer general wording such as "contributors" and +"readers" when the guidance applies to everyone. + +Canonical human-readable docs +----------------------------- + +Project rules and conventions belong in human-readable documentation, +``HACKING.rst``, or deterministic tooling. Avoid adding agent-only project +rules. If a rule matters for the project, document it where contributors can +find it and let agents discover the same source. + +Deterministic enforcement +------------------------- + +Prefer deterministic checks over prose-only instructions. When a convention can +be enforced by a hacking check, linter, test fixture, or validation script, use +that mechanism instead of relying on a person or an agent to remember prose. + +Minimal routing +--------------- + +Root ``AGENTS.md`` is a compact routing layer plus high-signal guardrails. It is +not a tutorial, not a contributor guide, and not a replacement for the project +documentation it links to. Keep it short enough that tools will read it and +then follow its pointers to authoritative sources. + +AGENTS.md +========= + +``AGENTS.md`` at the repository root provides terse routing pointers and a +small set of guardrails for common incorrect actions. It should contain only +information that is difficult for an agent to discover from source, config, or +existing docs, and that is likely to prevent an incorrect action. + +Appropriate content includes: + +* pointers to authoritative contributor, testing, and subsystem docs; +* guardrails for common incorrect actions not obvious from source or config; + +Avoid adding: + +* long-form explanations; +* summaries of docs that may drift from the source of truth; +* rules already enforced by ``tox -e pep8`` or other validation; +* personal preferences or in-progress notes. + +When ``AGENTS.md`` approaches 100 lines, prune existing content before adding +more, consolidating into contributor docs as required. + +Local scratch files +=================== + +Use ``.tmp/`` for local plans, scratch notes, generated prompts, and other +ephemeral output from coding tools. The directory is gitignored and should not +appear in upstream review history. + +Local tool configuration +======================== + +Local override files are tool-specific preferences. There is currently no +standard format or name for a project-local override file. Some tools support +files such as ``CLAUDE.local.md`` or ``AGENTS.override.md``; others use +different names or do not support local instruction files. Configure these +locally according to the tool being used. + +Do not assume another contributor's tool will read the same local files or use +the same precedence rules. Shared project guidance belongs in the checked-in +Nova docs and root ``AGENTS.md`` routing layer. + +Some tools do not natively support ``AGENTS.md`` and instead look for a +tool-specific file at the repository root. For these tools, the recommended +approach is a local symlink so that checked-in project guidance remains in +``AGENTS.md`` without duplicating it. + +``CLAUDE.md`` is gitignored so contributors can maintain a personal file +without it appearing in upstream review history. If a tool expects +``CLAUDE.md`` at the root, create a local symlink:: + + ln -s AGENTS.md CLAUDE.md + +Tool invocation +=============== + +Assume project tools are installed and available on ``$PATH``. Invoke them +directly in examples and automation, for example:: + + tox -e pep8 + +Do not install tools system-wide with a package manager or ``pip`` (``apt +install``, ``dnf install``, ``pip install``, ``pip install --user``, and +similar). If a required Python tool is not on ``$PATH``, ``uvx`` or ``pipx +run`` are acceptable ephemeral fallbacks. Do not use those wrappers as the +primary invocation for project commands. + +Commit messages +=============== + +See :doc:`/contributor/commit-messages` for Nova commit message guidance, +including Developer Certificate of Origin sign-off, Gerrit ``Change-Id`` +footers, and AI attribution trailers. diff --git a/doc/source/contributor/commit-messages.rst b/doc/source/contributor/commit-messages.rst new file mode 100644 index 00000000000..e3abf279433 --- /dev/null +++ b/doc/source/contributor/commit-messages.rst @@ -0,0 +1,84 @@ +.. _commit-messages: + +=============== +Commit Messages +=============== + +Nova follows the OpenStack Gerrit workflow. Commit messages should explain why +the change is needed, be readable in email, and preserve the footers required by +OpenStack tooling. + +For general OpenStack guidance, see the `OpenStack contributor guide commit +message documentation`__ and the `OpenStack Git commit message wiki`__. + +.. __: https://docs.openstack.org/contributors/common/git.html#commit-messages +.. __: https://wiki.openstack.org/wiki/GitCommitMessages + +Message style +============= + +Use an imperative subject line, normally around 50 characters, with no trailing +period. Wrap body text at 72 columns and explain why the change is needed, not +just what changed. + +Use ASCII-safe printable characters in commit messages. Prefer ``<=``, ``>=``, +``...``, and ``--`` rather than Unicode equivalents such as ``≤``, ``≥``, +``…``, and ``—``. This keeps messages readable across terminals, mail clients, +and review tools used in the OpenStack workflow. + +Gerrit Change-Id +================ + +Nova uses Gerrit. Do not remove existing ``Change-Id:`` footers when amending +or rewording a commit. The ``Change-Id`` links new patch sets to the existing +review. + +Developer Certificate of Origin +=============================== + +Nova follows the Linux Foundation Developer Certificate of Origin (DCO). Every +commit must include a ``Signed-off-by:`` line certifying that the contributor +has the right to submit the contribution:: + + Signed-off-by: Jane Smith + +When creating commits locally, ``git commit -s`` can add this footer. + +AI attribution trailers +======================= + +When AI coding tools materially contribute to a Nova change, use one of the +following trailers. The tool name is required. The model name or version is +optional but should be included when known. Guidance matches the +`OpenInfra Foundation policy for AI generated content`__. + +.. __: https://openinfra.dev/legal/ai-policy + +Notation: ```` = required, ``[x]`` = optional. Do not include the brackets +or angle brackets in the final trailer. + +``Generated-By: [model-version]`` + Use when the commit is mostly machine-authored with minimal human revision. + +``Assisted-By: [model-version]`` + Use when a human materially revised, amended, or curated the AI-generated + work. + +When multiple AI tools materially contributed, each tool may have its own +``Assisted-By`` or ``Generated-By`` trailer. + +Do not use ``Co-Authored-By:`` for AI tools. That trailer identifies human +co-authors and carries a DCO implication; applying it to a language model +misrepresents authorship. Use ``Generated-By`` or ``Assisted-By`` instead. + +Do not use ``Made-with:`` in this repository; Nova standardizes on +``Generated-By`` and ``Assisted-By`` as described in that OpenInfra policy. + +Example footer block +==================== + +:: + + Assisted-By: my-agent my-model-v1 + Signed-off-by: Jane Smith + Change-Id: I1234567890abcdef1234567890abcdef12345678 diff --git a/doc/source/contributor/dependencies.rst b/doc/source/contributor/dependencies.rst new file mode 100644 index 00000000000..341f6f27e0a --- /dev/null +++ b/doc/source/contributor/dependencies.rst @@ -0,0 +1,59 @@ +.. _dependencies: + +============ +Dependencies +============ + +Nova uses the standard OpenStack dependency management model. Keep dependency +changes in the file that matches how the dependency is used so that local +development, packaging, and CI jobs install the same inputs. + +Python dependencies +=================== + +Runtime Python dependencies belong in ``requirements.txt``. These are packages +needed when Nova services or libraries run. + +Test-only Python dependencies belong in ``test-requirements.txt``. These are +packages needed by unit, functional, and validation jobs but not by a deployed +Nova service. + +Documentation-only Python dependencies belong in ``doc/requirements.txt``. +These are packages needed to build or validate the documentation, such as +Sphinx extensions, documentation themes, release note tooling, and redirect +test tools. + +System packages +=============== + +Binary and operating-system package dependencies belong in ``bindep.txt``. +Use bindep profiles such as ``test``, ``doc``, or ``pdf-docs`` when a package +is only needed by a specific class of jobs. For example, a library or command +needed only when building documentation should be listed in ``bindep.txt`` with +the ``doc`` profile rather than documented as a manual setup step. + +Packaging metadata +================== + +Nova uses ``pbr`` for Python packaging metadata. Do not introduce alternative +version or dependency management systems such as ``setuptools-scm``. Keep +package dependencies in the requirement files above so OpenStack constraints +and CI can manage them consistently. + +``pyproject.toml`` is present and records static package metadata per +`PEP 621`_, such as the project name, classifiers, and entry points. The +version and main Python dependency list are dynamic and managed by ``pbr``; +they are not edited in the ``dependencies`` table. + +Runtime, test, and documentation Python dependencies must continue to be +declared in ``requirements.txt``, ``test-requirements.txt``, and +``doc/requirements.txt`` as described above. + +Nova also defines a small set of setuptools extra groups in the +``[project.optional-dependencies]`` table (for example ``osprofiler``, +``zvm``, and ``vmware``). Use those only for the same driver or tooling +extras already represented there. Do not add new general runtime or test +dependencies via ``optional-dependencies``; add them to the appropriate +requirement file instead. + +.. _PEP 621: https://peps.python.org/pep-0621/ diff --git a/doc/source/contributor/index.rst b/doc/source/contributor/index.rst index 5b6e0b8f929..ac8752c9d95 100644 --- a/doc/source/contributor/index.rst +++ b/doc/source/contributor/index.rst @@ -22,6 +22,13 @@ Getting Started * :doc:`/contributor/development-environment`: Get your computer setup to contribute +* :doc:`/contributor/repo-overview`: Terse map of the Nova repository + layout, package structure, and documentation sections. + +* :doc:`/contributor/agentic-coding`: Conventions for AI-assisted + development, including the ``AGENTS.md`` routing layer and local scratch + files. + .. # NOTE(amotoki): toctree needs to be placed at the end of the section to # keep the document structure in the PDF doc. .. toctree:: @@ -29,6 +36,8 @@ Getting Started how-to-get-involved development-environment + repo-overview + agentic-coding Nova Process ============ @@ -71,18 +80,54 @@ while keeping users happy and keeping developers productive. blueprints ptl-guide -Reviewing -========= +Code Conventions +================ + +* `OpenStack code and documentation guide`_: General OpenStack contributor + conventions covering style, testing, and documentation standards. + +* `Nova HACKING.rst`_: Nova-specific style rules and N-check descriptions; + enforced by ``tox -e pep8``. + +* :doc:`/contributor/dependencies`: Where to declare Python, documentation, + and system package dependencies. + +* :doc:`/contributor/commit-messages`: Nova commit message guidance, + including Gerrit footers, DCO sign-off, and AI attribution trailers. * :doc:`/contributor/releasenotes`: When we need a release note for a contribution. +* :doc:`/reference/i18n`: What we require for i18n in patches. + +* :doc:`/reference/database-migrations`: How to write schema and data + migrations when adding a feature or bugfix. + +* :doc:`/reference/upgrade-checks`: How to write automated upgrade checks + when adding a feature or bugfix. + +.. # NOTE(amotoki): toctree needs to be placed at the end of the section to + # keep the document structure in the PDF doc. +.. toctree:: + :hidden: + + dependencies + commit-messages + releasenotes + /reference/i18n + /reference/database-migrations + /reference/upgrade-checks + +.. _OpenStack code and documentation guide: https://docs.openstack.org/contributors/code-and-documentation/index.html +.. _Nova HACKING.rst: https://opendev.org/openstack/nova/src/branch/master/HACKING.rst + +Reviewing +========= + * :doc:`/contributor/code-review`: Important cheat sheet for what's important when doing code review in Nova, especially some things that are hard to test for, but need human eyes. -* :doc:`/reference/i18n`: What we require for i18n in patches - * :doc:`/contributor/documentation`: Guidelines for handling documentation contributions @@ -91,9 +136,7 @@ Reviewing .. toctree:: :hidden: - releasenotes code-review - /reference/i18n documentation Testing diff --git a/doc/source/contributor/repo-overview.rst b/doc/source/contributor/repo-overview.rst new file mode 100644 index 00000000000..9b64aee8ea2 --- /dev/null +++ b/doc/source/contributor/repo-overview.rst @@ -0,0 +1,86 @@ +.. _repo-overview: + +============= +Repo Overview +============= + +A terse map of the Nova repository for contributor orientation. + +Root Files +========== + +``HACKING.rst`` + Nova coding style rules and N-check descriptions. +``AGENTS.md`` + Agent routing index and policy. +``.tmp/`` + Gitignored local scratch directory for notes, plans, and ephemeral output. +``tox.ini`` + Test environments, commands, and environment variables. +``pyproject.toml`` + Build system (pbr), mypy configuration, optional dependency groups. +``.pre-commit-config.yaml`` + Linting hooks: hacking/flake8, mypy, codespell, sphinx-lint. +``requirements.txt`` / ``test-requirements.txt`` + Runtime and test dependencies (pinned via OpenStack constraints). + +nova/ Package +============= + +``nova/api/`` + REST API layer: WSGI, routing, controllers, policy enforcement. +``nova/compute/`` + Compute manager and resource tracker; runs on hypervisor hosts. +``nova/conductor/`` + Conductor manager: cross-service orchestration and DB operations. +``nova/scheduler/`` + Scheduler, filters, and weighers for instance placement decisions. +``nova/virt/`` + Hypervisor driver interface and per-driver implementations. +``nova/db/`` + Database API abstraction layer (SQLAlchemy models and API). +``nova/objects/`` + Versioned objects: the canonical data model for RPC payloads. +``nova/image/`` + Glance integration: image metadata and download for provisioning. +``nova/volume/`` + Cinder integration: volume attachment and detachment operations. +``nova/network/`` + Neutron integration; no in-tree networking logic. +``nova/cmd/`` + Entry points for Nova services (``nova-api``, ``nova-compute``, + ``nova-conductor``, ``nova-scheduler``, etc.). +``nova/conf/`` + oslo.config option declarations, one file per subsystem. +``nova/policies/`` + oslo.policy rule definitions for API access control. +``nova/tests/`` + Unit (``unit/``) and functional (``functional/``) test suites. + +doc/ Structure +============== + +``doc/source/admin/`` + Operator guides: deployment, configuration, architecture. +``doc/source/contributor/`` + Developer guides: process, testing, APIs, agentic coding. +``doc/source/reference/`` + Internal reference: threading model, RPC, scheduling, VM states. +``doc/source/user/`` + End-user guides and feature documentation. +``doc/source/cli/`` + nova-manage and nova-status CLI reference. +``releasenotes/`` + Reno release notes (``notes/`` source files + rendered output). + +API Docs +======== + +These live at the repository root, separate from ``doc/``. + +``api-ref/`` + REST API reference: per-resource ``.inc`` files and an index. Built and + published to docs.openstack.org. +``api-guide/`` + API usage guide covering authentication, faults, links, and general + concepts for API consumers. diff --git a/doc/source/contributor/testing.rst b/doc/source/contributor/testing.rst index 248d9f11c20..25dc57714a1 100644 --- a/doc/source/contributor/testing.rst +++ b/doc/source/contributor/testing.rst @@ -35,7 +35,20 @@ Running tests and reporting results Running tests locally --------------------- -Please see +To run the unit tests:: + + tox -e py3 + +To run the functional tests:: + + tox -e functional + +To run a specific test or subset of tests, pass a regex to stestr via tox:: + + tox -e py3 -- TestWSGIServer + tox -e py3 -- nova.tests.unit.compute.test_api + +For the full set of options and environment setup details, see https://opendev.org/openstack/nova/src/branch/master/HACKING.rst#running-tests @@ -48,9 +61,13 @@ For more details, please see: http://docs.openstack.org/infra/manual/developers.html#automated-testing Before merging any code, there is an integrate gate test queue, to ensure -master is always passing all tests. +master is always passing all tests. Nova uses Zuul gating, where each approved +change is tested together with other changes ahead of it before merging. For more details, please see: -http://docs.openstack.org/infra/zuul/user/gating.html +https://zuul-ci.org/docs/zuul/latest/gating.html + +Because Nova review series are not squashed before merge, each commit in a +series should be independently correct and testable. Infra vs Third-Party -------------------- diff --git a/doc/source/contributor/testing/eventlet-profiling.rst b/doc/source/contributor/testing/eventlet-profiling.rst index 96c58ba9112..e90c1797ef1 100644 --- a/doc/source/contributor/testing/eventlet-profiling.rst +++ b/doc/source/contributor/testing/eventlet-profiling.rst @@ -2,13 +2,21 @@ Profiling With Eventlet ======================= +.. note:: + + As of Nova 34.0.0 (2026.2 Hibiscus), most Nova services run in native threading + mode by default. This guide applies only to services running in eventlet + mode (``OS_NOVA_DISABLE_EVENTLET_PATCHING=False``). For services running + in native threading mode, the standard Python profiling tool cProfile_ can + be used directly instead. + When performance of one of the Nova services is worse than expected, and other sorts of analysis do not lead to candidate fixes, profiling is an excellent tool for producing detailed analysis of what methods in the code are called the most and which consume the most time. -Because most Nova services use eventlet_, the standard profiling tool provided -with Python, cProfile_, will not work. Something is required to keep track of +When a service is running in eventlet mode, the standard profiling tool +provided with Python, cProfile_, will not work. Something is required to keep track of changing tasks. Thankfully eventlet comes with ``eventlet.green.profile.Profile``, a mostly undocumented class that provides a similar (but not identical) API to the one provided by Python's ``Profile`` diff --git a/doc/source/reference/threading.rst b/doc/source/reference/threading.rst index 5b33e2dd770..97862807960 100644 --- a/doc/source/reference/threading.rst +++ b/doc/source/reference/threading.rst @@ -54,3 +54,30 @@ a discussion of the `impact on performance`_. .. _mailing list thread: https://lists.launchpad.net/openstack/msg08118.html .. _impact on performance: https://lists.launchpad.net/openstack/msg08217.html .. _PyMySQL: https://wiki.openstack.org/wiki/PyMySQL_evaluation + +Native threading +---------------- +Since the Flamingo release OpenStack started to transition away form +``eventlet``. During this transition Nova maintains support for running +services with ``eventlet`` while working to add support for running services +with ``native threading``. + +To support both modes with the same codebase Nova started using the +`futurist`_ library. In native threading mode ``futurist.ThreadPoolsExecutors`` +are used to run concurrent tasks and both the oslo.service and the +oslo.messaging libraries are configured to use native threads to execute tasks +like periodics and RPC message handlers. + +New code should avoid adding eventlet-specific primitives or spawning APIs. +Nova is transitioning toward native threading, and ``tox -e pep8`` enforces +several related rules, including N340, N373, and N374. Prefer Nova +compatibility helpers and standard-library primitives where applicable. + +New code should not introduce ``asyncio``. Nova services use native threading +(via futurist and oslo libraries) or, during transition, eventlet. Mixing +``asyncio`` into this model is unsupported. + +.. _futurist: https://docs.openstack.org/futurist/latest/ + +To see how to configure and tune the native threading mode read the +:doc:`/admin/concurrency` guide.