diff --git a/AGENTS.md b/AGENTS.md index c1a9c72d..59b95056 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,100 +2,65 @@ Mandatory rules for AI coding agents. Violations will result in rejected PRs. ---- - -## Codebase - -Neural-LAM: graph-based neural weather prediction for Limited Area Modeling. Models: `GraphLAM`, -`HiLAM`, `HiLAMParallel`. - -**Data flow:** Raw zarr/numpy → `Datastore` → `WeatherDataset` → `WeatherDataModule` → Model → -Predictions +**Read [CONTRIBUTING.md](CONTRIBUTING.md) first.** It covers the general workflow that applies to +every contributor (open an issue, fork, set up the environment, run `pre-commit` and `pytest`, +fill in the PR template, add a CHANGELOG entry, monthly dev meeting). Everything there applies. +This file adds the AI-specific rules on top. -**Key modules:** -- `datastore/` — `BaseDatastore` (abstract), `MDPDatastore` (zarr via mllam-data-prep) -- `models/` — `ARModel` (autoregressive base, Lightning) → `BaseGraphModel` (encode-process-decode) - → `GraphLAM` / `HiLAM` / `HiLAMParallel` -- `weather_dataset.py` — `WeatherDataset` + `WeatherDataModule` -- `config.py` — YAML config via dataclass-wizard -- `create_graph.py` — builds mesh graphs (must run before training) -- `interaction_net.py` — `InteractionNet` GNN layer (PyG `MessagePassing`) -- `utils.py` — `make_mlp`, normalization helpers +## Codebase reference -Config examples: `tests/datastore_examples/` - -## Commands - -```bash -# Install (torch variant selected via mutually-exclusive extras; uv.lock is committed) -uv sync --extra cpu --group dev --locked # CPU-only -uv sync --extra gpu --group dev --locked # GPU, CUDA 13.0 (default) -uv sync --extra gpu-cu128 --group dev --locked # GPU, CUDA 12.8 -``` +See the README architecture overview for the data flow and module map. +`git log --stat -- neural_lam/` shows which files have moved recently - prefer that over any +snapshot, which will rot. -The `cpu`/`gpu`/`gpu-cu128` extras route `torch` to the matching PyTorch index via `[tool.uv.sources]` in `pyproject.toml`. CI uses `uv` only (CPU + CUDA 13.0); the `pip` install path is still documented in the README for users who prefer it. +## AI-specific entry-point commands -The remaining commands need to be prepended with `uv run` or the virtual env activated with `source .venv/bin/activate` first: +The standard install / lint / test commands are in +[CONTRIBUTING.md > Before you push](CONTRIBUTING.md#before-you-push). The two CLI entry points +agents most often need: ```bash -# Lint -pre-commit run --all-files # black, isort, flake8, mypy, codespell - -# Test -pytest -vv -s --doctest-modules # all -pytest tests/test_training.py -vv -s # single file -pytest tests/test_training.py::test_fn -vv # single function - -# Run python -m neural_lam.create_graph --config_path --name python -m neural_lam.train_model --config_path --model graph_lam --graph python -m neural_lam.train_model --eval test --config_path --load ``` -W&B auto-disabled in tests. `DummyDatastore` used; example data downloaded from S3 on first run. +W&B is auto-disabled in tests. `DummyDatastore` is the in-memory test fixture; example data is +downloaded from S3 on first run. --- -## Rules +## AI-specific rules + +### Search before creating + +Duplicate issues and PRs from AI agents are a recurring problem. Search before opening anything: -### Issues +```bash +gh issue list --state all --search "" +gh pr list --state all --search "" +``` -1. **Search before creating.** Use any of: GitHub UI search, `gh issue list --state all --search ""`, or `curl "https://api.github.com/search/issues?q=+repo:mllam/neural-lam+type:issue"`. Duplicate issues will be closed. -2. **Every PR requires an issue.** No exceptions. Open one first if none exists. -3. **Include minimal example.** Each issue should include a minimal, reproducible example on how to easily recreate a bug, including all necessary module imports and data. Include full traceback if it is a bug-report. +If a PR already exists for the same issue, contribute there rather than opening a competing one. -### Pull Requests +### Re-read the thread before every action -1. **Search before creating.** Use any of: GitHub UI search, `gh pr list --state all --search ""`, or `curl "https://api.github.com/search/issues?q=+repo:mllam/neural-lam+type:pr"`. If a PR exists for the same issue, contribute there. -2. **Link the issue.** PR body must contain `closes #` or `refs #`. Unlinked PRs will be - rejected. -3. **Use the PR template.** Fill in every section of `.github/pull_request_template.md`. Do not - delete or skip sections. -4. **Read the full issue thread before writing code.** Rejected approaches and prior decisions are - there. Ignoring them wastes everyone's time. -5. **Run pre-commit hooks locally.** Linting needs to be done locally before each new commit with e.g. `uvx pre-commit run --all` -6. **Testing Mandate.** Run `pytest tests/` before opening a PR and if tests fail do not open the PR, fix the failure first. +Re-read the full issue / PR thread (including inline review comments via +`gh api repos/mllam/neural-lam/pulls//comments`) before every comment and every push. Never +repeat a question already answered or an approach already rejected. ### Communication - **Terse.** One sentence per point. No preamble. No summaries of visible diffs. -- **No filler.** Ban list: "Great question", "As mentioned above", "I hope this helps", "Let me know - if you have questions", "Happy to help". +- **No filler.** Ban list: "Great question", "As mentioned above", "I hope this helps", "Let me + know if you have questions", "Happy to help". - **No obvious narration.** Do not explain what self-explanatory code does. - **PR descriptions: what changed and why.** Nothing else. - **One question at a time.** No shotgun lists of open-ended questions. -### Context - -- **Re-read the entire thread** before every comment and every push. No exceptions. -- **After a context gap**, reload the full thread (GitHub UI, `gh issue view ` / `gh pr view `, or `curl "https://api.github.com/repos/mllam/neural-lam/issues/"`) before acting. -- **Never repeat** a question already answered or an approach already rejected in the thread. - ### Commits -- Imperative form, matching existing `git log` style. -- One concern per PR. No unrelated changes. -- AI attribution of tool names is mandatory if used and should be mentioned in the commit message trailer as `Co-authored-by ` +AI attribution is mandatory. Add a `Co-authored-by:` trailer to every commit produced with AI assistance, e.g. `Co-authored-by: Claude Opus 4.8 `. ### Changelog diff --git a/CHANGELOG.md b/CHANGELOG.md index 4ff85253..7c426943 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -83,6 +83,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Maintenance +- Add a root `CONTRIBUTING.md` walking a new contributor from fork to merged PR (issue triage, environment setup pointing at the README install section, pre-commit, code standards, the exact local test command CI runs, PR template / CHANGELOG expectations, monthly dev-meeting pointer, Slack + issues for help) [\#407](https://github.com/mllam/neural-lam/pull/407) @ANANYA542 + - Split the monolithic `neural_lam/utils.py` into a `neural_lam/utils/` package with one module per concern (`buffer_list`, `graph`, `networks`, `plot`, `logging`, `tensor`, `time`); `utils/__init__.py` re-exports the full public API so existing imports are unaffected. Pure code movement, no behavioural change. [\#682](https://github.com/mllam/neural-lam/pull/682) @Sir-Sloth-The-Lazy - Add comprehensive type hints to GraphLAM in `neural_lam/models/step_predictors/graph/graph_lam.py` [\#669](https://github.com/mllam/neural-lam/pull/669) @GiGiKoneti diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..95ef4721 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,174 @@ +# Contributing to Neural-LAM + +Thank you for your interest in contributing to Neural-LAM! +We welcome bug reports, bug fixes, documentation improvements, and new features. + +## Community + +Neural-LAM is developed in the open by a small, friendly group of researchers +and engineers working on ML-based limited area modelling. We try hard to keep +the community welcoming, open-minded, and constructive - whether you are +opening your first issue or proposing a substantial new feature. + +A few things that help keep it that way: + +- **Assume good intent.** Reviewers and contributors are mostly volunteering + their time. If a comment feels blunt, read it as direct rather than hostile. +- **Ask questions early.** It is always cheaper to discuss a design choice + before code is written. We would rather see a half-formed idea in an issue + than a finished PR going in the wrong direction. +- **Credit each other.** When your work builds on someone else's PR or issue, + link it. When you review, thank the author for the time they invested. +- **It is fine to be a beginner.** Several of the maintainers were + PhD students or first-time open-source contributors not long ago. Nobody + expects you to know everything about NWP, ML, or the codebase on day one. + +## Getting started + +1. **Open or find an issue.** Before writing code, open a + [GitHub issue](https://github.com/mllam/neural-lam/issues) describing what + you plan to do. If an issue already exists, comment so others know you are + working on it. New contributors looking for an approachable first task can + filter by the + [`good first issue` label](https://github.com/mllam/neural-lam/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22). + +2. **Read the existing discussion.** Review the full issue thread and any + linked PRs - someone may have already proposed a solution or identified a + blocker. + +3. **Set up your environment.** Fork the repo via the GitHub UI, then follow + the [Installing Neural-LAM section of the README](README.md#installing-neural-lam) + using the `--group dev` flag. Then activate the hooks: + + ```bash + pre-commit install + ``` + +## Code standards + +Code quality is enforced automatically by +[pre-commit hooks](.pre-commit-config.yaml) (Black, isort, Flake8, mypy, +Codespell and others). In addition: + +- Add **NumPy-style docstrings** and **type annotations** to every public + function and class. The + [numpydoc style guide](https://numpydoc.readthedocs.io/en/latest/format.html) + is the authoritative reference; + [`neural_lam/datastore/base.py`](neural_lam/datastore/base.py) is a good + in-repo example of the conventions we follow (sectioned `Parameters` / + `Returns`, types in the signature line, prose first). +- When annotating tensor shapes, use the canonical dimension names from the + [Dimension Glossary](README.md#dimension-glossary) so shapes stay consistent + across the codebase. +- Keep new code consistent with the patterns already in the codebase. + +## Before you push + +Run **both** checks locally - they are the same ones CI will run: + +```bash +pre-commit run --all-files +pytest -vv -s --doctest-modules +``` + +> **Note:** The first test run downloads ~50 MB of example data via +> [pooch](https://www.fatiando.org/pooch/). + +## Community roadmap + +Our community roadmap is defined by +[milestones](https://github.com/mllam/neural-lam/milestones) on the +[`neural-lam` GitHub repo](https://github.com/mllam/neural-lam). We use +[semantic versioning](https://semver.org/) `vX.Y.Z` for the milestones, so +issues on the roadmap carry the version they will be part of as their +milestone. + +The process for putting issues or PRs on the roadmap is: + +1. **Propose** - label your issue or PR with a milestone in the form + `vX.Y.Z (proposed)` (e.g. `v0.9.2 (proposed)`). +2. **Discuss** - at a development meeting the assignee explains the purpose + and how it fits the current roadmap. The group decides whether to accept + it and which revision it targets (it may be moved to a later release). +3. **Accept** - once accepted the `(proposed)` suffix is removed and the + issue or PR is placed on the milestone for that version. + +To propose something for the roadmap, all you need to do is add a milestone +label in the form `vX.Y.Z (proposed)`. + +## Pull requests + +1. Push your branch and open a PR against `main`. +2. Fill in the + [pull request template](.github/pull_request_template.md) - it contains the + full checklists for authors, reviewers, and assignees. +3. Write commit messages in **imperative form** matching the existing + `git log` style ("Add X" not "Added X"), and keep one concern per PR. +4. PRs land via **squash-and-merge**: the PR title becomes the single commit + message on `main` and the PR description becomes its body. Polish both + before requesting review - per-commit history on your branch is not + preserved in `main`. +5. A maintainer will review your PR. Small, focused PRs are reviewed faster. +6. After review, iterate on the feedback. Once the review is resolved and + CI is green: + - **Bugfixes and maintenance PRs** are merged directly by the assignee. + - **Feature PRs** (anything labelled `enhancement`) are proposed for the + [community roadmap](#community-roadmap) by adding a `vX.Y.Z (proposed)` + milestone, then discussed at the next + [monthly dev meeting](#monthly-development-meetings) so the team can + align on roadmap and scope. Feature PRs accepted for the roadmap + (i.e. assigned a `vX.Y.Z` milestone without `(proposed)`) are merged + by the assignee once the milestone is ready. +7. **Accepted feature PRs** are merged when the milestone closes (or + sooner if the feature is self-contained and the maintainers agree). + Bugfix and maintenance PRs are merged as soon as step 6 completes. + +## CHANGELOG entries + +**Every PR must add a line to [CHANGELOG.md](CHANGELOG.md)** under the +section matching the change type (`Added`, `Changed`, `Fixed`, or +`Maintenance`). Add a new section heading if it does not already exist +under the current `[unreleased]` block. + +The entry references the **PR number, not the issue number**, and tags the +author. Format: + +```markdown +- Short description of the change [\#NNN](https://github.com/mllam/neural-lam/pull/NNN) @your-handle +``` + +Example (from a real merged PR): + +```markdown +- Add bounds checking for `--var_leads_metrics_watch` indices to fail at + CLI parse time rather than mid-validation [\#306](https://github.com/mllam/neural-lam/pull/306) @your-handle +``` + +## Monthly development meetings + +The mllam team meets monthly to coordinate roadmaps across the three core +projects (`neural-lam`, `weather-model-graphs`, `mllam-data-prep`), review +proposed changes, and decide what lands in upcoming releases. The meeting is +on the **second Monday of each month, 10:00-11:00 CEST**, on Zoom (the link +is pinned in the +[`#general` channel of the mllam Slack workspace](https://kutt.to/mllam)). + +Everyone is welcome - contributors, users, lurkers, and people who are just +curious about ML-based weather forecasting. If you have a feature you would +like to discuss before opening a PR, or a design question that does not fit +neatly into a GitHub thread, the dev meeting is a good venue. It is also a +good place to learn where these tools are heading. + +## Getting help + +- Join the [mllam Slack workspace](https://kutt.to/mllam) - chat with + maintainers and other contributors, and find the Zoom link for the next + dev meeting in `#general`. +- [Open a GitHub issue](https://github.com/mllam/neural-lam/issues) - best + for anything that benefits from a written, searchable record. + +## License + +Neural-LAM is released under the [MIT License](LICENSE.txt). By contributing +to this repository, you agree that your contribution is licensed under the +same terms.