Skip to content

Add mesh_layout='prebuilt' for user-provided mesh graphs - #91

Open
prajwal-tech07 wants to merge 16 commits into
mllam:mainfrom
prajwal-tech07:feat/prebuilt-mesh-layout
Open

Add mesh_layout='prebuilt' for user-provided mesh graphs#91
prajwal-tech07 wants to merge 16 commits into
mllam:mainfrom
prajwal-tech07:feat/prebuilt-mesh-layout

Conversation

@prajwal-tech07

Copy link
Copy Markdown
Collaborator

Describe your changes

Add support for mesh_layout="prebuilt" in create_all_graph_components, enabling users to supply their own mesh graph (e.g. ICON icosahedral, MPAS Voronoi, custom observation networks) instead of relying on library-generated meshes.

Two modes are supported:

  • Nodes+edges mode: User provides a complete nx.DiGraph with validated node attributes (pos, type) and edge attributes (len, vdiff). The graph is used as-is after validation.
  • Nodes-only mode: User provides an nx.Graph with only nodes (having pos and type attributes). The library automatically builds connectivity edges using Delaunay triangulation (with KDTree fallback for collinear/degenerate cases).

Works with all three m2m_connectivity types: flat, flat_multiscale, and hierarchical.

Changes across 4 files:

File Change
src/.../mesh/kinds/prebuilt.py (new, 453 lines) Core module: validation functions, Delaunay edge builder, flat/multiscale/hierarchical constructors
src/.../create/base.py (+197/-38) Dispatch logic in create_all_graph_components for all 3 m2m_connectivity branches
src/.../mesh/__init__.py (+9) Public re-exports of prebuilt functions
tests/test_prebuilt_mesh.py (new, 2869 lines) 202 tests across 28 test classes covering validation, Delaunay, flat/multiscale/hierarchical, integration, edge cases, numerical correctness, graph invariants, and stress tests

Motivation: This is a key extensibility feature that decouples mesh topology from the graph construction pipeline, allowing researchers to use arbitrary mesh structures without modifying library internals.

Dependencies: No new dependencies — uses existing networkx, numpy, scipy.spatial (Delaunay, KDTree).

Issue Link

Closes #79

Type of change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📖 Documentation (Addition or improvements to documentation)

Checklist before requesting a review

  • My branch is up-to-date with the target branch - if not update your fork with the changes from the target branch (use pull with --rebase option if possible).
  • I have performed a self-review of my code
  • For any new/modified functions/classes I have added docstrings that clearly describe its purpose, expected inputs and returned values
  • I have placed in-line comments to clarify the intent of any hard-to-understand passages of my code
  • I have updated the documentation to cover introduced code changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have given the PR a name that clearly describes the change, written in imperative form (context).
  • I have requested a reviewer and an assignee (assignee is responsible for merging)

Checklist for reviewers

Each PR comes with its own improvements and flaws. The reviewer should check the following:

  • the code is readable
  • the code is well tested
  • the code is documented (including return types and parameters)
  • the code is easy to maintain

Author checklist after completed review

  • I have added a line to the CHANGELOG describing this change, in a section
    reflecting type of change (add section where missing):
    • added: when you have added new functionality
    • changed: when default behaviour of the code has been changed
    • fixes: when your contribution fixes a bug

Checklist for assignee

  • PR is up to date with the base branch
  • the tests pass
  • author has added an entry to the changelog (and designated the change as added, changed or fixed)
  • Once the PR is ready to be merged, squash commits and merge the PR.

@joeloskarsson

Copy link
Copy Markdown
Contributor

It seems to me that there are really two mesh layouts added here: the prebuilt and delaunay triangulation from nodes. These are quite different and should be different mesh_layouts IMO. For building mesh from a given set of nodes, it would be nice to make the solution flexible enough to also support other mesh generation methods, e.g. knn-graphs.

@prajwal-tech07

Copy link
Copy Markdown
Collaborator Author

Hi @joeloskarsson, thanks for the feedback!

This PR implements issue #79, which was split alongside #78 (mesh_layout="rectilinear", PR #81) and #80 (mesh_layout="triangular") as part of a planned series — @leifdenby helped scope these three issues from the original discussion in #71.

The two modes in this PR are intentionally both under mesh_layout="prebuilt":

That said, I think your point about supporting other mesh generation methods (KNN, radius graphs) for the nodes-only case is a great idea. I could make the connectivity method configurable via mesh_layout_kwargs, e.g.:


mesh_layout_kwargs=dict(
    mesh_graph=my_nodes_graph,
    connectivity_method="delaunay"  # or "knn", "radius"
)python
mesh_layout_kwargs=dict(
    mesh_graph=my_nodes_graph,
    connectivity_method="delaunay"  # or "knn", "radius"
)

This would keep both modes under "prebuilt" (since the user is still "bringing their own nodes") while making connectivity generation pluggable. Would that address your concern, or do you feel they should still be separate

mesh_layout
values?

Happy to adjust the design based on your and @leifdenby's input!

…tion

Add support for triangular mesh layout based on networkx.triangular_lattice_graph.
Implements flat, flat_multiscale and hierarchical triangular mesh connectivity.

Closes mllam#80
- Fix wording: 'can improve' -> 'is expected to improve' message-passing
- Set same axis limits across all subplots in side-by-side comparison
- Set aspect ratio 1.0 on all axes in side-by-side comparison
Move coordinate-creation functions from coords.py and connectivity/triangular.py into layout/rectilinear.py and layout/triangular.py. Drop triangular-specific convenience wrappers in favour of the generic two-step API. Retain backward-compatible re-exports in coords.py and connectivity/triangular.py.

Update all imports and rewrite tests to use the two-step API.
- docs/mesh_layout.ipynb: note mesh_layout is supported from v0.5.0 (was v0.4.0)
- layout/__init__.py: describe layouts in plain terms instead of naming the
  internal networkx helpers (grid_2d_graph / triangular_lattice_graph)
…ultiscale, simplify base.py

Implements the approved PR mllam#92 review comments:
- layout/ now produces only undirected coordinate primitives; all DiGraph
  construction lives in connectivity/. Removed create_single_level_2d_triangular_mesh_graph
  and the layout->connectivity import (mllam#4).
- Generalized create_flat_multiscale_from_coordinates to handle both layouts:
  rectilinear keeps the exact index-arithmetic merge (output unchanged),
  triangular falls back to position-based KD-tree matching. Per-layout selection
  uses the 'diagonal' adjacency signature and the odd-refinement-factor check now
  only guards the index path. Deleted create_flat_multiscale_from_triangular_coordinates (mllam#5, mllam#8).
- create_single_level_2d_triangular_mesh_primitive now accepts mesh_node_spacing
  and computes nx/ny internally, mirroring the rectilinear primitive (mllam#6).
- base.py: swapped the mesh_layout/m2m_connectivity if-nesting so m2m_connectivity
  is outer and the primitive function is picked by mesh_layout, removing duplicated
  spacing handling (mllam#7).
- Dropped the no-op triangular 'pattern' special-casing; the lattice fully
  determines triangular connectivity (mllam#2).
- Renamed the unsupported-layout test placeholder 'hexagonal' -> 'nonexistent_layout' (mllam#9).

All 299 tests pass; rectilinear output is unchanged by construction.
@prajwal-tech07

prajwal-tech07 commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator Author

Hi @leifdenby! Following up on the meeting - here's my plan for reviving this PR on top of the mesh_layout structure from #92, plus the design questions around input format and docs. Would love a sanity check before I start porting.

1. Branch strategy

This branch predates #81's merge (it carries parallel copies of those commits on the old mesh/kinds/ structure), so a literal merge of #92 would conflict on every file. Instead I'd reset this branch onto feat/triangular-mesh-layout and port the prebuilt code as fresh commits adapted to the layout/ + connectivity/ split - the PR and its history stay alive, and once #92 merges the diff here shrinks to just the prebuilt work.

2. How prebuilt fits the two-step architecture

prebuilt becomes a third layout module (mesh/layout/prebuilt.py) producing the same undirected mesh-primitive contract as rectilinear/triangular:

  • Nodes-only input → validate, build adjacency via Delaunay triangulation (adjacency_type="delaunay") → the existing connectivity step runs unchanged.
  • Nodes+edges input → the user's edges become the primitive adjacency, and the connectivity step still runs (directs edges, computes len/vdiff).

The second point is a deliberate change from this PR's old design, where a full nx.DiGraph bypassed connectivity entirely. Treating user edges as primitive adjacency keeps len/vdiff computed in exactly one place and keeps mesh_layout × m2m_connectivity fully orthogonal - prebuilt then works with flat, flat_multiscale, and hierarchical (multi-level via a level node attribute).
Does that match your thinking, or do you see a use case for accepting a fully-formed directed mesh graph as-is?

3. User input contract

Canonical: an in-memory networkx.Graph with node attr pos (ndarray (2,), same coordinate system as the grid xy - I'll make this loud in the docs), optional level for multi-level, edges optional.
Convenience: a bare np.ndarray [N, 2] of coordinates for the simplest nodes-only case.
No file-format commitment in the library — instead the docs show how to load real sources (ICON netCDF vertices, a CSV of station locations) into that graph in a few lines.
API: create_all_graph_components(mesh_layout="prebuilt", mesh_layout_kwargs=dict(mesh_graph=G), ...) - no mesh_node_spacing needed since spacing is implied by the user's coordinates.

import networkx as nx
import numpy as np
import weather_model_graphs as wmg

# user's own mesh: e.g. vertices loaded from an ICON grid file
my_mesh = nx.Graph()
for i, (x, y) in enumerate(my_mesh_coords):  # [N, 2], same coords as xy
    my_mesh.add_node(i, pos=np.array([x, y]), type="mesh")
# (optionally add edges here — otherwise Delaunay builds the adjacency)

graph = wmg.create.create_all_graph_components(
    xy=xy_grid,
    mesh_layout="prebuilt",
    mesh_layout_kwargs=dict(mesh_graph=my_mesh),
    m2m_connectivity="flat",
    g2m_connectivity="nearest_neighbour",
    m2g_connectivity="nearest_neighbour",
)

4. Docs

A new jupyter-book chapter prebuilt_mesh.ipynb (after mesh_layout.ipynb in the TOC): motivation (bring your own mesh - icosahedral, Voronoi, observation networks), the input contract, nodes-only vs nodes+edges walkthroughs with plots, multi-level → hierarchical, the validation errors, and an end-to-end create → save example. I'll also trim the old test file down to the essentials (validators, Delaunay properties, one end-to-end per connectivity type).

If this sounds right I'll start with the layout-module port and open the docs chapter alongside it.

…tidy tests

- Rename triangular layout primitives to drop the _triangular suffix and
  alias them on import in base.py (same for rectilinear) for symmetry.
- Use elif mesh_layout == 'triangular' in both coordinate-creation branches
  and raise NotImplementedError for unsupported layouts.
- Inline the triangular test helpers to use the generic two-step API and
  drop the redundant flat-multiscale passthrough.
- Clarify the 4-star/8-star pattern-equivalence test docstring.
- Fix mangled UTF arrows in the vdiff reciprocity docstring.
The pattern argument only filters cardinal vs diagonal edges, which is a
rectilinear distinction. Triangular primitives have only cardinal edges,
so 4-star and 8-star produce identical graphs -- the test verified a no-op.
…llam#79)

The prebuilt layout takes user-supplied mesh node positions (nx.Graph with
pos/type/level node attributes, or a bare [N, 2] array) and passes them
through as edge-less node-cloud primitives after validation. Following the
design agreed in mllam#79, no adjacency is invented in the layout step: the
connectivity step builds directed mesh edges straight from the node
positions (method='delaunay', the default), skipping the undirected
adjacency graph entirely.

The connectivity step now also validates an explicit 'pattern' against the
adjacency types present in the primitive and raises with the available
options instead of silently producing an empty mesh; when no pattern is
given, every edge the layout produced is used (behaviour-identical for the
generated layouts). intra_level for hierarchical connectivity accepts
method= for edge-less primitives alongside the existing pattern=.
- accept mesh_layout_kwargs=dict(mesh_graph=...) (nx.Graph or [N, 2]
  ndarray); no mesh_node_spacing needed since spacing is implied by the
  node positions
- flat: forward optional pattern/method to the connectivity step (no more
  hardcoded 8-star default; omitting pattern uses every layout edge, which
  is behaviour-identical for the generated layouts)
- hierarchical: multi-level primitives split by the nodes' integer 'level'
  attribute (lowest = finest); grid connects to the finest level as usual
- flat_multiscale with prebuilt raises NotImplementedError for now
- tests: input validation, primitive creation, Delaunay connectivity
  (incl. 1/2-node clouds, collinear degeneracy, pattern-vs-method errors),
  flat + hierarchical end-to-end, and a pattern-default equivalence guard
  for the generated layouts
New docs chapter walks through the input contract, the flat (Delaunay)
and hierarchical (level-attribute) nodes-only cases with plots, the
validation errors, and how to load real mesh sources (station CSV, ICON
vertices) into the node-cloud graph.
@prajwal-tech07
prajwal-tech07 force-pushed the feat/prebuilt-mesh-layout branch from ceede3a to 2b13e5a Compare July 18, 2026 06:08
Upstream main added nb-clean (v4.0.1) to the pre-commit config after this
branch's fork point; CI lints the merge with main, so the two notebooks
this branch touches need to be clean: strip execution counts/outputs from
mesh_layout.ipynb and the kernel version metadata from prebuilt_mesh.ipynb.
@prajwal-tech07

Copy link
Copy Markdown
Collaborator Author

The revival is implemented and pushed, following the design we converged on in #79:

  • mesh_layout="prebuilt" + mesh_layout_kwargs=dict(mesh_graph=...) - nodes-only input (nx.Graph or a bare [N, 2] array), validated and passed through as an edge-less node cloud
  • Mesh edges are built in the connectivity step directly from node positions (method="delaunay", default) - no undirected intermediate, per your suggestion
  • No pattern given → every layout edge is used; an explicit pattern that doesn't match the primitive's adjacency types raises listing what's available; pattern= on a node cloud points to method=
  • Hierarchical via integer level node attribute (lowest = finest), inter_level={"pattern": "nearest", "k": 1} unchanged
  • New jupyter-book chapter (prebuilt_mesh.ipynb) covering both nodes-only cases, 45 new tests, all 342 pass

Two conscious deferrals: user-provided edges (Cases 2/4 from the sketches - raises NotImplementedError pointing at #79) and flat_multiscale for prebuilt (level-merging assumes coincident cross-level positions, which arbitrary user clouds don't have - I think it needs the Delaunay-coarsening idea we paused).

Note: this branch is based on #92's branch, so the diff currently includes those commits - it'll shrink to just the prebuilt work once #92 merges, and I'll rebase then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feat] Add mesh_layout="prebuilt" and mesh_layout="prebuilt_hierarchical" support

3 participants