Skip to content

usp-utils: add NCAR-based library install + MPAS build workflow#8

Open
daniloceano wants to merge 8 commits into
developfrom
setup/installation
Open

usp-utils: add NCAR-based library install + MPAS build workflow#8
daniloceano wants to merge 8 commits into
developfrom
setup/installation

Conversation

@daniloceano

@daniloceano daniloceano commented Jun 9, 2026

Copy link
Copy Markdown

What

Adds a friendly, end-to-end installation/build path to usp-utils/, following
the NCAR MPAS-Atmosphere library workflow, so users can prepare the toolchain
and compile MPAS with clear guidance.

New under usp-utils/install/

  • download_mpas_lib_sources.sh — downloads & integrity-checks the 6 library
    source tarballs (MPICH, zlib, HDF5, PnetCDF, NetCDF-C/Fortran) in the exact
    versions the build script expects; PnetCDF from the official Argonne mirror.
  • mpas_lib_install.sh — the unmodified NCAR script (build MPICH → … → PIO).
    Kept pristine: users try it first.
  • INSTALL_GUIDE.md — full recommended flow (conda → julia → source env →
    build libs → compile MPAS → setup run dir), with a "Known build issues
    (optional fixes)" section (e.g. the gfortran ≥ 10 FFLAGS fix) to apply only
    if the build fails, and a troubleshooting section.
  • .gitignore — keeps machine-specific *.local.sh copies out of the repo
    (copy-template → edit .local.sh pattern).

README.md

Documents the install/build flow and the .local.sh pattern.

Notes

  • The temporary Julia Zeros workaround is intentionally not in this PR; it
    lives on the julia/zeros-compat-workaround branch and will be handled
    separately. The docs here only point to it.
  • Validated end to end on swell (gfortran 13.3): libraries built, both cores
    compiled (init_atmosphere_model, atmosphere_model), and setup_run_dir.py
    verified.

daniloceano and others added 4 commits June 9, 2026 18:11
@daniloceano @claude
usp-utils: add NCAR-based library install + MPAS build workflow
9406a97 
Add a friendly, end-to-end install path under usp-utils/install/:
- download_mpas_lib_sources.sh: fetch & verify the 6 library source tarballs
  (MPICH, zlib, HDF5, PnetCDF, NetCDF-C/Fortran) in the versions the build
  script expects; PnetCDF from the official Argonne mirror.
- mpas_lib_install.sh: fix the build for gfortran >= 10 by adding
  -fallow-argument-mismatch to FFLAGS (MPICH's F77 check uses FFLAGS; without
  it MPICH never builds, cascading into "C compiler cannot create executables"
  in later libs that use the missing mpicc).
- INSTALL_GUIDE.md: full recommended flow (conda -> julia -> source env ->
  build libs -> compile MPAS -> setup run dir), with the Julia LD_LIBRARY_PATH
  and Zeros caveats and a troubleshooting section.
- .gitignore: keep machine-specific *.local.sh copies out of the repo.

README.md: document the install/build flow and the copy-to-.local.sh pattern.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@daniloceano @claude
usp-utils: revise INSTALL_GUIDE for accuracy and completeness
2bab36f 
- Use mpas_build_env.local.sh (git-ignored) consistently, sourced for both
  building and running.
- Add mpirun launch commands for the cores and note that running
  atmosphere_model with no inputs aborts via MPI_Abort (expected).
- Complete prerequisites: make, bzip2, tar/gzip.
- Clarify that steps 4-6 (build/compile) don't require the conda/Julia envs.
- Note the template is already patched for gfortran >= 10.
- Document that PIO is optional (SMIOL fallback); NETCDF/PNETCDF always needed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@daniloceano @claude
usp-utils: keep mpas_lib_install.sh pristine; document gfortran fix a…
f214d9b 
…s optional

Revert the -fallow-argument-mismatch change in the versioned template so it
stays identical to the upstream NCAR script — users should try the original
first. Document the fix instead, in a new "Known build issues (optional fixes)"
section of INSTALL_GUIDE, to be applied in the user's .local.sh copy only if the
MPICH build fails on gfortran >= 10. The note is explicit that the fix is
compiler/OS-dependent and invites others to document alternative solutions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@daniloceano @claude
usp-utils: decouple install docs from the temporary Zeros workaround
cc06273 
The install/build docs no longer hard-reference the workaround installer or
UPSTREAM_ISSUE_zeros_compat.md (which live on the separate
julia/zeros-compat-workaround branch). The Julia Step 2 now keeps the universal
LD_LIBRARY_PATH guidance and points to that branch for the temporary Zeros
resolve conflict, so this PR is self-contained.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings June 9, 2026 18:31
Copilot AI reviewed Jun 9, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@daniloceano daniloceano assigned daniloceano and favba and unassigned daniloceano Jun 9, 2026
daniloceano and others added 4 commits June 11, 2026 21:49
@daniloceano @claude
gitignore: ignore WPS_GEOG and geog downloads
e9a2349 
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@daniloceano @claude
usp-utils: robust conda env activation + add install script shebang
300df6a 
- setup_environment.sh: stop inferring env existence from 'conda activate's exit
  code (which gives a misleading "Couldn't find the environment" when the env
  exists but a transient activate returns non-zero). Now: ensure the 'conda'
  shell function is loaded (source conda.sh if only the binary is on PATH),
  check the env in 'conda env list', then activate. Clearer not-found message.
- install_conda_environment.sh: add '#!/usr/bin/env bash' shebang. It uses
  bash-only features (BASH_SOURCE, '[ == ]'); running it with sh/dash failed
  with "Bad substitution" / "unexpected operator" and left SCRIPT_DIR empty.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@daniloceano @claude
gitignore: ignore stray HOME/per-user dirs created in the repo
84affa2 
When a tool runs with HOME pointing at the repo, conda (cache+envs), shell
history, and other per-user files get created here (.conda, .cache, .local,
.config, .claude, .viminfo, ...). Ignore them so they don't pollute git status;
the real ones live in $HOME and these copies are safe to delete.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@daniloceano @claude
install: add gcc/gfortran-12 toolchain template + guide
a736938 
MPAS v8.4 miscompiles with gfortran 13 (builds, crashes at run time);
gfortran 12 (the NCAR tutorial's version) produces a working model. Add
mpas_lib_install_gcc12.sh — a gfortran-12 variant of the lib-install
template (conda-forge gcc12 wrappers, separate libs-gcc12 prefix,
generic $HOME paths) — and document the gcc12 toolchain in INSTALL_GUIDE
(conda env, build steps, and the gcc12 build-environment block).
Personal *.local.sh copies stay git-ignored.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@favba

favba commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

@daniloceano, I'm not sure it is worth compiling your own MPICH.
My opinion is that GCC and MPI should really be provided by the system you are working with. Besides, hpc systems very often have their own mpi software optimized for their own architecture (for example, that might use InfiniBand network interfaces).

I also don't think Claude is right by saying that MPAS doesn't run with GCC 13. I've run it with all versions of gcc from v10 to v16 currently. But I might have used OpenMPI instead of MPICH...

Were you working in an environment where you had to compile MPI yourself in order to use MPAS?

I would also drop PIO, that library brings lots of headaches and dependencies. Besides GCC and MPI, you really only need PNETCDF if you don't use PIO. Otherwise, you need NETCDF, HFD5, ZLIB, and maybe others.

PNETCDF is its own implementation of the CDF file format; it doesn't need the NETCDF library to work. In summary, to use MPAS, you really only need GCC, some MPI implementation, and PNETCDF.

This is what I think this PR should aim at: Downloading and compiling PNETCDF only and optionally download and compile MPICH. But I wouldn't download MPICH from NCAR's website, don't know how "persistent" that link will be... I'd download it from the official website. Also, MIPCH is at version 5.0 already, why use the 3.3 version?

What do you think @pedrospeixoto ?

@favba

favba commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

@daniloceano, one last comment: I've always used make gnu CORE=... instead of make gfortran CORE=...
Looking at the MakeFile the only difference seems to be in some FFLAGS: gnu recipe uses -std=f2008 while gfortran flag doesn't. Does that make any difference? Could you ask your AI friend to investigate this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

Assignees

@favba favba

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@daniloceano @favba