Skip to content

New generate-history utility for creating synthetic history files#40

Open
ceblanton wants to merge 6 commits into
NOAA-GFDL:mainfrom
ceblanton:generate-history
Open

New generate-history utility for creating synthetic history files#40
ceblanton wants to merge 6 commits into
NOAA-GFDL:mainfrom
ceblanton:generate-history

Conversation

@ceblanton

@ceblanton ceblanton commented Jun 26, 2026

Copy link
Copy Markdown

Description
New pip-installed tool that uses pyfms to drive the FMS diag manager and produce synthetic but usable NetCDF history files without running a real model. It supports three grid types — regular lat/lon, cubed-sphere atmosphere (6-tile gnomonic, via mpirun -n 6), and tripolar ocean — and for each native grid it also writes the FMS mosaic and supergrid files needed to regrid the output with fregrid.

How Has This Been Tested?
I pip installed following the README instructions and used this example diag-yaml:

title: test_diag_manager
base_date: 2 1 1 1 1 1
diag_files:
- file_name: annual
  freq: 12 months
  time_units: days
  unlimdim: time
  varlist:
  - module: atmos
    var_name: tas
    reduction: average
    kind: r4
- file_name: annual_inst
  freq: 12 months
  time_units: days
  unlimdim: time
  varlist:
  - module: atmos
    var_name: tas
    reduction: none
    kind: r4

Then, I created 3 flavors of "history" output:

  1. Generic lat/lon

generate-history diag_table.yaml --nx 32 --ny 32 --nz 2 --calendar noleap --nsteps 8 --output-dir out.regular

  1. Cubed-sphere

mpirun -n 6 generate-history diag_table.yaml --grid-type cubed-sphere --ntile 6 --calendar=noleap --output-dir out.cubed --nsteps 8 --ntile 96

  1. Tripolar

generate-history diag_table.yaml --grid-type tripolar --nx 60 --ny 40 --nz 2 --calendar noleap --nsteps 8 --output-dir out.tripolar

The output looked plausible in all 3 cases. Then I regridded the cubed-sphere and tripolar files, with usable mosaic files generate along with the history files.

  1. Cubed-sphere to lat/lon

fregrid --input_mosaic C96_mosaic.nc --input_file annual --scalar_field tas --output_file regridded.nc --nlat 90 --nlon 180

  1. Tripolar to lat/lon

fregrid --input_mosaic ocean_mosaic.nc --input_file annual --scalar tas --output_file regridded.nc --nlat 90 --nlon 180

fregrid worked in both cases.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published in downstream modules
  • New check tests, if applicable, are included

ceblanton and others added 6 commits June 26, 2026 16:02
Extends the generate-history CLI with --grid-type cubed-sphere and
--grid-type tripolar, along with a new grid_spec.py module that
produces minimally functional FMS mosaic files for use with fregrid.

Cubed-sphere (--grid-type cubed-sphere --ntile N, requires mpirun -n 6):
- Uses define_cubic_mosaic with tile_count=6 so FMS writes per-tile
  history files (stem.tile1.nc ... stem.tile6.nc)
- Writes C{N}_mosaic.nc + C{N}_grid.tile{1..6}.nc with gnomonic
  equal-angle lat/lon geometry matching the GFDL make_hgrid convention
- Writes C{N}_scrip.nc for ESMF/esmpy compatibility

Tripolar ocean (--grid-type tripolar --nx N --ny N):
- Registers dual h/q grid axes (xh/yh for tracers, xq/yq for velocity)
- --q-vars selects which fields go on the q-grid
- Writes ocean_mosaic.nc + ocean_hgrid.nc + ocean_scrip.nc

Both grid types:
- Pad mosaic char variables with null bytes (not spaces) so fregrid
  can parse filenames correctly
- Stamp interp_method = conserve_order1 on all data variables so
  fregrid accepts the history files without extra configuration

All five verification tests pass: generate + fregrid for both grids.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- black + isort reformatted generate_history.py and grid_spec.py
- strip trailing whitespace from README.md
- remove spurious f-prefix from f"SCRIP grid file" (flake8 F541)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… line

black 20.8b1 requires spaces around the exponentiation operator in
expressions like `EARTH_RADIUS ** 2` (modern black omits the spaces).
Also restore the two blank lines after imports that isort requires.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@ceblanton

Copy link
Copy Markdown
Author

Installation instructions for GFDL systems:

module load gcc mpich netcdf-c netcdf-fortran

# install libyaml from source (https://pyyaml.org/wiki/LibYAML)
wget http://pyyaml.org/download/libyaml/yaml-0.2.5.tar.gz
tar -xf yaml-0.2.5.tar.gz
cd yaml-0.2.5
./configure --prefix=$HOME/local
make
make install
setenv PKG_CONFIG_PATH $HOME/local/lib/pkgconfig
setenv CPATH $HOME/local/include

python3.12 -m venv ~/python-envs/pyfms
source ~/python-envs/pyfms/activate

pip install -v -e '.[dev]'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant