Fix NEE flat-file var read; registry overlay hygiene; min_year default#129
Merged
Merged
Conversation
Flat-file variable resolution Preprocessed flat files store the relabelled evaluation-item variable (e.g. Net_Ecosystem_Exchange derived from an f_respc fallback/convert), but downstream readers hard-indexed ds[sim_varname] (f_respc) and raised xarray "No variable named 'f_respc'". Add util.names.select_data_array (resolve by name case-insensitively, else fall back to the sole data var) and use it in runner/masking, grid evaluation, and all _comparison_* modules. The write-side relabel to the item is intentional (output labels) and unchanged. Registry overlay hygiene (openbench registry diff/prune) Classify the user ~/.openbench overlay as redundant / stale full-copy / delta / custom and re-sparse it behavior-preservingly, so a legacy full snapshot no longer silently shadows bundled catalog fixes. Adds a throttled, silent-when-clean startup notice (suppressible via OPENBENCH_NO_REGISTRY_CHECK). min_year_threshold default -> 1 schema + loader default 3 -> 1 (matches GUI/smoke); init writes 1 for every project span.
4379b98 to
f4d0511
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three fixes from a debugging session on the persistent
Net_Ecosystem_Exchangeevaluation error (plus related cleanups).1. NEE flat-file variable resolution (the actual bug)
Preprocessing correctly derives NEE from
f_respc/f_assimand relabels the saved variable to the evaluation item (Net_Ecosystem_Exchange) for output-label correctness. But the flat file is named with the config varname (f_respc), and downstream readers hard-indexedds[sim_varname]→ xarrayNo variable named 'f_respc'. Variables on the dataset include ['Net_Ecosystem_Exchange', 'time', 'lat', 'lon'].util.names.select_data_array(ds, *names): resolve by name (case-insensitive), else fall back to the sole data variable (flat files have exactly one).runner/masking.py(the preprocess crash),core/evaluation.py(grid scoring), and all 7core/_comparison_*.pymodules.2. Registry overlay hygiene —
openbench registry diff|pruneA legacy full-snapshot
~/.openbenchoverlay silently shadows bundled catalog fixes (this is why the CLARA albedo unit fix didn't take effect until the overlay was reset). New tooling:registry prunere-sparses behavior-preservingly (drops redundant, minimizes stale full-copies; merged registry identical before/after).OPENBENCH_NO_REGISTRY_CHECK.3.
min_year_thresholddefault → 1schema+loaderdefault3 → 1(matches GUI/smoke);initnow writes 1 for every project span.Testing
tests/test_flat_var_resolution.py(reproduces the exact error + end-to-end masking read),tests/test_registry_overlay_audit.py.test_cli_stubs.py,test_config/test_schema.pyfor the new defaults/command.Remote sync note
After pulling + reinstalling on the HPC, run
openbench registry pruneonce to un-shadow the bundled catalog (the remote~/.openbenchoverlay is separate).