Skip to content

fix: add failback for shef infile import when tsid doesn't exist#218

Merged
msweier merged 4 commits into
mainfrom
fix/shef-infile
Jun 4, 2026
Merged

fix: add failback for shef infile import when tsid doesn't exist#218
msweier merged 4 commits into
mainfrom
fix/shef-infile

Conversation

@msweier

@msweier msweier commented May 13, 2026

Copy link
Copy Markdown
Collaborator

cwmscli/commands/shef/import_infile.py

  • Split store_group into _save (one store→update attempt) and store_group (orchestrator).
  • Added _filter_existing_tsids, which probes each TSID with cwms.get_timeseries_identifier and drops missing ones with a warning.
  • New flow: try the bulk save first; on failure, validate per-TSID, rebuild the payload from survivors, and retry. If every TSID validates, the original error re-raises (failure
    wasn't TSID-related).
  • store_group now accepts entries and category_id so the retry can rebuild the JSON.

cwmscli/commands/commands_cwms.py

  • Expanded the shef import_infile click help= so the auto-generated CLI reference describes the alias-id encoding and the new validate-and-retry fallback.

tests/commands/test_shef_import_infile_dry_run.py

  • Added test_missing_tsid_falls_back_to_validated_save: 3-entry fixture, mocks store_timeseries_groups to fail when the missing TSID is in the payload and update_timeseries_groups
    to fail outright, mocks get_timeseries_identifier to raise for one TSID. Asserts two store calls (3 then 2 members), the missing TSID is absent from the retry, and the expected
    warnings are logged.

@msweier msweier requested review from Enovotny and krowvin as code owners May 13, 2026 20:32
@msweier msweier marked this pull request as draft May 13, 2026 20:36
@msweier msweier marked this pull request as draft May 13, 2026 20:36
@msweier

msweier commented May 13, 2026

Copy link
Copy Markdown
Collaborator Author

hang on....getting more in files to make this importer more robust

@msweier msweier changed the title fix: add shef infile tests for nwo fix: add failback for shef infile import when tsid doesn't exist May 14, 2026
@msweier msweier marked this pull request as ready for review June 4, 2026 15:32
@msweier

msweier commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator Author

@Enovotny this is ready for review/merge. I forgot to follow up. Some of the issues we were having were group issues with empty timeseries. Should be good to go now

@Enovotny

Enovotny commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator

@msweier did you see a new version of shef-parsers that went out yesterday that should fix the empty ts issue?

@msweier

msweier commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator Author

@msweier did you see a new version of shef-parsers that went out yesterday that should fix the empty ts issue?

No i did not. We just got rid of the timeseries as a work around. Glad there is a fix!

@msweier msweier merged commit 77172d2 into main Jun 4, 2026
9 checks passed
@msweier msweier deleted the fix/shef-infile branch June 4, 2026 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants