Weekly hveto program added#176
Open
areeda wants to merge 46 commits into
Open
Conversation
* Add support for SNAX triggers * Pin gwtrigfind requirement for snax --------- Co-authored-by: Derek Davis <derek.davis@ldas-pcdev1.ligo-wa.caltech.edu>
…ore error checking
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #176 +/- ##
==========================================
- Coverage 39.88% 33.46% -6.42%
==========================================
Files 21 22 +1
Lines 1735 2104 +369
==========================================
+ Hits 692 704 +12
- Misses 1043 1400 +357
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Contributor
|
The idea of this code makes sense to me. It will be good to have. I have not done a thorough review of the code files though. |
eagoetz
reviewed
Aug 27, 2025
| This directory contains tempplates for Condor cron jobs to run hveto | ||
| on a daily basis as used by the Detchar account. | ||
|
|
||
| These templates are used to create the actual ssubmit files in |
Member
There was a problem hiding this comment.
Minor typo
Suggested change
| These templates are used to create the actual ssubmit files in | |
| These templates are used to create the actual submit files in |
eagoetz
reviewed
Aug 27, 2025
Comment on lines
+8
to
+9
| - hveto-weekly.su - runs once per day and covers the previous 7 days from | ||
| 00:00 to 00:00 UTc No newline at end of file |
Member
There was a problem hiding this comment.
Suggested change
| - hveto-weekly.su - runs once per day and covers the previous 7 days from | |
| 00:00 to 00:00 UTc | |
| - hveto-weekly.sub - runs once per day and covers the previous 7 days from | |
| 00:00 to 00:00 UTC |
eagoetz
reviewed
Aug 27, 2025
| default_end_date = now_utc.strftime('%Y-%m-%d') | ||
|
|
||
| parser.add_argument('-e', '--end', type=to_day, default=default_end_date, | ||
| help='Processng looks back duration days up to but not including this date. (default: %(default)s).') |
Member
There was a problem hiding this comment.
Suggested change
| help='Processng looks back duration days up to but not including this date. (default: %(default)s).') | |
| help='Processing looks back duration days up to but not including this date. (default: %(default)s).') |
eagoetz
reviewed
Aug 27, 2025
Comment on lines
+167
to
+168
| help='We process one day at a time from end daste back to star date each ' | ||
| 'run is duration days long (default: %(default)s).') |
Member
There was a problem hiding this comment.
Suggested change
| help='We process one day at a time from end daste back to star date each ' | |
| 'run is duration days long (default: %(default)s).') | |
| help='Process one day at a time from end date back to start date where ' | |
| 'each run is duration days long (default: %(default)s).') |
| #!/usr/bin/env condor_submit | ||
| # | ||
| # Condor submit file for Hveto weekly processing | ||
| # it runs inn the local universe creates and submits a DAG to the vbanilla universe |
There was a problem hiding this comment.
Suggested change
| # it runs inn the local universe creates and submits a DAG to the vbanilla universe | |
| # it runs in the local universe and creates and submits a DAG to the vanilla universe |
| help='We process one day at a time from end daste back to star date each ' | ||
| 'run is duration days long (default: %(default)s).') | ||
| parser.add_argument('-d', '--duration', type=int, default=7, | ||
| help='Number of days analyze for each day (default: %(default)s).') |
There was a problem hiding this comment.
Suggested change
| help='Number of days analyze for each day (default: %(default)s).') | |
| help='Number of days analyze for each run (default: %(default)s).') |
| parser.add_argument('--stride', type=int, default=1, | ||
| help='Number of days between each hveto run. ' | ||
| 'For example start=7/1, end=7/29, stride=7, duration=7' | ||
| 'will create 4 hveto reports ending on 7/28, 7/21, 7/14, 7/1 each covering 7 days') |
There was a problem hiding this comment.
Suggested change
| 'will create 4 hveto reports ending on 7/28, 7/21, 7/14, 7/1 each covering 7 days') | |
| 'will create 4 hveto reports ending on 7/28, 7/21, 7/14, 7/7 each covering 7 days') |
|
|
||
| To run hveto for every day for the month of July, each run covering 7 days: | ||
|
|
||
| python -m weekly_hveto --start 7/1 --end 7/31 --duration 7 --stride 1 |
There was a problem hiding this comment.
Wouldn't this need to have --end 8/1, to include 7/31?
| dag_file.parent.mkdir(parents=True, exist_ok=True) | ||
|
|
||
| with dag_file.open('w') as dag_fh: | ||
| print(f'# process long duration hveto for {start_day} to {end_day} each {duration} days', file=dag_fh) |
There was a problem hiding this comment.
@areeda I think you might have already fixed this during our meeting, but I think this should be something like '# process long duration hveto for {start_day} to {end_day}, running each {stride} days, with each run covering {duration} days' - but note that it could confuse the user a little, in that there is no run that includes 'end_day' (here and throughout the code).
added 18 commits
September 21, 2025 19:52
…ectory for the subdags
…ectory for the subdags
…ectory for the subdags, again
…ectory for the subdags, again and again
Update submit file templates to include Omega scans
…igure out quoting
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The hveto-weekly program creates a condor DAG and submit files to run the existing hveto program with and arbitrary duration, like 7 days, on a set of dates to make backfilling easy.