Skip to content

Added statistical test output to ocrreject_exam#197

Open
m-dallas wants to merge 4 commits into
spacetelescope:masterfrom
m-dallas:md_ocrreject_exam_updates
Open

Added statistical test output to ocrreject_exam#197
m-dallas wants to merge 4 commits into
spacetelescope:masterfrom
m-dallas:md_ocrreject_exam_updates

Conversation

@m-dallas

Copy link
Copy Markdown
Contributor

Major improvements
• Renamed avg_ratio to combined_ratio since the value isn’t really an average.
• Added the new method combined_ratio_threshold() which generates the highest combined_ratio possible under the null hypothesis that CRs are randomly distributed across the detector given a significance level (alpha) and a mean number of pixels each CR hit is expected to impact (cr_size). This was developed in response to feedback that there should be an included way for users to immediately see if a dataset was likely overflagged.
• Added overflagged_stat to the results output. The value is True if the observed combined_ratio >= to the threshold set by combined_ratio_threshold(). Comparing to manually checked datasets from 2016-2025 shows this stat is agrees with the manual classification of the data ~93% of the time.

Bug fixes:
• There was a bug in how the extraction box region was defined. Previously it would always extend 1 pixel lower than it should. This updated version is the same as what was shown in STIS ISR 2019-02 and ensures the correct extraction box size.
• There was a bug where subarray readouts wouldn’t properly render in the html files. This was fixed.
• Trying to only show figures in an interactive environment was buggy (especially with qt), so this was changed to the “show_plot” argument which allows users to choose if they want the plots to render or just to be saved to a directory.

The tests and docstrings have been updated to reflect these changes. This also shouldn’t break the plot rendering in the STIS notebooks but it’s worth double checking.

@m-dallas m-dallas requested a review from a team as a code owner June 12, 2026 20:53

@sean-lockwood sean-lockwood left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest making PEP8 changes as suggested via:

pycodestyle --ignore=E501,W503,W504 ocrreject_exam.py

I haven't tried running the tool yet.

Comment thread stistools/ocrreject_exam.py Outdated
Comment thread stistools/ocrreject_exam.py Outdated
Comment thread stistools/ocrreject_exam.py Outdated
@m-dallas

Copy link
Copy Markdown
Contributor Author

Updated the script to comply with PEP8, I also double checked that the tool is giving the results we expect.

@m-dallas m-dallas requested a review from sean-lockwood June 24, 2026 18:10
Comment thread stistools/ocrreject_exam.py Outdated
@m-dallas m-dallas requested a review from sean-lockwood June 27, 2026 23:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants