SVM-Based Glitch Detection and Removal

[katefoyle]

This pull request transforms the glitch detection pipeline into a machine learning approach (with an SVM classifier) capable of finding instrumental glitches at lower S/N thresholds and removing only those with glitch likelihoods exceeding a specified threshold (set to .45).

Key Changes

Added SVM glitch detection:

• Added SVM_dpi_eval function to analyze 64ms DPI segments and flag likely glitches
• Lowered SNR candidate thresholds ((stds > 6) and (stds / np.abs(stds2)) > 2) (previously hard-coded to 10 and 3) and passed them as changeable arguments
• Glitch probabilities of candidates logged in data_setup.log
• Intervals with glitch likelihood > .45 get returned and removed from event GTI

Interval handling:

• Updated get_btis_for_glitches to allow customizable S/N thresholds
• t_min, t_max tuples are returned and used to update GTI with true SVM-classified btis

Why?

• Lowering the heat pipe glitch parameters allows the SVM to detect more glitches, which has proven effective in distinguishing these types of glitches from GRBs, SGRs, and background.
• Increased glitch rejection lowers overall noise at any S/N, improving sensitivity to GRBs in ground-based searches.
• Decreases noise from undetected glitches caused by previously strict, hard-coded glitch parameters.

Testing:

• The hk mask, event, and attitude pat.fits files were ran through do_data_setup.py for event obsid 00013856003 (a high-probability glitch as found by the SVM)
  • data_setup.log printed its glitch probability and returned two bad time intervals that overlapped.
  • GTI was updated with the combined bad time interval now removed.
• The above was repeated for an event obsid that did not return any significant glitch probability (00012872142) with initial SVM testing
  • No glitch probability was printed and entire GTI remained intact since no glitch was found.

[jjd330]

Add what you did to test this code

[jjd330]

Once you change evdata to evdata0, can you run through one of the test cases one more time?

Also can you make this pull request not a draft?

[jjd330]

Looks good to me now!