Skip to content

Update mktrace from gfit to Astropy fitting#174

Draft
sean-lockwood wants to merge 5 commits into
masterfrom
sl_mktrace
Draft

Update mktrace from gfit to Astropy fitting#174
sean-lockwood wants to merge 5 commits into
masterfrom
sl_mktrace

Conversation

@sean-lockwood

@sean-lockwood sean-lockwood commented Jan 29, 2025

Copy link
Copy Markdown
Member

Version 3.0.0. Builds onto PR #37.

  • Remove dependence on deprecated stsci.tools.gfit Gaussian fitting; replace with Astropy fitting.
  • Allow for scipy API change for scipy.signal.boxcar to scipy.signal.windows.boxcar.
  • Changed Gaussian fitting initial guess to peak.
  • Aggregate Astropy fitting warnings (with ierr > 4 into a single warning for each call of Trace.gFitTrace().
  • Set data types to Float as they are created rather than after, avoiding conversions.
  • Update test stdout. Will still require updating files (after manual validation) on Artifactory.

Remove dependence on deprecated stsci.tools.gfit Gaussian fitting; replace with Astropy fitting.
Allow for scipy API change for scipy.signal.boxcar to scipy.signal.windows.boxcar.
Changed Gaussian fitting initial guess to peak.
Aggregate Astropy fitting warnings (with ierr > 4 into a single warning for each call of Trace.gFitTrace().
Set data types to Float as they are created rather than after, avoiding conversions.
@sean-lockwood sean-lockwood self-assigned this Jan 29, 2025
@sean-lockwood

Copy link
Copy Markdown
Member Author

Early checks show differences in _crj_1dt.fits trace of 2e-3 to 5e-7 pixels, depending on the dataset.

It appears that gfit was more robust to initial conditions in the mean. So while setting the Astropy Gaussian model's mean to the peak location seems to work, this could result in different answers from gfit's center-of-range initial value.

We should probably construct a test suite across flux levels and modes in order to compare results before merging this change.

@sean-lockwood

Copy link
Copy Markdown
Member Author

Validation work is ongoing. I compiled a test suite and have been measuring the scale of the resulting changes.

@sean-lockwood

Copy link
Copy Markdown
Member Author

Some rows are not fitting the same regardless of changes to model initial parameters, e.g.:
image

… in the middle of the fit region. Increased maxiter to 250. Simplified scipy imports. Updated user warning stacklevel to be more useful.
@sean-lockwood

Copy link
Copy Markdown
Member Author

Applying a Hann window to the fitting weights allows us to bias the result toward the center:
image

Going across all the columns of test data set o5f301020_crj.fits shows more consistent behavior:
image

@sean-lockwood

Copy link
Copy Markdown
Member Author

Hann window weighting:
image

Comment thread stistools/mktrace.py

# Bias 1D Gaussian fit toward the center of the range:
window = hann(specimage.shape[0], sym=True)
window /= window.sum()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this cause problems for E1 data?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be agnostic to Y-position unless it falls off the detector/subarray dimensions.

I did find some bad cases caused by the weights on the edges being zero, so changed the windowing slightly in the next commit.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I think I understand...

The window it's fitting each column in is 41 pixels tall (as set by the subim_size = 40 parameter).

…. Perform a global initial guess for the trace amplitude (scalar median) and peak position (3σ-clipped linear fit); default to scalar median peak position when this fails.
@sean-lockwood

Copy link
Copy Markdown
Member Author

Trace comparison: trace_results_5.pdf

Quick view of data for reference: quickview
(Ignore the echelle datasets for now; I'll remove these soon.)

@sean-lockwood

sean-lockwood commented Feb 17, 2025

Copy link
Copy Markdown
Member Author

@stscirij -
I'm planning to present this to the STIS team after I'm satisfied with tuning the algorithm and validation testing.

The current code produces 4 output FITS files with 1D images. I'm considering merging these into a binary FITS table of a single product. Do you know of any dependencies on mktrace products that might break?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants