Skip to content

Upgrading fermipy to numpy>2 and astropy>7#679

Open
facero wants to merge 2 commits into
fermiPy:masterfrom
facero:master
Open

Upgrading fermipy to numpy>2 and astropy>7#679
facero wants to merge 2 commits into
fermiPy:masterfrom
facero:master

Conversation

@facero

@facero facero commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

This PR proposes to fix remaining issues with numpy2 and also issues that I discovered with astropy 6&7.

This follows the PR fixing numpy2 in the fermitools and this PR should be tested in a numpy and astropy 7 environment in conjunction with the fermitools numpy2 version.

Issues solved:

All issues (for now) were related to catalog.py.
I haven't yet tested a longer fermipy analysis than just gta.setup().
I will try later and report if any other bugs are found.

So far this is what I've done.

Numpy 2 related

As numpy2 deprecated the np.string_ call here was failing.

the fix was to use : (np.bytes_, np.str_) to handle both string and bytes columns
Maybe just np.str_ was enough. But at least we should be safe with both.

Astropy >6 related

In astropy >6 , table.join() here strictly rejects missing values in key columns. The join_tables() function in catalog.py was calling join() directly on tables where the left key column (Extended_Source_Name) had masked/NaN values for non-extended sources, causing:
astropy.table.np_utils.TableMergeError: Left key column 'Extended_Source_Name' has missing values

The fix here is more painful and I'm not completly sure that the proposed solution is the best.
The solution is to add a pre-processing step before the join() call that fills masked values in the key column for both tables with 0 or empty strings.

Maybe this entire join_tables function could be revisited and simplified as it is a bit cryptic now.
But let's try this fix first.

facero added 2 commits June 25, 2026 13:56
Fill missing (masked) values in key columns before joining

 In astropy >v6 table.join() strictly rejects missing values in key columns while it was cool with this in astropy v5.
@facero

facero commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

if the tests are running with numpy 1.26, they might fail.

One would need to upgrade the CI environment to astropy >7, numpy 2>, gammapy>2

@facero

facero commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author
      The following packages are incompatible
      ├─ fermitools >=2.2.0 * is installable and it requires
      │  └─ python >=3.11,<3.12.0a0 *, which can be installed;
      └─ python =3.9 * is not installable because it conflicts with any installable versions previously reported.

python 3.9 could be removed from CI

@ndilalla ndilalla self-requested a review June 25, 2026 23:36
@ndilalla

Copy link
Copy Markdown
Contributor

Thanks @facero! I just dropped support for python 3.9 on master. I see though that tests are failing due to:

ImportError: cannot import name 'pixel_scale_angle_at_skycoord' from 'regions._utils.wcs_helpers' (/Users/runner/miniconda3/envs/fermipy/lib/python3.11/site-packages/regions/_utils/wcs_helpers.py)

which is called by gammapy/utils/regions.py. Yesterday region 0.12 was released and gammapy looks not compatible yet. Here you can find the changelog: https://astropy-regions.readthedocs.io/en/stable/changelog.html#changelog

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants