Reseed errors out with numpy legacy seeding

[jerry-reevo]

Python Version

3.13

pytest Version

8.4.2

Package Version

4.0.0

Description

When paired with numpy v2.3.2 using macOS arm64, I get the following error when --randomly-seed is not provided:

    def pytest_runtest_teardown(item: Item) -> None:
        if item.config.getoption("randomly_reset_seed"):
>           _reseed(item.config, _crc32(item.nodeid) + 1)

.venv/lib/python3.13/site-packages/pytest_randomly/__init__.py:212: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.venv/lib/python3.13/site-packages/pytest_randomly/__init__.py:180: in _reseed
    reseed(seed)
.venv/lib/python3.13/site-packages/thinc/util.py:96: in fix_random_seed
    numpy.random.seed(seed)
numpy/random/mtrand.pyx:4828: in numpy.random.mtrand.seed
    ???
numpy/random/mtrand.pyx:254: in numpy.random.mtrand.RandomState.seed
    ???
numpy/random/_mt19937.pyx:168: in numpy.random._mt19937.MT19937._legacy_seeding
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   ValueError: Seed must be between 0 and 2**32 - 1

numpy/random/_mt19937.pyx:182: ValueError

[adamchainz]

Hi, I'm struggling to reproduce this issue within the package. I created a new virtual env with Python 3.13, pytest 8.4.2, pytest-randomly 4, Numpy 2.3.2, on macOS ARM, and ran this test file:

import numpy as np


def test_something():
    assert np.random.rand() >= 0

Even running it 1000 times, I get no error.

Do you think the issue is in your own code? I see this in the stack trace:

.venv/lib/python3.13/site-packages/thinc/util.py:96: in fix_random_seed
    numpy.random.seed(seed)

It looks like 'thinc' has an entrypoint to reseed numpy, separate to the numpy support within pytest-randomly.

[adamchainz]

This issue did at least get me to look at the code again, and I found two optimizations related to the Numpy code path: #690 and #691. I don't think either will fix your issue, though!

[jerry-reevo]

@adamchainz Thanks for digging into this! I have thinc in my dependencies via spacy. Is there a way to just unhook thinc from the reseed calls?

[justkrys]

I can confirm this bug. Python 3.12 and pytest-randomly 4.0.1. I too have thinc via spacy. Hope this helps.

[adamchainz]

@adamchainz Thanks for digging into this! I have thinc in my dependencies via spacy. Is there a way to just unhook thinc from the reseed calls?

There's no easy way, since the connection is via package entry point, however you might be able to use patchy in your conftest.py to delete this block:

pytest-randomly/src/pytest_randomly/__init__.py

Lines 164 to 168 in 50de096

	if entrypoint_reseeds is None:
	eps = entry_points(group="pytest_randomly.random_seeder")
	entrypoint_reseeds = [e.load() for e in eps]
	for reseed in entrypoint_reseeds:
	reseed(seed)

I think thinc needs to change here, on this line: https://github.com/explosion/thinc/blob/20eec35566997d5272d083be1f1a0ec9006934ff/thinc/util.py#L101

I am not sure why it reseeds numpy when pytest-randomly already does that for it, so maybe the line can be removed. Otherwise it will need adjusting to add % 2**32 on the seed, to fix it within the supported range, as pytest-randomly already does:

pytest-randomly/src/pytest_randomly/__init__.py

Line 162 in 50de096

np_random.seed(seed % 2**32)

File an issue / PR there?

[jerry-reevo]

@adamchainz Thank you!

Closing this issue as the fix is not in the scope of pytest-randomly

[bnomis]

thinc has not changed in 7 months. pytest-randomly before 4 does not cause this issue.
This difference seems to be in the offset passed into the pytest_runtest_setup/teardown functions. pre-4 it was 1/-1 now it's
_reseed(item.config, (_crc32(item.nodeid) + 1) % 2**32) - the seed + offset now exceeding the size I guess.

[flieee]

wild idea, but could you use try/except blocks around entrypoint reseed calls so the errors/exceptions don't crash the test, but instead log some error?

[adamchainz]

That could work but then it would hide any real errors, which doesn't sound great. Pushing ahead with a test run when something may be very broken doesn't sound like a great idea.

[flieee]

what about this?

try:
    reseed(seed)
except:
    reseed(seed % 2**32)