Add ReuseServerTestCase for class-scoped server reuse by Fniakate8 · Pull Request #13 · valkey-io/valkey-test-framework

Fniakate8 · 2026-07-29T01:05:07Z

This PR implements server reuse for test classes, reducing per-test overhead from (FLUSHALL + CONFIG RESETSTAT).

Whats Added

ReuseServerTestCase now inherits ValkeyTestCase and overrides create_server() to cache the server on first call instead of spawning a new one per test
Configs are snapshotted on server creation and restored between tests to prevent pollution
FLUSHALL + CONFIG RESETSTAT runs between each test for data isolation
Bumps pytest to 7.4.3
How to use

To adopt server reuse in your module, you need to:

Import ReuseServerTestCase from the framework
Change your test base class to inherit from ReuseServerTestCase instead of ValkeyTestCase
your existingsetup_test, create_server()calls, and self.server/self.client assignments all work unchanged.

How I tested with other modules

Bloom module — Changed ValkeyBloomTestCaseBaseto inherit ReuseServerTestCase. Ran test_bloom_basic.py (34 tests). All 34 passed. Tests that use CONFIG SET
(like bf.bloom-memory-usage-limit, maxmemory) no longer pollute later tests thanks to the automatic config restore.
JSON module — Changed JsonTestCase to inherit ReuseServerTestCase. Ran test_json_basic.py (171 tests). 170 passed, 1 failed due to a pre-existing missing SOURCE_DIR env var (unrelated to server reuse).

Server reuse helps most when module loading is expensive (like JSON) and tests are lightweight data operations. Bloom sees less improvement because its tests are compute-heavy (filling filters, sleeping for expiration).

Test plan

Bloom: all 34 tests pass with only base class change
JSON: 170/171 pass with only base class change
Config-heavy tests no longer pollute subsequent tests
Tests that need a fresh server can stay on ValkeyTestCase

Instead of spawning a fresh valkey-server per test (~42ms), one server is started per class and FLUSHALL + CONFIG RESETSTAT (~5ms) run between tests to reset state. This gives ~5-7x speedup for test classes. Changes: - src/conftest.py: add class_port_tracker fixture (scope="class") - src/valkey_test_case.py: add ReuseServerTestCase class - tests/test_reuse_server.py: demo/test proving reuse and isolation Signed-off-by: Fanta Niakate <niakatf@amazon.com>

ReuseServerTestCase now inherits ValkeyTestCase and overrides create_server() to cache the server on first call. Modules only need to change their base class — no other code modifications required. Key changes: - Inherit ValkeyTestCase instead of ValkeyTestCaseBase so all fixtures (setup, port_tracker_fixture) work automatically - Override create_server() to return cached server on subsequent calls - Snapshot all configs on first creation and restore between tests to prevent config pollution across tests - FLUSHALL + CONFIG RESETSTAT between tests for data isolation - Bump pytest to 7.4.3 Tested against bloom (34/34 pass) and JSON (170/171 pass, 1 unrelated env var issue). JSON sees ~17x speedup (8s vs 138s). Signed-off-by: Fanta Niakate <niakatf@amazon.com>

chinguyen21

Overall looks better a lot after the 2nd change. Just a few things to improve. Please check the comments

chinguyen21 · 2026-07-30T20:52:27Z

+
+
+class TestReuseServer(ReuseServerTestCase):
+    """Verifies that server reuse works and tests are isolated."""


Lack of setup_test to call create_server() after redesigning the ReuseServerTestCase in the 2nd commit

you're correct. After the redesign of the server reuse, the server is only created when create_server() is first called. I'll add a setup_test fixture taht calls create_server().

chinguyen21 · 2026-07-30T20:53:21Z

@@ -1,4 +1,4 @@
 valkey
-pytest==6
+pytest==7.4.3


Why do we need this change? Doesn't pytest6 work?

The pytest 6 works but it throws deprecation warnings for the teardown(self) method and pytest 7 handles fixture scoping more reliably (like our class-scoped class_teardown fixture.) but for simplicity I'll remove it.

chinguyen21 · 2026-07-30T21:24:16Z

+    def teardown(self):
+        if hasattr(self.__class__, '_shared_server') and self.__class__._shared_server:
+            client = self.__class__._shared_client
+            client.flushall()


If the server somehow crashes during a test, this will raise an exception. Can we handle it? And make sure if exception happens, we should discard the shared server, so other tests can still run properly

Fixed. I wrapped the teardown in a try/except. If the server crashes, we can just discard _shared_server and _shared_client by setting them to None. The next test's create_server will see None and spin up a new server for concurrent tests.

chinguyen21 · 2026-07-30T21:56:32Z

+
+    def create_server(
+        self,
+        testdir=None,


Is there any benefit to not require testdir?

Added a fallback: if testdir isn't passed, it defaults to self.testdir (same approach as server_path). This way None never reaches the parent, and modules that already pass testdir=self.testdir still work the same.

zackcam · 2026-07-31T18:51:36Z

I let workflows run seems like there are some formatting issues can run black . to fix formatting issues.

zackcam · 2026-07-31T18:56:14Z

+
+
+@pytest.fixture(scope="class")
+def class_port_tracker(request):


Do we need this still? I saw we had an import for this but removed it

- Handle server crash in teardown: if flushall/config reset fails, discard the shared server so next test creates a fresh one - Default testdir to self.testdir when not passed (same pattern as server_path) so None never reaches the parent - Add setup_test fixture to test_reuse_server.py to call create_server() matching the pattern modules use - Remove unused class_port_tracker import from test file - Revert pytest version bump — not needed for server reuse Signed-off-by: Fanta Niakate <niakatf@amazon.com>

Fniakate8 force-pushed the server-reuse-upstream branch 2 times, most recently from e94d1d4 to 5006368 Compare July 29, 2026 01:14

Fanta Niakate added 2 commits July 30, 2026 17:45

Fniakate8 force-pushed the server-reuse-upstream branch from de8c642 to 88d0f96 Compare July 30, 2026 17:50

chinguyen21 reviewed Jul 30, 2026

View reviewed changes

Fniakate8 force-pushed the server-reuse-upstream branch from 482ecaa to 53af29f Compare July 30, 2026 23:59

zackcam reviewed Jul 31, 2026

View reviewed changes

Fniakate8 force-pushed the server-reuse-upstream branch from 53af29f to c3685d4 Compare July 31, 2026 21:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add ReuseServerTestCase for class-scoped server reuse - #13

Add ReuseServerTestCase for class-scoped server reuse#13
Fniakate8 wants to merge 3 commits into
valkey-io:unstablefrom
Fniakate8:server-reuse-upstream

Fniakate8 commented Jul 29, 2026 •

edited

Loading

Uh oh!

chinguyen21 left a comment

Uh oh!

chinguyen21 Jul 30, 2026

Uh oh!

Fniakate8 Jul 30, 2026 •

edited

Loading

Uh oh!

chinguyen21 Jul 30, 2026

Uh oh!

Fniakate8 Jul 30, 2026

Uh oh!

chinguyen21 Jul 30, 2026

Uh oh!

Fniakate8 Jul 30, 2026 •

edited

Loading

Uh oh!

chinguyen21 Jul 30, 2026

Uh oh!

Fniakate8 Jul 30, 2026

Uh oh!

zackcam commented Jul 31, 2026 •

edited

Loading

Uh oh!

zackcam Jul 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants



		class TestReuseServer(ReuseServerTestCase):
		"""Verifies that server reuse works and tests are isolated."""



		@pytest.fixture(scope="class")
		def class_port_tracker(request):

Uh oh!

Conversation

Fniakate8 commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chinguyen21 left a comment

Choose a reason for hiding this comment

Uh oh!

chinguyen21 Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

Fniakate8 Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chinguyen21 Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

Fniakate8 Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

chinguyen21 Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

Fniakate8 Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chinguyen21 Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

Fniakate8 Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

zackcam commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zackcam Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Fniakate8 commented Jul 29, 2026 •

edited

Loading

Fniakate8 Jul 30, 2026 •

edited

Loading

Fniakate8 Jul 30, 2026 •

edited

Loading

zackcam commented Jul 31, 2026 •

edited

Loading