Skip to content

chore: Rework testing to reduce time spent and to remove the hatch run cov command#131

Merged
jason-famedly merged 1 commit into
mainfrom
jason/parallel-test-runs-2
Apr 24, 2026
Merged

chore: Rework testing to reduce time spent and to remove the hatch run cov command#131
jason-famedly merged 1 commit into
mainfrom
jason/parallel-test-runs-2

Conversation

@jason-famedly

@jason-famedly jason-famedly commented Apr 3, 2026

Copy link
Copy Markdown
Member

Tests were taking to long. Let's speed them up!

Before, we ran unit tests and collected coverage with the command hatch run cov. This ran the tests on a single process/thread and (in CI) tested against both SQLite and Postgres as a matrix, consecutively.

Now, the built-in hatch test command can be used for the same purpose.
hatch test will run all tests without coverage on a single process/thread
Add -p as an option to run all tests using all available cores/threads of a CPU.
Add -c as an option to collect coverage data.

These are interchangeable. Previously, on my machine these tests with coverage took just over 2 minutes, now it is just over 30 seconds. Without coverage is even faster.

SQLite testing will be the default. Running in Postgres is only slightly more labor intensive. A Postgres installation is required of course(see the python.yml testing workflow for a quick example) and then add to the front of your command the ENV variables that the testing suite will automatically look for:
SYNAPSE_POSTGRES=1 SYNAPSE_POSTGRES_USER=<theDatabaseUser> SYNAPSE_POSTGRES_PASSWORD=<theDBUsersPassword> hatch test

@jason-famedly jason-famedly force-pushed the jason/parallel-test-runs-2 branch 2 times, most recently from cf9e507 to 96990f8 Compare April 3, 2026 17:23
@codecov

codecov Bot commented Apr 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.61%. Comparing base (90f0f8d) to head (5d25743).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #131      +/-   ##
==========================================
- Coverage   97.24%   95.61%   -1.63%     
==========================================
  Files           7        7              
  Lines         798      798              
  Branches      118      118              
==========================================
- Hits          776      763      -13     
  Misses         22       22              
- Partials        0       13      +13     

see 2 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 90f0f8d...5d25743. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jason-famedly jason-famedly force-pushed the jason/parallel-test-runs-2 branch 17 times, most recently from aa65675 to 989ce58 Compare April 6, 2026 15:06
Comment thread pyproject.toml
Comment thread pyproject.toml
@jason-famedly jason-famedly force-pushed the jason/parallel-test-runs-2 branch from ed27a62 to 9724fc2 Compare April 6, 2026 16:36
@jason-famedly jason-famedly changed the title experiment chore: Rework testing to reduce time spent and to remove the hatch run cov command Apr 6, 2026
@jason-famedly jason-famedly force-pushed the jason/parallel-test-runs-2 branch from 9724fc2 to c2542a1 Compare April 6, 2026 17:06
@jason-famedly jason-famedly marked this pull request as ready for review April 6, 2026 17:11
@jason-famedly jason-famedly requested a review from a team as a code owner April 6, 2026 17:11
@jason-famedly

Copy link
Copy Markdown
Member Author

This will need the required test checks adjusted before it can be merged, as they will have changed

Comment thread pyproject.toml
@jason-famedly jason-famedly force-pushed the jason/parallel-test-runs-2 branch from 9d5d3f7 to 5d25743 Compare April 20, 2026 12:56

@FrenchGithubUser FrenchGithubUser left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@jason-famedly jason-famedly merged commit 4da25e5 into main Apr 24, 2026
10 checks passed
@jason-famedly jason-famedly deleted the jason/parallel-test-runs-2 branch April 24, 2026 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants