Rust database access via Python database connection pool v2

[MadLittleMods]

Based on the explorations done in #19824 and #19846,

---

Rust database access via Python database connection pool

This is a stepping stone before we can go full Rust everywhere. We're providing a generic interface as we want database access to work in Synapse and synapse-rust-apps. In synapse-rust-apps, we will use a tokio-postgres based database connection pool so it's full Rust.

We want to avoid the situation where we have two database connection pools (one for Python, one for Rust) as we've run into connection exhaustion problems on Matrix.org before.

As an example of using it and sanity check for all this work (including tests), I've also ported over the /versions handler to the Rust side with database access. The /versions endpoint is the simplest endpoint I could find that still had some database access. Hopefully the refactor on /versions isn't that controversial as it's not really the point of this PR. We can always remove it from this PR but it's just here as a sanity check that all of this works.

Why runInteraction(...)?

Using the same runInteraction pattern that we already have in Synapse means we can port over existing Synapse code/endpoints without much thought. But this pattern also makes sense because we want¹ transactions to have repeatable-read isolation (easy to think about, less foot-guns). Having everything thappen in a function callback means we can do retries for serialization/deadlock errors.

When an application receives this error message, it should abort the current transaction and retry the whole transaction from the beginning. The second time through, the transaction will see the previously-committed change as part of its initial view of the database, so there is no logical conflict in using the new version of the row as the starting point for the new transaction's update.

Note that only updating transactions might need to be retried; read-only transactions will never have serialization conflicts.

-- https://www.postgresql.org/docs/current/transaction-iso.html#XACT-REPEATABLE-READ

As a note, this strategy is less of an impedance mismatch (aligns more closely) with Synapse so the glue code for the python_db_pool should also be simpler.

How does this interact with logcontext (LoggingContext)?

See docs on log contexts for more background.

We already support normal logging from Rust -> Python with pyo3-log and log but as soon as we pass a thread boundary, everything is logged against the sentinel log context. Normally, we want logs and CPU/DB usage correlated with the request that spawned the work.

You can see how I took a stab at fixing this in #19846 by capturing the logcontext in a Tokio task local and re-activating as necessary. For example, in that PR, I reactivated the logcontext in run_python_awaitable(...) which we use to call runInteraction(...) from the Rust side which means all of the database usage is correlated with the request as expected. It also means any log:info!(...) done in run_interaction(...) is correlated correctly. But there needs to be a better story for when you want to log everywhere else.

I haven't explored tracking CPU usage on the Rust side.

I've left all of this out of this PR as I think it will be better to tackle this as a dedicated follow-up. For example, I'm thinking about instead creating a new LoggingContext with the parent_context set to the calling context and try to avoid needing to call set_current_context(...) on the Python side where possible (like tracking CPU).

Testing strategy

Added some tests that exercise some async Rust handlers for the /versions endpoint:

SYNAPSE_TEST_LOG_LEVEL=INFO poetry run trial tests.rest.client.test_versions.VersionsTestCase

Real-world:

1. poetry run synapse_homeserver --config-path homeserver.yaml
2. GET http://localhost:8008/_matrix/client/versions

Dev notes

Example INSERT

self.db_pool
    .run_interaction("asdf", move |txn| {
        async move {
            let _rows = txn
                .query(
                    "INSERT INTO delayed_events (delay_id, user_localpart, delay, send_ts, room_id, event_type, content, is_processed) \
                     VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
                    &["delay_id", "user_localpart", "10", "999991782342348349", "!room", "m.room.message", "content", "0"],
                )
                .await?;

            Ok(())
        }
        .boxed()
    })
    .await?;

Todo

• Audit panic usage (unwrap, expect, panic!, unreachable!)
• Wait for Update FakeChannel.await_result(...) to drive async Rust (Tokio runtime/thread pool) #19879

Pull Request Checklist

• Pull request is based on the develop branch
• Pull request includes a changelog file. The entry should:
  • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
  • Use markdown where necessary, mostly for code blocks.
  • End with either a period (.) or an exclamation mark (!).
  • Start with a capital letter.
  • Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
• Code style is correct (run the linters)

Footnotes

1. To note: Ideally, we'd want the least isolation possible but the problem is that there is no tooling to yell at you when your queries/logic is wrong so repeatable-read isolation is a great balance. ↩

[MadLittleMods]

I suggest reviewing this commit by commit.

It first introduces the database interface structure, then Store usage, and then finally refactoring the /versions endpoint to be handled in Rust.

The /versions endpoint is the simplest endpoint I could find that still had some database access. Hopefully the refactor on /versions isn't that controversial as it's not really the point of this PR. We can always remove it from this PR but it's just here as a sanity check that all of this works.

I've looked over and refined all of this but feel free to call out whatever as I might be cargo-culting too much from the LLM splat that this is based on, #19846 (that PR is a combination of LLM splats and manual refinement)

[MadLittleMods]

As an update, twisted/twisted#12514 is finally part of a Twisted release 26.4.0 (2026-05-11). If we updated our Twisted version, we could probably get rid of all of this ugliness.

But that may be a few years away given our deprecation policy considers a "no-brainer" upgrade once it's available in both the latest Debian Stable (currently Twisted 24.11.0) and Ubuntu LTS repositories (currently Twisted 25.5.0)

[MadLittleMods]

Perhaps this will get fixed by #19876 (comment)

[MadLittleMods]

Made these a set to better represent what it is and more efficient lookups. Touched this because I had to model it in SynapseHomeServerConfig on the Rust side

[MadLittleMods]

For reference, this is a manual translation from synapse/rest/client/versions.py#L105-L213 (not a prompt and pray it's right). I also already asked an LLM to review and spot any mistakes.

[MadLittleMods]

For reference, you can also see what a Rust tokio-postgres based DatabasePool looks like in #19846

It's completely unscrutinized but the point is just that it's possible and we can care about it once we start using it in synapse-rust-apps.

[MadLittleMods]

You can see how handling logcontext might look like in #19846 but per my explanation in the PR description here, it's being left out in favor of a follow-up PR.

[MadLittleMods]

I'm relying on these tests to sanity check that the whole Rust database access thing works.

Is this good enough? Should I be writing tests on the Rust side? Seems hard/weird as our only DatabasePool implementation requires Synapse to be running. How do we make that happen?

I wish I could write some specific Rust db_pool.run_interaction(...)/txn.query(...) scenarios and test results better but generally I guess this is good enough. We could expose that function and run it from here 🤷

[MadLittleMods]

These changes are based on the same decisions being made in #19871

[MadLittleMods]

This seems to cause our trial tests not to finish in CI 🤔

Even tried these changes by themselves in #19879 which also reproduces.

I'll have to figure it out tomorrow but I think this PR is still good to review otherwise.

[MadLittleMods]

Split off to #19879

[MadLittleMods]

Note the tests will fail here until we figure out #19879

[MadLittleMods]

Opinions on self.rust_handlers.versions vs exposing handlers individually?

If this is the pattern we like, we should also move rust/src/rendezvous / rust/src/msc4388_rendezvous here as well (follow-up PR).

[MadLittleMods]

We have the UnstableFeatureMap abstraction separate from SynapseHomeServerConfig as we potentially want to re-use these handlers as part of synapse-rust-apps, Rusty homeserver, etc. We don't want to bog it down with Synapse specific things.

[MadLittleMods]

To make this actually generic (usable outside of Synapse), we probably want to pass this in as well but I've left it for now ⏩

[MadLittleMods]

This kind of thing may be possible. It seems like sea-orm accomplished this somehow:

• Feature Request: Support for async closures SeaQL/sea-orm#2749 (comment)
• https://blog.weiznich.de/blog/diesel-async-0-9/

In any case, we can address this in a follow-up PR.

[MadLittleMods]

One thing that is different from the Python is_feature_enabled(...) is that we don't have any cached lookups on the Rust version

synapse/synapse/storage/databases/main/experimental_features.py

Lines 101 to 104 in 807795e

	@cached()
	async def is_feature_enabled(
	self, user_id: str, feature: "ExperimentalFeature"
	) -> bool:

[erikjohnston]

I think this fine for now, it's quite a small table so lookups should be cheap.

[erikjohnston]

@MadLittleMods cargo test seems to have broken ftr

[erikjohnston]

Other than the test failures I think this looks good

[MadLittleMods]

Had to spread this to a few other places that deal with /versions since it uses async Rust now 😢

But this is our standard pattern for dealing with starting the Tokio runtime.

See #19878 (comment) for more details on how this could go away.

[erikjohnston]

Given we already subclass for lots of tests here:

synapse/tests/server.py

Line 585 in 026937b

class ThreadedMemoryReactorClock(MemoryReactorClock):

I wonder if we can patch it for now by overriding callWhenRunning to automatically run stuff if the reactor is already running?

[MadLittleMods]

I think it's possible. We can consider it in a follow-up PR ⏩

[erikjohnston]

Looks good! Whoop!

[MadLittleMods]

Thanks for the review @erikjohnston and @reivilibre for the good tips and references on a previous exploratory stab at things 🐘