Limit to-device EDU sizes by erikjohnston · Pull Request #19617 · element-hq/synapse

erikjohnston · 2026-03-27T15:45:04Z

This is based on #18416, which got reverted (#19614) due to it incorrectly rejecting to-device messages to users with many devices (and thus breaking message sending).

Fix #17035

A to-device message content looks like:

{
  "@user:domain": {"device1": {...}, "device2": {...}},
  ...
}

The previous PR would split up into multiple EDUs, each with a subset of the users. However, if one user's entry was too large it would not further split it up and then error out.

The main change in this PR is to allow splitting up a single user into multiple EDUs.

Other changes:

Rename to SOFT_MAX_EDU_SIZE to indicate that we sometimes send EDUs with larger size than that, and its more a target than a hard limit.
Check early if any to-device message (to a specific device) is too large to send, even if we're not going to send it over federation. This ensures that we catch issues where clients try to send too large to-device.

This still means that if a client send a large individual to-device message it will fail, but I don't believe we ever send such large to-device messages (normally they're in the range of a few KB).

I ended up changing the implementation a bunch to make it easy to reuse the code to split up dictionaries. Instead of repeatedly splitting up the EDU until each bit fits into the size, we instead record the size of each entry in the dict and instead split up based on cumulative size. This means we call encode_canonical_json on each entry rather than once on the entire struct, but its not significantly slower to do so.

--

cc @MatMaul @MadLittleMods

If a set of messages exceeds this limit, the messages are split across several EDUs. Fix #17035 (should) There is currently [no official specced limit for EDUs](matrix-org/matrix-spec#807), but the consensus seems to be that it would be useful to have one to avoid this bug by bounding the transaction size. As a side effect it also limits the size of a single to-device message to a bit less than 65536. This should probably be added to the spec similarly to the [message size limit.](https://spec.matrix.org/v1.14/client-server-api/#size-limits) Spec PR: matrix-org/matrix-spec#2340 --------- Co-authored-by: mcalinghee <mcalinghee.dev@gmail.com> Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>

This is based on #18416, which got reverted due to it rejecting to-device messages to users with many devices. The main change here is that if a to-device EDU for a single user is too large, then we split it up into multiple EDUs.

It's not that much more costly to just keep re-encoding the values.

Copilot

Pull request overview

This PR updates Synapse’s handling of outbound to-device messages so that outgoing m.direct_to_device EDUs are kept to a target size by splitting payloads more granularly (including splitting a single recipient’s devices across multiple EDUs), while also rejecting individual to-device messages that are too large.

Changes:

Introduce a generic dict-splitting helper (split_dict_to_fit_to_size) and unit tests for its behavior.
Refactor to-device sending to split remote to-device payloads across multiple EDUs (including per-user device splitting) and queue them separately for federation.
Add constants for EDU size targets / transaction EDU budgeting and expand tests to cover splitting and size-limit failures.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tests/util/test_split_dict.py	Adds unit tests for the new dict-splitting utility.
tests/rest/client/test_sendtodevice.py	Adds tests for to-device size rejection and EDU/transaction splitting behavior.
tests/handlers/test_appservice.py	Updates test setup to use the renamed datastore method for local to-device inbox seeding.
synapse/util/init.py	Adds `split_dict_to_fit_to_size` and helper state/size-calculation logic.
synapse/storage/databases/main/deviceinbox.py	Splits the old “add to-device inbox” method into separate local vs remote queueing APIs.
synapse/handlers/devicemessage.py	Enforces per-message size checking and implements splitting remote to-device messages into multiple EDUs.
synapse/federation/sender/per_destination_queue.py	Uses constants for EDU transaction limits and reserved EDU slots.
synapse/api/constants.py	Introduces `SOFT_MAX_EDU_SIZE` and transaction EDU constants.
changelog.d/19617.bugfix	Documents the bugfix around large to-device queues blocking outbound federation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-29T12:13:33Z

+def split_dict_to_fit_to_size(
+    original_dict: dict[str, Any],
+    *,
+    soft_max_size: int,
+    wrapping_object_size: int = 2,
+) -> Iterator[tuple[dict[str, JsonDict], int]]:
+    """Splits a dict up into a list of dicts, each of which is small enough to


The return type of split_dict_to_fit_to_size is too specific: it currently declares Iterator[tuple[dict[str, JsonDict], int]], but this helper is used (and tested) with non-dict values (e.g. strings) and should preserve the original value type. Consider making it generic over the value type (TypeVar) and returning dict[str, V] (or dict[str, Any]) to avoid mypy errors and misleading annotations.

Copilot · 2026-04-29T12:13:33Z

+    for key, payload in original_dict.items():
+        current_payload.subset[key] = payload
+        current_size = _len_with_wrapping_object(
+            current_payload.subset, wrapping_object_size
+        )


split_dict_to_fit_to_size recomputes encode_canonical_json(current_payload.subset) on every iteration as the subset grows, making the algorithm O(n^2) in the number of entries. For large to-device payloads (many devices/users), this can become a noticeable CPU cost; consider precomputing per-entry sizes once and tracking cumulative size (including commas/quotes/keys) to split without repeatedly re-encoding the whole subset.

The PR description mentions calling encode_canonical_json on each entry. But Copilot is correct; we are technically calling encode_canonical_json multiple times on each entry in current_payload.subset as we build it up.

I presume you'd have to store the length for each entry in subset to avoid this.

That was the original implementation, but you have to be careful of handling extra chars like commas, etc.

The common case should be that the to-device content is smaller than max, and therefore we take the fast path of only serializing once.

anoadragon453

Overall looks great. Just some minor things below.

anoadragon453 · 2026-04-29T14:33:55Z

+def split_dict_to_fit_to_size(
+    original_dict: dict[str, Any],
+    *,
+    soft_max_size: int,
+    wrapping_object_size: int = 2,
+) -> Iterator[tuple[dict[str, JsonDict], int]]:


We should note that this is a generator in the docstring.

We could annotate that it's a generator, but the fact its a generator is an implementation detail? A generator like this is an Iterator

anoadragon453 · 2026-04-29T14:40:26Z

+    for key, payload in original_dict.items():
+        current_payload.subset[key] = payload
+        current_size = _len_with_wrapping_object(
+            current_payload.subset, wrapping_object_size
+        )


The PR description mentions calling encode_canonical_json on each entry. But Copilot is correct; we are technically calling encode_canonical_json multiple times on each entry in current_payload.subset as we build it up.

I presume you'd have to store the length for each entry in subset to avoid this.

MatMaul · 2026-06-03T10:13:35Z

Thank you all for getting this through ❤️

MadLittleMods · 2026-06-03T21:55:03Z

+
+        self.assertEqual(number_of_edus_sent, 2)
+
+    def test_edu_large_messages_not_splitting_one_user(self) -> None:


Just saw this test flake in another PR,

❌ https://github.com/element-hq/synapse/actions/runs/26910432685/job/79387059438?pr=19648

Error: Traceback (most recent call last): File "/home/runner/.cache/pypoetry/virtualenvs/matrix-synapse-pswDeSvb-py3.14/lib/python3.14/site-packages/twisted/logger/_observer.py", line 81, in __call__ observer(event) File "/home/runner/.cache/pypoetry/virtualenvs/matrix-synapse-pswDeSvb-py3.14/lib/python3.14/site-packages/twisted/logger/_legacy.py", line 90, in __call__ self.legacyObserver(event) File "/home/runner/.cache/pypoetry/virtualenvs/matrix-synapse-pswDeSvb-py3.14/lib/python3.14/site-packages/twisted/trial/_dist/workertrial.py", line 44, in emit self.protocol.callRemote(managercommands.TestWrite, out=text) File "/home/runner/.cache/pypoetry/virtualenvs/matrix-synapse-pswDeSvb-py3.14/lib/python3.14/site-packages/twisted/protocols/amp.py", line 951, in callRemote return co._doCommand(self) File "/home/runner/.cache/pypoetry/virtualenvs/matrix-synapse-pswDeSvb-py3.14/lib/python3.14/site-packages/twisted/protocols/amp.py", line 1973, in _doCommand d = proto._sendBoxCommand( File "/home/runner/.cache/pypoetry/virtualenvs/matrix-synapse-pswDeSvb-py3.14/lib/python3.14/site-packages/twisted/protocols/amp.py", line 884, in _sendBoxCommand box._sendTo(self.boxSender) File "/home/runner/.cache/pypoetry/virtualenvs/matrix-synapse-pswDeSvb-py3.14/lib/python3.14/site-packages/twisted/protocols/amp.py", line 713, in _sendTo proto.sendBox(self) File "/home/runner/.cache/pypoetry/virtualenvs/matrix-synapse-pswDeSvb-py3.14/lib/python3.14/site-packages/twisted/protocols/amp.py", line 2358, in sendBox self.transport.write(box.serialize()) File "/home/runner/.cache/pypoetry/virtualenvs/matrix-synapse-pswDeSvb-py3.14/lib/python3.14/site-packages/twisted/protocols/amp.py", line 692, in serialize raise TooLong(False, True, v, k) twisted.protocols.amp.TooLong: tests.rest.client.test_sendtodevice.SendToDeviceTestCase.test_edu_large_messages_not_splitting_one_user

Previous discussions in the original PR:

Limit outgoing to_device EDU size to 65536 #18416 (comment)

Limit outgoing to_device EDU size to 65536 #18416 (comment)

How we worked around this before:

Work around twisted.protocols.amp.TooLong error by reducing logging in some tests. #18736

@user

This is based on element-hq#18416, which got reverted (element-hq#19614) due to it incorrectly rejecting to-device messages to users with many devices (and thus breaking message sending). Fix element-hq#17035 A to-device message content looks like: ```jsonc { "@user:domain": {"device1": {...}, "device2": {...}}, ... } ``` The previous PR would split up into multiple EDUs, each with a subset of the users. However, if one user's entry was too large it would not further split it up and then error out. The main change in this PR is to allow splitting up a single user into multiple EDUs. Other changes: 1. Rename to `SOFT_MAX_EDU_SIZE` to indicate that we sometimes send EDUs with larger size than that, and its more a target than a hard limit. 2. Check early if any to-device message (to a specific device) is too large to send, even if we're not going to send it over federation. This ensures that we catch issues where clients try to send too large to-device. This still means that if a client send a large individual to-device message it will fail, but I don't believe we ever send such large to-device messages (normally they're in the range of a few KB). --- I ended up changing the implementation a bunch to make it easy to reuse the code to split up dictionaries. Instead of repeatedly splitting up the EDU until each bit fits into the size, we instead record the size of each entry in the dict and instead split up based on cumulative size. This means we call `encode_canonical_json` on each entry rather than once on the entire struct, but its not significantly slower to do so. -- cc @MatMaul @MadLittleMods --------- Co-authored-by: Mathieu Velten <matmaul@gmail.com> Co-authored-by: mcalinghee <mcalinghee.dev@gmail.com> Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>

MatMaul and others added 3 commits March 27, 2026 12:01

Limit to-device EDU sizes

b98769a

This is based on #18416, which got reverted due to it rejecting to-device messages to users with many devices. The main change here is that if a to-device EDU for a single user is too large, then we split it up into multiple EDUs.

Newsfile

8c5b122

MadLittleMods added the A-Federation label Mar 27, 2026

MadLittleMods reviewed Mar 27, 2026

View reviewed changes

Comment thread synapse/handlers/devicemessage.py Outdated

Comment thread changelog.d/19617.bugfix

erikjohnston added 2 commits March 31, 2026 11:57

Avoid, where possible, using magic values to calculate json size

7afc60e

It's not that much more costly to just keep re-encoding the values.

Add fast path

ded7a66

MadLittleMods reviewed Mar 31, 2026

View reviewed changes

Comment thread synapse/handlers/devicemessage.py Outdated

Comment thread synapse/handlers/devicemessage.py

erikjohnston added 3 commits April 1, 2026 11:40

Add note about 413 not specced

32134a5

split_device_messages doesn't raise an error

448a17a

Add dedicated constant for to-device content limit

1f0ea43

erikjohnston marked this pull request as ready for review April 1, 2026 10:55

erikjohnston requested a review from a team as a code owner April 1, 2026 10:55

anoadragon453 requested a review from Copilot April 29, 2026 12:07

Copilot started reviewing on behalf of anoadragon453 April 29, 2026 12:07 View session

Copilot AI reviewed Apr 29, 2026

View reviewed changes

anoadragon453 approved these changes Apr 29, 2026

View reviewed changes

MadLittleMods mentioned this pull request May 11, 2026

A long queue of to-device messages can prevent outgoing federation working #17035

Closed

erikjohnston added 6 commits June 2, 2026 15:14

Merge remote-tracking branch 'origin/develop' into erikj/limit_to_device

ec6b7c1

Remove unused stuff from test

6d8ce63

Fix comment typo

f87d95a

Add message length in error

d2baa57

Fix docstring

65011eb

Don't oversplit EDUs

9f9935b

erikjohnston enabled auto-merge (squash) June 2, 2026 15:42

erikjohnston merged commit 5bf825c into develop Jun 2, 2026
79 of 81 checks passed

erikjohnston deleted the erikj/limit_to_device branch June 2, 2026 15:57

MadLittleMods reviewed Jun 3, 2026

View reviewed changes

MadLittleMods mentioned this pull request Jun 3, 2026

Limit outgoing to_device EDU size to 65536 #18416

Merged

3 tasks


		self.assertEqual(number_of_edus_sent, 2)

		def test_edu_large_messages_not_splitting_one_user(self) -> None:

Uh oh!

Conversation

erikjohnston commented Mar 27, 2026 • edited by MadLittleMods Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

anoadragon453 Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

erikjohnston Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

anoadragon453 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

anoadragon453 Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

erikjohnston Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

anoadragon453 Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MatMaul commented Jun 3, 2026

Uh oh!

MadLittleMods Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

erikjohnston commented Mar 27, 2026 •

edited by MadLittleMods

Loading

MadLittleMods Jun 3, 2026 •

edited

Loading