Skip to content

release: Famedly Synapse v1.152.1_1#256

Merged
jason-famedly merged 4 commits into
masterfrom
famedly-release/v1.152
May 7, 2026
Merged

release: Famedly Synapse v1.152.1_1#256
jason-famedly merged 4 commits into
masterfrom
famedly-release/v1.152

Conversation

@jason-famedly

Copy link
Copy Markdown
Member

🚨 This is a security release

Upstream changes:

  • Prevent CPU starvation (Denial of Service) under worker lock contention, additionally capping the WorkerLock time out interval to a maximum of 60 seconds. Contributed by Famedly. (#19394, ELEMENTSEC-2026-1706, GHSA-8q93-326v-3m7g, CVE pending)
  • Prevent pagination ending when a page is full of rejected events. (ELEMENTSEC-2025-1636, GHSA-6qf2-7x63-mm6v, CVE pending)

Famedly changes:

None (Although we did contribute one of the fixes 🎉 )

jason-famedly and others added 4 commits May 7, 2026 13:25
Fixes the symptoms of element-hq/synapse#19315
/ element-hq/synapse#19588 but not the
underlying reason causing the number to grow so large in the first
place.

```
ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit
```

Copied from the original pull request on [Famedly's Synapse
repo](#221) (with some edits):

Basing the time interval around a 5 seconds leaves a big window of
waiting especially as this window is doubled each retry, when another
worker could be making progress but can not.

Right now, the retry interval in seconds looks like `[0.2, 5, 10, 20,
40, 80, 160, 320, (continues to double)]` after which logging should
start about excessive times and (relatively quickly) end up with an
extremely large retry interval with an unrealistic expectation past the
heat death of the universe. 1 year in seconds = 31,536,000.

With this change, retry intervals in seconds should look more like:

```
[
0.2,
0.4,
0.8,
1.6,
3.2,
6.4,
12.8,
25.6,
51.2,
60, < never goes higher than this
]
```

Logging about excessive wait times will start at 10 minutes.

<details>
<summary>Previous breakdown when we were using 15 minutes</summary>

```
[
0.2,
0.4,
0.8,
1.6,
3.2,
6.4,
12.8,
25.6,
51.2,
102.4,  # 1.7 minutes
204.8,  # 3.41 minutes
409.6,  # 6.83 minutes
819.2,  # 13.65 minutes  < logging about excessive times will start here, 13th iteration
900,  # 15 minutes < never goes higher than this
]
```
</details>

Further suggested work in this area could be to define the cap, the
retry interval starting point and the multiplier depending on how
frequently this lock should be checked. See data below for reasons why.
Increasing the jitter range may also be a good idea

---------

Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
(cherry picked from commit 3f58bc5)
@jason-famedly jason-famedly marked this pull request as ready for review May 7, 2026 15:18
@jason-famedly jason-famedly requested a review from a team as a code owner May 7, 2026 15:18
Copilot AI review requested due to automatic review settings May 7, 2026 15:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Security release bump to Synapse 1.152.1, including upstream fixes for worker-lock contention (DoS mitigation) and /messages pagination through rejected events, plus accompanying regression tests and release metadata updates.

Changes:

  • Update worker lock retry/backoff behavior and add regression tests for retry timeout growth.
  • Fix /rooms/{roomId}/messages to continue pagination even when a page is entirely rejected/filtered events, with a regression test.
  • Bump release/version metadata (pyproject, Debian changelog, CHANGES).

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
synapse/handlers/worker_lock.py Introduces capped/jittered retry timeouts and improved warning logs while waiting for locks.
tests/handlers/test_worker_lock.py Adds regression tests ensuring lock retry timeouts don’t balloon over long simulated waits.
synapse/handlers/pagination.py Uses the limited flag to decide whether to omit the end token on empty pages.
synapse/storage/databases/main/stream.py Clarifies pagination return semantics (especially limited with rejected-event filtering).
tests/rest/client/test_rooms.py Adds regression test ensuring backpagination continues past batches of rejected events.
pyproject.toml Bumps Synapse version to 1.152.1.
debian/changelog Adds 1.152.1 packaging entry.
CHANGES.md Adds 1.152.1 security release notes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread synapse/handlers/worker_lock.py
Comment thread synapse/handlers/worker_lock.py
Comment thread synapse/handlers/worker_lock.py
Comment thread synapse/handlers/worker_lock.py
Comment thread tests/handlers/test_worker_lock.py
Comment thread tests/handlers/test_worker_lock.py
@codecov

codecov Bot commented May 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 60.00000% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.39%. Comparing base (3c6b5d7) to head (d4dd3f4).
⚠️ Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
synapse/handlers/worker_lock.py 58.82% 14 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #256      +/-   ##
==========================================
+ Coverage   80.38%   80.39%   +0.01%     
==========================================
  Files         501      501              
  Lines       72337    72351      +14     
  Branches    10906    10906              
==========================================
+ Hits        58148    58168      +20     
+ Misses      10907    10903       -4     
+ Partials     3282     3280       -2     
Files with missing lines Coverage Δ
synapse/handlers/pagination.py 81.38% <100.00%> (+0.86%) ⬆️
synapse/storage/databases/main/stream.py 79.26% <ø> (ø)
synapse/handlers/worker_lock.py 87.57% <58.82%> (-0.18%) ⬇️

... and 2 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3c6b5d7...d4dd3f4. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jason-famedly jason-famedly added this pull request to the merge queue May 7, 2026
Merged via the queue into master with commit dadce64 May 7, 2026
77 of 79 checks passed
@jason-famedly jason-famedly deleted the famedly-release/v1.152 branch May 7, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants