Complement `TestRoomSummaryAllowedRoomIDs/restricted_room_includes_allowed_room_ids` is flaky

[MadLittleMods]

The TestRoomSummaryAllowedRoomIDs/restricted_room_includes_allowed_room_ids Complement test is flaky (test was added in matrix-org/complement#873)

Added to #18537 (list of known flaky tests)

Complement test source: tests/room_summary_test.go#L59-L69

Example: https://github.com/element-hq/synapse/actions/runs/28601133960/job/84838454090?pr=19871

❌ TestRoomSummaryAllowedRoomIDs/restricted_room_includes_allowed_room_ids (410ms)
      room_summary_test.go:61: MatchResponse key 'join_rule' missing - http://127.0.0.1:32853/_matrix/client/v1/room_summary/%!l(MISSING)ZgQByxwZPjAuAnLnk:hs1 => {"room_id":"!lZgQByxwZPjAuAnLnk:hs1","room_version":"8","num_joined_members":1,"world_readable":false,"guest_can_join":false,"allowed_room_ids":["!NCUEzyGBEscdYvWgLu:hs1"],"membership":"join"}

Investigation

Synapse currently pulls out the join_rule from the room_stats_state table but this is populated in the background as replication traffic reaches the worker that is configured to run_background_tasks.

synapse/synapse/handlers/room_summary.py

Lines 782 to 804 in 5df6d1b

	stats = await self._store.get_room_with_stats(room_id)
	# currently this should be impossible because we call
	# _is_local_room_accessible on the room before we get here, so
	# there should always be an entry
	assert stats is not None, "unable to retrieve stats for %s" % (room_id,)
	entry: JsonDict = {
	"room_id": stats.room_id,
	"room_version": stats.version,
	"name": stats.name,
	"topic": stats.topic,
	"canonical_alias": stats.canonical_alias,
	"num_joined_members": stats.joined_members,
	"avatar_url": stats.avatar,
	"join_rule": stats.join_rules,
	"world_readable": (
	stats.history_visibility == HistoryVisibility.WORLD_READABLE
	),
	"guest_can_join": stats.guest_access == "can_join",
	"room_type": stats.room_type,
	"encryption": stats.encryption,
	}

synapse/synapse/handlers/stats.py

Line 320 in 5df6d1b

await self.store.update_room_state(room_id, state)

This means that a workerized deployment of Synapse will have some delay in the room summary being correct after the state changes. Currently, the Complement test expects things to be updated immediately.

What's extra strange is that allowed_room_ids appears in the /room_summary response but not the join_rule. This is because we look at the actual m.room.join_rule state event in the room for allowed_room_ids but the stats for the join_rule. Since, we have to fetch the m.room.join_rule state event in some cases, we could just always pull from that source of truth instead. This would fix this particular test 💪

Although, there is a bigger question around how the Complement test expects the /room_summary endpoint to give accurate results immediately. And is another example of a read-after-write inconsistency problem in Synapse.

[gamesguru]

I believe this may fix it, but i'm not sure why it should be required. Seems like database write-transaction/read-context-scope issue?

diff --git a/synapse/handlers/room_summary.py b/synapse/handlers/room_summary.py
index 0bc6021fa72..6c86969f2dd 100644
--- a/synapse/handlers/room_summary.py
+++ b/synapse/handlers/room_summary.py
@@ -814,6 +814,18 @@ async def _build_room_entry(self, room_id: str) -> JsonDict:
             )
         )
 
+        # If the background statistics processor is lagging behind room creation,
+        # stats.join_rules may be None. Fall back to fetching the join rule directly
+        # from the room's current state to ensure we always return it.
+        if entry["join_rule"] is None:
+            join_rules_event_id = join_rules_state_ids.get((EventTypes.JoinRules, ""))
+            if join_rules_event_id:
+                join_rules_event = await self._store.get_event(
+                    join_rules_event_id, allow_none=True
+                )
+                if join_rules_event:
+                    entry["join_rule"] = join_rules_event.content.get("join_rule")
+
         try:
             room_version = await self._store.get_room_version(room_id)
         except UnsupportedRoomVersionError: