Sliding Sync: Fix outlier re-persisting causing problems with sliding sync tables

[MadLittleMods]

Fix outlier re-persisting causing problems with sliding sync tables

Follow-up to #17512

When running on matrix.org, we discovered that a remote invite is first persisted as an outlier and then re-persisted again where it is de-outliered. The first the time, the outlier is persisted with one stream_ordering but when persisted again and de-outliered, it is assigned a different stream_ordering that won't end up being used. Since we call _calculate_sliding_sync_table_changes() before _update_outliers_txn() which fixes this discrepancy (always use the stream_ordering from the first time it was persisted), we're working with an unreliable stream_ordering value that will possibly be unused and not make it into the events table.

TODO

• Add test

Developer notes

SYNAPSE_TEST_LOG_LEVEL=INFO poetry run trial tests.storage.test_sliding_sync_tables.SlidingSyncTablesTestCase.test_joined_room_outlier_and_deoutlier

SYNAPSE_POSTGRES=1 SYNAPSE_POSTGRES_USER=postgres SYNAPSE_TEST_LOG_LEVEL=INFO poetry run trial tests.storage.test_sliding_sync_tables.SlidingSyncTablesTestCase.test_joined_room_outlier_and_deoutlier

Pull Request Checklist

• Pull request is based on the develop branch
• Pull request includes a changelog file. The entry should:
  • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
  • Use markdown where necessary, mostly for code blocks.
  • End with either a period (.) or an exclamation mark (!).
  • Start with a capital letter.
  • Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
• Code style is correct
  (run the linters)

[MadLittleMods]

This could be simplified to remove ORDER BY stream_ordering DESC. This just gives a more correct answer but we will end up with the correct answer anyway when _update_sliding_sync_tables_with_new_persisted_events_txn() runs right after this.

Suggested change

	(SELECT stream_ordering FROM events WHERE room_id = ? ORDER BY stream_ordering DESC LIMIT 1),
	(SELECT stream_ordering FROM events WHERE room_id = ? LIMIT 1),

Up to you

[MadLittleMods]

Found this logic bug because of the new tests. This is a separate problem though that we're fixing.

Now we're actually fetching the missing_event_ids if we have any.

[MadLittleMods]

Thanks for the review and merge @erikjohnston 🐁

[MadLittleMods]

We should have fallen back to "master" when this isn't filled in

Suggested change

	(SELECT instance_name FROM events WHERE event_id = ?)
	(SELECT COALESCE(instance_name, 'master') FROM events WHERE event_id = ?)

Fixed up in #17636 ⏩