Sliding sync notification uses "Special values" removed from MSC4186

[JadedBlueEyes]

This is apparently leading to notification issues. Ideally rust-sdk would set lazy_members and use the full user ID, as per the current version of the MSC.

https://forgejo.ellis.link/continuwuation/continuwuity/issues/1661

[Hywan]

Thanks for raising the problem.

The Matrix Rust SDK isn't compliant with the latest latest version of the MSC4186. The MSC is unmerged and is changing these past weeks to land in clean state, by breaking some part of the request/response formats. Note that Ruma isn't up-to-date neither.

As soon as the MSC is merged, it will be much easier to align all the implementations, both client- and server-side. For the moment, I'm just not sure what to do.

We can write a patch in matrix_sdk::sliding_sync that manually unfolds the special value $ME to the current user ID. It will solve the initial issue raised in continuwuity/issues/1661, but it doesn't resolve the underlying problem. The required_state format has radically changed recently, and I'm not sure which “version” of the MSC we should follow currently. I guess it's the price for using an unstable MSC, but this situation lasts a bit longer than expected.

[JadedBlueEyes]

It looks like the most recent changes are clarifications now, so merge could come soon!

In the meantime, I think the best course of action would be to make things as compatible as possible with the latest version of the MSC so that switching over is more smooth when the time comes. (We also depend on ruma, so for some of the type changes this probably means aliasing fields and then switching over the serialised key name in the next release, for example. For the required state tuple, I'm not sure how easy it will be but it is possible to handle with a custom deserialise implementation).

The alternative would be to leave everything for the merge and then to do all of the changes at once, but I think that will be more painful the more divergent things are, as to avoid breaking live clients either the server or the client will have to support both formats.

[JadedBlueEyes]

Checkboxes:

• Add set_presence URL param.
• Rename invite_state to stripped_state
• When state is deleted we return a stub {"type: "..", "state_key": ".." } in required_state.
• Rename unstable_expanded_timeline to expanded_timeline
• Add lists to room response
• Add membership field to room response.
• Change the format of required_state in the request.
• Move URL parameters to the request body. This is so that on web every request doesn't need to be pre-flighted for CORS.
• Convert ranges to range in SyncListConfig in the request.
• Make the lists request field "sticky".
• Rename is_invite to is_invited.
• pos can be used in /messages and /relations APIs.
• name can be null if the room name is removed.
• Add a limit of 100 to the maximum number of subscriptions.
• Remove notification_count and highlight_count.

This probably needs a tracking issue in Ruma, Continuwuity and Synapse (if one doesn't exist there already) too

[Hywan]

We are discussing about the migration. Propositions are:

• change the /versions feature flag
• change the URL (simplified_msc3575 to msc4186 for example)
• so that we can update Ruma
• and then we can update the Matrix Rust SDK

This is still under discussion with the MSC authors.

[Hywan]

We have all the tooling in the Matrix Rust SDK to detect the sliding sync version (see VersionBuilder), and do an upgrade automatically, or use the most appropriate version. It's been introduced when switching from MSC3575 to MSC4186. We need to repeat that, and that's what left to be decided in the MSC.

[kovapatrik]

MSC4186 got merged last week. I have little to no knowledge about how or when the MSCs gets implemented in the SDK, so I'm wondering whether the fix for this issue can be started working on or not. I can also work on implementing the new MSC, although I don't really have any knowledge on Matrix:D