From 3cc466cf24a9a1751221869b3fbb271906087d14 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 6 May 2024 01:24:29 -0600 Subject: [PATCH] MSC: Bot buttons & conversations --- proposals/4139-bot-buttons.md | 201 ++++++++++++++++++ proposals/images/4139-01-dice-bot-welcome.png | Bin 0 -> 5825 bytes 2 files changed, 201 insertions(+) create mode 100644 proposals/4139-bot-buttons.md create mode 100644 proposals/images/4139-01-dice-bot-welcome.png diff --git a/proposals/4139-bot-buttons.md b/proposals/4139-bot-buttons.md new file mode 100644 index 00000000000..5ee6e758517 --- /dev/null +++ b/proposals/4139-bot-buttons.md @@ -0,0 +1,201 @@ +# MSC4139: Bot buttons & conversations + +Nearly all bots and bridges in the Matrix ecosystem use a text-based interface to support their +operations. These interfaces are typically highly structured commands and require the user to know +the entire incantation for the action they want to invoke, making them feel like "power user" +features. + +Further, interacting with bots today is extremely transactional: the user sends a command and the +bot performs the action as-is or spews errors back at the user due to a typo. If an error was +returned, the entire command needs to be re-run. + +A more user-friendly approach is to have the user provide the bot with information as needed, +without having to guess at the bot's current state. This proposal calls such an approach a +"conversation" with the bot - the user does something to "start" the conversation, and the bot +provides a limited set of prompts to continue the conversation. This repeats until the conversation +ends (usually by the bot saying so explicitly). Users may hold multiple concurrent conversations +with bots. Conversation starters are deliberately left as a bot implementation detail in this +proposal to allow the ecosystem to explore this new interaction technique. Examples may include the +user opening a DM with the bot, sending a `!command` message, or, in future, sending a slash command +like `/start`. + +This conversation approach is heavily inspired by platforms like Telegram. + +## Proposal + +A new `m.prompts` [mixin](https://github.com/matrix-org/matrix-spec-proposals/blob/main/proposals/1767-extensible-events.md#mixins-specifically-allowed) +is specified which describes actions another user in the room can take to further the conversation. + +The `m.prompts` mixin contains some scoping parameters, rendering hints, and the actual prompts +themselves. For example, when applied to an `m.message` event, the `m.prompts` may look like the +following: + +*Note*: The JSON comments are normative, and irrelevant fields are not shown. + +```jsonc +{ + "type": "m.message", + "sender": "@bot:example.org", + "content": { + "m.text": [ + {"body": "Hello! Say !roll [dice] to roll some dice.", "mimetype": "text/html"}, + {"body": "Hello! Say `!roll [dice]` to get started."} + ], + "m.prompts": { + // Clients which recognize `m.prompts` would use `intro` to render the event instead. This + // allows the remainder of the event to be a fallback for unsupported clients. + "intro": { + "type": "m.message", + "content": { + "m.text": [ + {"body": "Hello! What would you like to roll today?"} + ] + } + }, + // These are the users who should see the `prompts`. Other users may see something like "you + // do not have permission to reply to this message" instead of prompts. `scope` is optional: + // when not supplied, all users who can see the message can respond. When an empty array, no + // one can respond. Clients SHOULD NOT show prompts to users who are descoped. + "scope": [ + "@alice:example.org", + "@bob:example.org", + ], + // These are the options a user has. Note the 2 distinct types and 3 label approaches. + "prompts": [ + { + // `type` is the prompt type: "preset" (show a button) or "input" (shown below) + "type": "preset", + // `id` is used by the bot to figure out what prompt the user picked. It is an opaque ID. + "id": "1d6", + // `label` is an extensible event with deliberately no `type`. + "label": { + "m.text": [{"body": "1 six sided die"}] + } + }, { + "type": "preset", + "id": "surprise", + "label": { + // This should render as an image event, hopefully + // Requires https://github.com/matrix-org/matrix-spec-proposals/pull/3552 + "m.text": [{"body": "🎲❓"}], // fallback + "m.file": { + "url": "mxc://example.org/abc123" + }, + "m.image_details": { + // Clients should impose maximums and minimums here. + "width": 16, + "height": 16 + }, + "m.alt_text": { + "m.text": [{"body": "An image of a 6 sided die with a red question mark over it"}] + } + } + }, { + "type": "input", + "id": "custom", + // Regex the client can use to test input locally. Optional - if not provided the client + // should accept *any* input, including an empty string. + "validator": "[0-9]+d[0-9]+", // `2d20`, etc + "label": { + "m.text": [{"body": "Other"}] + } + } + ] + } + } +} +``` + +In this example, clients which don't support the mixin will see the old-style `!roll 2d6` help text, +allowing the user to continue interacting if needed. Over time, bots may wish to drop this fallback +style and instead use a message like `Hello! Your client doesn't support talking to me :(`. + +Clients which do support `m.prompts` will instead render the `intro` object as the event. It's not +required that the `intro.type` matches the top level event `type`, though it is considered good +practice to do so. The `intro` block is primarily intended to allow senders to tailor their message +for supported clients, as the intent for this proposal is to discourage commands like `!roll` where +possible. + +Prompts SHOULD be rendered in order of the array, and appear below the `intro` rendering. Buttons +SHOULD be used for `preset` prompts, using the provided `label`, and text inputs with `label` as a +prefix or placeholder, and validation per `validator`, SHOULD be used for `input` prompts. For +example: + +![](./images/4139-01-dice-bot-welcome.png) + +[Codepen](https://codepen.io/turt2live/pen/gOyVvaY) (note: doesn't do validation) + +The user is then able to click on one of the buttons or submit text through the `input` option. That +reply looks as follows: + +```jsonc +{ + "type": "m.conversation.reply", + "sender": "@alice:example.org", + "content": { + "m.in_reply_to": { // TODO: Change to match Extensible Event replies + "event_id": "$previousMessage", + "rel_type": "m.thread" // yes, we use threads! + }, + // Whichever option the user clicked is described here in a new content block. + "m.used_prompt": { + "id": "surprise" + }, + // We then add all the fallback representations. For `preset` prompts, this is typically just + // the `label` verbatim. `input` prompts may require some creative editing, like "Other: 2d20". + "m.text": [{"body": "🎲❓"}], // fallback for the image + "m.file": { + "url": "mxc://example.org/abc123" + }, + "m.image_details": { + "width": 16, + "height": 16 + }, + "m.alt_text": { + "m.text": [{"body": "An image of a 6 sided die with a red question mark over it"}] + } + } +} +``` + +The bot can then process this and continue the conversation as needed, using more `m.prompts` mixins +to get the information it needs from the user. If the bot considers the conversation/thread to be +complete, it sends an event with no `m.prompts` mixin to the thread. In our example of a dice bot, +this could be the result of the roll. + +Once a user has picked (and sent) a prompt, the client SHOULD disable the user's ability to send +another. This could be done by hiding all options, or using the HTML `disabled` attribute. + +The example dice bot would then start a new conversation by sending a new welcome message, likely +with different text to feel less mechanical. For example: "What are we rolling next? [1d6] [...]". + +It is left as a bot implementation detail to handle multiple responses, responses from descoped +users, and invalid input. Typically this would be handled by the bot using a threaded reply to the +sender saying "sorry, you don't have permission to interact here" or "sorry, I didn't catch that. +[same prompts as original message]". + +## Potential issues + +TODO + +## Alternatives + +[MSC3006](https://github.com/matrix-org/matrix-spec-proposals/pull/3006) is very similar to this +proposal. Instead of starting per-message threads, it defines interactions via a state event. This +makes MSC3006 more akin to a "conversation starter" replacement, to use this MSC's terminology. + +## Security considerations + +TODO + +## Unstable prefix + +While this proposal is not considered stable, clients should use `org.matrix.msc4139.` in place of +`m.` in all identifiers. + +TODO: Language to support usage in room versions without Extensible Events support, similar to +[MSC3381: Polls](https://github.com/matrix-org/matrix-spec-proposals/blob/main/proposals/3381-polls.md). + +## Dependencies + +This MSC has no direct dependencies. diff --git a/proposals/images/4139-01-dice-bot-welcome.png b/proposals/images/4139-01-dice-bot-welcome.png new file mode 100644 index 0000000000000000000000000000000000000000..54a29679de7e0f7eaa7dadb6563a0ee3d77ece1b GIT binary patch literal 5825 zcmb7Ic{r5O+kQ!sWSt`0*d``hBujP~+t_9zLX^VT*JR(yHufchQ8Z?T2+5KqWEsjf zp_09_Z%Nq(!#DkwZ~g20<9*-ryytq(xzD-Jxu55H63tEZ&oc8e0|0Q=&_EXn0CX@~ zJCKQy_I6m0UZH*H{E_-PK;Y^bYk5oEvql!67p@_Q(0JMrQ7 zpJ=PVD$k#LyaNQ2C7@J{gDRPy^`5s{hqy`{y?sT&7sh4E$m@0I1{HFWP=I-|6yd*XEObtTZwtVEycLz!PJQD|+Q1Qd5B>0{~n)JsO&P6Y7}51e7w` zxg_0$qRspo=mDT8lU>IDbU7$#vkrTEgP*80t)bg|6l5*p5IWIN3jpTKkNfO8F2Q8} z)IO%4+c~X+@H?P+eQnT15G0(VSB!ssR+>P4-sgUCr1VFuyk*o8E3HmLMvsxOM(emO zy!=Jc4AcFC@hGb1DjAY~Gi7*PZwA6qMN+Ib!z0q#hj{_>OLn-@DD%0)KEu`S(i8CCV%HVjVe+H~OvcKX=w*)seH@HbfVDYVbH}>J;$JH!%jk*VNN1URct8U4Nu|z;;>3Ebi!# zQf-i+#LA~X;yg&GJTt|)Y2$H*^fh>n=Yhl8^Rw7DLq!~1sQISF&0=F6RzXQ z!3vK30`1h!#I5oSF0t>_E{<98(3SLZ5mR z78;@D&q$dF=ed=4rOgg!lIge-QhG3Z@H^uYP=|(t#7Ob^HmLbkhauUvLmf8FhmRR$ zJ%~+L1r+`)U#~^YdW8hEyYWUYwhXcOPU26YZg#q0l z@}R9G+Fz5%(PxADI_%858KgSZxB?7JY|&K&Ir6Eu2J%T;*C(!kbNLd@*_m%l-UDqc z%GhjR5U)TrBGra+p=R=ur9O!(WX1uthOi6$c}^_1QHZ=Ui99F895OH$|98fdw&N{h z-!=9Jax_PtuCfkGC#hdhj4ad@IGVhpxh>0a#0rWiE*$0a@n8<&PDM6$Jh zJ_d+O$mFj3$3+mT@zCbc?bKOZ2>eC&>inZTk-vA4xeyw{$-f;Wc{aiWJaud=PNhy> z&EK9}M4}zLC3IChXX*0yzu7@yoA8|I>pjCSgW*fynWw}9mww_g&$ZOJQ*ANOR8S4> z!{EY?--)SKKj)W&ok`i*X35!CNSDdxfvy`+8<$d=He)0V(0|_48gA02lAU^s0-KEa zSPM>2JlUo$?{e|hg|!`iblUHJlmUuQYsMa}6Ei*q&y^ah29Fyz`F`D7Dv1s@972m+ zxao{8V!Lfm| z3)hb${&)_ZlF>o`sO#GW(E&nd|AGPshiIDEL?%S+j~)*V=4>p*sT^hA>SvFaaedCm zxASq}%O}lTm>``W}OiVNPzNBo8=SpulM^lWN%Qp4n(1GefKTnek?|w$x(+*I2?(+ z=hI;TKx~-zw&)o1XDb;Aixg~KL)c87ObN|rjwv9yTMjPwhi%ZhzqVI|BD#`spe9Wgr#iuGb-7Mm!Zf5H)X0f@by~1(0j?)i|bM zqN^*e3kCDFl*XxXxgUZeN4e*KB2a^FJCcQ?raRqdV(`tirRteFMCX@Ig@@m{Xp$zn zz9)OUJbom11g`8@V+YKQ>~JWgh1@)fo`W9P=fp@Pg06QE>SRBBOZ$E=kp)<~XNSWh z8GtCicMu^Mp!Io{7Px_;72nea9DwEGjhF>err3Nss#n{l~H7 zJ6quS`K|2LUm17n#(*Ze=aUG9BS>-*swa(V5w4~SU10V zMzKqJ@vC?3jsM+LwifDq99#>1iI9VC1)1l2(^r#sS+b>o*F+9^vW31DxayEhkrK`89rEy#x) zu9-y)_h;dkg@;PEXhk}a}}SHxSoya8}mw=ZA|tX zf}(v}b?&#WMACnt@@$++vlhG<@$9Z`ty-?ToCo)t0r3|vziC=2dZxN7#1)QaQ`+lR zIX*+*KD%D)@3_-8edN~(aWg=-lW zLMW1+3ll{Pg)%8|x4aaXK6|bRG3{cVBo-Z7t#;1D{BeTc`=ZL(R6$mslk1M~n)@cc6uQVPx~ z%S$|=a_fUy$NKm?OI7ur&2AEz(1&v{wxM)~`a4{zmUur43m5Cv>Q{M}@){PhQCH!0 z7^&RudsoGErIPr4IP|?Amo7(Eludj#)%p^Ym0xV|#@nh)GPp)vOOqQ~HN>;Oy>yT3 zg-u*AKi>MhSfHXfY(pI$B<#xle0+H_cDDGl@DtEyy4l6@+Xe`MT=_=r4m89Adr8bm z3%j9`{aTmL>By1qhWAYo`Z|6{8!9pe8Wof8I0?YDm#H zkvK=aN&D7d?HyO$-TLnm5U^iy9RXbp5eqPqVbtd?{jQeZa?u zOWRDV-K@B;jTXHuE3T}#pqAIcxr<6=W-$^vFI`8riU~hSnIfZR)z`XG6GhVy6BweW zoN?nxhDD*~sh2hXptnng`P>e!msJ-$>o~*+d05TU)6=`N+aq&1&gbK6Zp2!Q#T&i{ zsdZVd_tGy0Mki5|vqgW#<0$bXWbZSU37qD8cnx=sIs1aYvwr_5_i&{x$nquq-z9$ZnqIw;{Gd| zxqIp`ZX;omRzmd#qfRogr6NNIWi}Fm)q721g6IH959E>sx>r_Stn6d5_ZGGd;9zLG zn2`FEY=D4s;m_>a&Mia==w@?&lHE~PY4bX)k^%ri2c*oM$TQsu-(PnTYBoR(g=@k> z`uh>&_^f9pkC}6!^dDs#bS(`tZohV`xngg(uo3K}L7ZekIJ?BQ9~E?*ae33}ezD!Wzc#lM4f^T$sTTHmEbB6>{eC8%DZnB?#uCnA%iHLrdqbDEqB+O4%+ZrqnH;z+Qc zg@#bDK^F#>lT@(0dDJhJZ5_6-^xVL-_wtT9Oq9(jk>8dPo0$?#OWmj`aEp3SU(}r0 zQ#xQiipI{#@3M89!py*Y3e53|F!|g;mPj# zeO$s>oZXQkw6C*u`c2h&S_rLXW0!aJ^dwAAPowo}wQFPDi>>Q@C$x{FH9q{{JSR!= z!-hi!a9t6m@=p>_XGXi-E=0K%-sH5i6(UK=AgYm4Z^o>KrHt%qC*__PTxHW5sy8}Y z>G1DP{h_aq5vdObxNZ4_Nk{+cpo5<0z+PsqSbZXC1(|GZ?f%kTaKbzohe%jgnsr~D zHIrINDq%^iw{-KN(nFp9qD=VuIK%9A)t3G9@|I(KdEPO>9dit6457F*uJ=BXT$59j z5pXL>f`bW_A4FfQ{>=E)5_I0_8tW@p|GVz@hJmd@vHNK0AT=xnr9_cXG#I+t5D^Wk zY0&jF*z960<9k(*-xw||-Ph7yN^~X8JAP!8@$CO4ujf9!28x1za)*CuiiPd8>;(tnu1;n$rSQPK#E8K~T^3?6G zhAY>wiTI#-4>l*(0E5cyLw--P-Ckl7dBGm(V14fXF3^YXnI_?D)_JJp<(~#I!k9sa;+hiqtC{D(>&P8e2X3;)mqtnyleH)%g^RsPssC-|DWT{BP~-gTOEB-{mxeWLKcURfpNG^5g+nX0 zk)NX)sa*|`4lW8;Kw*1<;wik>;!VTDqX&BQ7O-3az6Uv|ov%|{%2=7xwtR4Dm^zy! zdI!=}ARRa1)cPS(QKz;7`Dm5fKrRXX=_QEpt0Oc{FCR4~fHwarNmVELP^7wem=PcM z4fVykyi*GaRT`Qm0u|zOp)%mqzQsYrX!tz>3*=!MzJJuA+BNG{T?RO+;G=P}0>x+u z-boLoXTy{tp!-)kn=q!%BNDqDvwbcX)&DZLS~h*7*3kF+fF3#dt&QB*Olg&h2f3falx~Ki#~{^_ z$Cr&~BlQsu8hkxhXcxD>vc&A^(EYuRoZAX55s>xpw%an;jkZt-b-rdpRJh5YS}E)K zGfg#p^=A8p-IA^^-FQ0dLs@UE{Mp~gIo^X`Cy_5*?QnsFC5~xT(5I_1O099l5y7gv zRNd&m>&@Tns#|QK=7Ypj9dtvFKAAdzsTtiVs3NYOL0ug+|1#`APaHdLuTtcVtaQEN z0)Nzt$OUdDGkiuSTok4))Xj7~D&5g!ma|Tnq|tGaWwhytIE($3IGmZORd7=_0#Qytp^U_Vyph zigADGkN~e(uu<<7x3sUT!JnXjnWK@k!At76Hb$HM!7&RY7