Supporting rich chatbot messages/menus

@tim.kinnane @sing.li @rodrigo.nascimento @rafael.kellermann @JSzaszvari

Any inputs before we put a bounty (or bounties) on this work?

Great work guys.

Personally Iā€™m still a little confused by the need for templates - or rather, I thought the type and template_type would be synonymous. I thought the type would indicate everything youā€™d need as far as layout - then the templates would be implemented in the codebase, to represent a given type of payload. E.g. Type would be carousel, the payload would contain all the required elements for the carousel and the Rocket.Chatā€™s UI would determine how to display and style it.

I see that adding templates allows greater flexibility, but for me it comes at the cost of simplicity and feels overcooked. Perhaps you could give an example of a real world use case. Iā€™m generally more inclined to keep new features as simple as possible, even if it the solution is limited, otherwise I find that the more ā€˜openā€™ it is, the high cognitive load can be a barrier to adoption.

In saying that, I donā€™t want to slow you down in pushing this forward and Iā€™ll be happy to assist whoever takes the bounty with bouncing around any implementation details. I think the spec is solid and as long as the mobile interface finds a solution that can be effectively ported to the desktop, I think it will be a huge win. We can learn from your experience and use it to iterate something for the wider platform.

Oh, one last (very pedantic) point. Iā€™m not sure snake_case is consistent with the existing schemas and coding standard. Usually camelCase is preferred for javascript projects and might actually be enforced by the linter. Although youā€™re doing this for the native apps, so maybe I donā€™t know what Iā€™m talking about?

We also have a custom message type related requirement on our fork: A user can browse a list of private channels and ask for being added. Then, a message is created in the channel which includes interactive buttons with which an authorized user can accept or decline the join request.

In fact, we decided to go this route. We enhanced the rendering of the field (see here) which allows us to execute any type of interaction by providing a custom template (see sample here)

If the Rocket.Chat basis offered this infrastructure, it was very easy to make Rocketlets provide custom message interaction - However I do not think that this is (the only) way to go, due to its flipsides:

  • No easy integration into the mobile apps
  • No harmonization across message forms
  • Inflexible with respect to ever changing the way itā€™s integrated or getting rid of it

Thus, I very much favor any declarative approach. Going for the API-format of Slack is adopting a de-facto-standard and much appreciated.

1 Like

@tim.kinnane your reasoning around template and template_type makes sense to me. Iā€™ll fix up the doc and even deal with the camel_caseā€¦ Thanks for looking it over! I imagine that your continued input during implementation will be helpful.

Apologies for being absent @bizzbyster @tim.kinnane

Real life commitments, unfortunately, have recently taken a priority over my internet presence.

So far the spec looks amazing. So much detail - This is amazing.

So, Lets get this ball rolling shall we? :slight_smile:

@bizzbyster Have you picked which component you want to start working on first. How far would you say the spec is from being completed or are you satisfied with it in its current state?

The reason I ask is we should start working on bits and work from the ground up.

Personally, I would advise against trying to do all of this in one large chunk of work. Iā€™ve found my self in the situation where it gets overwhelming with the amount of work, as well as issues flying at you from every direction.

I think we should start with modifying the message schema and build up from there, First buttons, then input, then cards, etc and it will all fall into place

I very much agree with:

Slack pretty much set the standard for this stuff, itā€™s what the current format is based off so it should not be too difficult to get these changes in. Once that small body of work is done, then this can be extended for all the other amazing stuff you have come up with.

It also drives more people to the platform if they come from Slack as they immediately understand how things should be structured.

We should probably have a longer chat on open.rocket.chat. So @bizzbyster let me know where you are at with this project at the moment and lets start moving forward. Also if you could give me a rough idea on how you would imagine works best in terms of breaking this work up into chunks that are manageable or how you want to approach it at the moment. Just want to get a better understanding of your requirements to get this over the line.

Thanks all! :smile:

Thanks @ojaegle and @JSzaszvari.

So, first, about the message schema, our gang does not have strong preferences here. We just want it to be flexible and clear for all use cases ā€“ enterprise/team focused, consumer focused, desktop, and mobile. It gets complicated when we try to please everyone ā€“ but letā€™s try anyway! :slight_smile:

Can you send us an example of what youā€™d like to see for one of the rich message types so we can side-by-side the message schema in the spec with the one youā€™d like to see for just a single type? Or, if you prefer, just send a link to the section of the relevant Slack doc?

ā€œPersonally, I would advise against trying to do all of this in one large chunk of work.ā€

Sounds good. We were not sure which way to go. Hereā€™s what Iā€™m thinking:

  1. Get agreement on the message schema
  2. Weā€™ll update our RichMessageTestBot implementation so that it responds with the appropriate schema for each type of rich message so that the developer does not need to worry about mocking this up. Everyone will have access to this server as well to verify that we are following the spec defined in step 1.
    3a. Implement horizontal and vertical buttons performing the first two actions: type postback and type url. Saving WebView for later as that is more difficult.
    3b. In parallel we can work with thiago to get the UX designs completed for buttons and the remaining rich message types for mobile and web client.
  3. Implement the generic card
  4. Implement Webview action for button actions
  5. Implement Custom Keyboards
  6. Implement Carousel

So weā€™d open issues for each of these on all three client repos. Weā€™re assuming weā€™ll get the initial implementation done int he WideChat Android repo which is where weā€™d also be doing initial testing. Hopefully we can also figure out how to get the changes implemented for the other two clients this summer as well ā€“ lots of ways to make that happen.

How does this sound?

1 Like

Weā€™d really like to complete #1 asap. Our team already has some of #2 but is now paused until we can get agreement on message schema.

@sing.li @tim.kinnane @JSzaszvari @ojaegle @gabriel.engel @rafael.kellermann

From what I am reading and seeing, the most common bot platform for team/enterprise/desktop chat is Slack. The most common platform for consumer/mobile is Facebook messenger, at least in the US market. Rocket.chat will be able to support more bots if it is able to support bots of these types. So, if I have a FB messenger bot or Slackbot already, hosted by dialogflow/botkit/wit.ai, I can easily add it to Rocket. It is very much in our interest to be at least somewhat compatible with these platforms. Agree?

So by that reasoning, we should support adapters that will support both types which convert FB formatted and Slack formatted into a single message schema. Do you agree with this? If so then I think we are very flexible about the message schema since in the end an adapter will do the job of converting into it.

Anything wrong with my thinking here?

I do believe that the general agreement at this point is to start with Slackā€™s model first, and evolve from there.

Can we all agree to this approach and fast forward the effort. Thanks for the consideration.

1 Like

Our team will look into this approach and post back here if we see any issues with it today.

All,

We are working on a spec for the first ā€œrich messageā€, which will essentially be buttons inside the attachment object, based on the Slack spec. A couple of very basic things needed for rich messaging are not supported in the Slack spec from what we can tell:

  1. Buttons cannot have images associated with them ā€“ only text.
  2. Type 1 here is not supported https://github.com/WideChat/Rocket.Chat.Android/wiki/A-Survey-of-Rich-Messaging-in-Chatbots#actions-upon-button-click, meaning when clicking a button the rich message cannot send a message back to the bot via the chat window.

We will be including support for both of these in the ā€œbuttons inside attachmentsā€ spec we are working on. Unlike the above proposals, this will be a more incremental approach based on Slackā€™s message schema which will hopefully be easier for the community to digest and get behind and hopefully merge upstream when complete.

Thanks!

1 Like

Hey @bizzbyster, whatā€™s the progress on the rich message feature?

@abhishek.dubey Initial features are being implemented. See https://github.com/WideChat/Rocket.Chat.Android/projects/1 to follow progress.

Initial features are described in this spec: https://github.com/WideChat/Rocket.Chat.Android/wiki/Buttons-Inside-Attachment-Object-Spec.

2 Likes

Good afternoon, how is the development going? Is there documentation for creating message-bound buttons?

Does this Rich message feature available in rocket chat android?

Hi all

How are you doing with rich messages?
Where do you need help?

Cheers!

@MiXaiLL76 @sanal.k @lucbas

Phase 1 in the above has been implemented and merged. We can definitely use the help on Phase 2 work that still needs to be designed in more detail and implemented. Weā€™ll likely pick this work up soon so message me @bizzbyster if you would like to contribute.

Hello, bizzbyster! So rich messages are in core?
I tried finding the way to send rich messages from Phase 1 using Hubot adaptor and canā€™t find any example? Are we able to use it now? Or it is still in development?
Regards.

P.S. Sorry. I was not able to find the right way in example, because I thought I need to use some ā€œcustomMessageā€ function instead of simple ā€œreplyā€. So for anyone like me searching for posting rich messages in Rocket Chat using Hubot, you just need to send object with attachement populated to regulat res.reply() function.

Yes.

@karan.bedi which testbot repo would you recommend for @TooPro to demonstrate the phase 1 rich message functionality that is available in the web client?

@TooPro You can have a look at any of the following

Botkit : https://github.com/WideChat/botkit-richmessage-testbot
Botpress : https://github.com/WideChat/botpress-richmessage-testbot
Bbot: https://github.com/WideChat/bbot-richmessage-testbot
Rasa: https://github.com/WideChat/rasa-richmessage-testbot
Python: https://github.com/WideChat/python-richmessage-testbot

If youā€™re testing on your local machine, I recommend using the Python testbot.

1 Like