First off: this is a follow up to Possibility of Github moderation for community members - #36 by sing.li.
In lights of the breathtakingly high number of open issues I asked, whether it would make sense to give github moderation privileges to assorted community members. Together with with Theo Renck and @sing.li we got a prototype of a Github bot off the ground that I used to delve into the depths of the Rocket.Chat Github repository, more precisely their open bugs. At the time of writing this, I have closed 106 issues with the help of this bot and skimmed the >90 pages of open issues in the process. I won’t say that I have looked at every single issue but I combed through every page to find issues that are not relevant anymore, don’t provide enough information or in some other way justify to close the respective issue.
This post tries to capture this experience and derive actionable information from it.
Quite obviously, I have an interest in Rocket.Chat flourishing. Otherwise I wouldn’t have invested such a huge amount of my spare time (this post alone took me 3h). That being said: I am going to criticize things and I will most probably be using strong language. If you can’t handle that, please move on. I will try to restrain myself, but you know how it is, when you are involved with something…
Also, my critique might, in some places, not be fully justified because I don’t see the inner workings of the Rocket.Chat Company. I can only interpolate from the things I see on Github / other Rocket.Chat means of public communication.
Also I most certainly made mistakes or have thoughts that might be wrong. If you see those, point them out to me, correct them and put things right. I’m thankful in advance already
What I did and what I didn’t do
- I did close issues that were missing information or were filed against some ancient version of Rocket.Chat
- I did close issues that were, quite obviously support-only question where there was no bug involved (and where it was not likely that one was to be found, even when digging deeper)
- I did close issues that were duplicates of other issues. I tried to leave open the bug that had the most valuable information in it in order to, at some point, do the thing that was requested
- I did not close issues that were feature requests (only in very very rare cases, where it was extremely unlikely that this will ever be implemented) - although sometimes you take a step back and think for yourself: has this person EVER considered whether this feature request is of importance to anyone but him and his mum. I could have closed a lot of those, let’s say at least 50. Quite possibly more.
- I did not close issues that needed some kind of decision of the Rocket.Chat company (I will come back to that topic later in this post)
- I did not close issues that had recent activity by Rocket.Chat core members
Furthermore, in the process of interacting with those issues and the community members behind them, I deemed it necessary to justify actions and to do so in a positive and generic manner that, on the one hand doesn’t burn my time by individually replying to each case, and on the other hand, do so in a friendly and determined way. For that I wrote a (non-exhaustive) set of templates that I used.
Of those 106 issues, there was maybe a handful of issues that got a reply after I closed them. Some of them thanked me, some of them made me rethink and reopen the issue, as their argument to not close the issue was good and valid. Overall, I would estimate, that the majority (maybe 60%) of issues are actually dead for one or more of the following reasons:
- the person opening them lost their interest
- the issue they were reporting has already been fixed
- the issue is not relevant anymore because the feature has been superseded / replaced by some other feature
- the person opening them is not willing to provide meaningful information to enable community members to first narrow down and reproduce the issue and then fix it
Common themes, maybe priorities for the future?
In the process of hunting, there were some common themes amongst issues that not always justified closing them as duplicate but could be summarized into “the same category” after all. As a byproduct of this venture, I collected them. This list is not exhaustive but every topic went across my screen at least 3 or (sometimes far) more times:
- Make it possible to delete or limit (in a temporal dimension) chat history
- Support full Markdown speccs, also see: [FIX] Support complete markdown specification by gromain · Pull Request #7454 · RocketChat/Rocket.Chat · GitHub
- Support HTTP-Proxy Configuration for every component that fetches HTTP-stuff
- Backing up Rocket.Chat and restoring it
- Show various information in the user profile from various sources (sometimes from LDAP, sometimes stuff the people can enter themselves… you get the gist…)
- Theming Rocket.Chat (no, not only colors, fonts or CSS separately. Real theming, like in "download this theme zip and boom, completely different style, preferrably one default theme and then as a user preference, so every user can choose their theme (provided only by admins of course))
- Labelling or tagging messages (this is something that deserves a discussion I think, because some proper use case is not inherent in my opinion)
- Possibility to see all users on a server / in a channel / in a group … so… visibility of users as a general topic
- Reporting feature for abusive / malicious messages (maybe including moderation features of some kind)
- Big topic: more fine-grained control over notifications. On a multi-dimensional level, read as: on channel basis, on broadcast-level (@here and friends), on device level, on user-status basis
- Running Rocket.Chat from a subfolder (maybe not subfolder, but rather from something like https://my.host.name/Rocket1 and https://my.host.name/Rocket2)
You know what… why not do a community voting on this? To get some priorities. Of course, the Rocket.Chat team has their own priorities, but getting the community opinion? Why not!
Mistakes by Rocket.Chat in their handling of Github
Quite obviosly, there have been numerous attempts to somehow deal with the huge influx of new issues. Most of them were, if I may say so, implemented half-heartedly:
- labelling issues (there are currently ~100 labels) and never doing anything else with them except for labelling
- assigning issues to core developers - some of them are assigned until today and either dead or, as it seems, of very low priority
- request more information in issues that didn’t provide enough information (labels like “stat: need more info”) but then not closing those issues after a certain time or when there was no feedback
- accepting every feature request that’s relevant to maybe 2 but not more than 10 people in the world
- introducing an issue template far too late (and enforcing it by closing issues that do not adhere)
General stuff that goes wrong
- I still have the feeling there is barely any testing. Sometimes there are issues that have two or even three incarnations until they are truly fixed. Don’t believe me? Well there you go!
- no definition of supported platforms or versions. The practice seems to be to only support current stable. Supported (whatever that means by the way) platforms seem to be like: “well… you know, you try to build it on your Temple OS and we’ll even talk to you and try help you out.” That is surely very nice of the developers, but it’s an utter waste of time that could be invested more wisely!
- Think through issues from the start to the end. I have had multiple occasions where the implication of one new feature or a change in a feature has not been considered for other parts of the application. For example the rendering of a room name (stripping or converting specials chars, putting a hash sign in front of it, take case into account) … it’s been implemented differently in at least two areas of the application currently? This is madness! For details follow up the traces of this issue & this issue and you will have some proper facepalm moments.
- Documentation: I don’t even know where to start here. The documentation of Rocket.Chat is a mess currently. Point in case: it’s gotten better already, true, but try to get a list of every admin option and the way it works. Or how it may or may not intersect with other options. Good luck. This leads to soooo much support overhead and also to errors when developing new features (see bullet point before). Want a good example of how to do it? Sure! Check out the mattermost docs. While not being perfect, this is lightyears ahead of Rocket.Chat docs. EDIT: I didn’t bother to look into the documentation again before writing this. In fact, some things have changed and improved and it seems to be getting better, indeed. The direction is the right one
Actions to be taken
This is a list of actions I think would help solve lots of problems. You will most likely disagree with me, but let’s just have a proper discussion.
- implement regular issue triaging. Don’t say you already have because the last two or three weeks almost nobody else than me and @JSzaszvari have replied to new issues on Github. Rocket.Chat team has to manage the influx of new tickets. One realization of this whole Github Bot for community members is: it is a nice addition to enable community members, but by no means can they (nor should they) manage the influx of new issues
MAKE DECISIONS! (yes, I find this to be paramount, that’s why I’m screaming!) When there’s a new feature request, decide whether you implement it or not and then
- either close the issue (with a template) telling the reporter that this will not be implemented (and preferrably even why)
- or assign it to a realistic milestone (might also be meta-milestones like long-term or whatever) but do so only, if you have thought it through. There’s far more “nice-to-have’s” than you will ever be able to implement!
- then, do regular triaging of your milestones and weed them out
- do not allow new features without them being documented exhaustively. Yes. Sometimes writing documention may even take more time than actually implementing the feature. That’s OK. Make this a part of your culture
- decide for a release schedule. For that, see my issue. You currently have no resources to support more than current stable? It’s OK. Just state it somewhere, it gives you leverage when combatting issues that get reported against old versions.
- do PR reviews fast. Nothing is more disappointing for someone wanting to help, than having to wait for some feedback. One PR that’s taking weeks to review may put off that person forever!
- DO NOT TRY TO SUPPORT A NON-TECH AUDIENCE. This might collide with your business goals of spreading the word about Rocket.Chat but somebody running a server application should know the basics. Somebody not even able to fetch the logs of their instance is just a wast of time. Not as a human being but as someone to tackle issues with on eye level. The community can help those people, but you shouldn’t. Unless they pay you, of course. But even then it might be worth thinking about it…
- implement UI smoke testing. So much issues get reported just because of wrong symbols, wrong place of buttons, disappearance of buttons, non-scrollable pages and the likes
- implement general (automated) testing. Yes, more of it than currently. Write the test before the feature.
- decide for supported deployment methods and document them extensively
- remove all documentation that belongs to non-supported deployment methods (or introduce something like “Community sourced documentation” in a separate place). Make it clear there, that there will be NO official support for those deployment methods
- Consider dropping support for the Electron Client. There are better alternatives currently and the time saved could be invested somewhere else.
Considering the title of this post, you might ask yourselves: Where is the good? The good is, I think, that the Rocket.Chat developers and deciders are open to critique and suggestions and do communicate with the community very much. That’s why I think this is gonna be a nice discussion. I’m looking forward to your feedback!