Hello, we had an automatic upgrade from v3.14.4 to v3.15.0 and SlackBridge stopped working. Which is a very big issue for us since we work with another company using this bridge all day long.
I tried to rollback to 3.14.4 by changing the tag in docker-compose.yml and rebuilding the containers but Rocket.Chat throws “Can’t find migration version 225” which I guess is not surprising since mongodb’s structure/schema must have change between 3.14 and 3.15.
If we restore a mongodump backup from before the upgrade we are going to lose a few days of data which is something we would really like to avoid.
This is is errors we get in Rocket.Chat log at the start of the server:
[WARN] @slack/client:WebClient:0 http request failed self signed certificate
[WARN] @slack/client:WebClient:0 http request failed self signed certificate
[WARN] @slack/client:WebClient:0 http request failed self signed certificate
…
And this is what we get when someone posts something in the Slack:
server.js:204 SlackBridge ➔ Slack.error Unhandled error onMessage Error: Not enough permission
at validateMessage (app/lib/server/functions/sendMessage.js:151:10)
at sendMessage (app/lib/server/functions/sendMessage.js:165:2)
at RocketAdapter.createAndSaveMessage (app/slackbridge/server/RocketAdapter.js:499:5)
at SlackAdapter.processNewMessage (app/slackbridge/server/SlackAdapter.js:830:17)
at SlackAdapter.onMessage (app/slackbridge/server/SlackAdapter.js:453:9)
at app/slackbridge/server/SlackAdapter.js:108:11
at runWithEnvironment (packages/meteor.js:1286:24)
Is there any way to fix this without losing days of data ?
Another question :
Is there a procedure documented somewhere to manually migrate mongo documents from one Rocket.Chat install to another?
(mainly messages and attachments but anything else really)
Like which collections are use for which purpose.
Timestamps or other gotchas to take care of, etc.
As far as reverting your mongo, unfortunately it is backups only. The schema gets changed, sometimes (not absolutely always), at which point I don’t believe you can revert it (not that I have ever seen over a few years here)
I was hoping (before learning about the workaroud), to use a 5 days old backup (still using v3.14 so before the problematic update and potential schema change) in a new mongo container and then copy paste the appropriate mongo documents from the v3.15 production container to the new mongo container that still uses the v3.14 schema and once all my data was migrated, lock the version/tag to 3.14 and use that new mongo container as the production server. But I’m thankful for the workaround and glad I didn’t have to attempt any of this… Would you advise against something like this?
I already migrated from GridFS to filesystem but thanks for the reminder. Next time I need to restore things in mongo I will check this script then, thank you. And thank you for making Rocket.Chat!
Personally I wouldn’t mess with a DB - end of story. Way too many things to go wrong - little tiny things with big consequences once day…
If it is just docs then you can get those out and I am sure you’d be OK with a little messing about if you really know what you are doing.
In my case I wouldn’t touch with a barge pole
I do under stand the issue - you backup, upgrade, think it is OK, and 3 days later finally figure out there is an issue. Can’t go forwards, can’t go back.
In general terms on my own small systems I wait for at least a point release before upgrading (3.15 unusually didn’t have one). I always keep a good eye on bugs and wait to see what rolls in. If the coast is clear after a week or two I upgrade.
Note that after he React frontend upgrade we are more focused on bugs/stability/UI fixes at the minute.
I’d just dispense with GridFS entirely and use local storage via a mapped volume. That’ll save you quite a bit of performance.
You can run that, get it all out, and then forget about it - your DB backups will be a little bit smaller
On behalf of everyone at Rocket.Chat thank you very much. It is nice to be appreciated!!
