Sudden 502 error

same here but our mongo dont work

Same here. Ubuntu 18, snap rocketchat-server 2.4.12 , error in mongodb startup

txn-recover: unsupported WiredTiger file version: this build only supports major/minor versions up to 1/0, and the file is version 2/0: WT_ERROR: non-specific WiredTiger error

Did not try to revert to previous version. All services down :frowning:

I was able to repair mongo errors as suggested here but apparently that’s not enough, so I’m not advising it.

OK this means I am not crazy. Glad to see others are having the same problem and its not just me. Also looks like 2.4.12 has been pulled from snap as of minutes ago.

1 Like

“not enough”… What happened after you did the repair? Mongo didn’t start?

if you are not 100% sure what to do , you can spend 1000 of hours in that …
i just hope for the team

… my Company just start to love this Server :sob:

Mongo did start, but rocketchat-server is now failing to open some file or something like that.

thats what i mean … its impossible

propably its an “Upgrade” and our server got that

You need to revert to 2.4.11. You can find the fix here: https://github.com/RocketChat/Rocket.Chat/issues/17628

These are the steps to revert to previous version (From ubuntu-snap channel):

snap revert rocketchat-server # Revert to 2.4.11
sudo snap run --shell rocketchat-server
snapctl get snap-refreshing #which returned "true"
snapctl set snap-refreshing=false
exit

Then I restarted the service:

sudo systemctl restart snap.rocketchat-server.rocketchat-server

I freaked out for 3 hours trying to fix my server. All good now!

3 Likes

Hhmmm… but other users above did try a revert and didn’t succeed, right?

Correct, reverted to 2.4.11 and logs show everything started up, however rocketchat.server will not go “active” shown in “snap info rocketchat-server”. Nothing else shows up in the logs that would mention a problem.

Hmm - I did just this and it came right. Weird, I have been fighting this issue for hours and thought I had done this or something similar but obviously not. Anyway happy to have my users back on and everyone off my back for now.

Worked for me as well! Thank you!

I’m glad to see there is a fix that doesn’t require a restore, however since we couldn’t find anyone else having the issue and I had a snap rocket.chat backup that occurred prior to the upgrade from 2.4.11 to 2.4.12, I did a revert and then a restore to fix our issue. However, one other thing I did prior to starting the rocketchat-server service was to change the channel to 3.x/stable. Then once I started the service, I did a refresh to bring the server up to 3.2.2 to keep it from trying to update to 2.4.12 again.

Set backups to occur before refresh:
sudo snap set rocketchat-server backup-on-refresh=enable

1 Like

A refresh should now also solve. We have reverted the revision at the store level too. I’d recommend looking at Introducing SNAP Tracks! also to lock into a track. Cause when we goto 3.x we want to prevent this… but could happen. We want to make sure everyone has a chance to backup prior to a big upgrade

Reverting for me did not work, but switching channel to 3.x/stable and refreshing worked perfectly. Thank you!

I was in the exact same boat. Thanks a ton!

Today when trying to fix the 502 on our server, I saved my snap:

image

After some failed attemps to fix it, I want to restore it the saved snap, I ran “sudo snap restore 1”. It takes around 15 minutes processing, but when it finishes, nothing happens.

If I ran sudo snap list after running restore, I only have core:

Name Version Rev Tracking Publisher Notes
core 16-2.44.3 9066 latest/stable canonical✓ core

What should be the process to restore the saved snapshot? So I can revert to 2.4.11

I have the same issue on my localhost environment.
The rocketchat does not start after reboot, status is always failure “rocketchat.service: Failed with result ‘exit-code’.” Active: failed (Result: exit-code) since Sat 2020-09-19 19:59:26 EEST; 12min ago
Process: 551 ExecStart=/usr/local/bin/node /opt/Rocket.Chat/main.js (code=exited, status=1/FAILURE)

19 19:59:26 debian rocketchat[551]: at Object.onceWrapper (events.js:300:26)
19 19:59:26 debian rocketchat[551]: at Socket.emit (events.js:210:5)
19 19:59:26 debian rocketchat[551]: at emitErrorNT (internal/streams/destroy.js:92:8)
19 19:59:26 debian rocketchat[551]: at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
19 19:59:26 debian rocketchat[551]: at processTicksAndRejections (internal/process/task_queues.js:8
19 19:59:26 debian rocketchat[551]: name: ‘MongoNetworkError’,
19 19:59:26 debian rocketchat[551]: [Symbol(mongoErrorContextSymbol)]: {}

But when I start rocketchat, “sudo systemctl start rocketchat” it starts to work.

So every time when I launch my virtual machine, everything else, like mongodb etc works but the rocketchat does not start, and to fix it it just needs “sudo systemctl start rocketchat”.

What should I do to fix this ?

  • mongod --version db version v4.4.1
  • npm --version6.14.8
  • rocket chat version 3.6.1
  • node js version v12.14.0

Replying to my self
I was able to fix this with a dirty fix:
put to crontab “@reboot sleep 20 && /etc/startrockechat.sh”
and put in that script “sudo systemctl start rocketchat”

Not any kind of good fix but for some reason it needs to wait something before the start gets succeed. If I put it without delay, the rockechat service on my debian will stay in a fail state.