Empty Chats after Update to 8.2.0

Description

Yesterday, I updated to the latest version and after restarting, everything looked fine at first, but then a window popped up (reload to update). I pressed it and then all my chats were empty. Current messages were then visible, but today they are gone again. Can anyone help? Is there anything that can be saved? I had already deleted the snapshot when

Server Setup Information

  • 8.2.0
  • Debiann
  • No docker Version
  • 1
  • DB Replicaset Oplog:
  • 22.16.0
  • 8.2
  • No
  • No

Any additional Information

Check your logs.

Check your Mongo DB entries.

As a sysadmin you should have multiple regular backups, especially when upgrading. If you haven’t then you may be in trouble…

In the admin area under statistics, it still shows me the correct number of messages, and it also shows me everything in the database. I can’t find any errors in the log. However, when I write a new message, it disappears the next day.

Sounds like your messages are being automatically expired.

Go and check ALL your settings. It could be you had a setting “on” that didn’t actually work correctly before and has been “fixed”.

I have seen that happen before.

Put logs in debug mode and check both Rocket and Mongo to track what happens.

I’ve checked all the settings and can’t see anything wrong. I reinstalled the client, and now it says “not logged in,” but new messages are still coming into the chat. The web version looks the same, by the way. So the database seems to be connected correctly, but I think the client has an authentication problem.

You aren’t being very methodical here.

Test with a browser to eliminate the app (that you didn’t mention you were using…)

If the browser is ok completely dump the app cache & log back in. It often needs a cleanup, especially after an update.

Yes, yes, I just wrote in the previous post that it looks exactly the same in the web version, except that the “not logged in” message does not appear there.

Your original server details are a muddle. You make no mention of deployment type (apart from ‘no docker’, whatever that means) what OS or app versions or browser type or versions that you are using.

YOU might know what you have got but that doesn’t help anyone else who can’t see your setup.

Your debugging information is incomplete and so all we can do is make educated guesses at your issue.

The devil will be in your details, and probably the answer will be in your logs if it isn’t a cache issue with your app.

This. View, Force reload. Or go and find your profile and delete it completely.

I have Debian, where a normal MongoDB runs with the RocketChat installation, which is not installed as Docker, but normally via software package:

curl -L https://releases.rocket.chat/latest/download -o /tmp/rocket.chat.tgz
tar -xzf /tmp/rocket.chat.tgz -C /tmp
cd /tmp/bundle/programs/server && npm install

The client version is the latest 4.12.0.

After reinstalling the client, I also deleted everything under appdata.

DB Replicaset Oplog is rs01

Rocket.Chat does not support this deployment method. Using it may lead to unexpected challenges or compatibility issues. The recommended deployment methods are Docker and Kubernetes.

So that’s your first issue. Not my decision - I don’t work here.

it looks exactly the same in the web version

Second, can you confirm positively that it all works OK in the browser or not?

I’m still not 100% sure of this as your details are so disorganised. Again, YOU can see this. I can’t. Describe it methodically and accurately.

If the messages have disappeared in the web version as well then you need to look at your logs. They will tell you what is happening.

All we really know so far is ‘on an unsupported installation the client has an error’

Can you properly document exactly what you see in all clients, and start looking at your logs.

Okay, thanks for the information.

I can’t see anything in the web version either. I can see every participant’s status correctly, as well as the channel names, and I can also post. However, the posts disappear after a while or the next day. The old messages are all gone. Under statistics, however, I see the number 195k messages, which matches the content from the Mongodb.

Then I’ll see if I can find anything in any logs. Thank you.

I just noticed something. This morning, I started a thread in the empty team chat. It disappeared for one of my colleagues and was empty. Then someone else replied in the thread and it became visible again. The logs don’t mention anything about deletion, just normal logs for reading and writing.

Conflicts in the getRoomByTypeAndName state on the server (known pattern in Rocket.Chat since 4.x, see e.g. #22256, #24886)

getRoomByTypeAndName → error-invalid-room → status 400,

then in the same session:

getRoomByTypeAndName → 200,

I downgraded to 8.1.0 and now all my messages are back and everything is fine. So it must be something to do with 8.2.0.

The users got a reload to update button and all the data was back.

I’ve observed the same behavior between 8.1.1 and 8.2.0 using the official Docker images. I’m running a very small instance with a Community “subscription.” One team, less than 5 users, less than five channels. Same behavior in a web browser and in the Windows app. With 8.1.1, I can scroll back in any channel and see all previous messages. With 8.2.0, each channel is empty. Roll back to 8.1.1 and all previous messages reappear in the UI.

While running 8.2.0, I can “find” old messages if I search for a particular word. However, nothing stays in the UI once I exit search. I even tried pinning an old message.

FYI this thread has been referred to the engineering team.

For reference I will point out a few basics on running Rocket once again - these are generic for everyone.

They are to help you help yourself, help the Rocket team, and prevent the XY problem.

If you run Rocket, you are a sysadmin, not just a user. Get over it. With that position comes responsibility. Mainly your users personal data……. If your server gets hacked or fails, as admin it’s your responsibility.

Always read release notes and changelogs BEFORE you start. Read github for bugs.

Never user a ”latest” tag unless you love unexpected surprises. Leave it to developers and test setups only.

For production never use a .0 release. Wait patiently for a .1 release, or a couple of weeks, whichever comes first. For the same reasons above. Unexpected surprises.

ALWAYS test your upgrade first on a separate VM. I have a permanent test box running in a small vm and test on that before upgrading my production servers.

I’ve been doing it this way for almost 10 years now (almost since Rocket was open sourced) and have virtually zero issues with my production servers. One has been upgraded through every version since around 0.24 :slight_smile: And another since 0.32 :rofl:

Make sure you keep track of your upgrade history, and the steps you take when upgrading. Write it all down.

Be methodical. Write down all the details of your server as well - see the bug reportung templates for the info required. Keep it up to date. Hopefully any issue can then be reproduced from your notes.

Some further reading.

Admin.

Development.

Asking for help.

Thanks for reading.

Hi Folks,
Spoke with internal engineering team and current directive is exactly what Reetp shared.
Check query in the database & analyze the HTTP responses.