Description
I run a self hosted snap installation of RocketChat for the school class of my sun. It is used for teacher and parent communication. About a week ago the parents contacted me and told me that some chat content has disappeared. I was able to confirm that a large segments of written chat content in the main channel was no longer showing. This has now happened again (a week has gone by since the previous described loss and the current one) . I was unable to find a reason for the data loss.
Server Setup Information
- Version of Rocket.Chat Server: 2.4.11
- Operating System: Ubuntu 18.04.4 LTS (GNU/Linux 4.15.0-66-generic x86_64)
- Deployment Method:
- Number of Running Instances: 1
- DB Replicaset Oplog: sorry I do not know what that is
- NodeJS Version: v8.17.0
- MongoDB Version: 3.6.14
- Proxy:
- Firewalls involved: ufw
To Action From
OpenSSH ALLOW Anywhere
22/tcp ALLOW Anywhere
443 ALLOW Anywhere
80 ALLOW Anywhere
3000 ALLOW Anywhere
OpenSSH (v6) ALLOW Anywhere (v6)
22/tcp (v6) ALLOW Anywhere (v6)
443 (v6) ALLOW Anywhere (v6)
80 (v6) ALLOW Anywhere (v6)
3000 (v6) ALLOW Anywhere (v6)
Any additional Information
The OS is up to date. I have a standard snap install. Did not modify anything within the snap. I use fail2ban and UFW is running. The server is dedicated for RocketChat. No other software was installed. File uploads are stored in S3 bucket. At the moment of data loss nothing was modified by me as an admin. No restart of the server was found in the logs. The runtime of the server at the moment of data loss was 58 days. I restarted the RocketChat server 8 days before the date loss and that was the runtime at data loss. Two days before the first data loss I set up the connection to a self hosted Jitsi server that I run on a different server. I just added the Jitsi settings into RocketChat. I am the only admin on the system. Automated archiving is turned off.
There is no problem with space.
Filesystem Size Used Avail Use% Mounted on
udev 1.9G 0 1.9G 0% /dev
tmpfs 395M 6.6M 388M 2% /run
/dev/sda3 39G 9.4G 28G 26% /
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/loop0 94M 94M 0 100% /snap/core/8935
/dev/loop1 255M 255M 0 100% /snap/rocketchat-server/1422
/dev/loop2 255M 255M 0 100% /snap/rocketchat-server/1427
/dev/loop3 92M 92M 0 100% /snap/core/8689
/dev/sda2 976M 348M 562M 39% /boot
tmpfs 395M 0 395M 0% /run/user/1000
- Average server load is very low. The chat is used by only 25 people with 6 active users. VPS has 2 cores and 4 gigs of ram.
- I have not added any new cron tasks.
Thank you soooooooo much for taking the time to even read this. I spend two days trying to find the cause of the problem. A experienced sysadmin friend set down with me at night. We tried to use the API to see if we will see the lost text messages in the response. But the API returns the same information as the chat. So I just do not know what to do from here. Thank you! 
