Hello,
While troubleshooting frequent out-of-memory problems I have been having with the Digital Ocean installation of RocketChat, I have realize that there is no swap file in their standard installation, which explains the frequent crashes, so I made one. I have done absolutely nothing else to the server, but it seems that the RocketChat completely STOPPED running, no node, nor mongo, nor treafik get started. Nothing got deleted from the disk, but nothing is working either.
Actually, the situation is somewhat less clear. The Digital Ocean support team replied to my ticket. They checked the resource usage of my droplet and said that the resources of the server should not be a problem at all for the current usage, and that a memory leak may be more likely. They are looking into this more in details right now. The suspicious thing one notices in the memory logs is that the cached part of memory does not get cleared and it keeps increasing until around 256MB and then it crashes. So I wonder if there might be a limit set to 256MB for the amount of RAM that can be use to cache, because that would explain the crasches. I don’t know how to check that but I have already asked DO to. Although, the real question is why the cashed RAM does not get cleared.
These are the info of the droplet: 1 GB / 1 CPU 25 GB SSD Disk 1000 GB transfer
To answer the other question. The installation described there is the same as mine, the 1-click installer of Digital Ocean does use snap. The memory behaviour described in the question is also precisely the same I have been observing lately.
I understand that more resources might be necessary now and that is absolotely not a problem. I have just got a 2GB droplet to check if the same behaviour happens. What I want to understand is why the RAM usage to cache keeps increasing even when the RocketChat has not been used by anybody in hours, and why it does not get freed up.
Sorry there are some things here that just don’t make sense.
I don’t think it does. Snaps are still on 3.15.x You are running snapd, but that doesn’t mean a snap is installed (and to save some memory I’d disable it if you are not using it as it is a resource hog). I think DO use the rocketchatctl script and it does a manual install. Hence the version.
Note there is a lot of difference between 1Gb and 2Gb at this level. You can’t easily compare them. We have already told you that the OS will take around 1Gb of RAM just to run these days. Yes, you can get leaner systems than say Ubuntu, but for your typical installs that’s 2021 for you.
So you really need a min of 2Gb to run Rocket comfortably. 1Gb will almost certainly give you issues, as you have discovered.
This is not what it appears…
The kernel dynamically uses RAM for buffering & caching to speed up operations and will usually grab as much as it can, but will relinquish some of that if an application needs it directly. So it is not unusual for you to see almost all the RAM used no matter how much you have installed - you can see Freemem is 166.7M in your screenshot and the Cache is 923.4M - that’s nearly 1Gb the kernel has reserved for itself!!
As the applications use RAM the kernel has to reduce the buffers & cache. But it has a minimum requirement for itself. Once you start squeezing this you are in a world of pain. For Ubuntu 20 they recommend a minimum of 512Mb RAM just to run. That does not include ANY services. Just the OS.
On your 1Gb droplet the applications - both Rocket AND Mongo - will want RAM to operate. Check Mongo requirements for the application, and caching - it needs quite a lot. Probably both need more than the 512Mb left. And you have no swap. And almost no buffers or cache…
You can of course help a bit by using a swap file but it is just a slow alternative to real RAM.
I think you’ll find on your 2Gb droplet things will run without any issues. Let us know.
Oh, okay, I was wrong about the DO installation type indeed. Thank you for the explanation, I really appreciate the help you have been providing me in the last days.
I did no mean to be difficult. I expect to be using RocketChat on a relatively large scale in the future, which is why I moved to DO. However, since the managing of the servers at Digital Ocean is on us, I need to build a good understanding of what happens at a somewhat deeper level. And of course I’d rather do it now while still testing things out.
I will let you know if the 2GB droplet also gives me problems.
