Description
ISSUE 1: After setting up my storage, I am confused as to why my storage for my root volume is so high. When I run df -h I see the following:
/app/bundle $ df -h
Filesystem Size Used Available Use% Mounted on
overlay 100.0G 79.6G 20.3G 80% /
tmpfs 64.0M 0 64.0M 0% /dev
tmpfs 15.3G 0 15.3G 0% /sys/fs/cgroup
/dev/nvme0n1p1 100.0G 79.6G 20.3G 80% /tmp
/dev/nvme1n1 196.7G 24.0K 196.7G 0% /app/uploads
/dev/nvme0n1p1 100.0G 79.6G 20.3G 80% /etc/hosts
/dev/nvme0n1p1 100.0G 79.6G 20.3G 80% /dev/termination-log
/dev/nvme0n1p1 100.0G 79.6G 20.3G 80% /etc/hostname
/dev/nvme0n1p1 100.0G 79.6G 20.3G 80% /etc/resolv.conf
shm 64.0M 0 64.0M 0% /dev/shm
tmpfs 4.0G 12.0K 4.0G 0% /run/secrets/kubernetes.io/serviceaccount
tmpfs 15.3G 0 15.3G 0% /proc/acpi
tmpfs 64.0M 0 64.0M 0% /proc/kcore
tmpfs 64.0M 0 64.0M 0% /proc/keys
tmpfs 64.0M 0 64.0M 0% /proc/latency_stats
tmpfs 64.0M 0 64.0M 0% /proc/timer_list
tmpfs 64.0M 0 64.0M 0% /proc/sched_debug
tmpfs 15.3G 0 15.3G 0% /sys/firmware
It used to be 40 GiB used but now it doubled with the upgrade. Why is this occurring?
ISSUE 2: I came across another issue for the logs of /livez and /readyz. Before, I had /health and was working great until I upgraded and saw this:
{"level":40,"time":"2025-10-22T00:58:21.384Z","pid":1,"hostname":"rocketchat-rocketchat-58d8b79d86-8dbpr","name":"System","msg":"Deprecated /health endpoint was called. Please update to /livez or /readyz."}
{"level":40,"time":"2025-10-22T00:58:33.020Z","pid":1,"hostname":"rocketchat-rocketchat-58d8b79d86-8dbpr","name":"System","msg":"Deprecated /health endpoint was called. Please update to /livez or /readyz."}
{"level":40,"time":"2025-10-22T00:58:36.383Z","pid":1,"hostname":"rocketchat-rocketchat-58d8b79d86-8dbpr","name":"System","msg":"Deprecated /health endpoint was called. Please update to /livez or /readyz."}
{"level":40,"time":"2025-10-22T00:58:48.022Z","pid":1,"hostname":"rocketchat-rocketchat-58d8b79d86-8dbpr","name":"System","msg":"Deprecated /health endpoint was called. Please update to /livez or /readyz."}
So I figured I should change readinessProbe to /readyz and livenessProbe to /livez. Once I did this my deployment would continuously fail with the following message:
│ {"level":50,"time":"2025-10-22T01:01:49.769Z","pid":1,"hostname":"rocketchat-rocketchat-5977bdfd49-swq79","name":"System","msg":"Liveness check failed","details":{"status":"unavailable","checks":{"memory":{"status":"degraded","percentile":92.83},"eventLoop":{"status":"ok","lagMs":17.55},"mongo":{"status":"ok"}}}} │
│ {"level":40,"time":"2025-10-22T01:01:51.810Z","pid":1,"hostname":"rocketchat-rocketchat-5977bdfd49-swq79","name":"System","msg":"Readiness check failed","details":{"status":"unavailable","checks":{"memory":{"status":"degraded","percentile":92.8},"eventLoop":{"status":"ok","lagMs":11.02},"mongo":{"status":"ok"}}}}
I thought huh, that’s a pretty high memory. Let me increase my resources for my memory to 4Gi and I still receive this error.
So a twofold issue that I am experiencing. Would appreciate any help since we are wanting to move to Enterprise license with RocketChat.
Server Setup Information
- Version of Rocket.Chat Server: v7.11
- Operating System: Alpine
- Deployment Method: Helm Chart (AWS EKS)
- Number of Running Instances: 1
- MongoDB Version: v7.0.25
- Proxy: No proxy
- Firewalls involved: None