We migrated our Multi-node / replica set RC solution to a new Kubernetes cluster, after the migration notifications from chat rooms no longer illuminate/notify of new messages reliably. Multiple clients types (iOS, Electron, Web, Android) are all affected but not all rooms/conversations.
Server Setup Information
- Version of Rocket.Chat Server: 1.3.2
- Operating System: k8s
- Deployment Method: Azure AKS
- Number of Running Instances: 3
- DB Replicaset Oplog: yes
- NodeJS Version: v8.11.4
- MongoDB Version: 4.0.12
- Proxy: Nginx
- Firewalls involved: n/m
Any additional Information
Deployment is leveraging Helm for both database AND rocketchat. https://github.com/helm/charts/tree/master/stable/rocketchat
The migration process has dump the database, reload the database on the new site and turn up RC replica.
Post migration the only odd errors I saw were
StreamBroadcast ➔ Stream.error Stream broadcast from ‘10.61.25.47:3000’ to ‘10.61.17.198:3000/’ with name notify-room not authorized
and these have subsided.