Messages greyed out: Error "No real time data received recently"

Community Support
1

Description

Seeing this issue after migrating from one physical server to another. Not sure why. It’s docker, so ideally the move shouldn’t have any effect on how it runs. I literally moved the compose.yml, the data directory, and my uploads directory over, then just did

docker compose up -d

The site is accessible, messages show, but they show greyed out when sent, and others don’t get the notification of a new message. We have to refresh in the browser, and reload in the electron app. The Mobile app kind of does better, but still no notifications, and not sure why.

Even restarts don’t fix this issue for me.

Running RocketChat Community Edition 6.8 as of 5-21-24 with this issue btw.

Server Setup Information

  • Version of Rocket.Chat Server: RocketChat CE 6.8
  • Operating System: Ubuntu 22.04 LTS Server
  • Deployment Method: docker
  • Number of Running Instances: 1
  • DB Replicaset Oplog: Yes
  • NodeJS Version: Whatever is in the ‘latest’ docker image builds
  • MongoDB Version: 5.0 according to the docker-compose.yml
  • Proxy: NginX Proxy Manager (NGinX
  • Firewalls involved: None

Any additional Information

a bit of logging, this really just seems to repeat over and over…done with
docker compose logs -f

mongo-1       | {"t":{"$date":"2024-05-22T15:18:58.448+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"Checkpointer","msg":"WiredTiger message","attr":{"message":"[1716391138:448417][1:0x7f8e33503700], WT_SESSION.checkpoint: [WT_VERB_CHECKPOINT_PROGRESS] saving checkpoint snapshot min: 12970, snapshot max: 12970 snapshot count: 0, oldest timestamp: (1716224436, 1) , meta checkpoint timestamp: (1716224736, 1) base write gen: 23012970"}}
rocketchat-1  | {"level":50,"time":"2024-05-22T15:19:03.430Z","pid":8,"hostname":"50eb92aa526c","name":"System","msg":"No real time data received recently"}
rocketchat-1  | {"level":50,"time":"2024-05-22T15:19:23.430Z","pid":8,"hostname":"50eb92aa526c","name":"System","msg":"No real time data received recently"}
rocketchat-1  | {"level":50,"time":"2024-05-22T15:19:43.430Z","pid":8,"hostname":"50eb92aa526c","name":"System","msg":"No real time data received recently"}
mongo-1       | {"t":{"$date":"2024-05-22T15:19:58.665+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"Checkpointer","msg":"WiredTiger message","attr":{"message":"[1716391198:665733][1:0x7f8e33503700], WT_SESSION.checkpoint: [WT_VERB_CHECKPOINT_PROGRESS] saving checkpoint snapshot min: 12980, snapshot max: 12980 snapshot count: 0, oldest timestamp: (1716224436, 1) , meta checkpoint timestamp: (1716224736, 1) base write gen: 23012970"}}
rocketchat-1  | {"level":50,"time":"2024-05-22T15:20:03.430Z","pid":8,"hostname":"50eb92aa526c","name":"System","msg":"No real time data received recently"}
rocketchat-1  | {"level":50,"time":"2024-05-22T15:20:23.430Z","pid":8,"hostname":"50eb92aa526c","name":"System","msg":"No real time data received recently"}
rocketchat-1  | {"level":50,"time":"2024-05-22T15:20:43.430Z","pid":8,"hostname":"50eb92aa526c","name":"System","msg":"No real time data received recently"}
mongo-1       | {"t":{"$date":"2024-05-22T15:20:58.777+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"Checkpointer","msg":"WiredTiger message","attr":{"message":"[1716391258:777052][1:0x7f8e33503700], WT_SESSION.checkpoint: [WT_VERB_CHECKPOINT_PROGRESS] saving checkpoint snapshot min: 12994, snapshot max: 12994 snapshot count: 0, oldest timestamp: (1716224436, 1) , meta checkpoint timestamp: (1716224736, 1) base write gen: 23012970"}}
rocketchat-1  | {"level":50,"time":"2024-05-22T15:21:03.430Z","pid":8,"hostname":"50eb92aa526c","name":"System","msg":"No real time data received recently"}
rocketchat-1  | {"level":50,"time":"2024-05-22T15:21:23.430Z","pid":8,"hostname":"50eb92aa526c","name":"System","msg":"No real time data received recently"}
rocketchat-1  | {"level":50,"time":"2024-05-22T15:21:43.430Z","pid":8,"hostname":"50eb92aa526c","name":"System","msg":"No real time data received recently"}
mongo-1       | {"t":{"$date":"2024-05-22T15:21:58.969+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"Checkpointer","msg":"WiredTiger message","attr":{"message":"[1716391318:969441][1:0x7f8e33503700], WT_SESSION.checkpoint: [WT_VERB_CHECKPOINT_PROGRESS] saving checkpoint snapshot min: 13013, snapshot max: 13013 snapshot count: 0, oldest timestamp: (1716224436, 1) , meta checkpoint timestamp: (1716224736, 1) base write gen: 23012970"}}
rocketchat-1  | {"level":50,"time":"2024-05-22T15:22:03.430Z","pid":8,"hostname":"50eb92aa526c","name":"System","msg":"No real time data received recently"}

Any help is greatly appreciated.

2

Thanks for creating a new issue - it is easier to track and doesn’t annoy old posters!

First thing is can you check the registration is OK?

And make sure websockets are working correctly.

3

If you can guide me on how to check that I’m happy to do it.

For websockets, not sure how to test it. I can say I setup the same system (same migrated stuff) on my homelab without a proxy in front of it, just using IP and port, and I saw the same behavior. That doesn’t mean websockets are working, just helps rule out the reverse proxy.

In the subscription screen it shows “Community”. Not sure what other registration you want to know about.

4

Community - fine. No read receipts.

Check the license says it’s ok.

Websockets - wrote this years back. First lort of call.

I really suggest for your own security you run a reverse proxy + letsencrypt certs. Cost is zero. Apps won’t work without - at least ios.

Dehydrated is your lightweight friend.

Insane not too.

N.B. I am away for a long weekend - I’ll get back to you as and when I can!

5

License does say OK.

Will go over the GitHub issue in a bit to check websocket is working. I do use a reverse proxy with LetsEncrypt already.

Mobile App (iOS ) updates ok, but guessing that’s a push notification vs websocket.

6

Ok, I get a 101 response in the WS filter under Network in the dev tools. I only see a single websocket connection after refresh. Assuming that’s ok, but not certain. Image attached.

Screenshot from 2024-05-22 16-00-55
Screenshot from 2024-05-22 16-00-551910×417 62.3 KB

7

Websocket seems ok.

Can you post your compose file less sensitive bits please?

Note I’m away later today back Weds so I’m not ignoring you!

Will try and speak to a dev in the meantime.

8

Sorry, took me longer than I expected.

Here is the one I’ve been running:

---
services:
  rocketchat:
    image: rocketchat/rocket.chat:latest
    command: >
      bash -c
        "for (( ; ; )); do
          node main.js &&
          s=$$? && break || s=$$?;
          echo \"Could not start Rocket.Chat. Waiting 5 secs...\";
          sleep 5;
        done; (exit $$s)"
    restart: unless-stopped
    volumes:
      - ./uploads:/app/uploads
    environment:
      - PORT=3000
      - ROOT_URL=http://localhost:3000
      #- MONGO_URL=mongodb://mongo:27017/rocketchat?replicaSet=rs0&directConnection=true
      - MONGO_OPLOG_URL=mongodb://mongo:27017/local?replicaSet=rs0&directConnection=true
      - MONGO_URL=mongodb://mongo:27017/rocketchat
      #- MONGO_OPLOG_URL=mongodb://mongo:27017/local
      - MAIL_URL=smtp://smtp.email
    depends_on:
      - mongo
    ports:
      - 3000:3000
    labels:
      - "traefik.backend=rocketchat"
      - "traefik.frontend.rule=Host: your.domain.tld"

  mongo:
    image: mongo:5.0
    restart: unless-stopped
    volumes:
      - ./data/db:/data/db
      - ./data/dump:/dump
    labels:
      - "traefik.enable=false"
    environment:
      MONGODB_REPLICA_SET_MODE: primary
      MONGODB_REPLICA_SET_NAME: ${MONGODB_REPLICA_SET_NAME:-rs0}
      MONGODB_PORT_NUMBER: ${MONGODB_PORT_NUMBER:-27017}
      MONGODB_INITIAL_PRIMARY_HOST: ${MONGODB_INITIAL_PRIMARY_HOST:-mongodb}
      MONGODB_INITIAL_PRIMARY_PORT_NUMBER: ${MONGODB_INITIAL_PRIMARY_PORT_NUMBER:-27017}
      MONGODB_ADVERTISED_HOSTNAME: ${MONGODB_ADVERTISED_HOSTNAME:-mongodb}
      MONGODB_ENABLE_JOURNAL: ${MONGODB_ENABLE_JOURNAL:-true}
      ALLOW_EMPTY_PASSWORD: ${ALLOW_EMPTY_PASSWORD:-yes}

networks:
  default:
    name: my-main-net
    external: true

I have a couple of lines commented out for testing, but no matter what mix I use those in, it doesn’t change the behavior.

It’s very odd for sure. A friend found a github topic about something similar in 2022 / 2023, and I went all the way through it, and even created a “new” docker compose file, and tried it, but same result.

Here’s that fiel, just for reference:

---
services:
  rocketchat:
    image: registry.rocket.chat/rocketchat/rocket.chat:latest
    command: >
      bash -c
        "for i in `seq 1 30`; do
          node main.js &&
          s=$$? && break || s=$$?;
          echo \"Tried $$i times. Waiting 5 secs...\";
          sleep 5;
        done; (exit $$s)"
    restart: unless-stopped
    volumes:
      - ./uploads:/app/uploads
    environment:
      MONGO_URL: mongodb://mongo:27017/rocketchat?replicaSet=rs0
      MONGO_OPLOG_URL: mongodb://mongo:27017/local?replicaSet=rs0
      ROOT_URL: http://localhost:3000
      DEPLOY_METHOD: docker
      DEPLOY_PLATFORM: linux
      PORT: 3000
    depends_on:
      - mongo
    links:
      - mongo
    ports:
      - 3000:3000

  mongo:
    image: mongo:5.0.26
    restart: unless-stopped
    volumes:
     - ./data/db:/data/db
     - ./data/dump:/dump
    command: mongod --oplogSize 128 --replSet rs0

  mongo-init-replica:
    image: mongo:5.0.26
    command: >
      bash -c
        "for i in `seq 1 30`; do
          mongo mongo/rocketchat --eval \"
            rs.initiate({
              _id: 'rs0',
              members: [ { _id: 0, host: 'localhost:27017' } ]})\" &&
          s=$$? && break || s=$$?;
          echo \"Tried $$i times. Waiting 5 secs...\";
          sleep 5;
        done; (exit $$s)"
    depends_on:
      - mongo
    links:
      - mongo

networks:
  default:
    name: my-main-net
    external: true

No change between the two files. I made sure to use the same version of mongo in the “new” file as was shown in the UI information.

For reference, here’s the GitHub issue, in case any of you can get something more out of it than I did.

Thanks for all the help.

9

I think it is likely your compose file settings.

So I’d start with this compose files and the env example

I think the ROOT_URL needs to be your server name.

Note: When deploying Rocket.Chat, you must set the ROOT_URL parameter to a HTTPS address without including a port number. So instead of ROOT_URL=http://localhost:3000 , use something like https://your_hostname.com

For reference here’s one of my simple compose files (for just Rocket itself as I run a local Mongo)

version: '2'
services:
  rocketchat:
    image: rocketchat/rocket.chat:6.6.8
    container_name: rocketchat
    environment:
    - MONGO_URL=mongodb://127.0.0.1:27017/rocketchat?replicaSet=rs0
    - MONGO_OPLOG_URL=mongodb://127.0.0.1:27017/local?replicaSet=rs0
    - ROOT_URL=https://chat.mydomain.net
    - MAIL_URL=smtp://127.0.0.1:25
    - HTTP_FORWARDED_COUNT=1
    - Accounts_UseDNSDomainCheck=True
10

I’ve put the compose and env files on my system, and tried them, but I get mongo errors in the logs as it comes up, and 502 bad gateway when I try to visit the site.

Not sure why my compose works, and this doesn’t other than some incompatibility with mongo. It’s just odd.

11

I literally moved the compose.yml, the data directory, and my uploads directory over, then just did

I suspect that somehow this has something to do with it.

I think a backup/restore of the DB may have been better. But possibly something has changed that hasn’t been accounted for in your config.

I presume this was running without any issues prior to your moving it?

502 bad gateway when I try to visit the site.

Likely because you haven’t got your reverse proxy and server name etc set up correctly. They need fixing before it will work correctly.

Think we are missing something obvious here and we only know part of the story.

Can you show your logs as it starts up please? Up to the point that Rocket shows:

@rocket.chat/meteor:dsv: I20240604-22:56:48.355(2)? +-----------------------------------------------+
@rocket.chat/meteor:dsv: I20240604-22:56:48.355(2)? |                 SERVER RUNNING                |
@rocket.chat/meteor:dsv: I20240604-22:56:48.355(2)? +-----------------------------------------------+

Both Rocket and Mongo. Remove anything sensitive and use say pastebin or similar.

You could also try a clean install as a test and see what happens.

12

FWIW this thread is worth a good read.

Almost certainly this is because the Mongo DB is not running correctly and Rocket thereofre not able to communicate with it effectively.

Please check the oplogs in particular.

13

As an FYI - this was the issue with my server after the move to a new host. I had the ROOT_URL set as localhost (which it has been for years), but when I changed it to https://mychat.mydomain.com it started working as expected.

14

Yes as I did say:

Likely because you haven’t got your reverse proxy and server name etc set up correctly. They need fixing before it will work correctly.

Glad you got it sorted.