RocketChat 7 slow API responses with default docker deployment

Description

Default docker deployment with rocker.server v7 (7.2.2 and 7.3.1 tested) is super slow on each API request = approx 7-12 seconds to response.
Checked same configuration on v6 and it does work properly without any issue.

Server Setup Information

  • Version of Rocket.Chat Server: 7.2.2
  • Operating System: Docker 27.01 / Ubuntu 22.04 / Swarm
  • Deployment Method: docker
  • Number of Running Instances: 1
  • DB Replicaset Oplog: 1
  • NodeJS Version: 20.18.3 - x64
  • MongoDB Version: 6.0.13
  • Proxy: traefik
  • Firewalls involved:

Any additional Information

x-default-opts:
  &default-opts
  logging:
    options:
      max-size: "10m"

services:
  mongodb:
    <<: *default-opts
    image: bitnami/mongodb:6.0
    volumes:
     - mongodb:/bitnami/mongodb
    environment:
      MONGODB_REPLICA_SET_MODE: primary
      MONGODB_REPLICA_SET_NAME: rs0
      MONGODB_PORT_NUMBER: 27017
      MONGODB_INITIAL_PRIMARY_HOST: mongodb
      MONGODB_INITIAL_PRIMARY_PORT_NUMBER: 27017
      MONGODB_ADVERTISED_HOSTNAME: mongodb
      MONGODB_ENABLE_JOURNAL: 'true'
      ALLOW_EMPTY_PASSWORD: 'yes'
    networks:
      - traefik_proxy

  rocketchat:
    <<: *default-opts
    image: rocket.chat:7.2.2
    volumes:
      - uploads:/app/uploads
    environment:
      PORT: 3000
      ROOT_URL: https://chat.my.domain
      MONGO_URL: mongodb://mongodb:27017/rocketchat?replicaSet=rs0
      MONGO_OPLOG_URL: mongodb://mongodb:27017/local?replicaSet=rs0
      DEPLOY_METHOD: docker
      Accounts_UseDNSDomainCheck: 'false'
    networks:
      - chat_internal
      - traefik_proxy
    deploy:
      mode: replicated
      replicas: 1
      labels:
        - traefik.enable=true
        - traefik.docker.network=traefik_proxy
        - traefik.http.routers.rocketchat.rule=Host(`chat.my.domain`)
        - traefik.http.routers.rocketchat.entrypoints=websecure
        - traefik.http.routers.rocketchat.tls.certresolver=resolver
        - traefik.http.services.rocketchat.loadbalancer.server.port=3000  
    depends_on:
      - mongodb
 
networks:
  traefik_proxy:
    external: true
  chat_internal:
    external: true

volumes:
  uploads:
  mongodb:

You’ll have to tell us what you are actually trying to do.

What API calls?

I am sorry. Literally every single API call is that slow. Checking via Chrome debug tools.

For example endpoint /api/v1/settings.public took 5.76s in v7.3.1 but in v6.13 it took 10ms

What do your server logs tell you is going on?

What file storage are you using?

Nothing wrong in logs seems to me and storage is now local SSD.

LocalStore: store created at  
LocalStore: store created at  
LocalStore: store created at  
{"level":40,"time":"2025-02-15T07:38:16.227Z","pid":1,"hostname":"5bf00cf20b15","name":"VoIPAsteriskService","msg":"Voip is not enabled. Cant start the service"} 
{"level":51,"time":"2025-02-15T07:38:16.252Z","pid":1,"hostname":"5bf00cf20b15","name":"Migrations","msg":"Not migrating, already at version 318"} 
[2025-02-15T07:38:16.261Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/BROKER: Moleculer v0.14.35 is starting... 
[2025-02-15T07:38:16.262Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/BROKER: Namespace: <not defined> 
[2025-02-15T07:38:16.262Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/BROKER: Node ID: 0f673441-ee1f-4d80-b55a-d94dcc4235eb 
[2025-02-15T07:38:16.263Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/REGISTRY: Strategy: RoundRobinStrategy 
[2025-02-15T07:38:16.263Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/REGISTRY: Discoverer: LocalDiscoverer 
[2025-02-15T07:38:16.265Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/BROKER: Serializer: EJSONSerializer 
[2025-02-15T07:38:16.271Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/BROKER: Validator: FastestValidator 
[2025-02-15T07:38:16.273Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/BROKER: Registered 13 middleware(s). 
[2025-02-15T07:38:16.274Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/BROKER: Transporter: TcpTransporter 
{"level":51,"time":"2025-02-15T07:38:16.293Z","pid":1,"hostname":"5bf00cf20b15","name":"DatabaseWatcher","msg":"Using change streams"} 
[2025-02-15T07:38:16.299Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/TRANSIT: Connecting to the transporter... 
[2025-02-15T07:38:16.319Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/TRANSPORTER: TCP server is listening on port 38679 
[2025-02-15T07:38:16.319Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/TRANSPORTER: UDP Discovery is disabled. 
[2025-02-15T07:38:16.319Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/TRANSPORTER: TCP Transporter started. 
{"level":51,"time":"2025-02-15T07:38:16.671Z","pid":1,"hostname":"5bf00cf20b15","name":"License","msg":"License installed","version":"3.0","hash":"O3XdD1N0"} 
ufs: temp directory created at "/tmp/ufs" 
[2025-02-15T07:38:16.828Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/REGISTRY: '$node' service is registered. 
[2025-02-15T07:38:16.829Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/REGISTRY: 'matrix' service is registered. 
[2025-02-15T07:38:16.830Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/$NODE: Service '$node' started. 
[2025-02-15T07:38:16.830Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/MATRIX: Service 'matrix' started. 
[2025-02-15T07:38:16.830Z] INFO  0f673441-ee1f-4d80-b55a-d94dcc4235eb/BROKER: âś” ServiceBroker with 2 service(s) started successfully in 541ms. 
+--------------------------------------------+ 
|               SERVER RUNNING               | 
+--------------------------------------------+ 
|                                            | 
|  Rocket.Chat Version: 7.3.1                | 
|       NodeJS Version: 22.14.0 - x64        | 
|      MongoDB Version: 6.0.13               | 
|       MongoDB Engine: wiredTiger           | 
|             Platform: linux                | 
|         Process Port: 3000                 | 
|             Site URL: https://chat.my.dom  | 
|     ReplicaSet OpLog: Enabled              | 
|          Commit Hash: 6a2724740a           | 
|        Commit Branch: HEAD                 | 
|                                            | 
+--------------------------------------------+ 
{"level":51,"time":"2025-02-15T07:38:28.412Z","pid":1,"hostname":"5bf00cf20b15","name":"License","msg":"License installed","version":"3.0","hash":"x7MO8UVQ"} 

Are you using GridFS or “local storage”?

Check your collection sizes too.

It seems i figured it out. I am using just local storage. Later i’ll switch into MinIO. I realized mongodb was in traefik_proxy network instead of chat_internal. As rocketchat service is in both but chat_internal has as the main. Probably in v7 there is different dns resolve or something like that and it timeouts resolving mongo in different network.

However in v6 it worked well. And what is strange too is when i have set networks correctly and switched to mongo 7 it started to be slow again ;o)

Anyway now i am running mongo v6 and rocketchat v7 and it works well!!

This I guess?

Glad you got it sorted.

Actually this was ok but mongodb was in traefik_proxy instead of chat_internal. That was the problem. Why it has problem with mongo v7 i still don’t know. But it works with v6 so i’ll leave it ;o)

Kk.

I’d start figuring the upgrade. You’ll have to eventually!!

And finally i’ve figured it out with mongo version. All the problems were connected to the os version. Initially i’ve tested it on Ubuntu 22.04 LTS and it seems after upgrading to 24.04 LTS it starts to work correctly. Probably it was connected to mongo and it’s dependencies (even 22.04 is offically supported for version 7).

That’s a ballache but excellent feedback thanks.

Typical with this fangled rapid release, beta at best, who cares if we didn’t test throroughly nonsense…

Not good enough at Enterprise level IMHO.

It will get mentioned :wink:

1 Like