Stuck on 3.8.7; can I export data and import to new instance running 6.x?

What version are you coming from?

3.8.7

What version are you going to?

Supported, guide said to go to 4.x, then 5.x, et cetera; going to 4.x get issues like 4.4.0 Upgrade: Error creating index: users -> { bio: 1 } · Issue #24347 · RocketChat/Rocket.Chat · GitHub, and so on.

What deployment method did you use to deploy?

docker

Did you follow a particular doc? Which one?

What issues are you running into upgrading?

Instance won’t start up.

Question is maybe it is simpler to just install the current version, and either create new accounts for everyone, or (better) import data from the current instance

Do you have any logs from rocket.chat on startup?

Which logs are most helpful?

Typically can do:

docker-compose logs rocketchat

and see what kind of logs you get.

Additionally would be useful to see if any logs in mongo

docker-compose logs mongo

Documents help, but are thin on details.
What I did (have standalone config, docker, mongo4.0 with rocketchat 3.18.7)
System is started with a single docker-compose.yml file (the original here):

version: '2'
services:
  rocketchat:
    image: registry.rocket.chat/rocketchat/rocket.chat:3.18.7
    container_name: rocketchat
    command: >
      bash -c
        "for i in `seq 1 30`; do
          node main.js &&
          s=$$? && break || s=$$?;
          echo \"Tried $$i times. Waiting 5 secs...\";
          sleep 5;
        done; (exit $$s)"
    restart: unless-stopped
    volumes:
      - /var/data/chat/uploads:/app/uploads
    environment:
      - PORT=3131
      - ROOT_URL=http://chat.adapt-ip.com
      - MONGO_URL=mongodb://mongo:27017/rocketchat
      - MONGO_OPLOG_URL=mongodb://mongo:27017/local
    depends_on:
      - mongo
    ports:
      - 3131:3131

  mongo:
    image: mongo:4.0
    restart: unless-stopped
    volumes:
     - /var/data/chat/db:/data/db
    command: mongod --smallfiles --oplogSize 1024 --replSet rs0 --storageEngine=mmapv1
    labels:
      - "traefik.enable=false"

  # this container's job is just run the command to initialize the replica set.
  # it will run the command and remove himself (it will not stay running)
  mongo-init-replica:
    image: mongo:4.0
    command: >
      bash -c
        "for i in `seq 1 30`; do
          mongo mongo/rocketchat --eval \"
            rs.initiate({
              _id: 'rs0',
              members: [ { _id: 0, host: 'localhost:27017' } ]})\" &&
          s=$$? && break || s=$$?;
          echo \"Tried $$i times. Waiting 5 secs...\";
          sleep 5;
        done; (exit $$s)"
    depends_on:
      - mongo

Step 1) Stop rocketchat so no new messages are posted

$ docker ps
CONTAINER ID   IMAGE                                                COMMAND                  CREATED          STATUS          PORTS                              NAMES
8ecbeb37c67f   registry.rocket.chat/rocketchat/rocket.chat:3.18.7   "docker-entrypoint.s…"   11 minutes ago   Up 11 minutes   3000/tcp, 0.0.0.0:3131->3131/tcp   rocketchat
f275a439580c   mongo:4.0                                            "docker-entrypoint.s…"   11 minutes ago   Up 11 minutes   27017/tcp                          chat-mongo-1
$docker stop rocketchat

Step 2) back it up

$ docker exec f275a439580c   sh -c 'mongodump --archive' > db.dump

Step 3) edit the operational docker compose to reflect wildTiger

version: '2'
services:
  rocketchat:
    image: registry.rocket.chat/rocketchat/rocket.chat:3.18.7
    container_name: rocketchat
    command: >
      bash -c
        "for i in `seq 1 30`; do
          node main.js &&
          s=$$? && break || s=$$?;
          echo \"Tried $$i times. Waiting 5 secs...\";
          sleep 5;
        done; (exit $$s)"
    restart: unless-stopped
    volumes:
      - /var/data/chat/uploads:/app/uploads
    environment:
      - PORT=3131
      - ROOT_URL=http://chat.adapt-ip.com
      - MONGO_URL=mongodb://mongo:27017/rocketchat
      - MONGO_OPLOG_URL=mongodb://mongo:27017/local
    depends_on:
      - mongo
    ports:
      - 3131:3131

  mongo:
    image: mongo:4.0
    restart: unless-stopped
    volumes:
     # - /var/data/chat/db:/data/db
     - /var/data/chat/db.wt:/data/db
    command: mongod --oplogSize 1024 --replSet rs0 --storageEngine=wiredTiger --bind_ip_all
    # command: >
    #   bash -c
    #   "mkdir -p /var/data/chat/db.wt; mongod --oplogSize 1024 --replSet rs0 --storageEngine=wiredTiger --dbpath=/var/data/chat/db.wt --bind_ip_all;"
    # command: mongod --oplogSize 1024 --storageEngine=wiredTiger --dbpath=/var/data/chat/db.wt
    labels:
      - "traefik.enable=false"

  # this container's job is just run the command to initialize the replica set.
  # it will run the command and remove himself (it will not stay running)
  mongo-init-replica:
    image: mongo:4.0
    command: >
      bash -c
        "for i in `seq 1 30`; do
          mongo mongo/rocketchat --eval \"
            rs.initiate({
              _id: 'rs0',
              members: [ { _id: 0, host: 'localhost:27017' } ]})\" &&
          s=$$? && break || s=$$?;
          echo \"Tried $$i times. Waiting 5 secs...\";
          sleep 5;
        done; (exit $$s)"
    depends_on:
      - mongo

Step 4) start just the mongo and its replicate set; asking for mongo-init-replica will start its dependencies:

$ docker compose up -d  mongo-init-replica

Step 5) restore the backup:

$ docker exec -i 2c068129eff9 /usr/bin/mongorestore --authenticationDatabase admin --archive < db.dump

Step 6) monitor logs until it appears quiet:

$docker logs chat-mongo-1
...
2023-11-11T21:05:52.815+0000 I NETWORK  [conn4] end connection 127.0.0.1:54050 (4 connections now open)
2023-11-11T21:05:52.816+0000 I NETWORK  [conn3] end connection 127.0.0.1:35770 (3 connections now open)
2023-11-11T21:05:52.817+0000 I NETWORK  [conn5] end connection 127.0.0.1:54056 (2 connections now open)
2023-11-11T21:05:52.819+0000 I NETWORK  [conn6] end connection 127.0.0.1:46510 (1 connection now open)
2023-11-11T21:05:52.827+0000 I NETWORK  [conn2] end connection 127.0.0.1:35762 (0 connections now open)

Step 7) Now start rocket chat and watch the logs

$ docker compose up rocketchat
[+] Building 0.0s (0/0)                                                                                                                                                                                            
[+] Running 2/0
 ✔ Container chat-mongo-1  Running                                                                                                                                                                            0.0s 
 ✔ Container rocketchat    Created                                                                                                                                                                            0.0s 
Attaching to rocketchat
rocketchat  | Setting default file store to GridFS
rocketchat  | LocalStore: store created at 
rocketchat  | LocalStore: store created at 
rocketchat  | LocalStore: store created at 
rocketchat  | {"line":"120","file":"migrations.js","message":"Migrations: Not migrating, already at version 232","time":{"$date":1699737099178},"level":"info"}
rocketchat  | Loaded the Apps Framework and loaded a total of 0 Apps!
rocketchat  | Updating process.env.MAIL_URL
rocketchat  | Using GridFS for custom sounds storage
rocketchat  | Using GridFS for custom emoji storage
rocketchat  | Browserslist: caniuse-lite is outdated. Please run:
rocketchat  | npx browserslist@latest --update-db
rocketchat  | 
rocketchat  | Why you should do it regularly:
rocketchat  | https://github.com/browserslist/browserslist#browsers-data-updating
rocketchat  | ➔ System ➔ startup
rocketchat  | ➔ +--------------------------------------------------+
rocketchat  | ➔ |                  SERVER RUNNING                  |
rocketchat  | ➔ +--------------------------------------------------+
rocketchat  | ➔ |                                                  |
rocketchat  | ➔ |  Rocket.Chat Version: 3.18.7                     |
rocketchat  | ➔ |       NodeJS Version: 12.22.1 - x64              |
rocketchat  | ➔ |      MongoDB Version: 4.0.18                     |
rocketchat  | ➔ |       MongoDB Engine: wiredTiger                 |
rocketchat  | ➔ |             Platform: linux                      |
rocketchat  | ➔ |         Process Port: 3131                       |
rocketchat  | ➔ |             Site URL: -------------  |
rocketchat  | ➔ |     ReplicaSet OpLog: Enabled                    |
rocketchat  | ➔ |          Commit Hash: 660c9f5e89                 |
rocketchat  | ➔ |        Commit Branch: HEAD                       |
rocketchat  | ➔ |                                                  |
rocketchat  | ➔ +--------------------------------------------------+

Step 8) Stop and restartit to make sure all is well

$ docker stop rocketchat; docker stop chat-mongo-1
$ docker compose up -d

All is good now!

So, I have the original 3.18.7 rocket chat running on a different machine in our network. “raven” versus the one hosting the webserver, called “web”.

This machine can run the more modern mongodb that needs the Intel database instructions.

I attempted to change nginx to direct traffic to this machine new machine “raven”, at 10.1.10.47 (instead of to localhost:3000)

(see the lines marked # << )

server {
listen *:443 ssl;
ssl_protocols TLSv1.2;
ssl_session_timeout 5m;
ssl_ciphers ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA;
ssl_session_cache shared:SSL:50m;
ssl_dhparam /etc/letsencrypt/certs/dhparam.pem;
ssl_prefer_server_ciphers on;
client_max_body_size 200M;

server_name chat.adapt-ip.com;
location / {
# proxy_pass [http://127.0.0.1:3131/](http://127.0.0.1:3131/); # << this worked when rocketchat & mongo are on the web machine
proxy_pass [http://10.1.4.47:3131/](http://10.1.4.47:3131/); # << my attempt to send the traffic to a different machine in my network
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $http_host;

proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forward-Proto http;
proxy_set_header X-Nginx-Proxy true;
add_header Front-End-Https on;
proxy_redirect off;
}
ssl_certificate /etc/letsencrypt/live/adapt-ip.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/adapt-ip.com/privkey.pem; # managed by Certbot
}

What am I doing wrong / what else do I need to do?

On web I can run curl and see the rocket chat page, os connectivity is fine

web % curl -v 10.1.10.47:3131
<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" type="text/css" class="__meteor-css__" href="/a3e89fa2bdd3f98d52e474085bb1d61f99c0684d.css?meteor_css_resource=true">
<meta charset="utf-8" />
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
<meta http-equiv="expires" content="-1" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
...

When I access the page http://localhost:3000 on a browser (goggle-chrome) running on “raven” I get the rocketchat session, as usual (great happiness as all of the data is there!); so mongo & rocketchat are up and running

However, when I try to access rocketchat from other machines via the mapped name: chat.adapt-ip.com (which uses to work), I get

504 Gateway Time-out

My docker file:

version: '2'
services:
rocketchat:
image: registry.rocket.chat/rocketchat/rocket.chat:3.18.7
container_name: rocketchat
command: >
bash -c
"for i in `seq 1 30`; do
node main.js &&
s=$$? && break || s=$$?;
echo \"Tried $$i times. Waiting 5 secs...\";
sleep 5;
done; (exit $$s)"
restart: unless-stopped
volumes:
- /var/data/chat/uploads:/app/uploads
environment:
- PORT=3131
- ROOT_URL=[http://chat.adapt-ip.com](http://chat.adapt-ip.com)
- MONGO_URL=mongodb://mongo:27017/rocketchat
- MONGO_OPLOG_URL=mongodb://mongo:27017/local
depends_on:
- mongo
ports:
- 3131:3131

mongo:
image: mongo:4.0
restart: unless-stopped
volumes:
- /var/data/chat/db:/data/db
command: mongod --smallfiles --oplogSize 1024 --replSet rs0 --storageEngine=mmapv1
labels:
- "traefik.enable=false"

mongo-init-replica:
image: mongo:4.0
command: >
bash -c
"for i in `seq 1 30`; do
mongo mongo/rocketchat --eval \"
rs.initiate({
_id: 'rs0',
members: [ { _id: 0, host: 'localhost:27017' } ]})\" &&
s=$$? && break || s=$$?;
echo \"Tried $$i times. Waiting 5 secs...\";
sleep 5;
done; (exit $$s)"
depends_on:
- mongo

I set up error logs for the proxy:

server {
listen *:443 ssl;
# listen [::]:443 ssl;
# ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

    ssl_protocols TLSv1.2;
    ssl_session_timeout 5m;
    ssl_ciphers ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA;
    ssl_session_cache shared:SSL:50m;
    ssl_dhparam /etc/letsencrypt/certs/dhparam.pem;
    ssl_prefer_server_ciphers on;
    client_max_body_size 200M;

    **error_log /var/log/chat/error.log;**  
    **access_log /var/log/chat/access.log combined;**

    server_name chat.adapt-ip.com;
    location / {
            # proxy_pass http://127.0.0.1:3131/;
            proxy_pass http://10.1.4.47:3131/;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header Host $http_host;

            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forward-Proto http;
            proxy_set_header X-Nginx-Proxy true;
            add_header       Front-End-Https   on;
            proxy_redirect off;
    }

# ssl_certificate     /etc/letsencrypt/live/www.adapt-ip.com/fullchain.pem; # managed by Certbot
# ssl_certificate_key /etc/letsencrypt/live/www.adapt-ip.com/privkey.pem; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/adapt-ip.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/adapt-ip.com/privkey.pem; # managed by Certbot

}

And get:

2023/11/21 22:51:55 [error] 2522954#2522954: *5 upstream timed out (110: Connection timed out) while connecting to upstream, client: 103.211.180.250, server: chat.adapt-ip.com, request: "GET / HTTP/2.0", upstream: "http://10.1.4.47:3131/", host: "chat.adapt-ip.com"
2023/11/21 22:52:55 [error] 2522954#2522954: *5 upstream timed out (110: Connection timed out) while connecting to upstream, client: 103.211.180.250, server: chat.adapt-ip.com, request: "GET /favicon.ico HTTP/2.0", upstream: "http://10.1.4.47:3131/favicon.ico", host: "chat.adapt-ip.com", referrer: "https://chat.adapt-ip.com/"

on the machine, web running:

web:~$ wget http://10.1.10.47:3000/ 
--2023-11-21 22:58:40--  http://10.1.10.47:3000/
Connecting to 10.1.10.47:3000... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘index.html.3’

index.html.3                                     [ <=>                                                                                        ] 218.03K  --.-KB/s    in 0.02s   

2023-11-21 22:58:40 (11.6 MB/s) - ‘index.html.3’ saved [223262]
web:~$

ok, yes, I had a bug; was redirecting to 10.1.4.47 in the nginx file, but the machine was at 10.1.10.47.

After fixing that I don’t get the 504 Timeout, and instead the page comes up nearly blank. It does get the favicon (the rocket chat) and then a white screen. The error logs are empty, and the access log shows:

103.211.180.250 - - [21/Nov/2023:23:30:21 +0000] "GET / HTTP/2.0" 200 80119 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"
103.211.180.250 - - [21/Nov/2023:23:30:21 +0000] "GET /a3e89fa2bdd3f98d52e474085bb1d61f99c0684d.css?meteor_css_resource=true HTTP/2.0" 200 15840 "https://chat.adapt-ip.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"
103.211.180.250 - - [21/Nov/2023:23:30:21 +0000] "GET /css-theme.css?411026f4c5e281493db620041a0076c09cb191f8 HTTP/2.0" 200 15840 "https://chat.adapt-ip.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"
103.211.180.250 - - [21/Nov/2023:23:30:21 +0000] "GET /36d53d95895b64e18e0f537a585d957d3432874d.js?meteor_js_resource=true HTTP/2.0" 200 80119 "https://chat.adapt-ip.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"
103.211.180.250 - - [21/Nov/2023:23:30:21 +0000] "GET /meteor_runtime_config.js?hash=ddb31330322bf50f2c55116f64ce244574e99f3e HTTP/2.0" 200 80119 "https://chat.adapt-ip.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"
103.211.180.250 - - [21/Nov/2023:23:30:21 +0000] "GET /scripts.js?b171b893c4d536f3d43217054fec30899bbef512 HTTP/2.0" 200 80119 "https://chat.adapt-ip.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"
103.211.180.250 - - [21/Nov/2023:23:30:21 +0000] "GET /css-theme.css?411026f4c5e281493db620041a0076c09cb191f8 HTTP/2.0" 200 80119 "https://chat.adapt-ip.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"
103.211.180.250 - - [21/Nov/2023:23:30:21 +0000] "GET /assets/favicon.svg HTTP/2.0" 200 80119 "https://chat.adapt-ip.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"
103.211.180.250 - - [21/Nov/2023:23:30:21 +0000] "GET /assets/favicon_32.png HTTP/2.0" 200 80119 "https://chat.adapt-ip.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"
103.211.180.250 - - [21/Nov/2023:23:30:21 +0000] "GET /images/manifest.json HTTP/2.0" 200 80119 "https://chat.adapt-ip.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"
103.211.180.250 - - [21/Nov/2023:23:30:21 +0000] "GET /assets/favicon_16.png HTTP/2.0" 200 80119 "https://chat.adapt-ip.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"

right away and then stops