Multiple instances on one Ubuntu server

Hi,

I would like to setup multiple instances of RocketChat on one single physical server.

I am not talking about “Running Multiple Instances Per Host To Improve Performance” as described at https://rocket.chat/docs/installation/manual-installation/multiple-instances-to-improve-performance/

I need multiple instances that are running independently of each other. So for each instance there exists its own database, domain etc.

E.g.
chat.astronomy.example
chat.biology.example

A user that is registered at chat.astronomy.example is not registered at chat.biology.example

I know how to setup reverse proxies and everything that is needed for Apache.

What I would like to know is the best way of installing multiple instances of RocketChat on Ubuntu?What configuration files do I have to edit?
Should I setup multiple Docker instances instead?

Thanks!

3 Likes

@enigma969 Have you got any solution to this? I am also trying to do the same.

Hi!!

You can easily achieve this using docker in conjunction with traefik.

here goes an untested bunch of docker-compose.yml to do just that.

first, considering you have docker and docker-compose installed, create a traefik public network:

docker network create --attachable traefik-public

now, you will create a docker-compose.yml, inside a folder called, for example, traefik:

version: '3.4'

networks:
  default:
    external: false
  traefik-public:
    external: true

volumes:
  traefik_letsencrypt:

services:
  traefik:
    image: "traefik:v2.5"
    command:
      #- "--log.level=DEBUG"
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
      - "--providers.docker.network=traefik-public"
      - "--entrypoints.websecure.address=:443"
      - "--entrypoints.web.http.redirections.entryPoint.to=websecure"
      - "--entrypoints.web.http.redirections.entryPoint.scheme=https"
      - "--certificatesresolvers.le.acme.httpchallenge=true"
      - "--certificatesresolvers.le.acme.httpchallenge.entrypoint=web"
      - "--certificatesresolvers.le.acme.email=your@email.com"
      - "--certificatesresolvers.le.acme.storage=/letsencrypt/acme.json"
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - traefik_letsencrypt:/letsencrypt
    labels:
      - traefik.enable=true
      - traefik.docker.lbswarm=true
      - traefik.http.routers.traefik.rule=Host(`traefik.awesome-university.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
      - traefik.http.routers.traefik.middlewares=auth
      - traefik.http.routers.traefik.service=api@internal
      - traefik.http.services.traefik.loadbalancer.server.port=8080
      # Note: all dollar signs in the hash need to be doubled for escaping.
      # To create user:password pair, it's possible to use this command:
      # echo $(htpasswd -nb user password) | sed -e s/\\$/\\$\\$/g
      # reference: https://doc.traefik.io/traefik/v2.0/middlewares/basicauth/
      - traefik.http.middlewares.auth.basicauth.users=admin:$$apr1$$ZElyugQj$$lugDGWSna3jdhPjj3zvkV/
      - "traefik.http.routers.traefik.entrypoints=websecure"
      - "traefik.http.routers.traefik.tls.certresolver=le"      
    networks:
      - traefik-public

now go inside your traefik folder, and run

docker-compose up -d

this will bring up traefik. Now, whenever you want to create instances of new rocket.chat, you create a new folder, let’s say biology

mkdir biology
cd biology

and create a new docker-compose.yml file with the following content:

You will need to change some stuff. Mainly labels (that will ultimately define domain, admin password, root_url)

version: '3.4'

networks:
  default:
    external: false
  traefik-public:
    external: true

volumes:
  rocketchat_uploads:
  rocketchat_db:
  rocketchat_dump:

services:

  rocketchat:
    image: rocketchat/rocket.chat:3.8.1
    networks:
      - traefik-public
      - default
    command: >
      bash -c
        "for i in `seq 1 30`; do
          node main.js &&
          s=$$? && break || s=$$?;
          echo \"Tried $$i times. Waiting 5 secs...\";
          sleep 5;
        done; (exit $$s)"
    restart: unless-stopped
    volumes:
      - rocketchat_uploads:/app/uploads
    environment:
      - PORT=3000
      - ROOT_URL=https://chat.biology.awesome-university.com
      - MONGO_URL=mongodb://mongo:27017/rocketchat
      - MONGO_OPLOG_URL=mongodb://mongo:27017/local
      - MAIL_URL=smtp://smtp.email
      - OVERWRITE_SETTING_Show_Setup_Wizard=completed
      - ADMIN_USERNAME=adminrc
      - ADMIN_PASS=some_awesome_initial_password
      - TZ=America/Brasilia
    depends_on:
      - mongo
      - mongo-init-replica
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.biology-rocketchat.rule=Host(`chat.biology.awesome-university.com`)"
      - "traefik.http.services.biology-rocketchat.loadbalancer.server.port=3000"    
      - "traefik.http.routers.biology-rocketchat.entrypoints=websecure"
      - "traefik.http.routers.biology-rocketchat.tls.certresolver=le"
    networks:
      - traefik-public
      - default

  mongo:
    image: mongo:4.0
    restart: unless-stopped
    volumes:
     - ./data/rocketchat_db:/data/db
     - ./data/rocketchat_dump/dump:/dump
    command: mongod --smallfiles --oplogSize 128 --replSet rs0 --storageEngine=mmapv1
    labels:
      - "traefik.enable=false"

  # this container's job is just run the command to initialize the replica set.
  # it will run the command and remove himself (it will not stay running)
  mongo-init-replica:
    image: mongo:4.0
    command: >
      bash -c
        "for i in `seq 1 30`; do
          mongo mongo/rocketchat --eval \"
            rs.initiate({
              _id: 'rs0',
              members: [ { _id: 0, host: 'localhost:27017' } ]})\" &&
          s=$$? && break || s=$$?;
          echo \"Tried $$i times. Waiting 5 secs...\";
          sleep 5;
        done; (exit $$s)"
    depends_on:
      - mongo

  

What happens here is that traefik, which will act as a reverse proxy, listens to docker socket, and will configure the reverse proxy according to the labels you specify for each rocketchat service.

each folder/instance you create (biology, math, etc) will generate a folder_service_1 container, like biology_rocketchat_1, biology_mongo_1, etc.

now you should point all the domains you specified at each rocketchat service label to the public IP your server responds at (port 80 and 443), and everything should work fine, with SSL included.

Considering the number of instances you plan on deploying, you probably want to use docker swarm, or even K8s for that. But the idea is the same…

Let me know if this works for you.

Remember, this is just a guide on how to achieve this. I have created a cookiecutter (kind of a script) to easy the installation of great open source softwares and I have used this technique. Check it out:

1 Like

Thanks for the inspiration, this worked, but I added some more tweaks:

.env

#CHANGEME:
CUSTOMER=demo
RELEASE=6.3.3
MONGODB_VERSION=6.0

# Static
MONGODB_HOSTNAME=$CUSTOMER-mongodb
ROCKET_DOMAIN=$CUSTOMER.domain.com
CONTAINER_OWNER=rocketchat

docker-compose.yaml

networks:
  default:
    external: false
  traefik-public:
    external: true

volumes:
  rocketchat_uploads:
  rocketchat_db:
  rocketchat_dump:

services:

  rocketchat:
    image: rocketchat/rocket.chat:${RELEASE}
    command: >
      bash -c
        "for i in `seq 1 30`; do
          node main.js &&
          s=$$? && break || s=$$?;
          echo \"Tried $$i times. Waiting 5 secs...\";
          sleep 5;
        done; (exit $$s)"
    restart: unless-stopped
    volumes:
      - rocketchat_uploads:/app/uploads
    environment:
      - PORT=3000
      - ROOT_URL=https://${ROCKET_DOMAIN}
      - MONGO_URL=mongodb://${MONGODB_HOSTNAME}:27017/rocketchat
      - MONGO_OPLOG_URL=mongodb://${MONGODB_HOSTNAME}:27017/local
      - MAIL_URL=smtp://smtp.domain.com
      - TZ=Europe/Stockholm
    depends_on:
      - mongodb
      - mongo-init-replica
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.${CUSTOMER}-rocketchat.rule=Host(`${ROCKET_DOMAIN}`)"
      - "traefik.http.services.${CUSTOMER}-rocketchat.loadbalancer.server.port=3000"
      - "traefik.http.routers.${CUSTOMER}-rocketchat.entrypoints=websecure"
      - "traefik.http.routers.${CUSTOMER}-rocketchat.tls.certresolver=leresolver"
    networks:
      - traefik-public
      - default

  mongodb:
    image: mongo:${MONGODB_VERSION}
    container_name: ${MONGODB_HOSTNAME}
    restart: unless-stopped
    volumes:
     - ./data/rocketchat_db:/data/db
     - ./data/rocketchat_dump/dump:/dump
    command: mongod --oplogSize 128 --replSet rs0
    environment:
      - TZ=Europe/Stockholm
    labels:
      - "traefik.enable=false"

# this container's job is just run the command to initialize the replica set.
  # it will run the command and remove himself (it will not stay running)
  mongo-init-replica:
    image: mongo:${MONGODB_VERSION}
    command: >
      bash -c
        "for i in `seq 1 30`; do
          mongosh ${MONGODB_HOSTNAME}:27017/rocketchat --eval \"
            rs.initiate({
              _id: 'rs0',
              members: [ { _id: 0, host: 'mongodb:27017' } ]})\" &&
          s=$$? && break || s=$$?;
          echo \"Tried $$i times. Waiting 5 secs...\";
          sleep 5;
        done; (exit $$s)"
    depends_on:
      - mongodb
    environment:
      - TZ=Europe/Stockholm

Traefik

-rw-rw-r-- 1 rocketchat rocketchat 1156 sep  5 19:01 docker-compose.yaml
drwxrwxr-x 2 rocketchat rocketchat 4096 sep  5 15:50 letsencrypt/
-rw-rw-r-- 1 rocketchat rocketchat  817 aug 31 18:10 traefik-config.yaml

*traefik-config.yaml

entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https

  websecure:
    address: ":443"
    http:
      tls:
        certResolver: leresolver
#    http2:
#      maxConcurrentStreams: 42
    http3:
      advertisedPort: 443

api:
  dashboard: true
  insecure: false

certificatesResolvers:
  leresolver:
    acme:
      email: "email@email.com"
      storage: /letsencrypt/acme.json
      caserver: "https://acme-v02.api.letsencrypt.org/directory"
      keyType: 'EC256'
      httpChallenge:
        entryPoint: "web"

providers:
  file:
    directory: "/config"
    watch: true
  docker:
    network: "traefik-public"
    exposedByDefault: false

log:
  level: DEBUG

global:
  checkNewVersion: true
  sendAnonymousUsage: false

docker-compose

version: '3.7'

services:
  traefik:
    image: docker.io/traefik:3.0
    container_name: traefik
    restart: unless-stopped
    environment:
      - TZ=Europe/Stockholm
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "./traefik-config.yaml:/etc/traefik/traefik.yaml:ro"
      - "./letsencrypt:/letsencrypt"
    networks:
      - traefik-public
    # Dashboard
#    labels:
#      - 'traefik.enable=true'
#      - 'traefik.http.routers.traefik.rule=Host(`traefik.domain.com`)'
#      - 'traefik.http.routers.traefik.entrypoints=websecure'
#      - 'traefik.http.routers.traefik.service=api@internal'
#      - 'traefik.http.routers.traefik.tls=true'
#      - 'traefik.http.routers.traefik.tls.certresolver=leresolver'
#      - "traefik.http.services.traefik.loadbalancer.server.port=8080"
#      - "traefik.docker.lbswarm=true"

    ports:
      - 80:80
      - 443:443
    logging:
      driver: "json-file"
      options:
        max-size: "1m"

networks:
  default:
    external: false
  traefik-public:
    name: traefik-public
    driver: bridge

With the above config, you don’t need to add the network config first, it’s done automatically.

Actually, there is no need to use the Docker.
I’m using the systemd service file below.

[Unit]
Description=Rocket.Chat Server('%I')
After=network.target remote-fs.target nss-lookup.target mongodb.service nginx.service

[Service]
ExecStart=/your-install-path/npm/bin/node /your-install-path/main.js
PrivateTmp=true
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
User=nginx
Group=nginx
Environment=NODE_ENV=production UNIX_SOCKET_PATH=/run/rocketchat/%i.sock ROOT_URL=https://%i.your.domain/ MONGO_URL=mongodb://localhost:27017/%i?replicaSet=miyoshi-mongo-rs MAIL_URL=smtp://your-smtp-server MONGO_OPLOG_URL=mongodb://your-mongodb:27017/local?replicaSet=your-replica-set
KillSignal=9

[Install]
WantedBy=multi-user.target

This service makes a UNIX domain socket usable by the reverse proxy.
run systemctl enable rocketchat@your-new-instance-name to make your new instance.