Description
I’m currently trying to setup a 2 node RC and mongodb cluster and everything seemed fine until RC decided to just not connect to the mongodb cluster.
I followed this tutorial: https://rocket.chat/docs/installation/docker-containers/high-availability-install/
Using the exact same mongodb URI directly, works without a problem.
I really have no idea, what the problem could be. I have attached the docker-compose.yml, logs from starting and testing the URI and the users present on the mongodb cluster.
Server Setup Information
version: '2'
services:
rocketchat:
image: rocketchat/rocket.chat:latest
hostname: rocket-1
command: >
bash -c
"for i in `seq 1 30`; do
node main.js &&
s=$$? && break || s=$$?;
echo \"Tried $$i times. Waiting 5 secs...\";
sleep 5;
done; (exit $$s)"
restart: unless-stopped
volumes:
- ./uploads:/app/uploads
environment:
- PORT=3000
- ROOT_URL=http://chat.mbconnectline.com
- MONGO_URL=mongodb://rocket:supersecurepassword@rocket-1:27017,rocket-2:27017/rocketchat?authSource=admin&replicaSet=rs0&readPreference=nearest&w=majority
- MONGO_OPLOG_URL=mongodb://oploguser:supersecurepassword@rocket-1:27017,rocket-2:27017/local?authSource=admin&replicaSet=rs0
- INSTANCE_IP=172.16.0.2
# - MAIL_URL=smtp://smtp.email
# - HTTP_PROXY=http://proxy.domain.com
# - HTTPS_PROXY=http://proxy.domain.com
depends_on:
- mongo
ports:
- 172.16.0.2:3000:3000
extra_hosts:
- "rocket-1:172.16.0.2"
- "rocket-2:172.16.0.3"
- "rocket-proxy:172.16.0.4"
mongo:
image: mongo:3.6.9
hostname: rocket-1
restart: unless-stopped
volumes:
- /root/mongo/arb:/data/db
- /root/mongo:/opt/keyfile
command: mongod --smallfiles --keyFile /opt/keyfile/mongodb-keyfile --replSet rs0 --enableMajorityReadConcern false --storageEngine=mmapv1
ports:
- 172.16.0.2:27017:27017
extra_hosts:
- "rocket-1:172.16.0.2"
- "rocket-2:172.16.0.3"
- "rocket-proxy:172.16.0.4"
Any additional Information
Running docker-compose up
mongo_1 | 2020-05-06T12:16:29.373+0000 I ASIO [NetworkInterfaceASIO-RS-0] Successfully connected to rocket-2:27017, took 5ms (2 connections now open to rocket-2:27017)
mongo_1 | 2020-05-06T12:16:30.203+0000 I NETWORK [listener] connection accepted from 172.16.0.3:50200 #2 (2 connections now open)
mongo_1 | 2020-05-06T12:16:30.204+0000 I NETWORK [conn2] received client metadata from 172.16.0.3:50200 conn2: { driver: { name: "NetworkInterfaceASIO-Replication", version: "3.6.9" }, os: { type: "Linux", name: "PRETTY_NAME="Debian GNU/Linux 9 (stretch)"", architecture: "x86_64", version: "Kernel 4.15.0-99-generic" } }
mongo_1 | 2020-05-06T12:16:30.227+0000 I ACCESS [conn2] Successfully authenticated as principal __system on local
rocketchat_1 | /app/bundle/programs/server/node_modules/fibers/future.js:313
rocketchat_1 | throw(ex);
rocketchat_1 | ^
rocketchat_1 |
rocketchat_1 | Error: connect ECONNREFUSED 172.18.0.3:27017
rocketchat_1 | at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1137:16) {
rocketchat_1 | name: 'MongoNetworkError',
rocketchat_1 | errorLabels: [ 'TransientTransactionError' ],
rocketchat_1 | [Symbol(mongoErrorContextSymbol)]: {}
rocketchat_1 | }
rocketchat_1 | Tried 1 times. Waiting 5 secs...
The logs on rocket-2 (currently PRIMARY)
2020-05-06T12:16:30.202742818Z 2020-05-06T12:16:30.202+0000 I ASIO [NetworkInterfaceASIO-Replication-0] Connecting to rocket-1:27017
2020-05-06T12:16:30.228444987Z 2020-05-06T12:16:30.228+0000 I ASIO [NetworkInterfaceASIO-Replication-0] Successfully connected to rocket-1:27017, took 26ms (1 connections now open to rocket-1:27017)
2020-05-06T12:16:30.229085661Z 2020-05-06T12:16:30.228+0000 I REPL [replexec-10] Member rocket-1:27017 is now in state SECONDARY
2020-05-06T12:16:30.682224798Z 2020-05-06T12:16:30.681+0000 I NETWORK [listener] connection accepted from 172.16.0.2:41124 #646 (6 connections now open)
2020-05-06T12:16:30.696569069Z 2020-05-06T12:16:30.696+0000 I NETWORK [conn646] end connection 172.16.0.2:41124 (5 connections now open)
2020-05-06T12:16:37.043417526Z 2020-05-06T12:16:37.043+0000 I NETWORK [listener] connection accepted from 172.16.0.2:41128 #647 (6 connections now open)
2020-05-06T12:16:37.058876004Z 2020-05-06T12:16:37.058+0000 I NETWORK [conn647] end connection 172.16.0.2:41128 (5 connections now open)
2020-05-06T12:16:43.389324847Z 2020-05-06T12:16:43.389+0000 I NETWORK [listener] connection accepted from 172.16.0.2:41132 #648 (6 connections now open)
2020-05-06T12:16:43.404446929Z 2020-05-06T12:16:43.404+0000 I NETWORK [conn648] end connection 172.16.0.2:41132 (5 connections now open)
2020-05-06T12:16:44.394504693Z 2020-05-06T12:16:44.394+0000 I NETWORK [conn644] end connection 172.16.0.2:54640 (4 connections now open)
2020-05-06T12:16:44.428933376Z 2020-05-06T12:16:44.428+0000 I NETWORK [conn645] end connection 172.16.0.2:54642 (3 connections now open)
2020-05-06T12:16:44.428980073Z 2020-05-06T12:16:44.428+0000 I NETWORK [conn643] end connection 172.16.0.2:54636 (2 connections now open)
Testing the mongodb URI with docker exec -it rocketchat_mongo_1 /bin/bash
root@rocket-1:/# mongo 'mongodb://rocket:supersecurepassword-1:27017,rocket-2:27017/rocketchat?authSource=admin&replicaSet=rs0&readPreference=nearest&w=majority'
MongoDB shell version v3.6.9
connecting to: mongodb://rocket-1:27017,rocket-2:27017/rocketchat?authSource=admin&replicaSet=rs0&readPreference=nearest&w=majority
2020-05-06T12:22:37.160+0000 I NETWORK [thread1] Starting new replica set monitor for rs0/rocket-1:27017,rocket-2:27017
2020-05-06T12:22:37.162+0000 I NETWORK [thread1] Successfully connected to rocket-1:27017 (1 connections now open to rocket-1:27017 with a 5 second timeout)
2020-05-06T12:22:37.163+0000 I NETWORK [ReplicaSetMonitor-TaskExecutor-0] Successfully connected to rocket-2:27017 (1 connections now open to rocket-2:27017 with a 5 second timeout)
Implicit session: session { "id" : UUID("bab275f9-7ed8-42a7-91db-eb3d1b4f3c54") }
MongoDB server version: 3.6.9
rs0:PRIMARY> db.isMaster()
{
"hosts" : [
"rocket-1:27017",
"rocket-2:27017"
],
"arbiters" : [
"rocket-proxy:27017"
],
"setName" : "rs0",
"setVersion" : 3,
"ismaster" : true,
"secondary" : false,
"primary" : "rocket-2:27017",
"me" : "rocket-2:27017",
"electionId" : ObjectId("7fffffff0000000000000002"),
"lastWrite" : {
"opTime" : {
"ts" : Timestamp(1588767779, 1),
"t" : NumberLong(2)
},
"lastWriteDate" : ISODate("2020-05-06T12:22:59Z"),
"majorityOpTime" : {
"ts" : Timestamp(1588767779, 1),
"t" : NumberLong(2)
},
"majorityWriteDate" : ISODate("2020-05-06T12:22:59Z")
},
"maxBsonObjectSize" : 16777216,
"maxMessageSizeBytes" : 48000000,
"maxWriteBatchSize" : 100000,
"localTime" : ISODate("2020-05-06T12:23:06.856Z"),
"logicalSessionTimeoutMinutes" : 30,
"minWireVersion" : 0,
"maxWireVersion" : 6,
"readOnly" : false,
"ok" : 1,
"operationTime" : Timestamp(1588767779, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1588767779, 1),
"signature" : {
"hash" : BinData(0,"iHro1JLnUlJCdOlYRRTWyouRe/4="),
"keyId" : NumberLong("6823659583924338689")
}
}
}
root@rocket-1:/# mongo 'mongodb://oploguser:supersecurepassword@rocket-1:27017,rocket-2:27017/local?authSource=admin&replicaSet=rs0'
MongoDB shell version v3.6.9
connecting to: mongodb://rocket-1:27017,rocket-2:27017/local?authSource=admin&replicaSet=rs0
2020-05-06T12:41:38.956+0000 I NETWORK [thread1] Starting new replica set monitor for rs0/rocket-1:27017,rocket-2:27017
2020-05-06T12:41:38.958+0000 I NETWORK [ReplicaSetMonitor-TaskExecutor-0] Successfully connected to rocket-1:27017 (1 connections now open to rocket-1:27017 with a 5 second timeout)
2020-05-06T12:41:38.958+0000 I NETWORK [thread1] Successfully connected to rocket-2:27017 (1 connections now open to rocket-2:27017 with a 5 second timeout)
Implicit session: session { "id" : UUID("e28d1aa3-6603-4ff1-9342-8b49de128738") }
MongoDB server version: 3.6.9
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
http://docs.mongodb.org/
Questions? Try the support group
http://groups.google.com/group/mongodb-user
rs0:PRIMARY>
Output of show users
rs0:PRIMARY> show users
{
"_id" : "admin.oploguser",
"user" : "oploguser",
"db" : "admin",
"roles" : [
{
"role" : "read",
"db" : "local"
}
]
}
{
"_id" : "admin.rocket",
"user" : "rocket",
"db" : "admin",
"roles" : [
{
"role" : "readWrite",
"db" : "rocketchat"
}
]
}
{
"_id" : "admin.siteRootAdmin",
"user" : "siteRootAdmin",
"db" : "admin",
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}
{
"_id" : "admin.siteUserAdmin",
"user" : "siteUserAdmin",
"db" : "admin",
"roles" : [
{
"role" : "userAdminAnyDatabase",
"db" : "admin"
}
]
}