Rocket.Chat unexpectedly crashed out of blue and throws this kind of errors: 2020/08/13 08:56:07 [error] 337681#337681: *99747 connect() failed (111: Connection refused) while connecting to upstream, client: 9x.8x.2xx.2xx, server name, request: “GET /favicon HTTP/1.1”, upstream: “127.0.0.1:3000/favicon.ico”, host: “sub.domain.com”, referrer: “domain name”
Service status showed this: at /snap/rocketchat-server/1439/programs/server/boot.js:415:15 Aug 13 10:54:46 v2201 rocketchat-server.rocketchat-server[2067]: at /snap/rocketchat-server/1439/programs/server/boot.js:465:7 Aug 13 10:54:46 v2201 rocketchat-server.rocketchat-server[2067]: at Function.run (/snap/rocketchat-server/1439/programs/server/profile.js:280:14) Aug 13 10:54:46 v2201 rocketchat-server.rocketchat-server[2067]: at /snap/rocketchat-server/1439/programs/server/boot.js:463:13 { Aug 13 10:54:46 v2201 rocketchat-server.rocketchat-server[2067]: errno: -30, Aug 13 10:54:46 v2201 rocketchat-server.rocketchat-server[2067]: syscall: 'mkdir', Aug 13 10:54:46 v2201 rocketchat-server.rocketchat-server[2067]: code: 'EROFS', Aug 13 10:54:46 v2201 rocketchat-server.rocketchat-server[2067]: path: '/snap/rocketchat-server/1439/programs/server/https:' Aug 13 10:54:46 v2201 rocketchat-server.rocketchat-server[2067]: }
I’m not sure if the “everything should be working fine as of right now” applies to my situation, but I’ve tried restarting rocketchat-server and I’m still getting ‘502 Bad Gateway’.
Same here. I’ve restored a fresh copy of the VM from 2 days ago because I’ve mucked with the current one trying to get it working. Still the same ol “502 Bad Gateway” for me. I’ve noticed that if I try to start the snap.rocketchat-server service, it appears it starts OK but for some reason stops immediately - shows it’s Inactive instead of Active and running. Port 3000 also never opens up.
3.5.2 should fully solve this. Basically two parts of same issue.
HTTP calls weren’t deferred causing them to block until they got a response.
On our end we had an incident that caused a service that is never down to stop responding and our load balancer kept the connection open too long instead of more aggressively timing out.
3.5.2 protects rocket.chat from that now regardless. The service is fixed and we are making the changes to the loadbalancer
Unfortunately, it did not. I’ve tried ‘snap refresh rocketchat-server’ but it says there are no updates available, so I guess it’s already updated to 3.5.2.
Restarting rc, caddy, mongo, and ubuntu server didn’t help.
-- Logs begin at Sun 2020-08-02 04:52:27 CEST. --
Aug 14 08:51:45 v220190810296596149 rocketchat-server.rocketchat-server[1654]: at /snap/rocketchat-server/1440/programs/server/boot.js:415:15
Aug 14 08:51:45 v220190810296596149 rocketchat-server.rocketchat-server[1654]: at /snap/rocketchat-server/1440/programs/server/boot.js:465:7
Aug 14 08:51:45 v220190810296596149 rocketchat-server.rocketchat-server[1654]: at Function.run (/snap/rocketchat-server/1440/programs/server/profile.js:280:14)
Aug 14 08:51:45 v220190810296596149 rocketchat-server.rocketchat-server[1654]: at /snap/rocketchat-server/1440/programs/server/boot.js:463:13 {
Aug 14 08:51:45 v220190810296596149 rocketchat-server.rocketchat-server[1654]: errno: -30,
Aug 14 08:51:45 v220190810296596149 rocketchat-server.rocketchat-server[1654]: syscall: 'mkdir',
Aug 14 08:51:45 v220190810296596149 rocketchat-server.rocketchat-server[1654]: code: 'EROFS',
Aug 14 08:51:45 v220190810296596149 rocketchat-server.rocketchat-server[1654]: path: '/snap/rocketchat-server/1440/programs/server/https:'
Aug 14 08:51:45 v220190810296596149 rocketchat-server.rocketchat-server[1654]: }
Aug 14 08:51:45 v220190810296596149 rocketchat-server.rocketchat-server[1654]: Updating process.env.MAIL_URL
-- Logs begin at Sun 2020-08-02 04:52:27 CEST. --
Aug 14 08:51:07 v220190810296596149 systemd[1]: Stopped Service for snap application rocketchat-server.rocketchat-caddy.
Aug 14 08:51:07 v220190810296596149 systemd[1]: Started Service for snap application rocketchat-server.rocketchat-caddy.
Aug 14 08:51:07 v220190810296596149 rocketchat-server.rocketchat-caddy[1220]: 2020/08/14 08:51:07 loading Caddyfile via flag: open /var/snap/rocketchat-server/1440/Caddyfile: no such file or directory
Aug 14 08:51:07 v220190810296596149 systemd[1]: snap.rocketchat-server.rocketchat-caddy.service: Main process exited, code=exited, status=1/FAILURE
Aug 14 08:51:07 v220190810296596149 systemd[1]: snap.rocketchat-server.rocketchat-caddy.service: Failed with result 'exit-code'.
Aug 14 08:51:07 v220190810296596149 systemd[1]: snap.rocketchat-server.rocketchat-caddy.service: Scheduled restart job, restart counter is at 5.
Aug 14 08:51:07 v220190810296596149 systemd[1]: Stopped Service for snap application rocketchat-server.rocketchat-caddy.
Aug 14 08:51:07 v220190810296596149 systemd[1]: snap.rocketchat-server.rocketchat-caddy.service: Start request repeated too quickly.
Aug 14 08:51:07 v220190810296596149 systemd[1]: snap.rocketchat-server.rocketchat-caddy.service: Failed with result 'exit-code'.
Aug 14 08:51:07 v220190810296596149 systemd[1]: Failed to start Service for snap application rocketchat-server.rocketchat-caddy.
How to fix that?
I’m in /var/snap/rocketchat-server/1440/ right now and it is empty. But I haven’t touched that folder before.
Had you used caddy before for your https and reverse proxy? If so you’ll need to initialize it. The FAQ has instructions for that. If not that error is fine. Just fails to start and sits there unused. For rocket.chat operation caddy isn’t needed. It’s just there available in case you want it.
Seems like you should be able to access on port 3000 right now.
I haven’t. I’ve installed rocket chat with snap and haven’t touched anything that wasn’t in the instructions. I’m using nginx to be able to access my rocket chat instance via an actual domain name url and that’s it.
On the actual host it’s self can you curl http://localhost:3000/api/info and get a result? If so sounds like firewall. The logs indicate it’s probably up and running