Locked out of my server

Community Support
1

Description

I just set up a fresh install of RocketChat and after enabling a few options and getting my email to work I logged out to test an invite link. Now my admin account is locked out and I am getting the following error: Login has been temporarily blocked For IP
I am looking for a way to get around this and regain access to my system. I have searched the forum for similar posts, but they all are “I fixed it” and no-one is able to provide HOW they fixed it. I really do not want to rebuild this server if I can help it.
I do have 2FA enabled by default, and set to block failed logins, both by IP and username/password after 3 failed logins for 5 min.
I waited 10 min, but still get the same error, I even changed my IP address and get the same issue!
I found this link, but it did not help: Locked out by a two factor authentification loop
Any help would be greatly appreciated.

Server Setup Information

  • Version of Rocket.Chat Server: 3.9.0
  • Operating System: Ubuntu 18.04
  • Deployment Method: tar (Rocket.Chat in Ubuntu - Rocket.Chat Docs)
  • Number of Running Instances: 1
  • DB Replicaset Oplog: Not sure, default based on the install linked above
  • NodeJS Version: 6.14.6
  • MongoDB Version: 4.0.25
  • Proxy: Nginx via Synology NAS
  • Firewalls involved: Synology NAS and Fortigate 60W

Any additional Information

Here is the exert of my syslog when I attempt to log in.

Jul  1 20:18:04 chat rocketchat[1128]: Exception while invoking method login Error: Login has been temporarily blocked For IP [error-login-blocked-for-ip]
Jul  1 20:18:04 chat rocketchat[1128]:     at app/authentication/server/startup/index.js:307:9
Jul  1 20:18:04 chat rocketchat[1128]:     at packages/callback-hook/hook.js:131:22
Jul  1 20:18:04 chat rocketchat[1128]:     at packages/accounts-base/accounts_server.js:191:15
Jul  1 20:18:04 chat rocketchat[1128]:     at Hook.each (packages/callback-hook/hook.js:109:15)
Jul  1 20:18:04 chat rocketchat[1128]:     at AccountsServer._validateLogin (packages/accounts-base/accounts_server.js:188:29)
Jul  1 20:18:04 chat rocketchat[1128]:     at AccountsServer._attemptLogin (packages/accounts-base/accounts_server.js:377:10)
Jul  1 20:18:04 chat rocketchat[1128]:     at MethodInvocation.methods.login (packages/accounts-base/accounts_server.js:559:23)
Jul  1 20:18:04 chat rocketchat[1128]:     at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1771:12)
Jul  1 20:18:04 chat rocketchat[1128]:     at packages/ddp-server/livedata_server.js:1689:15
Jul  1 20:18:04 chat rocketchat[1128]:     at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1234:12)
Jul  1 20:18:04 chat rocketchat[1128]:     at packages/ddp-server/livedata_server.js:1687:36
Jul  1 20:18:04 chat rocketchat[1128]:     at new Promise (<anonymous>)
Jul  1 20:18:04 chat rocketchat[1128]:     at Server.applyAsync (packages/ddp-server/livedata_server.js:1686:12)
Jul  1 20:18:04 chat rocketchat[1128]:     at Server.apply (packages/ddp-server/livedata_server.js:1625:26)
Jul  1 20:18:04 chat rocketchat[1128]:     at Server.call (packages/ddp-server/livedata_server.js:1607:17)
Jul  1 20:18:04 chat rocketchat[1128]:     at Object.post (app/api/server/v1/misc.js:263:26)
Jul  1 20:18:04 chat rocketchat[1128]:     at app/api/server/api.js:394:82
Jul  1 20:18:04 chat rocketchat[1128]:     at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1234:12)
Jul  1 20:18:04 chat rocketchat[1128]:     at Object._internalRouteActionHandler [as action] (app/api/server/api.js:394:39)
Jul  1 20:18:04 chat rocketchat[1128]:     at Route.share.Route.Route._callEndpoint (packages/nimble_restivus/lib/route.coffee:150:32)
Jul  1 20:18:04 chat rocketchat[1128]:     at packages/nimble_restivus/lib/route.coffee:59:33
Jul  1 20:18:04 chat rocketchat[1128]:     at packages/simple_json-routes.js:98:9
Jul  1 20:18:04 chat rocketchat[1128]:  => awaited here:
Jul  1 20:18:04 chat rocketchat[1128]:     at Promise.await (/opt/Rocket.Chat/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/promise_server.js:60:12)
Jul  1 20:18:04 chat rocketchat[1128]:     at Server.apply (packages/ddp-server/livedata_server.js:1638:22)
Jul  1 20:18:04 chat rocketchat[1128]:     at Server.call (packages/ddp-server/livedata_server.js:1607:17)
Jul  1 20:18:04 chat rocketchat[1128]:     at Object.post (app/api/server/v1/misc.js:263:26)
Jul  1 20:18:04 chat rocketchat[1128]:     at app/api/server/api.js:394:82
Jul  1 20:18:04 chat rocketchat[1128]:     at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1234:12)
Jul  1 20:18:04 chat rocketchat[1128]:     at Object._internalRouteActionHandler [as action] (app/api/server/api.js:394:39)
Jul  1 20:18:04 chat rocketchat[1128]:     at Route.share.Route.Route._callEndpoint (packages/nimble_restivus/lib/route.coffee:150:32)
Jul  1 20:18:04 chat rocketchat[1128]:     at packages/nimble_restivus/lib/route.coffee:59:33
Jul  1 20:18:04 chat rocketchat[1128]:     at packages/simple_json-routes.js:98:9
2

Hi,

Your first real issue is your version is old and unsupported. Rocket versions are only supported for a couple of months at best.

Nonetheless, the link you provided does show you the mongo commands to disable 2FA and regain control of your server.

You should be able to run them, get control back, and then upgrade quickly! Your version has a serious security flaw that is not fixed. You need to be on at least 3.15.x or 3.16.x

3

John,
Thanks for the reply, but if you read my post you would know I attempted the process to disable 2FA, but still am getting the your IP is temporarily blocked every time I log in.
How can I disable this block from mongo, or even enable a new admin account?

4

First, make sure you have definitely disabled the 2FA - you need to view the settings and make sure they are off, and possibly restart the server.

Next you need to disable
Accounts - Block failed login attempts by Username
Accounts - Block failed login attempts by IP

We will try and find those settings but you wil have to bear with us.

5

I wanted to provide an update to anyone else that might run into this same issue.
I managed to regain access to my server by sheer luck!
Once I regained access, I promptly removed the “Failed Login” action for both Username AND IP. I have since only left my Block by IP enabled and whitelisted by public IP and my internal IP address(s).
@john.crisp I would still love to see how to remove this from mongo and/or create a new user/admin form mongo to resolve this issue should it happen again.

6

Well done (no need to @ anyone)!!

We’re investigating and will post next week.