Cannot oauth login with Keycloak (17.0.0) behind Nginx (1.23.1-alpine) reverse proxy

Description

I have a small RocketChat setup in Docker Compose that I use to deploy into production environments. I have an Nginx reverse proxy in front of RocketChat and Keycloak, where Keycloak is initially set up for simple username/password authentication.

When I attempt to log in as “Alice” in Keycloak, I get error messages in Rocketchat logs with a failure to log in:

Exception while invoking method login errorClass [Error]: No matching login attempt found [145546287]
    at MethodInvocation.<anonymous> (packages/accounts-oauth/oauth_server.js:35:21)
    at packages/accounts-base/accounts_server.js:594:31
    at tryLoginMethod (packages/accounts-base/accounts_server.js:1518:14)
    at AccountsServer._runLoginHandlers (packages/accounts-base/accounts_server.js:592:22)
    at AccountsServer.Accounts._runLoginHandlers (app/lib/server/lib/loginErrorMessageOverride.js:7:35)
    at MethodInvocation.methods.login (packages/accounts-base/accounts_server.js:652:31)
    at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1885:12)
    at packages/ddp-server/livedata_server.js:1803:15
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1257:12)
    at packages/ddp-server/livedata_server.js:1801:36
    at new Promise (<anonymous>)
    at Server.applyAsync (packages/ddp-server/livedata_server.js:1800:12)
    at Server.apply (packages/ddp-server/livedata_server.js:1739:26)
    at Server.call (packages/ddp-server/livedata_server.js:1721:17)
    at Object.post (app/api/server/v1/misc.ts:572:27)
    at app/api/server/api.js:459:96
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1257:12)
    at Object._internalRouteActionHandler [as action] (app/api/server/api.js:459:39)
    at Route._callEndpoint (packages/rocketchat_restivus/lib/route.coffee:150:32)
    at packages/rocketchat_restivus/lib/route.coffee:59:33
    at packages/simple_json-routes.js:100:9
 => awaited here:
    at Promise.await (/app/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/promise_server.js:60:12)
    at Server.apply (packages/ddp-server/livedata_server.js:1752:22)
    at Server.call (packages/ddp-server/livedata_server.js:1721:17)
    at Object.post (app/api/server/v1/misc.ts:572:27)
    at app/api/server/api.js:459:96
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1257:12)
    at Object._internalRouteActionHandler [as action] (app/api/server/api.js:459:39)
    at Route._callEndpoint (packages/rocketchat_restivus/lib/route.coffee:150:32)
    at packages/rocketchat_restivus/lib/route.coffee:59:33
    at packages/simple_json-routes.js:100:9 {
  isClientSafe: true,
  error: 145546287,
  reason: 'No matching login attempt found',
  details: undefined,
  errorType: 'Meteor.Error'
}
{"level":35,"time":"2022-07-31T18:27:00.396Z","pid":1,"hostname":"4559732a6468","name":"API","method":"POST","url":"/api/v1/method.callAnon/login","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.134 Safari/537.36","length":"189","host":"david.local.denizen.net","referer":"https://david.local.denizen.net/home","remoteIP":"172.20.0.1","status":200,"responseTime":22}
[448B blob data]
    at CustomOAuth.getAccessToken (app/custom-oauth/server/custom_oauth_server.js:135:18)
    at Object.handleOauthRequest (app/custom-oauth/server/custom_oauth_server.js:193:26)
    at OAuth._requestHandlers.<computed> (packages/oauth2/oauth2_server.js:10:33)
    at middleware (packages/oauth/oauth_server.js:171:5)
    at /app/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/fiber_pool.js:43:40
 => awaited here:
    at Promise.await (/app/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/promise_server.js:60:12)
    at Server.apply (packages/ddp-server/livedata_server.js:1752:22)
    at Server.call (packages/ddp-server/livedata_server.js:1721:17)
    at Object.post (app/api/server/v1/misc.ts:572:27)
    at app/api/server/api.js:459:96
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1257:12)
    at Object._internalRouteActionHandler [as action] (app/api/server/api.js:459:39)
    at Route._callEndpoint (packages/rocketchat_restivus/lib/route.coffee:150:32)
    at packages/rocketchat_restivus/lib/route.coffee:59:33
    at packages/simple_json-routes.js:100:9

My Keycloak and Rocketchat instances are configured as I documented in my docs/ repo. But also screenshotted, and linked below.

I’m almost certain this is just a configuration problem, as I had more success when I didn’t try to add Nginx to the frontend. We need Nginx there for added security benefits. So thanks in advanced for any help!

Server Setup Information

  • Version of Rocket.Chat Server: rocketchat/rocket.chat:5.0.0-alpine
  • Operating System: Manjaro (host), Docker containers for all services.
  • Deployment Method: Docker + Compose
  • Number of Running Instances: 1
  • DB Replicaset Oplog:
  • NodeJS Version:
  • MongoDB Version: mongo:5.0.9
  • Proxy: nginx:1.23.1-alpine
  • Firewalls involved:

The whole system is set up the Docker containers (from my .env file):

IMAGE_KEYCLOAK=quay.io/keycloak/keycloak:17.0.0
IMAGE_MONGO=mongo:5.0.9
IMAGE_MONGO_EXPRESS=mongo-express:1.0.0-alpha.4
IMAGE_NGINX=nginx:1.23.1-alpine
IMAGE_PGADMIN=dpage/pgadmin4:6.12
IMAGE_POSTGRES=postgres:14.4-alpine3.16
IMAGE_ROCKETCHAT=rocketchat/rocket.chat:5.0.0-alpine

My host system has this hosts file entry:

127.0.0.1       david.local.denizen.net

This allows me to use that david.local.denizen.net host locally. I then have that host as one of Nginx’s aliases in the Docker Compose config.

Nginx is set up with the deliberist/quick-pki certs, specifically the David server cert. I then loaded the Root CA1 and the Alice certs into the browser on my host.

Screenshots

Any secrets/passwords in logs or screenshots are OK, they are used in my local testing and change generally every time I restart the system.

Logs

Keycloak logs
Rocketchat logs

Logs continued…

Nginx logs

Repos:

Apologies for spamming my own thread. I tried to pull as much info together as I can, but the site prevented me from posting more than 2 links in any post/reply. I just wanted to make sure I captured as much info as would be helpful.

Thanks!