Rocket.Chat won't start after upgrade to 3.5.0 via docker - probably apn related?

Description:

After updating to 3.5.0 - Rocket.Chat will not start anymore. There is only one real exception in the logs:

Push ➔ debug Configure {
  sendTimeout: 60000,
  apn: {
    passphrase: '',
    key: 'WDBDR35MT8',
    cert: '-----BEGIN PRIVATE KEY-----\n' +
     xxx
      '-----END PRIVATE KEY-----'
  },
  gcm: undefined,
  production: true,
  gateways: undefined,
  uniqueId: 'xxx',
  getAuthorization: [Function: getAuthorization]
}
Push ➔ debug APN configured
Exception in callback of async function: Error: ENOENT: no such file or directory, open 'WDBDR35MT8'
    at Object.openSync (fs.js:457:3)
    at Object.readFileSync (fs.js:359:35)
    at resolveCredential (/app/bundle/programs/server/npm/node_modules/apn/lib/credentials/resolve.js:16:15)
    at loadCredentials (/app/bundle/programs/server/npm/node_modules/apn/lib/credentials/certificate/load.js:15:15)
    at loadAndValidate (/app/bundle/programs/server/npm/node_modules/apn/lib/credentials/certificate/prepare.js:11:20)
    at config (/app/bundle/programs/server/npm/node_modules/apn/lib/config.js:49:22)
    at new Client (/app/bundle/programs/server/npm/node_modules/apn/lib/client.js:11:19)
    at new Provider (/app/bundle/programs/server/npm/node_modules/apn/lib/provider.js:12:19)
    at initAPN (app/push/server/apn.js:114:18)
    at PushClass.configure (app/push/server/push.js:49:4)
    at configurePush (server/lib/pushConfig.js:109:8)
    at Function.time (/app/bundle/programs/server/profile.js:273:30)
    at /app/bundle/programs/server/boot.js:415:15
    at /app/bundle/programs/server/boot.js:465:7
    at Function.run (/app/bundle/programs/server/profile.js:280:14)
    at /app/bundle/programs/server/boot.js:463:13 {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: 'WDBDR35MT8'
}

Steps to reproduce:

I don’t know how to reproduce. Happened after the update to 3.5.0.
Reverting to 3.4.3 didn’t solve anything.

Expected behavior:

Rocket.Chat will start up.

Actual behavior:

Rocket.Chat does not start - the log does not provide any clues, except the one from above. The System ➔ startup part from the log is completely missing.

Server Setup Information:

  • Version of Rocket.Chat Server: 3.5.0
  • Operating System: ubuntu
  • Deployment Method: docker
  • Number of Running Instances: 1

Additional context

Maybe related to any push notification settings? Is there a way to disable this via ENV?

Relevant logs:

The complete logs are:

LocalStore: store created at
LocalStore: store created at
LocalStore: store created at
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Facebook
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Facebook_id
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Facebook_secret
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Facebook_callback_url
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Google
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Google_id
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Google_secret
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Google_callback_url
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Github
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Github_id
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Github_secret
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Github_callback_url
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Linkedin
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Linkedin_id
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Linkedin_secret
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Linkedin_callback_url
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Meteor
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Meteor_id
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Meteor_secret
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Meteor_callback_url
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Twitter
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Twitter_id
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Twitter_secret
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Twitter_callback_url
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Proxy_host
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Proxy_services
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Tokenpass
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Tokenpass_id
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Tokenpass_secret
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Tokenpass_callback_url
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Dolphin_URL
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Dolphin
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Dolphin_id
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Dolphin_secret
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Dolphin_login_style
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Dolphin_button_label_text
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Dolphin_button_label_color
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Dolphin_button_color
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Drupal
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Drupal_id
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Drupal_secret
CustomOAuth ➔ debug Init CustomOAuth tokenpass {
  serverURL: '',
  identityPath: '/oauth/user',
  authorizePath: '/oauth/authorize',
  tokenPath: '/oauth/access-token',
  scope: 'user,tca,private-balances',
  tokenSentVia: 'payload',
  usernameField: 'username',
  mergeUsers: true,
  addAutopublishFields: {
    forLoggedInUser: [ 'services.tokenpass' ],
    forOtherUsers: [ 'services.tokenpass.name' ]
  },
  accessTokenParam: 'access_token'
}
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Drupal_callback_url
rocketchat:lib ➔ oauth_updated Accounts_OAuth_GitHub_Enterprise
rocketchat:lib ➔ oauth_updated Accounts_OAuth_GitHub_Enterprise_callback_url
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Gitlab
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Gitlab_id
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Gitlab_secret
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Gitlab_merge_users
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Gitlab_callback_url
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Nextcloud
Setting default file store to GridFS
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Nextcloud_URL
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Nextcloud_id
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Nextcloud_secret
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Nextcloud_callback_url
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Nextcloud_button_color
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Wordpress
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Wordpress_id
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Wordpress_secret
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Wordpress_server_type
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Wordpress_identity_path
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Wordpress_identity_token_sent_via
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Wordpress_token_path
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Wordpress_authorize_path
CustomOAuth ➔ debug Init CustomOAuth dolphin {
  serverURL: '',
  authorizePath: '/m/oauth2/auth/',
  tokenPath: '/m/oauth2/token/',
  identityPath: '/m/oauth2/api/me/',
  scope: 'basic',
  addAutopublishFields: {
    forLoggedInUser: [ 'services.dolphin' ],
    forOtherUsers: [ 'services.dolphin.name' ]
  },
  accessTokenParam: 'access_token'
}
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Wordpress_scope
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Wordpress_callback_url
rocketchat:lib ➔ oauth_updated Accounts_OAuth_GitHub_Enterprise_id
rocketchat:lib ➔ oauth_updated Accounts_OAuth_GitHub_Enterprise_secret
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Gitlab_identity_path
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Apple
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Nextcloud_button_label_text
CustomOAuth ➔ debug Init CustomOAuth drupal {
  serverURL: '',
  identityPath: '/oauth2/UserInfo',
  authorizePath: '/oauth2/authorize',
  tokenPath: '/oauth2/token',
  scope: 'openid email profile offline_access',
  tokenSentVia: 'payload',
  usernameField: 'preferred_username',
  mergeUsers: true,
  addAutopublishFields: {
    forLoggedInUser: [ 'services.drupal' ],
    forOtherUsers: [ 'services.drupal.name' ]
  },
  accessTokenParam: 'access_token'
}
rocketchat:lib ➔ oauth_updated Accounts_OAuth_Nextcloud_button_label_color
Federation ➔ Setup.info Federation is disabled
CustomOAuth ➔ debug Init CustomOAuth github_enterprise {
  serverURL: '',
  identityPath: '/api/v3/user',
  authorizePath: '/login/oauth/authorize',
  tokenPath: '/login/oauth/access_token',
  addAutopublishFields: {
    forLoggedInUser: [ 'services.github-enterprise' ],
    forOtherUsers: [ 'services.github-enterprise.username' ]
  }
}
CustomOAuth ➔ debug Init CustomOAuth gitlab {
  serverURL: 'https://gitlab.com',
  identityPath: '/api/v4/user',
  scope: 'read_user',
  mergeUsers: false,
  addAutopublishFields: {
    forLoggedInUser: [ 'services.gitlab' ],
    forOtherUsers: [ 'services.gitlab.username' ]
  },
  accessTokenParam: 'access_token'
}
CAS ➔ info Disabling CAS login service
CustomOAuth ➔ debug Init CustomOAuth nextcloud {
  serverURL: '',
  tokenPath: '/index.php/apps/oauth2/api/v1/token',
  tokenSentVia: 'header',
  authorizePath: '/index.php/apps/oauth2/authorize',
  identityPath: '/ocs/v2.php/cloud/user?format=json',
  scope: 'openid',
  addAutopublishFields: {
    forLoggedInUser: [ 'services.nextcloud' ],
    forOtherUsers: [ 'services.nextcloud.name' ]
  }
}
SlackBridge ➔ Rocket.debug constructor
SlackBridge ➔ Class.debug Setting: SlackBridge_APIToken
SlackBridge ➔ Class.debug Setting: SlackBridge_AliasFormat
SlackBridge ➔ Class.debug Setting: SlackBridge_ExcludeBotnames
SlackBridge ➔ Class.debug Setting: SlackBridge_Reactions_Enabled true
SlackBridge ➔ Class.debug Setting: SlackBridge_Enabled false
CustomOAuth ➔ debug Init CustomOAuth wordpress {
  serverURL: '',
  identityPath: '/oauth/me',
  addAutopublishFields: {
    forLoggedInUser: [ 'services.wordpress' ],
    forOtherUsers: [ 'services.wordpress.user_login' ]
  },
  accessTokenParam: 'access_token'
}
Search Logger ➔ info create search provider defaultProvider
Search Logger ➔ info create search provider chatpalProvider
Chatpal Logger ➔ debug Using https://api.chatpal.io/v1/ as chatpal base url
{"line":"120","file":"migrations.js","message":"Migrations: Not migrating, already at version 197","time":{"$date":1596445508005},"level":"info"}
ufs: temp directory created at "/tmp/ufs"
SAUMonitor ➔ info [aggregate] - Start Cron.
SAUMonitor ➔ debug [start] - InstanceId: xxx
Loaded the Apps Framework and loaded a total of 0 Apps!
Using GridFS for custom sounds storage
Using GridFS for custom emoji storage
VersionCheck ➔ info Checking for version updates
Browserslist: caniuse-lite is outdated. Please run next command `npm update`
rocketchat:theme ➔ stop_rendering 138
Updating process.env.MAIL_URL
CROWD ➔ info Disabling CROWD Background Sync
LDAPSync ➔ info Disabling LDAP Background Sync
Meteor ➔ method createDirectMessage -> userId: rocket.cat, arguments: ["xxx"]
Search Logger ➔ debug Load data for all providers
Push ➔ debug Configure {
  sendTimeout: 60000,
  apn: {
    passphrase: '',
    key: 'WDBDR35MT8',
    cert: '-----BEGIN PRIVATE KEY-----\n' +
     xxx
      '-----END PRIVATE KEY-----'
  },
  gcm: undefined,
  production: true,
  gateways: undefined,
  uniqueId: 'xxx',
  getAuthorization: [Function: getAuthorization]
}
Push ➔ debug APN configured
Exception in callback of async function: Error: ENOENT: no such file or directory, open 'WDBDR35MT8'
    at Object.openSync (fs.js:457:3)
    at Object.readFileSync (fs.js:359:35)
    at resolveCredential (/app/bundle/programs/server/npm/node_modules/apn/lib/credentials/resolve.js:16:15)
    at loadCredentials (/app/bundle/programs/server/npm/node_modules/apn/lib/credentials/certificate/load.js:15:15)
    at loadAndValidate (/app/bundle/programs/server/npm/node_modules/apn/lib/credentials/certificate/prepare.js:11:20)
    at config (/app/bundle/programs/server/npm/node_modules/apn/lib/config.js:49:22)
    at new Client (/app/bundle/programs/server/npm/node_modules/apn/lib/client.js:11:19)
    at new Provider (/app/bundle/programs/server/npm/node_modules/apn/lib/provider.js:12:19)
    at initAPN (app/push/server/apn.js:114:18)
    at PushClass.configure (app/push/server/push.js:49:4)
    at configurePush (server/lib/pushConfig.js:109:8)
    at Function.time (/app/bundle/programs/server/profile.js:273:30)
    at /app/bundle/programs/server/boot.js:415:15
    at /app/bundle/programs/server/boot.js:465:7
    at Function.run (/app/bundle/programs/server/profile.js:280:14)
    at /app/bundle/programs/server/boot.js:463:13 {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: 'WDBDR35MT8'
}
Blockstack ➔ debug Blockstack not enabled {
  enable: false,
  loginStyle: 'redirect',
  generateUsername: false,
  manifestURI: 'https://rocket.xxx.de/_blockstack/manifest',
  redirectURI: 'https://rocket.xxx.de/_blockstack/validate',
  authDescription: 'Rocket.Chat login',
  buttonLabelText: 'Blockstack',
  buttonColor: '#271132',
  buttonLabelColor: '#ffffff'
}
Search Logger ➔ debug Start provider 'defaultProvider'

Maybe anyone has any idea? Thanks a lot in advance!

Cheers

1 Like

The environment variable OVERWRITE_SETTING_Push_enable=False helped me start up the server again.

Then redoing all the register workspace steps helped restart the server without the env.

I believe 3.5.1 also solved this for anyone else that comes across this

1 Like