Hm… sure enough looks like it didn’t get booted. Then we’re going to need more output for Rocket.Chat startup logs.
journalctl -u snap.rocketchat-server.rocketchat-server
Hm… sure enough looks like it didn’t get booted. Then we’re going to need more output for Rocket.Chat startup logs.
journalctl -u snap.rocketchat-server.rocketchat-server
journalctl -u snap.rocketchat-server.rocketchat-server output:
Aug 14 10:17:48 v2201 systemd[1]: Started Service for snap application rocketchat-server.rocketchat-server.
Aug 14 10:17:51 v2201 rocketchat-server.rocketchat-server[714]: Mongo is not available, can't start. Waiting 10 seconds and trying again
Aug 14 10:18:01 v2201 rocketchat-server.rocketchat-server[714]: Checking if oplog has been enabled, and enabling if not
Aug 14 10:18:02 v2201 rocketchat-server.rocketchat-server[1473]: MongoDB shell version v3.6.14
Aug 14 10:18:02 v2201 rocketchat-server.rocketchat-server[1473]: connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
Aug 14 10:18:02 v2201 rocketchat-server.rocketchat-server[1473]: Implicit session: session { "id" : UUID("26f6e030-794a-4680-8cf2-b01f3afa3821") }
Aug 14 10:18:02 v2201 rocketchat-server.rocketchat-server[1473]: MongoDB server version: 3.6.14
Aug 14 10:18:02 v2201 rocketchat-server.rocketchat-server[714]: Checking if mongo featureCompatibilityVersion is correct, changing if not
Aug 14 10:18:22 v2201 rocketchat-server.rocketchat-server[1614]: LocalStore: store created at
Aug 14 10:18:22 v2201 rocketchat-server.rocketchat-server[1614]: LocalStore: store created at
Aug 14 10:18:22 v2201 rocketchat-server.rocketchat-server[1614]: LocalStore: store created at
Aug 14 10:18:23 v2201 rocketchat-server.rocketchat-server[1614]: Setting default file store to GridFS
Aug 14 10:18:28 v2201 rocketchat-server.rocketchat-server[1614]: {"line":"120","file":"migrations.js","message":"Migrations: Not migrating, already at version 202","time":{"$date":1597393108974},"level":"info"}
Aug 14 10:18:29 v2201 rocketchat-server.rocketchat-server[1614]: ufs: temp directory created at "/tmp/ufs"
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: Loaded the Apps Framework and loaded a total of 0 Apps!
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: Using GridFS for custom sounds storage
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: Using FileSystem for custom emoji storage
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: Exception in callback of async function: Error: EROFS: read-only file system, mkdir '/snap/rocketchat-server/1440/programs/server/https:'
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: at Object.mkdirSync (fs.js:840:3)
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: at sync (/snap/rocketchat-server/1440/programs/server/npm/node_modules/mkdirp/index.js:71:13)
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: at sync (/snap/rocketchat-server/1440/programs/server/npm/node_modules/mkdirp/index.js:77:24)
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: at Function.sync (/snap/rocketchat-server/1440/programs/server/npm/node_modules/mkdirp/index.js:77:24)
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: at new RocketChatFile.FileSystem (app/file/server/file.server.js:154:10)
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: at app/emoji-custom/server/startup/emoji-custom.js:32:38
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: at Function.time (/snap/rocketchat-server/1440/programs/server/profile.js:273:30)
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: at /snap/rocketchat-server/1440/programs/server/boot.js:415:15
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: at /snap/rocketchat-server/1440/programs/server/boot.js:465:7
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: at Function.run (/snap/rocketchat-server/1440/programs/server/profile.js:280:14)
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: at /snap/rocketchat-server/1440/programs/server/boot.js:463:13 {
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: errno: -30,
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: syscall: 'mkdir',
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: code: 'EROFS',
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: path: '/snap/rocketchat-server/1440/programs/server/https:'
Aug 14 10:18:31 v2201 rocketchat-server.rocketchat-server[1614]: }
Aug 14 10:18:32 v2201 rocketchat-server.rocketchat-server[1614]: Updating process.env.MAIL_URL
__updated to insert the entire log.
It looks like something is trying to use the file system at a path that doesn’t exist and it can’t create at since read only.
Can you do: rocketchat-server.mongo
Then execute:
db.rocketchat_settings.find().forEach(function(s) {
if (!s._id.match(/^Assets/) && !s._id.match(/^Layout/) && s._id !== 'css' && s.value !== s.packageValue) {
print(s._id + ' - ' + s.value);
}
});
It will spit out any setting changed. So will need to censor out any secrets or domains. The file paths and file related settings we are most interested in.
I’ve inserted that in the rocketchat-server.mongo terminal and it returned nothing.
That piece of code is immediately followed by “rs0: PRIMARY>”
You might need to type use parties
first to select the database (this is default db name in snaps)
Should I post the custom .css output here as well? There’s a lot.
Nah that shouldn’t have any effect on boot up. Can leave that out
Accounts_TwoFactorAuthentication_Enabled - false
Accounts_TwoFactorAuthentication_By_Email_Enabled - false
Accounts_TwoFactorAuthentication_By_Email_Auto_Opt_In - false
uniqueID - zn*************
Accounts_SearchFields - username, name, bio
Accounts_ManuallyApproveNewUsers - true
Accounts_RegistrationForm_SecretURL - PPcZDimKCuAP2Jc9W
Accounts_Registration_AuthenticationServices_Enabled - false
Accounts_Default_User_Preferences_hideRoles - true
Show_Setup_Wizard - completed
Site_Url - https://sub.domain.com
Site_Name - Polises Messaging
DeepLink_Url - https://sub.domain.com
Statistics_reporting - false
Message_Attachments_GroupAttach - true
Message_Audio_bitRate - 128
Message_MaxAllowedSize - 15000
Hide_System_Messages -
Push_enable_gateway - false
Custom_Script_Logged_In -
UI_Use_Real_Name - true
Organization_Type - community
Organization_Name - name
Industry - ***
Size - 2
Country - worldwide
Website - https://domain.com
Server_Type - privateTeam
Allow_Marketing_Emails - false
Register_Server - false
Cloud_Workspace_Access_Token_Expires_At - Thu Jan 01 1970 01:00:00 GMT+0100 (CET)
EmojiUpload_Storage_Type - FileSystem
EmojiUpload_FileSystemPath - https://domain.com/emotes
Update_LatestAvailableVersion - 3.5.1
Message_Read_Receipt_Enabled - true
Message_Read_Receipt_Store_Users - true
FEDERATION_Status - Enabled
RetentionPolicy_DoNotPruneDiscussion - true
RetentionPolicy_DoNotPruneThreads - true
Alright this is what I suspected.
We’ll need to drop into a mongo shell again. Also switch to parties db just like above. Then choose which path forward
If you want custom emoji on file system you can update this value:
db.rocketchat_settings.update({_id:”EmojiUpload_FileSystemPath”},{$set:{value:”/var/snap/rocketchat-server/common/emoji”}})
Or replace with folder that is snap writable. See more details here: Deploy with Snaps - Rocket.Chat Docs
Alternatively switch it to use gridfs
db.rocketchat_settings.update({_id:”EmojiUpload_Storage_Type”},{$set:{value:”GridFS”}})
Once you do one or the other you should be able to restart Rocket.chat.
Looks like this value was changed while it was running and must have never needed to restart since set. Then the update caused it to and failed to start up because it had web address instead of file system address and it couldn’t find the folder
I get this:
db.rocketchat_settings.update({_id:" EmojiUpload_Storage_Type"},{$set:{value:"GridFS"}})
WriteResult({ "nMatched" : 0, "nUpserted" : 0, "nModified" : 0 })
After exit, I restart rocketchat and mongo, but still can’t access my rocketchat instance.
I’ve tried running the following code again:
db.rocketchat_settings.find().forEach(function(s) {
if (!s._id.match(/^Assets/) && !s._id.match(/^Layout/) && s._id !== 'css' && s.value !== s.packageValue) {
print(s._id + ' - ' + s.value);
}
});
And still get EmojiUpload_FileSystemPath - https://domain.com/emotes
I’m not an avid mongo user, I thought maybe I exited without saving but I can’t find a command that saves the changes to a database so I assume it saves the changes automatically.
What could be a problem?
Oh ouch looks like i typed on my mobile a space before the setting name. Shouldn’t be any space between the quote and setting name. Fixed previous post in case any others copy
db.rocketchat_settings.update({_id:"EmojiUpload_Storage_Type"},{$set:{value:"GridFS"}})
Great, everything works now.
I must note that was some stellar support of an open-source user with among the fastest turnarounds I’ve ever seen.
Thank you for your time and attention, Aaron.
Happened to be able to help and see it through. Definitely not always this quick
Glad you got it though!
dear aaron colud you please help me in this
Thanks to the RC Team for keeping the community updated. Much appreciated. Keep up the good work everyone!
I’ve restored a fresh copy of the VM from 2 days ago because I’ve mucked with the current one trying to get it working.
Its really helpful information
breakup shayari. Great Information.Thanks for sharing
Thank you so much, it is so helpful !