Snap 2.1 Update Issues

[Sarbakan]

got the same problem, when i revert to mongo 3.4, mongo is not starting with error txn-recover: unsupported WiredTiger file version: this build only supports major/minor versions up to 1/0, and the file is version 2/0: WT_ERROR: non-specific WiredTiger error
so i can’t run the command rocketchat-server.mongo --eval "printjson(db.adminCommand ({getParameter: 1, featureCompatibilityVersion: 1}))"

[abaldac]

The mongo service status is in “failed” and if I run the command:
snap run rocketchat-server.mongo --eval “printjson(db.adminCommand ({getParameter: 1, featureCompatibilityVersion: 1}))”

the output is:
MongoDB shell version v3.4.20
connecting to: mongodb://127.0.0.1:27017
2019-11-06T17:18:21.175+0100 W NETWORK [thread1] Failed to connect to 127.0.0.1:27017, in(checking socket for error after poll), reason: Connection refused
2019-11-06T17:18:21.175+0100 E QUERY [thread1] Error: couldn’t connect to server 127.0.0.1:27017, connection attempt failed :
connect@src/mongo/shell/mongo.js:240:13
@(connect):1:6
exception: connect failed

[Sarbakan]

Our problem is fixed. the database was still on version 3.2. so we had to start it with a mongodb v3.2 and then upgrade the DB to 3.6. after that i upgrade RocketChat to the latest version, stop mongodb service and replace the /var/snap/rocketchat-server/common folder for the updated one. and the server is now starting on 2.1.1

[lucia.guevgeozian]

what snap version are you running?

[lucia.guevgeozian]

@abaldac and @avenetj can you join #ubuntu-snap channel in https://open.rocket.chat, I think it will be easier to debug there your cases, thanks

[Melvins_Gunnarson]

Good evening,

since yesterday evening we are living some troubles too with our current Rocket Chat installation.

We are exactly living this situation:

Suddenly, mongod is not starting

Description I had a working Rocket Chat yesterday. This morning I saw a “502 Bad gateway” in the web page. After trying to figure out what happened, I stopped rocketchat-caddy and rocketchat-server snap services and try to start again. Then I start first with sudo systemctl start snap.rocketchat-server.rocketchat-mongo.service but it doesn’t start… Server Setup Information Version of Rocket.Chat Server: I don’t know but I did a caddy refresh about 1 month ago Operating System: Ubuntu Deploy…

Our current setup is:

rocketchat-server 2.1.1 1414 stable rocketchat✓ -
MongoDB shell version v3.6.14

I don’t understand if MongoDB updated too fast, or if our Rocketchat is not up to date (since the current version seems to be 1416).

Our update are usually done automatically.

How can we fix this?

Thanks in advance.

[lucia.guevgeozian]

Hello, we already saw some cases that when trying to check compatibility version with this command

rocketchat-server.mongo --eval "printjson(db.adminCommand ({getParameter: 1, featureCompatibilityVersion: 1}))"

get confused by the mongo shell version, and think featureCompatibilityVersion is 3.4, but in fact is 3.2

If the output is this, take a look at the last line, which is the result of the command executed

MongoDB shell version v3.4.20           ---->  not important
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.20
{ "featureCompatibilityVersion" : "3.2", "ok" : 1 }   ------>  this is the important part!!!

[lucia.guevgeozian]

Hello 1414 is the latest, please check SNAP NEWS - Rocket.Chat 2.1

In some cases previous revision was not installed or failed to bring mongo feature compatibility version to 3.4 before mongo upgrade to 3.6, that looks like your case.

Also check this one SNAP NEWS - Rocket.Chat 2.1

Thanks

[Melvins_Gunnarson]

Hello @lucia.guevgeozian,

thanks for the follow up, currently if I’m launching this command:

rocketchat-server.mongo --eval "printjson(db.adminCommand ({getParameter: 1, featureCompatibilityVersion: 1}))"

I have this output:

        MongoDB shell version v3.6.14
        connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
        2019-11-07T09:17:25.719-0500 W NETWORK  [thread1] Failed to connect to 127.0.0.1:27017, in(checking socket for error after poll), reason: Connection refused
        2019-11-07T09:17:25.720-0500 E QUERY    [thread1] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed :
        connect@src/mongo/shell/mongo.js:263:13
        @(connect):1:6
        exception: connect failed

[lucia.guevgeozian]

You probably saw this message in the logs too? ** IMPORTANT: UPGRADE PROBLEM: The data files need to be fully upgraded to version 3.4 before attempting an upgrade to 3.6, if so, you can follow the instructions in the post made by Aaron

[Melvins_Gunnarson]

Yay, thanks @lucia.guevgeozian!!

It is now working again. Do we need to be careful about something special after this manipulation?

Thanks for your great support, long live the .

[lucia.guevgeozian]

Great, so you are back in stable right? with mongo 3.6 installed and running? if so nothing special after this should be done, the tricky part was mongo upgrade, thanks

[Melvins_Gunnarson]

@lucia.guevgeozian : yes I’m seeing that Mongodb upgraded correctly to 3.6.

after launching:

rocketchat-server.mongo --eval "printjson(db.adminCommand ({getParameter: 1, featureCompatibilityVersion: 1}))"

I’m seeing:

    MongoDB shell version v3.6.14
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("b1b09cd6-8107-48c7-a42a-8d7136722a80") }
MongoDB server version: 3.6.14
{ "featureCompatibilityVersion" : { "version" : "3.4" }, "ok" : 1 }

Which seems to be good.

Thanks.

[sietdobra]

Hi guys,
I have the sam problem after automatic update:
*** IMPORTANT: UPGRADE PROBLEM: The data files need to be fully upgraded to version 3.4 before attempting an upgrade to 3.6; see http://dochub.mongodb.org/core/3.6-upgrade-fcv for more details.*

I was trying to follow your guide, but it failed just in the beginning:
error: cannot revert "rocketchat-server": cannot find revision 1416 for snap "rocketchat-server"

The only revision I’m able to run (and rocket works) is 1254
rocketchat-server 0.63.3 1254 stable/… rocketchat✓ -

After update it get, where mongo doesn’t start at all…
rocketchat-server 2.4.11 1427 stable/… rocketchat✓ -

And when I try to enter the command
rocketchat-server.mongo --eval "printjson(db.adminCommand ({getParameter: 1, featureCompatibilityVersion: 1}))"

MongoDB shell version: 3.2.7
connecting to: test
{ "ok" : 0, "errmsg" : "no option found to get" }

I get error message, because in revision 1254 is mongo shell 3.2 where featureCompatibilityVersion wasn’t implemented. I was able to connect via mongo shell to server tried several things, but no success. Plz how to update to working environment, so I can update the featureCompatilibyVersion variable…?

We would like to use current mobile app and have updated software packages.

Thanks

[aaron.ogle]

It looks like you are on a suuuuuuuper old version. Curious how in the world did you manage to stay back on 0.63.3?

To move forward here you’re going to have to actually jump to a snap that upgraded to mongo 3.4.

I’ll take a look and see if I can remember which snap version we upgraded to mongo 3.4 in.

[sietdobra]

We didn’t do any management, maybe that’s the reason. Other guy installed that, so after update we run into problems.

Isn’t there any archive of snaps to use?

[aaron.ogle]

Ok you will need to move to mongo 3.2 and then to mongo 3.4

snap refresh rocketchat-server --channel=stable/migrate-mongo-3.2

make sure it comes up. You’ll be on Rocket.Chat 0.65.2 and mongo 3.2

Then take it to mongo 3.4

snap refresh rocketchat-server --channel=stable/migrate-mongo-3.4

Once it comes up you’ll be on Rocket.Chat 1.3.2 and mongo 3.4

Then finally you can go to 2.x or 3.x

snap refresh rocketchat-server --channel=2.x/stable

snap refresh rocketchat-server --channel=3.x/stable

[sietdobra]

Fantastic, it worked like a charm!

[GPConnect]

I was able to make rocket chat run again using “snap refresh rocketchat-server --channel=stable/migrate-mongo-3.2” then “snap refresh rocketchat-server --channel=stable/migrate-mongo-3.4” and i was effectively at version 1.3.2 !
When i try to update to branch 3, the server went down again !
tried to downgrade to branch 2 but same issue !
I can not use the first command anymore i get this error
“error: requested a non-existing branch on 2.x/stable for snap “rocketchat-server”:
migrate-mongo-3.2”
(Get caddy down in the status and rocket chat can’t connect to mongodb again)
Any help on this ? thanks

[faded]

I am somehow stuck in the boat after ramping up a suspended VM, that I need to get it back into working order. The snap tried to update, and it failed to start. I had to revert to “rocketchat-server 0.74.3” snap to get it running again, and according to the tools above, it is running mongodb 3.2. When I try to use any of the “–channel=stable/migrate-mongo-3.2” or “–channel=stable/migrate-mongo-3.4” it fails to start because it immediately attempts to load in db version v3.6.14. Is there a specific way I can move from 3.2 to 3.4 to 3.6? The above commands in this thread do not seem to work.