# Docker deployment not starting after upgrade from 4.2.0 to 4.2.1/4.2.2/4.3.0

**URL:** <https://forums.rocket.chat/t/docker-deployment-not-starting-after-upgrade-from-4-2-0-to-4-2-1-4-2-2-4-3-0/12960>\
**Category:** Community Support\
**Tags:** docker\
**Created:** [January 2, 2022, 5:28pm UTC](https://forums.rocket.chat/t/docker-deployment-not-starting-after-upgrade-from-4-2-0-to-4-2-1-4-2-2-4-3-0/12960 "2022-01-02T17:28:17Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![bozzib](https://avatars.discourse-cdn.com/v4/letter/b/65b543/32.png) [@bozzib](https://forums.rocket.chat/u/bozzib)\
**Post date:** [January 2, 2022, 5:28pm UTC](https://forums.rocket.chat/t/docker-deployment-not-starting-after-upgrade-from-4-2-0-to-4-2-1-4-2-2-4-3-0/12960/1 "2022-01-02T17:28:17Z")

</div>

### Description

I’m using the official docker hub images for mongodb and rocket.chat. I’m not using docker-compose (I should, I know, but it’s not related to the issue), but the setup is similar, only done with basic docker commands . It’s working on 4.2.0 (and many others earlier; the setup is old), but updating the rocket.chat image (tried 4.2.1, 4.2.2 and 4.2.3) breaks it.

When running the service, it says nothing into “docker logs” and after a moment of silence, just shuts down. In MongoDB logs, I can see that it makes the normal amount of connections and authenticates into the DB successfully, but does nothing after that. Reverting back to 4.2.0 image fixes the issue.

Any tips on where to look for the cause?

### Server Setup Information

- Version of Rocket.Chat Server: 4.2.0 (works) 4.2.1+ (fails)
- Operating System: CentOS 7
- Deployment Method: Docker
- Number of Running Instances: 1
- DB Replicaset Oplog:
- NodeJS Version: 12.22.1 - x64
- MongoDB Version: 4.2.17 (wiredTiger)
- Proxy: nginx
- Firewalls involved: no

### Any additional Information

Updated from mmap to wiredTiger a while ago, but the setup has been working since then, through a couple of rocket.chat versions.

Containers are in a docker network, communicating via docker hostnames. Mongo has auth enabled on a separate user (rocket).

Mongo launched with args: --storageEngine wiredTiger --auth --replSet rs0

Rocket env-variables set via docker:  
-e “ROOT\_URL=https://-domain-here-”   
-e “MONGO\_URL=mongodb://rocket:-password-here-@mongo-db:27017/meteor”   
-e “MONGO\_OPLOG\_URL=mongodb://rocket:-password-here-@mongo-db:27017/local?authSource=meteor&replSet=rs0”   
-e “USE\_NATIVE\_OPLOG=true”

---

<div class="post-metadata">

**Author:** ![dudanogueira](https://sea1.discourse-cdn.com/flex021/user_avatar/forums.rocket.chat/dudanogueira/32/2529_2.png) [@dudanogueira](https://forums.rocket.chat/u/dudanogueira)\
**Post date:** [January 3, 2022, 1:19pm UTC](https://forums.rocket.chat/t/docker-deployment-not-starting-after-upgrade-from-4-2-0-to-4-2-1-4-2-2-4-3-0/12960/2 "2022-01-03T13:19:09Z")

</div>

Hi! Welcome to our forums!

Have you tried with latest 4.3.0? (Title says one thing, Server Setup info don’t)

It’s really strange to not spill any logs. Even if this was an auth problem.

I would suggest moving to a docker-compose, just to make sure, and creating an staging environment to test with latest.

---

<div class="post-metadata">

**Author:** ![bozzib](https://avatars.discourse-cdn.com/v4/letter/b/65b543/32.png) [@bozzib](https://forums.rocket.chat/u/bozzib)\
**Post date:** [January 3, 2022, 9:36pm UTC](https://forums.rocket.chat/t/docker-deployment-not-starting-after-upgrade-from-4-2-0-to-4-2-1-4-2-2-4-3-0/12960/3 "2022-01-03T21:36:53Z")

</div>

Thanks for the reply. I did try 4.3.0 as well and the result was the same.

I guess, I’ll move to compose soon anyhow, but I don’t see it affecting this. It would be more maintainable at any rate. Maybe I should try to tinker with it a bit, launching the app manually inside the container too see if it would be more verbose.

---

<div class="post-metadata">

**Author:** ![bozzib](https://avatars.discourse-cdn.com/v4/letter/b/65b543/32.png) [@bozzib](https://forums.rocket.chat/u/bozzib)\
**Post date:** [January 3, 2022, 10:51pm UTC](https://forums.rocket.chat/t/docker-deployment-not-starting-after-upgrade-from-4-2-0-to-4-2-1-4-2-2-4-3-0/12960/4 "2022-01-03T22:51:57Z")

</div>

Tried running the container with bash as entrypoint (now testing on 4.3.0).  
The environment variables within look fine to me:

```auto
I have no name!@rocket_test:/app/bundle$ printenv
HOSTNAME=rocket_test
PWD=/app/bundle
PORT=3000
NODE_ENV=production
Accounts_AvatarStorePath=/app/uploads
DEPLOY_METHOD=docker-official
RC_VERSION=4.3.0
HOME=/tmp
TERM=xterm
SHLVL=1
ROOT_URL=https://my.domain.here
MONGO_OPLOG_URL=mongodb://rocket:password_censored@mongo-db:27017/local?authSource=meteor&replSet=rs0
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
MONGO_URL=mongodb://rocket:password_censored@mongo-db:27017/meteor
NODE_VERSION=12.22.8
USE_NATIVE_OPLOG=true
_=/usr/bin/printenv

```

Running the app manually also dies after a silent while. There is an error, but not too verbose:

```auto
I have no name!@rocket_test:/app/bundle$ node main.js
Illegal instruction
I have no name!@rocket_test:/app/bundle$

```

Edit:  
I tried a few different tweaks into the config emulating the official compose-file from [Docker.Official.Image/docker-compose.yml at master · RocketChat/Docker.Official.Image · GitHub](https://github.com/RocketChat/Docker.Official.Image/blob/master/docker-compose.yml) . There were some differences, but changing them didn’t change the result. It just dies with “Illegal Instruction”. Is there a way to get more logging?

---

<div class="post-metadata">

**Author:** ![dudanogueira](https://sea1.discourse-cdn.com/flex021/user_avatar/forums.rocket.chat/dudanogueira/32/2529_2.png) [@dudanogueira](https://forums.rocket.chat/u/dudanogueira)\
**Post date:** [January 4, 2022, 11:56am UTC](https://forums.rocket.chat/t/docker-deployment-not-starting-after-upgrade-from-4-2-0-to-4-2-1-4-2-2-4-3-0/12960/5 "2022-01-04T11:56:44Z")

</div>

There were some segmentation fault issues with centOS recently.

Not sure if it’s related. The no log situation is already strange, so worth looking.  
The docker should isolate this kind of stuff.

For reference:

> <https://github.com/RocketChat/Rocket.Chat/pull/23796>
>
> \<!-- Your Pull Request name should start with one of the following tags
> \[NEW\]… For new features
> \[IMPROVE\] For an improvement (performance or little improvements) in existing features
> \[FIX\] For bug fixes that affect the end-user
> \[BREAK\] For pull requests including breaking changes
> Chore: For small tasks
> Doc: For documentation
> \--\>
> 
> \<!-- Checklist!!! If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code. 
> - I have read the Contributing Guide - https://github.com/RocketChat/Rocket.Chat/blob/develop/.github/CONTRIBUTING.md#contributing-to-rocketchat doc
> - I have signed the CLA - https://cla-assistant.io/RocketChat/Rocket.Chat
> - Lint and unit tests pass locally with my changes
> - I have added tests that prove my fix is effective or that my feature works (if applicable)
> - I have added necessary documentation (if applicable)
> - Any dependent changes have been merged and published in downstream modules
> \--\>
> 
> \## Proposed changes (including videos or screenshots)
> 
> \<!--
> Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request.
> If it fixes a bug or resolves a feature request, be sure to link to that issue below.
> This description will appear in the release notes if we accept the contribution.
> \--\>
> Upgrades \`sharp\` to avoid a segmentation fault on CentOS 7 during startup related to \`sharp.node\` being loaded via \`process.dlopen()\`.
> 
> Suggested as a fix for versions \`4.0.x\` and \`4.1.x\`.
> 
> 
> \## Issue(s)
> 
> Closes #23346
> Closes #23427
> 
> 
> \## Steps to test or reproduce
> 
> 1. Build Rocket.Chat in a temporary directory:
> \`\`\`sh
> meteor build --server-only --directory /tmp/meteor-build
> \`\`\`
> 2. Setup a VirtualBox VM with CentOS 7 (minimal), SSH access (this requires a \*Bridged Adapter\* on networking), and \`rsync\`
> 3. Connect into VM via SSH (assuming user \`user\` and host IP \`192.168.0.123\`)
> 4. Follow this section: https://docs.rocket.chat/quick-start/installing-and-updating/manual-installation/centos#install-necessary-dependency-packages
> 5. Follow only the steps related to MongoDB in this section: https://docs.rocket.chat/quick-start/installing-and-updating/manual-installation/centos#configure-the-rocket.chat-service
> 6. Create \`/opt/Rocket.Chat\` directory:
> \`\`\`sh
> sudo mkdir /opt/Rocket.Chat
> sudo chown tasso:tasso /opt/Rocket.Chat
> sudo chmod a+rwX /opt/Rocket.Chat
> \`\`\`
> 7. Logout from VM and copy the bundled files to it:
> \`\`\`sh
> rsync -avP --del /tmp/meteor-build/bundle/ user@192.168.0.123:/opt/Rocket.Chat/
> \`\`\`
> 8. Connect back into VM via SSH
> 9. Install dependencies and run Rocket.Chat:
> \`\`\`sh
> cd /opt/Rocket.Chat/programs/server
> npm install
> 
> ROOT\_URL=http://192.168.0.123:3000/ \\
> MONGO\_URL=mongodb://localhost:27017/rocketchat?replicaSet=rs01 \\
> MONGO\_OPLOG\_URL=mongodb://localhost:27017/local?replicaSet=rs01 \\
> PORT=3000 \\
> node /opt/Rocket.Chat/main.js
> \`\`\`
> 
> \## Further comments
> 
> \> Found it. The segfault occurs when \`delete\` is called here https://github.com/libvips/libvips/blob/master/cplusplus/VImage.cpp#L113-L119 on a \`std::list\`, which immediately made me suspect https://gcc.gnu.org/onlinedocs/libstdc++/manual/using\_dual\_abi.html
> \> 
> \> The prebuilt sharp binary is built with C++11 "old" ABI but it looks like the libvips binaries for musl-based Linux were built with C++11 "new" ABI. This hasn't been a problem with the "oldmalloc" allocator, but it is incorrect behaviour between sharp and libvips, and musl's new "mallocng" allocator has exposed this.
> \> 
> \> I'm going to switch all prebuilt Linux binaries to use "new" ABI for the next release, so the forthcoming libvips v8.10.6 and sharp v0.28.0 binaries will be aligned for both glibc and musl Linux.
> \> 
> \> https://github.com/lovell/sharp-libvips/commit/07de78c4ef31ffb02dac769b3ab7f8becc2a1118
> 
> \_Originally posted by @lovell in https://github.com/lovell/sharp/issues/2570#issuecomment-792659786\_

---

<div class="post-metadata">

**Author:** ![bozzib](https://avatars.discourse-cdn.com/v4/letter/b/65b543/32.png) [@bozzib](https://forums.rocket.chat/u/bozzib)\
**Post date:** [January 4, 2022, 6:01pm UTC](https://forums.rocket.chat/t/docker-deployment-not-starting-after-upgrade-from-4-2-0-to-4-2-1-4-2-2-4-3-0/12960/6 "2022-01-04T18:01:46Z")

</div>

I think you’re on to something. The fix is merged on 4.2.1 - exactly where my troubles started and the CentOS 7 matches as well. The fix really just updates the sharp library (and anything behind it of course). A bit of googling indicates that sharp has given Illegal Instruction errors in the past.

Digging through their github, I found this recent issue: [Getting "Illegal instruction" error when running sharp.js in Docker / Node container. · Issue #3030 · lovell/sharp · GitHub](https://github.com/lovell/sharp/issues/3030)

My server CPU is almost exactly the same as the one there (AMD Phenom II X6 1100T). The new sharp lib just doesn’t support it on Linux x64 apparently. Performance-wise it’s still more than enough, but I guess 10 years is considered a long time these days. Well, it’s been a loyal workhorse for a good while…

Edit: in case you get others wondering about similar issues with a CPU in the 10+ ageclass, you can refer to that, as well as this: [version 0.28.0 and later is not compatible with Intel Wolfdale E5xxx CPU · Issue #2723 · lovell/sharp · GitHub](https://github.com/lovell/sharp/issues/2723) . There are workarounds there as well, but I don’t know how much support you want to maintain for old hardware.

---

<div class="post-metadata">

**Author:** ![bozzib](https://avatars.discourse-cdn.com/v4/letter/b/65b543/32.png) [@bozzib](https://forums.rocket.chat/u/bozzib)\
**Post date:** [January 16, 2022, 12:58am UTC](https://forums.rocket.chat/t/docker-deployment-not-starting-after-upgrade-from-4-2-0-to-4-2-1-4-2-2-4-3-0/12960/7 "2022-01-16T00:58:02Z")

</div>

As a follow-up; I got some new hardware and indeed, the exact same setup now works (with the same db, etc.). So this was indeed a case of CPU incompatibility.

---

<div class="post-metadata">

**Author:** ![wdrvx](https://avatars.discourse-cdn.com/v4/letter/w/f1d935/32.png) [@wdrvx](https://forums.rocket.chat/u/wdrvx)\
**Post date:** [July 2, 2022, 11:48am UTC](https://forums.rocket.chat/t/docker-deployment-not-starting-after-upgrade-from-4-2-0-to-4-2-1-4-2-2-4-3-0/12960/8 "2022-07-02T11:48:59Z")

</div>

I’ve just decided to finally upgrade my rchat deployment to the latest version after few months have passed since the last attempt. I did not have time to dig deep into the issue so I’ve decided to stick with 4.2.0 for the time being. Now that a warning popped up about a recent CVE I thought it’s high time I updated. Firing up a new container with 4.2.1/4.2.2/4.3.0 was unsuccessful, similar to OP, I was getting no error messages while starting the container, bash’ing into it revealed ‘illegal instruction (core dumped)’

My CPU is Phenom II X4 965 which does not support AVX, my question is: do I really have to upgrade to a newer CPU to continue using rocket.chat?

This is harsh… ☹

---

<div class="post-metadata">

**Author:** ![bozzib](https://avatars.discourse-cdn.com/v4/letter/b/65b543/32.png) [@bozzib](https://forums.rocket.chat/u/bozzib)\
**Post date:** [July 4, 2022, 9:24am UTC](https://forums.rocket.chat/t/docker-deployment-not-starting-after-upgrade-from-4-2-0-to-4-2-1-4-2-2-4-3-0/12960/9 "2022-07-04T09:24:44Z")

</div>

As far as I can tell, there doesn’t seem to be much else we can do as users. The linux x64 binaries for sharp won’t work with CPUs as old as the phenom II. It doesn’t seem like they’re interested in providing that support (it was intentionally removed), so there’s not a lot of options. I guess you could try manually replacing the sharp lib for an older one to buy more time, but it’s likely quite a hassle and not very maintainable either.

---

<div class="post-metadata">

**Author:** ![dudanogueira](https://sea1.discourse-cdn.com/flex021/user_avatar/forums.rocket.chat/dudanogueira/32/2529_2.png) [@dudanogueira](https://forums.rocket.chat/u/dudanogueira)\
**Post date:** [July 4, 2022, 11:36am UTC](https://forums.rocket.chat/t/docker-deployment-not-starting-after-upgrade-from-4-2-0-to-4-2-1-4-2-2-4-3-0/12960/10 "2022-07-04T11:36:59Z")

</div>

Unfortunately that’s something beyond our control.

Also consider that we’ll have more situations like this. For instance, Mongo 5.X, like sharp, also seems to have this constraint:

> **[MongoDB 5.0 CPU Intel G4650 compatibility](https://www.mongodb.com/community/forums/t/mongodb-5-0-cpu-intel-g4650-compatibility/116610)**
>
> According to the Production notes, mongod and the legacy mongo shell require Sandy Bridge or later microarchitectures. I tried to execute mongo and mongod with an Intel G4560. This processor belongs to the Kaby Lake Intel CPU generation, that is...