Image uploads are getting scrambled

Description

I’ve been running a Rocket Chat Snap server since June. When I stood it up I connected file uploads to an S3 lookalike over at Linode (their object storage). This has functioned flawlessly until last Friday when file uploads started getting scrambled. I really don’t have any better word for it. It’s almost like the uploads are being communicated to the object storage wrong from Rocket Chat and I can’t figure out what is causing it.

I have tried searching around but can’t find anyone else who has experienced this behavior.

Here is an example of one of these images: https://chat.sonofdavid.org/file-upload/P3vRusPpGmPDbbiv4/2020-05-24%2018.29.28.jpg

Thats 1/5th of my daughter eating birthday cake and then a lot of scrambled data. If I upload direct to Object storage it looks fine. If I switch to GridFS it also works fine. But something about how Rocket Chat is communicating with the object storage is just not working.

Hoping someone else bumped into this before. I really like the cost savings of using Linode’s storage but obviously it is an issue if all the images come out looking like the matrix.

Server Setup Information

  • Version of Rocket.Chat Server: 2.5.0
  • Operating System: Ubuntu 20.04 LTS
  • Deployment Method: Snap
  • Number of Running Instances: 1
  • DB Replicaset Oplog:
  • NodeJS Version: Latest in Snap
  • MongoDB Version: Latest in Snap
  • Proxy: Caddy
  • Firewalls involved:

Any additional Information