I managed to fixed this by upgrading version by version: 3.2, 3.4, 3.6, and then finally 4.0. If anyone has a better way then please post it. Here are my steps when upgrading using docker-compose.yml:
# starting from 3.2, down the containers and update your mongo version in docker-compose.yml
docker-compose -f /path/to/rocketchat/docker-compose.yml down
sed -i "s/mongo:3.2/mongo:3.4/g" /path/to/rocketchat/docker-compose.yml
# starting up the updated docker-compose.yml will pull new mongo image
docker-compose -f /path/to/rocketchat/docker-compose.yml up -d mongo
# this command will update the db to use the new version features
docker exec -it rocketchat_mongo_1 bash -c 'mongo --eval "db.adminCommand( { setFeatureCompatibilityVersion: "3.4" } )"'
# should see an output like this
{ "ok" : 1 }
# initialize rocketchat server to load new db features and update your data files
# not sure if this step necessary but I'm not taking any risks
docker-compose -f /path/to/rocketchat/docker-compose.yml up -d
# go back to step 1, change values, rinse and repeat!