Noob question - Begin customizations but using existing Monbo db

fathom · May 29, 2021, 11:18am

Description

HI. We’ve been using the standard community version for a few years (and have a healthy Mongo db full of posts). I would like to setup a DEV environment using the latest Git repo source code but populate that DEV env with a copy of my production db. Basicly, how do I build the system from the source code, then deploy that code to use my copy of the production db?

Server Setup Information

Version of Rocket.Chat Server:
Operating System: Ubuntu 18.04.3
Deployment Method: Tar
Number of Running Instances: 1
DB Replicaset Oplog:
NodeJS Version: 12.21.1
MongoDB Version: 4.0.24
Proxy: Apache
Firewalls involved:

Any additional Information

john.crisp · May 29, 2021, 11:23am

If you are doing this on a separate box first setup Mongo and import your DB, then run your dev system connecting to the new test DB.

Just have plenty of backups everywhere!!

fathom · May 29, 2021, 12:03pm

Thanks John. It is on a brand new box.

My question is really around “replacing” the standard build with a new build. I’m not super familiar with NPM and Meteor yet, so the various steps for building are still foreign to me right now. Here are the steps I’ve used to simply install RC for the past couple of years: Rocket.Chat in Ubuntu - Rocket.Chat Docs. Here is what I’ve played with in cloing the repo and doing a build for it:https://developer.rocket.chat/guides/developer/quick-start-on-linux

If I use the repo clone/build steps, that seems to use .meteor/local/db for the path to the mongo db, and I’ve not been successful in getting that build up and running with the copy of my production db. Those build steps do a “meteor npm start”, which runs its own Mongo server (based on what I see in a ps -ef display). I’d love to get that “repo build” simply moved and running like I do with the “manual build steps” in the first doc above, which starts mongod and rocketchat as services.

Thanks!
Scott

john.crisp · May 29, 2021, 12:27pm

Ha - a good point I can now see. Doh.

I need to go find the brains department and get you some help!! Might be quiet as it is the weekend so could be Monday.

At least for now get the thing running and you can test on a fresh DB - I think your hands are going to be full finding your way around Meteor I believe!!

fathom · May 30, 2021, 12:41pm

Thank you John! Will look forward to some thoughts early this coming week.

Scott

anton.karlan · June 1, 2021, 9:41am

I can share my prod to dev copy method, but we are using docker deployment.

Do mongodb backup whithout these collections

rocketchat_message_read_receipt,
rocketchat_message,
rocketchat_statistics,
rocketchat_uploads,
rocketchat_sessions,
rocketchat_apps_log,
rocketchat_integration_history,
rocketchat__trash,
rocketchat_oembed_cache

Move that backup file to dev host server
On that dev host server stop running docker containers docker stop $(docker ps -q)
Delete current data

rm -rf /var/lib/docker/volumes/rocketchat_rocket_mongo_data_1/_data/ *
rm -rf /var/lib/docker/volumes/rocketchat_rocket_mongo_data_2/_data/ *
rm -rf /var/lib/docker/volumes/rocketchat_rocket_mongo_data_3/_data/ *

unpack backup tar xvf rc_mongo_dev_$DATE.tar -C /var/lib/docker/volumes/rocketchat_rocket_mongo_dump/_data
Start containers (3 for example) docker start rocketchat_s-rc-srv-01_1 rocketchat_s-rc-srv-02_1 rocketchat_s-rc-srv-03_1
Configure Replica Set and most important change PROD url to DEV

docker exec rocketchat_s-rc-srv-01_1 /usr/bin/mongo --ssl --sslAllowInvalidCertificates -- eval "rs.initiate({ _id: 'rs0', members: [ { _id: 0, host: 's-rc-srv-01:27017' } ]})"
docker exec rocketchat_s-rc-srv-01_1 mongorestore --ssl --sslAllowInvalidCertificates
docker exec rocketchat_s-rc-srv-01_1 /usr/bin/mongo --ssl --sslAllowInvalidCertificates -- eval 'db.rocketchat_settings.update({"value":"https://rc-prod.company.com"},{$set:{"value":"https://rc-dev.company.com"}})' rocketchat
docker exec rocketchat_s-rc-srv-01_1 /usr/bin/mongo --ssl --sslAllowInvalidCertificates -- eval 'rs.add( { host: "s-rc-srv-02:27017" } )'
docker exec rocketchat_s-rc-srv-01_1 /usr/bin/mongo --ssl --sslAllowInvalidCertificates -- eval 'rs.add( { host: "s-rc-srv-03:27017" } )'

Finaly start RC docker containers docker start rocketchat_rocketchat_1_1 rocketchat_rocketchat_2_1 rocketchat_rocketchat_3_1

I hope my method can help you to accomplish your goal

fathom · June 1, 2021, 6:04pm

Hi, Anton. Thanks for your detailed process!

My setup is a regular Ubuntu and Apache setup where Mongo and RC run as services. Part of what I’m trying to figure out is how to “build” the RC application code from a Git clone (then modified) so I can simply run it in in the same Ubuntu/Apache setup where Mongo and RC are run as services. I can successfully dump and restore a Mongo db in that kind of setup. For building the app itself, the instructions and shell script examples in the Git repo show samples of runing the app in several setups, but not the one I describe.

Thanks!
Scott

john.crisp · June 2, 2021, 10:20am

OK - I understand you need to do something akin to this:

mongorestore --db meteor folder_with_mongo_dump/

Can you have a play and let us know? I’d like to document it of possible.

fathom · June 10, 2021, 1:21pm

Hi, guys. Sorry I went dark for a while…travel and life got in the way.

So, I’ve made some good headway on this. I have managed to get a “build” of the RC source code with a copy of my production data in the mongo db.

My build steps come from the example script incluced with the repo (remember this is in an Ubuntu/Apache config with no Docker or snap):

meteor --allow-superuser npm install

npm audit fix

meteor --allow-superuser build --server-only --directory /storage/RC/rocket.chat

npm audit fix

cd /storage/RC/rocket.chat/bundle/programs/server

npm install

npm audit fix

meteor --allow-superuser npm install --save @nivo/core

npm audit fix

chown -R rocketchat:rocketchat /storage/RC

My next question is exactly what meteor does when it “builds”. For example, if I pull a new copy of the repo and execute these commands, all is well. If I then simply run the script again (i.e., rebuild the app again), it dies when running the “meteor --allow-superuser build --server-only” command with this output (is there a folder I need to wipe out before “rebuilding”?):

/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/meteor-promise/promise_server.js:218
throw error;
^

SyntaxError: Unexpected token u in JSON at position 1
at JSON.parse ()
at /home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/tools/fs/tools/fs/optimistic.ts:321:17
at wrap.makeCacheKey (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/tools/fs/tools/fs/optimistic.ts:36:15)
at recomputeNewValue (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/optimism/src/entry.ts:182:31)
at Slot.withValue (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/@wry/context/lib/context.js:73:29)
at reallyRecompute (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/optimism/src/entry.ts:165:19)
at Entry.recompute (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/optimism/src/entry.ts:85:9)
at optimistic (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/optimism/src/index.ts:101:25)
at /home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/tools/fs/tools/fs/optimistic.ts:366:19
at recomputeNewValue (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/optimism/src/entry.ts:182:31)
at Slot.withValue (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/@wry/context/lib/context.js:73:29)
at reallyRecompute (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/optimism/src/entry.ts:165:19)
at Entry.recompute (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/optimism/src/entry.ts:85:9)
at optimistic (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/optimism/src/index.ts:101:25)
at find (/tools/isobuild/package-source.js:1339:30)
at /tools/isobuild/package-source.js:1395:27
at Array.forEach ()
at find (/tools/isobuild/package-source.js:1374:22)
at /tools/isobuild/package-source.js:1395:27
at Array.forEach ()
at find (/tools/isobuild/package-source.js:1374:22)
at find (/tools/isobuild/package-source.js:1406:25)
at /tools/isobuild/package-source.js:1395:27
at Array.forEach ()
at find (/tools/isobuild/package-source.js:1374:22)
at find (/tools/isobuild/package-source.js:1374:22)
at /tools/isobuild/package-source.js:1395:27
at Array.forEach ()
at find (/tools/isobuild/package-source.js:1374:22)
at /tools/isobuild/package-source.js:1395:27
at Array.forEach ()
at find (/tools/isobuild/package-source.js:1374:22)
at /tools/isobuild/package-source.js:1395:27
at Array.forEach ()
at find (/tools/isobuild/package-source.js:1374:22)
at /tools/isobuild/package-source.js:1418:34
at Object.withCache (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/tools/fs/tools/fs/files.ts:1663:18)
at PackageSource.findSources (/tools/isobuild/package-source.js:1418:18)
at SourceArch.getFiles (/tools/isobuild/package-source.js:960:32)
at /tools/isobuild/compiler.js:406:23
at /tools/isobuild/compiler.js:186:28
at Object.withCache (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/tools/fs/tools/fs/files.ts:1663:18)
at /tools/isobuild/compiler.js:185:11
at Function..each._.forEach (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/underscore/underscore.js:186:9)
at Object.compile (/tools/isobuild/compiler.js:180:5)
at /tools/isobuild/bundler.js:3268:24
at Object.capture (/tools/utils/buildmessage.js:283:5)
at bundle (/tools/isobuild/bundler.js:3214:31)
at /tools/isobuild/bundler.js:3157:32
at Slot.withValue (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/@wry/context/lib/context.js:73:29)
at Object.withCache (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/tools/fs/tools/fs/files.ts:1663:39)
at Object.bundle (/tools/isobuild/bundler.js:3157:16)
at buildCommand (/tools/cli/commands.js:1082:30)
at /tools/cli/commands.js:945:25
at Function.run (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/tools/tool-env/tools/tool-env/profile.ts:289:14)
at /tools/cli/commands.js:943:18
at /home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/meteor-promise/fiber_pool.js:43:40
=> awaited here:
at Promise.await (/home/ubuntu/.meteor/packages/meteor-tool/.2.1.1.ixi3z9.hw65m++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/meteor-promise/promise_server.js:60:12)
at /tools/cli/main.js:1529:7

john.crisp · June 10, 2021, 2:38pm

Welcome back!

This is something I don’t know, but someone who may will be about some time soon.

Topic		Replies	Views
Best practices for deployment of modified Rocket.Chat Development	5	59	October 8, 2024
Install Rocket chat on fresh Ubuntu 18.04 LTS Community Support	3	2540	March 31, 2020
Unraid Community App Rocket.Chat with MongoDB Community Support	0	468	October 30, 2023
Migrating Rocket-Chat from Meteor built to Synology Docker containers Community Support	3	2111	October 17, 2020
Rocket-chat DATABASE Development	1	771	April 9, 2022

[Rocket.Chat v7.0: Join the webinar] Roadmap Reveal: on-prem AI, VoIP, and more! Save your seat ->

Noob question - Begin customizations but using existing Monbo db

Description

Server Setup Information

Any additional Information

meteor --allow-superuser npm install

npm audit fix

meteor --allow-superuser build --server-only --directory /storage/RC/rocket.chat

npm audit fix

cd /storage/RC/rocket.chat/bundle/programs/server

npm install

npm audit fix

meteor --allow-superuser npm install --save @nivo/core

npm audit fix

chown -R rocketchat:rocketchat /storage/RC

Related topics