Have problem with exporting file from GridFS

Description

Hello guys,

RocketChat become slowly when uploading/reading picture.

And @john.crisp (Community Manager) suggested me to export the file from GridFS with this script.

arminfelder/gridfsmigrate: RocketChat GridFS to filesytem migration script

To make the plan for doing it,I mongodump the rocketchat database from MongoDB which is working now.

Then do mongorestore to another VM which installed MongoDB.x86_64 (NOT in docker container. ver 4.4).

Executing the python script like below


python3 ./migrate.py -c dump -r rocketchat -t FileSystem -d /root/gridfs_dump

And here is the return error.


Traceback (most recent call last):

  File "/usr/local/lib64/python3.6/site-packages/gridfs/grid_file.py", line 755, in next

    chunk = self._next_with_retry()

  File "/usr/local/lib64/python3.6/site-packages/gridfs/grid_file.py", line 747, in _next_with_retry

    return self._cursor.next()

  File "/usr/local/lib64/python3.6/site-packages/pymongo/cursor.py", line 1215, in next

    raise StopIteration

StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "./migrate.py", line 229, in <module>

    obj.dumpfiles("rocketchat_uploads", store)

  File "./migrate.py", line 104, in dumpfiles

    data = res.read()

  File "/usr/local/lib64/python3.6/site-packages/gridfs/grid_file.py", line 565, in read

    chunk_data = self.readchunk()

  File "/usr/local/lib64/python3.6/site-packages/gridfs/grid_file.py", line 528, in readchunk

    chunk = self.__chunk_iter.next()

  File "/usr/local/lib64/python3.6/site-packages/gridfs/grid_file.py", line 759, in next

    raise CorruptGridFile("no chunk #%d" % self._next_chunk)

gridfs.errors.CorruptGridFile: no chunk #0

Server Setup Information

  • Version of Rocket.Chat Server: 3.6.1

  • Version of MongoDB server: 4.4

  • Operating System: CentOS 7

  • Deployment Method: both RocketChat and MongoDB in Docker container

  • Number of Running Instances: 2

  • Firewalls involved: no

Thanks for your help.

As per conversations on open…

Have a search for this:
gridfs.errors.CorruptGridFile: no chunk

eg
https://jira.mongodb.org/browse/PYTHON-1552

There are a number of potential reasons - out of space etc etc

Personally I would get a good backup and then update your whole Rocket install first - if you want to be cautious do it a version at a time with backups between just to make sure the DB modifications work.

When moving DB make sure you move between the SAME versions of Rocket or you will hit problems.

Do NOT export from 3.6.1 and import to say 3.14.0

Make a sure you have plenty of space to export from GridFS

Something else I have just seen is you are running 3.6.1 on Mongo 4.4 ?

Did you import to Mongo 4.4 first and THEN try to export from GridFS? I’m not sure how good 4.4 support is.

This may need to be updated but…

https://docs.rocket.chat/installation/manual-installation/mongo-versions

1 Like

I’m very sure that the space of disk is quite enough.

Let me make sure I have it right.

  1. Make the good BACKUP BEFORE DONING ANYTHING. Yes,I got it.

  2. Update the RocketChat version will modify the MongoDB schema. So hold the MongoDB version and update RocketChat.

After update RocketChat make sure it is working well.

  1. Do NOT export DB from RocketChat 3.6.1 and import to 3.14.0, and so on.

Yes,I’m running RocketChat 3.6.1 on MongoDB 4.4.

Yes,I import to MongoDB 4.4 first and try to export from GridFS, maybe this is the answer why I have problem when exporting GridFS ?

I’ll try to use MongoDB ver 3.6.9 as official recommend.

Thank you again.

Have a nice day