A have a python script witch takes a huge list of images and move them in a circle to mongoDB using gridfs (it stores files in two mongoDB collections, files and chunks).
After few thousands of successful transfers I am starting to get following error:
assertion C:\data\mci\77b70aa30ca8170c4ee9bee2ba42cf1f\src\src\mongo/util/concurrency/rwlock.h:204
I tried to debug it and it looks, that it becomes fail during chunk insert, but why this happens I don't know.
import pymongo
from pymongo import MongoClient
import gridfs
fs = gridfs.GridFS(db)
for image in images_list:
with open(vers['path'], 'rb') as fileObj:
try:
# try:
try:
grid_in = fs.new_file(filename=image[1]['name'],
metadata={"imageId": image[1]['_id'],
"version": version})
grid_in.write(fileObj)
file_id = grid_in._id
finally:
grid_in.close()
except pymongo.errors.WriteError as we:
print('%s %s:%s' % (image[1]['_id'], we, vers['path']))
grid_in.abort()
continue
fileObj.closed
I also tried gridfs.put instead of new_file and write and also GridFSBucket.open_upload_stream and GridfsBuscket.upload_from_stream - result always the same.
Since some moment I am getting error, which a mentioned above
Do you know why this happens and how can I solve it?
Or maybe you can at least answer following question:
- can be some problems because of inserting the same images few times? or images with same filename?
- can it be error because of access to mongoDB (lock mechanizm or smth) and how can I then change it to avoid any conflicts?
If you have some other ideas, they all are welcome.
Thank you!