Can you use GridFS to store large zip files?

945 views
Skip to first unread message

Khalid

unread,
Jun 19, 2014, 9:38:21 AM6/19/14
to mongod...@googlegroups.com
I have stored zip files in GridFS, but I could not retrieve the whole ZIP file. How can pull it from GridFS chunks collection into the file system using the Python driver? Is there a way to serve zip files in GridFS instead of loading the chunks in the memory? 

I looked at the examples in the documentation but they are too simple and do not show the power of GridFS.
If this is a limitation in the Python driver, can you show me how to do it in other drivers (Java or Ruby)?

Khalid

unread,
Jun 20, 2014, 1:41:49 AM6/20/14
to mongod...@googlegroups.com
It turned out it was straightforward to do in Java. Just get the input stream and copy it into a new file using Apache IOUtils.

GridFSDBFile zipFile = gridGfs.find(new ObjectId("539edfcc7836782a6bd2970b"));
InputStream inputStream = zipFile.getInputStream();
File file = new File("/Users/Khalid/file.zip");
if (!file.exists()) {file.createNewFile();}
FileOutputStream fileOutputStream = new FileOutputStream(file);
IOUtils.copy(inputStream, fileOutputStream);

Alex at Ikanow

unread,
Jun 20, 2014, 11:17:35 AM6/20/14
to mongod...@googlegroups.com
Seems like you have a working solution, but in case it was useful I just wanted to mention a utility I wrote and open-sourced:


This provides a zip file interface that backs directly onto the GridFS, ie you can extract individual files from within the zip archive without having to copy the whole thing to disk first:

    import com.ikanow.utility.GridFSRandomAccessFile;
   
import net.sf.jazzlib.GridFSZipFile;
   
//+MongoDB imports
   
String dbName = "test";
   
DBCollection db = new MongoClient().getDB(dbName);
   
String fsName = "fs"; // (the default)
   
ObjectId fileId = new ObjectId("FILEID");
   
GridFSRandomAccessFile shareAsFile = new GridFSRandomAccessFile(db, fsName, fileId);
    net
.sf.jazzlib.GridFSZipFile zipFile = new net.sf.jazzlib.GridFSZipFile("dummy", shareAsFile);
   
//Then use the zipFile object however you would use a java.util.zip.ZipFile...


Reply all
Reply to author
Forward
0 new messages