GridFS Streaming without loading entire file into memory(Java)

[Franklin De Los Santos]

I'm having a problem with large files. Does gridfs.createFile() load all the contents into memory? I have code to upload and download video files, but sometimes I get the buffer out of memory error, can someone review this and help out:

Here I'm uploading the file using a ServletOutputStream:

GridFS gridFS =  new GridFS(mongoDb, "videos");
ServletInputStream servletInputStream  = req.getInputStream();
GridFSInputFile gfsFile = gridFS.createFile(servletInputStream);

gfsFile.put("location", location);
gfsFile.put("username", AuthenticationService.userName);
gfsFile.put("contentType", req.getContentType());
gfsFile.put("filename", "noname");
gfsFile.put("likes", 0);
collection.insert(gfsFile);
gfsFile.save();

Does this code load the entire file into memory? Does gridFS support uploading parts of the file at a time?

code to stream video with seek functionality:

 InputStream inputStream = gridFSDBFile.getInputStream();
    ServletOutputStream out = resp.getOutputStream();
    String range = req.getHeader("Range");
    if (range == null) {
        try {
            IOUtils.copy(inputStream, out);
        } finally {
            inputStream.close();
        }

    }

    String[] ranges = range.split("=")[1].split("-");
    int from = Integer.parseInt(ranges[0]);
    int to = (int) gridFSDBFile.getChunkSize() + from;
    if (to > gridFSDBFile.getLength()) {
        to = (int) (gridFSDBFile.getLength() - 1);
    }
    if (ranges.length == 2) {
        to = Integer.parseInt(ranges[1]);
    }
    int len = to - from + 1;

   // resp.setStatus(HttpStatus.PARTIAL_CONTENT_206);
    resp.setHeader("Accept-Ranges", "bytes");
    final String responseRange = String.format("bytes %d-%d/%d", from, to, (int) gridFSDBFile.getLength());
    resp.setHeader("Content-Range", responseRange);
    resp.setContentLength(len);
    inputStream.skip(from);
    byte[] buf = new byte[1024];
    try {
        while (len != 0) {
            int read = inputStream.read(buf, 0, buf.length > len ? len : buf.length);
            out.write(buf, 0, read);
            len -= read;
        }
    } finally {
        inputStream.close();
    }

    //resp.setStatus(HttpStatus.PARTIAL_CONTENT_206);

}

[Justin Lee]

The bytes are streamed in/out of the database so all that's in memory is what your application is holding on to and some buffering in the driver.

--------------------------------

{ name     : "Justin Lee", 

  title    : "Software Engineer",

  twitter  : "@evanchooly",

  web      : [ "mongodb.com", "antwerkz.com" ],

  location : "New York, NY" }

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: https://docs.mongodb.org/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/7bdb4163-3c65-4f28-8435-e5f6f208b04a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.