“Document exceeds allowed max BSON size. The max is 16777216.” error using GridFS

[Thierry Leloup]

Hello everybody,

I am trying to insert a 33 MB video file in my mongo database using GridFS and ruby and I have a systematic "Document exceeds allowed max BSON size. The max is 16777216.". I thought the only way to insert file greater than 16 MB in a mongo collection was to use Gridfs so I suspect I do it wrong even though I copy/paste the Ruby driver example (http://docs.mongodb.org/ecosystem/tutorial/ruby-driver-tutorial/#gridfs).

I am using Ruby 2.2.1, mongo driver 2.0.4 and a mongo db 3.0.1. My code :

eDatabase = Mongo::Client.new(......)
@grid = @eDatabase.database.fs

myvideo = File.open("files/33mo.mov", 'rb')
grid_file = Mongo::Grid::File.new(myvideo.read, :filename => "myvideo.mov")

@grid.insert_one(grid_file)

And my logs :

21/05/2015 11:12:25] ERROR : Exception in main loop : Document exceeds allowed max BSON size. The max is 16777216.
[21/05/2015 11:12:25] ERROR : Backtrace : /Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/protocol/serializers.rb:157:in `serialize'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/protocol/message.rb:153:in `block in serialize_fields'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/protocol/message.rb:141:in `each'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/protocol/message.rb:141:in `serialize_fields'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/protocol/message.rb:70:in `serialize'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/server/connection.rb:123:in `block in write'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/server/connection.rb:122:in `each'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/server/connection.rb:122:in `write'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/server/connectable.rb:66:in `block in dispatch'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/loggable.rb:44:in `log'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/loggable.rb:65:in `log_debug'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/server/connectable.rb:65:in `dispatch'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/operation/executable.rb:35:in `block in execute'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/server/connection_pool.rb:99:in `with_connection'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/server/context.rb:63:in `with_connection'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/operation/executable.rb:34:in `execute'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/operation/write/insert.rb:71:in `execute_write_command'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/operation/write/insert.rb:61:in `execute'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/collection.rb:196:in `insert_many'
/Users/Atome/.rvm/gems/ruby-2.2.1/gems/mongo-2.0.4/lib/mongo/grid/fs.rb:78:in `insert_one'
server.rb:530:in `handle_addobject_message'

FYI, line server.rb:530 corresponds to

@grid.insert_one(grid_file)

Notes :

• the very same code is working fine with a < 16 MB file, i.e. both chance and file documents are inserted

• everything works fine when I use mongofiles command...

• other inserts in the db work fine from my application

Thanks,

Thierry

[Emily S]

Hi Thierry

Thanks for reporting this. I've opened a ticket in the ruby driver JIRA project if you'd like to track it:

https://jira.mongodb.org/browse/RUBY-946

A fix is in code review and will be released with 2.0.5 soon.

Thanks!

Emily

[Thierry Leloup]

OK, thanks, Emily.

Thierry

[Emily S]

Hi Thierry

We've released driver version 2.0.5 (mongo gem) with a fix for this. Feel free to open a ticket in our JIRA project, if you find any other issues.

https://jira.mongodb.org/browse/RUBY

Thanks

Emily

On Thursday, June 11, 2015 at 1:08:01 PM UTC+2, Thierry Leloup wrote:

OK, thanks, Emily.

Thierry