downloading files from GridFS::GridStore in Ruby
amcvega
I started using GridFS::GridStore since the uploads for a project I'm
working on will definitely be bigger than 4MB. So far, uploading and
storing stuff to gridfs doesn't seem to be a problem. However when I
want to download the file, it takes a long time.
Right now I use GridFS::GridStore#read to put the binary data into the
body of the response. Is this the best practice?
Kyle Banker
--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
Miguel Vega
http://pastie.org/743439
I'm using the excellent the mongomapper and mongomapper_ext libraries
to make life easier. But that doesn't really matter, the
"@attachment.file" in line 20 returns a GridFS::GridStore object so
the 'read' method there is the one from mongo.
I have a 3mb file uploaded and when I benchmark the
GridFS::GridStore#read it takes up to 40 seconds for it to finish
reading so I'm sure the bottleneck is at that method.
Thanks for the response.
Miguel Vega
Upstrat, Inc.
http://theavidmind.upstrat.com
Sunny Hirai
We've significantly increased read performance by using #fast_read as
implemented in this piece of code from a user named Nuanda:
http://gist.github.com/214124
It also improves write performance quite significantly as well (using
#fast_write).
Sunny Hirai
On Dec 14, 4:41 pm, Miguel Vega <amcv...@gmail.com> wrote:
> I'm probably using it wrong then. Here's the relevant codehttp://pastie.org/743439
> I'm using the excellent the mongomapper and mongomapper_ext libraries
> to make life easier. But that doesn't really matter, the
> "@attachment.file" in line 20 returns a GridFS::GridStore object so
> the 'read' method there is the one from mongo.
>
> I have a 3mb file uploaded and when I benchmark the
> GridFS::GridStore#read it takes up to 40 seconds for it to finish
> reading so I'm sure the bottleneck is at that method.
>
> Thanks for the response.
>
>
>
> On Mon, Dec 14, 2009 at 10:51 PM, Kyle Banker <k...@10gen.com> wrote:
> > Reading should be faster than writing. #read is the correct api. Maybe you
> > can share a bit more about you app?
>
Miguel Vega
Thanks for the link Sunny. I'll try it out right away.
Kyle Banker
This is great. I'm going to try and work it into the driver so that all the specs still pass. Look for it in an upcoming commit.
Kyle
Miguel Vega
Here's the ruby benchmark on a reading a 3.5 mb file
Rehearsal ---------------------------------------------
read 36.650000 4.930000 41.580000 ( 41.867265)
fast_read 3.800000 0.030000 3.830000 ( 3.843537)
----------------------------------- total: 45.410000sec
user system total real
read 37.040000 4.920000 41.960000 ( 42.013695)
fast_read 3.650000 0.050000 3.700000 ( 3.722122)
37.040000 4.920000 41.960000 ( 42.013695)
3.650000 0.050000 3.700000 ( 3.722122)
Kyle Banker
It is indeed a great improvement. Thanks, Sunny!
Damon P. Cortesi
I had some issues with GridFS speed before. As a random aside to tack
on this thread, I ended up hacking my rails app so it actually wrote
the data out to a temp file and then used the local mongofiles to read/
write to GridFS. Even with the fast_ additions, it was still faster to
do it that way. Not sure what size the files were off the top of my
head, but probably anywhere in the neighborhood of 14-20MB.
Damon
On Dec 16 2009, 7:13 am, Kyle Banker <k...@10gen.com> wrote:
> I've also pushed this change to the latest master, and added a benchmark:
> bin/gridfs_benchmark.
>
> It is indeed a great improvement. Thanks, Sunny!
>
> On Tue, Dec 15, 2009 at 11:08 PM, Miguel Vega <amcv...@gmail.com> wrote:
> > it worked like a charm! Definitely a lot faster.
>
> > Here's the ruby benchmark on a reading a 3.5 mb file
>
> > Rehearsal ---------------------------------------------
> > read 36.650000 4.930000 41.580000 ( 41.867265)
> > fast_read 3.800000 0.030000 3.830000 ( 3.843537)
> > ----------------------------------- total: 45.410000sec
>
> > user system total real
> > read 37.040000 4.920000 41.960000 ( 42.013695)
> > fast_read 3.650000 0.050000 3.700000 ( 3.722122)
> > 37.040000 4.920000 41.960000 ( 42.013695)
> > 3.650000 0.050000 3.700000 ( 3.722122)
>
> > >> >> mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>
> > .
> > >> >> For more options, visit this group at
> > >> >>http://groups.google.com/group/mongodb-user?hl=en.
>
> > >> > --
>
> > >> > You received this message because you are subscribed to the Google
> > Groups
> > >> > "mongodb-user" group.
> > >> > To post to this group, send email to mongod...@googlegroups.com.
> > >> > To unsubscribe from this group, send email to
> > >> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>
> > .
> > >> > For more options, visit this group at
> > >> >http://groups.google.com/group/mongodb-user?hl=en.
>
> > >> --
> > >> Miguel Vega
> > >> Upstrat, Inc.http://theavidmind.upstrat.com
>
> > > --
>
> > > You received this message because you are subscribed to the Google Groups
> > "mongodb-user" group.
> > > To post to this group, send email to mongod...@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>
> > .
> > > For more options, visit this group at
> >http://groups.google.com/group/mongodb-user?hl=en.
>
> > --
> > Miguel Vega
> > Upstrat, Inc.
> >http://theavidmind.upstrat.com
>
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "mongodb-user" group.
> > To post to this group, send email to mongod...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>