Is there a limit to the size of what can be stored in a given key?

88 views
Skip to first unread message

Gitted

unread,
Feb 23, 2011, 3:24:32 PM2/23/11
to project-voldemort
What is the limit to what can be stored in a key?

i.e. is storing an article appropriate? (so it would be of type string
UTF-8 I would presume)

Alex Feinberg

unread,
Feb 23, 2011, 4:50:04 PM2/23/11
to project-...@googlegroups.com
There is no limit in Voldemort itself. That said, there's a practical
limit: since the values are stored on disk and retrieved in a single
operation, rather than streamed. Storing whole article makes sense and
is reasonable. At LinkedIn, we have several values that exceed a few
mb (but they're very rare "one-offs") and routinely use values that
are 15kb in size (e.g., when you share an article on LinkedIn, a
thumbnail picture from that article is stored in Voldemort). On the
other hand, if you're looking to store full size photographs or video,
Voldemort is not the right tool (Amazon's S3, MogileFS or something
like Facebook's "Haystack" would be the right solution).

Thanks,
- Alex

> --
> You received this message because you are subscribed to the Google Groups "project-voldemort" group.
> To post to this group, send email to project-...@googlegroups.com.
> To unsubscribe from this group, send email to project-voldem...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/project-voldemort?hl=en.
>
>

Gitted

unread,
Feb 23, 2011, 5:19:48 PM2/23/11
to project-...@googlegroups.com
Say I wanted to create an application like google groups, and store each forum message/post in voldermort, would that be suitable?

I guess 'yes' since you said an article would be a good fit.

in this case, your reply to my quesiton was about .6kb in size, but ofcourse some replies may be much larger....

Liam Slusser

unread,
Feb 24, 2011, 4:27:29 AM2/24/11
to project-...@googlegroups.com
We have a V cluster of nearly 5 million protobuf objects with an avg
object size of around 18k. Works extremely well, even in our heavy
write environment.

liam

Alex Feinberg

unread,
Feb 24, 2011, 3:57:08 PM2/24/11
to project-...@googlegroups.com
Yes,

That configuration would work very well. We have number of use cases
where average value is small, but with some outliers.

Thanks,
- Alex

Reply all
Reply to author
Forward
0 new messages