views

[Jay Kreps]

Hey All,

I have a patch and proposal for adding views in voldemort. The details are here:

http://code.google.com/p/project-voldemort/issues/detail?id=178

This is a fairly minor code change but introduces what amounts to a
new client-facing interface. Because of this I would really like any
review or comments people could give. Any other use cases people have
would probably help strength then design.

Thanks!

-Jay

[ijuma]

On Nov 25, 9:22 am, Jay Kreps <jay.kr...@gmail.com> wrote:
> Hey All,
>
> I have a patch and proposal for adding views in voldemort. The details are here:

I haven't looked at the details yet, but this is definitely needed, so
thanks for looking into it. I added a few suggestions to the issue.

Best,
Ismael

[Eishay Smith]

Like it!

We have large objects and getting a subset 
would be very useful. One extreme usecase would be a transformation or Any to 
Boolean that would implement contains(key).

How would you do an update with the proposal? 
Assuming the value is a list of Foo and the view value is a Bar, don't you need also a:
    public String fromViewToStore(String k, Bar v, List<Foo> existingValue) {
        if (null != existing) return existing.add(new Foo(existingValue);
        return Lists.newArrayList(new Foo(existingValue);
    }

[Jay Kreps]

Yes that is a very good point. I will need to update that interface to
take the existing value.

Another potential problem with the interface is that the
transformation happens per-value. So if there are no values there are
no transformations. This means that you cannot create a view that
masks missing values. Not sure if this is important or not.

Anyhow, there is clearly some baking needed here. I think it would be
good for me to try to implement so of these use cases and see what
they look like. We could start a little contrib library with some
support for basic data structures.

One thing that would make this much more powerful was if there was a
way to specify additional query information that is not used for
partitioning and routing in the query. For example we have long had
the idea of storing personalized indexes in voldemort for things like
inbox search and addressbook search. And an inverted index like lucene
uses is inherently a serializable structure so this would be a natural
thing to do. Clearly a view would push this computation closer to the
data, but you need to be able to pass in additional query info aside
from the primary key (say a user id) to make this possible. This would
be a larger API change.

-Jay

> --
>
> You received this message because you are subscribed to the Google Groups
> "project-voldemort" group.
> To post to this group, send email to project-...@googlegroups.com.
> To unsubscribe from this group, send email to
> project-voldem...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/project-voldemort?hl=en.
>