gateway API proposal

[John Wang]

Hi folks:

     I'd like to propose the following API change to simply gateway.

     Gateway should be a factory for a standard java Iterable implementation, e.g.

     

     public Iterable<V> iterable(String version);

      The benefits:

1) more flexible and integrates with the standard JDK better

2) allows cleaner iteration code, .e.g for each works.

     Thoughts?

-John

[Mark Williams]

Are you saying that the interface DataProvider<V>, which currently is only a marker interface, will now have a new method "iterable(String version)"?

Is version the startingOffset?

And if so, how do you provide the other functionality such as pausing, stopping, or resetting? Would you have to discard the current iterable and go back to the factory to get a new one?

-mark

[Volodymyr Zhabiuk]

Hi John, Mark

The interface looks awesome. It's much cleaner than a previous one.

As for the pausing, stopping, or resetting functionality I don't
remember a single case somebody used it


With many thanks,
Volodymyr


2013/5/21 Mark Williams <zaido74...@yahoo.com>:

> --
> You received this message because you are subscribed to the Google Groups
> "Sensei" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sensei-searc...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Yonghui Zhao]

Reset should be used for failover, if sensei crashes the ram index is lost, we need reset the offset for data provider(kafka,  etc).

next() and reset(String version)  should be necessary.

2013/5/22 Volodymyr Zhabiuk <vzha...@gmail.com>

[Volodymyr Zhabiuk]

Hi Yonghui

If Sensei node crashes and restarts, we may just get the Iterable from
the Gateway by passing the last persistent version.
Is the current reset method been called from anywhere in Sensei eg
DefaultStreamingIndexingManager?

Thanks,
Volodymyr

2013/5/22 Yonghui Zhao <zhaoy...@gmail.com>:

[Yonghui Zhao]

Getting the iterable from gateway by version sounds good.

In this case we don't need reset interface.