SenseiDB getting more SenseiBA-like?

[Otis Gospodnetic]

Hi,

I looked at SenseiDB 1.6.0 release notes at
http://senseidb.github.io/sensei/2013/04/10/sensedb-1.6.0-released.html
and see more support for analytics queries.

At the same time there is a parallel effort (worked on by some of the
same people?) in form of Sensei BA. One of its FAQ entries says
full-text/Lucene was dropped and real-time indexing was dropped in
order to make all the other stuff possible.

How exactly do these two efforts.... work. While one is going away
from Lucene and SenseiDB pieces that have to do with search and
real-time, the other one is going more analytical, which I suppose
makes it more Sensei BA-like.

Thanks,
Otis
--
Search Analytics - http://sematext.com/search-analytics/index.html
SENSEI Performance Monitoring - http://sematext.com/spm/index.html

[Volodymyr Zhabiuk]

Hi Otis

Again, we need to improve documentation on this. In SenseiDB 1.6.0 we can do the queries like

select sum(cost), avg(year) where tag='automatic' group by groupid, color

or 

select distinctCount(year) where  tag='automatic'

The same functionality is available in SenseiBA as well. Internally it's implemented using Sensei's map reduce engine http://senseidb.github.io/sensei/map-reduce.html. We've spent lots of efforts to make sure that map reduce jobs could be run against Sensei's Lucene/Bobo indexes as well as against SenseiBA indexes in the same way 

With many thanks,

Volodymyr

2013/5/7 Otis Gospodnetic <otis.gos...@gmail.com>

--
You received this message because you are subscribed to the Google Groups "Sensei" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sensei-searc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Otis]

Thanks Volodymyr!

I didn't pose my question well. :)  My question was really a bit more high level, so to speak.  That is, from the outside it seems like both SenseiDB and BA are overlapping a lot and each wants something that the other one has.  DB wants analytical queries like in your example.  BA wants real-time indexing and full-text search (I presume.... if not today, then in the future - I'm willing to bet my left pinkie!).  So it almost feels like eventually they will collide and merge.  But this is how things look to me from the outside.  Maybe you know of some inside or under the hood reasons why that is not likely to happen in the future?

Thanks,

Otis
--
Search Analytics - http://sematext.com/search-analytics/index.html
SENSEI Performance Monitoring - http://sematext.com/spm/index.html

[Volodymyr Zhabiuk]

Hi Otis

Thank you for good insights. 

The major difference between Sensei and SenseiBA is that SenseiBA doesn't use Lucene. It would never do full text search, but on the other hand it might give much smaller memory footprint if the column cardinality (number of distinct values) is low. We may try to merge them together, but this would greatly increase system's complexity. So I'm not sure about this. Historically Sensei team  was trying to keep things simple

Vlad 

2013/5/8 Otis <otis.gos...@gmail.com>