The roadmap of Supersonic

[Zhichao Liang]

Hi,

  Is there a roadmap of Supersonic? Any additional and advanced functions such as data compression will be provided in the future?

Best Regards!

Zhichao Liang

[Piotr Tabor]

Hi Zhichao, 

There is no clean read-map for Supersonic.

In the open-sourced version:

 - A few students is working/thinking on columnar storage format for Supersonic (including lightweight column compression). 

 - One person is working on Join's API cleanup and maybe new join implementations. 

That's what I'm aware of. In fact I expected more involvement from community side it the project. Few examples from top of my head:

 - OSX / Windows build

 - any optimizations

 - support for other data formats / data sources.

 - there is idea to change Supersonic model of memory. Currently cursors own memory blocks that holds multiple columns.  

   If we changed this to a model where we have per-column granularity and get rid of cursor ownership of block, than we could get 

   probably better performance and more convenient API to write cursors. In the new model a memory should be refcounted and

   exists as long as any Column View pointing to this memory exists. 

Some internal focus is around adding better support of Protobufs, maybe full-structure. 

I will try to sync / release a new version once a quarter - so it should happen soon. 

Piotr

Zhichao Liang

--
You received this message because you are subscribed to the Google Groups "Supersonic Query Engine Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to supersonic-query-...@googlegroups.com.
To post to this group, send an email to supersonic-...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Zhichao Liang]

Hi Piotr,

  Thank you for your reply! I believe an usable columnar storage format for Supersonic is very necessary for users. Due to columnar storage missing in Supersonic, now i'm working on integrating Supersonic with another storage engine to exploit the excellent query function provided by Supersonic. However, if Supersonic has its own columnar storage, it will make things easy.

Zhichao

在 2013年4月9日星期二UTC+8下午11时58分18秒，ptab写道：

To unsubscribe from this group and stop receiving emails from it, send an email to supersonic-query-engine+unsub...@googlegroups.com.

[ajoy frank]

Hi Zhichao,

Did you integrate this with a columnar storage format? Did it work well for you?

Thanks!

[Zhichao Liang]

Hi ajoy,

 Actually, i implemented a columnar storage format in Redis and integrated Supersonic with it as a query engine. I think it work well for our application requirement. If you're interested in its detail performance, i can send the performance summary  to you.

Zhichao

2013/11/15 ajoy frank <ajo...@gmail.com>

--
Zhichao Liang
M.S Candidate
MSN:frank...@live.cn
Flash DataBase Group
Web and Mobile Data Management lab
Information School,Renmin University of China 

[Piotr Tabor]

Hi Zhichao, 

Have you considered sharing the code ? We could put this is contrib subdirectory or keep it in a separate repo, but link from the webpage. 

Piotr

To unsubscribe from this group and stop receiving emails from it, send an email to supersonic-query-...@googlegroups.com.

[Timothy Chen]

Please share the performance summary as well as I'd like to learn more about what the results look like.

Tim

[Ajoy Frank]

Zhichao,

That's awesome .. is it open sourced? Can you please point me to how I can use it?

Thanks!

-Ajoy

[Zhichao Liang]

Yes, i'd like to share but for now it's only an ugly prototype for evaluation. I believe there's still a lot of work to do before open source but i will improve it in my spare time.

在 2013年11月20日星期三UTC+8下午8时10分35秒，ptab写道：

2013/11/15 ajoy frank <ajo...@gmail.com>

To unsubscribe from this group and stop receiving emails from it, send an email to supersonic-query-engine+unsubscr...@googlegroups.com.

To post to this group, send an email to supersonic-...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Zhichao Liang
M.S Candidate
MSN:fra...@live.cn

Flash DataBase Group
Web and Mobile Data Management lab
Information School,Renmin University of China 

--
You received this message because you are subscribed to the Google Groups "Supersonic Query Engine Group" group.

[Zhichao Liang]

  Our application requirement is simple, just do some aggregations, sort, and two-tables join on the intermediate result generated by mapreduce. So the data to process is not very large but the process must be done very quickly. I implemented the redis columnar storage in three different ways, dynamic array, segmented

or linked dynamic array, segmented dynamic array with compression. In addition, i also implemented some extra commands for redis to support table create and table query. I did some evaluation on the prototype and you guys can find the performance summary in the attachment. All questions about it are welcome!

Zhichao

在 2013年11月20日星期三UTC+8下午8时23分09秒，Timothy Chen写道：

2013/11/15 ajoy frank <ajo...@gmail.com>

To unsubscribe from this group and stop receiving emails from it, send an email to supersonic-query-engine+unsubscr...@googlegroups.com.

To post to this group, send an email to supersonic-...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Zhichao Liang
M.S Candidate
MSN:fra...@live.cn

Flash DataBase Group
Web and Mobile Data Management lab
Information School,Renmin University of China 

--
You received this message because you are subscribed to the Google Groups "Supersonic Query Engine Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to supersonic-query-engine+unsub...@googlegroups.com.
To post to this group, send an email to supersonic-...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.