Changing SS memory materialization

[Usama Dar]

Hi,

 

Currently SS Table maintains a one large chunk of contigeous memory. We would like to change some of that and implement the memory storage in paritions down to segment level(Both horizontal and vertical partitions and the full monty).

 

I see the code and think extending BasicCursor to create our own cursor which canbe the data-source of the data would be the way to go, and i just want to validate my thinking here.

 

Would i be right in implementing a different "Table" API as a sink whereby doing an "AddRow" on the table will store data in memory according to our own scheme

 

In our query execution this table will be the datasource producing 1024 row vectors at a time and feeding it to other SS operations which don't need to change.

 

Is this the right way to do it? Can we run into other issues which are not forseen?

 

Thanks,

 

[Piotr Tabor]

Hi Usama,

In general it makes sense. 

Are you going to feel your 'storage' from other cursors output, or 'manually' by your private logic ?

If the source of data for this storage is other cursor, indeed your storage should implement Sink API ( 

basically: virtual FailureOr<rowcount_t> Write(const View& data) = 0;

that gives you flexibility to interpret whole block of data at once.

Otherwise your custom API should be sufficient. 

And than you can Create cursors backed by your table.

Piotr

 

--
You received this message because you are subscribed to the Google Groups "Supersonic Query Engine Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to supersonic-query-...@googlegroups.com.
To post to this group, send an email to supersonic-...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.