Major API changes in TSet

Niranda Perera

unread,

Jul 27, 2020, 10:55:05 PM7/27/20

to Twister2, Ahmet Uyar, Supun Kamburugamuve, Vibhatha Abeykoon, Pulasthi Supun Wickramasinghe

Hi all,

I made some major API changes to TSets.

1. Swapping the generics order in TSets

Previously TSets functions had generics as <O, I>. It is changed to <I, O>. (I= input type, O=output type). This is more in-line with other frameworks like Spark.

2. Removed Input type generic from ComputeTsets. Previously the signature was ComputeTSet<O, I>. But input types do not serve any purpose for a TSet, because no subsequent communication operations, do not care how that TSet was created. So, the new signature was changed to ComputeTSet<O> that only follows output type. Same logic follows for SComputeTSet<O>, KeyedTSet<K, V> and SKeyedTSet<K, V>.

These changes were added to this PR

https://github.com/DSC-SPIDAL/twister2/pull/935

Later I added "computeToTuple" method to TLinks. Previously, if someone wanted to create a KeyedTSet after a compute, they had to go through a direct link as follows.

...compute(some compute).direct().mapToTuple(some map)

Now this can be done in a single operation as follows

...computeToTuple(some compute with tuple output)

This was added with this PR

https://github.com/DSC-SPIDAL/twister2/pull/936

Please note that this PR is dependent on the #935 PR.

@Pulasthi Supun Wickramasinghe these changes would affect Beam runners AFAIU.

@Vibhatha Abeykoon These changes also affect windowing operations.

I made some changes there to fix compilation errors. Could you guys please double-check those changes?

Best

--

Niranda Perera