Aggregation Operations on Input as Groups

[Suminda Dharmasena]

If possible can you add the following aggregation operations:

• If all inputs have values available values then transform it into a structure and process
• If any input in group has a values then transform it into a structure and process
• Flatten inputs into one stream as they arrive
• Flatten inputs into one stream in order
• etc.

Have a look at:

• http://gpars.org/guide/index.html
  
• http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-M2/scala.html?_ga=1.21434555.1234628154.1418042628
  
• https://github.com/scalaz/scalaz-stream
  
• etc.

[Suminda Dharmasena]

In a nut shell merging ports into another port with an optional trastrasfomation in the process

[Jordan Halterman]

I think yes! That is definitely what I have been working towards - transformations. We should talk about what this should actually look like once the messaging is back in working order as I assume it's something that can be built on top of that.

On Jan 27, 2015, at 8:30 PM, Suminda Dharmasena <sirina...@gmail.com> wrote:

In a nut shell merging ports into another port with an optional trastrasfomation in the process

--
You received this message because you are subscribed to the Google Groups "vertigo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx-vertig...@googlegroups.com.
To post to this group, send email to vertx-...@googlegroups.com.
Visit this group at http://groups.google.com/group/vertx-vertigo.
To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/fbb0e23e-22e3-46ac-a094-28297b8290e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Suminda Dharmasena]

Another use case will be for a given input port to get inputs in particular order. E.g. A1, A2, B1, C1, B2, C2 => A1, B1, C1, A2, B2, C2

[Oliver Rolle]

If I remember correctly we had a similar discussion some time ago about transformation, selections, aggregation etc. on streams. Maybe we could create a github project which implements these functions as vertigo components + DSL (similar to spark streaming api?) to easily combine these operators?

[Jordan Halterman]

Indeed, indeed we did. I think Vertigo has been intentionally going in the direction of supporting functions and aggregations, and I would like to implement ten in a separate project.

Actually, a while back I prototyped such an API but never released it simply because Vertigo lacked the guarantees to make accurate calculations feasible. Presumably, that will no longer be the case.

On Jan 28, 2015, at 12:54 PM, Oliver Rolle <oliver...@googlemail.com> wrote:

If I remember correctly we had a similar discussion some time ago about transformation, selections, aggregation etc. on streams. Maybe we could create a github project which implements these functions as vertigo components + DSL (similar to spark streaming api?) to easily combine these operators?

--
You received this message because you are subscribed to the Google Groups "vertigo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx-vertig...@googlegroups.com.
To post to this group, send email to vertx-...@googlegroups.com.
Visit this group at http://groups.google.com/group/vertx-vertigo.

To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/756db2f8-3448-476f-9845-1ec0c29e6f57%40googlegroups.com.

[Oliver Rolle]

Do you think it makes sense to crowd source the implementation of these functions?

[Jordan Halterman]

I think we should wait at least until Vertigo's messaging abstractions are complete so we can get a better understanding of exactly what aggregations will look like. It should be quick to prototype at least and iterate.

To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/23205648-173d-4651-a7af-dd678135c1eb%40googlegroups.com.