Implementing Pipelines in Go programming language

[Ronoaldo Pereira]

Hi all!

App Engine Pipelines and MapReduce is a somewhat must-have tool in order to build some more complex workflows in Google App Engine task queues, specially if you are designing for scaling your jobs across multiple parallel computation, from multiple concurrent users. 

I would like to check with the list if there is planed support for something like this library for the Go language, or if there is any documentation on how to properly implement fan-in/fan-out for Go on App Engine.

Currently, I am using just task queues, but it is getting harder and harder to acomplish some coordination (fan-out specially) in a way that does not implodes the universe entirelly in my hands.

I started to draft something (no code yet), but I'm not yet done with something that would even look good. The issue I see for Go is that you can't do things dynamically as you can in Python (decoding parameters), nor you can build generics like in Java, so the pipeline signatures would end up looking like Run(interface{}...), with potential type errors on the fly. Either way, I would love to help porting, if someone already familiar with the project could guide me in the first design steps.

Thanks!