Integrating Binary Data and PJS: multiple results

[Herhut, Stephan A]

After some sleep, here is a continuation. In the meanwhile, Rick already mentioned flatten/partition and I agree that enabling (even a limited form) of a reinterpret cast is the best way to go here. Adding structure to otherwise flat data (think canvas) is very useful outside of parallel programming, too.

Another pattern that I think binary data could help allow is to enable multiple results from a single parallel operation. Such support gives the programmer (or a fancy runtime library on top of PJS) more flexibility to schedule work. An example I have come across is computing tangent and normal vectors for some mesh data. Both are computed from essentially the same input but the result of the computation is used as independent arrays in the surrounding code. Of course, one could compute both separately but this brings twice the overhead of parallel execution and leaves some nice data locality unused.

I have proposed this earlier in [1] using an unzip method. The idea was to compute the data like this

function foo (x) {
return {odd: (x%2), twice: 2*x};
}
R = [1,2,3,4].map(foo);

which creates an array of objects and in a second step one would call an unzip method on R, which does an array-of-structures (AoS) to structure-of-arrays (SoA) transformation. Of course, ideally the runtime would produce the SoA version in the first place to avoid the copy. Now, from previous discussions, I have learnt that such implicit magic seems not to resonate well with the JS engine reality, so I was looking for a way to make this more explicit. Looking at the binary data version of map, I had the following idea. How about writing

A = ... // assume A is a binary data version of [1,2,3,4] with type ArrayType(float64, 4)
R = A.map(function (x, out1, out2) { out1 = x%2; out2 = 2*x; }, float64, float64)

In the above example, I provide two type specifications to map, which in turn triggers two out pointer being passed to the elemental function. Map would produce two values of type ArrayType(float64, 4) and return them as an array (in lack of tuples). With destructuring, one could write

[R1, R2] = A.map(function (x, out1, out2) { out1 = x%2; out2 = 2*x; }, float64, float64)

Of course, I would like pbuild to have similar support. When using a map style API for pbuild this is straight forward. However, I have no good idea yet of how to do this for the version you proposed in your blog post.

Stephan

[1] https://mail.mozilla.org/pipermail/es-discuss/2012-May/022828.html