Broadcasting

[Daniel Holden]

Would there be any interest in broadcasting for numeric? In numpy broadcasting is a fairly essential tool to avoid looping without duplicating data (http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html). It also makes data manipulation nice when used in combination with slicing and constructs such as "newaxis". Because numeric does not call out to native code, avoiding loops isn't so important, but having broadcasting can still neaten up code, and allow you to write what are essentially loops using simple expressions. Secondly, in the future, there might be a day when numeric does call out to something like asm.js or native code. In which case broadcasting is preferable to loops.

I've implemented a kind of prototype in a local copy of numeric and I'm confident it would not "change" numerics existing behaviour in a non-backward-compatible way (it just resolve some failure cases) but I'm not sure if it will be possible to implement broadcasting fully without either some small performance impact, or some larger re-writes to some of the core code base. If there is interest I'd be more than happy to work on some re-writes, but if broadcasting is seen as useless or impossible to make fast enough, then probably I will get more comfortable with just writing loops.

Secondly, but related, I'd also be interested in implementing an optional "axis" argument for the mapreducers so that you can "sum" or "max" over a given dimension. Would this interest people at all?

[Sébastien Loisel]

Hi,

Thanks for your email.

I know about broadcasting; at the time when I implemented everything I tried it but couldn't get the performance I wanted. If you manage to get the same performance (or beat what I have) I'd love to see it!

As for the optional axis argument: again if you can make it run as fast as what I have I'd love to see it!

Thanks,

--
You received this message because you are subscribed to the Google Groups "numericjs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to numericjs+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Sébastien Loisel

Assistant Professor

Department of Mathematics, Heriot-Watt University

Riccarton, EH14 4AS, United Kingdom
web: http://www.ma.hw.ac.uk/~loisel/
email: S.Loisel at hw.ac.uk
phone: +44 131 451 3234
fax: +44 131 451 3249

[Daniel Holden]

Ah, I figured performance might have been the issue! We'll I suspect you are right, and that the performance overhead might be unavoidable, but I'll give it a shot either way :)

[Owen Densmore]

Speaking of performance, does numeric work with typed arrays?

I mention this because our project has moved to TAs because they are both faster and way more memory efficient, and also open us up to use of the GPU for rendering the "model" the arrays represent.

It turns out that fast.js, which is based on avoiding JS "corner cases" (sparse arrays for one), has shown that JS is faster, yup, faster, than native.  Sure made me a believer of jsPerf!  :)

   -- Owen

[Daniel Holden]

I've made good progress on broadcasting. I currently have a version working without any performance deficits (in fact I think it may be fractionally faster due to some other refactorings). You can check it out on my github if you want to give it a try: https://github.com/orangeduck/numeric

Naturally it still requires a little more testing but it should be functional enough to give a good try.

Currently it supports broadcasting with the exception that the numer of dimensions must be equal. More specifically, unlike numpy, it will not right-align the dimensions of Arrays and fill in the left hand side with new axes. Therefore the user is responsible for manually adding any new axes required, so that the numer of dimensions match. In the future I'll look into this, but if it makes the code too difficult or messy it may be fine to leave this as it is.

I also need to adapt the "pointwise" function to allow for users to efficently create their own broadcastable pointwise functions. While I don't think this will break backward compatibility, there might be some changes required for new users to get broadcasting on their newly defined pointwise functions as it requires more than a single iterator value `i`.

I've also added a few new functions. This includes a "slice" function which is very similar to numpy's array access. You can provide it with values, ranges, and a newaxis object, for each dimension. I'm also interested in allowing indexing via boolean arrays or integer arrays such as numpy has, but I've not completed that yet.

My plan now is to look into the mapreducers and the optional "axis" argument. This one might be a bit tougher to get working.

- Dan

[Sébastien Loisel]

Yes, numeric works on typed arrays. It won't allocate them though so you'll have to use addeq et al.

--

You received this message because you are subscribed to the Google Groups "numericjs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to numericjs+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sébastien Loisel]

Sounds good. Does it pass the unit tests?

[Daniel Holden]

Hey,

With all the new additions all of the unit tests are passing other than the following:

240 FAIL sol=numeric.dopri(0,2,1,function(x,y){returny;},1e-8,100,function(x,y){returny-1.3;})

It seems like some of the changes I introduced might have introduced some slightly different numeric results. I don't know why but it could be optimisations or a whole host of other things. Either way, something is causing this test to act slightly differently and give slightly different numeric values. I'll take a good look into it at some point, although its likely I'll need some help to see exactly where it is going wrong because I'm not so familiar with the function.

There are new tests to go with (almost) all of the new functions I've defined and I've briefly updated the documentation but it will require another going over to make sure it is all clear.

I've added in mapreduction on a given axis and it seems to be working well. One consiquence is that if you want to do reduction on an axis you now need to use numeric operators in the body of the function instead of normal javascript operators (for example "numeric.addeq" instead of "+=").

For defining new pointwise functions you need to use some new iterators `k`, `i`, `j` for the return value, lhs value and rhs value if you want your function to work with broadcasting. So I don't think my changes will break backward compatibility providing broadcasting isn't used in old code, but giving new definitions does require things to be specified slightly differently if you want broadcasting to work with them. I think this is unavoidable in both cases.

The final thing I want to look at is indexing arrays via integer or boolean values. After that I will do a thorough going over of the code. Then if you like you can take a look over it and see if there is anything wrong or that you don't like.

Thanks,

Dan

[Sébastien Loisel]

Sounds good!

[Daniel Holden]

I've pushed up various other updates just now including masking with boolean arrays and indexing with integer arrays to my github.

I also changed the interface to the "pointwise" function so that it can generate fully polymorphic functions, now that broadcasting essentially handles scalar values this seemed okay. Previously it wasn't documented and I already broke the interface to "mapreduce".

I have a minor thought regarding min and max. In the current version of numeric (on the website) "min" and "max" are pointwise functions, while "inf" and "sup" are the mapreduce versions which take the minimum and maximum of some array. With the additional axis argument to the mapreduce functions, the mapreduce versions "inf" and "sup" become a lot more useful. You can even use them in a kind of pointwise way by putting the arguments in an Array and doing the min/max on axis 0. But I don't think their names are very good. Maybe I am missing something obvious but I don't think I "get" why they are called what they are.

In NumPy the pointwise version are called "minimum" and "maximum" and the mapreduce versions are called "min" and "max". This works nicely because it matches "argmin" and "argmax" which are for finding the minimum and maximum indices (I've implemented these in my branch). It is really minor, but I think the NumPy naming is nicer. Would you consider this re-naming even though it breaks backward compatibility?

- Dan

[Sébastien Loisel]

Hi Daniel,

Thanks for your email.

I love all improvements to everything you're doing. That being said, I'm a bit reluctant to introduce incompatibilities in the API unless they're really important. I'm not convinced that changing the names now is a net win.

Thanks,

[Daniel Holden]

No problem, completely understand!

[Daniel Holden]

Hey,

Okay I've pushed up a final version I'm happy with to github:

https://github.com/orangeduck/numeric

Perhaps you can take a look and help me out with fixing the final test that is failing. I still can't tell exactly what is wrong.

Thanks,

Dan

[Daniel Holden]

I managed to fix the final test that was failing. It turned out to be a fairly significant bug so I'm glad I spent the time to check it out properly. I've also added in several more functions and updated the documentation to make it a little easier to make use of the cheat sheet. I'm going to do a few more checks and then I will send a pull request.

Thanks,

Dan

[Sébastien Loisel]

OK!

[Yotam G]

Will these changes be integrated? I look forward to numericjs having broadcasting.

Yotam

[Sébastien Loisel]

Hi Yotam,

I know I have a backlog of things to do on numeric. I'll try to get
around to it. Real life is happening to me -- apart from my newborn
kid, I have a bunch of PhD students. Maybe I should get some of my
undergrads to help with numeric.

Thanks,