I've been looking at using shedskin to translate some code using numpy.  In
this particular example, there is set of particles (in a 2d array) and a
simple computation related to the distance between particles.

There seem to be two issues with infering the type correctly:
 - the 'numpy.array' constructor can be used to create 1d or 2d arrays (or
higher dimensions, but not in this example).  Shedskin can't determine the
correct array type (even though it should have the type information it
needs).
 - slices of 2d arrays can either be a 1d array (single index - a[1]) or a
scalar (index is a tuple - a[1,2])

More generally, it seems there should be some way of conveying this
decision information to shedskin (the stub files don't necessarily work for
the case where the output type of a function requires a decision based on
the input type (?)).

Or have I done something wrong with the numpy stub file?  Both the example
and the numpy stub file are attached.

Mark

ex1.py < 1K Download

numpy.py 1K Download

hi mark,

indexing and slicing indeed seem the most difficult to support, as the
result in fact depends on both array rank and length of the index tuple..

I'm afraid 'isinstance' and 'len' don't work as expected during the type
inference that shedskin performs, so they can't be used to model this
behaviour.. :P shedskin currently doesn't track the length of things in any
way, which in general even seems impossible (comparable to the halting
problem), and 'isinstance' is in fact not supported at all anymore.
needless to say type inference is very nasty business.

but I guess the 'rank' of the data that goes into array() can be usually
known by shedskin. but we'd have to track things when combining arrays
further probably.. the index tuple forms an even more fundamental problem
perhaps, since we can't know its length beforehand in all cases.
restricting tuples to always be specified by its elements would make the
problem easier, as it would allow shedskin to insert the correct types
during analysis.

I'm sure we'd find other similar problems when trying to add direct support
for numpy. in short, I don't think we want to try and support it directly.
it's quite big and dynamic (in a way that guido himself dislikes - types
being dependent on non-type data essentially). we obviously can't support
every library out there, so perhaps it makes limit ourselves to the python
standard library. other libraries can also often be combined with a
shedskin-compiled extension module (as happens with for example pygame and
pygtk in shedskin/examples).

an alternative approach to integrate with numpy without the performance
loss of having to convert things, and that would probably also be useful in
other areas (the regular 'array' module would be one example) would be to
figure out a simple way to share memory between cpython and shedskin. I've
thought about this a few times, but have not come upon a satisfactory
solution yet..

thanks in any case for having me read through the numpy tutorial.. ;-)
mark.