[Swig-user] typemap problem: can't get arrays example to work
Russell E. Owen
example for getting arrays in structs to work.
<http://www.swig.org/Doc2.0/Typemaps.html#Typemaps_nn40>
Unfortunately I can't get it to work. I hope it's just a simple user
error.
I made a minimal test case that attempts to wrap this simple struct
using the in, out and memberin typemaps in the typemap example mentioned
above.
*** swigtest.h ***
#pragma once
struct TestBlk {
float f[3];
};
The code is appended, and a complete copy, including a scons builder
file and python test file, is available here:
<http://www.astro.washington.edu/users/rowen/testswig_cfloat.zip>
What I find is that the "out" typemap works perfectly (I can retrieve f
from TestBlk) but the "in" and "memberin" typemaps are ignored -- the
swig wrapper acts identically whether those are present or absent, and
it's not pretty: If I attempt to set elements of TestBlk.f individually
(e.g. test.f[0] = 4.0) the command is accepted but ignored. If I attempt
to set f to a list of 3 floats, the command is rejected.
Any ideas?
(By the way, my need is C++ and fixed-length-arrays of double, bool and
int, but I figured I would get the simplest possible example working
first. I also will have some arrays of objects, and that will clearly
need a different approach.)
-- Russell
*** swigtest.c ***
#include "swigtest.h"
*** swigtest.i ***
%module(docstring="test array in struct") swigtestLib
%{
#include "swigtest.h"
%}
%init %{
%}
%typemap(in) float value[ANY] (float temp[$1_dim0]) {
int i;
if (!PySequence_Check($input)) {
PyErr_SetString(PyExc_ValueError,"Expected a sequence");
return NULL;
}
if (PySequence_Length($input) != $1_dim0) {
PyErr_SetString(PyExc_ValueError,"Size mismatch. Expected $1_dim0
elements");
return NULL;
}
for (i = 0; i < $1_dim0; i++) {
PyObject *o = PySequence_GetItem($input,i);
if (PyNumber_Check(o)) {
temp[i] = (float) PyFloat_AsDouble(o);
} else {
PyErr_SetString(PyExc_ValueError,"Sequence elements must be
numbers");
return NULL;
}
}
$1 = temp;
}
%typemap(memberin) float [ANY] {
int i;
for (i = 0; i < $1_dim0; i++) {
$1[i] = $input[i];
}
}
%typemap(out) float [ANY] {
int i;
$result = PyList_New($1_dim0);
for (i = 0; i < $1_dim0; i++) {
PyObject *o = PyFloat_FromDouble((double) $1[i]);
PyList_SetItem($result,i,o);
}
}
%include "swigtest.h"
<http://www.astro.washington.edu/users/rowen/testswig_cfloat.zip>
------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure
_______________________________________________
Swig-user mailing list
Swig...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/swig-user
Bob Hood
The swig 2.0 manual section "10.6.1 Typemaps for arrays" has a nice example for getting arrays in structs to work. <http://www.swig.org/Doc2.0/Typemaps.html#Typemaps_nn40> Unfortunately I can't get it to work. I hope it's just a simple user error. I made a minimal test case that attempts to wrap this simple struct using the in, out and memberin typemaps in the typemap example mentioned above. *** swigtest.h *** #pragma once struct TestBlk { float f[3]; }; The code is appended, and a complete copy, including a scons builder file and python test file, is available here: <http://www.astro.washington.edu/users/rowen/testswig_cfloat.zip> What I find is that the "out" typemap works perfectly (I can retrieve f from TestBlk) but the "in" and "memberin" typemaps are ignored -- the swig wrapper acts identically whether those are present or absent, and it's not pretty: If I attempt to set elements of TestBlk.f individually (e.g. test.f[0] = 4.0) the command is accepted but ignored. If I attempt to set f to a list of 3 floats, the command is rejected. Any ideas?
I recall running into this same sort of issue while wrapping Python for a major 3D animation package. I posted to the list about it, and was mildly surprised to discover that SWIG does not perform individual element accesses, but rather migrates whole arrays at once (you might be able to find it if you search back through the archives).
What this comes down to is the fact that "out" is going to behave as you expect with individual element access because the entire array is transferred to the target language environment with each access. This means that when you access your array using:
f0 = test.f[0]
Then the entire array (f[3]) will be copied to Python's stack, and then the single element [0] will be pulled from it and assigned to your variable ('f0'). So, the following code:
f0 = test.f[0]
f1 = test.f[1]
f2 = test.f[2]
would actually end up copying your entire array three times!
Knowing that, then it becomes clearer that the "out" version won't set an individual element into the array and pass it back to C++. Instead, it will copy the entire array locally, set the value into the specific element...and then just discard the array.
To improve efficiency, I had to resort (in the Python examples I constructed) to setting up cases where exposed structure array members are pulled down to the Python environment locally in their entirety until the user is done working with them, and then providing a means for passing the modified array back to C++ in a single, whole step.
AFAIK, this is still the case with SWIG (2.0.4).
Render me gone, |||
Bob ^(===)^
---------------------------------oOO--(_)--OOo---------------------------------
I'm not so good with advice...can I interest you in a sarcastic comment?
Russell E. Owen
Bob Hood <bho...@comcast.net> wrote:
> expect with individual element access because /the entire array is
> transferred
> to the target language environment with each access/. This means that when
> you access your array using:
>
> f0 = test.f[0]
>
> Then the entire array (f[3]) will be copied to Python's stack, and then the
> single element [0] will be pulled from it and assigned to your variable
> ('f0'). So, the following code:
>
> f0 = test.f[0]
> f1 = test.f[1]
> f2 = test.f[2]
>
> would actually end up copying your entire array three times!
>
> Knowing that, then it becomes clearer that the "out" version won't set an
> individual element into the array and pass it back to C++. Instead, it will
> copy the entire array locally, set the value into the specific element...and
> then just discard the array.
>
> To improve efficiency, I had to resort (in the Python examples I constructed)
> to setting up cases where exposed structure array members are pulled down to
> the Python environment locally in their entirety until the user is done
> working with them, and then providing a means for passing the modified array
> back to C++ in a single, whole step.
>
> AFAIK, this is still the case with SWIG (2.0.4).
That sounds pretty bad. Your explanation of why setting a single value
fails certainly makes sense. Since I cannot set all values at once,
either, the object is read only. I'm very puzzled why the documented
method simply doesn't work for me -- I'm not sure if the documentation
is wrong or I'm doing something wrong. (I don't mind a bit of extra
copying).
I've decided to switch to STL arrays.
I'm hoping somebody will eventually wrap std::array (much like
std_vector.i). Meanwhile I'll use the appended hack, adapted from
<http://stackoverflow.com/questions/7713318/nested-structure-array-access
-in-python-using-swig>. It has some unfortunate limitations: no way to
set all values at once, no range indexing, no relative indexing. But it
works and it certainly avoids excessive copying.
-- Russell
*** swigtest.h ***
template <typename Type, size_t N>
struct wrapped_array {
boost::array<Type, N> data; // or use std::array in modern C++
};
struct TestBlk {
wrapped_array<double, 3> d;
};
*** swigtestLib.i ***
%module(docstring="test array in struct") swigtestLib
%{
#include <iostream>
#include "boost/array.hpp"
#include "swigtest.h"
%}
%init %{
%}
%include "std_except.i"
%include "swigtest.h"
%extend wrapped_array {
inline size_t __len__() const { return N; }
inline const Type& __getitem__(size_t i) const
throw(std::out_of_range) {
return self->data.at(i);
}
inline void __setitem__(size_t i, const Type& v)
throw(std::out_of_range) {
self->data.at(i) = v;
}
}
%template (doubleArray3) wrapped_array<double, 3>;
Russell E. Owen
"Russell E. Owen" <ro...@uw.edu> wrote:
> I'm hoping somebody will eventually wrap std::array (much like
> std_vector.i). Meanwhile I'll use the appended hack, adapted from
> <http://stackoverflow.com/questions/7713318/nested-structure-array-access
> -in-python-using-swig>. It has some unfortunate limitations: no way to
> set all values at once, no range indexing, no relative indexing. But it
> works and it certainly avoids excessive copying.
I was able to overcome these problems using the appended .i excerpt. The
resulting Python is as clean as I could hope for. One can do things like:
test.d[:] = (1, 2, 3)
atuple = test.d[:]
test.d[0] = 1
test.d[-1] = 3.0
...
One cannot do this: test.d = (1, 2, 3) but I am fine with that because
one cannot rebind test.d in any case.
The C++ side is not ideal because of the array data is one level down
from where it should be: in my example test.d.arr instead of test.d.
I tried wrapping boost::array directly to make the C++ look normal, but
unsurprisingly got an enormous number of errors -- enough to convince me
to live with my simple solution for now. I hope somebody more familiar
with SWIG will be able to figure out how to wrap std::array (the modern
version of boost::array).
-- Russell
%extend tcc::swigArray {
inline size_t __len__() const { return N; }
inline const Type& _get(size_t i) const throw(std::out_of_range) {
return $self->arr.at(i);
}
inline void _set(size_t i, const Type& v) throw(std::out_of_range) {
$self->arr.at(i) = v;
}
%pythoncode {
def __getitem__(self, key):
if isinstance(key, slice):
return tuple(self._get(i) for i in
range(*key.indices(len(self))))
if key < 0:
key += len(self)
return self._get(key)
def __setitem__(self, key, v):
if isinstance(key, slice):
for i in range(*key.indices(len(self))):
self._set(i, v[i])
else:
if key < 0:
key += len(self)
self._set(key, v)
}
}
%template (swigArray_int_2) tcc::swigArray<int, 2>;
%template (swigArray_int_3) tcc::swigArray<int, 3>;
%template (swigArray_double_2) tcc::swigArray<double, 2>;
%template (swigArray_double_3) tcc::swigArray<double, 3>;