make function return fw_ret_arg instead of (fw_ret_arg,)
Ondrej Certik
for fortran functions, why do they return (fw_ret_arg,) instead of
fw_ret_arg? It seems to me that it make sense for them to return the
number (object, whatever it is) directly. Isn't it?
E.g. change this:
cpdef api object func(fwi_integer_t a):
"""
func(a) -> (fw_ret_arg,)
Parameters
----------
a : fwi_integer, intent in
Returns
-------
fw_ret_arg : fwi_integer, intent out
"""
cdef fwi_integer_t fw_ret_arg
cdef fwi_integer_t fw_iserr__
cdef fw_character_t fw_errstr__[fw_errstr_len]
func_c(&fw_ret_arg, &a, &fw_iserr__, fw_errstr__)
if fw_iserr__ != FW_NO_ERR__:
raise RuntimeError("an error was encountered when calling the
'func' wrapper.")
return (fw_ret_arg,)
to this:
...
return fw_ret_arg
Then I thought --- well, it'd make sense to return fw_ret_arg whenever
there is just a list of length 1, so for example this patch fixes it:
diff -r e1bd3c1c4204 fwrap/cy_wrap.py
--- a/fwrap/cy_wrap.py Mon Aug 09 16:25:35 2010 -0500
+++ b/fwrap/cy_wrap.py Mon Aug 09 22:30:33 2010 -0700
@@ -585,8 +585,10 @@
def return_tuple(self):
ret_arg_list = []
ret_arg_list.extend(self.arg_mgr.return_tuple_list())
- if ret_arg_list:
+ if len(ret_arg_list) > 1:
return "return (%s,)" % ", ".join(ret_arg_list)
+ elif len(ret_arg_list) == 1:
+ return "return %s" % ret_arg_list[0]
else:
return ''
of course, some tests have to be adapted now (9 fail), but the changes
in tests seem trivial. What do you think of this?
I used to use f2py, and without this patch, I would have to change all
my python code from
S = integrate(x, f)
to
S, = integrate(x, f)
which seems to me like a step back.
Ondrej
Kurt Smith
> Hi,
>
> for fortran functions, why do they return (fw_ret_arg,) instead of
> fw_ret_arg? It seems to me that it make sense for them to return the
> number (object, whatever it is) directly. Isn't it?
The existing behavior is simply for uniformity: everything with
something to return returns a tuple; a wrapped function's return value
is the first element of the tuple. This lets us treat functions &
subroutines exactly the same way, no matter the number of arguments or
return values ("special cases aren't special enough to break the
rules" and all that).
That being said, I agree that your patch makes a certain sense so long
as we can figure out what to do with subroutines. Let's say we have a
subroutine with a huge argument list, every argument is intent(in)
except for one, which is intent(out). This patch will effectively
turn this subroutine into a function at the Python-level.
Would that be confusing? Perhaps not since it would be documented.
What would the common expectation be?
It seems that the expectation is slightly different between functions
& subroutines. If you have a fortran function that has all intent(in)
arguments, you *should* be able to do:
x = func(y, z, alpha, beta)
and have it work right, like in your example.
I guess it comes down to this question: how important is it to keep
the function/subroutine distinction in Fortran at the Python-level?
One possibility would be to special case *functions* for this return
behavior, and leave subroutines as-is. I'll need to think about this
a bit. Let me know your thoughts.
Kurt
> --
> You received this message because you are subscribed to the Google Groups "Fwrap Users" group.
> To post to this group, send email to fwrap...@googlegroups.com.
> To unsubscribe from this group, send email to fwrap-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/fwrap-users?hl=en.
>
>
Dag Sverre Seljebotn
1-length tuples. In my own code I tend to progress naturally from
x = f()
to
x, y = f()
once I need more than one return value. To me, tuples are just the way
multiple return values are transported...
The only time I'd ever do
x, = f()
is if f() returns a _list_, and in this special case is known to return
one element. I.e. the number of return values from the *same function*
varies. However, different arguments are definitely tuple-stuff, not
homogenous lists... and the number of return values stays fixed per
function.
Dag Sverre
Kurt Smith
Not to appeal to the Zen as though it's some unquestionable oracle,
but I guess you could say that "flat is better than nested" would
argue against length-one tuples :-)
Good points.
Kurt
Kurt Smith
This reminds me: I was considering supporting named tuples as the
return object in a future release, but only if it isn't a ridiculous
performance hit and it turns out to improve usability (I've not tried
them out very much).
http://docs.python.org/library/collections.html#collections.namedtuple
Seems like they have all the benefits of dictionaries and tuples in
one return object; only downside is speed issues.
If we could get fast(er) named tuples implemented in Cython that might do it.
Ondrej Certik
> On Tue, Aug 10, 2010 at 12:33 AM, Ondrej Certik <ond...@certik.cz> wrote:
>> Hi,
>>
>> for fortran functions, why do they return (fw_ret_arg,) instead of
>> fw_ret_arg? It seems to me that it make sense for them to return the
>> number (object, whatever it is) directly. Isn't it?
>
> The existing behavior is simply for uniformity: everything with
> something to return returns a tuple; a wrapped function's return value
> is the first element of the tuple. This lets us treat functions &
> subroutines exactly the same way, no matter the number of arguments or
> return values ("special cases aren't special enough to break the
> rules" and all that).
>
> That being said, I agree that your patch makes a certain sense so long
> as we can figure out what to do with subroutines. Let's say we have a
As a first iteration, you can disable this for subroutines, just keep
it for functions. That's all I need.
> subroutine with a huge argument list, every argument is intent(in)
> except for one, which is intent(out). This patch will effectively
> turn this subroutine into a function at the Python-level.
>
> Would that be confusing? Perhaps not since it would be documented.
> What would the common expectation be?
>
> It seems that the expectation is slightly different between functions
> & subroutines. If you have a fortran function that has all intent(in)
> arguments, you *should* be able to do:
>
> x = func(y, z, alpha, beta)
>
> and have it work right, like in your example.
Exactly, that's what I am talking about. I couldn't easily figure out
how to test if it's a function or subroutine, but I guess it would be
trivial for you to improve my patch.
>
> I guess it comes down to this question: how important is it to keep
> the function/subroutine distinction in Fortran at the Python-level?
>
> One possibility would be to special case *functions* for this return
> behavior, and leave subroutines as-is. I'll need to think about this
> a bit. Let me know your thoughts.
Definitely. Let's start with that and later we can make it work for
subroutines too if needed.
Another related issue with subroutines is that if some arguments are
intent(out), you still need to specify them as arguments in Python.
f2py didn't have this behavior. There is an issue what to do with
intent(out) arrays, as then the wrappers would have to allocate the
array, while the current approach leaves this to the user to do at
the Python level, which makes things more clear and I like this.
I am currently not sure what happens if you do intent(out) arrays for
functions, both in fortran and fwrap, as then you also hit the
problem.
Ondrej
Dag Sverre Seljebotn
> Another related issue with subroutines is that if some arguments are
> intent(out), you still need to specify them as arguments in Python.
> f2py didn't have this behavior. There is an issue what to do with
> intent(out) arrays, as then the wrappers would have to allocate the
> array, while the current approach leaves this to the user to do at
> the Python level, which makes things more clear and I like this.
>
> I am currently not sure what happens if you do intent(out) arrays for
> functions, both in fortran and fwrap, as then you also hit the
> problem.
>
- take an "out" argument
- if out=None, allocate the array, otherwise check that shape/strides
are OK
- no matter what, also return it
BTW, the ability to set any intent(out)-array to None to have them
auto-allocated, or pass them in to use existing buffers, is something
I'd like in general. I didn't check out fwrap yet so I don't know
whether that's supported.
The only problem is with functions with arguments named "out". I think
that is highly unlikely and am fine with a NotImplementedError there...
Dag Sverre
Dag Sverre Seljebotn
> Ondrej Certik wrote:
>> Another related issue with subroutines is that if some arguments are
>> intent(out), you still need to specify them as arguments in Python.
>> f2py didn't have this behavior. There is an issue what to do with
>> intent(out) arrays, as then the wrappers would have to allocate the
>> array, while the current approach leaves this to the user to do at
>> the Python level, which makes things more clear and I like this.
>>
>> I am currently not sure what happens if you do intent(out) arrays for
>> functions, both in fortran and fwrap, as then you also hit the
>> problem.
>>
> One could follow NumPy and have an out argument. I.e.:
>
> - take an "out" argument
> - if out=None, allocate the array, otherwise check that shape/strides
> are OK
> - no matter what, also return it
>
> BTW, the ability to set any intent(out)-array to None to have them
> auto-allocated, or pass them in to use existing buffers, is something
> I'd like in general. I didn't check out fwrap yet so I don't know
> whether that's supported.
meaning auto-allocation of intent(out)-arrays some day, though I realize
there will be other, higher priorities :-)
Dag Sverre
Kurt Smith
Of course :-) I'm leaning towards just applying your patch as-is for
both functions & subroutines. Thanks for the help.
>
>>
>> I guess it comes down to this question: how important is it to keep
>> the function/subroutine distinction in Fortran at the Python-level?
>>
>> One possibility would be to special case *functions* for this return
>> behavior, and leave subroutines as-is. I'll need to think about this
>> a bit. Let me know your thoughts.
>
>
> Definitely. Let's start with that and later we can make it work for
> subroutines too if needed.
>
>
>
> Another related issue with subroutines is that if some arguments are
> intent(out), you still need to specify them as arguments in Python.
> f2py didn't have this behavior. There is an issue what to do with
> intent(out) arrays, as then the wrappers would have to allocate the
> array, while the current approach leaves this to the user to do at
> the Python level, which makes things more clear and I like this.
intent(out) *scalars* are not in the argument list; they're returned
in the output tuple. If this is not the case then please submit a bug
report.
The one exception is intent(out) character arguments with len=*; the
wrappers need to figure out the actual argument length so you have to
pass in a dummy string. The contents don't actually matter. There's
probably a better way to handle this case.
I'm in the process of drawing up basic documentation detailing all of
this. Character arguments
character(len=20) arg
are Python strings at the python level. This means that intent(inout)
character arguments are copy-in and a new string is created for the
output to preserve string immutability.
Character arrays:
character(len=20), dimension(:, :) :: arg
are Numpy string arrays at the python level:
np.empty((10, 10), dtype='S20')
and are mutable (no copy-in/copy-out required).
If no intent is specified (for array or scalar argument), it's
analogous to intent(inout) and the argument must be supplied and will
be in the output tuple. f2py would assume that no intent ->
intent(in) in contrast to fwrap, and which breaks intent(inout) scalar
arguments, since you couldn't get it back.
>
> I am currently not sure what happens if you do intent(out) arrays for
> functions, both in fortran and fwrap, as then you also hit the
> problem.
It's just like a subroutine: the argument can be modified by the
function (both in fortran & fwrap) and is in the arugment list and
output tuple (for fwrap).
>
> Ondrej
>
Kurt Smith
+1. This is already in the great-big-design-document in my head,
which needs to be dumped to the wiki :-)
The only issue is for assumed-size and assumed-shape arrays: how do
you nicely tell the wrapper these array extents? This will probably
require something in an interface file, like "arg.default_shape = (10,
20)" Other suggestions?
Kurt
>
> Dag Sverre
>
Dag Sverre Seljebotn
arguments and/or environment, "set_resolution"-style functions called
earlier, etc....
I'd rather work on encouraging people to modify the pyx wrapper in cases
like this. fwrapc could even invoke "git init" in the wrapper dir by
default and do its changes in an "fwrap-generated" branch, bootstrapping
you into the mode of using git to merge fwrap-generated stuff and your
own wrapper stuff.
I think this is much more flexible than f2py-style comments, custom
config files and so on.
Dag Sverre
Dag Sverre Seljebotn
Note that this way it shouldn't be hard to support an '--update': Simply
switch branch to "fwrap-generated", wipe out everything, regenerate
wrapper, commit, and git/mercurial will figure out the rest when the
user merges into "master" (git-style workflow, mercurial is different I
guess).
Some stable order of wrapper functions strictly needed though.
Alphabetical by function name or (alphabetical Fortran source file, then
order in source file), perhaps.
(Me and Kurt talked about this earlier, but figured it should hit the list.)
Dag Sverre
Kurt Smith
The "(10, 20)" was just a placeholder of course; it could be any
arbitrary expression (a question of which wrapper level to put it on
arises, of course).
You could do arg.default_shape = "shape(other_arg)" and it would be
computed at runtime appropriately.
>
> I'd rather work on encouraging people to modify the pyx wrapper in cases
> like this. fwrapc could even invoke "git init" in the wrapper dir by default
> and do its changes in an "fwrap-generated" branch, bootstrapping you into
> the mode of using git to merge fwrap-generated stuff and your own wrapper
> stuff.
I realize we talked about this a long while ago, but I'm less and less
confident that this will work well. The pyx wrappers aren't
particularly easy to modify unless you know what's going on--name
mangling, exception handling, etc obfuscate things a lot, and I'm not
looking forward to the ugly bug reports that would result when things
don't work right. It would be similar to encouraging a Cython user to
modify the generated .c file which is not recommended.
My thinking for the interface file approach is to have the interface
file be a python script. No yet-another-interface-language to learn
(just a pretty simple API to learn instead) and there wouldn't be
arbitrary restrictions. Whatever fwrap does to generate the wrappers
the user would have access to. Fwrap can dump the default interface
.py file and the user can modify that.
Kurt Smith
Pushed, with some changes to get all tests to pass, and the docstrings
were changed to reflect the modified signature.
http://bitbucket.org/kwmsmith/fwrap-dev/changeset/bcab0ef9f65d
>
> I used to use f2py, and without this patch, I would have to change all
> my python code from
>
> S = integrate(x, f)
>
> to
>
> S, = integrate(x, f)
>
> which seems to me like a step back.
>
> Ondrej
>