[Numpy-discussion] Using logical function on more than 2 arrays, availability of a "between" function ?
Matthieu Rigal
I have actually a double question, which only aims to answer a single one :
how to get the following line being processed more efficiently :
array = numpy.logical_and(numpy.logical_and(aBlueChannel < 1.0, aNirChannel >
(aBlueChannel * 1.0)), aNirChannel < (aBlueChannel * 1.8))
One possibility would have been to have the logical_and being able to handle
more than two arrays
Another one would have been to be able to make a "double comparison" or a
"between", like following one :
array = numpy.logical_and((aBlueChannel < 1.0), (1.0 <
aNirChannel/aBlueChannel < 1.8))
Is there any way to get the things work this way ? Would it else be a possible
improvement for 1.7 or a later version ?
Best Regards,
Matthieu Rigal
RapidEye AG
Molkenmarkt 30
14776 Brandenburg an der Havel
Germany
Follow us on Twitter! www.twitter.com/rapideye_ag
Head Office/Sitz der Gesellschaft: Brandenburg an der Havel
Management Board/Vorstand: Ryan Johnson
Chairman of Supervisory Board/Vorsitzender des Aufsichtsrates:
Robert Johnson
Commercial Register/Handelsregister Potsdam HRB 24742 P
Tax Number/Steuernummer: 048/100/00053
VAT-Ident-Number/Ust.-ID: DE 199331235
DIN EN ISO 9001 certified
_______________________________________________
NumPy-Discussion mailing list
NumPy-Di...@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Richard Hattersley
faster? Or using less memory? Or have more concise source code?
Less memory:
- numpy.vectorize would let you get to the end result without any
intermediate arrays but will be slow.
- Using the "out" parameter of numpy.logical_and will let you avoid
one of the intermediate arrays.
More speed?:
Perhaps putting all three boolean temporary results into a single
boolean array (using the "out" parameter of numpy.greater, etc) and
using numpy.all might benefit from logical short-circuiting.
And watch out for divide-by-zero from "aNirChannel/aBlueChannel".
Regards,
Richard Hattersley
Matthieu Rigal
Well, actually I made a mistake lower... that you may have noticed...
On the faster (your) solution, even with a cleaner use of the out parameter,
the fact that the all has then to be used with parameter axis=0 takes more
time and makes it actually slower than the initial solution...
So I may go for the "multiplier" solution.
Regards,
Matthieu
On Tuesday 20 March 2012 13:13:15 you wrote:
> Hi Richard,
>
> Thanks for your answer and the related help !
>
> In fact, I was hoping to have a less memory and more speed solution.
> Something equivalent to a "raster calculator" for numpy. Wouldn't it make
> sense to have some optimized function to work on more than 2 arrays for
> numpy anyway ?
>
> At the end, I am rather interested by more speed.
>
> I tried first a code-sparing version :
> array = numpy.asarray([(aBlueChannel < 1.0),(aNirChannel > aBlueChannel *
> 1.0),(aNirChannel < aBlueChannel * 1.8)]).all()
>
> But this one is at the end more than 2 times slower than :
> array1 = numpy.empty([3,6566,6682], dtype=numpy.bool)
> numpy.less(aBlueChannel, 1.0, out=array1[0])
> numpy.greater(aNirChannel, (aBlueChannel * 1.0), out=array1[1])
> numpy.less(aNirChannel, (aBlueChannel * 1.8), out=array1[2])
> array = array1.all()
>
> (and this solution is about 30% faster than the original one)
>
> I could find another way which was fine for me too:
> array = (aBlueChannel < 1.0) * (aNirChannel > (aBlueChannel * 1.0)) *
> (aNirChannel < (aBlueChannel * 1.8))
>
> But this one is only 5-10% faster than the original solution, even if
> probably using less memory than the 2 previous ones. (same was possible
> with operator +, but slower than operator *)
>
> Regards,
> Matthieu Rigal
>
> On Monday 19 March 2012 18:00:02 numpy-discus...@scipy.org wrote:
> > Message: 2
> > Date: Mon, 19 Mar 2012 13:20:23 +0000
> > From: Richard Hattersley <rhatt...@gmail.com>
> > Subject: Re: [Numpy-discussion] Using logical function on more than 2
> > arrays, availability of a "between" function ?
> > To: Discussion of Numerical Python <numpy-di...@scipy.org>
> > Message-ID:
> >
> > <CAP=RS9=UBOc6Kmtmnne7W093t19w=T=oSrXUAW0W...@mail.gmail.com
> >
> > > Content-Type: text/plain; charset=ISO-8859-1
> >
> > What do you mean by "efficient"? Are you trying to get it execute
> > faster? Or using less memory? Or have more concise source code?
> >
> > Less memory:
> > - numpy.vectorize would let you get to the end result without any
> > intermediate arrays but will be slow.
> > - Using the "out" parameter of numpy.logical_and will let you avoid
> > one of the intermediate arrays.
> >
> > More speed?:
> > Perhaps putting all three boolean temporary results into a single
> > boolean array (using the "out" parameter of numpy.greater, etc) and
> > using numpy.all might benefit from logical short-circuiting.
> >
> > And watch out for divide-by-zero from "aNirChannel/aBlueChannel".
> >
> > Regards,
> > Richard Hattersley
>
RapidEye AG
Frédéric Bastien
make them faster.
[1] http://deeplearning.net/software/theano/
[2] https://code.google.com/p/numexpr/
Fred
Pierre Haessig
I have an off topic but somehow related question :
Le 19/03/2012 12:04, Matthieu Rigal a écrit :
> array = numpy.logical_and(numpy.logical_and(aBlueChannel < 1.0, aNirChannel >
> (aBlueChannel * 1.0)), aNirChannel < (aBlueChannel * 1.8))
Is there any significant difference between :
z = np.logical_and(x,y) and
z= x & y (assuming x and y are already numpy arrays and not just list)
I've always used the & (and | and ~) operator because it's of course
much shorter ;-)
I've seen no mention of the "&" operator in np.logical_and docstring so
I wonder...
Best,
Pierre
Eric Firing
> Hi,
>
> I have an off topic but somehow related question :
>
> Le 19/03/2012 12:04, Matthieu Rigal a écrit :
>> array = numpy.logical_and(numpy.logical_and(aBlueChannel< 1.0, aNirChannel>
>> (aBlueChannel * 1.0)), aNirChannel< (aBlueChannel * 1.8))
> Is there any significant difference between :
>
> z = np.logical_and(x,y) and
>
> I've always used the& (and | and ~) operator because it's of course
> much shorter ;-)
>
> I've seen no mention of the "&" operator in np.logical_and docstring so
> I wonder...
There is a big difference: &, |, and ~ are bitwise operators, not
logical operators, so they work like logical operators only if operating
on booleans (or at least arrays containing nothing but integer zeros and
ones) and only if you bear in mind that & and | have lower precedence
than their logical counterparts. Therefore you often need to use more
parentheses than you might have expected.
In [1]: a = np.array([1])
In [2]: b = np.array([2])
In [5]: np.logical_and(a,b)
Out[5]: array([ True], dtype=bool)
In [6]: a & b
Out[6]: array([0])
Using the bitwise operators in place of logical operators is a hack to
get around limitations of the language; but, if done carefully, it is a
useful one.
Eric
>
> Best,
> Pierre
Pierre Haessig
Thanks for the hints !
Le 25/03/2012 20:33, Eric Firing a écrit :
> Using the bitwise operators in place of logical operators is a hack to
> get around limitations of the language; but, if done carefully, it is a
> useful one.
What is the rationale behind not overloading __and__ & other logical
operations ?
Is it a requirement that boolean operators should always return *a bool*
and not an *array of bools* ?
--
Pierre
Eric Firing
> Hi Eric,
>
> Thanks for the hints !
>
> Le 25/03/2012 20:33, Eric Firing a écrit :
>> Using the bitwise operators in place of logical operators is a hack to
>> get around limitations of the language; but, if done carefully, it is a
>> useful one.
> operations ?
> Is it a requirement that boolean operators should always return *a bool*
> and not an *array of bools* ?
Pierre,
See http://www.python.org/dev/peps/pep-0335/
Eric