Timing comparisons between mahotas & scikits-image

[Luis Pedro Coelho]

Hello,

In the context of my previous email to pythonvision

https://groups.google.com/d/topic/pythonvision/SSCF4LUlRKQ/discussion

I made a timing comparison between mahotas & scikits-image.

operation | mahotas | pymorph | skimage
------------+----------+----------+---------
erode | 10.80 | 14.33 | 80.17
dilate | 11.44 | 8.93 | 41.59
open | 22.45 | 23.20 | 80.18
center mass | 7.05 | NA | NA
sobel | 75.03 | NA | 105.72
cwatershed | 201.03 | 56586.50 | 290.41
daubechies | 19.05 | NA | NA
haralick | 306.48 | NA | 7391.37

(Best viewed with fixed-width fonts)

The unit of measurement is the time it takes to run ``numpy.max(image)``
Mahotas is always faster than skimage (although pymorph is better for certain
morphological basic operations). I used GCLM in skimage to stand for Haralick,
which is a rough approximation.

I attach the script that generates these (against github skimage). If you
think that I have used skimage incorrectly, please let me know.

Best,
--
Luis Pedro Coelho | Institute for Molecular Medicine | http://luispedro.org

[Anthony Oliver]

Very impressive.   Would it be possible to integrate numba as well with mahotas?  I haven't tested it much but seems to get quite a bit of speed up using it.

https://github.com/numba/numba

You should put the speed test results on a website or blog somewhere.  I checked pythonvision.org but it looks like it is broken at the moment.

CTO

Sight Machine

Anthony Oliver

www.sightmachine.com

906-289-8169

On Thu, Aug 30, 2012 at 1:44 PM, Anthony Oliver <ant...@sightmachine.com> wrote:

Very impressive.   Would it be possible to integrate numba as well with mahotas?  I haven't tested it much but seems to get quite a bit of speed up using it.

https://github.com/numba/numba

You should put the speed test results on a website or blog somewhere.  I checked pythonvision.org but it looks like it is broken at the moment.

CTO

Sight Machine

Anthony Oliver

www.sightmachine.com

906-289-8169

[Luis Pedro Coelho]

I have not yet tried numba. I tend to be a bit behind the curve on these
things.

I updated the table with openCV, which is ridiculously fast, but only
supports C-Arrays (otherwise, you get an interpreter crash). Check Table
1 on page 7 on the new version of the paper (attached).

Stéfan: I have not yet fixed your reference, but I will have to redo all
of the references for the final manuscript.

Best,
Luis

PS: Since updating that server, I have had problems with configuration
on pythonvision.org. I will figure it out when I decide to lose a few
hours on nginx/apache configuration.

On 08/30/2012 06:46 PM, Anthony Oliver wrote:
> Very impressive. Would it be possible to integrate numba as well with
> mahotas? I haven't tested it much but seems to get quite a bit of speed
> up using it.
>
> https://github.com/numba/numba
>
>
> You should put the speed test results on a website or blog somewhere. I

> checked pythonvision.org <http://pythonvision.org/> but it looks like it

> is broken at the moment.
>
>
> CTO
> Sight Machine
> Anthony Oliver

> www.sightmachine.com <http://www.sightmachine.com>

> 906-289-8169
>
>
>
> On Thu, Aug 30, 2012 at 1:44 PM, Anthony Oliver
> <ant...@sightmachine.com <mailto:ant...@sightmachine.com>> wrote:
>
> Very impressive. Would it be possible to integrate numba as well
> with mahotas? I haven't tested it much but seems to get quite a bit
> of speed up using it.
>
> https://github.com/numba/numba
>
>
> You should put the speed test results on a website or blog

> somewhere. I checked pythonvision.org <http://pythonvision.org> but

> it looks like it is broken at the moment.
>
>
> CTO
> Sight Machine
> Anthony Oliver

> www.sightmachine.com <http://www.sightmachine.com>
> 906-289-8169 <tel:906-289-8169>

[Luis Pedro Coelho]

One good thing about benchmarks is that I always end up improving my code :)

Here is the new table with openCV and github master version of Mahotas (10~20%
faster on morphological operations):

operation | mahotas | pymorph| skimage | OpenCV
------------+----------+----------+----------+---------
erode | 10.3 | 14.2 | 41.5 | 0.4
dilate | 12.7 | 8.9 | 38.9 | 0.4
open | 23.0 | 23.3 | 80.8 | NA
center mass | 7.1 | NA | NA | NA
sobel | 75.4 | NA | 105.4 | 6.2
cwatershed | 191.3 | 60410.0 | 285.0 | 42.1
daubechies | 20.3 | NA | NA | NA
haralick | 304.5 | NA | 7615.8 | NA

(Best viewed in fixed-font)

OpenCV is really fast. Erosion is faster than simply count 1 pixels:

%timeit cv2.erode(fbin8, Bc8)
10000 loops, best of 3: 35.9 us per loop

%timeit np.sum(fbin8)
10000 loops, best of 3: 110 us per loop

Best,
Luis

[Stéfan van der Walt]

On Thu, Aug 30, 2012 at 9:46 AM, Luis Pedro Coelho <lu...@luispedro.org> wrote:
> I made a timing comparison between mahotas & scikits-image.
>
> operation   | mahotas  | pymorph  | skimage
> ------------+----------+----------+---------

> dilate      |    11.44 |     8.93 |    41.59

With the latest versions of skimage and mahotas, I have:

          |   mahotas  | skimage
----------+------------+--------
erode     |      9.60  |   16.18
dilate    |     11.69  |    8.01
open      |     20.53  |   16.16

Thanks to Johannes Schönberger for the speed-ups.

Stéfan

[Luis Pedro Coelho]

Great!

I will redo the timings in the next round of manuscript corrections (I
already submitted last weeks results yesterday) for the mahotas paper. I
might try to speed up things on mahotas' side too :)

A little friendly competition always gets users better software.

Best,
Luis

[Luis Pedro Coelho]

Great!

Now that I look at this, I realise that the comparison is not fair to mahotas because it includes a thresholding step. The correct comparison would run mahotas.sobel(f, just_filter=True) which runs in almost the exact same time as skimage because it is almost the exact same code ;-)

mahotas.center_of_mass also handles labeled images; btw.

Best,

Luis

On Tuesday, September 4, 2012 4:22:54 PM UTC+1, Johannes Schönberger wrote:

After another round of performance improvements for the sobel filter, I get:

erode       &     9.03 &    13.67 &    13.31 & \\
dilate      &    12.45 &     9.40 &     6.60 & \\
open        &    21.99 &    22.70 &    13.25 & \\
center mass &     6.63 &       NA &       NA & \\
sobel       &    74.66 &       NA &    43.72 & \\

Note, that there also exists a possibility to calculate the center of mass using the `regionprops` function and the properties "Centroid" and "WeightedCentroid". But due to the label overhead this is probably a lot slower than your implementation.