Performance getting contour points

623 views
Skip to first unread message

Huey

unread,
Nov 20, 2010, 9:50:19 AM11/20/10
to javacv
I'm using CvFindContours to get the contours in an image and then
iterating over all points in the returned contours. CvFindContours is
fast but iterating over the contour points takes a long time. For
example, in an image with about 350 contours consisting of a total of
2500 points it takes 50 seconds to iterate over the points.

Searching the mailing list I found this: http://code.google.com/p/javacv/issues/detail?id=10.
It mentions that JNA is slow for large structures and suggests setting
autoSynch to false. Surprisingly, doing that made it slower - it took
130 seconds to iterate over the points.

Am I doing this right? Is there a faster way? I'd like process images
in real time.

I've included the test program below. printContoursWithGetStructure()
gets the contours with autoSync defaulting to true. It takes about
400 milliseconds to execute firstContour.getStructure() and then takes
about 50 seconds to iterate over the points.
printContoursWithSyncFalse() gets the contours with autoSync set to
false and uses readField().

Here's the program:

import com.googlecode.javacv.jna.cxcore;
import com.sun.jna.Pointer;

import javax.swing.*;
import java.awt.*;
import java.awt.image.BufferedImage;
import java.io.IOException;

import static com.googlecode.javacv.jna.cxcore.*;
import static com.googlecode.javacv.jna.highgui.*;
import static com.googlecode.javacv.jna.cv.*;


/*
* Example results:

findContours(): numContours: 345
printContoursWithGetStructure(): took 443 millis for
firstContour.getStructure()
printContoursWithGetStructure(): took 52278 millis to iterate over all
points of all contours
printContoursWithGetStructure(): numPoints: 2457, numContours: 345
printContoursWithSyncFalse(): took 0 millis for new
CvContour(firstContour.getValue())
printContoursWithSyncFalse(): h_next is null
printContoursWithSyncFalse(): took 131262 millis to iterate over all
points of all contours
printContoursWithSyncFalse(): numPoints: 2457, numContours: 345
*/


public class TestCvFindContours
{
private static IplImage loadImage(String filename)
{
IplImage image = cvLoadImage(filename, 1);

if (image == null)
{
String msg = "loadImage(): Error calling cvLoadImage on `"
+ filename + "'";
System.err.println(msg);
throw new RuntimeException(msg);
}

return image;
}




private static IplImage cannyImage(IplImage inImage)
{
CvSize.ByValue size = cxcore.cvSize(inImage.width,
inImage.height);

IplImage grayImage = cxcore.cvCreateImage(size,
cxcore.IPL_DEPTH_8U, 1);
IplImage cannyImage = cxcore.cvCreateImage(size,
cxcore.IPL_DEPTH_8U, 1);

cvCvtColor(inImage, grayImage, CV_RGB2GRAY);

cvCanny(grayImage, cannyImage, 10, 100, 3);

return cannyImage;
}




private static CvSeq.PointerByReference findContours(IplImage
cannyImage,
CvMemStorage
storage)
{
CvSeq.PointerByReference firstContour = new
CvSeq.PointerByReference();
int sizeofCvContour =
com.sun.jna.Native.getNativeSize(CvContour.ByValue.class);

int numContours = cvFindContours(cannyImage, storage,
firstContour,
sizeofCvContour,
CV_RETR_EXTERNAL,
CV_CHAIN_APPROX_SIMPLE);

System.out.println("findContours(): numContours: " +
numContours);

return firstContour;
}



private static void
printContoursWithGetStructure(CvSeq.PointerByReference firstContour)
{
long start = System.currentTimeMillis();
CvSeq contour = firstContour.getStructure();
long millis = System.currentTimeMillis() - start;

System.out.println("printContoursWithGetStructure(): took " +
millis + " millis for firstContour.getStructure()");

int numPoints = 0;
int numContours = 0;

start = System.currentTimeMillis();
while (contour != null)
{
++numContours;

// System.out.println("Bounding box of contour: " +
cvBoundingRect(contour, 0));

// Print all points in contour
for (int i = 0; i < contour.total; ++i)
{
++numPoints;

Pointer ptrPoint = cvGetSeqElem(contour, i);
CvPoint point = new CvPoint(ptrPoint);
// System.out.println(point);
}

contour = contour.h_next;
}

millis = System.currentTimeMillis() - start;
System.out.println("printContoursWithGetStructure(): took " +
millis + " millis to iterate over all points of all contours");
System.out.println("printContoursWithGetStructure():
numPoints: " + numPoints + ", numContours: " + numContours);
}





private static void
printContoursWithSyncFalse(CvSeq.PointerByReference firstContour)
{
CvContour.autoSynch = false;

long start = System.currentTimeMillis();
CvContour cvContour = new CvContour(firstContour.getValue());

long millis = System.currentTimeMillis() - start;
System.out.println("printContoursWithSyncFalse(): took " +
millis + " millis for new CvContour(firstContour.getValue())");

int numPoints = 0;
int numContours = 0;

start = System.currentTimeMillis();
while (cvContour != null)
{
++numContours;

cvContour.readField("rect");
cvContour.readField("total");
cvContour.readField("h_next");

// System.out.println("Bounding box of contour: " +
cvBoundingRect(cvContour, 0));

// Print all points in contour
for (int i = 0; i < cvContour.total; ++i)
{
++numPoints;

Pointer ptrPoint = cvGetSeqElem(cvContour, i);
CvPoint point = new CvPoint(ptrPoint);
// System.out.println(point);
}


CvSeq.ByReference hNext = cvContour.h_next;
if (hNext == null)
{
System.out.println("printContoursWithSyncFalse():
h_next is null");
break;
}

cvContour = new CvContour(hNext.getPointer());
}

millis = System.currentTimeMillis() - start;
System.out.println("printContoursWithSyncFalse(): took " +
millis + " millis to iterate over all points of all contours");
System.out.println("printContoursWithSyncFalse(): numPoints: "
+ numPoints + ", numContours: " + numContours);
}




public static void main(String[] args) throws IOException
{
IplImage origImage = loadImage("Pong-splash.png"); // Pong-
mid-game.png

JFrame frame = new JFrame("Test");
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

ImagePanel panel = new ImagePanel();
frame.setContentPane(panel);

IplImage cannyImage = cannyImage(origImage);


CvMemStorage storage = CvMemStorage.create();

CvSeq.PointerByReference firstContour =
findContours(cannyImage, storage);

printContoursWithGetStructure(firstContour);
printContoursWithSyncFalse(firstContour);


cvClearMemStorage(storage);




panel.setImage(cannyImage.getBufferedImage());

cvReleaseImage(origImage.pointerByReference());
cvReleaseImage(cannyImage.pointerByReference());

frame.pack();
frame.setVisible(true);
}
}



class ImagePanel extends JPanel
{
private BufferedImage image;


@Override
public void paintComponent(Graphics g)
{
g.drawImage(image, 0, 0, null);
}


public void setImage(BufferedImage image)
{
this.image = image;

setPreferredSize(new Dimension(image.getWidth(),
image.getHeight()));
}
}

Samuel Audet

unread,
Nov 20, 2010, 10:47:22 AM11/20/10
to jav...@googlegroups.com
The JNA Structure class isn't very fast unfortunately.. It's practical,
but when you need to do things like that, it will be faster using the
Pointer class only. For each field and its known offset in the struct,
you may get its value by calling the appropriate get*() method.
(Ideally, JNA should provide this kind of access for Structure as well,
something like contour.getTotal() to get the value of the /total/ field,
for example, but it's not doing that right now.)

But even so, each Pointer.get*() makes a native call, going in and out
of the virtual machine takes maybe 200 instructions (which you may
reduce by using a Buffer returned by Pointer.getByteBuffer() instead),
so it may still be slow. In that case, you will have to program that
loop in C, which can then very easily be called using JNA. FYI, this is
what I did for "cvkernels".

Samuel

Samuel Audet

unread,
Nov 21, 2010, 6:48:52 AM11/21/10
to jav...@googlegroups.com
BTW, the best way to handle the case of CvSeq in situations like is to
use cvCvtSeqToArray(). This way, you only have to make a few native
calls, and the rest can be done via preallocated memory entirely and
efficiently in Java.

I guess something like
cvContour.readField("total");
CvPoint pts[] = CvPoint.createArray(cvContour.total);
cvCvtSeqToArray(cvContour, pts[0].getPointer(), CV_WHOLE_SEQ);
would work for you..

Samuel

Huey

unread,
Nov 22, 2010, 8:59:59 PM11/22/10
to javacv
Thanks! I tried using the Pointer class with the appropriate get*()
methods for each field with the offset of the field. It improved
performance from 50 seconds to 300 milliseconds.

I'll try cvCvtSeqToArray.

Samuel Audet

unread,
Nov 22, 2010, 9:24:45 PM11/22/10
to jav...@googlegroups.com
I thought about it a bit more, and CvPoint is still a Structure, so you
would probably get best performance doing something like this:
Memory m = new Memory(bigenoughsize > total)
IntBuffer b = m.getByteBuffer(0, m.size()).asIntBuffer();
cvCvtSeqToArray(cvContour, m, CV_WHOLE_SEQ);
for (int i = 0; i < total; i++) {
int x = b.get(2*i );
int y = b.get(2*i + 1)
...
}
That should be fast..

Samuel

Huey

unread,
Nov 24, 2010, 1:10:00 PM11/24/10
to javacv
I tried cvCvtSeqToArray and it is faster - 90 milliseconds for 345
contours.

I found that using the bulk get(byte[] dst) in ByteBuffer rather than
separate get(int)'s to get x and y is faster. For 345 short contours
the bulk get took 40 milliseconds less than individual gets.

My current code does "new CvContour(ptrToCvSeq)" and passes the result
as the first argument to cvCvtSeqToArray(). "new
CvContour(ptrToCvSeq)" takes 0.15 milliseconds. Multiplied by 345
contours this accounts for 51 milliseconds.

If there was a way to call cvCvtSeqToArray with a pointer to a CvSeq
rather than a CvContour that might be faster. The OpenCv
documentation for cvCvtSeqToArray has a first parameter of "const
CvSeq *" but I didn't see a declaration of cvCvtSeqToArray in
cxcore.java that takes a pointer first argument.
> >> would work for you..- Hide quoted text -
>
> - Show quoted text -

Samuel Audet

unread,
Nov 24, 2010, 5:50:24 PM11/24/10
to jav...@googlegroups.com
On 2010-11-25 03:10, Huey wrote:
> I tried cvCvtSeqToArray and it is faster - 90 milliseconds for 345
> contours.
>
> I found that using the bulk get(byte[] dst) in ByteBuffer rather than
> separate get(int)'s to get x and y is faster. For 345 short contours
> the bulk get took 40 milliseconds less than individual gets.

Hum, that's strange.. which JDK are you using? If it's a variant of
Sun's JDK like OpenJDK, are you running the "server" version of HotSpot?

> My current code does "new CvContour(ptrToCvSeq)" and passes the result
> as the first argument to cvCvtSeqToArray(). "new
> CvContour(ptrToCvSeq)" takes 0.15 milliseconds. Multiplied by 345
> contours this accounts for 51 milliseconds.

And CvContour.autoSynch = false? There is a protected method
useMemory().. You may call that to reuse the same CvContour object, but
if you're not going to do anything with the fields then might as well
use a Pointer yes

> If there was a way to call cvCvtSeqToArray with a pointer to a CvSeq
> rather than a CvContour that might be faster. The OpenCv
> documentation for cvCvtSeqToArray has a first parameter of "const
> CvSeq *" but I didn't see a declaration of cvCvtSeqToArray in
> cxcore.java that takes a pointer first argument.

Yes, you can add a declaration that accepts a Pointer, it will work..
You don't need to modify cxcore.java itself, you can declare your own
class like this:

public static class mycxcore extends cxcore {
public static final String libname = Loader.load(paths, libnames);
public static native Pointer cvCvtSeqToArray(Pointer seq,
Pointer elements, CvSlice.ByValue slice/*=CV_WHOLE_SEQ*/);
}

And calling mycxcore.cvCvtSeqToArray(ptrToCvSeq, m, CV_WHOLE_SEQ) should
work

Samuel

Huey

unread,
Nov 24, 2010, 7:46:47 PM11/24/10
to javacv
I'm using the 32-bit Sun 1.6.0_22 JDK (via Clojure but don't think
Clojure is impacting this). Looking at the source code for
DirectIntBufferU class (which is the implementation of IntBuffer
returned by m.getByteBuffer(0, m.size()).asIntBuffer()) I see that it
uses JNI to get data from the buffer, so that may explain why a bulk
get is faster.

The time for "new CvContour(ptrToCvSeq)" is with CvContour.autoSynch =
false.

The mycxcore class had a big impact. It now takes 15 milliseconds for
345 contours. Thanks!

Samuel Audet

unread,
Nov 24, 2010, 9:13:16 PM11/24/10
to jav...@googlegroups.com
On 2010-11-25 09:46, Huey wrote:
> I'm using the 32-bit Sun 1.6.0_22 JDK (via Clojure but don't think
> Clojure is impacting this). Looking at the source code for

What does "java -version" say?

> DirectIntBufferU class (which is the implementation of IntBuffer
> returned by m.getByteBuffer(0, m.size()).asIntBuffer()) I see that it
> uses JNI to get data from the buffer, so that may explain why a bulk
> get is faster.

No, direct NIO buffers don't use JNI

> The time for "new CvContour(ptrToCvSeq)" is with CvContour.autoSynch =
> false.
>
> The mycxcore class had a big impact. It now takes 15 milliseconds for
> 345 contours. Thanks!

great!

Samuel

Huey

unread,
Nov 24, 2010, 9:38:16 PM11/24/10
to javacv


On Nov 24, 9:13 pm, Samuel Audet <samuel.au...@gmail.com> wrote:
> On 2010-11-25 09:46, Huey wrote:
>
> > I'm using the 32-bit Sun 1.6.0_22 JDK (via Clojure but don't think
> > Clojure is impacting this).  Looking at the source code for
>
> What does "java -version" say?
>

java -version
java version "1.6.0_22"
Java(TM) SE Runtime Environment (build 1.6.0_22-b04)
Java HotSpot(TM) Client VM (build 17.1-b03, mixed mode, sharing)


> > DirectIntBufferU class (which is the implementation of IntBuffer
> > returned by m.getByteBuffer(0, m.size()).asIntBuffer()) I see that it
> > uses JNI to get data from the buffer, so that may explain why a bulk
> > get is faster.
>
> No, direct NIO buffers don't use JNI
>

Maybe I'm misinterpreting things. DirectIntBufferU.java has this:

public int get() {
return ((unsafe.getInt(ix(nextGetIndex()))));
}

where 'unsafe' is an instance of sun.misc.Unsafe which has the
following declaration:

public native int getInt(long address);

(according to
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/sun/misc/Unsafe.java#Unsafe.getInt%28long%29)

Samuel Audet

unread,
Nov 24, 2010, 9:41:37 PM11/24/10
to jav...@googlegroups.com
On 2010-11-25 11:38, Huey wrote:
> java -version
> java version "1.6.0_22"
> Java(TM) SE Runtime Environment (build 1.6.0_22-b04)
> Java HotSpot(TM) Client VM (build 17.1-b03, mixed mode, sharing)

There you go, you need the "Server VM" for performance, not the "Client
VM". You may try to run java using the -server option, but it does not
mean you have it installed..

> where 'unsafe' is an instance of sun.misc.Unsafe which has the
> following declaration:
>
> public native int getInt(long address);

That does not mean the VM uses JNI..

Samuel

Huey

unread,
Nov 25, 2010, 4:37:25 PM11/25/10
to javacv


On Nov 24, 9:41 pm, Samuel Audet <samuel.au...@gmail.com> wrote:
>
> There you go, you need the "Server VM" for performance, not the "Client
> VM". You may try to run java using the -server option, but it does not
> mean you have it installed..
>

Using -server improved average performance from 15 milliseconds to 4
milliseconds for 345 contours.

> > where 'unsafe' is an instance of sun.misc.Unsafe which has the
> > following declaration:
>
> > public native int     getInt(long address);
>
> That does not mean the VM uses JNI..

Thanks, I wasn't aware of that.
Reply all
Reply to author
Forward
0 new messages