AWT JNI Painting for Xuggler

[mail4732]

Hello JNI Cracks

I'm writing a Java Movie Player based on Xuggler. Now my problem is
that especially HD Content (1920x1080 or something) cannot be played
at all because painting the BufferedImage to the screen is much to
slow. If the content should fit into some component scaling is
impossible even on hardcore machines. At least not if you want more
than 0 fps :)

When i look at the Quicktime for Java component everything is so much
smoother and faster without using much CPU. My guess is that they
somehow paint directly on the screen using AWT JNI stuff.
http://en.wikipedia.org/wiki/Java_AWT_Native_Interface

For scaling i'm thinking of OpenGL which might be very fast used
directly from the C code. Also OpenGL is available on every platform
as far as i know. If using OpenGL directly cannot be done easily maybe
JOGL can somehow help.

My problem is that i really don't know much about JNI and AWT so i
have no idea if this would make any sense.
Please let me know if that way has truly the potential to play FullHD
stuff.

Is it correct that currently every BufferedImage i get is independent
and causes a lot of garbage collection during playback?
Maybe it would be simple enough to just "update" that bufferedImage in
some smart way?

Also i noticed playing without sound seems to cause a little
performance boost. Maybe sound could be played through C
stuff in a better way? No idea how difficult that would be.

[Art Clarke]

The short answer is yes.  The bad news is you may need to learn some JNI.  But here's the short recommendations:

1) JOGL would be the first thing to investigate.  If their API takes Direct java.net.ByteBuffer objects, then get the direct ByteBuffers from your IVideoPicture ojects (picture.getData().getByteBuffer(...)).  This will avoid any copy from native (C) memory to Java and back, which should result in MUCH faster renders.  You're probably going to need to use a IVideoResampler object to convert from YUV420P to RGB space for rendering, unless JOGL accepts YUV420 natively.  (I don't know a lot about JOGL).

2) If JOGL does not take java.net.ByteBuffers, then look into the AWT direct window draw functionality or OpenGL form C Code.  Create your own Java library with a JNI wrapper around your C methods.  For your Java API pass in a java.net.ByteBuffer object, and then there are JNI methods for getting the direct C native memory address.  See here: http://java.sun.com/j2se/1.4.2/docs/guide/jni/jni-14.html.  The build system we use for Xuggler uses something called SWIG to generate JNI bindings, and you may find some of the tricks we use their to be somewhat useful, but it's pretty advanced and probably more overkill than what you need.

3) If either of the two methods above is not fast enough, you could of course make your own version of Xuggler, adding new methods that link in the AWT libraries.  This will require you to be a java/swig/c/autoconf wizard, but it's doable.  I don't want to bundle that into standard Xuggler because right now we have no dependency on AWT meaning Xuggler runs in headless environments.

Hope that helps,

- Art

--

You received this message because you are subscribed to the Google Groups "xuggler-users" group.
To post to this group, send email to xuggle...@googlegroups.com.
To unsubscribe from this group, send email to xuggler-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/xuggler-users?hl=en.

--
http://www.xuggle.com/
xu‧ggle (zŭ' gl) v. To freely encode, decode, and experience audio and video.

Use Xuggle to get the power of FFmpeg in Java.

[Art Clarke]

And a few other notes from your e-mail:

a) if possible, try to reuse objects in your code rather than creating new objects each time.  Unfortunately the com.xuggle.xuggler.video.Converter object forces the creation of new BufferedImages each time, but you can see the code we use for that class (it's all Java) and use the IVideoResampler object to do the same thing while reusing the image.  Also try to re-use IVideoPicture and IAudioSamples objects and you should see a performance increase.

b) For audio playback, make sure you're always copying full sample ranges, not samples by samples (i.e. array-copy not short-by-short).  See the IBuffer.get(...) methods -- they will do a assembly based memcpy from native memory to the java short[] array you're probably using with java.sound

c) If the sound trick above doesn't work, you can play sound through C, but now you're into the world of platform dependent APIs.  Do you use Windows DirectSound, Linux ALSA or Jack, OSX's sound APIs, or other.  Way too complicated for me :)  In general though most sound-playback problems are because people are doing sample-by-sample gets, rather than array copies into the java.sound buffers.  See the demo playback audio samples we provide for an example of doing array copies to Java sound.

- Art