Xuggler video player in a BSAF (or SAF) application

[Daniel Blasco]

I've posted the same question in stackoverflow But maybe it's better to ask it in this forum. I'm just copying the same that I've posted in stackoverflow:

I'm trying to play a video file in a swing component with Xuggler. The Java application is built over the Better Swing Application Framework (BSAF).

I've found two examples of how to do this in the Xuggle blog and in navinbhutada blog. Then, I've adapted a little bit the code creating two classes: ImageComponent - that is extending JComponent and is displaying the video - and DecodeAndPlayVideo - that processes a video file with Xuggler and sends the images to the ImageComponent instance.

To try all of this I've coded a third class named App that adds an ImageComponent to a JFrame and plays a video. This is working fine and displaying the video:


ImageComponent.java

    public class ImageComponent extends JComponent {
        private Image mImage;
   
        public void setImage(Image image) {
            mImage = image;
            repaint();
        }
   
        @Override
        public synchronized void paint(Graphics g) {
            if (mImage != null) {
                g.drawImage(mImage, 0, 0, this);
            }
        }
    }


DecodeAndPlayVideo.java

    public class DecodeAndPlayVideo {
        private ImageComponent mScreen;
        private IStreamCoder videoCoder;
        private IContainer container;
   
        @SuppressWarnings("deprecation")
        public DecodeAndPlayVideo(ImageComponent mScreen) {
            this.mScreen = mScreen;
        }
   
        @SuppressWarnings("deprecation")
        public void PlayVideo(String filename) {
            if (!IVideoResampler.isSupported(
                IVideoResampler.Feature.FEATURE_COLORSPACECONVERSION)) {
                throw new RuntimeException("you must install the GPL version"
                    + " of Xuggler (with IVideoResampler support) for "
                    + "this demo to work");
            }
   
            container = IContainer.make();
   
            if (container.open(filename, IContainer.Type.READ, null) < 0) {
                throw new IllegalArgumentException("could not open file: " + filename);
            }
   
            int numStreams = container.getNumStreams();
            int videoStreamId = -1;
   
            for (int i = 0; i < numStreams; i++) {
                IStream stream = container.getStream(i);
                IStreamCoder coder = stream.getStreamCoder();
   
                if (coder.getCodecType() == ICodec.Type.CODEC_TYPE_VIDEO) {
                    videoStreamId = i;
                    videoCoder = coder;
                    break;
                }
            }
   
            if (videoStreamId == -1) {
                throw new RuntimeException("could not find video stream in container: "
                    + filename);
            }
   
            if (videoCoder.open() < 0) {
                throw new RuntimeException("could not open video decoder for container: "
                    + filename);
            }
   
            IVideoResampler resampler = null;
            if (videoCoder.getPixelType() != IPixelFormat.Type.BGR24) {
                resampler = IVideoResampler.make(videoCoder.getWidth(),
                                                videoCoder.getHeight(), IPixelFormat.Type.BGR24,
                                                videoCoder.getWidth(), videoCoder.getHeight(), videoCoder.getPixelType());
                if (resampler == null) {
                    throw new RuntimeException("could not create color space "
                        + "resampler for: " + filename);
                }
            }
   
            IPacket packet = IPacket.make();
            long firstTimestampInStream = Global.NO_PTS;
            long systemClockStartTime = 0;
            while (container.readNextPacket(packet) >= 0) {
                if (packet.getStreamIndex() == videoStreamId) {
                    IVideoPicture picture = IVideoPicture.make(videoCoder.getPixelType(),
                                                            videoCoder.getWidth(), videoCoder.getHeight());
   
                    int offset = 0;
                    while (offset < packet.getSize()) {
                        int bytesDecoded = videoCoder.decodeVideo(picture, packet, offset);
                        if (bytesDecoded < 0) {
                            throw new RuntimeException("got error decoding video in: "
                                + filename);
                        }
                        offset += bytesDecoded;
   
                        if (picture.isComplete()) {
                            IVideoPicture newPic = picture;
                            if (resampler != null) {
                                newPic = IVideoPicture.make(resampler.getOutputPixelFormat(),
                                                            picture.getWidth(), picture.getHeight());
                                if (resampler.resample(newPic, picture) < 0) {
                                    throw new RuntimeException("could not resample video from: "
                                        + filename);
                                }
                            }
                            if (newPic.getPixelType() != IPixelFormat.Type.BGR24) {
                                throw new RuntimeException("could not decode video"
                                    + " as BGR 24 bit data in: " + filename);
                            }
   
                            if (firstTimestampInStream == Global.NO_PTS) {
                                firstTimestampInStream = picture.getTimeStamp();
                                systemClockStartTime = System.currentTimeMillis();
                            } else {
                                long systemClockCurrentTime = System.currentTimeMillis();
                                long millisecondsClockTimeSinceStartofVideo =
                                    systemClockCurrentTime - systemClockStartTime;
                                long millisecondsStreamTimeSinceStartOfVideo =
                                    (picture.getTimeStamp() - firstTimestampInStream) / 1000;
                                final long millisecondsTolerance = 50; // and we give ourselfs 50 ms of tolerance
                                final long millisecondsToSleep =
                                        (millisecondsStreamTimeSinceStartOfVideo
                                            - (millisecondsClockTimeSinceStartofVideo
                                            + millisecondsTolerance));
                                if (millisecondsToSleep > 0) {
                                    try {
                                        Thread.sleep(millisecondsToSleep);
                                    } catch (InterruptedException e) {
                                        return;
                                    }
                                }
                            }
   
                            BufferedImage javaImage = Utils.videoPictureToImage(newPic);
                            updateJavaWindow(javaImage);
                        }
                    }
                } else {
                    do {
                    } while (false);
                }
   
            }
            close();
        }
   
        private void updateJavaWindow(BufferedImage javaImage) {
            mScreen.setImage(javaImage);
        }
   
        public void close() {
            if (videoCoder != null) {
                videoCoder.close();
                videoCoder = null;
            }
            if (container != null) {
                container.close();
                container = null;
            }
        }
    }


App.java

    public class App {
        private App() {
            ImageComponent mediaPlayerComponent = new ImageComponent();
   
            JFrame aFrame = new JFrame();
            aFrame.setPreferredSize(new Dimension(640, 500));
            aFrame.add(mediaPlayerComponent);
            aFrame.pack();
            aFrame.setLocationRelativeTo(null);
            aFrame.setVisible(true);
   
            DecodeAndPlayVideo decodeAndPlayVideo =
                            new DecodeAndPlayVideo(mediaPlayerComponent);
            decodeAndPlayVideo.PlayVideo("video.esf");
        }
   
        public static void main(String[] args) {
            new App();
        }
    }

All of this, as I said, is working fine and displaying a swing window with a video that is running smoothly. The problem is when I try to use BSAF:

AppBSAF.java

    public class AppBSAF extends SingleFrameApplication {
        public static void main(String[] args) {
            Application.launch(AppBSAF.class, args);
        }
   
        @Override
        protected void startup() {
            ImageComponent mediaPlayerComponent = new ImageComponent();
   
            JFrame aFrame = new JFrame();
            setMainFrame(aFrame);
            aFrame.setPreferredSize(new Dimension(640, 500));
            aFrame.add(mediaPlayerComponent);
            aFrame.pack();
            aFrame.setLocationRelativeTo(null);
            aFrame.setVisible(true);
   
            DecodeAndPlayVideo decodeAndPlayVideo =
                            new DecodeAndPlayVideo(mediaPlayerComponent);
            decodeAndPlayVideo.PlayVideo("video.esf");
        }
    }

With BSAF a swing window is created, as with App.java, but no video is displayed. The entire system slows down and even the mouse is sluggish, but no error is reported in the console.

While looking for a solution I've found this page: Xuggler_on_NetBeans that could provide some clue related to the heavyweight and lightweight components in swing, but I can't find out how it works in the BSAF case when launching the application.

My system:

- Ubuntu 11.10
- Java 1.6.0_31
- NetBeans 7.1
- Maven 3.0.4
- BSAF 1.9.2
- Xuggler 5.2

Any help would be very appreciate.