Syncing audio/video with xuggler

[Sean Brophy]

Hi all,

I've posted before. I'm trying to sync the audio/video using xuggler to decode the video files. I thought I had solved this, but it had turned out that my laptop's relatively crappy video hardware had made things appear to be syncing as designed. In reality, the video was just being decoded very slowly, and it happened to sync up with some tuning of the sync tolerances.

The Overview:

I'm trying to decode and play video files inside an OpenGL instance. I've got my video decoding being done inside a thread I call "PacketHandler", after which the decoded packets are parcelled off to their respective outputs.

For audio it goes to a thread which writes it out to a SourceDataLine object. For video, it's converted to a format that can be drawn as a texture.

I've had some success, the audio plays flawlessly, and the video plays relatively smoothly. My approach has been to use the audio playback as a gold standard from which to decide when video frames are updated. This is because I want to make sure that there is always audio being written out.

Audio gets written out as raw data, so the timestamps are lost when it goes into the buffer, and I've been using SourceDataLine.getMicrosecondPosition() to keep time. Keeping track of the last timestamp put into the buffer seems like a worse approach, because we don't necessarily know how big the buffer will get for a given hardware setup.

I'm beginning to suspect however that my approach has introduced some imprecision into the mix, and 

The Problem:
The audio seems to be out of sync still, but it seems to be out of sync by the same amount of time for every playthrough. The video is slightly ahead of the audio, and no amount of tuning seems to be able to change that. 

The Code:
The codebase is available here:
https://github.com/seannybgoode/libGDX-EvolveLiteMediaPlayerLibrary


I appreciate any advice you A/V geniuses have for me. I think I'm on the right track, but would welcome any suggestions toward a solution to the syncing issues so that I can continue to focus on improving the library.

Thanks in advance!

[Sean Brophy]

So a followup question:

Have I gone too low level? Should I be using IMediaReader?