libwebm + vorbis

[Ben Siroshton]

Hi, this is my first post here. First I would like to thank the WebM
team here for providing these great libraries. I am adding webm
playback capabilities in an iPhone app I am working on and the video
pipeline is super fast, thanks!

Now to some trouble I am having regarding audio. I am using the
libwebm library to parse the webm file and using libvorbis to decode
the audio. Here is what I am doing and what is going wrong, any help
would be greatly appreciated.

After the webm file is loaded I loop through the tracks and look for
the first audio track. Once I have it I then initialize the vorbis
libraries like so:

// initialize vorbis
vorbis_info_init(&_vorbis.info);
vorbis_comment_init(&_vorbis.comment);
memset(&_vorbis.dspState,0,sizeof(_vorbis.dspState));
memset(&_vorbis.block,0,sizeof(_vorbis.block));

size_t size;
unsigned char * data = (unsigned char *)audioTrack-
>GetCodecPrivate(size);

_vorbis.oggPacket.packet = data+3; // NOTE "vorbis" does not start
until +3 into this packet.
_vorbis.oggPacket.bytes = size-3;
_vorbis.oggPacket.b_o_s = true;
_vorbis.oggPacket.e_o_s = false;
_vorbis.oggPacket.granulepos = 0;
_vorbis.oggPacket.packetno = 0;

int r;

r = vorbis_synthesis_headerin(&_vorbis.info, &_vorbis.comment,
&_vorbis.oggPacket);
if( r<0 )
debug(DBG_WARNING,"vorbis_synthesis_headerin failed, error: %d", r);

r = vorbis_synthesis_init(&_vorbis.dspState, &_vorbis.info);
if( r<0 )
debug(DBG_WARNING,"vorbis_synthesis_init failed, error: %d", r);

r = vorbis_block_init(&_vorbis.dspState, &_vorbis.block);
if( r<0 )
debug(DBG_WARNING,"vorbis_block_init failed, error: %d", r);


If you notice the "NOTE" above I am having to offset the pointer from
GetCodecPrivate by 3 otherwise the vorbis header is incorrect. If I
add 3 then everything initializes ok. Is there a bug in libwebm
regarding the codecs private data?

Now, lets assume that everything is ok and that +3 offset is a bug but
by adding 3 were ok. The next problem I am having is in decoding the
audio itself.

To parse the audio I loop through the available clusters and block
entries for the audio track I am interested in. Once I have some
audio data I deal with it like so:

if( trackNum==_audioInfo.audioTrack && _soundSource )
{
// Use this Audio Track
const long size = block->GetSize();

if( size>_audioInfo.audioPageSize )
{
if( _audioInfo.audioPage ) delete [] _audioInfo.audioPage;
_audioInfo.audioPage = new unsigned char[size];
_audioInfo.audioPageSize = size;
}

if( size>0 ) // NOTE: First time through size is only 1, seems bad??
{
block->Read(_reader.reader, _audioInfo.audioPage);

_vorbis.oggPacket.packet = _audioInfo.audioPage;
_vorbis.oggPacket.bytes = size;
_vorbis.oggPacket.b_o_s = false;
_vorbis.oggPacket.packetno++;
_vorbis.oggPacket.granulepos = -1;

if( vorbis_synthesis(&_vorbis.block, &_vorbis.oggPacket)==0 ) //
NOTE: <-- Memory access violation.
{
vorbis_synthesis_blockin(&_vorbis.dspState, &_vorbis.block);
}

// send audio to audio device...
}


I am new to libwebm and vorbis so there is a good chance I am simply
doing something wrong. What happens for me now is that
vorbis_synthesis has a memory access viloation as it looks like
pointers in codec_setup_info.mode_param fields were not initialized.

Any ideas?

Thanks.

-=ben

[Ben Siroshton]

Update. I upgraded to libvorbis 1.3.1 and this fixes the crash I was
getting, but it does not help; 1.3.1 simply checks to make sure the
memory is valid before accessing it now. It does give a little
insight however, turns out vorbis_synthesis is returning an
OV_EBADPACKET error. Following what I am doing for video (and is
working) I should pass the data from block->Read to the vorbis decoder
correct? Should I be doing this for each block or do I need to
accumalate all blocks for a given cluster before passing to vorbis?

Thanks.

-=ben

[Ben Siroshton]

Update II. I see now that I am not reading all of the vorbis headers
needed to fully initialize vorbis. Question now, is how exactly do I
do that? The lack of libwebm documentation makes it difficult to
progress. Is there a way to get header data before I start decoding
the streams or is the first part of the stream data the header data?
If thats the case then how do I know when the data is no longer header
data? On the other hand if its not in the stream data and is all in
GetCodecPrivate then how should this be used to initialize vorbis?

Thanks again.

-=ben

> > -=ben- Hide quoted text -
>
> - Show quoted text -

[Ben Siroshton]

Solved.... sorta.

I figured out the vorbis header and how to parse the 3 needed from
GetCodecPrivate. Everything is initialized fine and I am being graced
with lots of static! Yay, progress! :) :/

For those interested this is how I am initializing vorbis from the
data in GetCodecPrivate.

// initialize vorbis
vorbis_info_init(&_vorbis.info);
vorbis_comment_init(&_vorbis.comment);
memset(&_vorbis.dspState,0,sizeof(_vorbis.dspState));
memset(&_vorbis.block,0,sizeof(_vorbis.block));

size_t size;
unsigned char * data = (unsigned char *)audioTrack-
>GetCodecPrivate(size);

_vorbis.oggPacket.e_o_s = false;
_vorbis.oggPacket.granulepos = 0;
_vorbis.oggPacket.packetno = 0;

int r;

for(int i=0;i<3;i++)
{
while( (size-6) && memcmp(data,"vorbis",6) )
{
data++;
size--;
}

_vorbis.oggPacket.packet = data;
_vorbis.oggPacket.packet--;
_vorbis.oggPacket.bytes = size;
_vorbis.oggPacket.b_o_s = _vorbis.oggPacket.packetno==0;

r = vorbis_synthesis_headerin(&_vorbis.info, &_vorbis.comment,
&_vorbis.oggPacket);
if( r<0 )
debug(DBG_WARNING,"vorbis_synthesis_headerin failed, error: %d", r);

_vorbis.oggPacket.packetno++;
data++;

}

r = vorbis_synthesis_init(&_vorbis.dspState, &_vorbis.info);
if( r<0 )
debug(DBG_WARNING,"vorbis_synthesis_init failed, error: %d", r);

r = vorbis_block_init(&_vorbis.dspState, &_vorbis.block);
if( r<0 )
debug(DBG_WARNING,"vorbis_block_init failed, error: %d", r);

Again I am new to these libraries so if you see something that does
not look right please let me know. Otherwise hopefully this will be
helpful to someone else. Now to solve my static issue....

Thanks.

-=ben

> > - Show quoted text -- Hide quoted text -

[Matthew Gregan]

At 2010-09-13T16:43:45-0700, Ben Siroshton wrote:
> I figured out the vorbis header and how to parse the 3 needed from
> GetCodecPrivate. Everything is initialized fine and I am being graced
> with lots of static! Yay, progress! :) :/
>
> For those interested this is how I am initializing vorbis from the
> data in GetCodecPrivate.

For what it's worth, the correct way to unpack the three Vorbis headers is
documented here: http://matroska.org/technical/specs/codecid/index.html
under A_VORBIS.

It's fairly simple, but if you'd like to look at example code, see
nestegg_track_codec_data_count here:
http://github.com/kinetiknz/nestegg/blob/master/src/nestegg.c#L1676

And nestegg_track_codec_data here:
http://github.com/kinetiknz/nestegg/blob/master/src/nestegg.c#L1708

Cheers,
-mjg
--
Matthew Gregan |/
/| kin...@flim.org

[Hwasoo Lee]

Hi Ben,

Here is the code snippet that I implemented in Vorbis decoder filter.
That is not officially release yet by Cristian Adam who is in charge
of
maintain of Vorbis code.

If you need more info about decode vorbis data, please let me know.
Based on your code it seems to ok.

mWorkPacket.b_o_s = 0;
mWorkPacket.bytes = packet_size;
mWorkPacket.e_o_s = 0;
mWorkPacket.granulepos = 0;
mWorkPacket.packet = const_cast<unsigned char*>(packet);
mWorkPacket.packetno = mPacketCount++;

mNumChannels = mVorbisInfo.channels;
mSampleRate = mVorbisInfo.rate;

if (vorbis_synthesis(&mVorbisBlock, &mWorkPacket) == 0)
{
const int ret = vorbis_synthesis_blockin(&mVorbisState,
&mVorbisBlock);
assert(ret == 0);
}

float** pcm;
int samples = 0;

while ((samples = vorbis_synthesis_pcmout(&mVorbisState, &pcm)) > 0)
{
for (int ch = 0; ch < mNumChannels; ++ch)
{
short* temp_buffer = pcm_samples + ch;
const float* const one_channel = pcm[ch];

for (int i = 0; i < samples; ++i)
{
int temp = static_cast<int>(one_channel[i] * 32767.0f);
*temp_buffer = clip16(temp);

temp_buffer += mNumChannels;
}
}
num_of_pcm_samples = samples;
vorbis_synthesis_read(&mVorbisState, num_of_pcm_samples);
}
return VORBIS_DATA_OK;


Basically as you mentioned you have to deal with code private data
first,
after that the audio stream is coming.

Thanks,
Hwasoo

[Ben Siroshton]

Ah, now it makes sense. :) Thanks for the links. Now my header code
is cleaned up properly.

-=ben

>                                   /|                    kine...@flim.org

[Ben Siroshton]

Thanks. I figured out my static problem, the audio buffer I was using
was unsigned char * instead of short *. Now for some reason though
the audio is playing back slower then it should be with some
interference (double checked im using the correct frequency, in this
case its 48000). I'm using OpenAL and I suspect my problems are no
longer related to libwebm. Thanks for posting the code it helped me
discover my memory type problem.

-=ben

[Ben Siroshton]

I feel silly.. I fixed my audio problems, turns out I was
miscalculating the byte size of my sample for OpenAL. Thanks for
everyones help.

-=ben

> > > > > >  - Hide quoted text -
>

> - Show quoted text -...
>
> read more »