No way to play audio from video file in android using AudioTrack

[shihabudheen pk]

Hi Sir,

You have done a great work providing wrappers for multiple essential libraries.

Into my issue.Iam trying to play a video file by grabbing each frames from the file using FFmpegFrameGrabber in a while loop,grabbing frames using 

Frame frame=grabber.grabFrame();

can access the image using 

frame.image

and i also manage to get the image displayed on ImageView after converting to Bitmap.And i am also getting a higher frame rate.But the biggest problem is audio.

I have tried converting resultant audio sample buffer to short.

Also tried to split data in an interleaved way for stereo.

But all iam hearing is noise.

Can u please tell me what is the correct way to play the

frame.samples

in android.

Hope you provide a better code snippet.

[Samuel Audet]

On 03/10/2015 04:33 AM, shihabudheen pk wrote:
> I have tried converting resultant audio sample buffer to short.
> Also tried to split data in an interleaved way for stereo.
>
> But all iam hearing is noise.
>
> Can u please tell me what is the correct way to play the
>
> frame.samples
>
> in android.
>
> Hope you provide a better code snippet.

There's code snippets for that everywhere. Here's one:
http://www.brokenteapotstudios.com/android-game-development-blog/2011/10/playing-a-wav-pcm-sample-with-audiotrack.html

Just call AudioTrack.write(samples), and there's no reason that
shouldn't work. There is no need to "convert" anything, it should just
simply work.

If you have some code sample that doesn't work for some reason, please
provide the problematic code, and we'll take a look at that, thanks!

Samuel

[shihabudheen pk]

Hey Sam,

Thanks for adding my question on your busiest days.

Yes here it is below.The code iam using currently which plays stereo audio from a video file.Also convert it to short.

Here it is the initialization of AudioTrack.

int frequency =44100;

int minBufSize = AudioTrack.getMinBufferSize(frequency,

AudioFormat.CHANNEL_OUT_STEREO,

AudioFormat.ENCODING_PCM_16BIT);

mAudioTrack = new AudioTrack(AudioManager.STREAM_MUSIC,

frequency, AudioFormat.CHANNEL_OUT_STEREO,

AudioFormat.ENCODING_PCM_16BIT, minBufSize,

AudioTrack.MODE_STREAM);

mAudioTrack.play();

Here it is below,the code which used to play the audio.

FloatBuffer fb;

ShortBuffer sb;

short[] shorts;

ByteBuffer byteBuffer =null;

byte[] bytez;

try {

for (Buffer b : samples) {

fb = (FloatBuffer)b;

byteBuffer = ByteBuffer.allocate(b.capacity()*4);

byteBuffer.order(ByteOrder.LITTLE_ENDIAN);

(byteBuffer).asFloatBuffer().put(fb);

byteBuffer.rewind();

bytez = new byte[byteBuffer.capacity()];

byteBuffer.get(bytez);

shorts=new short[bytez.length/2];

 ByteBuffer.wrap(bytez).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(shorts);

mAudioTrack.write(shorts, 0, shorts.length);

}

}

catch(Exception e)

{

e.printStackTrace();

}

I can here a slight voice with a lot of distortion like when tuning radio for channels.

Full of distortion.

Tried changing Indianness.

Tried converting byte to short and played

Tried playing byte and played.

And also if i try cast directly to ShortBuffer instead of FloatBuffer.There is a class cast exception.So i used FloatBuffer and converted to shorts.

Can u please provide the correct way to play the audio coming frame grabber.

And also have one more question will be posted after i got an answer for this.:)

Regards,

shihab

[Samuel Audet]

On 03/16/2015 11:19 PM, shihabudheen pk wrote:
> And also if i try cast directly to ShortBuffer instead of
> FloatBuffer.There is a class cast exception.So i used FloatBuffer and
> converted to shorts.

Do you have the possibility of using AudioFormat.ENCODING_PCM_FLOAT and
call the write() function in the following way?

float[] floats = new float[fb.limit()]
mAudioTrack.write(floats, 0, floats.length);

Samuel

[ShihabSoft]

Sorry Sam,
As I mentioned in the question title.

I can't play it correctly for weeks.

Here it is in Android there are only.

ENCODING_DEFAULT
ENCODING_INVALID
ENCODING_PCM_16BIT
ENCODING_PCM_8BIT

There is a reminder for you.
I can slightly hear the voice with lots of distortions with my code posted before.

Hope you help me :)

[ShihabSoft]

Hey everyone,

Here i have found my way to the destination with just a keyword provided by Sam.

PCM_FLOAT.

What is the way to play the raw float from javacv to android?Here it is i have created a class.Take it on for free.

package com.shihabsoft.castanythingtopc;

import android.media.AudioFormat;

import android.media.AudioManager;

import android.media.AudioTrack;

public class AndroidAudioDevice

{

   AudioTrack track;

   short[] buffer = new short[1024];

   public AndroidAudioDevice(int sampleRate,int channels)

   {

  

      int minSize =AudioTrack.getMinBufferSize( sampleRate, channels==1?AudioFormat.CHANNEL_CONFIGURATION_MONO:AudioFormat.CHANNEL_CONFIGURATION_STEREO, AudioFormat.ENCODING_PCM_16BIT );        

      track = new AudioTrack( AudioManager.STREAM_MUSIC, sampleRate, 

     channels==1?AudioFormat.CHANNEL_CONFIGURATION_MONO:AudioFormat.CHANNEL_CONFIGURATION_STEREO, AudioFormat.ENCODING_PCM_16BIT, 

                                        minSize, AudioTrack.MODE_STREAM);

      track.play();        

   }   

   public void writeSamples(float[] samples) 

   {

      fillBuffer( samples );

      track.write( buffer, 0, samples.length );

   }

   private void fillBuffer( float[] samples )

   {

      if( buffer.length < samples.length )

         buffer = new short[samples.length];

      for( int i = 0; i < samples.length; i++ )

         buffer[i] = (short)(samples[i] * Short.MAX_VALUE);;

   }

}

How to use my class.

AndroidAudioDevice aaD=new AndroidAudioDevice(sampleRate Hz,numberOfChannels);

aaD.writeSamples(floatArray);

But iam not so cruel :) ,to leave you after crossing half-quarter of the sea.Here it is i have created the code from top to end ie,Frame to float array.Feel free to use it.

final java.nio.Buffer[] samples=vFrame.samples;//Getting the samples from the Frame from grabFrame()
float[] smpls;
if(aaD.track.getChannelCount()==1)//For using with mono track

{

Buffer b=samples[0];

fb = (FloatBuffer)b;

fb.rewind();

smpls=new float[fb.capacity()];

fb.get(smpls);

}

else if(aaD.track.getChannelCount()==2)//For using with stereo track

{

FloatBuffer b1=(FloatBuffer) samples[0];

FloatBuffer b2=(FloatBuffer) samples[1];

smpls=new float[b1.capacity()+b2.capacity()];

for(int i=0;i<b1.capacity();i++)

{

        smpls[2*i]=b1.get(i);

smpls[2*i+1]=b2.get(i);

}

}

aaD.writeSamples(smpls);

Don't ever put the sampleRate and audioChannels manually,Just grab that both from FFmpegFrameRecorder.

Because incase the video have a sample rate of 44kHz and you are initializing the audio track with 8000Hz,the audio will be so distored you may never hear a voice.So be carefull on those two params.

I may suggest,Samuel to add a new class in the next release for reliability of developers using javacv for android playing audio.

From the above code provided.

Regards,

ShihabSoft

[Samuel Audet]

On 03/24/2015 09:09 PM, ShihabSoft wrote:
> Because incase the video have a sample rate of 44kHz and you are
> initializing the audio track with 8000Hz,the audio will be so distored
> you may never hear a voice.So be carefull on those two params.
>
> I may suggest,Samuel to add a new class in the next release for
> reliability of developers using javacv for android playing audio.
>
> From the above code provided.

Thanks! Actually, because most people will want more flexibility, I
would see this code being more appropriate as a great sample! Could you
package this into a very small application that uses this code, and that
we can add to the samples directory? That would be awesome :) Thanks!

Samuel

[ShihabSoft]

Ya to be great Sam.I 'll try to help.

How can i become a part of this project as a project member.

Is it possible so?

--

--- You received this message because you are subscribed to the Google Groups "javacv" group.
To unsubscribe from this group and stop receiving emails from it, send an email to javacv+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[ShihabSoft]

And please mention the way to post the sample application.Is it ok for mentioning my name or entity in that sample.

[Samuel Audet]

On 03/29/2015 01:32 PM, ShihabSoft wrote:
> Ya to be great Sam.I 'll try to help.
> How can i become a part of this project as a project member.
> Is it possible so?

Sure, you could for example start by sending pull requests here to add
new samples:
https://github.com/bytedeco/javacv/tree/master/samples
I've added some other ideas here:
http://bytedeco.org/contribute/
But I'm open to suggestions. What would be your ideas?

Samuel

[Samuel Audet]

On 03/29/2015 01:43 PM, ShihabSoft wrote:
> And please mention the way to post the sample application.Is it ok for
> mentioning my name or entity in that sample.

Of course! Check the existing samples.

Samuel