How to bind an audio with movie using FFFMPEGFramerecorder and FrameGrabber ?

[RAJESH K]

Hi All ,

I want to bind an audio with an video file using javacv/Android platform .
If any one has idea or any experience in that please advice me to come out .

Thanks ,
Rajesh K .

[Samuel Audet]

On 2012-07-24 19:51, RAJESH K wrote:
> Hi All ,
>
> I want to bind an audio with an video file using javacv/Android platform .
> If any one has idea or any experience in that please advice me to come
> out .

Something like this should work:

FrameGrabber grabber1 = new FFmpegFrameGrabber("video.avi");
FrameGrabber grabber2 = new FFmpegFrameGrabber("audio.mp3");
grabber1.start();
grabber2.start();
FrameRecorder recorder = new FFmpegFrameRecorder("output.mp4",
grabber1.getImageWidth(), grabber1.getImageHeight(),
grabber2.getAudioChannels());
recorder.setFrameRate(grabber1.getFrameRate());
recorder.setSampleFormat(grabber2.getSampleFormat());
recorder.setSampleRate(grabber2.getSampleRate());
recorder.start();
Frame frame1, frame2;
while ((frame1 = grabber1.grabFrame()) != null ||
(frame2 = grabber2.grabFrame()) != null) {
recorder.record(frame1);
recorder.record(frame2);
}
recorder.stop();
grabber1.stop();
grabber2.stop();

Samuel

[He Jibo]

Hi, Samuel,

Thanks for the code. Can we record the audio and video at the same time using opencv on windows platform?

---------------------------
He Jibo
Department of Psychology,
Beckman Institute for Advanced Science and Technology
University of Illinois, Urbana Champaign,
603 East Daniel St.,
Champaign, IL 61820
website: www.hejibo.info

[Samuel Audet]

On 2012-07-28 13:37, He Jibo wrote:
> Hi, Samuel,
>

> Thanks for the code.*Can we record the audio and video at the same time
> using opencv on windows platform?*

I don't think OpenCV offers anything in the way of audio, you need FFmpeg

[Rajesh]

Hi Samuel ,

Above shows few issues for me .  i am attached the issue as the image and also the issues .

issues are like as follows :
The method getAudioChannels() is undefined for the type FrameGrabber

The method getSampleFormat() is undefined for the type FrameGrabber
so please advice me .

[Samuel Audet]

Which version of JavaCV are you using?

On 08/23/2012 12:12 AM, Rajesh wrote:
> <https://lh5.googleusercontent.com/-AOQwylpnxuk/UDT2wWNXhCI/AAAAAAAAADA/it-yYLVA9Cw/s1600/Capture.PNG>

[RAJESH K]

Hi Samuel ,

I am using the latest update which is available in the javacv website.

now i am getting the issues while merging the files. i have added the issue line which i have got .

08-30 15:35:49.400: W/System.err(1542): com.googlecode.javacv.FrameRecorder$Exception: Audio codec not found
08-30 15:35:49.400: W/System.err(1542):     at com.googlecode.javacv.FFmpegFrameRecorder.start(FFmpegFrameRecorder.java:340)
08-30 15:35:49.630: W/System.err(1542): com.googlecode.javacv.FrameRecorder$Exception: No video output stream (Is imageWidth > 0 && imageHeight > 0 and has start() been called?)
08-30 15:35:49.630: W/System.err(1542):     at com.googlecode.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:493)
08-30 15:35:49.630: W/System.err(1542):     at com.googlecode.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:489)
08-30 15:35:49.630: W/System.err(1542):     at com.googlecode.javacv.FrameRecorder.record(FrameRecorder.java:210)
08-30 15:35:49.630: W/System.err(1542):     at com.example.javacv_latest_update.MainActivity$ImageconvertsionAsyncTask.doInBackground(MainActivity.java:88)
08-30 15:35:49.630: W/System.err(1542):     at com.example.javacv_latest_update.MainActivity$ImageconvertsionAsyncTask.doInBackground(MainActivity.java:1)


Please advice me to come out of this issue.

My video file length is 20 secs and format is mp4.
My audio file length is 20 secs and format is mp3.

Thanks
Rajesh K

[Samuel Audet]

On 2012-08-30 19:13, RAJESH K wrote:
> I am using the latest update which is available in the javacv website.
>
> now i am getting the issues while merging the files. i have added the
> issue line which i have got .
>

> *08-30 15:35:49.400: W/System.err(1542):

> com.googlecode.javacv.FrameRecorder$Exception: Audio codec not found
> 08-30 15:35:49.400: W/System.err(1542): at

> com.googlecode.javacv.FFmpegFrameRecorder.start(FFmpegFrameRecorder.java:340)*

Well, try to setAudioCodec() to something else before calling start()

[RAJESH K]

Hi Samuel ,
Still i am not able to get the answer for the solution which i have got . Please advice me . I have posted the issue also  in below.

[Deepak R K]

Hi Rajesh,

Any Luck on audio ? I am figuring it out too.

[Rajesh]

Sorry guys .. still i am also searching the solution . 

[Ram]

hi stuart did you solve this issue?i am also facing the same problem.

On Tuesday, September 4, 2012 8:02:01 PM UTC+5:30, Stuart Keough wrote:

Rajesh, did you ever fix this issue?  This is the exact issue i'm trying to work through over here.  Video works fine, but audio fails every time :(

[Samuel Audet]

On 12/07/2012 06:58 PM, Ram wrote:
> hi stuart did you solve this issue?i am also facing the same problem.

What is the problem?

[Samuel Audet]

On 12/10/2012 03:27 PM, Rajesh wrote:
> Hi samuel ,
>
> I need to combine both audio and video.
> 1. i have two options to do . please advice me from which one we can
> make it with the help of javacv.
>
> * option 1: *
> * while creating the video from images , we can add the
> audio by use of audio recording ?? *
> *option 2 : *
> * i can combine two files (Audio and video ) , which is
> already available in the mobile sd-card. *
> *

We can do both, just call FFmpegFrameRecorder.record() with whatever
audio and video data you want to save

Samuel

[Ram]

hi samuel,

   i am also facing same issue as rajesh...below i mentioned my error,how to solve this?

here is my code:

 String path ="/mnt/sdcard/Video_images"; 

File folder = new File(path);

File[]    listOfFiles = folder.listFiles();  

 if(listOfFiles.length>0)

   {

 

 iplimage = new opencv_core.IplImage[listOfFiles.length];

  

  for (int j = 0; j < listOfFiles.length; j++) {

String files="";  

   

  if (listOfFiles[j].isFile()) 

  {

  files = listOfFiles[j].getName();

  System.out.println(" j " +j   + listOfFiles[j]);

    }   

  

 

String[] tokens = files.split("\\.(?=[^\\.]+$)");

           String name=tokens[0]; 

           

          iplimage[j]=cvLoadImage("/mnt/sdcard/Video_images/"+name+".jpg");

         

          

  }

  

   } 

 

 

 

 

    FrameGrabber grabber1 = new FFmpegFrameGrabber("/mnt/sdcard/video.mp4"); 

    try {

grabber1.start();

} catch (com.googlecode.javacv.FrameGrabber.Exception e1) {

// TODO Auto-generated catch block

e1.printStackTrace();

} 

        FFmpegFrameRecorder recorder = new FFmpegFrameRecorder("/mnt/sdcard/DCIM/Camera/analysed"+System.currentTimeMillis()+".mp4",200,150,grabber1.getAudioChannels());

        

        

        

        

        try {

        recorder.start();

            recorder.setVideoCodec(a); //CODEC_ID_MPEG4 //CODEC_ID_MPEG1VIDEO

//             recorder.setFrameRate(grabber1.getFrameRate());

            

            

            recorder.setSampleFormat(grabber1.getSampleFormat());

            recorder.setSampleRate(grabber1.getSampleRate()); 

            

            

            

            recorder.setFrameRate(24);            

            recorder.setPixelFormat(PIX_FMT_YUV420P); //PIX_FMT_YUV420P

           

            

            

            Frame frame1;

            

            

            

            

            while ((frame1 = grabber1.grabFrame()) != null)

                     {

                

               

            

           for (int i=0;i<iplimage.length;i++)

              {

            recorder.record(frame1);

             

            recorder.record(iplimage[i]);

              }

           }

            recorder.stop();

            grabber1.stop();

           }

        catch (Exception e){

            e.printStackTrace();

          }

Errors:

08-30 15:35:49.400: W/System.err(1542): com.googlecode.javacv.FrameRecorder$Exception: Audio codec not found

08-30 15:35:49.400: W/System.err(1542):     at com.googlecode.javacv.FFmpegFrameRecorder.start(FFmpegFrameRecorder.java:340)
08-30 15:35:49.630: W/System.err(1542): com.googlecode.javacv.FrameRecorder$Exception: No video output stream (Is imageWidth > 0 && imageHeight > 0 and has start() been called?)
08-30 15:35:49.630: W/System.err(1542):     at com.googlecode.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:493)
08-30 15:35:49.630: W/System.err(1542):     at com.googlecode.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:489)
08-30 15:35:49.630: W/System.err(1542):     at com.googlecode.javacv.FrameRecorder.record(FrameRecorder.java:210)
08-30 15:35:49.630: W/System.err(1542):     at com.example.javacv_latest_update.MainActivity$ImageconvertsionAsyncTask.doInBackground(MainActivity.java:88)
08-30 15:35:49.630: W/System.err(1542):     at com.example.javacv_latest_update.MainActivity$ImageconvertsionAsyncTask.doInBackground(MainActivity.java:1)

[Samuel Audet]

On 12/13/2012 05:42 PM, Ram wrote:
> 08-30 15:35:49.400: W/System.err(1542): com.googlecode.javacv.FrameRecorder$Exception: Audio codec not found
> 08-30 15:35:49.400: W/System.err(1542): at com.googlecode.javacv.FFmpegFrameRecorder.start(FFmpegFrameRecorder.java:340)
> 08-30 15:35:49.630: W/System.err(1542): com.googlecode.javacv.FrameRecorder$Exception: No video output stream (Is imageWidth > 0 && imageHeight > 0 and has start() been called?)
> 08-30 15:35:49.630: W/System.err(1542): at com.googlecode.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:493)
> 08-30 15:35:49.630: W/System.err(1542): at com.googlecode.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:489)
> 08-30 15:35:49.630: W/System.err(1542): at com.googlecode.javacv.FrameRecorder.record(FrameRecorder.java:210)
> 08-30 15:35:49.630: W/System.err(1542): at com.example.javacv_latest_update.MainActivity$ImageconvertsionAsyncTask.doInBackground(MainActivity.java:88)
> 08-30 15:35:49.630: W/System.err(1542): at com.example.javacv_latest_update.MainActivity$ImageconvertsionAsyncTask.doInBackground(MainActivity.java:1)

It looks like you're trying to run an old version of
FFmpegFrameRecorder. Can you try again with JavaCV 0.3? thanks

Samuel

[Ram]

hi samuel,


        thanks for your reply.i am already using javacv0.3 only.in this version only i am getting error like that.i want to merge audio with video.please advice me.
thanks.

[Samuel Audet]

On 12/15/2012 10:57 PM, Ram wrote:
> hi samuel,
>
>
> thanks for your reply.i am already using javacv0.3 only.in this
> version only i am getting error like that.i want to merge audio with
> video.please advice me.

You may be trying to use JavaCV 0.3, but the line numbers in the
exception messages do not correspond to JavaCV 0.3, so what you are
actually executing is probably an older version of JavaCV. If you can
figure out how to make Android run the new version of JavaCV, you are
probably going to have better luck! Uninstalling and reinstalling
everything usually does the trick

[Samuel Audet]

Hello,

On 05/17/2013 11:18 PM, Ophir Levinson wrote:
> the problem is that when playing the produced video, the audio always
> finishes before the video (even though the sound mp3 file is a couple of
> minutes long. I would expect the final video to have full audio for the
> whole lengh of the video recorded (which is a about 10 seconds).

Maybe the final video frames aren't flushed properly:
http://code.google.com/p/javacv/issues/detail?id=302

Please try out the modifications here:
http://code.google.com/p/javacv/source/detail?r=9fab7522f5904904eaec4744e7f99b7bd8f48ac6

> 2. Is it possible to combine 2 audios with one video (having music
> (Audio 1), speech (Audio 2 ) , video ), is it possible to control the
> volume of the audio recorded?

FFmpeg probably has functionality for that, but it's not part of
FFmpegFrameRecorder, no. If you'd like to make a contribution, let me
know :)

Samuel

[Gui de Guinetik]

Hi, 

im using the latest version (0.5) and i also have the Audio Codec not found. my file is an mp3, here is some code:

FrameGrabber videoFrames = new FFmpegFrameGrabber(filePath + ".mp4");

FrameGrabber audioFrames = new FFmpegFrameGrabber(filePath + ".mp3");

videoFrames.setFormat("mp4");

audioFrames.setFormat("mp3");

videoFrames.start();

audioFrames.start();

FrameRecorder recorder = new FFmpegFrameRecorder(filePath + "_m.mp4",

videoFrames.getImageWidth(), videoFrames.getImageHeight(),

audioFrames.getAudioChannels());

recorder.setFrameRate(videoFrames.getFrameRate());

recorder.setSampleFormat(audioFrames.getSampleFormat());

recorder.setSampleRate(audioFrames.getSampleRate());

recorder.setFormat("mp4");

recorder.setAudioCodec(avcodec.AV_CODEC_ID_AC3);

recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);

recorder.start();

Frame frameVideo, frameAudio;

while(true)

{

frameVideo = videoFrames.grabFrame();

frameAudio = audioFrames.grabFrame();

if(frameVideo == null || frameAudio == null) break;

recorder.record(frameVideo);

recorder.record(frameAudio);

}

recorder.stop();

videoFrames.stop();

audioFrames.stop();

i think the version of the FFmpegFrameRecorder in my jar doesn't support mp3. i already have the lame lib in my project, so any way of accomplishing this merge?

thanks

[Samuel Audet]

On 07/24/2013 03:56 PM, Gui de Guinetik wrote:
> i think the version of the FFmpegFrameRecorder in my jar doesn't support mp3. i already have the lame lib in my project, so any way of accomplishing this merge?

That's not the problem. The problem is that you should try WITHOUT
calling setSampleFormat().

Samuel

[To Quang Ngoc]

I try this code below and merge two file successfully

 try {

            FrameGrabber grabber1 = new FFmpegFrameGrabber("/sdcard/Video/RCD_20131204_141146.mp4"); 

            FrameGrabber grabber2 = new FFmpegFrameGrabber("/sdcard/Video/tempAudio.mp3"); 

grabber1.start();

        grabber2.start(); 

        //Log.e("WIDTH",grabber2.getSampleRate()+"");

        recorder = new FFmpegFrameRecorder(ffmpeg_link, 

                grabber1.getImageWidth(), grabber1.getImageHeight(), 

                2); 

        recorder.setFormat("mp4");

        recorder.setFrameRate(grabber1.getFrameRate()); 

        //recorder.setSampleFormat(grabber2.getSampleFormat()); 

        recorder.setSampleRate(grabber2.getSampleRate()); 

        //recorder.setAudioCodec(avcodec.AV_CODEC_ID_AC3);

//recorder.setVideoCodec(avcodec.AV_CODEC_ID_MPEG4);

        recorder.start(); 

        Frame frame1, frame2 = null; 

        while ((frame1 = grabber1.grabFrame()) != null || 

              (frame2 = grabber2.grabFrame()) != null) { 

            recorder.record(frame1); 

            recorder.record(frame2); 

        } 

        recorder.stop(); 

        grabber1.stop(); 

        grabber2.stop(); 

        Log.e(LOG_TAG, "recorder initialize success");

} catch (com.googlecode.javacv.FrameGrabber.Exception e) {

// TODO Auto-generated catch block

e.printStackTrace();

} catch(Exception e1){

e1.printStackTrace();

}

Vào 17:51:23 UTC+7 Thứ ba, ngày 24 tháng bảy năm 2012, RAJESH K đã viết:

[Twos]

Hi,

    If I use && for below while, the audio can only be heard for only 3~4 seconds... What happened?

    while ((frame1 = grabber1.grabFrame()) != null && 

              (frame2 = grabber2.grabFrame()) != null) { 

            recorder.record(frame1); 

            recorder.record(frame2); 

        } 

在 2013年12月5日星期四UTC+8下午12时17分22秒，To Quang Ngoc写道：

[Samuel Audet]

On 08/28/2014 05:54 PM, Twos wrote:
> Hi,
> If I use && for below while, the audio can only be heard for only 3~4 seconds... What happened?
>
> while ((frame1 = grabber1.grabFrame()) != null &&
>
> (frame2 = grabber2.grabFrame()) != null) {
>
> recorder.record(frame1);
>
> recorder.record(frame2);
>
> }

Try to use two while loops, that should work better.

Samuel

[Illia Kaluzhniy]

I did like in my code in the bottom, and as a result i have 40 sec video ( 20 sec video + 20 sec audio). I want it one on top of other! Do any1 know how to mix them up? maybe there is startFromBeginning or smth?

ublic class AddAudioToVideoAsyncTask extends AsyncTask<Void, Void, Void> {


    String videoFromFilePath;
    String videoToFilePath;
    double frameRate;

    FFmpegFrameRecorder recorder;
    FFmpegFrameGrabber grabberFirst;
    FFmpegFrameGrabber grabberSecond;

    public AddAudioToVideoAsyncTask(String audioName){



        Log.d("MY", "WORKING FINE ON THE BEGINNING!");

        File workingFile = Environment
                .getExternalStoragePublicDirectory(Environment.DIRECTORY_DCIM);

        videoFromFilePath = workingFile + "/myvideo.3gp";
        videoToFilePath = workingFile + "/output.mp4";

        Log.d("MY_TAG", "result video - " + videoToFilePath);
        Log.d("MY_TAG", "source video - " + videoFromFilePath);

        grabberFirst = new FFmpegFrameGrabber(videoFromFilePath);
        grabberSecond = new FFmpegFrameGrabber(audioName);
    }

    @Override
    protected void onPreExecute() {
        super.onPreExecute();


        try {
            grabberFirst.start();
        } catch (FrameGrabber.Exception fge) {
            fge.printStackTrace();
        }
        try {
            grabberSecond.start();
        } catch (FrameGrabber.Exception fge) {
            fge.printStackTrace();
        }

        Log.d("MY", "STILL ALIVE AFTER GRABBERFIRST.START");


        frameRate = grabberFirst.getFrameRate();
        recorder = new FFmpegFrameRecorder(videoToFilePath, grabberFirst.getImageWidth(), grabberFirst.getImageHeight(), grabberSecond.getAudioChannels());




        try {
            recorder.start();
        } catch (FrameRecorder.Exception e) {
            e.printStackTrace();
        }


    }

    @Override
    protected void onPostExecute(Void aVoid) {
        super.onPostExecute(aVoid);
        try {
            recorder.release();
            grabberFirst.release();
            grabberSecond.release();
        } catch (FrameGrabber.Exception fge) {
            fge.printStackTrace();
        } catch (FrameRecorder.Exception fre) {
            fre.printStackTrace();
        }

        //ButtonClickActions.returnFromAddMusicFragment((MainActivity)act);
    }

    @Override
    protected Void doInBackground(Void... params) {
        Log.d("MY", "DOING GREAT! DOINBACKGROUND!");

        Frame frame1 = null;
        Frame frame2 = null;

        try {
            while ((frame1 = grabberFirst.grabFrame()) != null ||
                    (frame2 = grabberSecond.grabFrame()) != null) {



                recorder.record(frame1);
                recorder.record(frame2);

            }

            recorder.stop();
            grabberSecond.stop();
            grabberFirst.stop();

        } catch (FrameGrabber.Exception fge) {
            fge.printStackTrace();
        } catch (FrameRecorder.Exception fre) {
            fre.printStackTrace();
        }

        Log.d("MY", "I am done with merging");

        return null;
    }

} 

[Samuel Audet]

On 06/12/2015 10:29 PM, Illia Kaluzhniy wrote:
> I did like in my code in the bottom, and as a result i have 40 sec video (
> 20 sec video + 20 sec audio). I want it one on top of other! Do any1 know
> how to mix them up? maybe there is startFromBeginning or smth?

Do something like shown in this sample and I think this will do what you
want:
https://github.com/bytedeco/javacv/blob/master/samples/RecordActivity.java

Samuel

[Oben IŞIK]

Hello,

In my Samsung Galaxy S3 everything works fine with that code.

But in my huawei phone same code gets UnsupportedOperationException.

It can record audio easily, but the problem is with video.

In FFmpegFrameRecorder class in recordImage method there is a line below. The yellow highlighted side gave error. Please help me...

BytePointer data = image[0] instanceof ByteBuffer?new BytePointer((ByteBuffer)image[0].position(0)):new BytePointer(new Pointer(image[0].position(0)));

@Override
protected Void doInBackground(Void... params) {

    finalPath = Util.createFinalPath();

    try {
        FrameGrabber videoFrames = new FFmpegFrameGrabber(videoPath);
        FrameGrabber audioFrames = new FFmpegFrameGrabber(audioPath);

        videoFrames.start();
        audioFrames.start();

        FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(finalPath,
                videoFrames.getImageWidth(), videoFrames.getImageHeight(),
                audioFrames.getAudioChannels());

        recorder.setFrameRate(videoFrames.getFrameRate());
        recorder.setSampleRate(audioFrames.getSampleRate());
        //recorder.setSampleFormat(videoFrames.getSampleFormat());
        recorder.setFormat(videoFrames.getFormat());
        recorder.start();

        Frame frameVideo, frameAudio;

        boolean bothNull = false;

        while(true)
        {

            frameVideo = videoFrames.grabFrame();
            frameAudio = audioFrames.grabFrame();

            bothNull = frameVideo == null && frameAudio == null;

            if(bothNull){
                Util.log("frameVideo = " + frameVideo + ", frameAudio = " + frameAudio);
                break;
            }

            recorder.record(frameAudio);
            recorder.record(frameVideo);

        }

        recorder.stop();
        videoFrames.stop();
        audioFrames.stop();

    } catch (FrameGrabber.Exception e) {
        e.printStackTrace();
    } catch (FrameRecorder.Exception e) {
        e.printStackTrace();
    } catch (UnsupportedOperationException e){
        e.printStackTrace();

    }

    return null;
}

5 Aralık 2013 Perşembe 06:17:22 UTC+2 tarihinde To Quang Ngoc yazdı:

[Avani Bhatnagar]

Hey,

 I tried doing this, but it doesn't work.

I added this to my existing class which converts a sequence of images to a video. I created a method to add the audio using the code you provided. But my output still doesn't have the audio. Nothing changes in the output. Any help is appreciated! :)

On Friday, 27 July 2012 21:09:48 UTC-7, Samuel Audet wrote:

On 2012-07-24 19:51, RAJESH K wrote:
> Hi All ,
>
> I want to bind an audio with an video file using javacv/Android platform .
> If any one has idea or any experience in that please advice me to come
> out .

Something like this should work:

     FrameGrabber grabber1 = new FFmpegFrameGrabber("video.avi");
     FrameGrabber grabber2 = new FFmpegFrameGrabber("audio.mp3");
     grabber1.start();
     grabber2.start();
     FrameRecorder recorder = new FFmpegFrameRecorder("output.mp4",
             grabber1.getImageWidth(), grabber1.getImageHeight(),
             grabber2.getAudioChannels());
     recorder.setFrameRate(grabber1.getFrameRate());
     recorder.setSampleFormat(grabber2.getSampleFormat());
     recorder.setSampleRate(grabber2.getSampleRate());
     recorder.start();
     Frame frame1, frame2;

     while ((frame1 = grabber1.grabFrame()) != null ||
            (frame2 = grabber2.grabFrame()) != null) {
         recorder.record(frame1);
         recorder.record(frame2);
     }
     recorder.stop();
     grabber1.stop();
     grabber2.stop();

Samuel

[Avani Bhatnagar]

Hey were you able to figure this out?

On Tuesday, 24 July 2012 03:51:23 UTC-7, Rajesh Krishnamoorthy wrote:

Hi All ,

I want to bind an audio with an video file using javacv/Android platform .
If any one has idea or any experience in that please advice me to come out .

[Samuel Audet]

Could you reduce your existing code into a few lines of code that can reproduce this issue?

2018年7月12日(木) 7:09 Avani Bhatnagar <avanibha...@gmail.com>:

--

---
You received this message because you are subscribed to the Google Groups "javacv" group.
To unsubscribe from this group and stop receiving emails from it, send an email to javacv+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Avani Bhatnagar]

public static void addMP3toMovie() throws FrameGrabber.Exception, FrameRecorder.Exception {
    FrameGrabber grabber1 = new FFmpegFrameGrabber(vidPath);
    FrameGrabber grabber2 = new FFmpegFrameGrabber(audioPath);
    grabber1.start();
    grabber2.start();
    FrameRecorder recorder = new FFmpegFrameRecorder(outputPath,
            grabber1.getImageWidth(), grabber1.getImageHeight(),
            grabber2.getAudioChannels());
    recorder.setFormat("mp4");
    recorder.setVideoQuality(1);
    recorder.setFrameRate(grabber1.getFrameRate());
    recorder.setSampleFormat(grabber2.getSampleFormat());
    recorder.setSampleRate(grabber2.getSampleRate());
    recorder.start();   //I get an error here 

    while ((frame1 = grabber1.grabFrame()) != null ||

            ((frame2 = grabber2.grabFrame()) != null)) {

        recorder.record(frame1);
        recorder.record(frame2);
    }
    recorder.stop();
    grabber1.stop();
    grabber2.stop();

} 

These are the errors I get: 

Exception in thread "main" org.bytedeco.javacv.FrameRecorder$Exception: avcodec_open2() error -22: Could not open audio codec.

at org.bytedeco.javacv.FFmpegFrameRecorder.startUnsafe(FFmpegFrameRecorder.java:581)

at org.bytedeco.javacv.FFmpegFrameRecorder.start(FFmpegFrameRecorder.java:288)

The code works fine till the call to my video conversion method. But when I call this method to add audio to the video, it gives me these errors. 

Does it have something to do with the audio format?

[Samuel Audet]

Looks like the default codec (AV_CODEC_ID_AAC) doesn't support that sample format.
Try not to set it to use the default, and see what that does.

[Avani Bhatnagar]

Hey,

I removed the setSampleFormat, and made a few changes. Now the audio has been added to my video. According to the frame rate I set, the video was only 20 seconds since that's how long my audio is. But when I added the audio, the length of the video increased and so the audio is ending before the video. Any ideas on how to resolve that?

ALSO, thanks a TON for the above help! :) 

[Samuel Audet]

Try to call recorder.setTimestamp(grabber.getTimestamp()), that should help,
but FFmpeg doesn't usually support going back in time, so if you get any errors,
make sure to never decrease the value you give to setTimestamp().

Samuel

[Avani Bhatnagar]

I got that fixed. Thanks a lot!! I was wondering if you have any ideas on how to add text on different video frames? 

[Samuel Audet]

Great!

There's a "drawText" filter for that, some examples in issues here:
https://github.com/bytedeco/javacv/issues/254
https://github.com/bytedeco/javacv/issues/164#issuecomment-134609198

[Samuel Audet]

Works fine from these tests:
https://github.com/bytedeco/javacv/blob/master/platform/src/test/java/org/bytedeco/javacv/FrameFilterTest.java

On 07/20/2018 01:10 AM, Avani Bhatnagar wrote:
> I'm unable to do that because I can't seem to import FFmpegFrameFilter.
>
> I get Can't resolve 'FFmpegFrameFilter'...

[Avani Bhatnagar]

Hey,

To be able to use the FFmpegFrameFilter, I have to update my javacv version from 1.0 to 1.4 in the dependencies and on doing so everything else that I have in my java class breaks...
I have a lot of features implemented which I don't wanna lose.. And when I change the version back to 1.0, it can't resolve FFmpegFrameFilter

[Avani Bhatnagar]

It works fine when I don't have the FFmpegFrameFilter, but when I add that and hence its dependencies, I get these errors:

[Samuel Audet]

What breaks when upgrading from 1.0? Maybe we can fix that instead.

2018年7月21日(土) 4:26 Avani Bhatnagar <avanibha...@gmail.com>:

[Avani Bhatnagar]

When I have the version 1.0, my java class converts a series of images to a video and then adds audio to create a movie. It works perfectly so far:

I am trying to add text to it as well. So when I update to version 1.4, to be able to use the FFmpegFrameFilter, then this happens:

And two more in the method which converts images to movie at this line: FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(vidPath,640,720);

And another one at the call for that method in main. 


Is there a way to use the drawtext filter without updating to version 1.4 and just using 1.0 instead?

[Avani Bhatnagar]

Above is a complete screenshot which shows the exception and the internal calls. 

[Samuel Audet]

Have you included the "javacv-platform" dependency in your build, and not just "javacv"?

2018年7月24日(火) 0:29 Avani Bhatnagar <avanibha...@gmail.com>:

[Avani Bhatnagar]

Yes I did. It still didn't work.. :(

I acually moved forward without the text for now. 
The problem I'm facing is that the video works on the server, but on the browser I can just hear the audio..
According to this post, https://stackoverflow.com/questions/39229930/amazon-s3-mp4-video-not-playing-in-html-video-tag, I should change the video codec to AVC codec (aka H.264). Currently it is this:

recorder.setVideoCodec(avcodec.AV_CODEC_ID_MPEG4);

How should I change it?

[Avani Bhatnagar]

I changed it to 

recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);

But it still doesn't show on some browsers including Chrome..

Did I change the video codec correctly?

Do you have any idea about how to make the video and audio both show. Currently I'm using the HTML video tag. If there is something else that can make it work, I can try that..

[Avani Bhatnagar]

I changed the codec to  only the video.. When I change the codec in the method that adds the audio to the video like this: 

recorder.setFormat("mp4");
recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
recorder.setVideoQuality(1);

Then a 0Kb file was created and this happened...

Output #0, mp4, to 'onlyVid2290.mp4':

    Stream #0:0: Video: h264 (libx264), yuv420p, 640x720, q=-1--1, 9 kb/s, 0.34 tbn, 0.34 tbc

[libx264 @ 000000001bb56e00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2

[libx264 @ 000000001bb56e00] profile High 4:4:4 Predictive, level 3.1, 4:2:0 8-bit

[libx264 @ 000000001bb56e00] 264 - core 146 r2538 121396c - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=0 mixed_ref=1 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=0 chroma_qp_offset=0 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=2 keyint=250 keyint_min=1 scenecut=40 intra_refresh=0 rc=cqp mbtree=0 qp=0

[swscaler @ 00000000213759a0] Warning: data is not aligned! This can lead to a speedloss

[libx264 @ 000000001bb56e00] frame I:4     Avg QP: 0.00  size:164748

[libx264 @ 000000001bb56e00] mb I  I16..4: 57.0%  0.0% 43.0%

[libx264 @ 000000001bb56e00] 8x8 transform intra:0.0%

[libx264 @ 000000001bb56e00] coded y,uvDC,uvAC intra: 95.2% 93.9% 93.8%

[libx264 @ 000000001bb56e00] i16 v,h,dc,p: 61% 37%  1%  0%

[libx264 @ 000000001bb56e00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 46% 37%  2%  2%  4%  3%  3%  2%  2%

[libx264 @ 000000001bb56e00] i8c dc,h,v,p:  6% 42% 53%  0%

[libx264 @ 000000001bb56e00] kb/s:448.11

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'onlyVid2290.mp4':

  Metadata:

    major_brand     : isom

    minor_version   : 512

    compatible_brands: isomiso2avc1mp41

    encoder         : Lavf56.37.100

  Duration: 00:00:11.77, start: 0.000000, bitrate: 449 kb/s

    Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv420p, 640x720, 448 kb/s, 0.34 fps, 0.34 tbr, 17408 tbn, 0.68 tbc (default)

    Metadata:

      handler_name    : VideoHandler

[aac @ 000000001bc21680] Estimating duration from bitrate, this may be inaccurate

Input #0, aac, from 'https://s3-us-west-2.amazonaws.com/creatics-static-resources/HelloSunshine_20SecondVersion_Final.aac':

  Duration: 00:00:16.79, bitrate: 152 kb/s

    Stream #0:0: Audio: aac (LC), 44100 Hz, stereo, fltp, 152 kb/s

Output #0, mp4, to 'Movie2290.mp4':

    Stream #0:0: Video: h264 (libx264), yuv420p, 640x720, q=-1--1, 400 kb/s, 0.34 tbn, 0.34 tbc

    Stream #0:1: Audio: aac (libvo_aacenc), 44100 Hz, stereo, s16, 64 kb/s

[libx264 @ 00000000223366c0] -qscale is ignored, -crf is recommended.

[libx264 @ 00000000223366c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2

[libx264 @ 00000000223366c0] profile High, level 3.1

[libx264 @ 00000000223366c0] 264 - core 146 r2538 121396c - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=1 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00

Output #0, mp4, to 'onlyVid2290.mp4':

    Stream #0:0: Video: h264 (libx264), yuv420p, 640x720, q=-1--1, 9 kb/s, 0.34 tbn, 0.34 tbc

[libx264 @ 000000002a883ea0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2

[libx264 @ 000000002a883ea0] profile High 4:4:4 Predictive, level 3.1, 4:2:0 8-bit

[libx264 @ 000000002a883ea0] 264 - core 146 r2538 121396c - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=0 mixed_ref=1 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=0 chroma_qp_offset=0 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=

[Samuel Audet]

It might not work with high profiles with such high quality settings. Try a lower quality (higher number).

2018年8月8日(水) 7:46 Avani Bhatnagar <avanibha...@gmail.com>:

[Avani Bhatnagar]

I tried higher numbers, but it still just creates a .mp4 file, but it is empty..

[Avani Bhatnagar]

Can I convert it to .ogv instead of .mp4 or do both since a lot of browsers support .ogv?

[Samuel Audet]

If the file is empty, it's probably because you just forgot to call stop() somewhere.

Sure, we can call setFormat("ogv"), but we would probably need to build FFmpeg with libtheora first...

[Avani Bhatnagar]

This is the method:

    public static void addMP3toMovie() throws FrameGrabber.Exception, FrameRecorder.Exception {

        FrameGrabber grabber1 = new FFmpegFrameGrabber(vidPath2);

        FrameGrabber grabber2 = new FFmpegFrameGrabber(audioPath);

        grabber1.start();

        grabber2.start();

        FrameRecorder recorder = new FFmpegFrameRecorder(outputPath,

                grabber1.getImageWidth(), grabber1.getImageHeight(),

                grabber2.getAudioChannels());

        recorder.setFormat("mp4");

        recorder.setVideoQuality(7);

        recorder.setFrameRate(grabber1.getFrameRate());

        recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);

        recorder.setSampleFormat(AV_SAMPLE_FMT_S16);

        recorder.start();

        while ((frame1 = grabber1.grabFrame()) != null ||

                ((frame2 = grabber2.grabFrame()) != null)) {

            recorder.record(frame1);

            recorder.record(frame2);

        }

        recorder.stop();

        grabber1.stop();

        grabber2.stop();

    }

the vidPath is just a video. And this method is supposed to add an .aac audio file to it. It works with the MPEG4 Codec..

[Samuel Audet]

Ok, if it does the same thing with JavaCV 1.4.2, could you post an issue on GitHub:
    https://github.com/bytedeco/javacv/issues
And attach sample files to reproduce this bug?
Then I'll be able to understand better what's happening.
Thanks for your time!

[Samuel Audet]

JavaCV 1.4.2 comes with more JAR files, make sure to add them all to your class path and it will work.

On 08/10/2018 05:35 AM, Avani Bhatnagar wrote:

I'm actually using version 1.0 because when I change it to 1.4.2, everything else stops working: 

    Exception in thread "main" java.lang.reflect.InvocationTargetException

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:497)

at com.intellij.rt.execution.CommandLineWrapper.main(CommandLineWrapper.java:67)

Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.bytedeco.javacpp.avutil

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:348)

at org.bytedeco.javacpp.Loader.load(Loader.java:941)

at org.bytedeco.javacpp.Loader.load(Loader.java:882)

at org.bytedeco.javacpp.avformat$Write_packet_Pointer_BytePointer_int.<clinit>(avformat.java:640)

at org.bytedeco.javacv.FFmpegFrameRecorder.<clinit>(FFmpegFrameRecorder.java:310)

at com.creatics.web.IAmVideoController.convertJPGtoMovie(IAmVideoController.java:113)    ///the line 113 is:   FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(file,640,720);

at com.creatics.web.IAmVideoController.main(IAmVideoController.java:50) //this is just a call to this method

... 5 more

[Avani Bhatnagar]

I added the dependency like this:

That should add everything right?

<dependency>
   <groupId>org.bytedeco</groupId>
   <artifactId>javacv-platform</artifactId>
   <version>1.4.2</version>
</dependency>

[Avani Bhatnagar]

I resolved the other errors, just this one now...
Error caused at recorder.record(frame1);


Caused by: org.bytedeco.javacv.FrameRecorder$Exception: avcodec_encode_video2() error -542398533: Could not encode video packet.

[Samuel Audet]

Might be caused by this line:

    recorder.setSampleFormat(AV_SAMPLE_FMT_S16);

Support for that has been removed from FFmpeg, and we never needed that line anyway, so you shouldn't use it.

[Avani Bhatnagar]

I even removed it, but it still causes this error..

Can I change the codec to Vorbis or vp8?

[Avani Bhatnagar]

My audio and video both work with MPEG4 codec, so is there a way I can make that entire video with MPEG codec and then write another method to convert the codec to Vorbis or vp8 or H264. (Preferably H264)

[Samuel Audet]

Yes, FFmpeg supports those codecs...

In any case, could you file an issue on GitHub and attach a sample file that causes that error? I'd like to fix that if possible, thanks!
https://github.com/bytedeco/javacv/issues

[Samuel Audet]

It might just be libx264 that doesn't support threads well on your platform:
https://github.com/bytedeco/javacv/issues/745#issuecomment-336786816
Try to setVideoOption("threads", "1");

[Avani Bhatnagar]

Just filed an issue.