FFmpegFrameRecorder produces low framerate video

[Szabó Barna]

Hi Everyone!

I must start with the fact that javacv helped me a lot to create a very responsive vide recorder activity, it works great, BUT I have a problem with the produced vide FPS, is to slow 15fps for resolution 480x360.

- I downloaded the newest javacv and dependencies

- I initialized the camera preview to 30fps

- I initialized the recorder to 30fps

                recorder = new FFmpegFrameRecorder(ffmpeg_link,480,360, 1);

                recorder.setFormat("mp4");

recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);

recorder.setVideoCodec(avcodec.AV_CODEC_ID_MPEG4);// AV_CODEC_ID_FLV1

recorder.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);

recorder.setSampleRate(44100);

recorder.setFrameRate(30);

recorder.setVideoBitrate(800000);

- I record the preview data[] just like in the RecordActivity example : 

public void onPreviewFrame(byte[] data, Camera camera) {

...

                                        yuvIplimage.getByteBuffer().put(data);

long t = 1000 * (System.currentTimeMillis() - pauseDelay - startTime);

if (t > recorder.getTimestamp()) {

recorder.setTimestamp(t);

}

recorder.record(yuvIplimage);

....

}

-> Result is a qualityt good video, but with the frame rate only ~12-15fps, which is bed for me. I need at least 24fps

Tests : 

- the onPreviewFrame() gives me 30fps if I remove this line of code "recorder.record(yuvIplimage);"

- if "recorder.record(yuvIplimage);" is working, than it slows down the processing time to half, so only 15fps are recorded.

- I measured and the "recorder.record(yuvIplimage);" is slowing down.

If the device isn't fast enough to save all frames, we got this low fps video, byt I use a Samsung Galaxy S4, so I don't think that it cannot handle those processing.

If I lower the desired recording resolution to 320x240, it will produce a 25fps video.

The conclusion is that the FFmpegFrameRecorder is slow. 

Questions : 

- what is worng with my code ? my setting are not right ? (in above code snippet)

- how can I keep 480x360 and to have ~ 25fps ?

Please help me.

Thank you

[Szabó Barna]

Updates : 

My oppinions is that the "record" must be executed on separate thread after we stop the recording, and on that separate thread we must synchronize the audio with the video and record it.

Has anybody a workaroudn for this ?

Thank you

[Jose Gómez]

During my tests in a PC, the fastest format was MPEG4, which is the video format you are using.

However, if the phone has a hardware H264 encoder, that might actually speed things up.

Can you try changing the format? For example, with AV_CODEC_ID_H264.

[Szabó Barna]

Thanks Jose!

I tested with H264 too, but it is even slower :( I repeat, if I use a low resolution preview and format 320x240 it works great, but in 480x360 it is skipping frames, because the record(...) ,method tekso too much time.

In applications like "Vine" for Android, which is using  the same arhitecture, they managed to record MP4 on 480x480, by using AsyncTask for record(...) method, this means that they cache all data from onPreviewFrame(...) and o a separate thread are writting the file. This is tricky because you must keep synced teh audio with video, and also teh PAUSE timestamp.

I need only javacv because I need to pause the recording and with Android MediaRecorder we cannot, and I don't want to use mp4parser for concatenating videos.

Thanks

[Klommenz]

Hey Szabo,

I wonder - did you find a solution for this problem? I am running into the same problem. Ideas?

Thanks,

Klommenz

[Samuel Audet]

Hello,

Something like Vine records only 6 seconds, so with a couple of tricks
it's entirely possible to save the whole content in memory, and then
have it encode slowly in the background...

Samuel

On 10/04/2013 11:09 PM, Klommenz wrote:
> Hey Szabo,
>
> I wonder - did you find a solution for this problem? I am running into
> the same problem. Ideas?
>
> Thanks,
> Klommenz
>

[Ahmed Nawaz]

Hi Everyone.

I am also facing same problem on 640x480 with AV_CODEC_ID_H264 I am getting only 2~5 fps. Using Galaxy S3.

Code is following.

recorder = new FFmpegFrameRecorder(mFilePath, 640,480, 1);

recorder.setSampleRate(sampleAudioRateInHz);

recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);

recorder.setFormat("mp4");

recorder.setFrameRate(30);

If someone solved the issue please share the code. 

[Marco Burrafato]

Hi everyone,

Szabo, Samuel, what do you mean with "Vine did it with AsyncTask"? Could you link me to some sample code? I'm trying to do the same thing, but I keep getting a black video (audio is recording correctly, though)

[Naeem Shah]

@Szabo Barna!

I have exactly same issue, Did you resolve the issue Please help me.

[Samuel Audet]

Try to use the "ultrafast preset" by calling:

recorder.setVideoCodec(AV_CODEC_ID_H264);
recorder.setVideoOption("preset", "ultrafast");

Samuel