Simple ffmpegFrameRecorder error

[rudba556]

I am trying to learn JavaCV by making simple projects,  I want to convert images displayed in a imageview to a video using FFmpegFrameRecorder, and add audio using mic as the source

 I have imported JavaCV to my project(javacv.jar, javacpp.jar, opencv.jar , openv-android-arm.jar, ffmpeg.jar, ffmpeg-android-arm.jar; added them under project structure/dependencies)

I am using this code https://github.com/gderaco/JavaCVTest/tree/masterI get error as "record (org.bytedeco.javacv.Frame) in FFmpegFrameRecorder cannot be applied to (org.bytedeco.javacpp.opencv_coreiplImage)"

 where have i gone wrong?

CODE:
import..

import org.bytedeco.javacpp.opencv_core;
import org.bytedeco.javacv.FFmpegFrameRecorder;

import static org.bytedeco.javacpp.opencv_imgcodecs.cvLoadImage;

public class MyAndroidAppActivity extends Activity {

ImageView image;

@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);

addListenerOnButton();

opencv_core.IplImage img = cvLoadImage("ImageView");

FFmpegFrameRecorder recorder = new                
FFmpegFrameRecorder("/sdcard/test.mpeg",200,150);

    try {
        recorder.setFrameRate(30);
        recorder.start();

        for (int i=0;i<100;i++)
        {
            recorder.record(img);
        }
        recorder.stop();
    }
    catch (Exception e){
        e.printStackTrace();
    }

}

public void addListenerOnButton() {

image = (ImageView) findViewById(R.id.imageView1);
button = (Button) findViewById(R.id.btn1);

button.setOnClickListener(new OnClickListener() {
    @Override
    public void onClick(View arg0) {
        image.setImageResource(R.drawable.image1);
    }
});

//...........

button = (Button) findViewById(R.id.btn10);

button.setOnClickListener(new OnClickListener() {
    @Override
    public void onClick(View arg0) {
        image.setImageResource(R.drawable.image10);
    }

});

}

 

[rudba556]

Also i am not getting the exact code to be put at opencv_core.IplImage img = cvLoadImage(?); the images in the image view change as the buttons are pressed.

Please help in compiling my first JavaCV app..

Thanks...

[Samuel Audet]

On 11/12/2015 03:51 PM, rudba556 wrote:
> I am trying to learn JavaCV by making simple projects, I want to convert
> images displayed in a imageview to a video using FFmpegFrameRecorder, and
> add audio using mic as the source
>
> I have imported JavaCV to my project(javacv.jar, javacpp.jar, opencv.jar ,
> openv-android-arm.jar, ffmpeg.jar, ffmpeg-android-arm.jar; added them under
> project structure/dependencies)
>
> I am using this code https://github.com/gderaco/JavaCVTest/tree/masterI get
> error as "record (org.bytedeco.javacv.Frame) in FFmpegFrameRecorder cannot
> be applied to (org.bytedeco.javacpp.opencv_coreiplImage)"
>
> where have i gone wrong?

That sample code wasn't updated for recent versions of JavaCV. Please
read these posts for some information:
http://bytedeco.org/news/2015/04/04/javacv-frame-converters/
http://bytedeco.org/news/2015/07/12/now-apache-license/

Let me know if you still have some questions after that! Thanks

Samuel

[rudba556]

Thanks for your Valuable reply, for learning sake i just need latest example code for converting displayed image into a video...

   Thanks. 

 

[Samuel Audet]

Check out the Demo class in the README.md file then. That's basically what it does. :)

Samuel

2015/11/14 20:16 "rudba556" <abdu...@gmail.com>:

Thanks for your Valuable reply, for learning sake i just need latest example code for converting displayed image into a video...

   Thanks. 

 

--

---
You received this message because you are subscribed to the Google Groups "javacv" group.
To unsubscribe from this group and stop receiving emails from it, send an email to javacv+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[rudba556]

Thanks again :-)

[rudba556]

I am not able to understand why it's trying to open the file which actually is to be created...? Where have I gone wrong in coding?

[rudba556]

I tried with following code :

import

..

import org.bytedeco.javacv.FFmpegFrameRecorder;
import org.bytedeco.javacv.Frame;
import org.bytedeco.javacpp.avcodec;

...

public ImageView image;
Button play,stop,record;
private Buffer[] yuvImage = null;

...

int imageWidth = 500;
int imageHeight = 320;
final FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(Environment.getExternalStorageDirectory().toString()+ "recording.mp4", imageWidth, imageHeight);
final Frame frame = new Frame();
frame.image = yuvImage;

record.setOnClickListener(new View.OnClickListener() {
    @Override
    public void onClick(View v) {


        try {
            recorder.setFrameRate(5);
            recorder.setVideoCodec(avcodec.AV_CODEC_ID_MPEG4);
            recorder.setFormat("mp4");
            recorder.setVideoBitrate(400000);
            recorder.setPixelFormat(2); //PIX_FMT_YUV420P

            recorder.start();



            for (int i=0;i<100;i++)
            {

                recorder.record(frame);
            }
        }
        catch (Exception e){
            e.printStackTrace();
        }

        record.setEnabled(false);
        stop.setEnabled(true);

        Toast.makeText(getApplicationContext(), "Recording started", Toast.LENGTH_LONG).show();
    }
});

stop.setOnClickListener(new View.OnClickListener() {
    @Override
    public void onClick(View v) {
        try {

            recorder.stop();
        } catch (Exception e) {
            e.printStackTrace();
        }

        stop.setEnabled(false);
        play.setEnabled(true);

        Toast.makeText(getApplicationContext(), "Video recorded successfully", Toast.LENGTH_LONG).show();
    }
});

Error:

11-21 23:20:19.516   W/art﹕ Suspending all threads took: 22.574ms
11-21 23:20:20.550   W/System.err﹕ org.bytedeco.javacv.FrameRecorder$Exception: avio_open error() error -13: Could not open '/storage/emulated/0recording.mp4'
11-21 23:20:20.550   W/System.err﹕ at org.bytedeco.javacv.FFmpegFrameRecorder.startUnsafe(FFmpegFrameRecorder.java:639)
11-21 23:20:20.550   W/System.err﹕ at org.bytedeco.javacv.FFmpegFrameRecorder.start(FFmpegFrameRecorder.java:288)
11-21 23:20:20.550   W/System.err﹕ at com.examples.videomaker.MainActivity$3.onClick(MainActivity.java:85)
11-21 23:20:20.550   W/System.err﹕ at android.view.View.performClick(View.java:4789)
11-21 23:20:20.550   W/System.err﹕ at android.view.View$PerformClick.run(View.java:19881)
11-21 23:20:20.550   W/System.err﹕ at android.os.Handler.handleCallback(Handler.java:739)
11-21 23:20:20.550   W/System.err﹕ at android.os.Handler.dispatchMessage(Handler.java:95)
11-21 23:20:20.550   W/System.err﹕ at android.os.Looper.loop(Looper.java:135)
11-21 23:20:20.550   W/System.err﹕ at android.app.ActivityThread.main(ActivityThread.java:5293)
11-21 23:20:20.550   W/System.err﹕ at java.lang.reflect.Method.invoke(Native Method)
11-21 23:20:20.550   W/System.err﹕ at java.lang.reflect.Method.invoke(Method.java:372)
11-21 23:20:20.550   W/System.err﹕ at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:904)
11-21 23:20:20.550   W/System.err﹕ at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:699)

[Samuel Audet]

On 11/20/2015 03:00 AM, rudba556 wrote:
> W/System.err﹕ org.bytedeco.javacv.FrameRecorder$Exception: avio_open
> error() error -13: Could not open '/storage/emulated/0recording.mp4'

This probably just means that you forgot to give your application
permission to write in /storage/. Are you sure you've given it permission?

Also, could you try to call FFmpegLogCallback.set() during
initialization? It should give us more messages in the log to debug
this, thanks!

Samuel

[rudba556]

Thanks for your valuable reply sir;

I had added that permission already, after i added (FFmpegFrameRecorder.exception e) to (exception e) that error has gone , 

I have added FFmpegLogCallback.set() at initiallization.

but now it comes with new error :((

749-28802/com.arapps.actionsmileys I/OpenGLRenderer﹕ Initialized EGL, version 1.4

11-23 12:36:56.579  28749-28802/com.example.Videomaker D/OpenGLRenderer﹕ Enabling debug mode 0

11-23 12:36:56.690  28749-28749/com.example.Videomaker I/Timeline﹕ Timeline: Activity_idle id: android.os.BinderProxy@36ad619e time:5108432

11-23 12:36:59.063  28749-28749/com.example.Videomaker I/System.out﹕ Output #0, mp4, to '/storage/emulated/0recording.mp4':

11-23 12:36:59.065  28749-28749/com.example.Videomaker I/System.out﹕ Stream #0:0

11-23 12:36:59.066  28749-28749/com.example.Videomaker I/System.out﹕ : Video: mpeg4, rgb24, 512x320, q=2-31, 400 kb/s

11-23 12:36:59.066  28749-28749/com.example.Videomaker I/System.out﹕ ,

11-23 12:36:59.066  28749-28749/com.example.Videomaker I/System.out﹕ 5 tbn,

11-23 12:36:59.067  28749-28749/com.example.Videomaker I/System.out﹕ 5 tbc

11-23 12:36:59.067  28749-28749/com.example.Videomaker I/System.out﹕ [ 11-23 12:36:59.068 28749:28749 I/System.out ]

    Stream #0:1

11-23 12:36:59.068  28749-28749/com.example.Videomaker I/System.out﹕ : Audio: aac, 44100 Hz, 44100 channels, fltp, 64 kb/s

11-23 12:36:59.068  28749-28749/com.example.Videomaker I/System.out﹕ [ 11-23 12:36:59.070 28749:28749 W/System.err ]

    Error: [mpeg4 @ 0xb4a36000] Specified pixel format rgb24 is invalid or not supported

11-23 12:36:59.072  28749-28749/com.example.Videomaker W/System.err﹕ org.bytedeco.javacv.FrameRecorder$Exception: avcodec_open2()

[Samuel Audet]

So, if you want to encode using the the rgb24 pixel format, try to
setVideoCodecName("libx264rgb"):
https://code.google.com/p/javacv/issues/detail?id=369

Samuel

On 11/23/2015 04:14 PM, rudba556 wrote:
> Thanks for your valuable reply sir;
>
> I had added that permission already, after i added
> (FFmpegFrameRecorder.exception e) to (exception e) that error has gone ,
> I have added FFmpegLogCallback.set() at initiallization.
>
> but now it comes with new error :((
>
> 749-28802/com.arapps.actionsmileys I/OpenGLRenderer﹕ Initialized EGL, version 1.4
> 11-23 12:36:56.579 28749-28802/com.example.Videomaker D/OpenGLRenderer﹕ Enabling debug mode 0
> 11-23 12:36:56.690 28749-28749/com.example.Videomaker I/Timeline﹕ Timeline: Activity_idle id: android.os.BinderProxy@36ad619e time:5108432
> 11-23 12:36:59.063 28749-28749/com.example.Videomaker I/System.out﹕ Output #0, mp4, to '/storage/emulated/0recording.mp4':
> 11-23 12:36:59.065 28749-28749/com.example.Videomaker I/System.out﹕ Stream #0:0
> 11-23 12:36:59.066 28749-28749/com.example.Videomaker I/System.out﹕ : Video: mpeg4, rgb24, 512x320, q=2-31, 400 kb/s
> 11-23 12:36:59.066 28749-28749/com.example.Videomaker I/System.out﹕ ,
> 11-23 12:36:59.066 28749-28749/com.example.Videomaker I/System.out﹕ 5 tbn,
> 11-23 12:36:59.067 28749-28749/com.example.Videomaker I/System.out﹕ 5 tbc
> 11-23 12:36:59.067 28749-28749/com.example.Videomaker I/System.out﹕ [
> 11-23 12:36:59.068 28749:28749 I/System.out ] Stream #0:1
> 11-23 12:36:59.068 28749-28749/com.example.Videomaker I/System.out﹕ : Audio: aac, 44100 Hz, 44100 channels, fltp, 64 kb/s
> 11-23 12:36:59.068 28749-28749/com.example.Videomaker I/System.out﹕ [

> 11-23 12:36:59.070 28749:28749 W/System.err ] Error: [mpeg4 @ 0xb4a36000] *Specified pixel format rgb24 is invalid or not supported*

> 11-23 12:36:59.072 28749-28749/com.example.Videomaker W/System.err﹕

> *org.bytedeco.javacv.FrameRecorder$Exception: avcodec_open2()*
>

[rudba556]

Thanks a lot again it worked,

but again :((

hope you don't mind..

   4488-4523/com.arapps.actionsmileys I/OpenGLRenderer﹕ Initialized EGL, version 1.4

11-23 12:56:25.321    4488-4523/com.example.Videomaker D/OpenGLRenderer﹕ Enabling debug mode 0

11-23 12:56:25.384    4488-4488/com.example.Videomaker  I/Timeline﹕ Timeline: Activity_idle id: android.os.BinderProxy@36ad619e time:6277125

11-23 12:56:26.791    4488-4508/com.example.Videomaker  I/art﹕ Background sticky concurrent mark sweep GC freed 50525(2MB) AllocSpace objects, 11(356KB) LOS objects, 0% free, 43MB/43MB, paused 11.543ms total 54.385ms

11-23 12:56:26.840    4488-4501/com.example.Videomaker  W/art﹕ Suspending all threads took: 6.687ms

11-23 12:56:27.820    4488-4488/com.example.Videomaker  I/System.out﹕ Output #0, mp4, to '/storage/emulated/0recording.mp4':

11-23 12:56:27.820    4488-4488/com.example.Videomaker  I/System.out﹕ Stream #0:0

11-23 12:56:27.821    4488-4488/com.example.Videomaker  I/System.out﹕ : Video: h264 (libx264rgb), rgb24, 512x320, q=-1--1, 400 kb/s

11-23 12:56:27.821    4488-4488/com.example.Videomaker  I/System.out﹕ ,

11-23 12:56:27.821    4488-4488/com.example.Videomaker  I/System.out﹕ 5 tbn,

11-23 12:56:27.821    4488-4488/com.example.Videomaker  I/System.out﹕ 5 tbc

11-23 12:56:27.821    4488-4488/com.example.Videomaker  I/System.out﹕ [ 11-23 12:56:27.821  4488: 4488 I/System.out ]

    Stream #0:1

11-23 12:56:27.821    4488-4488/com.example.Videomaker  I/System.out﹕ : Audio: aac, 44100 Hz, 44100 channels, fltp, 64 kb/s

11-23 12:56:27.821    4488-4488/com.example.Videomaker I/System.out﹕ [ 11-23 12:56:27.834  4488: 4488 I/System.out ]

    [libx264rgb @ 0xb4a33000] using cpu capabilities: ARMv6 NEON

11-23 12:56:27.864    4488-4488/com.example.Videomaker  I/System.out﹕ [libx264rgb @ 0xb4a33000] profile High 4:4:4 Predictive, level 2.1, 4:4:4 8-bit

11-23 12:56:27.866    4488-4488/com.example.Videomaker  I/System.out﹕ [libx264rgb @ 0xb4a33000] 264 - core 148 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=5 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=400 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00

11-23 12:56:27.905    4488-4488/com.example.Videomaker  I/System.out﹕ [libx264rgb @ 0xb4a33000] final ratefactor: 14.89

11-23 12:56:27.909    4488-4488/com.example.Videomaker  W/System.err﹕ org.bytedeco.javacv.FrameRecorder$Exception: avcodec_open2() error -22: Could not open audio codec.

[rudba556]

I have added audio input through the mic using:

http://code.google.com/p/javacv/source/browse/samples/RecordActivity.java

where is the audiocodec specified?

[rudba556]

Also i have added permission for recording audio...

[rudba556]

i tried adding:

recorder.setAudioCodec(avcodec.AV_CODEC_ID_AC3);

no luck:((

[rudba556]

I also tried adding:

recorder.setAudioCodecName("aac");

and others mentioned on: https://trac.ffmpeg.org/wiki/SupportedMediaTypesInFormats

NO LUCK :((

Help Please!

[Samuel Audet]

On 11/23/2015 04:31 PM, rudba556 wrote:
> 11-23 12:56:27.821 4488-4488/com.example.Videomaker I/System.out﹕ : Audio: aac, 44100 Hz, 44100 channels, fltp, 64 kb/s

I don't know of any codec that supports 44100 channels. You're going to
need to rethink what you're trying to do here.

Samuel

[rudba556]

Thanks for the all the help till now sir, I did finish generating the video file(but only with audio) few days back :-)

Right now i am trying to figure out the steps for finishing with video part, and i did read about the frameconverters and many forums based on this topic , but i have not been able to figure out the right SEQUENCE :(

In my case the image is displayed on the imageview; which is from the drawable and can be changed by a button's press, 

So,

how do i pass this jpeg towards the recorder.record(....?....)..?

   I tried jpg->Frameconverter->Frame->record(frame)-> application crashes

   i read about the iplimage [convert jpg to iplimage and then pass this to record()..] about which in some forum it was advised not to be used since its deprecated..

These things have led me to confusion, kindly help!

Thanks..

[rudba556]

Also if i use this method:

Frame frame = converter.convert(bitmap);

Compiler shows me "unsafe operation" and on running the app the application crashes..:-(

[Samuel Audet]

On 12/11/2015 01:02 AM, rudba556 wrote:
> how do i pass this jpeg towards the recorder.record(....?....)..?

You could use imread() to get a Mat object from a JPEG file, and then we
can get a Frame for FFmpegFrameRecorder using
OpenCVFrameConverter.convert(). Sounds good?

Samuel

[Samuel Audet]

Could you provide a bit more details about the crash and about the
bitmap object? Thanks

Samuel

[rudba556]

Thanks for the reply, i'll give try..

That error is shown in initRecorder too....

On console nothing is shown as such...( i have used FFmpegLogCallback.set() )

[rudba556]

Is there any way the video recording can be paused/resumed?

Or should we always start and stop recording and then combine all mp4's using mp4parser lib,, like we do it for mediaRecorder?

I found an article telling the above can be done using "STOP" and "CONT" functions in ffmpeg:

http://superuser.com/questions/944113/can-ffmpeg-be-paused-while-recording-and-then-resume

Can this be done in android too? and how?

Thanks...! 

[Samuel Audet]

If we don't call stop(), start(), or record(), it is effectively paused.

Or are you talking about appending new content to an existing file?

Samuel

2015/12/31 0:00 "rudba556" <abdu...@gmail.com>:

[rudba556]

Thanks!

Ya i meant appending new content to the file...

I tried giving few tries like removing recorder.release() etc... didn't work..! 

IF recorder.start() is executed it writes a fresh file again..

[Samuel Audet]

I don't believe FFmpeg supports this, but I might be wrong. Anyway, we
can simply create a new file and copy all the old frames before adding
the new ones. To avoid the encoding overhead, we can also copy the raw
packets, but this isn't something that is supported by
FFmpegFrameGrabber or FFmpegFrameRecorder yet:
https://github.com/bytedeco/javacv/issues/93
It shouldn't be to hard to enhance them for that, and a lot of people
have been requesting this enhancement, so if you feel like making a
contribution, adding support for this would be very welcome!

Samuel

[rudba556]

PAUSE FUNCTIONALITY:

Thanks to robpuk38 and LV426 @androidforums.com.

http://androidforums.com/threads/writing-filename-in-sequence.985745/#post-7185580

http://androidforums.com/threads/weird-java-io-filenotfoundexception-error.993941/#post-7207505 

[rudba556]

These many days ffmpeg was working fine, now all of a sudden i get a weird fps and bitrate issue.

I am using recorderActivity.java, with a slight change, where the yuv bytes come from an image processing method.

here is what happens to the video:

(fps set to 5, bitrate is default, audiobitrate is 32kbps)

Recording times:

1s - 3.8fps, 90kbps,

2s - 0.5fps, 54kbps,

3s - 0.12fps,44kbps,

.

.

10s - 0.02fps, 40kbps.

Everything was working fine, since 2 days i am getting this; I have cross checked the code thrice, i dont feel i have changed anything.... don't know whats happening..

Kindly help.

Thanks...!

[Samuel Audet]

On 02/26/2016 04:15 AM, rudba556 wrote:
> Everything was working fine, since 2 days i am getting this; I have cross
> checked the code thrice, i dont feel i have changed anything.... don't know
> whats happening..

You might want to try to uninstall your app completely and reinstall
everything. Some devices are known for not installing things properly:
The Perils of Loading Native Libraries on Android
https://medium.com/keepsafe-engineering/the-perils-of-loading-native-libraries-on-android-befa49dce2db

Samuel