High CPU usage when using FFmpeg

[Bashar HajHamad]

Hi,

I'm Bashar Al-haj Hamad, working as a JAVA developer.

I'm using the FFmpeg library for getting images from videos and write the images to the disk, but I'm facing an issue that the usage of CPU is high while extracting the frames from the video, could you please help me with how I can optimize the usage of CPU?

Attached the code, dependency in the POM file, and the CPU usage while extracting video frames.

Regards,
Bashar  

[Samuel Audet]

Could you put your code on some place like https://gist.github.com/ and send the link here? Thanks!

2020年8月23日(日) 14:55 Bashar HajHamad <bashar....@exalt.ps>:

--

---
You received this message because you are subscribed to the Google Groups "javacv" group.
To unsubscribe from this group and stop receiving emails from it, send an email to javacv+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/javacv/866f3f5c-38d3-4b03-a4fa-cbcd2aa5b16bn%40googlegroups.com.

[Bashar HajHamad]

Sure, attached the link:

https://gist.github.com/BasharHajhamad-exalt/8ef995f9e2bd8a1dd53a1cc1f8ac8d61 

Also, sorry I missed sending the attachments in the previous email, attached to this email.

Regards,
Bashar   

[Bashar HajHamad]

Hi,

Could you please help me with the issue?

Thanks,
Bashar

[Samuel Audet]

You're using a lot of APIs in there, but I would start by getting rid of Java 2D. That's always pretty slow.

[Bashar HajHamad]

What is best to replace it with? I need to get the Frame as BufferedImage

[Samuel Audet]

Then that's a problem. That's always going to be slow...

[Samuel Audet]

The way you're dong it is fine. It's just that BufferedImage is never going to be fast.

If you need it to be faster, you'll need to use something else than BufferedImage.

On 8/24/20 4:10 PM, Bashar HajHamad wrote:

Oh, so what is the solution to get bufferedImages from a video?

[Samuel Audet]

Yes, that's what imwrite() is for:
https://docs.opencv.org/master/d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce

On 8/24/20 5:14 PM, Bashar HajHamad wrote:
> So is there a way to write Mat or IplImage as JPG and PNG images?

[Bashar HajHamad]

Yes but I could find it in JAVA, is it supported on these dependencies?

<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>javacv</artifactId>
<version>1.5</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.bytedeco/ffmpeg -->
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>ffmpeg-platform</artifactId>
<version>4.1.3-1.5</version>
</dependency>

Regards,
Bashar

[Samuel Audet]

Yes:
http://bytedeco.org/javacpp-presets/opencv/apidocs/org/bytedeco/opencv/global/opencv_imgcodecs.html#imwrite-org.bytedeco.javacpp.BytePointer-org.bytedeco.opencv.opencv_core.Mat-org.bytedeco.javacpp.IntPointer-

[Bashar HajHamad]

Thanks,
Just a question, when I use:

Frame frame = frameGrabber.grabImage();
Mat matImage = Java2DFrameUtils.toMat(frame);

Or use the converter:

OpenCVFrameConverter.ToMat converter = new OpenCVFrameConverter.ToMat();

Frame frame = frameGrabber.grabImage();
Mat matImage = converter.convert(frame);

on both the execution stuck and throw exception when reach  Java2DFrameUtils.toMat(frame) or  OpenCVFrameConverter.ToMat converter = new OpenCVFrameConverter.ToMat();

Could you please tell me why this happened?

Regards,
Bashar

[Samuel Audet]

It sounds like OpenCV is failing to load? What's the error message?

2020年8月24日(月) 20:50 Bashar HajHamad <bashar....@exalt.ps>:

[Bashar HajHamad]

Should the OpenCV be installed on my device?

Regards,
Bashar

[Samuel Audet]

No, you just need to make sure you have javacv-platform or opencv-platform in your dependencies.

[Bashar HajHamad]

I changed the Java2DFrameConverter to OpenCVFrameConverter.ToOrgOpenCvCoreMat but still the CPU usage very high. 

attached the code  https://gist.github.com/BasharHajhamad-exalt/27f02020cb9e75582eb49c13cf1d879b

[Bashar HajHamad]

Could you please help me with the issue?

[Samuel Audet]

I told you, BufferedImage is going to be slow, no matter what you do, so I recommend to get rid of that first.

[Bashar HajHamad]

Yes, but the is issue is not that the performance is slow, the issue is that when extracting the frames the CPU usage on the device is very high ( more than 100% ), how I can make the library work with one thread and use less resources?

[Samuel Audet]

Ah, I see, that's a different question. We can usually disable
multithreading for video decoders this way:

frameGrabber.setVideoOption("threads", "1")

[Bashar HajHamad]

Thanks, I will check it.

[Bashar HajHamad]

I still see that all CPU cores still working when I start extracting frames, I think that it still working with multi instances (threads).

[Samuel Audet]

It might be one of the other libraries. Make sure to check them all one by one.

[Samuel Audet]

Yes: https://github.com/bytedeco/javacpp-presets/wiki/Reducing-the-Number-of-Dependencies

2020年9月9日(水) 0:53 Bashar HajHamad <bashar....@exalt.ps>:

another thing, can I have 2 dependencies one for windows and one for mac? to minimize the size of JAR

On Tue, Sep 8, 2020 at 6:51 PM Bashar HajHamad <bashar....@exalt.ps> wrote:

Thank you, I have tried many dependencies but sometimes extracting frames fails or something else.

Could you please tell me which dependencies is necessary for extracting frames from the video as buffered images.

On Tue, Sep 8, 2020 at 2:14 PM Samuel Audet <samuel...@gmail.com> wrote:

You can customize them and build them for source: https://github.com/bytedeco/javacpp-presets/#build-instructions

2020年9月8日(火) 19:55 Bashar HajHamad <bashar....@exalt.ps>:

Hi,

Just a question, I'm using these dependencies, I just need to extract the Frames from the video as buffered images.

<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>javacv</artifactId>
<version>1.5</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.bytedeco/ffmpeg -->
<dependency>
<groupId>org.bytedeco</groupId>

<artifactId>ffmpeg</artifactId>
<version>4.2.2-1.5.3</version>
</dependency>

Are there other dependencies I can use and the size of them are smaller?

Regards,
Bashar