Possible memory leak when using JavaCPP and FFmpeg for audio transcoding

[AppietraLata]

Hello,

I'm facing a memory issue with JavaCPP and FFmpeg when transcoding audio files from AMR-NB to WAV-G711.

Performing a stress test I see the memory continuously growing: it seems that resources are never deallocated.

Converting in the opposite direction (from WAV-G711 to AMR-NB) does not lead to the same behavior.

From the java heap dump i can see a lot of org.bytedeco.javacpp.Pointer$NativeDeallocator that seem to be never released.

Following an extract of my code:

        FFmpegFrameGrabber grabber    = null;
        FFmpegFrameRecorder recorder  = null;
       
        try {
            grabber = new FFmpegFrameGrabber(inFile);
            grabber.start();

[...]
           
            recorder = new FFmpegFrameRecorder(outFile, grabber.getAudioChannels());
            recorder.setSampleRate(sampleRate <= 0 ? outSampleRate : sampleRate);
            recorder.setFormat(outFormat);
            recorder.setAudioCodec(outCodec);
           
            if(outBitRate > 0) {
                recorder.setAudioBitrate(outBitRate);
            }
           
            recorder.start();

            Frame frame = null;
            while ((frame = grabber.grabSamples()) != null) {
                recorder.record(frame);

            }
           
           
        } finally {
            try { if(recorder != null) { recorder.stop(); } } catch (Exception ignore) { }
            try { if(recorder != null) { recorder.release(); } } catch (Exception ignore) { }
            try { if(recorder != null) { recorder.close(); } } catch (Exception ignore) { } // Calling close() method should not have effect since the object has been already stopped and released
            recorder = null;
           
            try { if(grabber != null) { grabber.stop(); } } catch (Exception ignore) { }
            try { if(grabber != null) { grabber.release(); } } catch (Exception ignore) { }
            try { if(grabber != null) { grabber.close(); } } catch (Exception ignore) { } // Calling close() method should not have effect since the object has been already stopped and released
            grabber = null;

        }
       
        return outFile;

Am I missing something?

Hope you could help.

Thank you.

Regards.

[Samuel Audet]

Try to use PointerScope:
http://bytedeco.org/news/2018/07/17/bytedeco-as-distribution/

It's probably something we should try adding to the methods of the
FFmpeg* classes themselves. Actually, let me try that. Please follow
these issues to receive updates about that:
https://github.com/bytedeco/javacv/issues/1366
https://github.com/bytedeco/javacv/issues/1383
https://github.com/bytedeco/javacv/issues/1425

Samuel

On 6/10/20 2:11 AM, AppietraLata wrote:
> Hello,
> I'm facing a memory issue with JavaCPP and FFmpeg when transcoding audio
> files from AMR-NB to WAV-G711.
> Performing a stress test I see the memory continuously growing: it seems
> that resources are never deallocated.

> *Converting in the opposite direction (from WAV-G711 to AMR-NB) does not
> lead to the same behavior*.

[AppietraLata]

Hi Samuel,

I tried to add PointerScope in this way:

        try (PointerScope scope = new PointerScope()) {



            grabber = new FFmpegFrameGrabber(inFile);
            grabber.start();

[...]


but this lead to a core dump:

11:00:08,798 INFO  [org.hibernate.validator.internal.util.Version] (default task-1) HV000001: Hibernate Validator 6.0.18.Final
11:00:09,496 INFO  [stdout] (default task-1) Input file: ENG_P0067.wav - format: AMRNB - samplerate: null - bitrate: null
Input #0, wav, from '/tmp/ENG_P00673120941168953070674.wav':
  Duration: 00:00:12.93, bitrate: 64 kb/s
    Stream #0:0: Audio: pcm_alaw ([6][0][0][0] / 0x0006), 8000 Hz, 1 channels, s16, 64 kb/s
[libopencore_amrnb @ 0x3a63740] bitrate not supported: use one of 4.75k, 5.15k, 5.90k, 6.70k, 7.40k, 7.95k, 10.20k, 12.20k, using 12.20k
Output #0, amr, to '/tmp/ENG_P00673120941168953070674.wav.amrnb':
  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0: Audio: amr_nb, 8000 Hz, mono, s16, 64 kb/s
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007fd50266472a, pid=23068, tid=0x00007fd50f214700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_101-b13) (build 1.8.0_101-b13)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.101-b13 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libavcodec.so.58+0x1a872a]  av_init_packet+0xa
#
# Core dump written. Default location: /opt/wildfly-19.1.0.Final/bin/core or core.23068
#
# An error report file with more information is saved as:
# /opt/wildfly-19.1.0.Final/bin/hs_err_pid23068.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

Am I doing something wrong?

Thanks.

[Samuel Audet]

If you enclose the constructor call inside the scope, you won't be able to use that instance once the scope is closed.

In any case, I've added PointerScope inside the methods in the latest commit:

    https://github.com/bytedeco/javacv/commit/28b90ef25cc54d4deeba89f078270793abdd525f

Please give it a try with the snapshots:

    http://bytedeco.org/builds/

Samuel

[AppietraLata]

Hi Samuel,

thanks for suggestions!

I really tried everything, but still got the problem: I tried the javacv 1.5.4 snapshot and also placing PointerScope everywhere in the code.

I still see the memory continuously growing up, but only in one way transcoding: from AMR to WAV (G711).

There should be something strange only in this case, in fact converting from WAV to AMR does not lead to the same behavior: memory seems to be properly managed.

Any other ideas?

Thank you.

[Samuel Audet]

There might very well be bugs in some codecs. Can you reproduce this with the ffmpeg program?

[AppietraLata]

What I can say is that the problem does not seem to occur when using the JavaCPP-FFmpeg with ProcessBuilder:

private static final String FFMPEG = Loader.load(org.bytedeco.ffmpeg.ffmpeg.class);
[...]
ProcessBuilder pb = new ProcessBuilder(FFMPEG, "-i", inFile.getAbsolutePath(), "-b:a", String.valueOf(bitRate), "-acodec", sOutCodec, outFile);
pb.inheritIO().start().waitFor();

Initially memory is going to grow, but after a while it seems to stabilize.

Hope this could help to identifying (or limiting) the impacted code.

[Samuel Audet]

Well, each time the process dies, all allocated memory goes away, so you need to try with a super large file, right?

In any case, could you post your code snippet along with some data to reproduce that with JavaCV on this GitHub issue?

https://github.com/bytedeco/javacv/issues/1366

Thanks!

[AppietraLata]

Honestly I did not try with a large file.

I can reproduce the issue by submitting always the same input file (about 100KB) repeatedly: I made a shell script for this.

In this way I can see the memory to grow up and reach the limit (911MB) in a few hours.

Anyway I will try to update the issue 1366 as soon as I can.