Can javacv change resolution when use hardware acceleration(cuvid)

[Aier karl]

Hi, i am a java developer.

Recently i found that i can't change resolution when i use cuvid. No matter what i set the option or set the ImageWidth and ImageHeight, it's very strange.Even when i use command to use cuvid and option('-s 1280x720') together, it always say error.

that is part of my code and command:

code:

grabber = new CustomFFmpegFrameGrabber(cameraPojo.getRtsp());

grabber.setOption("y", "");
grabber.setOption("vsync", "0");
grabber.setOption("hwaccel", "cuvid");
grabber.setVideoCodecName("h264_cuvid");
grabber.setOption("rtsp_transport", "tcp");

if (cameraPojo.getToHls() == 1) {
record = new FFmpegFrameRecorder(cameraPojo.getHls(), 1280, 720);// it doesn't work
} else {
record = new FFmpegFrameRecorder(cameraPojo.getRtmp(), 1280, 720);// it doesn't work
}
record.setVideoOption("crf", "40");
record.setGopSize(2);
record.setFrameRate(framerate);
record.setVideoCodecName("h264_nvenc");
// record.setVideoCodec(AV_CODEC_ID_H264);
record.setVideoBitrate(20000);

command:

ffmpeg -y -vsync 0 -hwaccel cuvid -c:v h264_cuvid -i rtsp://admin:xxxxxxxxxx:554/h264/ch1/main/av_stream -s 1280x720 -f flv -c:v h264_nvenc rtmp://127.0.0.1:1935/live

my javacv version is 1.5.2

Thanks for your help!

[Samuel Audet]

If I understand correctly, that's an issue with the cuvid codec?
Have you tried to report this issue upstream?

[Aier karl]

No, i haven't did it.

And this situation just happen in recordPacket(AVPacket) this method.

If i use recordFrame(Frame) method, it seem that i can change the resolution.

So should i report an issue in github project?

[Samuel Audet]

recordPacket() won't reencode the frame, so you won't be able to resize,
no. You'll need to use recordFrame() for that.

[Aier karl]

Yep, but there is a question.

When i use recordFrame(), CPU usage is very high, and the process will use cuvid at the same time.

So how can i decrease the CPU usage with cuvid and change the resolution at the same time?

[Samuel Audet]

What happens with the ffmpeg program? How much CPU does it use?

[Samuel Audet]

Ah, I see, what you're looking for is the npp_scale filter:
https://developer.nvidia.com/blog/nvidia-ffmpeg-transcoding-guide/

That's not something that JavaCV currently supports, but it should be
possible to make it work:
https://github.com/bytedeco/javacpp-presets/issues/857

[Aier karl]

Thanks for you guide.

And npp_scale is work on command.However, when i add lines 'record.setOption("vf", "scale_npp=1280:720")' and 'record.setOption("gpu", "")';, it doesn't work like command.

It seem that this parameter can't work in javacv?

[Samuel Audet]

Like I said, no, it's not currently supported, but it should be possible
to make it work. Please see this issue for more details:
https://github.com/bytedeco/javacpp-presets/issues/857

[Aier karl]

I see it. And not matter i set 'record.setOption("gpu", "")' or 'grabber.setOption("gpu", "")'. It can't change the resolution.

Maybe i need to wait javacv support it in the future? 

By the way, because ffmpeg pull and push stream with high CPU usage(almost 50-60% when push a stream, even higher when push more streams), so i have to choose cuvid to decrease CPU usage.

And i use recordPacket can use cuvid in low CPU usage. The only shortage is that it can't change the resolution. So I think i have choose changing the resolution or decreasing CPU usage in balance.

[Samuel Audet]

If you have a budget that you can use, please consider using it:
https://xscode.com/bytedeco/javacv
This is something I could be working on if you need it.

[Aier karl]

Do you mean if i buy this service, you can help me to add npp_scale, etc parameter to javacv?

[Samuel Audet]

Yes, that's what it's for. The first thing we need to do is to update
the builds of FFmpeg to include the npp_scale filter. We'll see if we
also need to update JavaCV after that.

[Aier karl]

I have try to rebuild javacpp-presets to add --enable-libnpp and it is successful.

But i can use the new ffmpeg.jar to change the resolution. That is very strange. How can i deal with this problem?

Thanks.

[Samuel Audet]

What error are you getting?

[Aier karl]

well, when i use the coed like that:

String ffmpeg = Loader.load(org.bytedeco.ffmpeg.ffmpeg.class);
ProcessBuilder pb = new ProcessBuilder(ffmpeg, "-y", "-vsync", "0", "-hwaccel", "cuvid", "-c:v", "h264_cuvid", "-i", "rtsp://admin:xxxxxxx/h264/ch1/main/av_stream", "-f", "flv", "-c:v", "h264_nvenc", "-vf", "scale_npp=1280:720", "rtmp://127.0.0.1:1935/live");
pb.inheritIO().start().waitFor();

the console print :

ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers

  built with gcc 7 (GCC)

  configuration: --prefix=.. --disable-iconv --disable-opencl --disable-sdl2 --disable-bzlib --disable-lzma --disable-linux-perf --enable-shared --enable-gpl --enable-version3 --enable-nonfree --enable-runtime-cpudetect --enable-zlib --enable-libmp3lame --enable-libspeex --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-openssl --enable-libopenh264 --enable-libx264 --enable-libx265 --enable-libvpx --enable-libfreetype --enable-libopus --enable-cuda --enable-cuvid --enable-nvenc --enable-pthreads --enable-libxcb --cc='gcc -m64' --extra-cflags=-I../include/ --extra-ldflags=-L../lib/ --extra-libs='-lstdc++ -lpthread -ldl -lz -lm '

  libavutil      56. 31.100 / 56. 31.100

  libavcodec     58. 54.100 / 58. 54.100

  libavformat    58. 29.100 / 58. 29.100

  libavdevice    58.  8.100 / 58.  8.100

  libavfilter     7. 57.100 /  7. 57.100

  libswscale      5.  5.100 /  5.  5.100

  libswresample   3.  5.100 /  3.  5.100

  libpostproc    55.  5.100 / 55.  5.100

[rtsp @ 0x17478c0] max delay reached. need to consume packet

[rtsp @ 0x17478c0] RTP: missed 7 packets

[h264 @ 0x174b280] error while decoding MB 88 83, bytestream -5

[h264 @ 0x174b280] concealing 1081 DC, 1081 AC, 1081 MV errors in P frame

Input #0, rtsp, from 'rtsp://admin:xxxxxxxxx/h264/ch1/main/av_stream':

  Metadata:

    title           : Media Presentation

  Duration: N/A, start: 0.240000, bitrate: N/A

    Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 2560x1440, 25 fps, 25 tbr, 90k tbn, 50 tbc

Stream mapping:

  Stream #0:0 -> #0:0 (h264 (h264_cuvid) -> h264 (h264_nvenc))

Press [q] to stop, [?] for help

[AVFilterGraph @ 0x17d0240] No such filter: 'scale_npp'

Error reinitializing filters!

Failed to inject frame into filter network: Invalid argument

Error while processing the decoded data for stream #0:0

scale_npp it seem like doesn't work.

[Aier karl]

By the way, i rebuild javacpp-presets/ffmpeg with new parameter "--enable-libnpp". And replace the ffmpeg.jar with new one.

But it seem does not rebuild successfully from the console log, isn't it?

在2020年12月1日星期二 UTC+8 上午11:44:47<Samuel Audet> 写道：

[Samuel Audet]

That's correct. Your build doesn't have that filter:

> [AVFilterGraph @ 0x17d0240] No such filter: 'scale_npp'

[Aier karl]

Maybe i add parameter in a wrong way? So what place should i change in the ffmpeg/cppbuild.sh

Would you tell me something about the keypoint of build.

Thanks a lot. :-)

[Samuel Audet]

To modify presets by yourself, please take the time to read this:
https://github.com/bytedeco/javacpp-presets/wiki/Create-New-Presets
If you have any questions after that, please let me know!