Green video when frameserving from Premiere Pro 2023 to FFMpeg

[Stefan Smietanowski]

Renderer: Adobe Premiere Pro 2023

Frameserver: CuminCode FrameServer 4.2.1

Encoder: FFMpeg latest.

PP Settings: Video and Audio enabled.

Frameserver settings: v210 , virtual file.

FFMpeg commandline: ffmpeg.exe -i "D:\CCFS\virtual\Input.avi" -c:v hevc_nvenc -preset p7 -tune hq -profile:v main10 -pix_fmt p010le -tag:v hvc1 -rc:v constqp -qp:v 12 -y "Output.mp4"

So, as you can see, it picks up the file just fine, and the AUDIO works, however the video is just a massive green frame throughout.

I have tried the following and it WORKED:

Encode to Apple ProRes 4444 XQ in Premiere Pro (huge file!), and then ran that exact same command line (except obviously different input file), and it works just fine. Am I doing something wrong?

I get ~40fps if I encode from ProRes, and ~25-26fps if I run through the frameserver, but there are obvious benefits to going straight: 1) I don't have to store the 709GiB intermediate file (!), and I can skip one encoding step.

Why I want to use FFMPEG is that I get 40fps (or 25-26 via FrameServer) using the hardware encoder, vs 0.6fps inside Premiere Pro, and the resulting file in Premiere Pro is worse at the same bitrate.

Anyone got a clue what I'm doing wrong?

// Stefan

[Stefan Smietanowski]

Oh, and the original file is in ProRes LT 422, if that matters. I can't encode straight from that as I have processing in Premiere Pro.

[Stefan Smietanowski]

ProRes HQ 422, not LT, but same, same.

[Poz]

It's likely that Frameserver doesn't understand the ProRes HQ 422 bitdepth/format and instead of throwing an error it does wrong math which ends up being a green screen.

Do videos with other bitdepths/formats work when frameserving from Premiere to ffmpeg?

- Poz

[Stefan Smietanowski]

Hi, thanx for replying.

Well, would the source file matter at all? I mean, my understanding would be :

Premiere Pro input => changed to internal "raw" format => processed using filters and everything else => output

output in this case is either an encoder (built in or plugin), which has no understanding (nor does it have to) of the input file, since it's given the raw frames in some format (RGB, YUV, etc) processed data to encode, or it's for instance the FrameServer, which again just gets the raw frames, in some YUV 10-bit format, converts it internally to v210 and serves those frames to the encoder (ffmpeg in my case).

I'm currently encoding another small test-project (originally 8K DCI ProRes LT 422), and I'm seeing 76.9kbit/s, so I'm guessing it's super green (points if you get the reference). I'll try another input format after that. Need to wait for my vmaf run to complete, it's eating most of my resources.

// Stefan

[Stefan Smietanowski]

My other theories go around : ffmpeg's v210 decoder is busted. Since v210 is a YUV format, that explains the green screen. If it's looking at the bits in the wrong place...

I will get hold of some sample v210 file somewhere and try with that straight in ffmpeg, just to rule it out, but I'd be surprised if it didn't work.

[Poz]

> Well, would the source file matter at all? I mean, my understanding would be :

> Premiere Pro input => changed to internal "raw" format => processed using filters and everything else => output

Premiere and other NLEs typically give the input in the same format to plugins such as Frameserver, so that there is minimal color and format conversion done. Hence I wonder if that's where the disconnect is.

Please try other formats and see if anything works.

[Stefan Smietanowski]

Ok, so I ran some more tests:

Done with a clean project, i.e. started from scratch, created project, imported just one file, and hit CTRL+M.

I used an input file I had used for testing. FFMpeg software encoded at 400Mbps VBR:

That was the input file, so a plain 10bit YUV, the settings when I encoded this were yuv420p10le.

Look at the bitrate, and you'll see that it's again super-green.

Video
ID                          : 1
Format                      : HEVC
Format/Info                 : High Efficiency Video Coding
Format profile              : Main 10@L6.2@High
Codec ID                    : hvc1
Codec ID/Info               : High Efficiency Video Coding
Duration                    : 46 s 547 ms
Bit rate                    : 113 kb/s      <===
Maximum bit rate            : 800 Mb/s <===
Width                       : 3 840 pixels
Height                      : 2 160 pixels
Display aspect ratio        : 16:9
Frame rate mode             : Constant
Frame rate                  : 59.940 (59940/1000) FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 10 bits
Scan type                   : Progressive
Bits/(Pixel*Frame)          : 0.000
Stream size                 : 643 KiB (89%)
Codec configuration box     : hvcC

[Stefan Smietanowski]

And now I've done another test. Same input file, output to libx265 instead of hevc_nvenc. Same deal, super-green.

[Stefan Smietanowski]

Looks like I may have found the culprit: It is FFMpeg after all. I need to check if I need to build my own ffmpeg that has more support or something, but neither that or my ffmpeg-based codecs can show the sample files I've found online.

On Sunday, November 6, 2022 at 9:47:31 PM UTC+1 Poz wrote:

[Stefan Smietanowski]

Weirdly enough, I get no error message from ffmpeg that it can't decode the input.

[Stefan Smietanowski]

Or... maybe not.

I found a v210 file that works just fine when I run it through ffmpeg, so clearly it handles v210 properly.

[Stefan Smietanowski]

I've played around with using Avisynth+ to convert colorspaces, in case that was the problem, but nothing I do works, even if I just tell it to do a simple v210 to yuv420p10le conversion or yuv444p10le or to RGB48.

Has anyone gotten the v210 output to work? If yes, with what?