Issue 114 in ffmbc: RGB DPX frame to Prores 422/444 HQ produces colour shift to yellow

[ff...@googlecode.com]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 114 by thomasur...@thelooklondon.com: RGB DPX frame to Prores
422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

What steps will reproduce the problem?
1. DPX frame with full RGB values to Prores HQ 422 or 444 encode with ffmbc
command
2.
3.

What is the expected output? What do you see instead?
I wouldnt expect any colour shift from RGB to YUV or from DPX RGB to Prores
RGB


What version of the product are you using? On what operating system?
FFmbc 0.7rc7 for Windows (64-bit) 5-Jul-2012
Windows 7 64bit


Please provide any additional information below:

C:\Users\User\Desktop\ffmpeg-PRORES_ENCODING>ffmbc -f image2 -r 25 -i
JUNK_DPX_TEST\%06d.dpx -vcodec prores -profile hq
N:\ffmpeg_encodes\output.mov
FFmbc version 0.7-rc7
Copyright (c) 2008-2012 Baptiste Coudurier and the FFmpeg developers
Input #0, image2, from 'JUNK_DPX_TEST\%06d.dpx':
Duration: 00:00:40.04, start: 0.000000, bitrate: N/A
Stream #0.0(und): Video: dpx, rgb48le, 1920x1080p [PAR 65536:65536 DAR
16:9], 25.00 fps
File 'N:\ffmpeg_encodes\output.mov' already exists. Overwrite ? [y/N] y
Incompatible pixel format 'rgb48le' for codec 'prores', auto-selecting
format 'yuv422p10le'
[scale @ 0000000001AFD200] w:1920 h:1080 fmt:rgb48le -> w:1920 h:1080
fmt:yuv422p10le flags:0x4
Output #0, mov, to 'N:\ffmpeg_encodes\output.mov':
Metadata:
encoder: FFmbc 0.7
Stream #0.0(und): Video: prores, yuv422p10le, 1920x1080p [PAR 1:1 DAR
16:9], 183500 kb/s, 25.00 fps
Stream mapping:
Stream #0.0 -> #0.0
Press [q] to stop, [?] for help
frame= 256 fps= 19 q=1.0 size= 10534kB time=00:00:10.24
bitrate=8427.5kbits/s eta=00:00:39.63


Attachments:
prores422.jpg 1.0 MB
dpx.jpg 507 KB

[ff...@googlecode.com]

Comment #1 on issue 114 by thomasur...@thelooklondon.com: RGB DPX frame to

Prores 422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

This actually seems to be effecting a conversion from DPX to rawvideo 422
QT as well

[ff...@googlecode.com]

Comment #2 on issue 114 by baptiste...@gmail.com: RGB DPX frame to Prores

422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

I will need the original dpx to reproduce

[ff...@googlecode.com]

Comment #3 on issue 114 by thomasur...@thelooklondon.com: RGB DPX frame to

Prores 422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

Here you go Baptiste. If you convert this frame to a QT and then compare
the colour you'll see the output gets slighter warmer. Have tried adding
-vf colormatrix=bt709:bt709 with no change.

Attachments:
dpx.dpx 3.5 MB

[ff...@googlecode.com]

Comment #4 on issue 114 by baptiste...@gmail.com: RGB DPX frame to Prores

422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

Hummm, djv and imagemagick cannot convert this dpx, their output is a black
frame.
What are you using to view and create the dpx ?

[ff...@googlecode.com]

Comment #5 on issue 114 by thomasur...@thelooklondon.com: RGB DPX frame to

Prores 422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

That was from a Quantel Pablo, this one is from SGO Mistika, be aware the
frame is very dark

Attachments:
001000.dpx.zip 3.3 MB

[ff...@googlecode.com]

Comment #6 on issue 114 by thomasur...@thelooklondon.com: RGB DPX frame to

Prores 422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

Having now also done a Uncompressed YUV422 QT to Prores I also get the same
shift. I think this is the same as this issue:
http://web.archiveorange.com/archive/v/DUtyPyCWLv29zJqkZkPl
Maybe as we're in a colour critical environment we pick up on this when
others dont care, however this means ffmpeg can not be used for true 1:1
color critical work. A snapshot of our scope shows a change in the RGB
parade (the last three). The Red and Blue channel is different,
unfortunately i cant upload these as this forum says the storage quota for
this thread is exceeded. Thanks

[ff...@googlecode.com]

Comment #7 on issue 114 by jakub.kr...@gmail.com: RGB DPX frame to Prores

422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

Hey guys, I can confirm that we at Weta are experiencing the same issue. I
have the latest ffmpeg and ffmbc versions compiled and i've ran a few tests
encoding a dpx sequence into a prores quicktime. We're trying to find a
replacement for our current in-house solution that runs only on mac os x
and uses native quicktime libraries (we are finding the code maintenance to
be very difficult, which is why we are wanting to find an alternative) and
i can generate an awesome looking prores movie, but as OP posted, it is
more yellow than the source. There is a definite color shift. The quality
of the video and encoding is better (prores HQ, 48fps, 2k res, yuv44410p,
170Mbit data rate) and the codec we use ATM (apple intermediate), but the
color is off.

I'd be happy to provide some dpx's and resulting sequences generated with
our in house solution vs. ffmpeg for comparison if this could help solve
the problem...

thanks
jakub

[ff...@googlecode.com]

Comment #8 on issue 114 by baptiste...@gmail.com: RGB DPX frame to Prores

422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

Humm, the other frame looks plain black to me, I cannot distinguish
anything.
I'll close this issue unless a good dpx file is supplied that clearly
illustrates the problem,
I just cannot fix anything without a good test case

[ff...@googlecode.com]

Comment #9 on issue 114 by flehnerh...@gmail.com: RGB DPX frame to Prores

422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

I have noticed some of the same symptoms. If the limit of attachments
hadn't been full I'd attached an archive with two dpx'es, two quicktimes,
and two screenshots from video scopes in FCP. I know it's not the most
accurate, but our hardware video scope confirms the shift.

I'll try emailing you the archive :)

Commands used to produce quicktimes:

ffmbc -i calibrate.%04d.dpx -r 25 -vcodec prores -profile std -vf
colormatrix=bt601:bt709 -y dpx_to_prores_709_filtering.mov

ffmbc -i calibrate.%04d.dpx -r 25 -vcodec prores -profile std -y
dpx_to_prores_no_filtering.mov

[ff...@googlecode.com]

Comment #10 on issue 114 by thomasur...@thelooklondon.com: RGB DPX frame to

Prores 422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

I've uploaded a kodak marcie(rec709) dpx and its converted to prores 422
output, and a split screen as a tiff Baptiste. I've shared the files with
you via your gmail account.
Hopefully this will clearly show you the issue. Its clear that the guys at
Weta have the same problem and they know what they're doing!
Best regards

[ff...@googlecode.com]

Comment #11 on issue 114 by thomasur...@thelooklondon.com: RGB DPX frame to

Prores 422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

Has this topic now been closed?

[ff...@googlecode.com]

Comment #12 on issue 114 by nicolas....@gmail.com: RGB DPX frame to Prores

422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

The issue still hasn't be fixed as of FFmbc-0.7-rc8 or FFmpeg 2.1.1

--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[Troy Sobotka]

On Jan 6, 2014 6:29 PM, <ff...@googlecode.com> wrote:
>
>
> Comment #12 on issue 114 by nicolas....@gmail.com: RGB DPX frame to Prores 422/444 HQ produces colour shift to yellow
> http://code.google.com/p/ffmbc/issues/detail?id=114
>
> The issue still hasn't be fixed as of FFmbc-0.7-rc8 or FFmpeg 2.1.1

What colorspace is the source DPX data in?

[ff...@googlecode.com]

Comment #13 on issue 114 by tho...@thelooklondon.com: RGB DPX frame to

Prores 422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

disappointing that this has never been fixed, would be great to use a
command line render farm on windows to do these! oh well

[ff...@googlecode.com]

Comment #14 on issue 114 by stevem...@gmail.com: RGB DPX frame to Prores

422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

It would be a game changer...

[ff...@googlecode.com]

Comment #15 on issue 114 by troy.sob...@gmail.com: RGB DPX frame to Prores

422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

Has anyone tried the colormatrix filter? IIRC there is a 601 default baked
into the code somewhere.

As far as I can see, only Rec.709 (sRGB primaries), FCC, Rec.601 (ITU-R
BT.470-2/SMPTE 170M), and SMPTE 240M are supported, but it would appear
they should work for the src / dst conversions.

"-vf colormatrix=src:dst" where src / dst are one of:
'bt709'
'bt601'
'smpte240m'
'fcc'

Example:

"-vf colormatrix=bt601:bt709"

If someone could test this and report back, that would be excellent.

[ff...@googlecode.com]

Comment #16 on issue 114 by tho...@thelooklondon.com: RGB DPX frame to

Prores 422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

I tried this a year ago, i still had an issue. would be interested to hear
if anyone else had more success




*I'm often with clients and unavailable for fast email replies, so for
urgent bookings and enquiries please call us, or email: *
book...@thelooklondon.com* and you will get a response*.
*Book early to avoid disappointment*.

*THE LOOK*
29-35 Rathbone Street
London, W1T 1NJ
T: 0207 2875313
E: tho...@thelooklondon.com
www.thelooklondon.com
2013: Voted 5th in Televisual's Top Colourists survey
2012: Voted 2nd in the 'Craft' category in Broadcast magazine's 'Hot 100
People in the industry'
Go to our website to see our latest drama, features and commercials work

[ff...@googlecode.com]

Comment #17 on issue 114 by troy.sob...@gmail.com: RGB DPX frame to Prores

422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

Is there any way someone following this thread can do a test with the Bella
Nuit (http://www.belle-nuit.com/test-chart) test pattern and sample the
luma across the two 709 / 601 sets?

I have a strong suspicion that RGB to YCbCr conversion is using the 601
matrices. This would account for the filter effects being skipped perhaps.

[ff...@googlecode.com]

Comment #18 on issue 114 by bengt...@hocusfocus.no: RGB DPX frame to Prores

422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

I've identified a number of issues causing this shift (all in libswscale).
First of all, the bt601 coefficients are hard coded in swscale.c (which
does rgb->yuv). Even when changing those, the luma will still be wrong if
mmx is enabled (because mmx also assumes bt601 for any yuv material).
swscale.c.rc8patch will change the defult coefficients to bt709 and disable
mmx for the scaler.

When converting back from yuv->rgb, bt601 coefficients will also be used.
swscale.h.rc8patch changes yuv->rgb coefficients to bt709.

However, when converting from yuv444 to rgb, rc8 will not give you full
color resolution.
utils.c.rc8patch fixes this with some lines borrowed from latest ffmpeg.

NB! Even though utils.c.rc8patch seems to do the trick in this case, I
might have broken something else. If you only need rgb->yuv and not the
other way, you only need swscale.c.rc8patch

Bengt

Attachments:
swscale.c.rc8patch 1.3 KB
swscale.h.rc8patch 369 bytes
utils.c.rc8patch 1.3 KB

[Troy Sobotka]

On Mar 8, 2014 4:46 PM, <ff...@googlecode.com> wrote:

> Bengt
>
> Attachments:
>         swscale.c.rc8patch  1.3 KB
>         swscale.h.rc8patch  369 bytes
>         utils.c.rc8patch  1.3 KB

These are excellent. I would think the YCbCr -> RGB is required for any and all frame dumping from codecs, so all three of these would be critically important in my estimation.

Do the patches hook into the -vf colormatrix filter?

Amazing work.

[ff...@googlecode.com]

Comment #19 on issue 114 by tho...@thelooklondon.com: RGB DPX frame to

Prores 422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

Thanks for this Bengt, unfortunately I've been unable to compile the RC8
with these patches for a Win 7 64 machine which is very frustrating (the
main source of info online is for 32bit). Does anyone know how to do this
and can send me the outputted ffmbc.exe for testing Bengt's fix?
Thanks all

[ff...@googlecode.com]

Comment #20 on issue 114 by tho...@thelooklondon.com: RGB DPX frame to

Prores 422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

Unless ofcourse it is possible to disable MMX in the command line encode in
RC8? in which case what would be the correct command for RGB DPX rec709 to
Prores 422 HQ?

[ff...@googlecode.com]

Comment #21 on issue 114 by baptiste...@gmail.com: RGB DPX frame to Prores

422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

I like the utils.c patch I will apply it.
Other patches seem intrusive to me, if it could be conditional if would be
better.

[ff...@googlecode.com]

Comment #22 on issue 114 by troy.sob...@gmail.com: RGB DPX frame to Prores

422/444 HQ produces colour shift to yellow
http://code.google.com/p/ffmbc/issues/detail?id=114

I would think that full range RGB should only be dumped if the source is
full range (yuvj)?

Does it dump full range if the source is yuvj range?