transformPixel()

[laszlo...@primefocusworld.com]

Hi Jonathan,

Great work on the OpenDCX - huge improvement from last I've seen it. Really exciting!

I do however have an observation on the transformPixel() call. I tested it using different values of offsets and it appears that on certain subpixels the same pixel values are returned. As an example if I transform an image by 0.2 or 0.8 pixels on the x, the same values are returned. I tested this on the hairy_potter_nofur_nomask image.

Also, do you have a Nuke implementaion of this call? I can write it but if you do then it would make it much faster for me.

Thanks,
_laszlo

[laszlo...@primefocusworld.com]

Also, would it be possible to change the transferPixel() call to have both source and destination pixel location too?

For example:

xform.transformPixel(inDeepTile, inX, inY, outDeepTile, outX, outY);

this way we could set individual pixels location separately - the benefit at first might not be obvious but then warp functions can be added to deep layers. 

[Jonathan Egstad]

Hi Laszlo, welcome to the forum!

> I tested it using different values of offsets and it appears that on certain subpixels the same pixel values

> are returned. As an example if I transform an image by 0.2 or 0.8 pixels on the x, the same values are

> returned. I tested this on the hairy_potter_nofur_nomask image.

I've attached a Nuke test script to sanity-check the DeepTransform vs flat Transform results. Taking into account the difference in filtering you should see the correct subpixel offsets for 0.2 and 0.8.when you switch between inputs 1&2.

However after running a render test using deepTransformExample I'm seeing the same issue as you in both fwd & bwd modes. The nuke plugin calls DeepTransform::sample() and I'm very confident that's working, and both sample() and pixelTransform() end up calling sampleSegments() which does the actual filtering work, so it's likely a fairly trivial problem somewhere.

I'll track it down.

> Also, do you have a Nuke implementaion of this call? I can write it but if you do then it would make it much faster for me.

I think the challenge with making transformPixel() a Nuke call is that Nuke's deep tiles are not random access so there's no easy way to cache all the output deep pixels that the input deep pixels will transform to. Since Nuke is limited to outputting individual scanlines you'd need to create a full frame Dcx::DeepImageOutputTile to cache the results (just like in deepTransformExample.cpp) then iterate through the input pixels, perhaps with multiple worker threads. Then the DeepImageOutputTile results can be read out into the Nuke deep tile scanline by scanline.

That's gonna eat gobs of memory...

It can be done but I don't think I can work on it anytime soon due to other priorities.

> Also, would it be possible to change the transferPixel() call to have both source and destination pixel location too?

The challenge with this is that a single input pixel can map to multiple output pixels, so specifying a single output pixel (assuming the input coord maps into the output coord at all) will produce a partial contribution of deep samples. However I *think* the partial sample combiner can now handle repeated partial additions to a single output deep pixel, but that's something to double-check.

Like the previous modification I'm not sure when I can get to it at this point in time.

Cheers,

-j

[Jonathan Egstad]

The problem is trivial - the output deep file of deepTransformExample does not contain the dcx channels since the source file did not contain them, so the filtering information is gone. If you run the same deepTransformExample command on hairy_potter_nofur.exr the filtering is there since the input file has the dcx channels, and the result matches the Nuke DeepTransform output.

Sort of an easy fix but I didn't make it very easy to add channels to the tile classes - so I'm going to expand the channel functionality a little.

[Jonathan Egstad]

Hi Laszlo,

Please try downloading the latest 2.2.2 tar balls from the opendcx web site (http://www.opendcx.org/download)

I've fixed the channel issue with deepTransformExample and added/cleaned up the channel handling in the lib a bit.

Sorry to keep posting tar balls but I haven't had time yet to get in sync with github and transition over to it fully.

Cheers,

-j

[laszlo...@primefocusworld.com]

Thanks Jonathan,

I'll give it a go now.

[laszlo...@primefocusworld.com]

Hi Jonathan,

I've been testing out the Nuke deepTransform example and just noticed something. if you transform by half a pixel (.5) the flattening seems to output the right pixel values but the deepSample node reads exactly the same deep values as before the transformation. I'm not sure if it makes any sense so I attached an example comp. I compiled the openDCX deepTransform as deepTransform_SP.  

thanks

laszlo

[Jonathan Egstad]

It's normal to have the channel values not change as often all that's happening when samples are transformed is that the spmask patterns are being resampled. Recall that OpenDCX rgba values do not have coverage baked into them and coverage is communicated solely through the spmask while alpha is purely for surface opacity. This is to avoid cracks which often occur along common-edge boundaries. The teapot example image has these kinds of seams.

I believe you're trying to use DeepTransform with deep data that does not contain spmask data, so I'm not sure what the correct workflow would be in that case. I think you need to clearly communicated your workflow, what kind of deep data you're feeding in, and what you're expecting from OpenDCX's api. If you're attempting to transform hard-surface renders that have coverage baked into the channels and no spmask info then I'm not sure DCX's transform will always produce the correct result.

Internally we have a nuke plugin called DeepPremult (I didn't release it as one of the example) which baked/unbakes the spmask coverage value into the channels - I suppose DeepTransform could have that as an option.

[laszlo...@primefocusworld.com]

This is just an observation I made while trying to make sense of the openDCX backend. What throw me off was that I was getting those deepSample values and didn't add up when flattening them. But what really matters is that the backend is working as it should.

thanks,

laszlo

[Jonathan Egstad]

No worries - I'm very happy someone else is exercising the code since you're likely doing things I haven't thought of.

However I'm not seeing the problem...

Hm. I built the dcx DeepTransform plugin as 'DeepTransform_DCX' and simplified your test script (attached.) Check the difference between inputs 1, 2.& 3. Both the stock & dcx plugins produce 0.75 as a result.and the 2D sanity check also produces 0.75.

I also built the lib and plugins for Nuke 10.5v2 in case it was a Nuke version issue and I still don't see a difference.

You're running under Windows right? Are you having to jump through hoops to compile it and possibly hitting some win/linux oddness?

Can you grab a screenshot split between the two results? Perhaps showing the Nuke pixel sampler results at the bottom of the Viewer?

[laszlo...@primefocusworld.com]

I feel I might have not explained myself properly, happens quite often ;) 

As I said (and you found) the output pixel values are the right values but inspecting the deep sample values are not correct after subpixel transformations. For example, if you offset a constant (rgba==0.5) lets say by x.7, you'll find that in the viewer the pixel values will be as you expect it to be (rgba==0.5). However, now we have 3 deep samples of 0.25, 0.25, 0.5 (rgba). the problem I find is that if you collapse these samples you would get a different final pixel value than 0.5, instead you'd get 0.71875. 

Hope this is a better explanation. Also, just to avoid confusion I'm using Centos 6.5 and Nuke 10.0v5. 

[Jonathan Egstad]

However, now we have 3 deep samples of 0.25, 0.25, 0.5 (rgba)

I don't think this is correct at all - are you sure that all three samples have R=0.25, G=0.25, B=0.25, but A=0.5? I don't see this and would be surprised if that was the case. I see this from DeepSample_DCX (see the attached screenshots as well):

5 {1.0 1.0   0.25 0.25 0.25 0.25} partial,additive

4 {1.0 1.0   0.25 0.25 0.25 0.25} partial,additive

3 {1.0 1.0   0.5  0.5  0.5  0.5 } full

2 {0.9 0.9   0.25 0.25 0.25 0.25} partial,additive

1 {0.9 0.9   0.25 0.25 0.25 0.25} partial,additive

0 {0.9 0.9   0.5  0.5  0.5  0.5 } full

If you can verify against the screenshots your sample values and please respond with screenshots of your own, if you are seeing those anomalous values then something is wrong.

You may already know this from our previous correspondence but I'll go over it again, I'm still in the process of writing this part of the docs up:

This is where the new 2.2.2 partial subpixel-coverage encoding comes into play. You should open the DeepSubpixelMask node in that script and change the 'do sample' knob to see each sample's metadata values.

At certain fractional subpixel offsets the 8x8 spmask abuffer doesn't have enough resolution to adequately capture the coverage weight, leading to aliasing of the transformed image. Increasing the spmask resolution is not a practical option so to compensate I expanded the partial-spcoverage flag from a single bit to 9bits (only 8 bits are actually stored since 0x00 is logically the same as 0x100, so the 9th bit is implied) which gives 0-256 levels of partial subpixel-coverage weight.

A sample who's transformed subpixel bins straddle multiple output subpixel bins is split into multiple output deep samples, where one output sample has the full-spcoverage mask with no value weighting and the other output samples are the partial-spcoverage samples that include value weighting and are additive. Additive-ness is a key detail when flattening these samples!

So, if DeepSubpixelMask is looking at pixel 512,93 in the test script and you're looking at sample 0 you should see metadata 'flags [none(Log,NoMatte]' and an spmask pattern that's missing column 5. Sample 1 has metadata 'flags [Log,Additive,spCvg(0.500)]' with a spmask pattern that fills in column 5. Continue down the sample list and you'll see the alternating pattern of full/partial spmasks.

The reason that samples 1,2 and 4,5 are duplicates is that the combiner method failed to combine those samples together into a 0.5 full-spcoverage sample. The combiner can be improved in this respect as there are cases where the output does not flatten to exactly the right answer - try transforming by x500.7 y0.3 which outputs 0.74609 instead of 0.75.

[Jonathan Egstad]

Ok, I think I misread you again...   :(

You meant this, right?

However, now we have 3 deep samples:

 2 {0.25 0.25 0.25 0.25}

 1 {0.25 0.25 0.25 0.25}

 0 {0.5  0.5  0.5  0.5 }

If so the channel values are ok, however only having 3 samples isn't correct as there should be both the 0.9 surface and 1.0 surface represented in the sample list, unless you turned one of them off.

Please include Zfront/Zback values as well!

[laszlo...@primefocusworld.com]

AHH, I SEE - its an additive you use to calc the the full pixel instead of the OpenEXR deep pixel flattening algo. It makes sense now. I'll go back and start again.

thanks