As Base Material I'm using a Vray Mtl. By the way I've noticed that turning on the Refraction to maximum white prevent the cube to receive color reflection from the plane, even though I guess it's supposed to don't affect the cube.
I'm not entirely sure I understand what you are trying to do, but I can tell you that you did setup the MtlWrapper coreectly. You could have also just made these adjustments in the VRay Propertins menu of the objects you want to matte. You really shouldn't have to make it fully refactive to remove color, but there is an occasional need to un-tick Receive GI.
There are some workarounds for this issue. But what I'd do is set the base material color in your wrapper material to a neutral gray... even better would be setting it to something similar to what you'll composite with, and then play with it (saturation and brightness) to get the desired result. After all, color bleeding is a natural thing. I try to stay away from "artificial workarounds"... but it depends on your workflow. You could also play with the color mapping settings. Some of them can handle this bleedings but they also have other "side-effects" to consider. Other option is the vray material overide, where you can set a diferent material for the GI's response.
Behind model I have vrayplane like a wall of the house with vraymtlwrapper for masking model to photo but on glass windows I see this vraymtlwrapper color and I cant see background house through the windows. Everything else is OK, shadows, GI, only cant see through the windows. When I disable Visible in refraction in Vray parameters of vrayplane, I can see through the windows, and its what I want, but there is on shadows and GI on the wall. How can I render it only with shadows and GI on the wall of the background building but with transparent window glasses?
If you've ticked it to be a matte object it shouldn't appear in the rgb channel. What does your alpha channel look like? It should be black and white, black where that object is. Save you image as a .tga or a file type you are used to using that will save an alpha channel. Then when you select your objects, all of the background gets cutout.
Looking at it quickly, although there are some missing maps, I think you can leave the mtlWrapper on as it's effectively doing the same as the vray properties, but put the alpha contribution on -1.0. Then on your glass material tick affect shadows in the refraction area and change the Affect channels to Color and Alpha in the refraction area too. Also, remove your reflection map and change the color the 250,250,250 and tick Fresnel reflections.
Thanks a lot for your maxfile.. there was different setting. When in reflection for windows glass is selected Affect channels to Color + Alpha, then in alpha channel of rendered picture is only front wall of the model, then I switch it to Color only and everything is OK. Now I can select it and retouch it in photoshop. But still I dont understand, why refraction is still displaying mtl wrapper in windows. If you scale matted plane behind model to hide whole house on background glasses will display this mtl wrapper.
I'm trying to use the 'vraymtlwrapper' to gather the shadows under some simple teapots (I'm setting up a template so don't need the real objects) in an HDRI lighting scene. I have a plane under the teapots and the shadows are being generated but the plane is still just barely visible. I can't get it to disappear.
I've tried a 'Standard, VRayMtl and a Matte/Shadow' in the Base Material. Neigher removes the lingering base. It looks like it's about 10% density. The non-occluded areas is not going pure alpha black it seems.
Hey there! I'd been looking for a solution to this issue for a long time and had been managing by masking and feathering edges in compositing. But I finally found a solution, which can be done all inside of max. Its not a 100% perfect solution but it works pretty well. You need to create a square jpg file which is essentially black and has a feathered white circle in the centre. The size and feather need to be adjusted in a way the edge still remains pure black ie. 0,0,0. Now you can use this image as an opacity map for the base material of your materialwrapper material. This also works with matte reflections but I've seen some minor issues there, but with shadows, it works pretty well.
Looks like we both came up with the same solution. I'm glad you posted it though so it will help others. I negelected to come back and report what I was doing to mitigate the issue. I wish there was a more direct way but this works well enough for now.
Sure, here you go, in fact, I just spent 45-minutes trying to get it to work at all. I've included a screen capture that shows the render and the settings. Forget about the edges, now I can't get it to even begin to work.
It's possible I have accidentally turned something off or on that prevents it from working but I can't find it to save my life. Let me know what you find. I'd love for it to be some ingorant user error but I don't believe it is.
I hear what you say about using a Standard Material in the base slot. Before I sent the first file to you I looked up 3 different tutorials on MtlWrapper and two of them use a standard material and one used a V-Ray material. Makes no difference here. If I use a V-Ray Mtl I get the same result. In fact, during that 45-minutes that I played with it tonight, it was a V-Ray Mtl most of the time. The screenshot I attached was of my last attempts where I did use a Standard material and that's where I left. it.
First of all; if your "ground" object is too small, you wont be able to see full shadows in some cases which depends on the angle of your light, hitting your object. Also it's perfectly normal to see such artifacts because V-Ray uses global illumination and your scene scatters photons everywhere which need to be sampled and the thing you see in your renders are these samples on your mtl wrapper object.
On the other hand, putting a geometry under MtlWrapper isn't something we usually want to do, because it's just a material which is being carried by a geometry so your geometry here creates the shadows on the plane under MtlWrapper.
Ah ha, so, it was an ignorant user error. I shot myself in the foot by having that plane under my wrapper while I was testing. So, all of my tests were destined to fail because the base file was problematic. I was thinking that by having something underneath, I'd have the wrapper create a shadow on top of it just floating there. But by having the upper plane with 'cast shadow' enabled, I couldn't see the effect. I didn't realize the wrapper would cut through all of the layers below it. I just stacked 3 and it cut through them all. Interesting. I'm guessing you can control that too.
I just now moved that bottom plane a lot farther down so that the upper plane's cast shadow was not cast directly underneath and it worked. So then I just removed the lower plane and turned off the 'cast shadow' of the upper plane as you suggested and it looks like it's working.
Now that I know that I was doing a few things wrong I'll keep an eye on it next time I need to use it. I don't know why I was having so much trouble with something so simple. Sometimes we can't see the forest for the trees.
I appreciate your looking at this. I'll post back here should the edge issue crop back up but I think I have a better understanding about how it works. No matter how well we know Max it's always some little thing that you rarely use that trips you up. I've used MtlWrapper maybe 3 times in 29 years in Max. But, you'd think that after using Max professional for that long I'd be able to figure it out. Go figure. At least it's working now.
I looked at this again and I was making two mistakes. I did not have 'cast shadow' turned off for the wrapper plane and my wrapper planes have been too small. I usually made them just big enough to catch the length of the shadow being cast and missing the fact that there is still other GI in the scene as you pointed out. That subtle GI farther away from my object was causing a faint shadow edge where the plane terminated. But making that wrapper plane larger, the GI finally dies out and the edges can't be seen.
e59dfda104