Maya Batch Bake question

[Lamont]

I am having issues with Maya's Batch bake and objects with alpha. I can't get it to respect the alpha on the mesh when I am trying to bake in AO. It goes by the poly. I am pretty sure I had it set up right.

Any input would be helpful :)

[MM]

are you baking from high poly to low poly ?
because the batch bake is used to bake from itself, and the alpha it bakes is the alpha of the UV region i think

if you are baking alpha from high poly to low poly you need to use "transfer maps" utility

[Lamont]

No, not going high to low. Just selecting the mesh and hitting the button :), I could try transfer maps and just duplicate the mesh.

[MM]

i dont understand then, where are you trying to bake the alpha from and why ?

if you already have a alpha texture then why bake it again ?

[Lamont]

Well, it's a layered texture (leaves on a bush). Top layer is supposed to be AO (on multiply), second is the alpha'd leaves/branch texture.

The AO, when baked doesn't clip to the alpha of the leaves, no matter what order I put the AO in on the shader. So I wanted to bake the AO w/alpha from the leaves to the target UV layout.

http://www.digitalweaponx.net/Forums/GameA/bridge00.jpg

[MM]

that occlusion shadows are cast based on geometry and not texture alpha. unless someone knows how to bake raytraced shadows i dont know how you would do that.

[Lamont]

Yeah, I am gonna figure it out :), it's like the last couple of things I need to do to finish this model:P

[ciprian0077]

Why don't you put a mask over your ambient baked map to hide the dark parts of the planes?...

[Lamont]

Why don't you put a mask over your ambient baked map to hide the dark parts of the planes?...There is a mask on the diffuse color. The AO should be above diffuse on multiply, and if there is nothing underneath to multiply, then it shouldn't show up... but it is :(.

[MM]

one thing you can do is instead of using quad planes for the leaves, actually model out the leaves based on your texture then lay them out and bake the shadows.

because it does not matter what texture you have and whether it is layered or not, the occlusion will be based on mesh always.

[Lamont]

True. But I think I can just use Unreals AO post process... gonna mess with it later today.

[Lamont]

I think I figured it out on doing it within Maya: Just make a quick AO light rig and turn on ray-tracing or GI, will be slow, but it might work. I'll take a stab at it when I get home from work.