Normal maps and smoothing groups

[Chase]

When separating Smoothing Groups for a normal map bake, I'm wondering if my process would work...

I watched Alec Moody's process at 3DMotive.com on how to render a normal map. He used the Text Tool script to automatically create the Smoothing Groups wherever he had made his UV splits. He doesn't worry about how the smoothing will look on the model until the very last step, which would drive me insane having shading errors everywhere. Is there any rhyme or reason to create UV splits first and then the smoothing groups?

I tried using the Text Tool operation that would create Smoothing Groups from UV Shells and all it does it make a mess of the Smoothing Groups I had already established. Not sure if you have to have everything in one smoothing group and then apply the operation. My process consists of selecting my entire model after I completing the unwrap, and Splitting the UV Shells by Smoothing Groups in the default 3ds Max Unwrap? Would that be the same thing? Would that be as effective in terms of creating less seems?

[swag]

What function in Textools lets you create smoothing groups based on UV splits? You can split the UVs according to your smoothing groups but not the other way, unless you have no smooth groups, then it assigns a temporary one (60 degrees) but not to the mesh itself. Still, I don't see how you derive smooth groups from UV shells.

Honestly this issue still confuses me sometimes because people seem to handle it differently. The way I do it is set up the SG before you unwrap. And keep in mind that for hard edges you want to separate the UV islands. Sometimes you might have to break the rule and see what the bake looks like or just go back and adjust the SM after UVing. I also use a combination of chamfered edges and hard edges, but it depends on the asset.

I never use the Uv to smoothing group function in textools though, so I can't comment on that directly.

Hope that helps.

[Chase]

I may have just misunderstood what he said. So do you just use the Select Smoothing Groups by Faces button in the UV Unwrap and then just split them from there? This wouldn't seem like the best resolution since it would double my vert count.

Here's what it looks like when I select faces by smoothing group 1


Not quite sure what the next step is since this selects the hard edges yet splitting them would give me a ton more seems...

[swag]

Selecting Smoothing Groups by Faces button in the UV Unwrap mod won't change your vertex count. I don't know where you are getting that from. Nothing you do in the UV mod will change your geometry.

Have you just started unwrapping this model? If so, I always start with Flatten Mapping. Then depending on how your smoothing groups are set up, you can then start breaking/stitching pieces together. Use the select faces by smoothing groups to see where the groups are if you forget.

I'll be nice and do an example! It's just a quick and dirty one though, but hopefully it helps.

In the first pic the smoothing groups are the top selected faces, and the middle section. For the middle section there will be hard edges of course and the faces are split accordingly. But forget about that section for now. The selected faces should not have hard edges and so they are in one smoothing group. And when unwrapped they are also attached (one UV island). So everything will be ok that you won't have hard edge seams in your normals.



In this pic I used a combination of chamfered edges and hard edges. The smoothing groups are set up in the same way as above (top and bottom the same group, middle one group). It illustrates the concept but in a different scenario. Basically the middle section will be smoothed properly since the edges are chamfered and the UVs are all one island. You'll get hard edges at the top/bottom edges. Also the chamfered edges won't produce a normal map seam even if they aren't all one island. That's the benefit of chamfering them.



The models with the normals applied pretty much looked like the LP versions, so I didn't bother posting them. Also you might still get some errors, but they get be fixed later in photoshop. Sometimes it's not worth the time to bake something again and again to fix a little problem.

That's it I might have made some mistakes, but it should be ok.

[Chase]

This process is only if you're baking a normal map right? When I said doubles the very count I meant in game, not in Max. Thanks for the example! I just want to be clear, hard edges are where one smoothing group starts and another one ends?

[swag]

Yeah, but you would want to keep the smoothing groups after you apply the normal map (but I think the normal overrides them anyway).

I still don't see how this would double your vert count in game.

And to answer your last question, yes. Might be exceptions...don't think so.

I recommend reading this, since there might be stuff I didn't mention or got wrong:

NormalMap - Polycount Wiki

[Chase]

It should be in that wiki. I've already read it haha. But thanks anyways