In my last blog post, How the 3ds Max Scanline Renderer Computes Tangent and Binormal Vectors for Normal Mapping, I talked about the 3ds Max tangent basis computation for normal map baking and rendering. Today I add some useful new information regarding face angle weighting.

The algorithm I shared in my previous post was the algorithm used by ComputeTangentAndBinormal() exposed in gutil.h for computing tangent basis. The other part of the normal baking algorithm which is very important is how normals are computed by Mesh::buildNormals().

I can't share the whole algorithm here, it is too big and hairy. However there is an important step which I want to highlight:

// Get the face angle at this vertex - the arccosine of the dot prod of the vectors:
float faceAngle = 1.0f;
if (useFaceAngleWeighting) {
  float dp = DotProd (edgeDir[j], -edgeDir[(j+2)%3]);
  // Correct for possible acosf-breaking floating-pt error:
  if (dp>1.0f) dp=1.0f;
  else if (dp<-1.0f) dp = -1.0f;
  faceAngle = acosf (dp);
}

Here is the current version of the code used under the hood for computing vertex normals:

static void ComputeVertexNormals(Mesh *mesh, std::vector<BasisVert> &FNorms, std::vector<VNormal> &VNorms, bool NegScale, Tab<int>m_MapChannels)
{
  int NumVert = mesh->getNumVerts();
  int NumFace = mesh->getNumFaces(); 
  Face* face = mesh->faces;
  Point3* Verts = mesh->verts;

  VNorms.resize(NumVert);
  FNorms.resize(NumFace);

  for(int i=0; i < NumFace; i++, face++)
  {
    int A = face->v[0];
    int B = face->v[1];
    int C = face->v[2];

    Point3 Vt0 = Verts[A];
    Point3 Vt1 = Verts[B];
    Point3 Vt2 = Verts[C];

    Point3 Normal = FNormalize((Vt1 - Vt0) ^ (Vt2 - Vt0));
    unsigned long Sg = face->smGroup;

    if(Sg)
    {
      VNorms[A].AddNormal(Normal,Sg);
      VNorms[B].AddNormal(Normal,Sg);
      VNorms[C].AddNormal(Normal,Sg);
    }
    else
    {
      FNorms[i].m_Normal = Normal;
    }
  }

  for(int i=0; i < NumVert; i++) 
    VNorms[i].Normalize();
}

What is missing here is the face weighting step used by the Mesh::buildNormals() method. Here is a new version of the algorithm that corrects this problem:

static void ComputeVertexNormals(Mesh *mesh, std::vector<BasisVert> &FNorms, std::vector<VNormal> &VNorms,bool NegScale, Tab<int>m_MapChannels)
{
  int NumVert = mesh->getNumVerts();
  int NumFace = mesh->getNumFaces();
  Face* face = mesh->faces;
  Point3* Verts = mesh->verts;

  VNorms.resize(NumVert);
  FNorms.resize(NumFace);

  bool useFaceAngleWeighting = GetVertexNormalsControl()->GetUseFaceAngles();

  for (int i=0; i < NumFace; i++, face++)
  {
    int A = face->v[0];
    int B = face->v[1];
    int C = face->v[2];

    Point3 Vt0 = Verts[A];
    Point3 Vt1 = Verts[B];
    Point3 Vt2 = Verts[C];

    Point3 Normal = FNormalize((Vt1 - Vt0) ^ (Vt2 - Vt0));
    unsigned long Sg = face->smGroup;

    // Compute edge vector directions (so we can get vertex angles later)
    float faceAngle[3];
    faceAngle[0] = 1.0f;
    faceAngle[1] = 1.0f;
    faceAngle[2] = 1.0f;
    if (useFaceAngleWeighting) {
      Point3 edgeDir[3];
      edgeDir[0] = FNormalize(Vt1 - Vt0);
      edgeDir[1] = FNormalize(Vt2 - Vt1);
      edgeDir[2] = FNormalize(Vt0 - Vt2);


      // Get the face angle at this vertex - the arccosine of the dot prod of the vectors:
      faceAngle[0] = GetFaceAngleFromEdgeDir(edgeDir[0], edgeDir[2]);
      faceAngle[1] = GetFaceAngleFromEdgeDir(edgeDir[1], edgeDir[0]);
      faceAngle[2] = GetFaceAngleFromEdgeDir(edgeDir[2], edgeDir[1]);
    }

    if(Sg) {
      VNorms[A].AddNormal(faceAngle[0] * Normal, Sg);
      VNorms[B].AddNormal(faceAngle[1] * Normal, Sg);
      VNorms[C].AddNormal(faceAngle[2] * Normal, Sg);
    }
    else
      FNorms[i].m_Normal = Normal;
  }

  for(int i=0; i < NumVert; i++)
    VNorms[i].Normalize();
}

I should make a big caveat that even if you were to apply this code to a custom hardware rendered mesh object it isn't guaranteed not to break something else in 3ds Max. Nor is it guaranteed to fix all of the possible sources of discrepancies between the generated normal map and the normal map displayed in the viewport. For example there are indications that there may be discrepancies in the orthogonalization method used between the tangent basis used when generating the map and and the basis used when rendering map. This is not clear yet ... well at least not to me!

I hope nonetheless that this information can be helpful to our SDK users by making the process of normal mapping used in 3ds Max more transparent.

I should also point out that while our engineers are working on the problem the community may have a workaround soon! I suggest keeping an eye open on the PolyCount forums.

As a final note, for those who aren't aware, there is an interesting partial workaround of the normal map baking problem in 3ds Max (i.e. normal maps generated by 3ds Max not displaying as expected in the viewport) identified by PolyCount users Undoz and CW, which involves applying the "Edit Normals" modifier to your models.

Big thank you to the members of the PolyCount forums for their patience and hard work in investigating the problem!