Followup: Normal Mapping Without Precomputed Tangents

This post is a fol­low-up to my 2006 ShaderX5 arti­cle [4] about nor­mal map­ping with­out a pre-com­put­ed tan­gent basis. In the time since then I have refined this tech­nique with lessons learned in real life. For those unfa­mil­iar with the top­ic, the moti­va­tion was to con­struct the tan­gent frame on the fly in the pix­el shad­er, which iron­i­cal­ly is the exact oppo­site of the moti­va­tion from [2]:

Since it is not 1997 any­more, doing the tan­gent space on-the-fly has some poten­tial ben­e­fits, such as reduced com­plex­i­ty of asset tools, per-ver­tex band­width and stor­age, attribute inter­po­la­tors, trans­form work for skinned mesh­es and last but not least, the pos­si­bil­i­ty to apply nor­mal maps to any pro­ce­du­ral­ly gen­er­at­ed tex­ture coor­di­nates or non-lin­ear deformations.

Intermission: Tangents vs Cotangents

The way that nor­mal map­ping is tra­di­tion­al­ly defined is, as I think, flawed, and I would like to point this out with a sim­ple C++ metaphor. Sup­pose we had a class for vec­tors, for exam­ple called Vector3, but we also had a dif­fer­ent class for cov­ec­tors, called Covector3. The lat­ter would be a clone of the ordi­nary vec­tor class, except that it behaves dif­fer­ent­ly under a trans­for­ma­tion (Edit 2018: see this arti­cle for a com­pre­hen­sive intro­duc­tion to the the­o­ry behind cov­ec­tors and dual spaces). As you may know, nor­mal vec­tors are an exam­ple of such cov­ec­tors, so we’re going to declare them as such. Now imag­ine the fol­low­ing function:

Vector3 tangent;
Vector3 bitangent;
Covector3 normal;
Covector3 perturb_normal( float a, float b, float c )
    return a * tangent +
           b * bitangent + 
           c * normal; 
           // ^^^^ compile-error: type mismatch for operator +

The above func­tion mix­es vec­tors and cov­ec­tors in a sin­gle expres­sion, which in this fic­tion­al exam­ple leads to a type mis­match error. If the normal is of type Covector3, then the tangent and the bitangent should be too, oth­er­wise they can­not form a con­sis­tent frame, can they? In real life shad­er code of course, every­thing would be defined as float3 and be fine, or rather not.

Mathematical Compile Error

Unfor­tu­nate­ly, the above mis­match is exact­ly how the ‘tan­gent frame’ for the pur­pose of nor­mal map­ping was intro­duced by the authors of [2]. This type mis­match is invis­i­ble as long as the tan­gent frame is orthog­o­nal. When the exer­cise is how­ev­er to recon­struct the tan­gent frame in the pix­el shad­er, as this arti­cle is about, then we have to deal with a non-orthog­o­nal screen pro­jec­tion. This is the rea­son why in the book I had intro­duced both \vec{T} (which should be called co-tan­gent) and \vec{B} (now it gets some­what sil­ly, it should be called co-bi-tan­gent) as cov­ec­tors, oth­er­wise the algo­rithm does not work. I have to admit that I could have been more artic­u­late about this detail. This has caused real con­fu­sion, cf from

The dis­crep­an­cy is explained above, as my ‘tan­gent vec­tors’ are real­ly cov­ec­tors. The def­i­n­i­tion on page 132 is con­sis­tent with that of a cov­ec­tor, and so the frame \left(\vec{T}|\vec{B}|\vec{N}\right) should be called a cotan­gent frame.

Intermission 2: Blinns Perturbed Normals (History Channel)

In this sec­tion I would like to show how the def­i­n­i­tion of \vec{T} and \vec{B} as cov­ec­tors fol­lows nat­u­ral­ly from Blinn’s orig­i­nal bump map­ping paper [1]. Blinn con­sid­ers a curved para­met­ric sur­face, for instance, a Bezi­er-patch, on which he defines tan­gent vec­tors \vec{p}_u and \vec{p}_v as the deriv­a­tives of the posi­tion \vec{p} with respect to u and v.

In this con­text it is a con­ven­tion to use sub­scripts as a short­hand for par­tial deriv­a­tives, so he is real­ly say­ing \vec{p}_u = {\partial\vec{p} \over \partial u}, etc. He also intro­duces the sur­face nor­mal \vec{N} = \vec{p}_u \times \vec{p}_v and a bump height func­tion f, which is used to dis­place the sur­face. In the end, he arrives at a for­mu­la for a first order approx­i­ma­tion of the per­turbed normal:

    \[\vec{N}' \simeq \vec{N} + \frac{f_u \vec{N} \times \vec{p}_v + f_v \vec{p}_u \times \vec{N}}{|\vec{N}|} ,\]

I would like to draw your atten­tion towards the terms \vec{N} \times \vec{p}_v and \vec{p}_u \times \vec{N}. They are the per­pen­dic­u­lars to \vec{p}_u and \vec{p}_v in the tan­gent plane, and togeth­er form a vec­tor basis for the dis­place­ments f_u and f_v. They are also cov­ec­tors (this is easy to ver­i­fy as they behave like cov­ec­tors under trans­for­ma­tion) so adding them to the nor­mal does not raise said type mis­match. If we divide these terms one more time by |\vec{N}| and flip their signs, we’ll arrive at the ShaderX5 def­i­n­i­tion of \vec{T} and \vec{B} as follows:

    \begin{align*} \vec{T} &= -\frac{\vec{N} \times \vec{p}_v}{|\vec{N}|^2} = \nabla u, & \vec{B} &= -\frac{\vec{p}_u \times \vec{N}}{|\vec{N}|^2} = \nabla v, \end{align*}

    \[\vec{N}' \simeq \hat{N} - f_u \vec{T} - f_v \vec{B} ,\]

where the hat (as in \hat{N}) denotes the nor­mal­ized nor­mal. \vec{T} can be inter­pret­ed as the nor­mal to the plane of con­stant u, and like­wise \vec{B} as the nor­mal to the plane of con­stant v. There­fore we have three nor­mal vec­tors, or cov­ec­tors, \vec{T}, \vec{B} and \vec{N}, and they are the a basis of a cotan­gent frame. Equiv­a­lent­ly, \vec{T} and \vec{B} are the gra­di­ents of u and v, which is the def­i­n­i­tion I had used in the book. The mag­ni­tude of the gra­di­ent there­fore deter­mines the bump strength, a fact that I will dis­cuss lat­er when it comes to scale invariance.

A Little Unlearning

The mis­take of many authors is to unwit­ting­ly take \vec{T} and \vec{B} for \vec{p}_u and \vec{p}_v, which only works as long as the vec­tors are orthog­o­nal. Let’s unlearn ‘tan­gent’, relearn ‘cotan­gent’, and repeat the his­tor­i­cal devel­op­ment from this per­spec­tive: Peer­cy et al. [2] pre­com­putes the val­ues f_u and f_v (the change of bump height per change of tex­ture coor­di­nate) and stores them in a tex­ture. They call it ’nor­mal map’, but is a real­ly some­thing like a ’slope map’, and they have been rein­vent­ed recent­ly under the name of deriv­a­tive maps. Such a slope map can­not rep­re­sent hor­i­zon­tal nor­mals, as this would need an infi­nite slope to do so. It also needs some ‘bump scale fac­tor’ stored some­where as meta data. Kil­gard [3] intro­duces the mod­ern con­cept of a nor­mal map as an encod­ed rota­tion oper­a­tor, which does away with the approx­i­ma­tion alto­geth­er, and instead goes to define the per­turbed nor­mal direct­ly as

    \[\vec{N}' = a \vec{T} + b \vec{B} + c \hat{N} ,\]

where the coef­fi­cients a, b and c are read from a tex­ture. Most peo­ple would think that a nor­mal map stores nor­mals, but this is only super­fi­cial­ly true. The idea of Kil­gard was, since the unper­turbed nor­mal has coor­di­nates (0,0,1), it is suf­fi­cient to store the last col­umn of the rota­tion matrix that would rotate the unper­turbed nor­mal to its per­turbed posi­tion. So yes, a nor­mal map stores basis vec­tors that cor­re­spond to per­turbed nor­mals, but it real­ly is an encod­ed rota­tion oper­a­tor. The dif­fi­cul­ty starts to show up when nor­mal maps are blend­ed, since this is then an inter­po­la­tion of rota­tion oper­a­tors, with all the com­plex­i­ty that goes with it (for an excel­lent review, see the arti­cle about Reori­ent­ed Nor­mal Map­ping [5] here).

Solution of the Cotangent Frame

The prob­lem to be solved for our pur­pose is the oppo­site as that of Blinn, the per­turbed nor­mal is known (from the nor­mal map), but the cotan­gent frame is unknown. I’ll give a short revi­sion of how I orig­i­nal­ly solved it. Define the unknown cotan­gents \vec{T} = \nabla u and \vec{B} = \nabla v as the gra­di­ents of the tex­ture coor­di­nates u and v as func­tions of posi­tion \vec{p}, such that

    \begin{align*} \mathrm{d} u &= \vec{T} \cdot \mathrm{d} \vec{p} , & \mathrm{d} v &= \vec{B} \cdot \mathrm{d} \vec{p} , \end{align*}

where \cdot is the dot prod­uct. The gra­di­ents are con­stant over the sur­face of an inter­po­lat­ed tri­an­gle, so intro­duce the edge dif­fer­ences \Delta u_{1,2}, \Delta v_{1,2} and \Delta \vec{p_{1,2}}. The unknown cotan­gents have to sat­is­fy the constraints

    \begin{align*} \Delta u_1 &= \vec{T} \cdot \Delta \vec{p_1} , & \Delta v_1 &= \vec{B} \cdot \Delta \vec{p_1} , \\ \Delta u_2 &= \vec{T} \cdot \Delta \vec{p_2} , & \Delta v_2 &= \vec{B} \cdot \Delta \vec{p_2} , \\ 0 &= \vec{T} \cdot \Delta \vec{p_1} \times \Delta \vec{p_2} , & 0 &= \vec{B} \cdot \Delta \vec{p_1} \times \Delta \vec{p_2} , \end{align*}

where \times is the cross prod­uct. The first two rows fol­low from the def­i­n­i­tion, and the last row ensures that \vec{T} and \vec{B} have no com­po­nent in the direc­tion of the nor­mal. The last row is need­ed oth­er­wise the prob­lem is under­de­ter­mined. It is straight­for­ward then to express the solu­tion in matrix form. For \vec{T},

    \[\vec{T} = \begin{pmatrix} \Delta \vec{p_1} \\ \Delta \vec{p_2} \\ \Delta \vec{p_1} \times \Delta \vec{p_2} \end{pmatrix}^{-1} \begin{pmatrix} \Delta u_1 \\ \Delta u_2 \\ 0 \end{pmatrix} ,\]

and anal­o­gous­ly for \vec{B} with \Delta v.

Into the Shader Code

The above result looks daunt­ing, as it calls for a matrix inverse in every pix­el in order to com­pute the cotan­gent frame! How­ev­er, many sym­me­tries can be exploit­ed to make that almost dis­ap­pear. Below is an exam­ple of a func­tion writ­ten in GLSL to cal­cu­late the inverse of a 3×3 matrix. A sim­i­lar func­tion writ­ten in HLSL appeared in the book, and then I tried to opti­mize the hell out of it. For­get this approach as we are not going to need it at all. Just observe how the adju­gate and the deter­mi­nant can be made from cross products:

mat3 inverse3x3( mat3 M )
    // The original was written in HLSL, but this is GLSL, 
    // therefore
    // - the array index selects columns, so M_t[0] is the 
    //   first row of M, etc.
    // - the mat3 constructor assembles columns, so 
    //   cross( M_t[1], M_t[2] ) becomes the first column
    //   of the adjugate, etc.
    // - for the determinant, it does not matter whether it is
    //   computed with M or with M_t; but using M_t makes it
    //   easier to follow the derivation in the text
    mat3 M_t = transpose( M ); 
    float det = dot( cross( M_t[0], M_t[1] ), M_t[2] );
    mat3 adjugate = mat3( cross( M_t[1], M_t[2] ),
                          cross( M_t[2], M_t[0] ),
                          cross( M_t[0], M_t[1] ) );
    return adjugate / det;

We can sub­sti­tute the rows of the matrix from above into the code, then expand and sim­pli­fy. This pro­ce­dure results in a new expres­sion for \vec{T}. The deter­mi­nant becomes \left| \Delta \vec{p_1} \times \Delta \vec{p_2} \right|^2, and the adju­gate can be writ­ten in terms of two new expres­sions, let’s call them \Delta \vec{p_1}_\perp and \Delta \vec{p_2}_\perp (with \perp read as ‘perp’), which becomes

    \[\vec{T} = \frac{1}{\left| \Delta \vec{p_1} \times \Delta \vec{p_2} \right|^2} \begin{pmatrix} \Delta \vec{p_2}_\perp \\ \Delta \vec{p_1}_\perp \\ \Delta \vec{p_1} \times \Delta \vec{p_2} \end{pmatrix}^\mathrm{T} \begin{pmatrix} \Delta u_1 \\ \Delta u_2 \\ 0 \end{pmatrix} ,\]

    \begin{align*} \Delta \vec{p_2}_\perp &= \Delta \vec{p_2} \times \left( \Delta \vec{p_1} \times \Delta \vec{p_2} \right) , \\ \Delta \vec{p_1}_\perp &= \left( \Delta \vec{p_1} \times \Delta \vec{p_2} \right) \times \Delta \vec{p_1} . \end{align*}

As you might guessed it, \Delta \vec{p_1}_\perp and \Delta \vec{p_2}_\perp are the per­pen­dic­u­lars to the tri­an­gle edges in the tri­an­gle plane. Say Hel­lo! They are, again, cov­ec­tors and form a prop­er basis for cotan­gent space. To sim­pli­fy things fur­ther, observe:

  • The last row of the matrix is irrel­e­vant since it is mul­ti­plied with zero.
  • The oth­er matrix rows con­tain the per­pen­dic­u­lars (\Delta \vec{p_1}_\perp and \Delta \vec{p_2}_\perp), which after trans­po­si­tion just mul­ti­ply with the tex­ture edge differences.
  • The per­pen­dic­u­lars can use the inter­po­lat­ed ver­tex nor­mal \vec{N} instead of the face nor­mal \Delta \vec{p_1} \times \Delta \vec{p_2}, which is sim­pler and looks even nicer.
  • The deter­mi­nant (the expres­sion \left| \Delta \vec{p_1} \times \Delta \vec{p_2} \right|^2) can be han­dled in a spe­cial way, which is explained below in the sec­tion about scale invariance.

Tak­en togeth­er, the opti­mized code is shown below, which is even sim­pler than the one I had orig­i­nal­ly pub­lished, and yet high­er quality:

mat3 cotangent_frame( vec3 N, vec3 p, vec2 uv )
    // get edge vectors of the pixel triangle
    vec3 dp1 = dFdx( p );
    vec3 dp2 = dFdy( p );
    vec2 duv1 = dFdx( uv );
    vec2 duv2 = dFdy( uv );
    // solve the linear system
    vec3 dp2perp = cross( dp2, N );
    vec3 dp1perp = cross( N, dp1 );
    vec3 T = dp2perp * duv1.x + dp1perp * duv2.x;
    vec3 B = dp2perp * duv1.y + dp1perp * duv2.y;
    // construct a scale-invariant frame 
    float invmax = inversesqrt( max( dot(T,T), dot(B,B) ) );
    return mat3( T * invmax, B * invmax, N );

Scale invariance

The deter­mi­nant \left| \Delta \vec{p_1} \times \Delta \vec{p_2} \right|^2 was left over as a scale fac­tor in the above expres­sion. This has the con­se­quence that the result­ing cotan­gents \vec{T} and \vec{B} are not scale invari­ant, but will vary inverse­ly with the scale of the geom­e­try. It is the nat­ur­al con­se­quence of them being gra­di­ents. If the scale of the geomtery increas­es, and every­thing else is left unchanged, then the change of tex­ture coor­di­nate per unit change of posi­tion gets small­er, which reduces \vec{T} = \nabla u = \left( \frac{\partial u}{\partial x}, \frac{\partial u}{\partial y}, \frac{\partial u}{\partial z} \right) and sim­i­lar­ly \vec{B} in rela­tion to \vec{N}. The effect of all this is a dimin­ished per­tu­ba­tion of the nor­mal when the scale of the geom­e­try is increased, as if a height­field was stretched.

Obvi­ous­ly this behav­ior, while total­ly log­i­cal and cor­rect, would lim­it the use­ful­ness of nor­mal maps to be applied on dif­fer­ent scale geom­e­try. My solu­tion was and still is to ignore the deter­mi­nant and just nor­mal­ize \vec{T} and \vec{B} to whichev­er of them is largest, as seen in the code. This solu­tion pre­serves the rel­a­tive lengths of \vec{T} and \vec{B}, so that a skewed or stretched cotan­gent space is sill han­dled cor­rect­ly, while hav­ing an over­all scale invariance.

Non-perspective optimization

As the ulti­mate opti­miza­tion, I also con­sid­ered what hap­pens when we can assume \Delta \vec{p_1} = \Delta \vec{p_2}_\perp and \Delta \vec{p_2} = \Delta \vec{p_1}_\perp. This means we have a right tri­an­gle and the per­pen­dic­u­lars fall on the tri­an­gle edges. In the pix­el shad­er, this con­di­tion is true when­ev­er the screen-pro­jec­tion of the sur­face is with­out per­spec­tive dis­tor­tion. There is a nice fig­ure demon­strat­ing this fact in [4]. This opti­miza­tion saves anoth­er two cross prod­ucts, but in my opin­ion, the qual­i­ty suf­fers heav­i­ly should there actu­al­ly be a per­spec­tive distortion.

Putting it together

To make the post com­plete, I’ll show how the cotan­gent frame is actu­al­ly used to per­turb the inter­po­lat­ed ver­tex nor­mal. The func­tion perturb_normal does just that, using the back­wards view vec­tor for the ver­tex posi­tion (this is ok because only dif­fer­ences mat­ter, and the eye posi­tion goes away in the dif­fer­ence as it is constant).

vec3 perturb_normal( vec3 N, vec3 V, vec2 texcoord )
    // assume N, the interpolated vertex normal and 
    // V, the view vector (vertex to eye)
    vec3 map = texture2D( mapBump, texcoord ).xyz;
    map = map * 255./127. - 128./127.;
    map.z = sqrt( 1. - dot( map.xy, map.xy ) );
    map.y = -map.y;
    mat3 TBN = cotangent_frame( N, -V, texcoord );
    return normalize( TBN * map );
varying vec3 g_vertexnormal;
varying	vec3 g_viewvector;  // camera pos - vertex pos
varying vec2 g_texcoord;
void main()
    vec3 N = normalize( g_vertexnormal );
    N = perturb_normal( N, g_viewvector, g_texcoord );
    // ...

The green axis

Both OpenGL and Direc­tX place the tex­ture coor­di­nate ori­gin at the start of the image pix­el data. The tex­ture coor­di­nate (0,0) is in the cor­ner of the pix­el where the image data point­er points to. Con­trast this to most 3‑D mod­el­ing pack­ages that place the tex­ture coor­di­nate ori­gin at the low­er left cor­ner in the uv-unwrap view. Unless the image for­mat is bot­tom-up, this means the tex­ture coor­di­nate ori­gin is in the cor­ner of the first pix­el of the last image row. Quite a difference!
An image search on Google reveals that there is no dom­i­nant con­ven­tion for the green chan­nel in nor­mal maps. Some have green point­ing up and some have green point­ing down. My artists pre­fer green point­ing up for two rea­sons: It’s the for­mat that 3ds Max expects for ren­der­ing, and it sup­pos­ed­ly looks more nat­ur­al with the ‘green illu­mi­na­tion from above’, so this helps with eye­balling nor­mal maps.

Sign Expansion

The sign expan­sion deserves a lit­tle elab­o­ra­tion because I try to use signed tex­ture for­mats when­ev­er pos­si­ble. With the unsigned for­mat, the val­ue ½ can­not be rep­re­sent­ed exact­ly (it’s between 127 and 128). The signed for­mat does not have this prob­lem, but in exchange, has an ambigu­ous encod­ing for −1 (can be either −127 or −128). If the hard­ware is inca­pable of signed tex­ture for­mats, I want to be able to pass it as an unsigned for­mat and emu­late the exact sign expan­sion in the shad­er. This is the ori­gin of the seem­ing­ly odd val­ues in the sign expansion.

In Hindsight

The orig­i­nal arti­cle in ShaderX5 was writ­ten as a proof-of-con­cept. Although the algo­rithm was test­ed and worked, it was a lit­tle expen­sive for that time. Fast for­ward to today and the pic­ture has changed. I am now employ­ing this algo­rithm in real-life projects for great ben­e­fit. I no longer both­er with tan­gents as ver­tex attrib­ut­es and all the asso­ci­at­ed com­plex­i­ty. For exam­ple, I don’t care whether the COLLADA exporter of Max or Maya (yes I’m rely­ing on COLLADA these days) out­put usable tan­gents for skinned mesh­es, nor do I both­er to import them, because I don’t need them! For the artists, it does­n’t occur to them that an aspect of the asset pipeline is miss­ing, because It’s all nat­ur­al: There is a geom­e­try, there are tex­ture coor­di­nates and there is a nor­mal map, and just works.

Take Away

There are no ‘tan­gent frames’ when it comes to nor­mal map­ping. A tan­gent frame which includes the nor­mal is log­i­cal­ly ill-formed. All there is are cotan­gent frames in dis­guise when the frame is orthog­o­nal. When the frame is not orthog­o­nal, then tan­gent frames will stop work­ing. Use cotan­gent frames instead.


[1] James Blinn, “Sim­u­la­tion of wrin­kled sur­faces”, SIGGRAPH 1978

[2] Mark Peer­cy, John Airey, Bri­an Cabral, “Effi­cient Bump Map­ping Hard­ware”, SIGGRAPH 1997

[3] Mark J Kil­gard, “A Prac­ti­cal and Robust Bump-map­ping Tech­nique for Today’s GPUs”, GDC 2000

[4] Chris­t­ian Schüler, “Nor­mal Map­ping with­out Pre­com­put­ed Tan­gents”, ShaderX 5, Chap­ter 2.6, pp. 131 – 140

[5] Col­in Bar­ré-Brise­bois and Stephen Hill, “Blend­ing in Detail”,

111 Gedanken zu „Followup: Normal Mapping Without Precomputed Tangents

  1. Out of inter­est, what impli­ca­tions does this have for tan­gent-space cal­cu­la­tions when you *bake* nor­mal-maps (for exam­ple, inside 3dsmax).
    Would­n’t you have to also use this method when cre­at­ing the orig­i­nal normal-map?

    • Hi MoP, absolute­ly, you’re cor­rect in your assumption.

      If your nor­mal map is the result of sam­pling high poly geom­e­try, then the nor­mal map sam­pler should use the same assump­tions about tan­gent space than the in the engine, if you want high­est fideli­ty. This is gen­er­al­ly true, whether you use pre­com­put­ed tan­gents or not.

      If on the oth­er hand you have a paint­ed nor­mal map, let’s say, a gener­ic rip­ple tex­ture, this just fol­lows what­ev­er tex­ture map­ping is applied to the mesh. If this map­ping is sheared or stretched, and so vio­lates the square patch assump­tion, the algo­rithm from above is still able to light it correctly.

    • Thanks for the fast reply, Christian!
      Absolute­ly, that makes sense… so I’m won­der­ing if, for “per­fect” results, if peo­ple are bak­ing their nor­mal maps in 3dsmax or Maya or XNor­mal or what­ev­er, those pro­grams will need to have their tan­gent space cal­cu­la­tions updated/reworked to match the in-game calculation?
      I’m guess­ing that cur­rent­ly the 3dsmax cal­cu­la­tion is not exact­ly synced with the method you describe here… or am I wrong?

    • You would be sur­prised to know that the method in 3dsMax or any­where else is most like­ly not synced with any engine, even when using the clas­sic method. There is an entire the­sis devot­ed to this top­ic (look for Mikkelsen here). If you want such per­fec­tion, then the engine should have its own tools for bak­ing nor­mal maps because only then it is going to know what their shaders do. 3dsMax can nev­er know this!

    • Yep, that makes sense — in fact that’s what the Doom 3 engine (id Tech 4) did to ensure the nor­mal maps were per­fect … they import­ed LWOs into the engine and baked the nor­mal-maps using the game engine so that the cal­cu­la­tions would match up perfectly.
      I haven’t seen any oth­er engine (released pub­li­cal­ly) do that yet, but I work in game devel­op­ment and I know that we have been try­ing to solve this prob­lem for years with no sat­is­fac­to­ry results. Cur­rent­ly the best method seems to be to mod­i­fy a third-par­ty pro­gram (eg. XNor­mal) to cal­cu­late tan­gent space in exact­ly the same way as the tar­get engine cal­cu­lates it, as this is the only way to ensure the nor­mal-maps will be absolute­ly correct.

  2. Pingback: Normal Mapping Without Precomputed Tangents « Interplay of Light

  3. I am con­fused to what space your TBN matrix the nor­mal from the nor­mal map tex­ture con­verts! You need this in order to do light­ing cal­cu­la­tions. Should I trans­form the light vec­tor and view vec­tor with the TBN matrix, or leave them in cam­era space?

    Do you assume in the frag­ment shad­er that g_vertexnormal, g_viewvector are in the ver­tex shad­er mul­ti­plied with the gl_NormalMatrix and gl_ModelViewMatrix respec­tive­ly? Oth­er­wise how do you account for the trans­for­ma­tions on the object.


    • Sor­ry I meant in the pre­vi­ous post “light vec­tor” and “eye vec­tor” instead of view vec­tor for the fist paragraph.

    • Hi cinepi­vates,
      the TBN matrix that I build in the pix­el shad­er is the trans­form from cotan­gent space to world space. After the vec­tor from the nor­mal map is mul­ti­plied with the TBN matrix, it is a world space nor­mal vec­tor. This is due to the fact that both the inter­po­lat­ed ver­tex nor­mal and the view vec­tor are sup­plied in world space by the ver­tex shad­er (not shown). Alter­na­tive­ly, if the ver­tex shad­er sup­plies these vec­tors in eye space, the pix­el shad­er should con­struct the TBN matrix to con­vert into eye space instead. So you can choose your way.

    • Hi
      thanks for the reply.
      Since you are mul­ti­ply­ing per pix­el with a mat3 (TBNxNor­mal) on the pix­el shad­er have you mea­sured any per­for­mance decrease com­pared to the more tra­di­tion­al method of pre­com­put­ed Tan­gents where you usu­al­ly only trans­form in the ver­tex shad­er the light,eye vec­tors into tan­gent space using the TBN matrix.

    • I don’t use the TBN in the ver­tex shad­er. That is a thing of the past when there was pix­el shad­er mod­el 1.x. It pre­vents you to use world space con­stants in the pix­el shad­er. For exam­ple, a reflec­tion vec­tor to look up a world space envi­ron­ment map. So the matrix mul­ti­ply by TBN in the pix­el shad­er has always been there for me.

  4. Hi,I just won­der­ing if light­ing cal­cu­la­tions can be done in tan­gent space.My tex­tures could not match the Square Patch Assumption,but I don’t want to lose tan­gent space lighting,since some tech­niques are par­tic­u­lar designed for that.

    • Hi April,
      this is tricky, as light­ing in tan­gent space does only work if it is orthog­o­nal. Think about it: the angle between any vec­tors (say, the angle between \mathbf{N} and \mathbf{L}) will change under a non-orthog­o­nal trans­for­ma­tion. So dot(N,L) is going to result in a dif­fer­ent val­ue, depend­ing on which space it is in. If you can live with that, then take the inverse-trans­pose of the TBN matrix (this is in essence what you get when you do the non-per­spec­tive opti­miza­tion men­tioned in the post), and use that to trans­form \mathbf{L}, \mathbf{V}, \mathbf{H} and any oth­er vec­tor you need into tan­gent space, and do the light­ing com­pu­ta­tions there.

    • Thanks for reply :)
      I man­aged to orthog­o­nal­ize the matrix and then used the inverse-​trans­pose one to do light­ing in world space, the result became a mess.I checked the orgin matrix, and found out the matrix can not be inversed where UV mir­rors. The deter­mi­nant() func­tion returns 0.
      Still look­ing for reason.

    • I’m not sure I can fol­low your argu­ment. If you want to do light­ing in world space, then you don’t need to change any­thing. If you want to do light­ing in tan­gent space instead (which only yields sim­i­lar results if the tan­gent space is rough­ly orthog­o­nal), you’d need a matrix to con­vert your light, view, etc vec­tors into tan­gent space. Edit: For this you need the inverse of the TBN. You can get the inverse-tran­pose already very sim­ply by ignor­ing the two cross prod­ucts and using dp2perp = dp1 and dp1perp = dp2. Then the trans­pose of this would be the inverse of the TBN (ignor­ing scale). In the shad­er you don’t need to explic­it­ly trans­pose, you can just mul­ti­ply with the vec­tors from the left (eg vector * matrix instead of transpose(matrix) * vector). Then you can orthog­o­nal­ize this matrix if you want, but this won’t help much if the tan­gent space is not orthog­o­nal to begin with. I don’t under­stand why you are will­ing to go though such hoops instead of sim­ply doing the light­ing in world space.

  5. Fan­tas­tic arti­cle! I switched my engine from using pre­cal­cu­lat­ed tan­gents to the method your describe — very easy to imple­ment. I’m using WebGL, but I’m pre­sum­ing my find­ings will cor­re­spond to what OpenGL + GLES pro­gram­mers see. On desk­top, I don’t real­ly notice much of a per­for­mance dif­fer­ence (if any­thing, the tan­gent-less approach is slight­ly slow­er). But on mobile (tried on both Galaxy Nexus and Nexus 7), this method rough­ly twice as slow as using pre­com­put­ed tan­gents. Seems dFdx and dFdy are par­tic­u­lar­ly slow on mobile GPUs. Just cut­ting out those calls takes my FPS from ~24FPS to ~30FPS. So although I think this method is ele­gant to the extreme, I’m not sure it’s fast enough to be the bet­ter option (at least, not on mobile). I real­ly hope some­one can con­vince me oth­er­wise though! :)

    • Hi Will, thanks for shar­ing. Of course those addi­tion­al ~14 shad­er instruc­tions for cotangent_frame() are not free, espe­cial­ly not on mobile (which is like 2005 desk­top, the time when the arti­cle was orig­i­nal­ly writ­ten). For me today, this cost is invis­i­ble com­pared to all the oth­er things that are going on, like mul­ti­ple lights and shad­ows and so forth. On the newest archi­tec­tures like the NVidia Fer­mi, it could already be a per­for­mance win to go with­out pre­com­put­ed tan­gents, due to the rea­sons men­tioned in the intro­duc­tion. But while that is nice, the main rea­son I use the method is the boost in productivity.

  6. Pingback: Martin Codes – Cool Link Stash, January 2013

  7. For me the equa­tions in this are screwed up. The are ran­dom LaTeX things in the images (Delta­math­bf). Same with Safari, Chrome and Firefox.

    Would be nice if you could fix that.

  8. Hal­lo there, I have fixed the math for­mu­lae in the post. It turned out to be an incom­pat­i­bil­i­ty between two plu­g­ins, and that got all \text{\LaTeX} back­slash­es eat­en. Sor­ry for that! Should there be a bro­ken for­mu­la that I have over­looked, just drop a line.

  9. So glad I found your blog. Though I am an artist these more tech­ni­cal insights real­ly help me in under­stand­ing what to com­mu­ni­cate to our coders to estab­lish cer­tain looks.

  10. Hi, I want to ask you to elab­o­rate on 2 fol­low­ing moments. First of all I would like to know why cross(N,Pv)/length(N)=gradient(u) and cross(Pu, N)/length(N)=gradient(v). I just can not find any math­e­mat­i­cal deriva­tion of this fact. Also it is unclear for me why deltaU = dot(T, deltaP), and how this fol­lows from def­i­n­i­tion of gra­di­ent. Oth­er­wise I found this arti­cle very excit­ing and my exper­i­ments showed that imple­men­ta­tion works very well and is a good drop-in replace­ment for con­ven­tion­al tan­gent basis.

    • Hi Mykhai­lo,
      thanks for shar­ing your expe­ri­ence. Indeed, I did not explain why \vec{N} \times \vec{P}_v / |\vec{N}|^2 = \nabla u, and that may not be obvi­ous, so here it goes: The gra­di­ent vec­tor is always per­pen­dic­u­lar to the iso-sur­face (aka. “lev­el-set”). In a skewed 2‑D coor­di­nate sys­tem, the iso-line of one coor­di­nate is sim­ply the oth­er coor­di­nate axis! So the gra­di­ent vec­tor for the u tex­ture coor­di­nate must be per­pen­dic­u­lar to the v axis, and vice ver­sa. (The scale fac­tor makes it such that the over­all length equals the rate of change, which is depen­dent on the assump­tion that |\vec{N}| = |\vec{P}_u \times \vec{P}_v|.) Your sec­ond ques­tion is also relat­ed to the fact that the gra­di­ent vec­tor is per­pen­dic­u­lar to the iso-sur­face. If a posi­tion delta is made par­al­lel to the iso-sur­face, then the tex­ture coor­di­nate does­n’t change, because in this case the dot prod­uct is zero.

    • Thank you for an answer, now every­thing is a bit clear­er. Still was able to ful­ly fig­ure it out only after I under­stood that para­metri­sa­tion of u, v is lin­ear in the plane of every tri­an­gle. For some rea­son I just missed that fact. And after you men­tioned that gra­di­ent is per­pen­dic­u­lar to iso-line every­thing made sense. And now it make sense why change of u is pro­jec­tion of posi­tion delta onto gra­di­ent vector.

  11. Hi Chris­t­ian, thanks for sharing!

    I am work­ing on an enhanced ver­sion of the Cry­tek-Spon­za scene and I had prob­lems with my per-pix­el nor­malmap­ping shad­er based on pre­com­put­ed tan­gents. I had most sur­faces light­ed cor­rect­ly, but in some cas­es they weren’t. It turned out that — unknown to me — some faces had flipped UVs (which is not that uncom­mon) and there­fore the cotan­gent frame was messed up, because the inverse of the tan­gent was used to cal­cu­late the binor­mal as the crossprod­uct with the ver­tex normal.

    Now, I replaced it with your approach and every­thing is alright; it works instant­ly and all errors van­ished. Awesome!! :-)

    • Yes, but you must pro­vide the ver­tex posi­tion in view space also. The view vec­tor is used as a proxy to dif­fer­en­ti­ate the ver­tex posi­tion, there­fore a con­stant view vec­tor will not do.

    • Ah ok, I just nor­mal­ized the view posi­tion and its work­ing perfectly


  12. Hi,
    Your idea seem to be very interesting.
    But I’ve tried it in real scene, and found a glitch — if uv (tex­ture coor­di­nates) are mir­rored, nor­mal is also become mirrored.

    Is there any way to fix it?

    • The nor­mal itself should not be flipped, only the tan­gents. If the u tex­ture coor­di­nate is mir­rored, then the \vec{T} tan­gent should reverse sign and sim­i­lar with v and the \vec{B} tangent.

  13. In the code you show in the arti­cle, you’re using ‑V as the posi­tion for tak­ing deriv­a­tives. But V is nor­mal­ized per-pix­el, so your posi­tion deriv­a­tives effec­tive­ly have their radi­al com­po­nent (toward/away from the cam­era) pro­ject­ed out. Does­n’t this cre­ate some arti­facts when the sur­face is at a glanc­ing angle to the camera?

    (You also lose scale infor­ma­tion, but since you’re nor­mal­iz­ing to make it scale-invari­ant any­way, I sup­pose this does not matter.)

    • Hi Nathan the arti­cle does­n’t men­tion it explic­it­ly but the view vec­tor is meant to be passed unnor­mal­ized from ver­tex to pix­elshad­er. The nor­mal­iza­tion should hap­pen after it has been used to com­pute the cotan­gent frame.
      (EDIT: I cor­rect­ed the nor­mal­iza­tion mis­take in the exam­ple code, now code + text are in agreement)

  14. Hel­lo Christian,

    What can be done with the tex­co­ords if we sam­ple from 2 nor­mal maps with dif­fer­ent uv’s

    We add them? or? i guess the rate of change will be the same

    Exam­ple code is:
    float3 normal1 = tex2d(normalmap1, 3*uv1);
    float3 normal2 = tex2d(normalmap2, 5*uv2);

    float3 nor­mal = nor­mal­ize (lerp(normal1, normal2, 0.5f) );

    Remarks: This oper­a­tion is not good blend­ing at all and my ques­tion is, if it was, how i will can use the uv’s?

    Best Regards

    • float2 uv_for_perturbation_function = lerp(3*uv1, 5*uv2, 0.5f)

      Like this?

    • Hi Ste­fan,
      I assume that the map­ping of uv1 and uv2 is dif­fer­ent. Then you are going to need a tan­gent frame for each, trans­form both nor­mals into a com­mon space (here: world space) and then mix them.

      float3 normalmap1 = tex2d(normalmap1, uv1);
      float3 normalmap2 = tex2d(normalmap2, uv2);
      float3x3 TBN1 = cotangent_frame( vertex_normal, -viewvector, uv1 );
      float3x3 TBN2 = cotangent_frame( vertex_normal, -viewvector, uv2 );
      float3 nor­mal = nor­mal­ize( lerp( mul( TBN1, normalmap1 ), mul( TBN2, normalmap2 ), 0.5f ) );

      hope this helps

  15. Hel­lo,

    I try to do impor­tance sam­pling of the Beck­mann Dis­tri­b­u­tion of a Cook Tor­rance BRDF. I use your descrip­tion to cre­ate a TBN-Matrix in order to trans­form the halfvec­tor from tan­gent-space to world-space.
    The prob­lem is that I can see every frag­ment (tri­an­gle) of my geom­e­try and not a glossy shad­ed sur­face. I’m not sure if the prob­lem is real­ly con­nect­ed with the TBN-Matrix. Does it work for light­ing cal­cu­la­tions with­out restric­tions? Or do you think I have to search the mis­take some­where else?

    • Hal­lo Phil,
      the par­tial deriv­a­tives of tex­ture coor­di­nates (the dFdx and dFdy instruc­tions) are con­stant over the sur­face of a tri­an­gle, so both tan­gent direc­tions, \vec{T} and \vec{B}, are going to be faceted (aka ‘flat shad­ed’). How­ev­er the nor­mal vec­tor \vec{N} is not. So unless you use are very low spec­u­lar expo­nent (high RMS slope in case of Beck­mann) it should not visu­al­ly matter.

  16. I tried this, but am hav­ing some problems.
    I’m using a left-hand­ed coor­di­nate sys­tem and I pro­vide the nor­mal and the view vec­tor in view space. It seems from one of the com­ments, that this should work.
    At first I though every is look­ing good, but than I noticed, that the per­turbed nor­mals are not quite right in some cas­es. Light­ing code should be fine, because results with unper­turbed nor­mals look as expected.
    If I use the cotan­gent frame to trans­form the view vec­tor into cotan­gent space for par­al­lax map­ping, the results look cor­rect as well.
    I’m also using D3D-style UVs where (0,0) is the top left cor­ner, but that should not mat­ter, right?
    Any ideas what could wrong?

    • Hi Ben­jamin,
      as I said in the arti­cle, the tex­ture coor­di­nate ori­gin is at the start of the image array in both OpenGL and D3D so the UV mir­ror­ing must be done in both APIs. If your coor­di­nate sys­tem is left hand­ed, you’ll need to negate both \mathbf{T} and \mathbf{B}, that’s all.

  17. I’ve tried imple­ment­ing this, but found that you get a faceted appear­ance on the world space nor­mal, pre­sum­ably due to the deriv­a­tives being per-tri­an­gle. Am I doing some­thing wrong, or is this a lim­i­ta­tion of this technique?

    • Hi James,
      as some oth­er peo­ple have com­ment­ed, the tan­gen­tial direc­tions are faceted, due to fact tha par­tial deriv­a­tives (dFdx) of tex­ture coor­di­nates are con­stant per tri­an­gle. How­ev­er for the nor­mal direc­tion the inter­po­lat­ed nor­mal is used so the faceting can only appears if there is a dif­fer­ence in the UV gra­di­ents from one tri­an­gle to the next.

  18. Could you please post your ver­tex shad­er code? I’m doing some­thing dumb wrong and I’ve been try­ing to get this to work all day.

    • Hi Rob,
      there is real­ly noth­ing to the ver­tex shad­er, just trans­forms. Below is an exam­ple (transforms[0] is the mod­el-to-world trans­form and transforms[1] is world-to-clip).

      uniform mat4 transforms[2];
      uniform vec4 camerapos;
      varying vec2 texcoord;
      varying vec3 vertexnormal;
      varying vec4 viewvector;

      void main()
      vec4 P = transforms[0] * gl_Vertex;
      gl_Position = transforms[1] * P;
      tex­co­ord = gl_MultiTexCoord0.xy;
      vertexnormal =
      ( transforms[0] * vec4( gl_Normal, 0. ) ).xyz;
      viewvec­tor = cam­er­a­pos — P;

  19. Ohhh, I was hop­ing my prob­lem might be in the ver­tex shad­er code but that’s what I have.

    The prob­lem I have is that if I use the nor­mal from perturb_normal, the light­ing rotates with the mod­el so it is always the same side of the mod­el that is lit (beau­ti­ful­ly) no mat­ter how the mod­el is ori­ent­ed rel­a­tive to the light source. If I light the mod­el with just the inter­po­lat­ed ver­tex nor­mal I get light­ing which works as expected.

    I wrote out the details of what I’m doing at I’m going to have to look at this again tomor­row to see if I can fig­ure out what it is I’m doing wrong. Thank you for your help.

  20. (This is a fol­low-up to my pre­vi­ous post, but I can’t find out how to reply to your reply.)

    I noticed you saying
    “Hi Nathan the arti­cle doesn’t men­tion it explic­itly but the view vec­tor is meant to be passed unnor­mal­ized from ver­tex to pix­elshader. The nor­mal­iza­tion should hap­pen after it has been used to com­pute the cotan­gent frame.” In one of the replies.
    In the code how­ev­er the view vec­tor is nor­mal­ized before it is passed to per­turb­Nor­mal(). Isn’t this contradictory?

    The rea­son I’m ask­ing is that I’m still strug­gling to get this work­ing for view space nor­mals. If the nor­malmap only con­tains (0, 0, 1), the final per­turbed nor­mal is unchanged and every­thing works as expect­ed, as the nor­mal effec­tive­ly is not per­turbed at all.
    Per­turb­ing by (0, 1, 0) how­ev­er gives me strange results for instance. It is espe­cial­ly notice­able at graz­ing angles. Look­ing at a plane there is a ver­ti­cal line where the nor­mal sud­den­ly flips dras­ti­cal­ly although the sur­face nor­mal did not change at all.
    I tried flip­ping B and T etc., but the “flip­ping” effect is not imme­di­ate­ly relat­ed to this.

    • Hi Ben­jamin,
      thanks for the sug­ges­tion, that’s a seri­ous gotcha, and I cor­rect­ed it.
      The code that I post­ed should pass g_viewvector into the func­tion perturb_normal.
      If you fol­low the argu­ment in the arti­cle it should be obvi­ous: perturb_normal offi­cial­ly wants the ver­tex posi­tion, but any con­stant off­set to that is going to can­cel when tak­ing the deriv­a­tive. There­fore, the view vec­tor can be passed in as a sur­ro­gate of the ver­tex posi­tion, but this is only true of the unnor­mal­ized view vector.

      Have you tried that?

      (BTW, while you can­not reply to my reply, you can reply to your orig­i­nal com­ment. The max­i­mum nest­ing lev­el here is 2)

  21. Hi,
    I’m new to shaders, but I have to imple­ment some fea­tures in a webGL project (based on this tem­plate:
    I have already imple­ment­ed a spec­u­lar map, but here are some new things and anoth­er shad­er language.

    What I actu­al­ly want to ask:
    — where comes the ‘map­Bump’ from? Is it already imple­ment­ed in GLSL (and webGL)?
    — How does ‘map = map * 255./127. — 128./127.;’ work? What means the ‘.’ after each number?

    • Nev­er­mind!
      In the end, every­thing was self-explaning.
      Also I just got it working.
      The webGL shad­er lan­guage seems to be a bit imma­ture which caused many issues, but final­ly, I got it working.

      Thanks for the great tutorial!

    • Hal­lo Basti,
      WebGL is mod­eled after OpenGL ES (and not Desk­top OpenGL), so there may be some incom­pat­i­bil­i­ties. But the shad­ing lan­guage as such should be the same GLSL. Of course, the name ‘map­Bump’ is only an exam­ple and you can use any old name. You should be able to leave out a lead­ing or trail­ing zero in float­ing point num­bers, so you can write 2. instead of 2.0 etc.

  22. Hi Chris­t­ian,
    Thank you for writ­ing this arti­cle to explain the con­cept of the cotan­gent frame, and also for your atten­tion to imple­men­ta­tion effi­cien­cy details.

    I’m still hazy on a num­ber of things though, espe­cial­ly these two:
    1.) In the Inter­mis­sion 2 sec­tion, you write:
    T = cross(N, Pv)/length(N)^2 = gradient(u)
    B = cross(Pu, N)/length(N)^2 = gradient(v)
    You seem to imply that if the tan­gent frame is orthog­o­nal, T = Pu and B = Pv, and the dif­fer­ences only occur when the frame has skew. How­ev­er, I’m con­fused about the con­ven­tions you’re using: If we assume cross prod­ucts fol­low the right-hand rule, and if we assume Pu points right and Pv points up, then T = ‑Pu and B = ‑Pv for an orthog­o­nal tan­gent frame. In oth­er words, T = Pu and B = Pv only holds for orthog­o­nal tan­gent frames if either:
    a.) We’re using the left-hand rule (thank you Direct3D for ruin­ing con­sis­tent lin­ear alge­bra con­ven­tions forever)
    b.) Pv points down (i.e. the tex­ture ori­gin is in the top-left cor­ner, rather than the bot­tom right corner)
    …but not both.
    What convention/assumption are you using for the above rela­tion­ships? Are you using the same conventions/assumptions through­out the rest of the math­e­mat­i­cal deriva­tion? Would it be more appro­pri­ate in a ped­a­gog­i­cal sense to swap the order of the cross prod­ucts, or am I miss­ing some­thing important?

    2.) I don’t have the back­ground to deeply under­stand the pre­cise dif­fer­ence between a tan­gent frame and a cotan­gent frame. Here is the extent of my back­ground knowl­edge on the subject:
    a.) Sur­face nor­mals are covectors/pseudovectors, what­ev­er the heck that means.
    b.) Covectors/pseudovectors like nor­mals trans­form dif­fer­ent­ly from ordi­nary vec­tors. Con­sid­er matrix M, col­umn vec­tor v, and nor­mal [col­umn] vec­tor n in the same coor­di­nate frame as v. If you use mul(M, v) to trans­form v into anoth­er coor­di­nate frame, you need to use mul(transpose(inverse(M)), n) to trans­form the nor­mal vec­tor. This reduces to mul(M, n) if M is orthog­o­nal, but oth­er­wise the inverse-trans­pose oper­a­tion is nec­es­sary to main­tain the nor­mal vector’s prop­er­ties in the des­ti­na­tion frame (specif­i­cal­ly, to keep the nor­mal vec­tor per­pen­dic­u­lar to the sur­face). I’ve tak­en this for grant­ed for some time, but I don’t have a deep under­stand­ing of why the inverse-trans­pose is the mag­ic solu­tion for trans­form­ing nor­mal vec­tors, and I believe this might be why I’m hav­ing trou­ble under­stand­ing the nature of the cotan­gent frame.

    Now, if I were to cre­ate a “tra­di­tion­al” TBN matrix from deriv­a­tives, I’d do it rough­ly like this (no opti­miza­tion, for clarity…and for­give me if some Cg-like con­ven­tions slip through):
    vec3 dp1 = dFdx( p );
    vec3 dp2 = dFdy( p );
    vec2 duv1 = dFdx( uv );
    vec2 duv2 = dFdy( uv );
    // set up a lin­ear sys­tem to solve for the TBN matrix:
    vec3 p_mat = mat3(dp1, dp2, N);
    vec3 uv_mat = mat3(duv1, duv2, vec3(0.0, 0.0, 1.0));
    // TBN trans­forms [reg­u­lar, non-nor­mal] vec­tors from
    // tan­gent space to world­space, so:
    // p_mat = mul(TBN, uv_mat)
    // there­fore, solve as:
    mat3 TBN = mul(p_mat, inverse(uv_mat));
    // exam­ple transforms:
    vec3 arbitrary_worldspace_vector = mul(TBN, arbitrary_tangent_space_vector);
    vec3 worldspace_normal = mul(transpose(inverse(TBN)), normal_map_val);

    First, a quick ques­tion: Due to the inverse-trans­pose oper­a­tion, this bla­tant­ly inef­fi­cient “tra­di­tion­al TBN” solu­tion should work cor­rect­ly even for non-orthog­o­nal TBN matri­ces, correct?

    Any­way, it fol­lows from the nature of the tan­gent-to-world­space trans­for­ma­tion that p_mat = mul(TBN, uv_mat) above. After all, the whole point of the world­space edges is that they’re the uv-space edges trans­formed from tan­gent-space to worldspace.

    For this rea­son, I still share some of the con­fu­sion of the poster at You men­tioned the dis­crep­an­cy has to do with your TBN matrix being a cotan­gent frame com­posed of cov­ec­tors, but I’m not yet clear on the impli­ca­tions of that.

    How­ev­er, I noticed some­thing inter­est­ing that I’m hop­ing will make every­thing fall into place: You trans­form your nor­mal vec­tor *direct­ly* from tan­gent-space to world­space using your cotan­gent frame. That is, you just do some­thing like:
    vec3 worldspace_normal = mul(cotangent_TBN, normal_map_val);

    Is the your cotan­gent frame TBN matrix sim­ply the inverse-trans­pose of the tra­di­tion­al tan­gent frame TBN matrix? If so, and cor­rect me if I’m wrong here…if you want­ed to trans­form ordi­nary vec­tors like L and V from world­space to tan­gent-space (for e.g. par­al­lax occlu­sion map­ping), you would sim­ply do:
    tangent_space_V = mul(transpose(cotangent_TBN), worldspace_V);

    If I’m under­stand­ing all this cor­rect­ly, con­struct­ing your cotan­gent frame is inher­ent­ly more effi­cient than the tra­di­tion­al TBN for­mu­la­tion for non-orthog­o­nal matri­ces: It lets us trans­form [co]tangent-space nor­mals to world­space nor­mals with a sim­ple TBN mul­ti­pli­ca­tion (instead of an inverse-trans­pose TBN mul­ti­pli­ca­tion), and it lets us trans­form ordi­nary vec­tors from world­space t [co]tangent-space with a trans­pose-TBN mul­ti­pli­ca­tion instead of an inverse-TBN multiplication.

    Do I have this cor­rect, or is the cotan­gent frame matrix some­thing entire­ly dif­fer­ent from what I am now thinking?

    • Hi Michael,
      that’s a hand full of a com­ment so i’m try­ing my best to answer.

      Good obser­va­tion! The for­mu­la which says \vec{T} = \vec{N} \times \vec{p_v} has the cross prod­uct back­wards, and I should cor­rect it in the arti­cle. Note that the code in the shad­er does the right thing, e.g. it says dp2perp = cross( dp2, N ), since my deriva­tion is based on the assump­tion that \vec{T} behaves the way I describe it. 

      So since I took the for­mu­la out of Blinns Paper, I checked it to see what he has to says about his cross prod­uct busi­ness — and in one of his draw­ings, the vec­tor \vec{N} \times \vec{p_u} is indeed in the oppo­site direc­tion as \vec{p_v}:

      Co-vec­tors are a fan­cy name for “coor­di­nate func­tions”, i.e. plane equa­tions (as in a x + b y + c z + \dots). The a, b, c, \dots behave like any old vec­tor space and so they’re called the coor­di­nates of a co-vec­tor, while the x, y, z, \dots are the coor­di­nates of an ordi­nary vector. 

      The ordi­nary vec­tor tells you “the x axis is in this direc­tion”. The co-vec­tor tells you the func­tion for the x‑coordinate.

      If you have a matrix of col­umn basis vec­tors, then the rows of the inverse matrix are the co-vec­tors, the plane equa­tions that give the coor­di­nate functions.

      If you stick with the dis­tinc­tion to treat co-vec­tors as rows, you don’t need any of this “inverse trans­pose” busi­ness — just the ordi­nary inverse will do, and you mul­ti­ply nor­mal vec­tors as rows from the left.

      If your TBN matrix was con­struct­ed as (\vec{p_u} | \vec{p_v} | \vec{N}), then yes, you would need to take the inverse (trans­pose) of that to cor­rect­ly trans­form a nor­mal vec­tor in the gen­er­al case. But the whole point of the arti­cle is to con­struct the TBN matrix as (\vec{T} | \vec{B} | \vec{N}) instead, which you can use direct­ly, since it is a co-tan­gent frame!

      The author of the post at uses the aster­isk sym­bol “*” to describe a matrix prod­uct in the first case, and a dot prod­uct in the sec­ond case. He won­ders why, seem­ing­ly, the for­mu­la to com­pute the same thing has the “prod­uct” on dif­fer­ent sides by dif­fer­ent authors. That’s his “dis­crep­an­cy”. The truth is: The for­mu­lae do not com­pute the same thing!

      Is the your cotan­gent frame TBN matrix sim­ply the inverse-​trans­pose of the tra­di­tional tan­gent frame TBN matrix?

      Yes, you can say that \vec{T} and \vec{B} are the “inverse-trans­pose” of \vec{p_u} and \vec{p_v}. The third one, \vec{N}, does not change direc­tion under the inverse-trans­pose oper­a­tion, since it is orthog­o­nal to the oth­er two. And if it is unit length, it will stay unit length.

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert

Please answer the following anti-spam test

Which thing makes "tick-tock" and if it falls down, the clock is broken?

  1.    chair
  2.    watch
  3.    ruler
  4.    table
  5.    pencil