# Followup: Normal Mapping Without Precomputed Tangents

This post is a fol­low-up to my 2006 ShaderX5 arti­cle [4] about nor­mal map­ping with­out a pre-com­put­ed tan­gent basis. In the time since then I have refined this tech­nique with lessons learned in real life. For those unfa­mil­iar with the top­ic, the moti­va­tion was to con­struct the tan­gent frame on the fly in the pix­el shad­er, which iron­i­cal­ly is the exact oppo­site of the moti­va­tion from [2]:

Since it is not 1997 any­more, doing the tan­gent space on-the-fly has some poten­tial ben­e­fits, such as reduced com­plex­i­ty of asset tools, per-ver­tex band­width and stor­age, attribute inter­po­la­tors, trans­form work for skinned mesh­es and last but not least, the pos­si­bil­i­ty to apply nor­mal maps to any pro­ce­du­ral­ly gen­er­at­ed tex­ture coor­di­nates or non-lin­ear deformations.

## Intermission: Tangents vs Cotangents

The way that nor­mal map­ping is tra­di­tion­al­ly defined is, as I think, flawed, and I would like to point this out with a sim­ple C++ metaphor. Sup­pose we had a class for vec­tors, for exam­ple called Vector3, but we also had a dif­fer­ent class for cov­ec­tors, called Covector3. The lat­ter would be a clone of the ordi­nary vec­tor class, except that it behaves dif­fer­ent­ly under a trans­for­ma­tion (Edit 2018: see this arti­cle for a com­pre­hen­sive intro­duc­tion to the the­o­ry behind cov­ec­tors and dual spaces). As you may know, nor­mal vec­tors are an exam­ple of such cov­ec­tors, so we’re going to declare them as such. Now imag­ine the fol­low­ing function:

Vector3 tangent; Vector3 bitangent; Covector3 normal;   Covector3 perturb_normal( float a, float b, float c ) { return a * tangent + b * bitangent + c * normal; // ^^^^ compile-error: type mismatch for operator + }

The above func­tion mix­es vec­tors and cov­ec­tors in a sin­gle expres­sion, which in this fic­tion­al exam­ple leads to a type mis­match error. If the normal is of type Covector3, then the tangent and the bitangent should be too, oth­er­wise they can­not form a con­sis­tent frame, can they? In real life shad­er code of course, every­thing would be defined as float3 and be fine, or rather not.

### Mathematical Compile Error

Unfor­tu­nate­ly, the above mis­match is exact­ly how the ‘tan­gent frame’ for the pur­pose of nor­mal map­ping was intro­duced by the authors of [2]. This type mis­match is invis­i­ble as long as the tan­gent frame is orthog­o­nal. When the exer­cise is how­ev­er to recon­struct the tan­gent frame in the pix­el shad­er, as this arti­cle is about, then we have to deal with a non-orthog­o­nal screen pro­jec­tion. This is the rea­son why in the book I had intro­duced both (which should be called co-tan­gent) and (now it gets some­what sil­ly, it should be called co-bi-tan­gent) as cov­ec­tors, oth­er­wise the algo­rithm does not work. I have to admit that I could have been more artic­u­late about this detail. This has caused real con­fu­sion, cf from gamedev.net:

The dis­crep­an­cy is explained above, as my ‘tan­gent vec­tors’ are real­ly cov­ec­tors. The def­i­n­i­tion on page 132 is con­sis­tent with that of a cov­ec­tor, and so the frame should be called a cotan­gent frame.

## Intermission 2: Blinns Perturbed Normals (History Channel)

In this sec­tion I would like to show how the def­i­n­i­tion of and as cov­ec­tors fol­lows nat­u­ral­ly from Blinn’s orig­i­nal bump map­ping paper [1]. Blinn con­sid­ers a curved para­met­ric sur­face, for instance, a Bezi­er-patch, on which he defines tan­gent vec­tors and as the deriv­a­tives of the posi­tion with respect to and .

In this con­text it is a con­ven­tion to use sub­scripts as a short­hand for par­tial deriv­a­tives, so he is real­ly say­ing , etc. He also intro­duces the sur­face nor­mal and a bump height func­tion , which is used to dis­place the sur­face. In the end, he arrives at a for­mu­la for a first order approx­i­ma­tion of the per­turbed normal:

I would like to draw your atten­tion towards the terms and . They are the per­pen­dic­u­lars to and in the tan­gent plane, and togeth­er form a vec­tor basis for the dis­place­ments and . They are also cov­ec­tors (this is easy to ver­i­fy as they behave like cov­ec­tors under trans­for­ma­tion) so adding them to the nor­mal does not raise said type mis­match. If we divide these terms one more time by and flip their signs, we’ll arrive at the ShaderX5 def­i­n­i­tion of and as follows:

where the hat (as in ) denotes the nor­mal­ized nor­mal. can be inter­pret­ed as the nor­mal to the plane of con­stant , and like­wise as the nor­mal to the plane of con­stant . There­fore we have three nor­mal vec­tors, or cov­ec­tors, , and , and they are the a basis of a cotan­gent frame. Equiv­a­lent­ly, and are the gra­di­ents of and , which is the def­i­n­i­tion I had used in the book. The mag­ni­tude of the gra­di­ent there­fore deter­mines the bump strength, a fact that I will dis­cuss lat­er when it comes to scale invariance.

## A Little Unlearning

The mis­take of many authors is to unwit­ting­ly take and for and , which only works as long as the vec­tors are orthog­o­nal. Let’s unlearn ‘tan­gent’, relearn ‘cotan­gent’, and repeat the his­tor­i­cal devel­op­ment from this per­spec­tive: Peer­cy et al. [2] pre­com­putes the val­ues and (the change of bump height per change of tex­ture coor­di­nate) and stores them in a tex­ture. They call it ’nor­mal map’, but is a real­ly some­thing like a ’slope map’, and they have been rein­vent­ed recent­ly under the name of deriv­a­tive maps. Such a slope map can­not rep­re­sent hor­i­zon­tal nor­mals, as this would need an infi­nite slope to do so. It also needs some ‘bump scale fac­tor’ stored some­where as meta data. Kil­gard [3] intro­duces the mod­ern con­cept of a nor­mal map as an encod­ed rota­tion oper­a­tor, which does away with the approx­i­ma­tion alto­geth­er, and instead goes to define the per­turbed nor­mal direct­ly as

where the coef­fi­cients , and are read from a tex­ture. Most peo­ple would think that a nor­mal map stores nor­mals, but this is only super­fi­cial­ly true. The idea of Kil­gard was, since the unper­turbed nor­mal has coor­di­nates , it is suf­fi­cient to store the last col­umn of the rota­tion matrix that would rotate the unper­turbed nor­mal to its per­turbed posi­tion. So yes, a nor­mal map stores basis vec­tors that cor­re­spond to per­turbed nor­mals, but it real­ly is an encod­ed rota­tion oper­a­tor. The dif­fi­cul­ty starts to show up when nor­mal maps are blend­ed, since this is then an inter­po­la­tion of rota­tion oper­a­tors, with all the com­plex­i­ty that goes with it (for an excel­lent review, see the arti­cle about Reori­ent­ed Nor­mal Map­ping [5] here).

## Solution of the Cotangent Frame

The prob­lem to be solved for our pur­pose is the oppo­site as that of Blinn, the per­turbed nor­mal is known (from the nor­mal map), but the cotan­gent frame is unknown. I’ll give a short revi­sion of how I orig­i­nal­ly solved it. Define the unknown cotan­gents and as the gra­di­ents of the tex­ture coor­di­nates and as func­tions of posi­tion , such that

where is the dot prod­uct. The gra­di­ents are con­stant over the sur­face of an inter­po­lat­ed tri­an­gle, so intro­duce the edge dif­fer­ences , and . The unknown cotan­gents have to sat­is­fy the constraints

where is the cross prod­uct. The first two rows fol­low from the def­i­n­i­tion, and the last row ensures that and have no com­po­nent in the direc­tion of the nor­mal. The last row is need­ed oth­er­wise the prob­lem is under­de­ter­mined. It is straight­for­ward then to express the solu­tion in matrix form. For ,

and anal­o­gous­ly for with .

## Into the Shader Code

The above result looks daunt­ing, as it calls for a matrix inverse in every pix­el in order to com­pute the cotan­gent frame! How­ev­er, many sym­me­tries can be exploit­ed to make that almost dis­ap­pear. Below is an exam­ple of a func­tion writ­ten in GLSL to cal­cu­late the inverse of a 3×3 matrix. A sim­i­lar func­tion writ­ten in HLSL appeared in the book, and then I tried to opti­mize the hell out of it. For­get this approach as we are not going to need it at all. Just observe how the adju­gate and the deter­mi­nant can be made from cross products:

mat3 inverse3x3( mat3 M ) { // The original was written in HLSL, but this is GLSL, // therefore // - the array index selects columns, so M_t[0] is the // first row of M, etc. // - the mat3 constructor assembles columns, so // cross( M_t[1], M_t[2] ) becomes the first column // of the adjugate, etc. // - for the determinant, it does not matter whether it is // computed with M or with M_t; but using M_t makes it // easier to follow the derivation in the text mat3 M_t = transpose( M ); float det = dot( cross( M_t[0], M_t[1] ), M_t[2] ); mat3 adjugate = mat3( cross( M_t[1], M_t[2] ), cross( M_t[2], M_t[0] ), cross( M_t[0], M_t[1] ) ); return adjugate / det; }

We can sub­sti­tute the rows of the matrix from above into the code, then expand and sim­pli­fy. This pro­ce­dure results in a new expres­sion for . The deter­mi­nant becomes , and the adju­gate can be writ­ten in terms of two new expres­sions, let’s call them and (with read as ‘perp’), which becomes

As you might guessed it, and are the per­pen­dic­u­lars to the tri­an­gle edges in the tri­an­gle plane. Say Hel­lo! They are, again, cov­ec­tors and form a prop­er basis for cotan­gent space. To sim­pli­fy things fur­ther, observe:

• The last row of the matrix is irrel­e­vant since it is mul­ti­plied with zero.
• The oth­er matrix rows con­tain the per­pen­dic­u­lars ( and ), which after trans­po­si­tion just mul­ti­ply with the tex­ture edge differences.
• The per­pen­dic­u­lars can use the inter­po­lat­ed ver­tex nor­mal instead of the face nor­mal , which is sim­pler and looks even nicer.
• The deter­mi­nant (the expres­sion ) can be han­dled in a spe­cial way, which is explained below in the sec­tion about scale invariance.

Tak­en togeth­er, the opti­mized code is shown below, which is even sim­pler than the one I had orig­i­nal­ly pub­lished, and yet high­er quality:

mat3 cotangent_frame( vec3 N, vec3 p, vec2 uv ) { // get edge vectors of the pixel triangle vec3 dp1 = dFdx( p ); vec3 dp2 = dFdy( p ); vec2 duv1 = dFdx( uv ); vec2 duv2 = dFdy( uv );   // solve the linear system vec3 dp2perp = cross( dp2, N ); vec3 dp1perp = cross( N, dp1 ); vec3 T = dp2perp * duv1.x + dp1perp * duv2.x; vec3 B = dp2perp * duv1.y + dp1perp * duv2.y;   // construct a scale-invariant frame float invmax = inversesqrt( max( dot(T,T), dot(B,B) ) ); return mat3( T * invmax, B * invmax, N ); }

### Scale invariance

The deter­mi­nant was left over as a scale fac­tor in the above expres­sion. This has the con­se­quence that the result­ing cotan­gents and are not scale invari­ant, but will vary inverse­ly with the scale of the geom­e­try. It is the nat­ur­al con­se­quence of them being gra­di­ents. If the scale of the geomtery increas­es, and every­thing else is left unchanged, then the change of tex­ture coor­di­nate per unit change of posi­tion gets small­er, which reduces and sim­i­lar­ly in rela­tion to . The effect of all this is a dimin­ished per­tu­ba­tion of the nor­mal when the scale of the geom­e­try is increased, as if a height­field was stretched.

Obvi­ous­ly this behav­ior, while total­ly log­i­cal and cor­rect, would lim­it the use­ful­ness of nor­mal maps to be applied on dif­fer­ent scale geom­e­try. My solu­tion was and still is to ignore the deter­mi­nant and just nor­mal­ize and to whichev­er of them is largest, as seen in the code. This solu­tion pre­serves the rel­a­tive lengths of and , so that a skewed or stretched cotan­gent space is sill han­dled cor­rect­ly, while hav­ing an over­all scale invariance.

### Non-perspective optimization

As the ulti­mate opti­miza­tion, I also con­sid­ered what hap­pens when we can assume and . This means we have a right tri­an­gle and the per­pen­dic­u­lars fall on the tri­an­gle edges. In the pix­el shad­er, this con­di­tion is true when­ev­er the screen-pro­jec­tion of the sur­face is with­out per­spec­tive dis­tor­tion. There is a nice fig­ure demon­strat­ing this fact in [4]. This opti­miza­tion saves anoth­er two cross prod­ucts, but in my opin­ion, the qual­i­ty suf­fers heav­i­ly should there actu­al­ly be a per­spec­tive distortion.

## Putting it together

To make the post com­plete, I’ll show how the cotan­gent frame is actu­al­ly used to per­turb the inter­po­lat­ed ver­tex nor­mal. The func­tion perturb_normal does just that, using the back­wards view vec­tor for the ver­tex posi­tion (this is ok because only dif­fer­ences mat­ter, and the eye posi­tion goes away in the dif­fer­ence as it is constant).

vec3 perturb_normal( vec3 N, vec3 V, vec2 texcoord ) { // assume N, the interpolated vertex normal and // V, the view vector (vertex to eye) vec3 map = texture2D( mapBump, texcoord ).xyz; #ifdef WITH_NORMALMAP_UNSIGNED map = map * 255./127. - 128./127.; #endif #ifdef WITH_NORMALMAP_2CHANNEL map.z = sqrt( 1. - dot( map.xy, map.xy ) ); #endif #ifdef WITH_NORMALMAP_GREEN_UP map.y = -map.y; #endif mat3 TBN = cotangent_frame( N, -V, texcoord ); return normalize( TBN * map ); }
varying vec3 g_vertexnormal; varying vec3 g_viewvector; // camera pos - vertex pos varying vec2 g_texcoord;   void main() { vec3 N = normalize( g_vertexnormal );   #ifdef WITH_NORMALMAP N = perturb_normal( N, g_viewvector, g_texcoord ); #endif   // ... }

### The green axis

Both OpenGL and Direc­tX place the tex­ture coor­di­nate ori­gin at the start of the image pix­el data. The tex­ture coor­di­nate (0,0) is in the cor­ner of the pix­el where the image data point­er points to. Con­trast this to most 3‑D mod­el­ing pack­ages that place the tex­ture coor­di­nate ori­gin at the low­er left cor­ner in the uv-unwrap view. Unless the image for­mat is bot­tom-up, this means the tex­ture coor­di­nate ori­gin is in the cor­ner of the first pix­el of the last image row. Quite a difference!
An image search on Google reveals that there is no dom­i­nant con­ven­tion for the green chan­nel in nor­mal maps. Some have green point­ing up and some have green point­ing down. My artists pre­fer green point­ing up for two rea­sons: It’s the for­mat that 3ds Max expects for ren­der­ing, and it sup­pos­ed­ly looks more nat­ur­al with the ‘green illu­mi­na­tion from above’, so this helps with eye­balling nor­mal maps.

### Sign Expansion

The sign expan­sion deserves a lit­tle elab­o­ra­tion because I try to use signed tex­ture for­mats when­ev­er pos­si­ble. With the unsigned for­mat, the val­ue ½ can­not be rep­re­sent­ed exact­ly (it’s between 127 and 128). The signed for­mat does not have this prob­lem, but in exchange, has an ambigu­ous encod­ing for −1 (can be either −127 or −128). If the hard­ware is inca­pable of signed tex­ture for­mats, I want to be able to pass it as an unsigned for­mat and emu­late the exact sign expan­sion in the shad­er. This is the ori­gin of the seem­ing­ly odd val­ues in the sign expansion.

## In Hindsight

The orig­i­nal arti­cle in ShaderX5 was writ­ten as a proof-of-con­cept. Although the algo­rithm was test­ed and worked, it was a lit­tle expen­sive for that time. Fast for­ward to today and the pic­ture has changed. I am now employ­ing this algo­rithm in real-life projects for great ben­e­fit. I no longer both­er with tan­gents as ver­tex attrib­ut­es and all the asso­ci­at­ed com­plex­i­ty. For exam­ple, I don’t care whether the COLLADA exporter of Max or Maya (yes I’m rely­ing on COLLADA these days) out­put usable tan­gents for skinned mesh­es, nor do I both­er to import them, because I don’t need them! For the artists, it does­n’t occur to them that an aspect of the asset pipeline is miss­ing, because It’s all nat­ur­al: There is a geom­e­try, there are tex­ture coor­di­nates and there is a nor­mal map, and just works.

### Take Away

There are no ‘tan­gent frames’ when it comes to nor­mal map­ping. A tan­gent frame which includes the nor­mal is log­i­cal­ly ill-formed. All there is are cotan­gent frames in dis­guise when the frame is orthog­o­nal. When the frame is not orthog­o­nal, then tan­gent frames will stop work­ing. Use cotan­gent frames instead.

## References

[1] James Blinn, “Sim­u­la­tion of wrin­kled sur­faces”, SIGGRAPH 1978
http://research.microsoft.com/pubs/73939/p286-blinn.pdf

[2] Mark Peer­cy, John Airey, Bri­an Cabral, “Effi­cient Bump Map­ping Hard­ware”, SIGGRAPH 1997
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.42.4736

[3] Mark J Kil­gard, “A Prac­ti­cal and Robust Bump-map­ping Tech­nique for Today’s GPUs”, GDC 2000
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.18.537

[4] Chris­t­ian Schüler, “Nor­mal Map­ping with­out Pre­com­put­ed Tan­gents”, ShaderX 5, Chap­ter 2.6, pp. 131 – 140

[5] Col­in Bar­ré-Brise­bois and Stephen Hill, “Blend­ing in Detail”,
http://blog.selfshadow.com/publications/blending-in-detail/

## 111 Gedanken zu „Followup: Normal Mapping Without Precomputed Tangents“

1. You do this in a ter­ri­bly round­about way, it’s much eas­i­er to do

”’

vec3 denorm­Tan­gent = dFdx(texCoord.y)*dFdy(vPos)-dFdx(vPos)*dFdy(texCoord.y);
vec3 tan­gent = normalize(denormTangent-smoothNormal*dot(smoothNormal,denormTangent));

vec3 nor­mal = normalize(smoothNormal);
vec3 bitan­gent = cross(normal,tangent);
”’

• Hi devsh,
that would be equiv­a­lent to what the arti­cle describes as the “non-per­spec­tive-opti­miza­tion”. In this case you’re no longer mak­ing a dis­tinc­tion between tan­gents and co-tan­gents. It only works if the tan­gent frame is orthog­o­nal. In case of per-pix­el dif­fer­ences tak­en with dFdx etc, it would only be cor­rect if the mesh is dis­played with­out per­spec­tive dis­tor­tion, hence the name.

2. Hel­lo!
First I want to thank for the algo­rithm, this is a very nec­es­sary thing. He works on my HLSL!
But there is a bug. When approach­ing the cam­era close to the object, a strong noise of pix­el tex­ture begins. This is a known prob­lem; no one has yet solved it. Do you have any deci­sions on this, have you thought about this?

• Hi Zagol­s­ki
This behav­iour is like­ly that the dif­fer­ences between pix­els of the tex­ture coor­di­nate become too small for float­ing point pre­ci­sion and are then round­ed to zero, which leads to a divide by zero down the line.

Try to elim­i­nate any „half“ pre­ci­sion vari­able that may affect the tex­ture coor­di­nate or view vec­tor, if there are any.

3. Does this approach work with mir­rored uv’s? We’re see­ing some light­ing issues where the light­ing is incor­rect on the mir­rored parts of shapes. Maya knows to flip the nor­mals, but does­n’t know to flip the uv direc­tions. So then the TBN frame looks incorrect.

• Yes, it should work with mir­rored tex­ture coor­di­nates out of the box. Then the frame spanned by T and B sim­ply changes handedness.

4. Nice work. It was real­ly sim­ple to inte­grate in my code, the results look very good, and the expla­na­tions are just great!

I observed, how­ev­er, that the first two col­umn vec­tors of TBN — those that replace the clas­si­cal inter­po­lat­ed pre-ver­tex tan­gents and bitan­gents — are smooth over a tri­an­gle, how­ev­er, they are not smooth across tri­an­gle edges. Note that the tan­gents and bitan­gents are smooth across tri­an­gle edges.

Any thoughts on whether this is a prob­lem? To be clear: The final light­ing does­n’t seem to look wrong and the nor­mal used for light­ing do not appear to have those dis­con­ti­nu­ities, but I am still curi­ous on your expert’s opin­ion on that.

Thanks

Quirin

• Hal­lo Quirin,
as you have noticed, the pseudovec­tors T and B are faceted. This stems from the fact that all deltas (, etc) are con­stant for all pix­els of a giv­en tri­an­gle. If you browse through the ear­li­er com­ment his­to­ry then you’ll find many more dis­cus­sions about this fact. In my opin­ion this has nev­er been a prob­lem in prac­tice. As you notice, the inter­po­lat­ed nor­mal is used for N, so that is smooth.

5. Hi, as a bit of a 3D gen­er­al­ist, and rel­a­tive­ly new to shaders at that, I can appre­ci­ate both the thor­ough math analy­sis and the sheer prac­ti­cal­i­ty of in-shad­er com­pu­ta­tion from noth­ing else than nor­mals and UVs. It is a bless­ing to have found this at an ear­ly point of my jour­ney, while putting togeth­er my first scene view­er app. Thank you for shar­ing your insight the way you did.

That said, I am not writ­ing just for the sake of praise. I do have a cou­ple of ques­tions, of the “did I get this right?” kind, which may prove use­ful to oth­er peo­ple who try this code in a “toy” project.

1) In the GLSL main() exam­ple, g_vertexnormal and g_viewvector are in world space, right? (Because there would­n’t be a “cam­era pos” in view or frus­tum space.) Sup­pos­ing the pro­gram pass­es raw ver­tices and nor­mals (in mod­el space) to the shad­er, along with a bunch of matri­ces, g_vertexnormal would need to be trans­formed as a cov­ec­tor, right? (as in, N = normalize(transpose(inverse(mat3(M))) * g_vertexnormal ), M being the 4x4 mod­el matrix such that vertex_pos_world = mat3(M * vec4(vertex_pos_model,1))
And of course the adjoint mod­el matrix would only be dif­fer­ent from the plain mod­el matrix if non-uni­form scal­ing is involved (right?).

2) Despite N being smooth across tri­an­gle edges (in the­o­ry at least), I am see­ing some sub­tle light­ing dis­con­ti­nu­ities, espe­cial­ly when I add spec­u­lar to the mix. My under­stand­ing is that the code is real­ly fine, and I did­n’t mess up imple­ment­ing it either, it’s just OpenGL’s default tex­ture fil­ter­ing. If the code is imple­ment­ed from scratch (e.g., in a “toy project” such as mine), then dis­con­ti­nu­ities at edges are to be expect­ed at some view­ing angles (despite the algorithm’s accu­ra­cy), and will only dis­ap­pear once I set up prop­er mipmap­ping and anisotrop­ic fil­ter­ing. Right?

Thanks again,
Sergey

• Hal­lo Sergey,
thanks for your comments!
For 1)
Both vec­tors g_vertexnormal and g_viewvector must be in the same space, and then the cal­cu­lat­ed nor­mal is in that space too.
— Rea­son not to do it in mod­el space: pos­si­ble non-uni­form scale, don’t!
— Rea­son not to do it in view space: while pos­si­ble, need to trans­form all envi­ron­men­tal light data from world into view space -> arkward
So my rec­om­men­da­tion is world space ori­en­ta­tion, but camera-relative.
For 2)
As was writ­ten as response to many oth­er com­menters: Your code is prob­a­bly fine. The arti­facts are there because the deriv­a­tives of the tex­ture coor­di­nates are con­stant across a tri­an­gle. So the T and B vec­tors do not change over the sur­face of a tri­an­gle. There­fore, the stronger your nor­mal map, the stronger the faceted arti­facts, because T and B are only rel­e­vant for mod­i­fy­ing N accord­ing to the nor­mal map, and the direc­tion in which this mod­i­fi­ca­tion is done may change abrubtly across tri­an­gle bor­ders. Mipmap­ping will reduce the nor­mal map inten­si­ty by aver­ag­ing every­thing out.

6. Hi Chris­t­ian,

Checked on the page yes­ter­day. Glad to see my post reg­is­tered after all, and thanks for the answer!

1) Thanks for the con­fir­ma­tion. I have tried a bunch of dif­fer­ent coor­di­nates while debug­ging, and indeed cam­era-rel­a­tive world coor­di­nates make per­fect sense for your algo­rithm. (My first shad­er attempt was in view space because of some light­ing code exam­ples that were writ­ten that way, but as you say work in view space would become less and less intu­itive for com­plex lighting.

2) In your pre­vi­ous reply to Quirin (about a year ago), you made it sound like your algo­rithm nev­er leads to faceted looks in prac­tice, and that a smooth inter­po­lat­ed nor­mal N is some­how suf­fi­cient to ensure a smooth per­turbed nor­mal PN, even when T ad B are dis­con­tin­u­ous across an edge. But now you make a valid point about how a dis­con­tin­u­ous T or B will be per­turb­ing the nor­mal by dif­fer­ent amounts and in dif­fer­ent direc­tions (for the same nor­mal map pix­el) on both sides of an edge. So now I am real­ly curi­ous which it is! In the gen­er­al case, the lat­ter is prob­a­bly true, i.e., there will always be some mesh­es that have arti­facts because of dis­con­tin­u­ous T and B (sad!). But clear­ly for a sym­met­ric enough edge the dis­con­tin­u­ous con­tri­bu­tions can can­cel out, so that the per­turbed nor­mal is still smooth and facet-less. What, then, is the con­di­tion for such a nice com­pen­sa­tion to occur, and could­n’t the algo­rithm be adjust­ed so that it hap­pens more often?

It turns out that, in my case, there actu­al­ly *was* a bug: my spec­u­lar code involves a nor­mal­ized eye vec­tor, so I end­ed up pass­ing that to perturb_normal instead of a non-nor­mal­ized one. While debug­ging, I cre­at­ed sev­er­al test shapes with well-behaved UVs (antiprisms, tes­se­lat­ed cubes, etc), and it was a relief to see the facets dis­ap­pear once I fixed that nor­mal­iza­tion mis­take. (I will try stronger nor­mal maps on those test shapes, and maybe write up some shape-spe­cif­ic math, but for sym­met­ric enough edges/UVs I don’t expect dis­con­tin­u­ous T and B to cause faceted­ness per se.) I also have a lot of mesh­es (third-par­ty FBX files, most­ly) for which facets can be seen in many areas, but part of that is prob­a­bly due to bad UVs/normals/geometry. I’ll try and pin­point the sim­plest pos­si­ble mesh for which your algo­rithm has sig­nif­i­cant “faceting”, and then report back.

Cheers,
Sergey

• Hal­lo Sergey,
sor­ry for not check­ing more often. But legit com­ments will get through eventually.
As for 2) this con­di­tion was orig­i­nal­ly termed by Peer­cy et al as the “square patch assump­tion”: The UV map­ping should look like a square grid, with­out shear, and with­out non-uni­form scale. Con­se­quent­ly then, the TBN frame is orthog­o­nal, mod­u­lo a scale fac­tor. Then no vis­i­ble change occurs across tri­an­gles. While not exact­ly real­iz­able in prac­tice, “good” work on part of the artist may get close to that ide­al, and is a desire­able qual­i­ty also for oth­er reasons.
Cheers!
Christian

7. Hi, thanks for sharing.
Is it nec­es­sary to use view vec­tor as p in cotangent_frame? Would it be suf­fi­cient to use world position?

• Hal­lo Tomáš,
the view vec­tor and the world vec­tor only dif­fer by the the eye vec­tor, which is a con­stant. There­fore, either view vec­tor or world posi­tion may be used for , since their deriv­a­tives are the same.

Using the view vec­tor should behave bet­ter numer­i­cal­ly because the world posi­tion may have large mag­ni­tudes that can­cel bad­ly when the deriv­a­tive func­tion is applied.