Tangent Space

... space article](The http://www.terathon.com/code/tangent.html) has excellent descriptions and source code. [1]: /u/img/ff5e4cc94010.svghttp://www.terathon.com/code/tangent.html) has excellent descriptions and source code. [1]: /u/img/ff5e4cc94010.svg

... space article](The http://www.terathon.com/code/tangent.html) has excellent descriptions and source code. [1]: /u/img/ff5e4cc94010.svghttp://www.terathon.com/code/tangent.html) has excellent descriptions and source code. [1]: /u/img/ff5e4cc94010.svg

Tangent space lies on the surface of geometry, allowing operations relative to the surface. Unlike other spaces, it bends and warps relative to world space. The basis vectors are the normal and two tangents at each vertex of a mesh. Like the normals, tangent space is typically interpolated over the geometry faces. Tangent space is most popular in *normal mapping*, which perturbs the geometry normals based on a normal map texture. The texture coordinates of the normal map provide the two tangent vectors implicitly, as described below. One of them is usually precomputed before rendering for efficiency. As tangent space is based on the geometry normals, normal mapping simply uses the normals from the texture in tangent space directly. They are then an additional direction applied on top of the geometry normal. Parallax mapping also uses tangent space and in particular parallax occlusion mapping, or relief mapping, involves tracing through the texture as a heightmap within tangent space. # Transforming to tangent space It may be simpler to describe how to use a tangent space mapping before how to creating it, i.e. find the tangent vectors. In the following example, we have a normal $n$ and single precomputed tangent $t$ for each vertex in a triangle mesh. We want to find a direction vector towards a light $l_T$ for a point on a triangle in tangent space. Then for example in the case of normal mapping, the dot product between the normal in the texture and $l_T$ gives a lambert diffuse scalar. Firstly, we need all input vectors in the same space. The light vector is often in world or eye space, and it can be more convenient to transform the normal and tangent vector to the same as often this is done anyway add part of rendering, rather than convert the light to object space. However, since the transformation is not necessarily orthonormal, it may be necessary to re-normalize the normal and tangent and adjust the tangent vector so that it remains orthographic/perpendicular. Also note that the tangent is transformed with the vertices by the regular transformation matrix while the normal is transformed by the inverse of the transpose to keep it perpendicular. The second tangent vector, $b$ often called the bi-normal (sometimes bi-tangent) is implicit, being orthonormal and can be found using the cross product. This can be used to make sure the original tangent vector is orthographic if it isn't already. $$ \begin{align} b &= t \times n \\ t_\perp &= n \times b \end{align} $$ Now that we have all the basis vectors, the direction vector $l$ can be found in tangent space $l_T$ as $x, y, z$ amounts of $t$, $b$ and $n$ respectively. Note that there is a strong convention of $z$ being the direction of the normal. $$ \begin{align} l_{T_x} &= t \cdot l \\ l_{T_y} &= b \cdot l \\ l_{T_z} &= n \cdot l \\ \end{align} $$ These dot products can be rewritten in matrix form. The tangent space transformation matrix aptly named TBN is built by $t$, $b$, and $n$ forming rows. $$ l_T = \mathsf{TBN} \, l $$ With $l_T$ known for each vertex, it can be interpolated to find the value at a point on the triangle. This is equivalent to interpolating TBN, although the vectors need to be normalized after interpolation. The position in tangent space is typically the 2D interpolated texture coordinate. Tangent space is then generally used for direction only, hence the $3\times3$ $\mathsf{TBN}$ matrix. A more general tangent space transformation including position is discussed later. # Finding the tangents Tangent space lies on the surface of an object. Its up direction must point away from the surface, but there is no orientation constraint from the geometry alone. Texture coordinates can provide a constraint for a tangent space relative to a mapped texture. The constraint could be chosen arbitrarily, but this is essentially attempting automatic texture mapping, which is a very challenging problem. Texture mapping is often constructed by hand to reduce seams and provide good texel density and automatic methods often produce poor results. With that said, seams and stretching of tangent space may not be so important depending on the application. ## From texture coordinates To compute tangents for each vertex in a triangle mesh, imagine a triangle of your mesh drawn over its texture, as in the image below. We need to create a transformation between object space and texture space. This happens at each triangle and is not valid as a global transformation. Rather than map the basis vectors globally, as described in the [matrices](/9/rendering/matrices/) page, a more direct tangent space derivation maps triangle edge vectors. This is described first, but for completeness we discuss the relation to global basis vectors later. ![tangent space and texture visualization][1] Given three vertices, $A$, $B$, $C$, the vectors $u=\overrightarrow{AB}$ and $v=\overrightarrow{AC}$ in object space must match the same vectors in texture space $s=\overrightarrow{AB}_T$ and $t=\overrightarrow{AC}_T$. A third dimension is added for the normal, although the normal direction in tangent space is implicitly $(0, 0, 1)$. It can be ignored but is shown here for context. Note the construction of matrices using row and column vectors. $$ \begin{bmatrix}s & t & (0, 0, 1) \end{bmatrix} = \begin{bmatrix}t \\ b \\ n \end{bmatrix} \begin{bmatrix}u & v & n \end{bmatrix} $$ Here we have six unknowns and six equations using them. I.e. the components for $t$ and $b$ are unknown but we have equations for $s$ and $t$. This can be solved in matrix form by multiplying both sides by the inverse of $\begin{bmatrix}u & v & n \end{bmatrix}$. $$ \begin{bmatrix}t \\ b \\ n \end{bmatrix} = \begin{bmatrix}s & t & (0, 0, 1) \end{bmatrix} \begin{bmatrix}u & v & n \end{bmatrix}^{-1} $$ Alternatively, the inverse of a $2\times2$ matrix is simpler and this equation can be rearranged as follows. This is possible as we assume the $\mathsf{TBN}$ matrix is orthonormal and can be inverted by taking the transpose. $n$ is ignored as it is known and the tangents are perpendicular and linearly independent. $$ \begin{align} \begin{bmatrix}t & b \end{bmatrix} &= \begin{bmatrix}u & v \end{bmatrix} \begin{bmatrix}s & t \end{bmatrix}^{-1} \\ &= \frac{1}{s_x t_y - s_y t_x} \begin{bmatrix}u & v \end{bmatrix} \begin{bmatrix} t_y & -t_x \\ -s_y & s_x \end{bmatrix} \end{align} $$ Expanding the matrix representation to vector components gives the final answer. Remember, $(u, v)$ are the triangle edge vectors in object space and $(s, t)$ are the same edges in texture space. $$ \begin{align} d &= \frac{1}{s_x t_y - s_y t_x} \\ t_x &= d (u_x t_y - v_x s_y) \\ t_y &= d (u_y t_y - v_y s_y) \\ t_z &= d (u_z t_y - v_z s_y) \\ b_x &= d (v_x s_x - u_x t_x) \\ b_y &= d (v_y s_x - u_y t_x) \\ b_z &= d (v_z s_x - u_z t_x) \\ \end{align} $$ ## Smooth tangent vectors ## Relative to basis vectors ## Position ## Without texture coordinates # External Links - [Eric Lengyel's tangent space article](The http://www.terathon.com/code/tangent.html) has excellent descriptions and source code. [1]: /u/img/ff5e4cc94010.svg

Tangent space lies on the surface of geometry, allowing operations relative to the surface. Unlike other spaces, it bends and warps relative to world space. The basis vectors are the normal and two tangents at each vertex of a mesh. Like the normals, tangent space is typically interpolated over the geometry faces. Tangent space is most popular in *normal mapping*, which perturbs the geometry normals based on a normal map texture. The texture coordinates of the normal map provide the two tangent vectors implicitly, as described below. One of them is usually precomputed before rendering for efficiency. As tangent space is based on the geometry normals, normal mapping simply uses the normals from the texture in tangent space directly. They are then an additional direction applied on top of the geometry normal. Parallax mapping also uses tangent space and in particular parallax occlusion mapping, or relief mapping, involves tracing through the texture as a heightmap within tangent space. # Transforming to tangent space It may be simpler to describe how to use a tangent space mapping before how to creating it, i.e. find the tangent vectors. In the following example, we have a normal $n$ and single precomputed tangent $t$ for each vertex in a triangle mesh. We want to find a direction vector towards a light $l_T$ for a point on a triangle in tangent space. Then for example in the case of normal mapping, the dot product between the normal in the texture and $l_T$ gives a lambert diffuse scalar. Firstly, we need all input vectors in the same space. The light vector is often in world or eye space, and it can be more convenient to transform the normal and tangent vector to the same as often this is done anyway add part of rendering, rather than convert the light to object space. However, since the transformation is not necessarily orthonormal, it may be necessary to re-normalize the normal and tangent and adjust the tangent vector so that it remains orthographic/perpendicular. Also note that the tangent is transformed with the vertices by the regular transformation matrix while the normal is transformed by the inverse of the transpose to keep it perpendicular. The second tangent vector, $b$ often called the bi-normal (sometimes bi-tangent) is implicit, being orthonormal and can be found using the cross product. This can be used to make sure the original tangent vector is orthographic if it isn't already. $$ \begin{align} b &= t \times n \\ t_\perp &= n \times b \end{align} $$ Now that we have all the basis vectors, the direction vector $l$ can be found in tangent space $l_T$ as $x, y, z$ amounts of $t$, $b$ and $n$ respectively. Note that there is a strong convention of $z$ being the direction of the normal. $$ \begin{align} l_{T_x} &= t \cdot l \\ l_{T_y} &= b \cdot l \\ l_{T_z} &= n \cdot l \\ \end{align} $$ These dot products can be rewritten in matrix form. The tangent space transformation matrix aptly named TBN is built by $t$, $b$, and $n$ forming rows. $$ l_T = \mathsf{TBN} \, l $$ With $l_T$ known for each vertex, it can be interpolated to find the value at a point on the triangle. This is equivalent to interpolating TBN, although the vectors need to be normalized after interpolation. The position in tangent space is typically the 2D interpolated texture coordinate. Tangent space is then generally used for direction only, hence the $3\times3$ $\mathsf{TBN}$ matrix. A more general tangent space transformation including position is discussed later. # Finding the tangents Tangent space lies on the surface of an object. Its up direction must point away from the surface, but there is no orientation constraint from the geometry alone. Texture coordinates can provide a constraint for a tangent space relative to a mapped texture. The constraint could be chosen arbitrarily, but this is essentially attempting automatic texture mapping, which is a very challenging problem. Texture mapping is often constructed by hand to reduce seams and provide good texel density and automatic methods often produce poor results. With that said, seams and stretching of tangent space may not be so important depending on the application. ## From texture coordinates To compute tangents for each vertex in a triangle mesh, imagine a triangle of your mesh drawn over its texture, as in the image below. We need to create a transformation between object space and texture space. This happens at each triangle and is not valid as a global transformation. Rather than map the basis vectors globally, as described in the [matrices](/9/rendering/matrices/) page, a more direct tangent space derivation maps triangle edge vectors. This is described first, but for completeness we discuss the relation to global basis vectors later. ![tangent space and texture visualization][1] Given three vertices, $A$, $B$, $C$, the vectors $u=\overrightarrow{AB}$ and $v=\overrightarrow{AC}$ in object space must match the same vectors in texture space $s=\overrightarrow{AB}_T$ and $t=\overrightarrow{AC}_T$. A third dimension is added for the normal, although the normal direction in tangent space is implicitly $(0, 0, 1)$. It can be ignored but is shown here for context. Note the construction of matrices using row and column vectors. $$ \begin{bmatrix}s & t & (0, 0, 1) \end{bmatrix} = \begin{bmatrix}t \\ b \\ n \end{bmatrix} \begin{bmatrix}u & v & n \end{bmatrix} $$ Here we have six unknowns and six equations using them. I.e. the components for $t$ and $b$ are unknown but we have equations for $s$ and $t$. This can be solved in matrix form by multiplying both sides by the inverse of $\begin{bmatrix}u & v & n \end{bmatrix}$. $$ \begin{bmatrix}t \\ b \\ n \end{bmatrix} = \begin{bmatrix}s & t & (0, 0, 1) \end{bmatrix} \begin{bmatrix}u & v & n \end{bmatrix}^{-1} $$ Alternatively, the inverse of a $2\times2$ matrix is simpler and this equation can be rearranged as follows. This is possible as we assume the $\mathsf{TBN}$ matrix is orthonormal and can be inverted by taking the transpose. $n$ is ignored as it is known and the tangents are perpendicular and linearly independent. $$ \begin{align} \begin{bmatrix}t & b \end{bmatrix} &= \begin{bmatrix}u & v \end{bmatrix} \begin{bmatrix}s & t \end{bmatrix}^{-1} \\ &= \frac{1}{s_x t_y - s_y t_x} \begin{bmatrix}u & v \end{bmatrix} \begin{bmatrix} t_y & -t_x \\ -s_y & s_x \end{bmatrix} \end{align} $$ Expanding the matrix representation to vector components gives the final answer. Remember, $(u, v)$ are the triangle edge vectors in object space and $(s, t)$ are the same edges in texture space. $$ \begin{align} d &= \frac{1}{s_x t_y - s_y t_x} \\ t_x &= d (u_x t_y - v_x s_y) \\ t_y &= d (u_y t_y - v_y s_y) \\ t_z &= d (u_z t_y - v_z s_y) \\ b_x &= d (v_x s_x - u_x t_x) \\ b_y &= d (v_y s_x - u_y t_x) \\ b_z &= d (v_z s_x - u_z t_x) \\ \end{align} $$ ## Smooth tangent vectors ## Relative to basis vectors ## Position ## Without texture coordinates # External Links - [Eric Lengyel's tangent space article](The http://www.terathon.com/code/tangent.html) has excellent descriptions and source code. [1]: /u/img/ff5e4cc94010.svg

Tangent space lies on the surface of geometry, allowing operations relative to the surface. Unlike other spaces, it bends and warps relative to world space. The basis vectors are the normal and two tangents at each vertex of a mesh. Like the normals, tangent space is typically interpolated over the geometry faces. Tangent space is most popular in *normal mapping*, which perturbs the geometry normals based on a normal map texture. The texture coordinates of the normal map provide the two tangent vectors implicitly, as described below. One of them is usually precomputed before rendering for efficiency. As tangent space is based on the geometry normals, normal mapping simply uses the normals from the texture in tangent space directly. They are then an additional direction applied on top of the geometry normal. Parallax mapping also uses tangent space and in particular parallax occlusion mapping, or relief mapping, involves tracing through the texture as a heightmap within tangent space. # Transforming to tangent space It may be simpler to describe how to use a tangent space mapping before how to creating it, i.e. find the tangent vectors. In the following example, we have a normal $n$ and single precomputed tangent $t$ for each vertex in a triangle mesh. We want to find a direction vector towards a light $l_T$ for a point on a triangle in tangent space. Then for example in the case of normal mapping, the dot product between the normal in the texture and $l_T$ gives a lambert diffuse scalar. Firstly, we need all input vectors in the same space. The light vector is often in world or eye space, and it can be more convenient to transform the normal and tangent vector to the same as often this is done anyway add part of rendering, rather than convert the light to object space. However, since the transformation is not necessarily orthonormal, it may be necessary to re-normalize the normal and tangent and adjust the tangent vector so that it remains orthographic/perpendicular. Also note that the tangent is transformed with the vertices by the regular transformation matrix while the normal is transformed by the inverse of the transpose to keep it perpendicular. The second tangent vector, $b$ often called the bi-normal (sometimes bi-tangent) is implicit, being orthonormal and can be found using the cross product. This can be used to make sure the original tangent vector is orthographic if it isn't already. $$ \begin{align} b &= t \times n \\ t_\perp &= n \times b \end{align} $$ Now that we have all the basis vectors, the direction vector $l$ can be found in tangent space $l_T$ as $x, y, z$ amounts of $t$, $b$ and $n$ respectively. Note that there is a strong convention of $z$ being the direction of the normal. $$ \begin{align} l_{T_x} &= t \cdot l \\ l_{T_y} &= b \cdot l \\ l_{T_z} &= n \cdot l \\ \end{align} $$ These dot products can be rewritten in matrix form. The tangent space transformation matrix aptly named TBN is built by $t$, $b$, and $n$ forming rows. $$l_T = \mathsf{TBN} \, l$$ With $l_T$ known for each vertex, it can be interpolated to find the value at a point on the triangle. This is equivalent to interpolating TBN, although the vectors need to be normalized after interpolation. # Finding the tangents *TODO*

Tangent space lies on the surface of geometry, allowing operations relative to the surface. Unlike other spaces, it bends and warps relative to world space. The basis vectors are the normal and two tangents at each vertex of a mesh. Like the normals, tangent space is typically interpolated over the geometry faces. Tangent space is most popular in *normal mapping*, which perturbs the geometry normals based on a normal map texture. The texture coordinates of the normal map provide the two tangent vectors implicitly, as described below. One of them is usually precomputed before rendering for efficiency. As tangent space is based on the geometry normals, normal mapping simply uses the normals from the texture in tangent space directly. They are then an additional direction applied on top of the geometry normal. Parallax mapping also uses tangent space and in particular parallax occlusion mapping, or relief mapping, involves tracing through the texture as a heightmap within tangent space. # Transforming to tangent space It may be simpler to describe how to use a tangent space mapping before how to creating it, i.e. find the tangent vectors. In the following example, we have a normal $n$ and single precomputed tangent $t$ for each vertex in a triangle mesh. We want to find a direction vector towards a light $l_T$ for a point on a triangle in tangent space. Then for example in the case of normal mapping, the dot product between the normal in the texture and $l_T$ gives a lambert diffuse scalar. Firstly, we need all input vectors in the same space. The light vector is often in world or eye space, and it can be more convenient to transform the normal and tangent vector to the same as often this is done anyway add part of rendering, rather than convert the light to object space. However, since the transformation is not necessarily orthonormal, it may be necessary to re-normalize the normal and tangent and adjust the tangent vector so that it remains orthographic/perpendicular. Also note that the tangent is transformed with the vertices by the regular transformation matrix while the normal is transformed by the inverse of the transpose to keep it perpendicular. The second tangent vector, $b$ often called the bi-normal (sometimes bi-tangent) is implicit, being orthonormal and can be found using the cross product. This can be used to make sure the original tangent vector is orthographic if it isn't already. $$ \begin{align} b &= t \times n \\ t_\perp &= n \times b \end{align} $$ Now that we have all the basis vectors, the direction vector $l$ can be found in tangent space $l_T$ as $x, y, z$ amounts of $t$, $b$ and $n$ respectively. Note that there is a strong convention of $z$ being the direction of the normal. $$ \begin{align} l_{T_x} &= t \cdot l \\ l_{T_y} &= b \cdot l \\ l_{T_z} &= n \cdot l \\ \end{align} $$ These dot products can be rewritten in matrix form. The tangent space transformation matrix aptly named TBN is built by $t$, $b$, and $n$ forming rows. $$l_T = \mathsf{TBN} \, l$$ With $l_T$ known for each vertex, it can be interpolated to find the value at a point on the triangle. This is equivalent to interpolating TBN, although the vectors need to be normalized after interpolation. # Finding the tangents *TODO*