I3D 2005 Poster; Brown Web Report Morgan McGuire, Brown (now at Williams College) Max McGuire, Iron Lore Entertainment (now at Unknown Worlds Entertainment) Web paper Images Bibtex I3D Extended Abstract (PDF) I3D Poster (PDF) Bump Maps ZIP Industry Adoption Shaders

We propose Steep Parallax Mapping, a new bump mapping scheme that can produce parallax, self-occlusion, and self-shadowing for arbitrary bump maps. It uses existing data formats, is a straightforward extension to the state-of-the-art parallax mapping, and can shade every pixel at 1024 x 768 at 30fps with 4x FSAA, making it practical for games.

Blinn and Newell's (1976) texture mapping first mapped images over surfaces to create the illusion of detail without adding geometry. Blinn's bump maps (1978), today implemented using Cook's normal map format (1984), allow flat surfaces to have not only the color but the appropriate shading of the perceived detail geometry. Texture and normal mapping are standard effects in modern games. Oliviera and Bishop's Relief Textures (1999) are a method for pre-distorting textures based on depth information. Although it has not caught on in the games community, the idea fostered the current work on parallax mapping and its improvements.

Kaneko et. al (2001) introduced parallax mapping, which for the first time allowed efficient self-occlusion and parallax effects for bump mapped surfaces. Welsh (2004) demonstrated parallax mapping on a programmable GPU architecture and added an offset limiting term to reduce texture swim. Parallax mapping is about as efficient as plain texture mapping on recent GPUs and appears more realistic than normal mapping alone. The input data structures are very efficient as well. It uses both a normal and bump map as input. Typically, the bump heights are packed into the w-components of the normal map pixels.

Figure 1: Comparision of four real-time bump-mapping methods applied to a single quadrilateral.

Parallax mapping with offset limiting is restricted to low-frequency bump maps. Steep bumps are rendered incorrectly by the algorithm and appear as parallel sheets of texture instead of bumps. Parallax mapping also has inherent swim, where the texture appears to slide over the surface due to a missing (1 / N dot V) factor, which Welsh intentionally removed the dot product to stablize the algorithm at glancing angles where the dot product is nearly zero. Nonetheless, parallax mapping with offset limiting is today considered the state of the art for real-time bump mapping.

We implemented parallax mapping with offset limiting-- including a recent forum suggestion by flipcode.org poster 'ReedBeta' of scaling the offset by N dot V to reduce swim where the gradient is large-- in the Brown University CAVE. The parallax effect is especially convincing in this environment because the user is much more free to move the viewpoint (his own head) in arbitrary directions. The CAVE is stereo, so each eye sees a different image rendered with a different view vector. This makes the the texture swim especially pronounced as the texture is shifted between eyes and the surface appears slightly out of focus.

Several methods have recently been proposed for improving parallax mapping. Donnelly (2005) describes a voxel ray tracer that uses 3D textures and a sphere tracing data structure. Tatarchuk (2004, 2005) and Policarpo (2004) describe a methods closer to our own, whic operate on a traditional packed normal/bump map. For many game applications the traditional 2D formats are superior to 3D textures-- they are supported by an existing art pipeline, the art assets already exist, 2D textures are better accellerated by current graphics cards, and 2D texture require less memory per surface area, enabling higher fidelity at the same space cost.

All of these recent ideas, including our own, are essentially the same: implement a ray tracer inside a pixel program. The crucial issues to resolve are performance and filtering to avoid undersampling (missing ray intersections), and each employs a different strategy. Our unique contributions are:

• Self-shadowing
• PS2.0 fallback in addition to a preferred PS3.0 implementation
• Use of MIP-map LOD bias for filtering
• Successful application to fur/grass rendering
• Correct depth-buffer values for bumpy surfaces
• First demonstration on a "real", difficult texture
• First implementation shown to be fast enough for immediate integration into games

The main difference between Steep Parallax Mapping and Real-Time Relief Mapping is that Policarpo and Oliveira use binary search. That gives a better appearance for smooth surfaces at slightly worse theoretical performance (the branch isn't very predictable in hardware). For thin, sharp features like hair and the raised text in the figures on our web page, binary search can miss the first intersection entirely and give incorrect results. Because this is a case we care about particularly, we instead use mip-map LOD to guarantee sampling below the Nyquist rate (MIP-maps aren't tuned for zero aliasing by default; for textures one normally wants to balance between aliasing artifacts and too-blurry). Since the texture reads are all adjacent and not dependent, they are performed very quickly from cache and the net result for our technique is high performance. The loop branch in our shader is almost perfectly predictable.

We believe that our previous technique for casting stenciled shadows from nailboards (2004) can be combined with Steep Parallax Mapping so that the only remaining cue that a surface is not actually bumpy is the silhouette. Several tricks for creating correct silhouettes for convex objects, but doing so for arbitrary shapes in an efficient manner remains an open and hard problem.

Full shaders are below and in G3D. The critical section appears on the I3D poster above; we have since extended it with Tatarchuk's idea of letting numSteps = lerp(60,10,tsE.z) and with glFragDepth = 1/(z + ray distance).

Mentions we've seen of people using our technique include:

• We've heard that Steep Parallax was used in Superman Returns pre-visualization
• S.T.A.L.K.E.R.
• Dreamscape HL2 Mod (they eventually switched to Relief Mapping)
• Oblivion IV Mod
• kbEditor
• SIGGRAPH 2006 Parallax Course
• ZootFly Xubi Engine

Here is a basic PS3.0 Steep Parallax pixel shader to help with implementing the technique. See also Philippe David's implementation.

/**
 SteepParallax.glsl.vrt
 Morgan McGuire 2005 morgan@cs.brown.edu
 */

/** Color texture (with alpha) */
uniform sampler2D texture;
uniform vec3 wsEyePos;

/** xyz = normal, w = bump height */
uniform sampler2D normalBumpMap;

/** Multiplier for bump map.  Typically on the range [0, 0.05]
  This increases with texture scale and bump height. */
uniform float bumpScale;

varying vec3 tsL;

varying vec3 _tsE;
varying vec2 texCoord;
varying vec4 tan_X, tan_Y, tan_Z, tan_W;

void main(void) {

    // We are at height bumpScale.  March forward until we hit a hair or the 
    // base surface.  Instead of dropping down discrete y-voxels we should be
    // marching in texels and dropping our y-value accordingly (TODO: fix)
    float height = 1.0;

    // Number of height divisions
    float numSteps = 5;

    /** Texture coordinate marched forward to intersection point */
    vec2 offsetCoord = texCoord.xy;
    vec4 NB = texture2D(normalBumpMap, offsetCoord);

    vec3 tsE = normalize(_tsE);

    // Increase steps at oblique angles
    // Note: tsE.z = N dot V
    numSteps = mix(numSteps*2, numSteps, tsE.z);

    // We have to negate tsE because we're walking away from the eye.
    //vec2 delta = vec2(-_tsE.x, _tsE.y) * bumpScale / (_tsE.z * numSteps);
    float step;
    vec2 delta;


    // Constant in z
    step = 1.0 / numSteps;
    delta = vec2(-_tsE.x, _tsE.y) * bumpScale / (_tsE.z * numSteps);

        // Can also step along constant in xy; the results are essentially
        // the same in each case.
        // delta = 1.0 / (25.6 * numSteps) * vec2(-tsE.x, tsE.y);
        // step = tsE.z * bumpScale * (25.6 * numSteps) / (length(tsE.xy) * 400);

    while (NB.a < height) {
        height -= step;
        offsetCoord += delta;
        NB = texture2D(normalBumpMap, offsetCoord);
    }

    height = NB.a;

    // Choose the color at the location we hit
    const vec3 color = texture2D(texture, offsetCoord).rgb;

    tsL = normalize(tsL);

    // Use the normals out of the bump map
    vec3 tsN = NB.xyz * 2 - 1;

    // Smooth normals locally along gradient to avoid making slices visible.
    // The magnitude of the step should be a function of the resolution and
    // of the bump scale and number of steps.
//    tsN = normalize(texture2D(normalBumpMap, offsetCoord + vec2(tsN.x, -tsN.y) * mipScale).xyz * 2 - 1 + tsN);

    const vec3 tsH = normalize(tsL + normalize(_tsE));

    const float NdotL = max(0, dot(tsN, tsL));
    const float NdotH = max(0, dot(tsN, tsH));
    float spec = NdotH * NdotH;

    vec3 lightColor = vec3(1.5, 1.5, 1) * 0.9;
    vec3 ambient = vec3(0.4,0.4,0.6) * 1.4;

    float selfShadow = 0;

    // Don't bother checking for self-shadowing if we're on the
    // back side of an object.
    if (NdotL > 0) {

        // Trace a shadow ray along the light vector.
        const int numShadowSteps = mix(60,5,tsL.z);
        step = 1.0 / numShadowSteps;
        delta = vec2(tsL.x, -tsL.y) * bumpScale / (numShadowSteps * tsL.z);

            // We start one iteration out to avoid shadow acne 
            // (could start bumped a little without going
            // a whole iteration).
            height = NB.a + step * 0.1;

            while ((NB.a < height) && (height < 1)) {
                height += step;
                offsetCoord += delta;
                NB = texture2D(normalBumpMap, offsetCoord);
            }

            // We are in shadow if we left the loop because
            // we hit a point
            selfShadow = (NB.a < height);

            // Shadows will make the whole scene darker, so up the light contribution
            lightColor = lightColor * 1.2;
        }
        }

        gl_FragColor.rgb = 
            color * ambient + color * NdotL * selfShadow * lightColor;

}

@article{ mcguire05parallax,
  author = "Morgan McGuire and Max McGuire",
  title = "Steep Parallax Mapping",
  journal = "I3D 2005 Poster",
  month = {April},
  day = {1},
  year = {2005}, 
  URL = {http://www.cs.brown.edu/research/graphics/games/SteepParallax/index.html}
}

@misc{ mcguire05parallaxreport,
  author = "Morgan McGuire and Max McGuire",
  title = "Steep Parallax Mapping",
  note = "Report at Brown University",
  location = "Providence, RI",
  month = {April},
  day = {1},
  year = {2005}, 
  URL = {http://www.cs.brown.edu/research/graphics/games/SteepParallax/index.html}
}