Depth buffer emulation II

I already wrote about how N64 games can use frame and depth buffers to create various special effects. Arsenal of tricks is very diverse. Due to difference in architecture of PC and N64 hardware almost any trick requires special support from graphics plugin. I invented almost all frame buffer emulation methods for Glide64 and GLideN64. As a frame buffer emulation expert, I thought that I knew about all tricks used by N64 games. However, during the work on new version I met new class of tricks with buffers. I want to explain some of these tricks while I still remember how I solved them.

Before you start reading, it can be useful to refresh in mind my previous articles about N64 frame and depth buffer emulation:

• Frame buffer emulation. Intro
• Frame buffer emulation. Part I.
• Frame buffer emulation. Part II.
• Depth buffer emulation

This article also is about depth buffer emulation; more exactly it is about

Direct rendering to depth buffer

N64 depth buffer is an area in N64 common memory, RDRAM. Depth buffer format is unsigned 16bit integer. N64 color frame buffer is also an area in RDRAM. Most common format for color buffer is 5-5-5-1 : 16bit packed integer with 15 bits for RGB and 1 bit for alpha. When Reality Display Processor (RDP) renders a polygon, it stores color of polygon's pixels in color buffer and depth values in depth buffer. This is the only "valid" method for RDP to update depth buffer. However, game can allocate a color buffer with address of the depth buffer and RDP will be able to render to that buffer as to usual 16bit color buffer. This is how N64 fills depth buffer with initial MAX_Z values before rendering starts: game allocates 16bit color buffer with address of depth buffer and calls fillrect (fill rectangle) command, which fills buffer area with color equal to MAX_Z. Then games switches color buffer pointer to address of the main color buffer and rendering starts.

Depth buffer clear is the most common case of rendering to depth buffer. Until recently I knew about only three other cases, when RDP rendered something to depth buffer directly.

1. Depth buffer as temporal color buffer. N64 memory is limited, so why not use depth buffer area when we don't need depth compare? Some games use depth buffer area as temporal color buffer when depth is not used, for example on pause screen.

   

   Mario Golf. Depth buffer is used as aux color buffer to show previous frame.

2. Depth buffer copy. 'The Legend of Zelda - Majora's Mask' uses many frame buffer tricks. One trick is related to Lens Of Truth (LoT). When Lens is activated, it reveals hidden objects. How it works? First, most of visible objects rendered as usual. When everything is ready, game copies depth buffer to another place. New 16bit color buffer allocated and depth buffer is rendered to it as texture, using BgCopy command. Then game renders fullscreen textured rectangle with Lens texture. That rectangle has minimal depth, so whole depth buffer should be filled with MIN_Z. However, Lens circle has zero alpha, so all its pixels discarded by alpha compare and depth buffer inside the Lens remained intact. Now game renders "hidden" objects. These objects discarded by depth compare outside of the Lens circle. By the way, any "nice" texture for LoT in texture packs breaks LoT functionality: LoT circle must be fully transparent. When rendering of "hidden" objects completed, game copies depth buffer from temporal buffer back to depth buffer, that is restores its state on the moment before Lens texture drawn. Now rest of geometry, which should be above the Lens, can be rendered. For example, show flakes.
   
3. Pre-rendered depth buffer. There are games, where 3D model moves over 2D background. 2D background represents 3D environment with various objects, which are just plain picture, but we see them having shape and position on the scene. Our 3D model can be visually behind these objects:
   
   How it is possible? Obviously, part of 3D model discarded by depth compare, otherwise it would look like this:
   
   2D objects have no depth. How make the depth compare work there? Zelda uses simple solution: first it renders dummy 3D objects, which corresponds to objects on picture, like this:
   
   These dummy object allow game to get valid depth buffer. When it is ready, 2D background rendered over and finally our 3D models added:
   
   There is another famous game, which uses 3D models over 2D backgrounds: Resident Evil 2. However, developers chose another way to get valid depth buffer. For each 2D color background the game has pre-rendered depth background. Each frame that pre-rendered depth background is rendered as texture to depth buffer area. Color background rendered to color buffer, then 3D models rendered over:

   

   Resident Evil 2 emulated by Glide64

Long time I thought that these are all cases of direct rendering to depth buffer. I was wrong. There are many of them.

NFL Quarterback Club 98

That game have TV monitor on menus. The monitor should display spinning logos. Logo itself is a 3D model. It is rendered to an auxiliary color buffer, which then rendered to TV monitor as texture. Nothing looks hard for hardware frame buffer emulation (HWFBE), but logo did not shown. When I worked on Glide64, I noticed that logo becomes visible when I force disable depth compare. I made a hack for this game, so it works with Glide64 and nobody noticed that it actually works a bit incorrect. For GLideN64 I decided to find why it does not work without that hack. To my surprise, this game also uses pre-rendered depth buffer texture, similar to RE2. Spinning logo is displayed on TV screen, thus no one piece of it must cross bounds of TV display. TV screen is 2D texture of non-rectangular shape, so tools like scissor can not help to cut those pixels of logo, which crosses screen bounds. The game uses 16bit depth texture, which has MAX_Z for texels corresponding the area inside the screen and MIN_Z outside of that area. Auxiliary color buffer with depth buffer address created, and that texture rendered to it. Then aux color buffer for logo selected, and logo rendered with working depth compare.
Ok, I found it, but how to reproduce that with OpenGL? First: I can't render directly to depth buffer. Second: depth texture format differs from format of GL depth buffer. Thus, I added special mode for fragment shader. When in that mode, fragment shader stores calculated color as its depth. Not all mobile versions of GL support it, but on desktop GL it works fine. Plugins renders depth texture, shader passes it to depth buffer and the logo finally works as it should.

International Superstar Soccer 64

This game has old problem with players shadows. Shadows rendered dynamically and look incorrect with most of graphics plugins. This is because the process of shadow rendering is multi-pass and quite tricky. First, the game renders shadow as set of overlapped co-planar polygons. In fact the shadow consist of several thick polylines, created by these polygons. Overlapped polygons would look ugly, but they are invisible! The only purpose to render them is to create shadow silhouette in the depth buffer. When the silhouette is ready, game renders one solid polygon, which covers all the silhouette. This polygon is co-planar with first auxiliary polygons. Special "decal" depth compare mode is used there. This mode rejects all pixels of the shadow polygon, which are out of the shadow silhouette or above other objects on the playground. Result is solid dynamic shadow. This technique does not use direct write to depth buffer, but modification of depth buffer by means of special invisible polygons is very close to the technique I used to emulate Logo in Quarterback Club 98. I should note, that "N64 depth compare" option is required to emulate "decal" depth compare mode.
When I worked on Glide64, I used hack to show shadows: I made invisible polygons visible and removed the final shadow polygon. Shadows were ugly, but much better than nothing:

Glide64

With  GLideN64 shadows finally look properly:

GLideN64

Mario Golf

Mario Golf has problem with depth compare on some levels:

When user rotates scene with the control stick, problem disappears:

For a long time I could not understand, how it should work. When camera moves, the game renders all objects. At some point, almost ready picture copied to temporal color buffer. When  rotation stops, content of that temporal buffer used as 2D background, and only non-static objects rendered above it, for example model of the golfer. Water is also not static and it is rendered over 2D background. It must be rendered with depth compare to reject pixels, which are hidden by nearby objects. But at this time these nearby objects are not 3D, they are just part of 2D picture. How the game can get depth values to compare? Analysis of log dumps helped to find that not only color buffer copied to temporal location, but depth buffer copied too. When camera not moves, that depth buffer background is copied from temporal buffer to the depth buffer before rendering starts and non-static objects correctly rendered with depth compare. This mechanism is similar to depth buffer copy for Lens Of Truth, but it uses texrect commands for buffer copy, while LoT uses BgCopy. I modified shader-based depth buffer write method invented for NFL Quarterback Club 98 to allow render to depth buffer with texrects and other drawing commands. The problem was solved.

Mario Tennis

The game has problem with VS screen. It could like like this:

or like this:

anyway, one of the players was missing. I started investigation and found, that the game renders two complete scenes, one for each player, to the same frame buffer. Second scene is rendered over the first one, and we see only one player, whose scene was rendered last. Log dump shows that game clears depth buffer before rendering the second scene, so the scene has no depth related glitches. I looked closely to the log and noticed that depth buffer clear works differently for second scene. The game clears depth buffer with MAX_Z as usual, but then it fills part of the buffer with MIN-Z using set of narrow fillrect commands. The part with MIN_Z corresponds to area from the left to the white diagonal line on the first screenshot. Thus, using depth buffer rendering, the game divided the frame on two areas. Area from the left of the white line is protected by depth compare from writing. Second scene rendered on the right and finally the white line rendered to cover junction points between scenes:

Pilot Wings

Shadows for all vehicles in Pilot Wings rendered dynamically. For a long time these shadows remained as one of the most mysterious element, impossible to emulate with hardware rendering. The game is popular, users don't want to see huge ugly black polygons, covered half of the screen.

Thus, most emulators use cheat codes to remove the shadows at all.
I spent many time trying to understand, how the shadows work. When I finally found it, it became clear why it is so hard to emulate. The mechanism is really tricky.
The game first prepares silhouette of the shadow in the depth buffer, as in International Superstar Soccer 64, but this time game renders right to depth buffer. Ok, we already met this. Surprise is that it renders to depth buffer with depth compare! Polygon, which has to be drawn to depth buffer has color and depth. If polygon's depth passes depth compare test, its color stored to depth buffer. That is depth buffer is used as color buffer and as depth buffer at the same time. When silhouette of the shadow is ready, large polygon with shadow color rendered to color buffer. Pixels out of the silhouette discarded by depth compare.
Of course, such tricky mechanism required special efforts to support on PC hardware. The task was successfully solved and the game finally looks as it should: