Frame buffer emulation. Part III. Emulation of Video Interface

Why emulate Video Interface

What is a Video Interface (VI)?

 Let me cite N64 programming manual:

”The video interface performs the following functions.

• Creates signal timing when outputting to television monitor
• Specifies resolution and depth of color frame buffer.
• Transfers from color frame buffer to video DAC (D-A converter) and specifies filters used during transfers.
• Provides current display position information.

Note: The video DAC creates analog data from digital data and outputs said data as a television video image. “

Thus, VI transforms digital frame buffer into analogues TV signal. PC video card has its own Video Interface which does the same: transfers frame buffer from video memory to PC monitor. Complete emulation of N64 VI presents only in emulators/plugins with software rendering. HW accelerated graphics plugins usually emulate VI in very small degree. Plugins take information about frame buffer parameters from VI registers and also use UpdateScreen command as a signal that color buffers must be swapped. Some plugins emulate Gamma Correction, which N64 also performs on VI side. In most cases that's all. Plugin renders into video card's color buffer, video card's VI outputs the buffer to the monitor as is.

Which problems can be caused by incomplete VI emulation? Let's see. N64 VI may not only put whole color buffer on whole TV screen. It also can take any rectangular part of the color buffer and place it on any part of the screen; move the buffer in any direction, stretch or shrink it. And all these manipulations need no work on RDP side. Color image will be the same, but picture on your TV will move. While it's rarely used possibility, some games look incorrect without its emulation. Most known example is slide film effect in Beetle Adventure Racing menus. Here VI slides vertically two successive color buffers.

About stretching or shrinking: N64 color buffer standard resolution is usually just a quarter of maximal TV screen resolution. TV signal in NTSC standard has 483 visible scan lines. N64 VI uses 480 scan lines. Maximal resolution is 640x480 interlaced. Standard color buffer resolution is 320x240. It's VI's job to stretch it on whole TV screen. For example, buffer of that size can be displayed in non-interlaced mode to avoid screen flickering. N64 also supports high-resolution displays at 640x480 dots. Such buffer can be displayed only in interlaced mode, thus with flickering. Some games use compromise: 640x240 color buffer. It has high horizontal resolution, but can be displayed non-interlaced. From the other side, there are games which use 320x480 color buffer in hi-res mode, with only vertical resolution increased.

Such buffer sizes can be a problem for graphics plugins, which expect 4:3 screen aspect ratio. Common problem with such games was that they either displayed only on top half of the screen or half of the image is stretched on the whole screen. PAL games, which were made especially for Europe, could use 320x288 color buffer, which is also not 4:3. Common problem with emulation of such games was that they stretched vertically too much and bottom part of the image was cut. The root of such problems is that graphics plugin tries to do work of RDP and VI in one step. It has to calculate scale factors for N64 color image to stretch it to selected PC resolution. The scale factors are applied to all rendered primitives, and ready color buffer in video card must have the same width-to-height proportions as the picture on TV screen. Scale factors calculation is based on values in VI registers and is performing before start of current frame rendering. If real size of the color buffer differs from values set in VI registers, scaling can be wrong. Actually, VI has information how color image will be scaled to TV picture and careful use of that information allows correct calculation of scales for PC resolution in most cases. However, I remember various weird cases, which required special hacks in Glide64 code.

N64 uses 480 half-lines, 240 for each half-frame. 320x240 color buffer can be displayed using only one half-frame, in non-interlaced mode. This is how it is described in N64 programming manual. VI_ORIGIN register keeps start address inside the displayed color buffer. Transfer to DAC will be started from that address. It's logical to expect that the start address corresponds to the origin of the color buffer. But it's not true. Start address is usually shifted vertically by one line from the origin. That is, the first line of the color buffer is not visible on TV screen. It's not the only surprise. VI_V_START register keeps start and end scan lines, which are used for output image. Usually, (End_Scan_Line - First_Scan_Line) == 474, not 480 as said in manual. This gives us 237 half-lines in non-interlaced mode. That means that not only the first line of the color buffer is not visible on screen, but the last two lines of standard 320x240 color buffer are ignored too. As I said, graphics plugin calculation of scale factors is based on values set in VI registers. The value for width of output image usually correspond to the width of the color buffer, but that 237 makes vertical scale factor wrong, because actual height of the color buffer is 240. Wrong scaling leads to wrong output image in case of traditional rendering to main color buffer. Thus, many plugins apply special correction to VI image height to get desired 240. However, 320x240 is not fixed size for color buffer. The color buffer can be of any size, VI will scale to it to screen as necessary. Some games use color buffer a bit smaller than 320x240, and height correction for such games may produce incorrect result.

Note, that I said "usually" about actual values. While VI_ORIGIN usually point on the second line of the color buffer, it may point on any address inside the color buffer. VI usually use 474 scan lines for NTSC, but it also can use less lines. Image shrinking effect can be obtained by increasing the first scan line and decreasing the last one at the same time in VI_V_START register. Shifting in horizontal direction can be obtaining by playing with VI_H_START register. All these manipulations can be used for special effects. If they can be used they surely have been used.

VI Emulation

New frame buffer emulation mechanism open me new way for VI emulation. Without frame buffer emulation (FBE) plugin renders directly to video card’s color buffer, which displayed on screen as is. With FBE I have separated rendering of frame buffer from its displaying. I have bunch of Frame Buffer Objects (FBO) and I have to copy one of them into the video card’s color buffer to display it on screen. That is I have the same pair of entities as N64: color buffer in FBO corresponds to N64 color buffer in RDRAM, video card’s color buffer corresponds to TV screen. The task of VI emulation is to copy color buffer from FBO into main color buffer the same way as real VI transfers color buffer into video DAC. Since color buffer in FBO is usual RGBA texture, the task is reduced to render textured rectangle on screen. Just need to correctly take into account values of VI registers. Incidentally, this way of VI emulation is not my invention. Similar mechanism was implemented by Vincent ‘Ziggy’ Penne in its LLE graphics plugin Z64 years ago.

What the impact of new VI emulation on resulted image? In most cases it can be considered as zero or even negative. Since plugin now displays only that part of color image which is visible on real console, the lines ignored by N64 VI are not rendered by the plugin too. This adds narrow black areas on top and bottom of the screen, and users don’t like black areas. From the other side, plugin now emulates manipulations with scan-lines and buffer origin. An example of scan-lines manipulation is Jet Force Gemini intro movie. With old VI emulation it looks like this:

With new VI emulation:

I implemented special hack in Glide64 to make it work right. With new VI emulation I got it for free.

Most known effect based on manipulation with output image origin is slide film in Beetle Adventure Racing menus. Two color buffers are stored in RDRAM one by one. The game just movies the origin from the top to the bottom of the first buffer. Now this effect is emulated too:

The video is a bit jerky, I don't know why. The actual emulation goes pretty smooth.

VI-based vertical image shrink is used in several soccer games, e.g. Mia Hamm Soccer. Top Gear Overdrive menu uses VI-based horisontal image shrink. These games are currently not supported by my plugin, so I can’t show you screen shots or videos for these effects.