Saturday, August 31, 2013

Chapter 1. Combiner

Problems with emulation of N64 graphics sub-system using PC hardware rendering induced by the difference between N64 and PC graphics hardware capabilities. Many things in N64 work very different from PC approach. Twelve years ago the situation was much worse: some N64 features were impossible to reproduce on PC hardware; others required complex code, which also had to take into account capabilities of user's video card and these capabilities varies greatly. Now every modern card supports shader programs, and using shaders we can program N64 most difficult to emulate features.


One of complex tasks was emulation of N64 color combiner. The color combiner equation itself is quite simple and the formula is the same for all combine modes. Nevertheless, the problem was complex. The combine equation could not be completely reproduced with standard Glide3x API as well as with standard OpenGL of that time (1998-2001). Thus, many N64 combine modes could not be reproduced correctly on PC, only an approximation was possible, with lost of color information.


OpenGL extensions introduced with GeForce cards helped to reduce the problem. Glide API also got new extensions after Voodoo4/5 release. In particular, Glide3x combiner extension was more flexible than N64 combiner and with it I was able to reproduce N64 combiners 100% correct … but not always. The main problem is the variety of color inputs, which N64 combiner can substitute to the equation. N64 can use in the same equation two standard constant color input, plus some additional parameters can be used as another constant color input, plus noise plus fractional part of per-pixel calculated LOD plus vertex color plus textures color. And besides that N64 often uses two cycle rendering when combiner consists of two equations and the second one uses output color from the first one as its input. Glide API has only one constant color (and another one can be used in texture combiner with an API extension). Two cycles mode can be reproduced if the first cycle combines textures and the second cycle blends output of the first cycle with non-texture color inputs. But crafty N64 programmers seldom used good to emulate combine modes. Most of time they used several constant color inputs and blend them with texture and vertex colors in both cycles. That made automatic translation of N64 combiners to Glide3x ones impossible. I had to use various tricks like pre-calculation of constant expressions. For each combine mode I had to think, how it can be reproduced. Thus, each combine mode in Glide64 is manually implemented. Circa ten thousands of modes, several thousands of corresponding static functions, 16665 lines of code. Titanic work. Sisyphean toil, as some modes I could not reproduce 100% correct despite of all my contrivances.


So, the first goal I wanted to achieve in the new plugin was automatic translator of combine modes. That is where my acquaintance with OpenGL Shading Language or GLSL began. With GLSL the task proved to be not just simpler, it became easy. I don't need vertex transformation, so vertex shader is trivial. Fragment shader can have as many external constant parameters as necessary, so there was no problem to pass all necessary constant color inputs to it. Combiner equation is always the same, so I just parsed its inputs and substitute them to the body of fragment shader. Initial implementation was circa 300 lines of useful code. 300 lines for universal translator, which automatically produces 100% correct combiners. Compare with 16665 lines of code in Glide64.


Initial implementation of combiner emulation put the project on the same level with Glide64: all combine modes are supported, but not all color inputs are emulated. In particular, LOD fraction and noise emulation is missing. LOD calculation is complex task by itself. Glide64 has only rough software approximation of that parameter.  Noise by itself is not complex, but Glide API does not support random color, and I did not find a good way to implement it. To my surprise, GLSL also does not have a function which generates random output, at least at the moment. From the other side, the language is powerful enough to implement such a function. Quick dig in the Internet led me to the webgl-noise project. This is an open-source project with variety of GLSL implementations of noise function. I added one to my project. Now I have a tool to implement noise input as well as randomized color and alpha dithering. My combiner implementation became more complex: 400 lines of code.

The screen shots below illustrate how it works:
Super Smash Bros. Character selection screen: noise is used for unknown characters icons.
Zelda Majora's Mask: noise in icons

Mission Impossible: randomized color dithering in menu screen


In year 2002 I started to work on Glide64 – a graphics plugin for Nuntendo 64 emulators. 10 years I was spending most of my free time on that hobby. The goal was ambitious: to make the best graphics plugin with hardware rendering. Today several top N64 emulators use Glide64-based builds as their main graphics plugin, so the goal seems to be achieved. For those 10 years I fed up with emulation development. In 2012 I stopped to work on the project and stopped to deal with emulators. Only work, family and native applications.


After one year of such simple life I found that something is missing. I've been working on the same projects using the same tools. No need to learn new stuff. When I participated in the emu project, I had to learn a lot of stuff, which did not intersect with my main work at that moment. Later situation changed and some of that “redundant” knowledge became indispensable. Permanent learning is essential. But learning “in advance” is boring, and I'm lazy. So, I had to find interesting topic to learn and motivation to work on it.


From the other side, while I was happy to forget about debugging all these strange graphics glitches and digging in old manuals, I felt regret that some my ideas left unimplemented. Some ideas were impossible to implement with Glide API, for others I just did not find time.


Thus some day in 2013 I decided to return to N64 emulation and start new graphics plugin project. Due to severe lack of spare time, project's goals are not ambitious:

·         implement my old ideas

·         support features and games, which I failed to implement with Glide64

·         make Linux port

·         make Android port

So, main focus is on new stuff. Problems which are already solved in Glide64 (e.g. hi-res textures support) have low priority. All end-user features and demands like GUI, stability, speed, compatibility also have low priority. Such minimalist and selfish approach makes development much easier.


Next topics describe milestone features implemented in the project.