DX(t)UT - 01 - Empty on the inside

Hi and welcome to my new and understated series! As promised quite a while ago, I will try and share some of the knowledge I have acquired in the fascinating but daunting domain of 3D engine writing. I have acquired a lot of knowledge, some useful, some that I will never use again and created a few shaders that would be a shame to waste. So I'll create a few tutorials based on DXUT to introduce some basic concepts and show how these shaders can be put into use. I have named the series DX(t)UT. See what I did there? I r wordsmith. Think of it LL3DLGLD version 2.0, with the exception that all the presented material will be fully explained, together with source snippets and even full compileable source code when needed.

Since this is the first post in the series I must stop and focus on some introductory points. First off all the tutorials will be targeting C++ with DirectX. Not the best choice. DirectX, while the best at what it does, even after multiple interface clean-ups is still quite obtuse and unpleasant to use. Before you can use DirectX you need to set up a basic window and event handling and since I am targeting low level APIs, this would end up being done directly with Windows API, which is even more unpleasant to use and understand. This is where DXUT comes in. It is a very thin wrapper over some parts of the API that takes care of initializing the main window, event handling and the nitty gritty of DirectX initialization. DXUT is an utility library and after you use it to initialize all that must be initialized you still end up using full DirectX. So you will both learn to use DirectX and generally be able to copy and paste DirectX snippets into your DXUT application without problems. We will put this to the test around lesson 5, where I'll convert some very useful DirectX samples to our DXUT application framework.

As said before, DXUT with C++ is a pretty poor choice. I think this series would be more useful if it targeted XNA 4.0. As a famous graphics programmer before me said, "that's where the shiznit is at, dawg"! This series is dual purposed. While I want to record some stuff and make it available for posterity, I will also try to teach you a few concepts and make you understand them. In consequence, a primary audience would be collage students into computing and graphics who are not taking a course on this subject. Or maybe high school students. I did a lot of graphics related programming in high school and if you are as smart as I was you'll have no problems understanding any of it. So a young and skilled but lacking experience individual, who likes games and would like to create and engine might find some very useful information here. No, you won't be available to create competitive engine. Get it out of your head. Not because the content here will be lacking (which it will), but because the number of man hours and shear effort needed to create an engine as powerful as the ones the big boys are using for commercial games is too much even for a small team to handle, yet alone one individual. But you can learn a lot of small stuff. And if this is what you want to do, I recommend XNA. It is a great platform, well documented, everybody is using it, the Internet is full of information regarding it and it can be quite pleasant to use. There is no good reason to use DirectX/DXUT directly if you want to create a game or even a high level engine capable of multiple games. But if you are interested in the low level stuff and would like to have total control of every single bit, create everything by yourself and have a playground, read on.

Boy, I sure am rusty at writing and teaching. The above paragraphs are all over the place. I promise to improve upon this in the future. So I won’t stall and dive right into it at a moment’s notice, but there is one more thing that I want to mention. I’ll start out with a few introductory lessons, but the main focus of the series is shaders! This is why I don't mind that much that I am targeting the lesser platform here. The way graphics work today, you'll end up relying on the GPU and shaders for most tasks. In a few years even more so. Even today you can do tons of stuff without invoking the CPU in a meaningful way (like fully GPU based terrains that do not create vertex buffers) and once you understand the GPU part you can easily adapt if from DXUT to XNA or some high level engine.

The shaders will be written in HLSL (duh, DirectX). I will not investigate Cg as a "cross platform" shader language and in the future I'll explain why I am not targeting OpenGL and GLSL. I will also try to make full use of shaders version 2.0, trying to push them to the limits, but some things cannot be done without version 3.0 or even 4.0. I will also be mainly targeting DirectX 9.

That's it for the introduction. Let's dive right into it!

For the first lesson I will analyze and explain one of the DirectX SDK samples: EmptyProject. This is the shortest official example that does absolutely nothing so it is well suited for explaining how DXUT works. Even so, it is relatively long and verbose for a program that does nothing except to show an empty window. But is a lot shorter that the classical DirectX/Windows API equivalent.

A huge reminder here in the middle of the post: I expect you to be comfortable with C++ and Visual Studio. In the future I'll distance myself from it, but for starting lessons this offers the path of least resistance. I won't explain step by step how to set up your environment. Try to figure it out on your own. But here are the main steps:

• Download the Windows 7 SDK free from the Microsoft site. This includes the compiler, header files and necessary libraries. This is fully command line, so you'll need and IDE too. You only need the C++ parts, but you can fully install it to avoid problems if you don't know what you are doing. 
• Download Visual Studio 2010 Express free from the Microsoft site. The Express edition is pretty good editor. It won't get you too far because the libraries that it ships with are very limited. This is where step one comes in. The Windows 7 SDK will have all the platform specific headers and libraries that the Express edition is missing. Or at least this is how the 2008 version worked. 
• Download the Direct X SDK free from the Microsoft site. 

Obviously, Microsoft is paying me. I am a corporate sell out and... DIE GNU! No, seriously, these are just a few free tools that are easy to set up and have a large user base so you can Google stuff if it is broken. I have a setup where I can use MINGW as an alternative for compilation with the press of a button and have not used Visual Studio in years. But for this series, if you followed all the steps, this will get you going a lot faster.

Now that you environment is all set up (bonus points if you already had one set up or you managed to get it right on your first try) you should have all the DirectX samples installed. For simplicity I'll consider that you installed under the default paths, so navigate to "c:\Program Files\Microsoft DirectX SDK (June 2010)\Samples\C++\Direct3D" (your path may be slightly different). Here you will find all the samples. Go to the "Bin/x86" folder. Here you will find all the samples compiled. Run a few of them to see what they do and to make sure that your GPU can handle them. Up to date DirectX and drivers are a good idea.

Do the samples work? Good. Now go back to the first path and go into the "EmptyProject" folder. Double click on "EmptyProject_2010.sln" and after Visual Studio boots up, compile and run it. If everything went without an incident, you will be greeted by this lovely window:

Lovely, just lovely! I bet you are all excited and tingly on the inside about becoming a real graphics programmer!

This application is perfect to show off how DXUT works. Let's start at the end of the program, where the "main" method is located:

Above is the code for the main method. Since Blogger is not that great at formatting and spacing text, I'll use screenshots. You can find the code in the folder I mentioned above, so you don't need to type it out. When I'll post original code I'll provide download links for the full project.

So let's analyze this "main" function. Since this is Windows API application, the entry point is somewhat different from the standard main function, but is serves the same role. The first 3 lines (#if... #endif) set up some debugging utility checks which I'll leave in for now.

The next "block" sets up all the callbacks. DXUT is callback based. You need to call a set of "DXUTSetCallback..." functions. These callbacks have very specific role and meaning and do a nice job of abstracting away the glue code so you can focus on the actual part you are interested in. DXUT supports DirectX 9 and 10 callbacks (and 11, but I won't talk about 11). You have a set of functions that set up the callbacks for DirectX 9 and equivalents for other versions. As an example, the DXUTSetCallbackD3D9DeviceAcceptable function has a DXUTSetCallbackD3D10DeviceAcceptable counterpart for DirectX 10. You should set all the callbacks for the version you are targeting. If you want to support multiple versions, you should fully specify all callbacks for all versions. DXUT will pick the highest version that is available, but there are methods that give you full control.

Let's skip this block for now and see how we create our application. The first method is DXUTInit. This does the main initialization and set up tasks so that DXUT can work and is the first thing you should call after having the callbacks set up. If you do not call it, DXUT will call it when you try create the device with default parameters. The first parameter is a boolean that tells DXUT to parse the command line. DXUT supports quite a few command line arguments. You could pass "-fullscreen" as an example to any DXUT app to force fullscreen mode. These parameters work as overrides, but only the first time. Let’s say you have your application set up to start in windowed mode. If you run it with a command line to force fullscreen, it will be run in full screen. If the user than goes into a possible setting dialog your application provides and disables fullscreen, the command line argument will not override it a second time. You can find the full list of parameters in the SDK documentation. The second parameter tells DXUT if it should display error messages for possible errors it has found. This function takes more parameters that I will not describe here.

The next function that we call is DXUTSetHotkeyHandling. This sets up a few basic keyboard shortcuts. The first parameter will enable Alt-Enter as a fullscreen toggle. The second will allow Esc to be used to exit the application. The third will allow Pause to be used to pause the frame processing loop. As you can see, these are very basic features that are at most a useful little feature that you can use while prototyping something. Later you will probably enhance/replace these small touches. This is very much in line with the design philosophy of DXUT.

Next we have DXUTSetCursorSettings, the third useful features set up function. By default the application will display a cursor when in windowed mode. Setting the first parameter to true will keep that cursor visible when you switch to fullscreen mode. If you want fullscreen windows without a cursor, use false. The second parameter clips the mouse movement to the window boundaries in fullscreen mode. To be quite honest, I don't get this flag. It does not seem that useful unless you have a multiple monitor setup. On the other hand, a toggleable mouse clip for windowed mode would be very much appreciated. Get on it, DXUT authors!

Together with the callbacks, this concluded the set up step of our application. We told it to parse a few default commands line arguments, display error messages, show cursor, handle a few shortcuts and gave the window a title. All this without using Windows API or even DirectX API!

The final step is to create the device: DXUTCreateDevice. This is the first function that actually does something "important" directly. The first parameter controls windowed mode, with true meaning that the application will not run in fullscreen mode. The second parameters control the suggested width and height of the window. DXUT might adjust these values depending on your system. In fullscreen mode the width and height should describe a resolution your adapter supports for optimal functionality. Based on what callbacks are set up and what your system supports, this function will choose a DirectX version. It will take all the options your provided (and if you did not call some of the setup functions, they should be called automatically with default parameters), gather them and after some steps it will call the callback you set with DXUTSetCallbackDeviceChanging so you can inspect and review these setting before they are applied. And then it will create everything and your window should be ready to go.

The main function has still two lines left. As customary for Windows applications and games too, they have a main loop that pools events and does game logic and rendering. In DXUT you use DXUTMainLoop to run this loop. 

And finally, you return with the value provided by DXUTGetExitCode. DXUTMainLoop will exit from the main loop when your application finalizes correctly or when an unrecoverable error has occurred. DXUTGetExitCode will return zero if the application finalized as expected and a positive value if an error is encountered. I won't list the values here, but most can be described with "an error occurred when creating/initializing...".

So, that's about it for the main function. I did not go into great detail, yet still the post is quite lengthy. And it will get even longer because I did not go into the callback functions. Still, this is a lot shorter and abstract than the equivalent plain DirectX application.

After we get used to all this, I'll try and isolate the "glue code" core and only have smaller parts that you need to change from application to application. So on to the callbacks!

First, we have IsD3D9DeviceAcceptable (set by DXUTSetCallbackD3D9DeviceAcceptable):

This callback is used to test the capabilities of your GPU in order to determine if it is capable of running your application. Here you will test hardware features, shader support and other major features. You won't test parameters that are unique to you current execution of the program, like resolution or colors. In this sample it only checks to see if the device supports alpha blending and rejects it otherwise. I won't explain that since you will probably copy & paste that everywhere. If your device does not support basic blending, you probably don't have any worth-while 3D acceleration. In future samples we'll keep this check and also verify the shader versions that are supported.

Then we have ModifyDeviceSettings (set by DXUTSetCallbackDeviceChanging):

This function is called before each device settings change. DXUTCreateDevice gathers all the settings and presents them to you before it creates the device so you can review and change them. Every time you change the settings this function will be called. If the function returns true, the DXUT will try to apply the settings. If it returns false, it will keep the current settings. So it makes sense that you return true the first time, so at least you have a window here something can be displayed. More specific applications might refuse some resolutions and other combination of settings, but generally an application should be flexible because you have no way to predict what setup the user will use to run it. In this skeleton application we simply return true and accept anything.

Then we have OnD3D9CreateDevice (set by DXUTSetCallbackD3D9DeviceCreated), which initializes the resources that our application might need for rendering. There are two kinds of resources. There are persistent resources that once created can be used until the application exits (and frees the resources before the exit). Other resources do not live such a long life. These resources can depend on the buffer size or other variables. The back buffer is such a resource, that get's reset every time the size changes. Or they can get reset because of other reasons. A texture might get freed when you Alt-Tab away from your game and run another application that requires that memory. That resource needs to be reset when you switch back to your application. OnD3D9CreateDevice is used to create only persistent resources and in our empty sample we simply return success:

The counterpart to this function is OnD3D9ResetDevice (set by DXUTSetCallbackD3D9DeviceReset), which creates resources that are no persistent. It is called "reset" because the line between having to initialize a non persistent resource and having to reset it due to some event is a blurry one. In general you'll just call a "reset" on all the appropriate resources here and this is mostly a filler function. One case where you actually need it is when the reset is caused by you changing the device settings. Each time you do that this function will be called (only if the setting were accepted by ModifyDeviceSettings ) and here you will probably do some computation based on the device size and reposition you camera and GUI. Again, we return just the OK value:

The resources that are reset here and were created in OnD3D9CreateDevice must also be released somehow. These creation functions have they counterpart that releases them. As a general rule, what you create somewhere you destroy in the counterpart function. These functions can be quite confusing if you are a beginner and since the empty project example only returns a success value if needed, I can't really explain them here. I'll try to do so in the second example. Here is the body for the two functions:

This concludes my description of the second phase of the project. Phase one was initializing everything and phase two was adding the bodies for the resource creating/freeing functions. The final phase is handling frames, rendering content and handling events. This is done again by callbacks and yet again they are very simple in this example.

The first one is OnFrameMove (set by DXUTSetCallbackFrameMove). This function is called every frame before it is rendered and here you will have the bulk of your game's logic. You will update counters, timers, run animation, collision, path finding, A.I. etc. here. As long as it is a short enough task. You don't want to have a long task because this is called in single threaded mode and no new frames will be rendered before this function exits:

Next we render the frame with OnD3D9FrameRender (set by DXUTSetCallbackD3D9FrameRender):

This is the first function we actually do something that is not initialization/book keeping/glue code. Here we clear the buffer and the Z-buffer and we call BeginScene and EndScene. Between these two calls the actual 3D rendering calls would be put. "D3DCOLOR_ARGB( 0, 45, 50, 170 )" is a macro that specifies the color in ARGB format using integers. And I'll leave as homework the presence of the apparently unused "hr" local variable. Hint: it is not a type/oversight.

You are not allowed to do logic in this function. While normally for each OnFrameMove OnD3D9FrameRender will be called only once, this is not guaranteed. If you include logic here and it changes some internal state, two frames that were supposed to be rendered identically might turn out differently.

And finally, we have MsgProc (set by DXUTSetCallbackMsgProc), used as the callback for event handling. Empty as usual:

Wow! That was long and hard! Avoiding obvious default joke. But bringing attention to me avoiding it!

Putting all this together, you will find that you have 6 KiB of code that create an empty blue window that I have spent pages describing. I tried to keep it as short as possible, but giving basic information on all the elements is a must so we can get on to real task and lessons on shaders.

If you came from a Unity/other drag and drop engine background you probably left the moment you saw the WinMain function. If you came from an OpenGL/GLUT background you probably find this long and verbose. I hope you are not patting yourself on the back and saying "LOL, FAIL M$!!!". If you come from Ogre3D then...

Anyway, this is the end of part one. In part two we'll go over another simple skeleton app that does nothing (but this one is going to be a lot more complicated) from the DirectX SDK samples. In part 3 we'll try and abstract away all the glue code so we can have a nice little OOP playground for playing around with DirectX, while adding some parts from another sample application. In part 4 we'll completely recreate the before mentioned sample application using this mini-framework. In part 5 we'll take some very useful DirectX snippets as they are from the Internet and see how easy is it to integrate them into a normal DXUT app and then how easy it is to integrate them in our test framework. By lesson 7 we'll load meshes from external files and render them with lighting and CSAA antialiasing.

And if any aspiring DirectX-er reads this, please let me know what you think!

Oh, and stay tuned for status updates on Dwarves & Holes, the engine and the future of this blog.

March of madness in review

You may have seen my video with that awesome disco scene and wondered what it was all about. Is it subtle (but surprisingly deep if looked from a certain perspective) social commentary on the state of disco as a musical genre and the entire movement after its collapse in the 80s? I also put out another video, showing a close-up of the stone stool: 

The model is the same as it has been since its creation, yet somehow it looks a lot better. You guessed it: it is lighting! But let us rewind. 

Be warned! Massive massive rant incoming. I love to write and not having done so in quite a while makes the ranty go all bendy! OK, I never claimed to be good at writing! And I am always way too lazy to spell-check!

The original plan was for March to be a month of wild experimentation, where I try out a bunch of seemingly divergent techniques to get a feel for them and based on the results of each individual experiment and the condensed conclusion on their ability to be merged into a single project, decide how to continue and what exactly to do with this game and following which style. But I encountered far more obstacles than expected, which lead me to an increasingly disproportional amount of research versus coding and experimenting. After a few days of trying unsuccessfully to implement caves I was on the net trying to find solutions and before I knew it I was neck neck in research papers. Research papers are a strange beast. They are often very obtuse,  math heavy and otherwise barely readable by mere mortals and in the end they only give you hints on how to overcome the problems that are presented. I guess you need to protect your ideas. They also come in exactly two templates, together with that blasted font that a lot of them use. What is this? A lot of them seem to be created with LaTeX . I'll never understand the fascination of academia with LaTeX. Sure, it has a deep tradition and can solve a very specific set of typesetting issues with unparalleled ease. I am a strong believer in the right tool for the right job, so when one of those typesetting issues arises I will point you in the direction of LaTeX. But if you are not in that situation and are basically writing a Word document but would like to hide that fact with fancy words and redacting it with LaTeX, I have news for you: 2012, Unicode 6.1 and printing industry wide standards called! They said to tell you to stop being a hipster. Heck, you were a hipster before hipsters existed!

Wow, way to come back from an absence of writing: making up for lack of content with ranting and alienating all LaTeX users. As said before, I like LaTeX when it is used properly. But we have a better standard that keeps getting better and better, yet in 2012 the general implementations for it are still lacking. I don't want to type \c{c} to get a ç. I'll just write a ç, it will appear as ç on the screen and it will be encoded as ç in the saved document. And yes, that character is completely different from c̦, and if they show up the same to you, then you can see why I said that support is still lacking.

But back on track. I encountered several blocking issues. One of them was caves. I worked and researched the least amount regarding caves, because this research quickly degenerated into a general terrain algorithm research. I went over CLOD (Continuous LOD), which is quick and simple, offering medium sized CPU terrain with an average amount of pop-in. I studied ROAM and ROAM2. ROAM2 seems hell-bent on solving all the issues with terrain rendering and thus raises quite a few issues that it tries to cover in its implementation, making it a tough pill to swallow at first. Geo-clipmapping seems easier to understand and has more implementation attempt available for study. Also, a distinct move away from CPU can be seen in the realm of terrain rendering. Even I was able to get a tiny patch generated using GPU, just to see if it is possible.

Now one problem that you will find when researching terrain and come to hate is seams! Seams in the terrain. Places where it is not watertight. Changing terrain LOD levels always results in seams. This is not a problem in research papers. They will acknowledge the problem, almost all will give a high level solution and quite a few will go into detail. The problem arises with third parties of similar status to myself (dudes interested in terrain that implement something based on what they have read). Almost all implementations lack any form of stitching. Another interesting thing I have discovered is the infiltration of such geeky topics in social media. With today's level of technological incursion in popular culture and a prevalence of gaming, you see young people on social media who have never coded in their life having genuine interest in game engines, rendering and often terrain. Just check out YouTube and the amount of terrain rendering going on there, and the amount of comments from random people written in typical YouTube comment style ("LOL, TEH TERNAIN ROKS WUH IS THAT, roam2?" or "my cat is called poo-poo-cacku! how hard is to do that land in unity?"). Anyway, what I'm trying to say is: most of the random  implementations lack support for any kind of stitching, it is almost always acknowledged and then hand-waived ("lol, stitching. i'll do that latter"). I find that funny for some reason and both motivating to never-ever release any terrain without stitching, even if I know for sure that the very next update which I can do in 30 minutes with one hand tied behind my back will add great stitching.

One thing that caught my eye is CDLOD by Filip Strugar. A very unique and quite elegant idea and implementation. I think that at the time of writing there are no commercial games or other serious big budget software using CDLOD for their terrain, but I tried it extensively and I think it is quite innovative and works great. The research paper is very readable but kind of short. OH MY GOD! What have these papers done to me. I am complaining that a research paper is too short! It also comes with full source code that for once works as claimed and without issues, in a normal brute force variant and a more advanced streaming variant. And a few large data sets are provided. The terrain needs to be pre-processed and written to disk in a custom format, but after that is done, the streaming client is capable of loading even gibibytes of terrain data near instantly (relatively speaking of course, based on the actual size of the data) and then life streaming it. The performance is great and you can adjust the detail and processing power needed to render the often hundreds of kilometer wide terrain both before pre-processing and on the fly while running the application. You can reach several compromises this way. I created a setup where terrain was rendered at very high detail, but far from maximum, and had over 60 FPS. Then I created a setup where terrain renders at over 180 FPS. And I created a setup where the same terrain is always rendered with between 300k-500k triangles. And I did not mention one of the biggest advantages (IMHO) of CDLOF: morphing! Traditional terrain systems suffer from pop-in. You take a step and somewhere in you view frustrum the geometry of the terrain changes. Often this is very jarring and if you would have a sound effects specialist make a sound for that effect just based on the visual ques, it will probably be some sort of "POP" or a metallic slightly reverberating "SCHLONG". All terrain implementations that feature LOD have pop-in. In the good ones you won't notice it. CDLOD is no exception, as in the geometry of the terrain changes. But it is not a pop effect. It is a morph effect. CDLOD does not use fixed LOD levels, instead it always morphs smoothly between two LOD levels. So "pop-in" is always present, at every step, not just at fixed thresholds, but it is not a pop, but a smooth morph. If you miss configured you detail parameters, sampling rate and view distance you will get very ugly LOD changing artifacts. Only they are not a pop, but a morph. If on the other hand you created a good compromise of quality/rendering speed there are almost zero visible morphs. And if you go all out and create a very high setting, I think you get a incredibly smooth engine, capable of rendering a closeup of the land from the view-point of a normal person, and then fly-up like Superman with hundreds of kilometers/hour, gaining a great overview of the land, while doing all this seamlessly.

If you can't tell, I am quite impressed and excited about CDLOD. I encourage you to download the demo and try it out for yourselves. It is fairly large download, but if you are interested in terrain rendering, I think it is well worth it and the algorithm will warm its way into peoples hearts. Just check out the elegance of this morphing sequence from a higher LOD level to a smaller LOD level, especially when compared to most approaches that simple exchange the chunk with one that uses 1/4 so many faces:

That image is from Filip Strugar's paper and all intellectual rights regarding CDLOD belong to him.

Did I loose you already? I am only getting started! The first think I researched was caves, but that became a terrain rendering research. Very interesting, but not helping me directly right now. A second major issue exists that is an obstacle in my plans to create this engine. Let me show a screenshot, again from and external source, that should give you a hint on what this problem is:

That, ladies an gentleman, is Dungeon Keeper 2. Released in 1999. I have never played a Dungeon Keeper game before. I was somewhat aware of it when it was released and I can recall reading back then an article about it in a printed magazine that specialized in games. Back then it was hard to get a hold of games and I greatly enjoyed reading those magazines, even if I would rarely if ever get my hands on specific games. Probably the only reason I remember DK is because of the very iconic red devil dude, who I think is the Dungeon Keeper. He is no Mario, but still a recognizable icon today. Well, maybe. If you are old! And so is that woman in the black leather that often appears near him. I have no idea who these people are and can't really find out, because being from 1999, DK2 has problems running on Windows 7 64 bit, even the GoG version. I can play about half a level before a crash, 1 an 1/2 if I'm lucky, in software mode and I can't save. But from what I managed to play, I can certainly see the similarities. So if I ever get accused of cloning DK2, while technically not true, I can see why that person would think so. Squad of imps digging out caves, squad of dwarves digging out caves. Sure, the similarities are just superficial and this becomes apparent from even playing just 2 levels from DK2, but you know the drill by now with appearances and the way they influence our conclusion reaching mechanism and impulsive behavior. But I was aware even back then of the great Bullfrog studio.

I have higher polygon count, better textures and even fancy bump mapping techniques, yet there is something about that screenshot that screams "I am prettier than you!". I am talking about the lighting. It is even worse if you run the game, because in software mode, those light even manage to flicker. Now 1999 was a long time ago and as said back then I had limited resources to get my hands on games so I did not get to see every game like I do today. But still, if my memory serves me well, this looked pretty good for 1999.  Not quite cutting edge technologically speaking, but still very solid lighting. Imps even cast shadows from multiple light sources (limited to 4; in software mode at least; determined empirically) and the charm of character design, color palette, overall personality of the world and the narrator/dungeon master, makes this a pleasing experience, even today.

So how did they do it? Maybe it is all trickery. Clearly, that is not vertex lighting. Unless they actually used a super high vertex count and made it intentionally look like the vertex count was low (you can count the polygons in that screenshot with ease), that looks like (not the best by today's standards) pixel lighting. The top of the map, the tiles that are not dug out are clearly vertex colored and involve no actual lighting. Or at least I can create with ease an almost identical effect by vertex colors. But the walls? The trickery argument would imply texturing tricks combined with lighting and some vertex colors. The other option is that they had an actual no trickery lighting scheme, as in if you see an area lit, then there is a light placed there and there are lighting calculations done to render the scene. The lighting model would not work that well for first person view point (judging from how it looks from top down; I read that you can poses an imp and control him first person, but I have no idea how to do this), but It is blast from top down view.

And how would such a game look in our modern ages? Maybe something like this:

Oh god, the lighting on that is crazy. Not especially pleasing aesthetically, but still good. That (if I did not upload the wrong picture) is a screenshot from Dungeons, developed by Realmforge Studios and published in 2011. Another game that I did not try or have any real contact with before this research phase, but this can be easily remedied and I hope to get my hands on it soon and do some speculative reverse engineering based on sight purely: looking at lighting at trying to think of ways to achieve a similar lighting engine. The game did not have too high reviews, but some reviewers liked it, so I may even be worth a shot just on its own merits. I think there were some crushed hopes of it being a new DK or some other form of complicated history. Anyway, I would love to have such a lighting engine.

But why is lighting so hard? First of all, it is just plain hard even if you have an engine with infinite and unimaginable lighting capabilities, because lighting comes down to aesthetics. You need to create an aesthetically pleasing lighting scheme first. This is very hard and completely dependent on your artistic aptitude. Even AAA titles sometimes look muddy. Why is that? Often a combination of factors, but more often than not it is either poor resolution and wrongly chosen textures (the composite albedo of you scene which without lighting determined the look of you game can have an equally disastrous effect if poorly chosen as very bad lighting) or just plain bad lighting. Or both. In the indie scene, on those projects that never make it out of some alpha/beta stage and never become a real and finished product (fingers-crossed that this is not fate's way of doing some fore-shadowing) you can often see competent lighting, but which is still somehow wrong.

And then there is the technical implementation of lighting, which, depending on who you ask, may or may not be a lot harder. There are a series of obstacles. Let's take standard hardware lighting. This is limited to 8 lights per object and is only per-vertex. Having a limited amount of lights per object defines the way you build you world. You need to figure out/compromise on the number of maximal moving lights (people carrying torches, projectile light, etc.), take that value into consideration and then divide the world into small enough pieces that while rendering the fixed lighting for those pieces you lave enough buffer space for the dynamic light. Let's say you divide the world into axis aligned square areas and pick a reasonably large chunk. You can't pick a very small chunk, because of the maximal number of batches constrain for the GPU. The chunk size is also dependent on how mobile and flexible you camera needs to be. So you pick a fairly large chunk size and most of the time you get only 2-4 light on a chunk, but sometimes you get 7, let's say in an area where a corridor merges with a room near a corner. Then if you have a single worker there you are fine. If you have more than one, you need to turn off either a worker light or one of the other lights. And you can't just turn off a light at random, because as workers walk the light totals will shift wildly and you will get very ugly flickering. You need a very complex light management component. And this without taking into account that dynamic entity movement can be truly freeform. In such an environment worker movement patterns may not mix well with that cell structure created by the aa boxes.

And after you coded this beast, you get only vertex lighting.

If you want pixel lighting, you need to use shaders. Here is where things become even more complicated. You need to take into account the capabilities of the GPU. You'll have a hard time coding such a scheme with pixel shader version prior to 3. With pixel shader 3 I created a lighting scheme that support any number of lights. Theoretically. But there are issues. First of all, the shader has a maximum number of lights. If you create a shader that can render up to M lights, but you parametrize it to render N, where N <= M, your performance will depend on both N an M. A shader that only currently renders 1 light but can render up to 4 will be faster than one that currently render 1 light but can render up to 8. Or so have all my practical experiments shown. So you need a bunch of shaders, probably one for 1, 2, 4, 6, 8 lights, and then probably a few more to account for a few extreme circumstances where you get a lot more lights. The lighting manager becomes simpler because it is less under strain to chunk the world in such a way that you have at most 8 lights active at once. You can target it for common scenarios and the suffer a performance penalty for uncommon ones. On the other hand, the lighting manager becomes more complicated because you need a bunch of shaders. A whole lot actually. Not only do you need a set with different maximal light counts, but a shader that can render a point light and a directional light is not identical to one that can render two point light, a point and a spot or all other combinations. If you need more light types than spot lights, the number of shaders increases dramatically.

And this without taking into account effects. Two very important effects are normal and parallax bump mapping. These depend on lighting. So if you want bump mapping, you take the total number of lighting shaders, and create a variant for each with bump mapping. And repeat for parallax. I have created a shader system that handles bump mapping for a theoretically unlimited number of lights. I still need to write one for parallax. How does on do that? First you study really in depth the lighting shader. Make sure you fully understand it. Then you study the normal mapping shader. Do this until it becomes trivial and you smirk arrogantly and say "that's it?". Normal mapping is a really easy trick and involves the same normal based calculations you do tho normally light an object. Make sure you can switch between the two with ease. Then study the multiple lighting shader, marking the differences in calculations from the single lighting one. By now, you should instinctually be able to update the multiple lights calculations to take into account normal mapping. Then study the differences between normal and parallax mapping, and repeat the process, starting from the multiple light normal mapped shader.

Easy! The hard part comes next if you want o support pixel shader 2. Small indie titles do well IMO to support a wide range of hardware. Pixel shader 2 is quite limited when compared to pixel shader 3. I managed to create a PS2 shader that support up to two lights in single pass mode and up to 5 lights in multi pass mode, as in one light per pass. I am working right now on a shader that supports up to 2 lights in one pass, up to 4 lights in two passes and up to 5 lights in 3 passes. For now I can't seem to get over 2 light per pass because I reach the limit of shader instructions and can't get over 5 lights total because I run out of constant registers to pass the lighting parameters. This while using "object local" multipass, as in a object is rendered using a single multi-pass shader. I am researching to see if it possible to do true multipass, as in you render either each object in turn or the entire scene for each pass, and each pass has its own set of parameters. Maybe this way I can get more than 5 lights. And of course, multipass gets almost linearly slower as the number of passes increases. Whatever batch count you obtained by chunking in you light manager, multiply that by the number of passes.

Then there is normal mapping, which for now I only managed to do with two lights, one pass with pixel shader 2. Parallax is in the works, and so is normal mapping multi pass, but I'm not sure if it will turn out okay.

And all this to get the first half of the universal lighting model. A set of shaders for each maximal light count (or a single one if you want to be slow, but general), doubled for normal normal, tripled for parallax mapping and multiplied by who know what if you want pixel shader 2 fall backs. Oh, and you can mix vertex with pixel lighting as a fall back method. If you pull this of and you also create a good light manager, you can get as good results as Dungeons. I hope! Is this how the big boys code lights? Seem prohibitively hard for indies.

And one final very important note: a lot of per-pixel light, even if single pass, are slow. No matter how good your chunking is, you will hit a brick wall sooner or latter depending on the fillrate of you GPU. It is easy to find this limit even with small scenes and a modest number of lights. A forward rendering scheme has the worst case scenario of number of objects x number of lights.

And then there is the second half of the universal lighting model: shadows. There is also post processing effect, but let's just stop at shadows. Shadows are an especially hard beast to tame and up to date, now in 2012, we do not have a perfect shadowing model. Shadow mapping is kind and the same principle in some form or another offers pretty much the only scalable solution. The only problem is that it produces very aliased edges. You can smooth them out, but this is not perfect. The quality also degrades a lot with the size of the scene. Anyway, shadow mapping is widely used and instantly recognizable. I think I can tell both the size of the shadow map and the scope of scene from a Skyrim screenshot. Lately, anything that I play gets inspected for all the lighting and shadowing visual cues I can gather. Cascading shadow maps and parallel split seem to greatly improve shadow quality. And  even smoothing is starting to give great results using percentage closer filtering. I am not a native English speaker (DUH!!!!!!), but does "percentage closer filtering" sound awkward to anybody else? Like they did not finish their sentence. So while shadow mapping is full of faults, there are solution out there to correct them and they keep getting better and better. But I am having a hard time wrapping my head around normal plain-and-simple shadow mapping. With normal lights. You can just forget about omnidirectional lights and those dammed cube maps and volumetric rendering.

Lightmaps are another tools that can be used in conjunction with the other techniques. Like shadow mapping, lightmapping is again hard to wrap you mind around and even with a fully GPU based implementation, it is not fast enough for real time updates to a dynamic scene. If you have a game with fixed levels created by a level designer baked lightmaps are probably your best friend. If you engine support real time blending of lightmaps with dynamic shadow, the better. Just bake you light maps and you have great scene ambiance. But my "levels" are all procedurally generated. And there are volumetric light maps, that affect dynamic geometry that passes through the lightmapped volume and renders correctly, similar to dynamic shadows. Yup, they exist. I can certainly confirm that. Those are a real thing. Yup.

And there are also volumetric lights. I just love what you can achieve with those: turn a bland scene into an almost eerily charming fairy tale scene full of warmth and hot pockets. I would love to have a partially volumetric lights based implementation. Would suit the look I am going for very well. Another thing I am studying.

I think the best results would be given by soft shadow casting lights that can be blended with a procedurally incremental pseudo-dynamic lightmap and a pixel shader based volume effect added to standardish spot lights.

Shadow mapping also has implications on you batch count. You already have a high batch count and shadow mapping renders the scene from the perspective of the light first. No my engine has been created from start to support a very high number of objects. More that other engines. I have my own method with which I am fairly satisfied that offers the possibility to render a huge amount of objects. You saw it, I kept showing it off. It does so by using a lot of GPU, a lot of CPU and a lot of normal RAM and GPU RAM. So it is fairly intense and the only things that makes it as practical as it is that my algorithm is designed to have a very good batching behavior. So if you combine my algorithm with the need of the light manager to have small objects and a high batch count, the whole thing goes to hell. I also expanded upon my algorithm and created a new super fast but super blurry version. Too blurry to use for normal objects, but for distant objects it is about 20 times faster in the worst case scenario. 3 to 6 times faster for normal scenarios. This implementation is slightly lighter on the CPU, but is is also undone by forced high batch counts.

So while I plan to keep this implementation around, being the crown jewel of my engine and all, I have also researched other ways to get a huge number of items rendered: instancing. I know now about 5 variants of instancing that give mixed results, all better than no instancing, and all except one far worse than my implementation. That one good instancing implementation is not as strong as my own method that routinely handle 90k objects. It is more suited for 65K items. But is does so with very low CPU use and considerably lower RAM requirements, so it might be a solution for the batching issues with lighting.

So as you can see, creating such and engine is extremely hard and will probably take me over a year. But take a look at this picture:

And the Disco video. A large number of lights and bump mapping. Actually, the disco video has a whole bunch of wrong normals used for mapping. The picture above has these issues corrected thanks to some further splendid effort from BrewStew.

Another closeup of how I can render stuff now, in case that the video is too blurry to see the details: 

So how was this done? With deferred rendering. Not deferred lighting. Deferred rendering offers some huge advantages. It is fairly geometry agnostic, as in the performance of lighting does not depend a lot on the complexity of the scene, so you can really go over board with huge poly count if you have great batching. Countering this is shadowing. Current shadowing schemes care a lot about geometric complexity, so even if you can light your super complex scene with ease, you won't be able to shadow it. Still, this means that you need to balance your load around shadows, not around lighting. Another advantage is that is is very fast, even with hundreds of lights. With very small lights you can even get thousands of light at once on the screens. My disco video uses 550 lights. Another advantage is that it is fairly simple. When you first read about it, the idea sounds very outlandish and has absolutely nothing to do with the fixed lighting pipeline. Zero in common. But it is still pretty easy to do.

So these are the advantage: huge number of lights, very fast, geometry agnostic. What are the disadvantages? Quite the truckload! No out of the box hardware antialiasing. Do you like your games with MSAA. Can you tell the difference and swear by the quality improvement and performance of CSAA? Well, they don't work with differed rendering. In the screenshots and videos you can see antialiasing and there are quite a few solutions for it. They don't give superb results, but are good enough. The only problem is that they are quite the performance hogs, especially since deferred rendering is really scraped for bandwidth. DirectX 10.1 gives you access in shader to each sample when using multisampling and this makes hardware MSAA viable again, but it does limit your choices of GPU and operating system. I don't think that I have a setup capable of this and anyway and I don't understand yet the code for it. Using deferred lighting you can again get MSAA working, but this implies an extra full geometry pass. Which does not play well with another disadvantage: performance and fill rate. Deferred rendering eats huge amount of resources, on paper at least. I was genuinely surprised of how fast it is, being faster with 550 light than forward rendering with 20. Still, deferred rendering is very fillrate dependant and completely changes the GPU compatibility graph. With forward rendering you have a curve, with low end working so poorly that you probably wont get interactive framerates and steadily increasing from there. With deferred rendering you get a plateau where no older GPU are capable of even creating the context for deferred rendering and the curve starts up from the plateau of zero compatibility and the first jump is a big one. My Disco demo runs with the integrated Intel GPU on my laptop with Optimus. 

Another disadvantage is that I can't handle transparency. As in alpha blending. Alpha masking works fine, as seen in the video. There are several solutions, but the prominent one, depth peeling is not really feasible yet in a general way. You need to multiply the amount of memory and fill rate requirements with the number of depth you want to "peel", making an already expensive pipeline a lot more expensive.

Another disadvantage is that is forces your lighting model on the entire scene. It is harder to create a setup where you do tricks by lighting different parts of the scene in different ways. And of course, having multiple materials is again very hard. I am currently tackling this problem of having universal specularity, making a polished metal object as shinny as simple cardboard box. Even if I manage to get specular mapping working, making truly different materials available is even harder. Like rendering an object with Phong, another with Blinn Phong and another with Lambert shading. Raise your hand! Who here does not love micro-facets?

So I am not really sure if the advantages outweigh the disadvantages. I'll keep experimenting with both forward and deferred rendering for a while.

But how did I do this? I must whole-heartily thank Catalin Zima . His blog, explanation and implementation of deferred shading for XNA 2.0 were pivotal in getting me to understand the technique. Then I must thank Roy Triesscheijn, who ported the implementation over to XNA 4.0. And last, by no mans least, Emil Persson (a.k.a. Humus) who had the second implementation of deferred rendering I managed to understand and who works under a more familiar C++ environment. Anyway, Humus's site is absolutely incredible. There are samples there, some even really old, that do very impressive things. Like that gold glow effect from 2003 written with GLSL assembly shaders. Or that interior scene with Phong lighting model and moving omnidirectional shadow-casting lights. Wow! Also let's not forget all the reading I've done beforehand, from which I will only mention a the chapter from GPU Gems dedicated to deferred rendering and a very interesting paper written by the developers of S.T.A.L.K.E.R. about their use of deferred rendering in the game with the same name. There was also a forum that escapes my mind right now.

Now, in all honesty, I really don't think it is possible for one man to create the engine and then code the game I am trying to create in any reasonable amount of time. So besides researching things that will allow me to create the engine my game deserves, but not the engine it needs, I also started researching engines in hopes that maybe I can find an engine that offers all the features I need and then I can concentrate 100% on the game. This is not an easy task because most engines seem to be centered around the pre-designed level structure while my game world is procedurally populated. Even if I do not transition over to an engine, I am so done with Irrlicht. I had my gripes with it for ages now, and researching any advanced topic and how to do it in Irrlicht is painfully depressing. You will find some ancient forum posting, with some promising results, but eventually all links stop working and none of the concerns and incomplete implementations ever get finished and production ready. Also, I have been suspecting for a while now that Irrlicht is to be blamed partially for the difficulties I had in creating such a large number of objects in engine. I whipped together a little DirectX sample that did shader based per vertex lighting and rendered 90k cubes without any frustum culling or instancing. And it runs with 2 FPS on one machine and 13 FPS with high AA and AF on another. 13 is really low, but with Irrlicht I can't get even 1/4 so many items to render at that framerate. Maybe Irrlicht is doing a lot of useful stuff I am not doing right now in my DirectX mock-up test, things that I will eventually have to do and then well get the same performance, but still, I don't consider this a good start for Irrlicht and taking into considerations all my other issues, including that while abstracting away all the low level stuff, if makes it extremely hard to implement things it was not designed for, it is time that we end out collaboration. Still, Irrlicht is not all that bad and if you don't have an overly ambitious project it may be a good place to learn the basics. And I will miss being able to switch at will between DirectX and OpenGL.

Next I investigated Ogre and I did not like it at all. It is "just" a rendering engine so you need a lot of plugins to get things that you need working. Even without any plugins, with the absolute bare minimum hello world application, it takes ages to compile. Maybe it gets better with precompiled headers (which I do not have routine access to and I don't need them), but precompiled headers are at most a band aid solution to fix the symptoms of the absolutely abyssal module support in C++. More precisely, zero module support. They are not enough and you need other tools, like making sure you do not have a horrendous include hierarchy. I precompiled that hello world Ogre example and it had over 800k lines of code. Geeee... I wonder while it takes so long to compile? Look, we are programmers. As programmers we wait a lot for stuff to compile, to load, to process. The sad truth is that there is no way around it. Even modern languages like Java or .NET based ones eventually start to compile slow, once your code base becomes large. Sure, the margin is much higher than for C++. But eventually we'll have to wait a lot for stuff to finish and there is no way around that. At the work place! Not on my personal projects. Those compile blazingly fast indifferent on what language I use. The load fast and are generally fast. A 3 second compile and link is very slow for what I am used to. So there is no way I wouldn't get annoyed as hell while working with Ogre. To make matters worse, Ogre uses a resource abstraction system where you set up folders with textures, materials, etc. and then you load the resource based on identifiers, not full disk based path. So far so good, I really want and need such a system, The problem is that in Ogre this is super slow. Granted, my hello world was only loading a single mesh (the ogre head), but the resource locator was configured to use the entire Ogre Demo resource bundle, almost 50 MiB of data. This causes a pretty large delay when the application loads, even if you only use one mesh. I tested and created some resource folders where only the needed mesh files were present and start-up was instant. You mean to tell me that startup takes so long only with 50 MiBs of data? That is not actually used? My game will use more than that. In the time it takes Ogre to index or what not those folders, I'm pretty sure I can load 3 times as many resources in anything else, even in Irrlicht. Heck, when compared to Ogre, the instant gratification provided by Irrlicht makes me remember our time together with slightly more rose colored glasses. And while the Ogre samples are more advanced that the Irrlicht samples, they still seem pretty last-gen. DirectX samples from the SDK are more advanced and interesting than that. So a massive no no for Ogre.

Speaking of DirectX, I became quite comfortable with it, more precisely with DXUT. It is one of my primary candidates for my tutorial series (not the game). What tutorials do you say? As said in a previous post, once I finish developing shaders I will do a small series on them. Since I changed my mind, and I won't be doing just some series on it. I will do full fledged tutorials with full public source code, including of course the shader code. I don't know when exactly, but soon. My other candidate for the same tutorials is XNA 4.0. While not the biggest fan of C#, I like and respect in enough for me to work comfortably with it and as a plus I have nothing but good things to say about XNA. Sure, the way you sometimes have to write custom pipeline classes can seem awkward, but generally speaking I believe that XNA is the go to platform for young coders who wish to learn the ropes of graphics programming. And I am not saying that because I am some Xbox fanboy. I have PCs and a PS3. But being able to port easily to the Xbox is a huge plus. Too bad for the licensing agreement and all the limitation on your port though, including the cost. So my tutorial will be either in DXUT or under XNA. The shaders are going to be identical anyway. Just the code that invokes them is different. XNA is easier to learn and develop for, but I find the levels of tutorial materials more than enough and XNA seems well represented on the learning front. DXUT is more powerful but a lot harder to learn and there is not enough learning material out there, so I fell like DXUT tutorials for my shaders would do more good. I was dead set on XNA, but while writing the lesson plan on complexity flow, I realized that I underestimated the amounts of glue code needed under DXUT. I have no problems with the glue code, especially since you can of repeat it unchanged from project to project, but I'm not sure I can present it in an approachable manner.

What do you think? XNA or DXUT?

As for my game, I am probably not going to port it to either XNA or DXUT, but if I ever start a new 3D project, it will 98% be a XNA one, even if this means that I need to write tighter code to compensate for the relative speed difference from C++ to .NET.

As for real finished engines that I could use for my game, after going over Ogre, DXUT and XNA, or during, I first investigated C4. C4 is pretty powerful and scalable. It has marching cubes terrain with built in LOD switching and stitching, so seemingly perfect for caves. It also features quite the bit of coding in addition to the drag & drop stuff, so building levels procedurally seems to be in. On the other hand, while the engine looks good and has extremely good bump mapping capabilities (I don't remember of the top of my had what the technique was called, but it even has some self shadowing and works superbly with brick like surfaces), I can't say that the engine looks that great. Everything created in it looks slightly muddy, and the lighting is somewhat foggy, like if there was some grain filter over the scene which there is none. If this were the only option, I would gladly accept it, because it still looks good, tons better than what I had and has the marching cubes terrain. As they say, you do not look a gift horse in the mouth. There is a licensing cost. The basic version is 100$ and you don't get sources or the ability to sell a game created with it. The standard version is 250$ and has the sources plus ability to sell. While I won't be using it on this project, I will probably buy the basic version.

Another engine I considered in Torque3D. This one I did not get the opportunity to play around in depth and don't understand its capabilities as well, but it certainly seems more advanced than C4. If I understand correctly, this one focuses more on drag & drop functionality, while having scripting tie together the capabilities of the engine. I'm not sure if this is ideal for my needs. Also, the scripting language it has uses a C++ inspired syntax. Really? Really? Out of all the well designed scripting languages out there... The information on licensing is slightly more contradictory here, so I can only say that it is between 75$ and 179$ (with some super expensive extra premium version that offers something important I think), but don't know exactly which price range offers which features.

Another engine I superficially reviewed is the NeoAxis engine. This one is based on Ogre3D, but it is in C# and you build the world in an editor, but does have quite a lot (too much for C# and Ogre?) coding behind the curtains. It scales well (except for shadows, which either work as expected on adequately powerful machines, but pretty much disappear on low hardware). It has a start-up time, but this time it makes sense with the amount of resources the demos actually load. The indie version is 95$ but has no source code. The commercial version (including source code) is 395. Out of all these engines I am the least inclined to go with this one, because it is just a (as far as I can tell) very competent engine at what it wants to do, without having anything to recommend for or against taking into account my needs. It is also more expensive, but from 250$ to 395$ the jump is not that big, and you can always buy the indie version first and then upgrade if you are fully committed.

And then there is Unity! Without Unity the things on this list seem more attractive. But Unity is huge, powerful, easy to use and fast. Or so they tell me. It is definitively together with XNA to a somewhat lesser degree the focal point of this generation's development efforts, big AAA producing companies not included. There are 3 potential obstacles. One, I don't know how to use it yet, because I barely installed it today. Two, it is heavily based on the editor and I'm not sure how complex procedural levels will work out. And let's not forget the price, which is a whooping 1500$. Sure, there is always the free version, but this one does not support dynamic shadows and deferred rendering: the very two features I want the engine to do so I don't have to do them myself. Also, I hope that C# from Unity is good (not a standard C#; based on Mono so it should do fine though), because there is no way I am using JavaScript. I want to see JS destroyed and begging for mercy, not coding in it :).

And then there are the two overkill engines: the Cryengine and the Unreal engine. Both have a free SDK (UMK for Unreal). Unreal is old and venerable. Cryengine not as old. Both these engines would be serious overkill. Sure, Unity can also be considered so, but Unity is quite used, so not all games will look good and be high profile. But using Cryengine for this? What's next? The FrostByte engine, with it's three absolutely gorgeous cutting edge next super dooper envelope pushing games, Battlefield 3, Need for Speed: The Run and Dwarves & Holes!

So I'll keep you updated and in April I'll focus more on the practical, leaving a few unread research papers for others too.

Oh, just because I have studied a lot and am starting to amass quite the knowledge base, it does not mean that I am not talking out of my ass. Take everything I say with a grain of salt because I may be wrong. I'm sure there is a graphics guru out there who when reading this would do several quadruple out of phase facepalms while coding with their feet an engine that is 10 billion times as advances as I will ever have.

Disco is back!

Filler - Splat

Yup, still totally stuck!

In an attempt to get a few things done I will try to regroup, focus on a few things, simplify and cut corners everywhere and reuse existing code and features from whatever resources I can find.

Terrain is very dependent on texturing, not just because of how it will end up looking, but because the kind of terrain you have determines what you can do with texturing, thus forming a vicious circle.

And the tile based texture atlas terrain does not mix well with what I want to do with the terrain. Plus is it ugly.

To make up for lost time and to make implementing terrain easier I am going with a much more streamlined vertex structure: as easy as it gets, with the minimum possible vertices and an index list. This in contrast to the more complex terrain I had implemented. Stretching a single texture (or multiple large ones) over such a simple mesh will end up looking bad any time you view it from even normal distances. Only when viewing it from far will it look not blurry.

So I learned texture splatting:

The idea is to take the stretched out blurry texture and splat on top of it a few other highly tiled textures. This is done on the fly by a pixel shader. The only input this shader takes is a special bitmap that gives information about where and how much to splat and the textures that are going to be blended together. In the above video both the heightmap and the splatter-alpha map are only 256x256, while the textured that are used for detail are higher resolution.

In theory this is a simple process and I have fully understood the shader. In practice things are far from smooth. Irrlicht has a 4 texture limit. I think you can get it to support 8 somehow, probably by recompiling. So one texture is the alpha map, and 3 textures used for detail. There is no room for the master stretched out terrain texture, so the above video uses only the detail textures without a master texture. Great, more obstacles!

Also, because I am using a custom shader, built in lighting does not work, so I'll have to merge my terrain splatting shader with my lighting shader.

In the spirit of code reuse, I have used the built in terrain class from Irrlicht, rather than simply my vertex setup model and implement terrain LOD switching. While not really visible in the video, the terrain switches dynamically in complexity based on distance. And it is a good thing that you can't notice that it does this. A slight modification was done to the terrain behavior from that class in order to not apply the default detail mapping but use a shader.

I'll use that class as a learning tool, maybe even adopt parts of it into my code.

Screens of the day 24 - One light to rule them all

I must confess I am kind of stuck on the caves. There are at least half a dozen ways to do it and none of them is without any disadvantages or easy to implement. Caves are the hardest task I have encountered up to this moment. Path-finding is hard, but I am not at the stage where it becomes tricky because I still limit it to 2D. Lighting is hard, but this comes down mostly to my inexperience with it. The real technical challenge is coding around the hardware limits on the number of lights. This will probably be the biggest issue that I will ever encounter, but for now that spot is resolved for caves.

I also read a lot about the marching cubes/tetrahedrons method. It is a very interesting method that mostly comes up in research projects and looking extremely promising yet meanwhile abandoned projects. It is also almost always either poorly documented or using a far too scientific language. Let's face it: with the wealth of information out there, if you are having serious and lasting problems with midpoint-displacement or Perlin noise terrain, then one probable cause could be that you are not cut out for programming (or are just a beginner). But not so for marching cubes. While the basics are fairly simple, solving the very common ambiguity problem and making seamless LOD switching is hard as ******** *** ******* ****.

So while I am not done yet with caves, terrain has been improved. Selection works again, there is a single cell focus "cursor" and mouse movement now tracks the top of the surface, whatever that is, making it a lot easier to navigate and select stuff.

Until I manage to do something meaningful with the caves, I need to write about something, so it's filler time again. This time: shaders!

I get the impression that Irrlicht's support for shaders is not the best in the world. Here are some observations:

• Still can't get global variables to take on the value they are initialized with. I have to set these values manually from C++ code. Not a big issue, but I see code out there that does not need to do this. Today we live in pretty XNA dominated era: the C# game centered library for Windows and Xbox.  From what I saw, I really like XNA. It is like DirectX on crack (the positive interpretation of that statement; not the addicted to drugs one)! You can find a ton of XNA resources out there and a lot of what I have learned about shaders comes from there.
• There is no support for the "technique" section in the shader sources. This is quite problematic because you can't just use a shader file to create post processing effects, deferred rendering and other composition techniques.
• You set shader variables (actually they are called constants) on a shader type basis, that is you need tot specify the name and know if the variable is a vertex shader or a pixel shader one. Setting a pixel shader variable as a vertex shader one (and vice-versa) generates an error and wrong rendering. But what if the variable is shared between the two shaders? Experimentally I found that I need to set them both for it to render correctly. Again, there is little to no indication found in Internet samples about having to do this.

These caveats not withstanding, I managed to get per-pixel Blinn-Phong lighting working, but only with one light:

Above you can see a sample. Bottom left is a shaded table. Bottom right a shaded table with transparency. flying higher you have a non-shaded non-lit model. You can't see it here very well, so let's take a look at the same scene from another view-point:

It is not really fair to compare a non-shaded non-lit model to a shaded one, but you saw how these models look with fixed pipeline lighting in the past, either using a two or tree lights setup. BrewStew suggested to make the table more bright for outside lighting conditions, and this is as bright as it gets:

Now this is per pixel lighting, meaning that every single visible pixel should be effected by lighting. In the future front-to-back ordering for rendering should be implemented to speed up the rendering. Light can be directional, point or spotlight shaped, and have ambient, diffuse, specular and emissive components (emissive is ignored for now because I don't need it and no use slowing down rendering even more). Materials also have diffuse, specular and ambient reaction values, together with a global ambient color. So the lighting model should be compatible with both the fixed pipeline lighting system of the GPU and things you can get in a 3D modeling program. I tested the values a lot. Global ambient is a little bit weird and so is specular. Specular not only creates those very shinny highlights that everybody want from specularity, but also globally shades the object based on the direction of lighting. This means that you can't put shininess to zero if you want no highlights, but instead need to set also the specular parameter of the material zero too or risk having the object rendered incorrectly. Playing around with the parameters I managed to get this very ugly and hard pseudo-self-shadowing to render:

Very ugly, but it is free, as in it does not take more time to render like this.

The problem is that this is an one light setup. Having object textured helps reduce the problems with such a setup and some shapes are better suited than others. But take a look at this untextured column to see the problems with a one light setup:

The side that faces the light has shading, but the shadowy areas are completely "flat".

I played a lot around with the parameters and got a lot of results, like this one:

Except for the one light limit there is one more problem: vertex colors are too washed out. Let's try and use a pure red vertex coloring and see how it looks:

The unshaded object is very open about its coloring, blasting away with its very explicit red coloring. The shaded object only takes a hint about its reddishness, becoming slightly pink. You can improve this with lowering the ambient material parameters:

The problem is that this makes the unlit areas of the object too dark:

A solution is to modify the color of the light and make it red:

Now you can play around with brightness:

This approach has one serious disadvantage: performance! This engine has been about one think from day one, and one thing only: huge amount of objects in scene with smooth performance. My engine does not care about coloring: it is free. You can have all objects in the scene have a single color or all a random color (from a limited pool of available colors). Even if you distribute colors with the worst possible mathematical probability to maximize color change, the engine doesn't care. Now, if I were to shine a different light on each object to color it using pixel shader the rendering performance would become dependent on the color distribution. Having just a few colors with large chunks colored the same would have a minor impact. Using the above mentioned worse case distribution would reduce performance. And I am not talking about 2-3 times lower performance. More like 30-100 times lower framerate. And this if we consider that rendering per-pixel lighting is free. Just having a large number of colors with worst case distribution and zero cost shaders would drop the performance a lot.

So instead I'll try to implement a dual light shader and see if we can get vertex colors to shine though like that. Let's see how that works out first.

But still, I managed to get some interesting results. Here is a metal shader I created:

And seen from above:

So... yeah... filler...

81 – Dig(gity)

I had this post so long on mind that I forgot half what I was going to say, so abridged version recapping the week:

I started working on caves and managed to implement a very powerful but very complicated solution. Any convex surface without an underside could be used to create caves underneath if it was high enough. Increase the complexity a little and I'm sure I would have wound up with full boolean mesh operations. Early on I realized that this is far too complicated for what I need but I still finished the implementation, tested it, saw that it was great and immediately deleted it.

I think that a simpler design and terrain overall would benefit my game. I scaled back a lot on the complexity until I reached a fair compromise between ease of creation and what can be done with terrain. The time spent creating the complex solution means that Snapshot 9 will be a little bit behind feature wise: no level switching, no slicing and no caves.

The first good reason to have relatively simple terrain is that terrain is often just eye-candy. You take a look at it, go "WOOOOOOOOOW" and then you spend 99.99% of your time staring either at a horizontal section though the landscape or and underground level.

The second good reason is that complicated terrain requires complicated tools. I could create a full on set of terrain editing tools for the GUI, but this is not really the scope for the game.

Instead my focus was to create an accesible terrain that looks good and is very natural, both initially and after you interact with it. In my last video I have shown and early digging prototype and that had very straight edges around the part that was removed. I don't like that because it brings back memories of the cubes. MUST... AVOID... PUBES... JOKE... I want softer surfaces and I'll show what I came up with at the end of this post, but first something has to be done about the way the landscape looks.

While there is plenty of detail in the map, because of the flat shading you can barely tell what is going on. Using Irrlicht's built in normal calculator I tried to add definition to the landscape. The result did not turn out too great. I obviously needed something more powerful, a solution implemented by me, but by using this first I learned a ton about normals, Irrlicht implementation for them, how Blender exports them and why using flat shading requires more vertices that smooth shading. Here is the result of the Irrlicht solution:

This solution, while far from perfect gave the required detail to the landscape. It also highlighted some problems with terrain creation. When I learned midpoint displacement I remember reading about rectangular artifacts that get carried up and create undesired pyramidal structures. I knew about this bug but for a cube world it was never a problem.

So the next step was to write a powerful terrain generator class to handle all these tasks, using the same algorithm as the above one, but replacing square midpoint-displacement with diamond-square midpoint displacement. And since I needed normal calculations, I made the class provide these too. The class not only works for discrete values, i.e. each point in the heigh-map, but can provide the correct values for height and normals for any floating point coordinate using interpolation. Currently the class only supports discrete and interpolated height and normal operations, but on a need by need basis I'll add support for other useful values for a 3D engine, like tangents, binormals, bitangents and bisexuals.

The next step was to improve lighting. Correct normals and bad lighting does not a pretty map make. I learned a ton about lighting in general. How did I do that? By trying to learn shaders. This is how you learn shaders: you start of with a "neutral" shader, one that simply converts from logical 3D coordinates to coordinates the GPU can use using the world-view-projection matrix and fill the areas with a single hard coded color. This is the most basic shader that you really need to fully understand. Then you start expanding upon it, adding ambient color, texture, directional lighting, specular highlights, normal mapping, etc. I am not ready yet to write my shader lessons, but I'll get there. First I need to solve a problem: out there on the Internet people often write shaders that have global variables initialized with some value. Pretty straightforward. Yet when I use those shaders, the initial value is never assigned and is instead zero, resulting in incorrect rendering. I need to figure out first why these values work for other people and not for me.

Using my new knowledge I created a new lighting model that only uses two lights. This might solve the problems with Irrlicht's default PS 1 shaders that only support two lights for some effects. The new model is far from perfect, and I am still iterating upon it, but this was a good result:

I further tweaked terrain generation, digging softness and lighting.

Terrain generation is done. As in the shape of it. I will not improve upon it more. Not all randoms maps are equally good, but I haven't found a single bad one yet. The difference between them comes down to subjectivity. I like hilly/rocky landscapes, but some might enjoy smoother ones. The random terrain generator creates both.

Lighting for terrain is also done, at least until I switch from fixed pipeline lighting to a terrain shader.

The only thing left is texturing. A new texturing scheme will come creating more realistic and modern looking landscapes, but not now.

The terrain generator has a resolution. It creates an amount of discrete points in concordance with that resolution. Increasing the resolution does not simply create a good looking terrain with twice the size and useful data. It creates a terrain with twice the size but half the detail. Every time I change resolution a new set of parameters for the terrain generator must be manually and experimentally determined. It is similar to a picture. Scaling it up won't give you any more detail. You may use some filtering, but eventually you'll need a higher resolution image. So I am going with a fixed resolution terrain.

So I implemented strectching. The terrain data can now be used to create terrain at any scale. Currently the unit borders are aligned with the heightmap to create the best looking map. I have not yet investigated how this looks if they are not aligned.

The terrain is also seedable, as in there are a few values that when reused always create the same terrain. Useful, this way I can always play around with my favorite maps.

Putting it all together we get this:

There are some coloring bugs that I have intentionally left in for terrain. While examining the physics and terrain model you can easily prove that these values are wrong, but I actually like it this way because it gives the terrain a little bit of personality and outlandishness. The texture stretching bugs are not intentional and will get fixed!

Overall this was a great first week for March's experiment-a-thon. A smashing success even. There is absolutely no way I am going back to cubes! Or a 2D engine!

Now just let's hope that I can hack together a Snapshot 9 out of all this stuff!