Today, in part two, we'll try and fix the nasty exaggerated normal mapping effect.
To not overly spam the hell out of you, I will be including just the non-normal mapped result and the normal mapped result, skipping the middle one:
This was the number one problem area and the result is better. Let's see it from behind:
Good, good. Now for some grass:
And dirt:
Now I really don't know what to say. Which to prefer? For now, I'll stick to the three terrain rendering qualities: low, medium and high, but in parallel you will be able to choose the softness of the normal mapping. This theoretically would give 6 types of rendering, but low soft and hard soft are identical. A little bit too many options for the standard user. In the future I hope to have even better shaders that blow both soft and hard options right out of the water, thus having fewer options available.
The above option is hard medium. Hard high option is almost identical to what you see in the pictures. But this method has some problems: a nasty repeating pattern can become quite obvious. Hard high tries to hide this pattern, but is slightly slower. Both hard options are faster than the soft ones.
Actually, after dozens of shader permutations I tried, I am very confused. What is normal mapping? Why doesn't it work on facing away polygons? Why does spherical harmonics work? Why was the detail so soft before? What did I do to make it harder? Why is my cat orange? These are all questions that blend together right now. I need to clear my head and take a few days break from normal mapping terrain shaders.
But don't worry, I have material gathered for 3 more posts already!
I am currently doing frantic clean up and restructuring for the creation of the game library, that is a somewhat more general self contained engine in a box. This will take time, especially since I spent some time dicking around with terrain shaders, so have two filler posts about that. Feel free to skip this post and the next one, because the end results that are more stable will be mentioned in a normal post.
So we saw last time how spherical harmonics can be used to enhance the perception a normal mapping in surfaces that are facing away from the light. An added benefit is that the ambient lighting is dependent on the skybox, making objects more blue on a clear sky and more purple/orange with an intense sunset skybox, but since I did not manage to finish the skyboxes, you can't see this benefit. But even with this effect, the normal mapping is quite subtle.
Let's put into overdrive! Do kids still say that?
So I will be presenting a set of 3 pictures, the first being simple blending, the second normal mapping with spherical harmonics, and the third the new EXTREEEEME normal mapping.
First the high terrain textured zone:
Ahhh, the effect is far too powerful here, but the bumpiness is accentuated.
The back side is a bit better, but still too bright. but notice the difference in bumpiness when compared to the second sample.
Grassy surfaces fare a lot better. And finally, some soil textured surfaces:
So this shader has potential. I need to do something about the exaggerated problematic areas. Which I'll do in part two.
I started creating the standalone library that will be the base of the engine. Right now engine code and game code are all in the same project. Before the split can occur, I need to clean up both and then make sure that the library only has generic functionality.
So while cleaning up the code, I got an idea: what if I investigate the range of texture weights that a terrain chunk has. If a chunk can be made to only sample 1/2/3 textures instead of the 4 that it currently does, I'm sure I can greatly improve the performance. But the numbers were not as prone to cooperate and most chunks seem to use 3 textures. I was hoping for more chunks to have 1/2. This distribution depends a lot on the properties of the terrain. Large smooth hills will probably have a more desirable distribution.
But since my terrain has its current structure, I decided to move this feature to the backlog. It will be a worthwhile optimization and might allow terrain than uses any number of textures, not just maximum 4 (with the caveat that a single chunk can have maximum 4 textures), but the benefits I would get right now are not worth it.
But since I started going over the terrain and shaders, I decided to try out spherical harmonics for terrain. Now, I am still not an expert in this domain and would greatly appreciate any input, but let's see my results.
So here is a terrain with basic shading and texturing:
Texturing looks good, but there is no fine surface detail. We traditionally fix this with normal mapping:
This add a fair amount of detail to surfaces that have such an angle that they receive enough light. But if you take a look at the area under the crosshair, which that is not under direct light, there is zero detail added by normal mapping. So here is where adding spherical harmonics might help, giving this result:
Better? I think so. Not sure if this is the way it should look. The effect is also quite subtle. Which is not ideal, since it is a fairly expensive one. It only works with pixel shader 3 or higher. Surprisingly, while the effect is pretty expensive, it is not significantly more expensive than normal normal mapping. Normal. This running joke is getting so old! Anyway, one could optimize it. Experimentally I noticed that for distant terrain, while the difference between no normal mapping and normal mapping is big, at least shading wise, the difference between normal mapping and spherical harmonics is not noticeable, except that the terrain gets a just a little bit darker. So the engine could decide to skip this effect for distant terrain.
Here are three more samples, in the same order:
Hopefully I did not mess up the upload order :)! Here the effect is harder to see because the whole surface is facing directly away from the sun. To better see the effect, download the pictures and use a program that allows fast swapping between them.
And a third sample:
OK, so what do you think? Does it look good? Is it worth it? Anyway, the engine now supports 3 quality settings for terrain. If your PC is beefy enough, there is nor reason not to turn one the spherical harmonics based rendering (which in the options menu will be called "high"), since I don't think it looks uglier.
Here is another screenshot, this time with antialiasing for a little bit of eye candy:
XNA support antialiasing out of the box and my engine also supported it pretty much from day one, but this feature was lost once I switched over form normal rendering to rendering to offscreen targets. Antialiasing is back and more customizeable than ever. Previously you could only choose if you wanted it or not, and XNA did its best to try and follow that suggestion. Now you can choose the number of samples.
Also, you may notice that the terrain is slightly more vibrant. I started messing around with the parameters and brought terrain lighting more in line with object lighting, creating a better illusion that everything is illuminated by the same sun.
And I continued experimenting. Not all of them turned out OK:
I'll continue to clean up the code and prepare for finalizing terrain code.
Hey, this is technically a screens/journal post, but since I managed to break terrain lighting there is no screenshot I currently wish to put out there :P. Still I have some data that I want to record for posterity.
Starting with the mention that sometimes I am stupid! My current terrain is not 2 square kilometers like I said two posts back! It is 2 kilometers wide. So that's 4 square kilometers, with 1-2 kilometers view distance. The speed calculation is still correct though.
So in my quest to reach that 10 square kilometers terrain (or maybe even limitless) I started profiling memory allocation and enhancing terrain load/save.
Now terrain is created by a new component that only has this responsibility and what it creates it always saves on disk, then it does some preparation for an eventual load of said data, then it clears up all resources.
My application starts with consuming 32 MiB of RAM. I don't know what eats up so much, ask XNA. Terrain generation for a heighmap of 2048x2048 takes this up to 65 MiB. After the save and cleanup phase, I am left again with 32 MiB consumed RAM and a 16 MiB map on the disk. Because the maps are saved pre-chunked and I don't want to deal with non-square chunks while still keeping the game running under the XNA reach profile, a little bit from the map is always lost and the actual saved size is 2032x2032.
The saved data contains the height information, together with other useful information, like global min/max height, grid spacing, chunk bounding box, etc. After a load of the map memory consumption grows by about 16 MiB and before using the data, one can change some properties, like grid spacing, without any actual performance hit or need to process the data again.
So after loading the data, we are now ready to create the vertex data for the map in memory. This is when memory consumption jumps up to 199 MiB. Then we create the vertex and index buffer objects, raising memory to 690.
Interestingly, BEPU is very smart about heighmap handling, and as long as the data loaded from disk is in a format it likes, it will not use any significant amount of extra RAM besides the actual height data.
Then finally after everything is loaded and prepared I am left with 777 MiB total memory consumption. Sometimes even over 800. It's over 800!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! OK, not funny unless it is over 9000.
I did some ugly hacks, getting rid of some data and breaking device loss recovery in the process, but I did manage to get the RAM down to 620 MiB.
So this won't work.
I'll try a few things. The first solution will be to keep only basic heighmap in memory. Taking my current 4 square kilometers map and adding 3 more equal sized chunks for a total of 16 square kilometers would only eat up 64 MiB. Then I'll try to only load initially the terrain around the character, up until the fog limit. Then as I move around, I'll dynamically add and remove chunks from memory.
My guess is that this will be far too slow. So I'll probably have to add multithreading.
Yet again I find myself working on so many things at once, that I couldn't get a new stable and finished build. So new features/changelog:
Implemented basic INI file settings. C# uses XMLs more and I might migrate to XML, but now there is an INI file in the game folder that holds a lot of common settings. A lot of the in game variables are now loaded from the INI so I can tweak the game settings without recompiling the game. Also, the INI is hot-loadable, so you can edit it and the game can pick up the changes without reload.
Light and material information are now stored in more advanced classes, that know how to talk with the shaders and get their parameters from INI files.
Worked on world scale in order to give the world its final and consistent scale.
Implemented ray-casting picking. Using it I created an experimental system for placing items in world without having to drop them from heights. You want to have item placement as smooth as possible Imagine that you are decorating a shelf and placing a bowl on it an this causes everything to be blown away, like sometimes they do in Oblivion/Skyrim. Not cool :). The placement algorithm also tries to place items only on relatively smooth and horizontal surfaces, so you don't end up with placements that immediately get changed by gravity. So you can't place a barrel vertically on a wall. The system is far too finicky at the moment of writing, sometimes refusing apparently smooth surfaces, other times placing items on a much too highly inclined surface. There is also now an extraordinarily ugly cross-hair for better spacial awareness when placing objects.
Take a look at this terrain:
Hmm, it seem to be using a single texture. Which is actually quite a good result. There are no obviously repeating patterns thanks to the shader I talked about a while ago. But why is it only using a single texture? Well I updated terrain to support custom texturing, so the texture of a zone is no longer tied to its height. Sure, I'll use this setup for initial terrain, but then you can change it. Using the cross-hair, I wrote hello on the terrain with grass:
The resolution of the terrain is not that great and texturing is tied to the resolution, so here is how the "H" looks from close up:
I think I did these screen shots without normal mapping, but normal mapping still works.
Terrain editing. Because of the low resolution of the terrain, I am yet to find a good algorithm to smoothly edit the terrain, but it is still doable:
Hey, been working on a lot of things all at once so I couldn't manage to get a stable build yet, so today I'll do a short preview of thing to come. Did not have time to investigate the Meteor engine this weekend, but I do have my monitor back and working.
First, I optimized a little the rendering pipeline. There is now one less CPU matrix multiplication per object and shaders are starting to get optimized for pre-shaders. Considering that for single render target shadow mapping you are rendering things at lest two times, that can mean two CPU matrix multiplications per object. I also optimized the shadow mapping part of the rendering pipeline and it uses even less multiplications than normally are needed for rendering.
What I forgot to mention last time is that there is now support for fine tuned control of render targets. You can render normally, into a new render target, into multiple, etc.
I also optimized the load order of the texture streaming system to first load all diffuse maps, the all bump maps, the all specular maps. This way even if the load time is exactly the same, loading seems faster because the more important maps are brought in with less delay.
Then I started working on model import/export. As said, loading 7 models with a total size of a few hundred KiBs takes 3.2 seconds. Unacceptable! By my calculations, a busy scene might take over 20 seconds just to load the meshes. I really did not want to resort to a custom binary format for storing meshes, but in the end I had to. I wrote a fairly lengthy mesh importer/export specially for static meshes (no bones/no animations). In the future I might add support for bone animations to the format. Here I had the opportunity to get rid of another bone multiplication. See, the exported mesh is exported weirdly and you need to take into account the first bone transformation, and also the standard loading process that I want to bypass computes normals and tangents, but again based on this bone information and... I have no idea what I'm doing! So the bone information is stored into my custom format (but animations are not) and still used because I couldn't convert the normal and tangent information into something that makes the rendered output look the same.
Anyway, loading time is now 10 milliseconds. Let's compare the two values: before it was 3200 milliseconds, now it is 10. 320 times faster. Even if I load 10 times as many meshes, it will still take only 100 ms, so I think we're done here.
And now the great advantage of my system becomes apparent. You can modify a mesh/texture and by pressing a hot key in game the assets are reloaded on the fly, without having to exit the game or reload the level. Currently everything is reloaded, but adding support for the file modification time-stamp is trivial so I won't bother with it :).
I also further enhanced the procedural building generator, but it is still at least a week away from its first underwhelming version.
Then I tried to do something about the horrible back sides of objects that are facing away the light. I think some form of global illumination is needed, but that exceeds my current level of knowledge, so I hacked together a shader that is not particularly convincing, but still looks better and performs OK. There is just one particular graphics bug with it and currently I am not sure if the technique is flawed or it has to do something with my custom mesh import. Once I figure it out and decide if the technique is good or not, I'll list all the advantages and disadvantages.
But enough word-thingies. Let's bring out the screen! First, we have a normal scene with traditional one directional light = sun lighting:
The backsides are too dark and you can't see the bump mapping. Now the after shot, with the new shader:
Better, but still not great. I need to investigate that visual bug and decide if this method is a keeper or not.
Finally, let me show you a closeup of the procedural tree:
It uses the same technique to keep the backsides from going dark, but it is not normal mapped (yet). Also, the leaves are billboards, so going close to the tree and spinning the camera around causes the typical billboard display bug.
The system comes with a few tree species out of the box but currently only one can be loaded at a time. On the other hand, every single tree is unique. It is not the same mesh repeated over and over and... I made a few tests, and generating 400 trees (not pictured) takes a few minutes and eats up over 600 MiBs of RAM, so clearly each tree can't be unique, not even on small maps.
Wow, a screens of the day post! The last one was a few months ago! Time to correct this dire lackor random screens!
Not exactly the ideal moment for such a post, but I had some good results today and I'm excited about them so I might as well show them of, even if the results are very early, rough around the edges and incredibly buggy. You see, I don't just write tutorials these days. I am actually putting the base of the XNA engine in place, while at the same time learning and evaluating the potential of XNA. I'll write a formal review of XNA after I get a few months of experience with it. Writing basic stuff lends itself very well for beginner tutorials. Sure, since I'm going with the flow and don't have a real lesson plan, the quality is not that great.
Anyway, I am always ahead of the tutorials I have written development wise and more specific features of my project don't lend themselves so well for tutorials, but you'll see this when I get to tutorial number 5, which is all over the place and will probably be skipped and rewritten some day. And while I am not done with the feature I am working on, here is an early WIP screenshot:
Yes, that' right! Grass! Quite the unfortunate implementation, but it does not look half bad. Keep in mind that this is on my development computer with and integrated GPU, so no antialiasing, filtering or performance. In this screenshot we only have one grass texture.
Now let's try another one:
Since in this one the camera is closer, it does not look that good, but the grass that is farther away looks decent.
Now let's combine the two grass types (doubling the density):
Decent variety but kind of random and colorful. I'll explain why these WIP screenshots are so colorful and what the bugs are in tutorial number 5.
Let's try three textures with little bit of extra variation in the geometry (T-T-T-TRIPLE DENSITY) :
How does that look? I say promising!
Now if these were the good old days, I'd read GPU Gems 2 - Toward Photorealism in Virtual Botany and GPU Gems - Rendering Countless Blades of Waving Grass until I memorize them, google everything on the subject and spend 2-4 weeks implementing a great grass system. But now I have very strict productivity goals, so we'll see what I can hammer out in a couple of days and then move on. My game is not about grass anyway. It hurts to grow up!
PS: I wonder if one can buy the GPU Gems book series and if you get access to any source code?
You may have seen my video with that awesome disco scene and wondered what it was all about. Is it subtle (but surprisingly deep if looked from a certain perspective) social commentary on the state of disco as a musical genre and the entire movement after its collapse in the 80s? I also put out another video, showing a close-up of the stone stool:
The model is the same as it has been since its creation, yet somehow it looks a lot better. You guessed it: it is lighting! But let us rewind.
Be warned! Massive massive rant incoming. I love to write and not having done so in quite a while makes the ranty go all bendy! OK, I never claimed to be good at writing! And I am always way too lazy to spell-check!
The original plan was for March to be a month of wild experimentation, where I try out a bunch of seemingly divergent techniques to get a feel for them and based on the results of each individual experiment and the condensed conclusion on their ability to be merged into a single project, decide how to continue and what exactly to do with this game and following which style. But I encountered far more obstacles than expected, which lead me to an increasingly disproportional amount of research versus coding and experimenting. After a few days of trying unsuccessfully to implement caves I was on the net trying to find solutions and before I knew it I was neck neck in research papers. Research papers are a strange beast. They are often very obtuse, math heavy and otherwise barely readable by mere mortals and in the end they only give you hints on how to overcome the problems that are presented. I guess you need to protect your ideas. They also come in exactly two templates, together with that blasted font that a lot of them use. What is this? A lot of them seem to be created with LaTeX . I'll never understand the fascination of academia with LaTeX. Sure, it has a deep tradition and can solve a very specific set of typesetting issues with unparalleled ease. I am a strong believer in the right tool for the right job, so when one of those typesetting issues arises I will point you in the direction of LaTeX. But if you are not in that situation and are basically writing a Word document but would like to hide that fact with fancy words and redacting it with LaTeX, I have news for you: 2012, Unicode 6.1 and printing industry wide standards called! They said to tell you to stop being a hipster. Heck, you were a hipster before hipsters existed!
Wow, way to come back from an absence of writing: making up for lack of content with ranting and alienating all LaTeX users. As said before, I like LaTeX when it is used properly. But we have a better standard that keeps getting better and better, yet in 2012 the general implementations for it are still lacking. I don't want to type \c{c} to get a ç. I'll just write a ç, it will appear as ç on the screen and it will be encoded as ç in the saved document. And yes, that character is completely different from c̦, and if they show up the same to you, then you can see why I said that support is still lacking.
But back on track. I encountered several blocking issues. One of them was caves. I worked and researched the least amount regarding caves, because this research quickly degenerated into a general terrain algorithm research. I went over CLOD (Continuous LOD), which is quick and simple, offering medium sized CPU terrain with an average amount of pop-in. I studied ROAM and ROAM2. ROAM2 seems hell-bent on solving all the issues with terrain rendering and thus raises quite a few issues that it tries to cover in its implementation, making it a tough pill to swallow at first. Geo-clipmapping seems easier to understand and has more implementation attempt available for study. Also, a distinct move away from CPU can be seen in the realm of terrain rendering. Even I was able to get a tiny patch generated using GPU, just to see if it is possible.
Now one problem that you will find when researching terrain and come to hate is seams! Seams in the terrain. Places where it is not watertight. Changing terrain LOD levels always results in seams. This is not a problem in research papers. They will acknowledge the problem, almost all will give a high level solution and quite a few will go into detail. The problem arises with third parties of similar status to myself (dudes interested in terrain that implement something based on what they have read). Almost all implementations lack any form of stitching. Another interesting thing I have discovered is the infiltration of such geeky topics in social media. With today's level of technological incursion in popular culture and a prevalence of gaming, you see young people on social media who have never coded in their life having genuine interest in game engines, rendering and often terrain. Just check out YouTube and the amount of terrain rendering going on there, and the amount of comments from random people written in typical YouTube comment style ("LOL, TEH TERNAIN ROKS WUH IS THAT, roam2?" or "my cat is called poo-poo-cacku! how hard is to do that land in unity?"). Anyway, what I'm trying to say is: most of the random implementations lack support for any kind of stitching, it is almost always acknowledged and then hand-waived ("lol, stitching. i'll do that latter"). I find that funny for some reason and both motivating to never-ever release any terrain without stitching, even if I know for sure that the very next update which I can do in 30 minutes with one hand tied behind my back will add great stitching.
One thing that caught my eye is CDLOD by Filip Strugar. A very unique and quite elegant idea and implementation. I think that at the time of writing there are no commercial games or other serious big budget software using CDLOD for their terrain, but I tried it extensively and I think it is quite innovative and works great. The research paper is very readable but kind of short. OH MY GOD! What have these papers done to me. I am complaining that a research paper is too short! It also comes with full source code that for once works as claimed and without issues, in a normal brute force variant and a more advanced streaming variant. And a few large data sets are provided. The terrain needs to be pre-processed and written to disk in a custom format, but after that is done, the streaming client is capable of loading even gibibytes of terrain data near instantly (relatively speaking of course, based on the actual size of the data) and then life streaming it. The performance is great and you can adjust the detail and processing power needed to render the often hundreds of kilometer wide terrain both before pre-processing and on the fly while running the application. You can reach several compromises this way. I created a setup where terrain was rendered at very high detail, but far from maximum, and had over 60 FPS. Then I created a setup where terrain renders at over 180 FPS. And I created a setup where the same terrain is always rendered with between 300k-500k triangles. And I did not mention one of the biggest advantages (IMHO) of CDLOF: morphing! Traditional terrain systems suffer from pop-in. You take a step and somewhere in you view frustrum the geometry of the terrain changes. Often this is very jarring and if you would have a sound effects specialist make a sound for that effect just based on the visual ques, it will probably be some sort of "POP" or a metallic slightly reverberating "SCHLONG". All terrain implementations that feature LOD have pop-in. In the good ones you won't notice it. CDLOD is no exception, as in the geometry of the terrain changes. But it is not a pop effect. It is a morph effect. CDLOD does not use fixed LOD levels, instead it always morphs smoothly between two LOD levels. So "pop-in" is always present, at every step, not just at fixed thresholds, but it is not a pop, but a smooth morph. If you miss configured you detail parameters, sampling rate and view distance you will get very ugly LOD changing artifacts. Only they are not a pop, but a morph. If on the other hand you created a good compromise of quality/rendering speed there are almost zero visible morphs. And if you go all out and create a very high setting, I think you get a incredibly smooth engine, capable of rendering a closeup of the land from the view-point of a normal person, and then fly-up like Superman with hundreds of kilometers/hour, gaining a great overview of the land, while doing all this seamlessly.
If you can't tell, I am quite impressed and excited about CDLOD. I encourage you to download the demo and try it out for yourselves. It is fairly large download, but if you are interested in terrain rendering, I think it is well worth it and the algorithm will warm its way into peoples hearts. Just check out the elegance of this morphing sequence from a higher LOD level to a smaller LOD level, especially when compared to most approaches that simple exchange the chunk with one that uses 1/4 so many faces:
That image is from Filip Strugar's paper and all intellectual rights regarding CDLOD belong to him.
Did I loose you already? I am only getting started! The first think I researched was caves, but that became a terrain rendering research. Very interesting, but not helping me directly right now. A second major issue exists that is an obstacle in my plans to create this engine. Let me show a screenshot, again from and external source, that should give you a hint on what this problem is:
That, ladies an gentleman, is Dungeon Keeper 2. Released in 1999. I have never played a Dungeon Keeper game before. I was somewhat aware of it when it was released and I can recall reading back then an article about it in a printed magazine that specialized in games. Back then it was hard to get a hold of games and I greatly enjoyed reading those magazines, even if I would rarely if ever get my hands on specific games. Probably the only reason I remember DK is because of the very iconic red devil dude, who I think is the Dungeon Keeper. He is no Mario, but still a recognizable icon today. Well, maybe. If you are old! And so is that woman in the black leather that often appears near him. I have no idea who these people are and can't really find out, because being from 1999, DK2 has problems running on Windows 7 64 bit, even the GoG version. I can play about half a level before a crash, 1 an 1/2 if I'm lucky, in software mode and I can't save. But from what I managed to play, I can certainly see the similarities. So if I ever get accused of cloning DK2, while technically not true, I can see why that person would think so. Squad of imps digging out caves, squad of dwarves digging out caves. Sure, the similarities are just superficial and this becomes apparent from even playing just 2 levels from DK2, but you know the drill by now with appearances and the way they influence our conclusion reaching mechanism and impulsive behavior. But I was aware even back then of the great Bullfrog studio.
I have higher polygon count, better textures and even fancy bump mapping techniques, yet there is something about that screenshot that screams "I am prettier than you!". I am talking about the lighting. It is even worse if you run the game, because in software mode, those light even manage to flicker. Now 1999 was a long time ago and as said back then I had limited resources to get my hands on games so I did not get to see every game like I do today. But still, if my memory serves me well, this looked pretty good for 1999. Not quite cutting edge technologically speaking, but still very solid lighting. Imps even cast shadows from multiple light sources (limited to 4; in software mode at least; determined empirically) and the charm of character design, color palette, overall personality of the world and the narrator/dungeon master, makes this a pleasing experience, even today.
So how did they do it? Maybe it is all trickery. Clearly, that is not vertex lighting. Unless they actually used a super high vertex count and made it intentionally look like the vertex count was low (you can count the polygons in that screenshot with ease), that looks like (not the best by today's standards) pixel lighting. The top of the map, the tiles that are not dug out are clearly vertex colored and involve no actual lighting. Or at least I can create with ease an almost identical effect by vertex colors. But the walls? The trickery argument would imply texturing tricks combined with lighting and some vertex colors. The other option is that they had an actual no trickery lighting scheme, as in if you see an area lit, then there is a light placed there and there are lighting calculations done to render the scene. The lighting model would not work that well for first person view point (judging from how it looks from top down; I read that you can poses an imp and control him first person, but I have no idea how to do this), but It is blast from top down view.
And how would such a game look in our modern ages? Maybe something like this:
Oh god, the lighting on that is crazy. Not especially pleasing aesthetically, but still good. That (if I did not upload the wrong picture) is a screenshot from Dungeons, developed by Realmforge Studios and published in 2011. Another game that I did not try or have any real contact with before this research phase, but this can be easily remedied and I hope to get my hands on it soon and do some speculative reverse engineering based on sight purely: looking at lighting at trying to think of ways to achieve a similar lighting engine. The game did not have too high reviews, but some reviewers liked it, so I may even be worth a shot just on its own merits. I think there were some crushed hopes of it being a new DK or some other form of complicated history. Anyway, I would love to have such a lighting engine.
But why is lighting so hard? First of all, it is just plain hard even if you have an engine with infinite and unimaginable lighting capabilities, because lighting comes down to aesthetics. You need to create an aesthetically pleasing lighting scheme first. This is very hard and completely dependent on your artistic aptitude. Even AAA titles sometimes look muddy. Why is that? Often a combination of factors, but more often than not it is either poor resolution and wrongly chosen textures (the composite albedo of you scene which without lighting determined the look of you game can have an equally disastrous effect if poorly chosen as very bad lighting) or just plain bad lighting. Or both. In the indie scene, on those projects that never make it out of some alpha/beta stage and never become a real and finished product (fingers-crossed that this is not fate's way of doing some fore-shadowing) you can often see competent lighting, but which is still somehow wrong.
And then there is the technical implementation of lighting, which, depending on who you ask, may or may not be a lot harder. There are a series of obstacles. Let's take standard hardware lighting. This is limited to 8 lights per object and is only per-vertex. Having a limited amount of lights per object defines the way you build you world. You need to figure out/compromise on the number of maximal moving lights (people carrying torches, projectile light, etc.), take that value into consideration and then divide the world into small enough pieces that while rendering the fixed lighting for those pieces you lave enough buffer space for the dynamic light. Let's say you divide the world into axis aligned square areas and pick a reasonably large chunk. You can't pick a very small chunk, because of the maximal number of batches constrain for the GPU. The chunk size is also dependent on how mobile and flexible you camera needs to be. So you pick a fairly large chunk size and most of the time you get only 2-4 light on a chunk, but sometimes you get 7, let's say in an area where a corridor merges with a room near a corner. Then if you have a single worker there you are fine. If you have more than one, you need to turn off either a worker light or one of the other lights. And you can't just turn off a light at random, because as workers walk the light totals will shift wildly and you will get very ugly flickering. You need a very complex light management component. And this without taking into account that dynamic entity movement can be truly freeform. In such an environment worker movement patterns may not mix well with that cell structure created by the aa boxes.
And after you coded this beast, you get only vertex lighting.
If you want pixel lighting, you need to use shaders. Here is where things become even more complicated. You need to take into account the capabilities of the GPU. You'll have a hard time coding such a scheme with pixel shader version prior to 3. With pixel shader 3 I created a lighting scheme that support any number of lights. Theoretically. But there are issues. First of all, the shader has a maximum number of lights. If you create a shader that can render up to M lights, but you parametrize it to render N, where N <= M, your performance will depend on both N an M. A shader that only currently renders 1 light but can render up to 4 will be faster than one that currently render 1 light but can render up to 8. Or so have all my practical experiments shown. So you need a bunch of shaders, probably one for 1, 2, 4, 6, 8 lights, and then probably a few more to account for a few extreme circumstances where you get a lot more lights. The lighting manager becomes simpler because it is less under strain to chunk the world in such a way that you have at most 8 lights active at once. You can target it for common scenarios and the suffer a performance penalty for uncommon ones. On the other hand, the lighting manager becomes more complicated because you need a bunch of shaders. A whole lot actually. Not only do you need a set with different maximal light counts, but a shader that can render a point light and a directional light is not identical to one that can render two point light, a point and a spot or all other combinations. If you need more light types than spot lights, the number of shaders increases dramatically.
And this without taking into account effects. Two very important effects are normal and parallax bump mapping. These depend on lighting. So if you want bump mapping, you take the total number of lighting shaders, and create a variant for each with bump mapping. And repeat for parallax. I have created a shader system that handles bump mapping for a theoretically unlimited number of lights. I still need to write one for parallax. How does on do that? First you study really in depth the lighting shader. Make sure you fully understand it. Then you study the normal mapping shader. Do this until it becomes trivial and you smirk arrogantly and say "that's it?". Normal mapping is a really easy trick and involves the same normal based calculations you do tho normally light an object. Make sure you can switch between the two with ease. Then study the multiple lighting shader, marking the differences in calculations from the single lighting one. By now, you should instinctually be able to update the multiple lights calculations to take into account normal mapping. Then study the differences between normal and parallax mapping, and repeat the process, starting from the multiple light normal mapped shader.
Easy! The hard part comes next if you want o support pixel shader 2. Small indie titles do well IMO to support a wide range of hardware. Pixel shader 2 is quite limited when compared to pixel shader 3. I managed to create a PS2 shader that support up to two lights in single pass mode and up to 5 lights in multi pass mode, as in one light per pass. I am working right now on a shader that supports up to 2 lights in one pass, up to 4 lights in two passes and up to 5 lights in 3 passes. For now I can't seem to get over 2 light per pass because I reach the limit of shader instructions and can't get over 5 lights total because I run out of constant registers to pass the lighting parameters. This while using "object local" multipass, as in a object is rendered using a single multi-pass shader. I am researching to see if it possible to do true multipass, as in you render either each object in turn or the entire scene for each pass, and each pass has its own set of parameters. Maybe this way I can get more than 5 lights. And of course, multipass gets almost linearly slower as the number of passes increases. Whatever batch count you obtained by chunking in you light manager, multiply that by the number of passes.
Then there is normal mapping, which for now I only managed to do with two lights, one pass with pixel shader 2. Parallax is in the works, and so is normal mapping multi pass, but I'm not sure if it will turn out okay.
And all this to get the first half of the universal lighting model. A set of shaders for each maximal light count (or a single one if you want to be slow, but general), doubled for normal normal, tripled for parallax mapping and multiplied by who know what if you want pixel shader 2 fall backs. Oh, and you can mix vertex with pixel lighting as a fall back method. If you pull this of and you also create a good light manager, you can get as good results as Dungeons. I hope! Is this how the big boys code lights? Seem prohibitively hard for indies.
And one final very important note: a lot of per-pixel light, even if single pass, are slow. No matter how good your chunking is, you will hit a brick wall sooner or latter depending on the fillrate of you GPU. It is easy to find this limit even with small scenes and a modest number of lights. A forward rendering scheme has the worst case scenario of number of objects x number of lights.
And then there is the second half of the universal lighting model: shadows. There is also post processing effect, but let's just stop at shadows. Shadows are an especially hard beast to tame and up to date, now in 2012, we do not have a perfect shadowing model. Shadow mapping is kind and the same principle in some form or another offers pretty much the only scalable solution. The only problem is that it produces very aliased edges. You can smooth them out, but this is not perfect. The quality also degrades a lot with the size of the scene. Anyway, shadow mapping is widely used and instantly recognizable. I think I can tell both the size of the shadow map and the scope of scene from a Skyrim screenshot. Lately, anything that I play gets inspected for all the lighting and shadowing visual cues I can gather. Cascading shadow maps and parallel split seem to greatly improve shadow quality. And even smoothing is starting to give great results using percentage closer filtering. I am not a native English speaker (DUH!!!!!!), but does "percentage closer filtering" sound awkward to anybody else? Like they did not finish their sentence. So while shadow mapping is full of faults, there are solution out there to correct them and they keep getting better and better. But I am having a hard time wrapping my head around normal plain-and-simple shadow mapping. With normal lights. You can just forget about omnidirectional lights and those dammed cube maps and volumetric rendering.
Lightmaps are another tools that can be used in conjunction with the other techniques. Like shadow mapping, lightmapping is again hard to wrap you mind around and even with a fully GPU based implementation, it is not fast enough for real time updates to a dynamic scene. If you have a game with fixed levels created by a level designer baked lightmaps are probably your best friend. If you engine support real time blending of lightmaps with dynamic shadow, the better. Just bake you light maps and you have great scene ambiance. But my "levels" are all procedurally generated. And there are volumetric light maps, that affect dynamic geometry that passes through the lightmapped volume and renders correctly, similar to dynamic shadows. Yup, they exist. I can certainly confirm that. Those are a real thing. Yup.
And there are also volumetric lights. I just love what you can achieve with those: turn a bland scene into an almost eerily charming fairy tale scene full of warmth and hot pockets. I would love to have a partially volumetric lights based implementation. Would suit the look I am going for very well. Another thing I am studying.
I think the best results would be given by soft shadow casting lights that can be blended with a procedurally incremental pseudo-dynamic lightmap and a pixel shader based volume effect added to standardish spot lights.
Shadow mapping also has implications on you batch count. You already have a high batch count and shadow mapping renders the scene from the perspective of the light first. No my engine has been created from start to support a very high number of objects. More that other engines. I have my own method with which I am fairly satisfied that offers the possibility to render a huge amount of objects. You saw it, I kept showing it off. It does so by using a lot of GPU, a lot of CPU and a lot of normal RAM and GPU RAM. So it is fairly intense and the only things that makes it as practical as it is that my algorithm is designed to have a very good batching behavior. So if you combine my algorithm with the need of the light manager to have small objects and a high batch count, the whole thing goes to hell. I also expanded upon my algorithm and created a new super fast but super blurry version. Too blurry to use for normal objects, but for distant objects it is about 20 times faster in the worst case scenario. 3 to 6 times faster for normal scenarios. This implementation is slightly lighter on the CPU, but is is also undone by forced high batch counts.
So while I plan to keep this implementation around, being the crown jewel of my engine and all, I have also researched other ways to get a huge number of items rendered: instancing. I know now about 5 variants of instancing that give mixed results, all better than no instancing, and all except one far worse than my implementation. That one good instancing implementation is not as strong as my own method that routinely handle 90k objects. It is more suited for 65K items. But is does so with very low CPU use and considerably lower RAM requirements, so it might be a solution for the batching issues with lighting.
So as you can see, creating such and engine is extremely hard and will probably take me over a year. But take a look at this picture:
And the Disco video. A large number of lights and bump mapping. Actually, the disco video has a whole bunch of wrong normals used for mapping. The picture above has these issues corrected thanks to some further splendid effort from BrewStew.
Another closeup of how I can render stuff now, in case that the video is too blurry to see the details:
So how was this done? With deferred rendering. Not deferred lighting. Deferred rendering offers some huge advantages. It is fairly geometry agnostic, as in the performance of lighting does not depend a lot on the complexity of the scene, so you can really go over board with huge poly count if you have great batching. Countering this is shadowing. Current shadowing schemes care a lot about geometric complexity, so even if you can light your super complex scene with ease, you won't be able to shadow it. Still, this means that you need to balance your load around shadows, not around lighting. Another advantage is that is is very fast, even with hundreds of lights. With very small lights you can even get thousands of light at once on the screens. My disco video uses 550 lights. Another advantage is that it is fairly simple. When you first read about it, the idea sounds very outlandish and has absolutely nothing to do with the fixed lighting pipeline. Zero in common. But it is still pretty easy to do.
So these are the advantage: huge number of lights, very fast, geometry agnostic. What are the disadvantages? Quite the truckload! No out of the box hardware antialiasing. Do you like your games with MSAA. Can you tell the difference and swear by the quality improvement and performance of CSAA? Well, they don't work with differed rendering. In the screenshots and videos you can see antialiasing and there are quite a few solutions for it. They don't give superb results, but are good enough. The only problem is that they are quite the performance hogs, especially since deferred rendering is really scraped for bandwidth. DirectX 10.1 gives you access in shader to each sample when using multisampling and this makes hardware MSAA viable again, but it does limit your choices of GPU and operating system. I don't think that I have a setup capable of this and anyway and I don't understand yet the code for it. Using deferred lighting you can again get MSAA working, but this implies an extra full geometry pass. Which does not play well with another disadvantage: performance and fill rate. Deferred rendering eats huge amount of resources, on paper at least. I was genuinely surprised of how fast it is, being faster with 550 light than forward rendering with 20. Still, deferred rendering is very fillrate dependant and completely changes the GPU compatibility graph. With forward rendering you have a curve, with low end working so poorly that you probably wont get interactive framerates and steadily increasing from there. With deferred rendering you get a plateau where no older GPU are capable of even creating the context for deferred rendering and the curve starts up from the plateau of zero compatibility and the first jump is a big one. My Disco demo runs with the integrated Intel GPU on my laptop with Optimus.
Another disadvantage is that I can't handle transparency. As in alpha blending. Alpha masking works fine, as seen in the video. There are several solutions, but the prominent one, depth peeling is not really feasible yet in a general way. You need to multiply the amount of memory and fill rate requirements with the number of depth you want to "peel", making an already expensive pipeline a lot more expensive.
Another disadvantage is that is forces your lighting model on the entire scene. It is harder to create a setup where you do tricks by lighting different parts of the scene in different ways. And of course, having multiple materials is again very hard. I am currently tackling this problem of having universal specularity, making a polished metal object as shinny as simple cardboard box. Even if I manage to get specular mapping working, making truly different materials available is even harder. Like rendering an object with Phong, another with Blinn Phong and another with Lambert shading. Raise your hand! Who here does not love micro-facets?
So I am not really sure if the advantages outweigh the disadvantages. I'll keep experimenting with both forward and deferred rendering for a while.
But how did I do this? I must whole-heartily thank Catalin Zima . His blog, explanation and implementation of deferred shading for XNA 2.0 were pivotal in getting me to understand the technique. Then I must thank Roy Triesscheijn, who ported the implementation over to XNA 4.0. And last, by no mans least, Emil Persson (a.k.a. Humus) who had the second implementation of deferred rendering I managed to understand and who works under a more familiar C++ environment. Anyway, Humus's site is absolutely incredible. There are samples there, some even really old, that do very impressive things. Like that gold glow effect from 2003 written with GLSL assembly shaders. Or that interior scene with Phong lighting model and moving omnidirectional shadow-casting lights. Wow! Also let's not forget all the reading I've done beforehand, from which I will only mention a the chapter from GPU Gems dedicated to deferred rendering and a very interesting paper written by the developers of S.T.A.L.K.E.R. about their use of deferred rendering in the game with the same name. There was also a forum that escapes my mind right now.
Now, in all honesty, I really don't think it is possible for one man to create the engine and then code the game I am trying to create in any reasonable amount of time. So besides researching things that will allow me to create the engine my game deserves, but not the engine it needs, I also started researching engines in hopes that maybe I can find an engine that offers all the features I need and then I can concentrate 100% on the game. This is not an easy task because most engines seem to be centered around the pre-designed level structure while my game world is procedurally populated. Even if I do not transition over to an engine, I am so done with Irrlicht. I had my gripes with it for ages now, and researching any advanced topic and how to do it in Irrlicht is painfully depressing. You will find some ancient forum posting, with some promising results, but eventually all links stop working and none of the concerns and incomplete implementations ever get finished and production ready. Also, I have been suspecting for a while now that Irrlicht is to be blamed partially for the difficulties I had in creating such a large number of objects in engine. I whipped together a little DirectX sample that did shader based per vertex lighting and rendered 90k cubes without any frustum culling or instancing. And it runs with 2 FPS on one machine and 13 FPS with high AA and AF on another. 13 is really low, but with Irrlicht I can't get even 1/4 so many items to render at that framerate. Maybe Irrlicht is doing a lot of useful stuff I am not doing right now in my DirectX mock-up test, things that I will eventually have to do and then well get the same performance, but still, I don't consider this a good start for Irrlicht and taking into considerations all my other issues, including that while abstracting away all the low level stuff, if makes it extremely hard to implement things it was not designed for, it is time that we end out collaboration. Still, Irrlicht is not all that bad and if you don't have an overly ambitious project it may be a good place to learn the basics. And I will miss being able to switch at will between DirectX and OpenGL.
Next I investigated Ogre and I did not like it at all. It is "just" a rendering engine so you need a lot of plugins to get things that you need working. Even without any plugins, with the absolute bare minimum hello world application, it takes ages to compile. Maybe it gets better with precompiled headers (which I do not have routine access to and I don't need them), but precompiled headers are at most a band aid solution to fix the symptoms of the absolutely abyssal module support in C++. More precisely, zero module support. They are not enough and you need other tools, like making sure you do not have a horrendous include hierarchy. I precompiled that hello world Ogre example and it had over 800k lines of code. Geeee... I wonder while it takes so long to compile? Look, we are programmers. As programmers we wait a lot for stuff to compile, to load, to process. The sad truth is that there is no way around it. Even modern languages like Java or .NET based ones eventually start to compile slow, once your code base becomes large. Sure, the margin is much higher than for C++. But eventually we'll have to wait a lot for stuff to finish and there is no way around that. At the work place! Not on my personal projects. Those compile blazingly fast indifferent on what language I use. The load fast and are generally fast. A 3 second compile and link is very slow for what I am used to. So there is no way I wouldn't get annoyed as hell while working with Ogre. To make matters worse, Ogre uses a resource abstraction system where you set up folders with textures, materials, etc. and then you load the resource based on identifiers, not full disk based path. So far so good, I really want and need such a system, The problem is that in Ogre this is super slow. Granted, my hello world was only loading a single mesh (the ogre head), but the resource locator was configured to use the entire Ogre Demo resource bundle, almost 50 MiB of data. This causes a pretty large delay when the application loads, even if you only use one mesh. I tested and created some resource folders where only the needed mesh files were present and start-up was instant. You mean to tell me that startup takes so long only with 50 MiBs of data? That is not actually used? My game will use more than that. In the time it takes Ogre to index or what not those folders, I'm pretty sure I can load 3 times as many resources in anything else, even in Irrlicht. Heck, when compared to Ogre, the instant gratification provided by Irrlicht makes me remember our time together with slightly more rose colored glasses. And while the Ogre samples are more advanced that the Irrlicht samples, they still seem pretty last-gen. DirectX samples from the SDK are more advanced and interesting than that. So a massive no no for Ogre.
Speaking of DirectX, I became quite comfortable with it, more precisely with DXUT. It is one of my primary candidates for my tutorial series (not the game). What tutorials do you say? As said in a previous post, once I finish developing shaders I will do a small series on them. Since I changed my mind, and I won't be doing just some series on it. I will do full fledged tutorials with full public source code, including of course the shader code. I don't know when exactly, but soon. My other candidate for the same tutorials is XNA 4.0. While not the biggest fan of C#, I like and respect in enough for me to work comfortably with it and as a plus I have nothing but good things to say about XNA. Sure, the way you sometimes have to write custom pipeline classes can seem awkward, but generally speaking I believe that XNA is the go to platform for young coders who wish to learn the ropes of graphics programming. And I am not saying that because I am some Xbox fanboy. I have PCs and a PS3. But being able to port easily to the Xbox is a huge plus. Too bad for the licensing agreement and all the limitation on your port though, including the cost. So my tutorial will be either in DXUT or under XNA. The shaders are going to be identical anyway. Just the code that invokes them is different. XNA is easier to learn and develop for, but I find the levels of tutorial materials more than enough and XNA seems well represented on the learning front. DXUT is more powerful but a lot harder to learn and there is not enough learning material out there, so I fell like DXUT tutorials for my shaders would do more good. I was dead set on XNA, but while writing the lesson plan on complexity flow, I realized that I underestimated the amounts of glue code needed under DXUT. I have no problems with the glue code, especially since you can of repeat it unchanged from project to project, but I'm not sure I can present it in an approachable manner.
What do you think? XNA or DXUT?
As for my game, I am probably not going to port it to either XNA or DXUT, but if I ever start a new 3D project, it will 98% be a XNA one, even if this means that I need to write tighter code to compensate for the relative speed difference from C++ to .NET.
As for real finished engines that I could use for my game, after going over Ogre, DXUT and XNA, or during, I first investigated C4. C4 is pretty powerful and scalable. It has marching cubes terrain with built in LOD switching and stitching, so seemingly perfect for caves. It also features quite the bit of coding in addition to the drag & drop stuff, so building levels procedurally seems to be in. On the other hand, while the engine looks good and has extremely good bump mapping capabilities (I don't remember of the top of my had what the technique was called, but it even has some self shadowing and works superbly with brick like surfaces), I can't say that the engine looks that great. Everything created in it looks slightly muddy, and the lighting is somewhat foggy, like if there was some grain filter over the scene which there is none. If this were the only option, I would gladly accept it, because it still looks good, tons better than what I had and has the marching cubes terrain. As they say, you do not look a gift horse in the mouth. There is a licensing cost. The basic version is 100$ and you don't get sources or the ability to sell a game created with it. The standard version is 250$ and has the sources plus ability to sell. While I won't be using it on this project, I will probably buy the basic version.
Another engine I considered in Torque3D. This one I did not get the opportunity to play around in depth and don't understand its capabilities as well, but it certainly seems more advanced than C4. If I understand correctly, this one focuses more on drag & drop functionality, while having scripting tie together the capabilities of the engine. I'm not sure if this is ideal for my needs. Also, the scripting language it has uses a C++ inspired syntax. Really? Really? Out of all the well designed scripting languages out there... The information on licensing is slightly more contradictory here, so I can only say that it is between 75$ and 179$ (with some super expensive extra premium version that offers something important I think), but don't know exactly which price range offers which features.
Another engine I superficially reviewed is the NeoAxis engine. This one is based on Ogre3D, but it is in C# and you build the world in an editor, but does have quite a lot (too much for C# and Ogre?) coding behind the curtains. It scales well (except for shadows, which either work as expected on adequately powerful machines, but pretty much disappear on low hardware). It has a start-up time, but this time it makes sense with the amount of resources the demos actually load. The indie version is 95$ but has no source code. The commercial version (including source code) is 395. Out of all these engines I am the least inclined to go with this one, because it is just a (as far as I can tell) very competent engine at what it wants to do, without having anything to recommend for or against taking into account my needs. It is also more expensive, but from 250$ to 395$ the jump is not that big, and you can always buy the indie version first and then upgrade if you are fully committed.
And then there is Unity! Without Unity the things on this list seem more attractive. But Unity is huge, powerful, easy to use and fast. Or so they tell me. It is definitively together with XNA to a somewhat lesser degree the focal point of this generation's development efforts, big AAA producing companies not included. There are 3 potential obstacles. One, I don't know how to use it yet, because I barely installed it today. Two, it is heavily based on the editor and I'm not sure how complex procedural levels will work out. And let's not forget the price, which is a whooping 1500$. Sure, there is always the free version, but this one does not support dynamic shadows and deferred rendering: the very two features I want the engine to do so I don't have to do them myself. Also, I hope that C# from Unity is good (not a standard C#; based on Mono so it should do fine though), because there is no way I am using JavaScript. I want to see JS destroyed and begging for mercy, not coding in it :).
And then there are the two overkill engines: the Cryengine and the Unreal engine. Both have a free SDK (UMK for Unreal). Unreal is old and venerable. Cryengine not as old. Both these engines would be serious overkill. Sure, Unity can also be considered so, but Unity is quite used, so not all games will look good and be high profile. But using Cryengine for this? What's next? The FrostByte engine, with it's three absolutely gorgeous cutting edge next super dooper envelope pushing games, Battlefield 3, Need for Speed: The Run and Dwarves & Holes!
So I'll keep you updated and in April I'll focus more on the practical, leaving a few unread research papers for others too.
Oh, just because I have studied a lot and am starting to amass quite the knowledge base, it does not mean that I am not talking out of my ass. Take everything I say with a grain of salt because I may be wrong. I'm sure there is a graphics guru out there who when reading this would do several quadruple out of phase facepalms while coding with their feet an engine that is 10 billion times as advances as I will ever have.
