91 – I hate shadows

Last week I've been hard at work and managed to put out 3 videos! Now it's time to write about them, because YouTube videos generate just an insignificant amount of blog views. New posts and referrals from actually popular blogs and itesbare where the meat is at!

So I started a new video series, "Modular City". I experimented with procedural, now I'm trying modular. Can't tell yet which one will work out better. Not without a consultant at least. This will be a long running series that started with me putting down a single building foundation and it will end when the engine is capable of creating a city of roughly the size and complexity of Balmora from Morrowind (including persistence). So pretty long. Here are the first 2 videos from the series, generously annotated, so please make sure you watch them under a video player that supports annotations:

During and after the creation of these two videos I did a ton of polishing of the engine and adding new features that I realized I needed, so I did not have time to create the third video in the series as I hoped. So I quickly added a few effects and kinetic responses to everything and put out this video:

The next video in the series will be about placing exterior decorations.

There are literally tons (I know what literally means, this is a joke) of changes and improvements to the engine and I am not going to list them all as I originally wanted. To tell you the truth, I forgot at least part of them. I should keep a changelog updated life, not trying to remember code I wrote a week ago. So I am going to list only the major new features.

Material system

While adding a bunch of new items I realized that keeping the material information coupled with the mesh information was not a good idea. So I created standalone material representations that are loaded asynchronously from disk when needed. Now if I wish to assign the barrel material to a stool, this is all I need to do (in pseudocode): stoolMesh.Material = barrelMesh.Material. Materials react to life update, so changing a material changes the way every single item that uses the material is rendered without having to inform the item of the change. This made the texture streaming algorithms significantly simpler. It is now only a few lines of code.

Active entities

I created a system where entities can be given quite complex behaviors and the system transparently manages the activity of these entities. Each such behavior has its own class. I really like the design! This is what I used for the third video. I created a general timed entity. This entity has a duration, and once it expires, it becomes inactive and triggers some events. From this I created two entities: one that translates a model and once it reaches its destination it optionally triggers a camera shake and it deactivates so the CPU no longer computes translations, and another that shatters an input model in place and once it finishes it will remove itself from the game-world completely so there are no leftover shattered object pieces under the geometry.

Now theoretically I can create an active entity for anything I can imagine. The best part is that even if you have an incredibly CPU intensive active entity, if you just render if without triggering its action, it has the same performance as a simple model render, so you can have any number of active entities that are currently inactive.

Polish and performance

The engine is starting to become pretty polishes and there is an alarmingly low number of hacks. I am starting to find the code base pleasing.

I also improved rendering performance, both in general and especially for terrain. Terrain is currently the biggest GPU power eater, so a quick fix was to make it be drawn last. In active scenes where a lot of objects are obstructing the terrain, this gives a ton of extra FPS.

I also optimized the low quality terrain renderer created for weaker machines and it is now two times as fast. If you have a scene only with terrain and you turn on the LQ renderer, you framerate will pretty much double.

But not everything I did was a success. I noticed some lighting oddities:

This looked to me normals were not exported properly, so I added a test cube and this is what I got:

And this is what I was expecting:

It took me quite a while to track down what was causing this. Eventually I narrowed it down to Blender. It is not an export bug as I first thought, but Blender interprets its vertex information differently based on the format you export to, and the format I am exporting to right now plus the more complex lighting scheme causes these issues. Marking edges as sharp and applying a modifier in blender fixes this. Or exporting as *.obj, which works exactly as expected. Speaking of which, the engine can now import *.obj meshes, a thing that XNA does not support out of the box.

But the main thing that caused problems were shadows. Again! I had not one, not two, not three but four failed attempts at creating shadow mapping, and one semi successful one. And not in this order.

I based my implementation on a Microsoft SSM sample and a Deferred CSM implementation  The SSM one is so ugly that it is unusable, but I started with this and had 3 failed attempts of adding it to my game. Each time I failed I started from scratch, with a new approach. But no matter what I tried, I couldn't get one that works from all angles. As long as the light was behind you, it worked. As soon as you turned toward the light, the entire scene got covered in darkness. The MS SSM example does not have this problem and the code was the same, so I couldn't figure out what is wrong.

Then I tried experimenting with the Deferred CSM sample. This was written for XNA 3 so I had to convert it, fixing a ton of compilation issues, but I got it to work. It had reasonable shadow radius  pretty good quality and was a solid implementation. There were just two problems:

• Not even cascades can make shadow mapping look good. It looks like ass. At a glance it is fine, but as soon as you stop and check out the fine detail it makes your inner graphics nazzi pop a vein.
• It is deferred. I don't want deferred. Naturally it killed MSAA instantly and it has the same disadvantages as normal deferred techniques.

So I took the view frustum calculations from the CSM sample an plugged them into the SSM sample and all the problems the SSM one had (except the ugliness) were fixed. Then I plugged it into the port of the SSM into my engine, and it worked just as good, only with massive shadow jittering.

I wasted hours trying to find what was the problem and I finally narrowed it down to world coordinates. As long as you are around point (0, 0, 0) the jiterring compensation works. My world center on a small map is around (2560, 0, 2560), so jittering is a huge problem. I have no idea how to fix this.

You have no idea how much I worked on shadows! Saturday night I even missed a concert because after working the entire Saturday on shadows, I continued working until 4:40 in the morning  Then i woke up at 9:30 and continued working on shadows until the afternoon.

I think it is time to face the facts: I will never have good looking shadows! That is very depressing. Even if I fix the jitterring and go with a good CSM implementation, CSM is still full of artifacts. It just looks like ass. Roof shadows are all wrong, having slivers of light in areas that should be completely in shadow and the foundation of a stool is so small that due to bias correction, it has a distant and unpronounced shadow. I'll probably add the option to have these ugly ass shadows.

I don't get it how shadow mapping is the industrial standard for shadows today! Not only are they universally fucking ugly, but they have ruined me for all games. There is not a game out there with shadow mapping where I don't notice the artifacts and get irritated by them.

And stencil shadows are pretty much dead. There is pretty much zero stencil shadow information out there for XNA 4.0, but I am still going to try and get an implementation working. I know they are dead. I know they are not scalable. But Doom 3 managed to pull it off years ago. And Doom3 got a HD re-relase recently. But I'm no John Carmack. Maybe they don't work with large outdoor scenes. I don't know. But sill, my game will not look as good as Doom 3 or be as demanding on the system, so if Doom 3 pulled it off, there is a chance that theoretically stencil shadows won't be prohibitively expensive.