87 – Not slacking

While I did not manage to find time and update the blog, work continued on as normal and I uploaded two YouTube videos.

I continued enhancing the content management system and also added some content to it. All textures are now loaded by it except for grass. I could make it be aware of grass, but currently the grass system is a little bit hacky, so I'll postpone adding this until I clean up the grass system. Grass textures are still loaded with multithreading, but the content management system is not aware of them. In the first video you are about to see, terrain textures are not loaded asynchronously  but since I added this, and like I said, only grass remains to be added to the system.

Model loading with the standard XNA method is really out of control. Still experiencing the massive bottleneck, and now it needs 3.2 second to load all the meshes. A few kilobytes of them! That is absolutely bonkers. Let me check the texture folders: 86.4 MiBs. That are loaded faster than the meshes! WTF?????? I am preparing to say bye-bye to XNA's content manager and go with solutions that are blazing fast, but I am yet undecided in what format to store the meshes that is fast, flexible, un-obtuse, artist-friendly and hot-swapable. There isn't one quality OBJ loader for C# out there and I don't want to write my own.

In the next video I try to break physics by creating a ton of objects in the same place. I also show off some features of the engine, like placing objects anywhere with physics, new terrain and object shaders, mass, etc. The video is annotated, so you can follow along with the explanations:

Please excuse a few hasty textured objects. Most of the objects that I modeled using Blender are not textured. But since I am using normal mapping, all objects must have valid normals and at least semi-reasonable UV mapping or they will not render correctly. It is not about looking good: I could use completely wrong and ugly (which I did) textures. A reasonable UV mapping is needed for lighting to not freak out!

Another thing you'll notice that the sides of objects that are on the opposite side of the light are very dark. A long time ago I was but a teenager and I picked up a 3D modeling book for 3D Studio Max. That's why I am not helpless at 3D modelling and can create some low res but passable objects. But ever since the day I created my first light in 3D Studio Max, I had this problem with the back sides of objects being dark. I now believe that for outdoor scenes traditional lighting schemes don't work. I don't know about you, but I do go outside and things outside are never dark during the day. There is global illumination, light reflecting and illuminating objects from all sides and shadows are not too dark. I don't think this can be reproduced with lights. I'll try and do some experiments with some form of environment mapping based SSAO. The problem is that without any form of shading 3D objects look flat. So I need to shade them and I also make sure that the shading scheme works with normal mapping. I can't remember the last time I saw a game having this problem, so there must be a way to do it.

But back to lighting. I implemented shadow mapping. Simple shadow mapping. It look ugly as fuck. I won't even show it to you. I am too embarrassed by it. Sure, there where two problems. One was that I did not stabilize the light position to make it snap to texel coordinates, so every time you moved the mouse to look around or moved shadows would flicker like crazy. And I did not move the frustums back, so things that were behind the player or above, like roofs, did not always cast correct shadows because they moved out of the frustum. While these two are big problems, they are also fixable. What is not fixable is the main problem with shadow mapping and large scenes: aliasing. There is just not enough shadow map to go around and a texel from the shadow map was about 1/4 the size of a barrel.

This is where techniques like Cascaded Shadow Maps or Parallel Split Shadow Maps or whatever come in. I read a few research/tech papers on them and they are more complicated than SSM, so I decided to not implement them from scratch. Actually, while I plan to finish and selectively use my super duper forward rendering shaders, I am actually looking for a ready to be used solution for all my lighting needs, free or commercial.

What I am looking for must follow at least this set of requirements:

• It must play nicely with the standard XNA environment, making code interoperability easy.
• It must not play too nice with XNA, as in being 100% married to XNA's content manager since I am phasing it out.
• It must be light weight.
• It must be simple enough that I eventually can reach 100% understanding of it so I can fix bugs and add new effects and tweak shaders.
• It must tolerate procedural generation. I do not open a visual editor and place items manually. I write code.
• And finally, it must be fast. A low tech XNA Reach backend would also be great, but I'll write my own if needed, no problem.

Finding something like this will be hard, so I'll try everything I can find. In no particular order (OK, I am lying, this is the order I would like to try them out, but this may change): J. Coluna's LPP implementation, the Meteor engine, Ploobs engine, Sunburn engine, QuickStart engine. And maybe others. Probably I'll exclude the QuickStart engine from the list  because I started testing it and it has comparable lighting to my shaders and only SSM, plus it uses a different physics system.

So after a few hours of hacking the engine and plugging in my content management system and physics, this was the result:

This was done using J.Coluna's Light Pre-Pass implementation (J.Coluna | Game development) as a rendering engine. Shadows are still a little bit aliased, but the texel size is a lot smaller than my SSM. I did not like the SSAO effect, so I turned it off. I need to update my terrain texturing and grass animation shaders for LPP, so this is why you can't see them, but otherwise everything works fine, looks good and performs great. As a bonus, skinned animations are also implemented. So I must thank J. Coluna for making this available.  It is a very good implementation, where I just need to increase then number of cascades and find a new SSAO implementation. Even if I eventually don't end up using it, I already learned a ton from it and by screwing around with his CSM implementation, I'll surely further enhance my own knowledge.

But even here you can see the problem with the back sides of objects being too dark.

For this weekend I have a second engine lined up for testing: the Meteor Engine. This has some advantages, like more flexible shadows implementation, a SSAO that from videos seems more to my liking and even the ability to switch on the fly from LPP to Deferred Rendering. I still don't want Deferred Rendering because it is still as complicated as ever, but LPP is fine.

Unfortunately, my monitor on the machine I wanted to use is no longer working. It is in service right now and it may get fixed by the weekend, but repairing it costs about as much as half from a new monitor, so I might end up buying a new one. I might be able to set up a development environment on my laptop and try out the Meteor engine still.

The thing I've been working on the last few days is procedural buildings. There is no way I will realistically have a lot of building models for my cities. At most I'll manage to get my hands on 1 3D "tileset" that can be combined to form various buildings.

So what am I to do? Even if I do eventually get more buildings than I need, until then I am stuck with my current ugly placeholders. So I decided to invest a week or two into procedural buildings, maybe I can get an implementation going that can generate a lot of unique buildings.

Literature on this subject is scarce and lacking detail, so I came up with my own system. The idea is to take a small grid that represents the maximum size of you building. You start off with 4 points representing a rectangle (but all angles must not be square) and from here you determine 4 segments  This is the main room. Then you can further select one or more from this segments and glue to it another square, updating segment sizes and marking data on the grid. Finally, you assign meaning to each segment and try to create a mesh for it. 

First I ended up with a coordinate based system that was very flexible, but only worked in one direction, so I had to replicate it for additional directions. Then I rewrote it to use vectors and now it works for any direction.

The system is in its very early stages and I am not ready to show a video yet, but let's see some pictures:

Here you can see hundreds of procedural one room building, right now with only one doorway each. Size of building, door position and size is all random, and I plan to add windows of different sizes and shapes and then add furniture inside. Also wall patterns. Then add multiple rooms. And here you can see my biggest concern for this system: I might end up with a system that while is capable of producing each time a theoretically unique room, these rooms might end up being so similar that the human mind perceives them as identical. Too early to tell. anyway, still better than a single building copy pasted 10000 times.

Above you can see the insides of the building. There is no floor yet. Once this is done and I have enough furniture variety, a single such room should have a similar setup and item density to one of the single room shacks form Skyrim. Only 10 bazillion times more ugly.

And now let's get to the problem:

From inside, some walls are facing away from the light. You can also see the back side of building. Both are dark and lacking detail in shadow mapping. And they blend together, lacking 3D detail and looking flat. I could place a point light in each building, but I can't place on on the backside of each building.

Or could I? Maybe if I place a selective light, that only illuminates the two walls that are facing away from the light, without affecting any item near it, the terrain of the player? Lighting is so hard :(!