103 – I suck at shaders

After so much time, I still haven't mastered shaders! Not even something simple, like bump mapping with ambient bump mapping and spherical harmonics. I've been always suspecting that there is something wrong with my bump mapping and this comes down to three things:

• My bump maps suck. I have done them myself and while things like the stool seems to work OK, the barrel from some angles always seems to have the metal rings protrude instead of extruding.
• I think that the equations I am using are not exactly correct. The point light one was clearly wrong and I fixed it, but the fix makes no sense mathematically, at least for me.
• The weights of each component are not well enough tuned.

In order to better locate my errors and increase quality, I am going to do three things:

• As a background task, I am going to study in depth (again) normal mapping and spherical harmonics. All the theory that I can get my hands on, making sure that my equations are correct and I understand why.
• Get my hands on some sample normal mapped meshes that I can use as reference and try to improve my own maps.
• Introduce a secondary rendering mode to the engine.

For now I only managed to introduce the new rendering mode. The engine still renders things as it used to, but with the press of a button ('l' in my case) you can switch to a second rendering profile on the fly. Using this I managed to compare parameters more easily and came with an all new rendering profile. Be prepared, because I am going to spam you with before/after screenshots.

So here is a before scene with a  lot of barrels and stools:

And the after shot:

Notice the added depth illusion on the barrels and stools, but also it is pretty clear that the rings on the barrel are not extruding the right way. I am going to go around this scene and post before/after comparisons form different angles:

Getting those images aligned in Blogger was harder than needed :). I hope they are in the correct order. The effect is customizable. Here is another profile that should look exactly the same to you, unless you know what to look for, and even then it will probably looks the same. Well, it is subtly different and alters the properties of light circling objects.

You can come up with a ton of settings and the parameters have a wide range of valid values. Here is what happens if you go outside the valid values:

I kind of like the new profile. I can't wait to get my hand on some proper normal mapped meshes done by professionals and use them to fine tune my new profile. Also, spherical harmonics produce some unnatural results for quite a few input values with this new profile (carefully avoided these values for the screenshots), so that should be also put on the list of things to fix.

But in my engine I don't have only one light source. Let's try out the personal torch, which in the new skill tree will actually be a spell in which you need to invest ability points:

From the front it makes a big difference, because there are two lights meeting heads on. From behind, there is less of a difference between the two profiles. The personal light is best used at night though:

This was from behind, with the moon illuminating the front of the objects. Let's take a look at a shot from the front, where we can see first only the moon light in the two profiles in dark, then with the personal light on:

I grow tired of these orderly scenes:

So, what do you think? I think this is a massive improvement. I still need to try it out with some proper normal maps. I hope I can fix spherical harmonics and get my hands on multiple sky-boxes, so we can see the harmonics in action, with the skybox contributing in real time to the illumination of objects (the contribution is minor). If I somehow am not able to fix the harmonics with this new profile, I might get rid of the atmospheric illumination element of shading. It is cool, but not really needed.

102 – Snapshot 2 is going to be great

The theme for snapshot 2 was "RPG", so let's leave that for last. Take a look at this screenshot from the engine:

I should really start cutting out the window border for dramatic effect! Anyway, in this screenshot you can see the new "hub" approach I have taken. The engine has now two centralized states, one meant for exploring the worlds and one for using the tools available in several windows with a traditional mouse  driven interface. You can swap anytime form game mode to system mode by hitting "`" on your keyboard.

When the little menu in the top left corner is visible, you are in system mode. Here you will have access to normal tools, like the world manager which is basically a much more advanced save manager and the game options. Normal things for a game. But you also have access to the engine's constructions set, which is build in seamlessly into the experience. You don't need a separate executable, learn a different tool, wait for loading, push updates or fight a sluggish interface. The tools from the construction set work instantly at any point and the changes are reflected live in the running game. The price to pay for this is minimal, except for the development time :). One note though: when hiding the system hub you will revert back to your loaded game. If no game has been loaded, you are left with an empty screen. Take a look at the screen that greats you when you boot up the engine:

OK, you probably won't have any saved maps. After creating a few you will get this. The purple background signals that no map is loaded. In the future there will be a clearer indication for this. If you load a map you will be placed normally into game and the system hub will disappears. Press "`" and it will be shown again, with all windows in the same state you left them (except for the world manager, which closes by default when loading a map; but if you show it again it will be in the same state as you left it before the close).

The world manager now lets you also give the height in percentage for the newly created maps. By snapshot 002 it will be updated to support entering the world seed too.

Another new comer is the options window. Currently it only supports just the main graphical options and it does not read or save them from disk, so every launch of the engine you have to change the options if you don't like the defaults, but it is a good start. It will be updated soon.

You may also have noticed the skybox. Adding this was done through the material editor, which received a healthy upgrade in functionality.

But before this I added primordial support for modules. Modules describe all the rules and content of the game and are designed with mod-ability in mind. Once the support is rich enough, the era of hardcoding is done for. For the engine to work and look the way it does, you need tot know what materials and meshes to load, what materials to assign to which part of meshes, what meshes to assign to the abstract in-game objects, what physic behavior to assign to them, etc. This was done in the past by hardcoding these relations, but a fully moddable experience needs to add support for creating these relations visually in the construction set and save them as data in a format designed to be easily modded. So now the material editor can add new materials and it also saves them to the main module, which for now is in XML format.

After the editor and and core engine skybox support was added, I was left with a white skybox. Using the material editor, I assigned the textures one by one, seeing the changes reflected live on screen:

I also added night support for the skybox:

I know believe the night is not dark enough.

But the skybox raises a new set of problems. In the past with the sky being a single color and fog having the same color at its maximum intensity, the far clip plane of the terrain was perfectly hidden. Now you can see it and it is rather ugly! My solutions for this: don't look at it. This is very hard to fix and impossible with my current view distance that is very low. So for now I'll ignore it, especially since it will take me months to find a solution for it.

There has been another major change: the engine now uses logical coordinates. In the past I have had many issues with the world scale. I had to change it around several times, a lengthy and error prone process. Now the engine uses logical coordinates and every dimension is expressed in abstract "meters". When going from logical coordinates to real ones, meters are converted to unit-less vertex coordinates at a fixed scale. Every world can have a unique meter size, but once I find the perfect one, all worlds will use the same scale. The scale is a hidden value anyway.

Adding logical coordinates support took me about a day. After adding support, I reduced the scale of the world several orders of magnitude. This had the unfortunate effect of changing subtly the look and feel of the world. Physics behaves differently and fog had its look changed.

So finally let's talk about the RPG elements. I'll write a post about my design for character progression, but for now I added a stamina bar with fairly complex rules to it. You have different rates of gaining and loosing stamina, together with an exhausted state again with its own rates. Governing this stamina bar and your overall mobility is the new skill tree:

Currently it is only a single tree, but the skill points can have drastic effects. 8 of the 9 points are implemented, with the last one being scheduled somewhere in the next few weeks. Instead I'll add further trees.

The progression in the trees is horizontal. The tree has mostly passive abilities, but the passive ones can have such a drastic effect that they will change your active gameplay style. Green outline means a new ability, yellow outline is an upgrade to a previously unlocked green ability and blue outlined abilities are mutually exclusive with another blue outlined ability near by:

This is the build I use during testing. The abilities are 100% implemented and even slightly balanced, but the tooltips are not complete or correct. Since balancing changes every week based on my experience with the ever evolving trees, the tooltips won't be 100% accurate for months. This is a common problem in games, with world of Warcraft being a prime example :P.

The skill tree works closely with a new class added to the engine: the Player class, which manages all stats and attributes and is responsible of spending and gathering resources like stamina. 

Currently the player character is reset to default at each game launch so you'll have to reassign you ability points.

So that's about it for the main features that I manged to squeeze into Snapshot 002. After some more polish and testing, it should be uploaded on Monday.

A fair warning regarding snapshots. At this early stage, save game compatibility is basically 0. Most new snapshots will render your previous saves useless, so a new set of generated worlds for each snapshot is called for.

Changelog:

• Implemented to system hub.
• Implemented primordial module system.
• The world manager can now control the height of generated maps.
• Enhanced material manager: you can add materials and it saves the material list. 
• Basic options menu added.
• Player character class has been added to keep track of the player's status.
• Ability trees have been added. Currently only about half of the mobility tree has been added and implemented.
• Skybox support.

Bugfixes:

• Fixed most of the bugs with changing states in the system hub.
• You can now copy & paste in the engine as you can in any windows application.
• Material editor no longer crashes when loading a non-existing texture.

99 – GUI guy

I've been away from town from Friday to Tuesday for a small family reunion and I barely managed to edit and upload this video:

Since the video is so commented I think I don't need to repeat everything in the blog too. Some small enhancements are still to be made (like a file selection dialog), but this is the base of the material editor. Another editor will be used to modify the meshes, assign materials and physics to them and preview them in a small 3D view. A third editor will manage the item lists, defining in game items and assigning meshes and other properties to them.

So let's talk about something else. In the past I used Ruminate for GUI. It is feature poor, slightly buggy and at places ill designed, but small and does the job. Still, a ton of work would be needed to bring this up to par. So I tried Neoforce. Neoforce has itself a ton of problems and I think that fixing them will take just as much time as fixing Ruminate, but Neoforce is much more feature rich. Imagine that you are trying to create a complex and distinct good looking GUI on top of an existing system. You could base it on Windows 3.1 or on Windows Vista. It is not guaranteed, but there is a better chance of getting a better result if you base your widgets on Windows Vista than on 3.1. This is a purely metaphorical example, but you get the idea.

So some problems with Neoforce:

• It is no longer maintained, neither for XNA 3 or 4.
• Getting it to run is a lot harder than necessary. Look at any forum thread and you will see that people have problems getting the damn skins to work. I can't make the skins work on the sample projects included, but I did manage to get them to work in my project.
• There are some problems with making the GUI active/inactive. I am using an imperfect workaround for this right now, but a perfect solution must be found.
• There are issues with focusing. Imagine that you execute two actions with side effects, A and B. If you execute them in this order, B will have a deterministic secondary side effect: a focus change. Now if you add a third action between A and B that touches GUI active state, so A > action > B, A and B will execute their side effect, but the focus change of B is no longer deterministic.
• The event chain order for focus loosing is wrong. If you want to create complex GUI you need to create more events than button presses. Things like actions being taken on edit and focus change are a must. Unfortunately the order Neoforce tigers these in a stupid order, making it impossible to do this.
• Calling Init on all widgets is awkward and stupid.
• There are strange things going on when making widgets larger than their parent.
• Porting this to SharpDX or something will be quite some work.

This is just from the top of my head not having touched Neoforce since Thursday. I'm sure there were more.

As you may know, I am a huge GUI nazi. I worked with GUIs all my life, designing both GUIs and GUI toolkits. I am one of those pretentious assholes who say things like "GUIs are art" and "nyeaaahh". So finding a GUI that satisfies my requirements is pretty hard. If I had time I'd love to fork and fix Neoforce.

Engine features 01 (A.K.A. I am really really done with the terrain; fo' reals...)

OK, a little bit latter as promised, here is the first video showing off engine features that are in a near final form:

First time doing any "real" video editing. Took me quite some time and hopefully my productivity and results will improve on future videos.

Since this video is late and I did not stop working meanwhile, I have enough new features for another two similar videos (but probably shorter) that I could record/post even right now. I still need to space out the content on the blog and YouTube, so today I am going the ignore the rest of the features and focus only on terrain.

So let me walk you quickly though the contents of the video. First I show the dynamic quality switching. It might be hard to tell, but every 1-2 seconds I change the quality of the terrain as I am moving around. I am fairly happy with the preset quality levels and any concerns are more stylistic than implementation related. The presets have predictable performance, with higher quality being slower, but unfortunately my terrain is bus capped so I can't get accurate GPU times on the rendering. A future version 2 of the terrain might use again vertex buffers to reduce the CPU time of passing the terrain chunks around, but this is fairly complicated since a complex cache system must be implemented. Terrain is meant to be editable in real time and creating/destroying vertex buffers is too slow for this. So this theoretical caching system should give timers to a chunk. If a chunk has been edited in the last N seconds it is very likely that it will be edited again, so no vertex buffer should be used. A chunk that hasn't been poked in ages should use vertex buffers. And the system should handle fast buffer migration for high speed terrain traversal. Complicated.

Next I show dynamic view distance. Again, every 1-2 seconds I increase view distance, from 7 to 19 (radius). Farther view distance is not really feasible, not because of the far plane, but because the terrain does not support LOD and is also bus capped, so after a point you spend a fixed and far too large time span just passing on blocks. More that it takes the GPU to render them. Version 2 of the terrain should have at least a simple LOD solution to approximately half the resolution of distant blocks, maybe even insert them into the same buffer.

Next is adaptive detail mapping. The engine supports this feature separately for medium and high quality terrain rendering, each with its own settings. By default for medium we have a distance of 700 and for high of 2000, but for the video I lowered it to 100 to show what the effect does. Alternatively there is a super fast variant of the algorithm that does not do anything for pixels outside the radius, instead of the smooth blending. This can boost performance somewhat, but does give a visible circular edge around the character. At a distance of 2000 this edge is only noticeable when you look for it, but by default I use the higher quality blending. Performance be dammed, look good!

Next is a new arrival: day and night cycles! I have a very complicated lighting scheme and making it behave consistently and also transition smoothly from day to night is a real nightmare. I am not 100% happy with the results, but it is a good start. The night is intentionally bright and clear.

And finally we have the personal light source/torch. During the day it over illuminates everything as it should and by night it gives good visibility for close-by objects. It was very hard to find a configuration that did not produce very desaturated/washed out colors on objects. Oh god, dynamic lighting is so hard. I don't like the way the point light illuminated the terrain though. It is unrealistic, with some points creating a wrong illusion of depth and facing. But I'll keep it like this for now.

With version 1 the terrain is finished. I did not show it in this video, but the terrain is still editable in real time, changing the height and texturing. I will try to release the source code of the core of it soon. I won't start working on version 2 of the terrain, which should one day allow for LOD switching and smart vertex buffer usage.

The next engine feature videos will show off the in game material editor and container/character inventory.

With everything going seemingly so well, it is important to mention things that don't work properly.

Number one is point lighting. It is just wrong. I have the feeling that there is somewhere a very basic error that permeates everything and once I find, it will fix everything. I upgraded the normal G-buffer to show normal-map enhanced normals, not just vertex based normals, and this causes the normals to rotate around wildly as objects are rotating. This clearly illustrates that there is a fundamental flaw in normals and the way point lights interpret them. 

Second is physics timing. The calculations I had in the past where wrong. I have fixed it but I'm not sure yet that the values are correct. With real values, physics actually is a huge performance bottleneck (it was before, only the measurements were lying to you). The good news is that I added multi-threaded support for physics and it scales very well with the number of cores, so if you have 2/4 cores instead of one you should notice a good improvement on physics times. The bad news is that I will probably have to scale back the map size. My i7 can handle the huge 64 square kilometer map decently, but slower machines might have a lot of troubles. Maybe even do a lot of small areas that are separated by caves and passages to hide the loading bars and limit the physics to only one region at a time.

98 – Terrain? More like ter-done! Am I right??!

I think I finally finished with the base of the terrain texturing!

I refactored the terrain shaders, using a very modular approach  with tons of function calls and a clean design. I sure hope that the shader compiler is really good. If not, a final version of the shader might have to be written someday that flattens out the implementation and uses all manner of optimizations.

I also started using branching heavily. I am using both good and bad kind of branching. The good one relies on constants passed to the pixel shader body and I am pretty sure the compiler does compile time evaluation of the constants and removes unnecessary branches from the code. The bad kind of branching is the use of run-time "if"s in shaders. GPU really don't love branching. Pixels are evaluated in a clustered fashion and branches can cause the entire cluster to wait for a sync. This can be mitigated if there is a high probability that all the parallel shaders executed for the cluster will take the same path. I done some testing and the results are inconclusive, maybe tending to go a little bit toward having lower performance if I use branching, even thought the body of the branch that is skipped is more expensive.

Using these methods I created two shader implementations, one very basic for the low quality and one that handles higher quality rendering. These are further parameterized with compile time flags to create all variants that I need. I also managed to greatly optimize the implementation, giving a 10-15 FPS increase on weak hardware. On strong hardware I can't tell, because currently I am bus capped.

I also implemented adaptive detail mapping, allowing you to specify a radius for detail mapping. Medium quality setting use this, not for the performance, but because it reduces repeating patterns in terrain somewhat. On high I am not using it because the small performance gain is not worth it when compared to the quality loss. It is a high quality setting for a reason.

The final step was to do something about the view distance. I determined that the landscape looks the best when I use a very distant and aggressive fog. The farther the fog start is, the larger the terrain seems. The fog is exponential and does a good job (but not a perfect one) of hiding polygons entering though the far plane. This small pop-in is so minor that you won't notice it unless you are really looking for it.

One thing that I need to dos till is make the view distance adjustable at run-time.

Using all the above I finished my hardest task: out of the dozens of permutations, choose only 5 quality settings. This was ridiculously hard because all were tough compromises. Just now I changed the spherical harmonics computation just like that and I'm not sure which one I like better. Anyway, there are 3 quality settings: low, medium and high. You can also choose to have enhance the harmonics for better quality, but this does not work for low quality, thus giving 5 quality levels instead of 6. I am fairly happy with these setting. I also made sure that they have comparable color warmness and intensity, but some minor differences are present.

Here is a video showing a 64 square kilometer map with small view distance at maximum terrain quality, large item density using 8xMSAA and SMAA while the character is running at very high speed traversing the map not quite diagonally (I wanted to go from corner to corner but I messed up :) ):

Now that the terrain shaders are finished (I hope) I need to add day and night cycles to it and see about those lights.

For the rest of the post let me entertain you with some very interesting shader variants I managed to produce:

These are not photoshoped or using any other textures than the one from the video. Just a shader variant that produces strange colors a more wet look:

If I ever need an alien looking landscape  I know where to start. I did not manage to produce workable shaders out of this method because the output is too noisy and weird in lot of places. It also has pretty bad temporal aliasing.

97 – I am so behind

Today I tried to fix the need for a hardcoded rotation for trees. I did again small incremental changes, the first being to save/load the number of bones in the mesh. Then I wrote a small API to load/save bones to disk. Then I made it that all bones are loaded and taken into consideration when rendering and removing the hardcode gave this result:

Wow! I tried to fix it a lot but I failed. So instead I tried to do something about distant trees. They look a little bit flat. Before: 

After:

Take a very close look at the trees. The effect is very subtle but I think it is an improvement. It is a custom shader that does a very approximate edge glow which is relatively cheap, but I still have it disabled for low quality settings. The shader has no name, I just made it up on the spot.

Maybe I should modulate it a little based on diffuse lighting for the backside of trees. Maybe they are a little bit too bright. Before:

After:

Too bright? Sometimes I wish I still had my DH audience so that questions would get answered :P. I won't bother with it more, it looks good in my eyes.

To bad about the bones issue though...

HA! Fooled you! Giving up is the first sigh of failure! Let's fix this!

The first thing I do is open up the Blender three file and add a new box that has the same general shape as the tree trunk and I test with it. The bones seem to work fine. But the actual tree trunk still doesn't work. After further testing I determine two things: 1. there isn't an uniform up direction for exporting from Blender; it depends from file to file and 2. the rotation of the tree trunk is ignored. But only on the tree. The rest of rotations are not ignored. But at least now I have a certain level of certainty that the bones do work correctly  In order to no longer loose time on this I just apply the rotation to the vertices and this fixes the problem. The real issue might be an import/export bug somewhere.

Then I fix the frustum culling for entities without physics (like tree canopies) and get this result:

The objects in view is a lot lower right now because more things are culled.

Finally, I add 2 more trees. Unfortunately  the canopy shader needs custom parameters for each tree depending on the leaf texture quality. Here is a shot of a random forest:

I still didn't get around to implementing grass streaming, but that does not mean that it does not work. Let's take a look at the engine, with all details turned to max:

Pretty good. Must add that skybox. And grass is certainly the most "last-gen" thing I have. I need to greatly improve it somehow.

96 – Run forest, run!

Today's post ain't no filler. I am trying to make and interesting landscape that uses a higher number of assets. So one boulder mesh won't be enough. So let's add another, first without textures:

Now let's see the two meshes together, using the same texture:

If you squint you can tell that there are two different meshes, but otherwise it looks like there is only one. To marginally improve this situation, I change the texture on the new boulder:

It is a little bit easier to tell the difference, but still not great. I need better textures (with some baked AO) and better UV mapping. Up close you can tell that the two boulders are different:

I also added huge barrels just for fun in the idea that I'll stop right here and refactor the physics mesh support. Because it is badly needed. Physics impostor meshes are polluting the resource list and the whole system does more than it is was designed to through hacks.

Here is a sample of the barrel landscape:

The barrels use their current mesh as a physics mesh. This is probably too complicated and could do with a reduction is complexity:

The interesting fact is that barrels can have multiple physics forms and representations. The above one is a full detailed static mesh, but my mobile barrels use a simple mobile cylinder. You can see the two interacting when I place a mobile barrel on a huge static one:

The plan was to stop right here and refactor the physics mesh system, but Chris from Electronic Meteor was kind enough to give me a link to the tree models he is using. I could use any mesh and I have a few available (plus the procedural trees I showed before and never touched since then), but using the same mesh allows for easier screenshot comparison. So I also added tree support.

Still, I couldn't just add trees like that. I did a partial refactor. I started with the resource locations. In the beginning I though that a very convention based system, with a rigid structure where you can easily tell where a resource is located was a good idea. This turned out not to be a great idea. It complicates the pipeline and ultimately flexibility is better.

So I transitioned over all terrain clutter resources and the new one were added using this paradigm. All other resources are yet to be converted. This was done by small incremental changes, each time processing all resources in order to not to break the pipeline and existing resources.

Then I updated the physics impostor system. They no longer pollute the resource list. If you want an object to use a physics impostor, you just set a flag for that mesh and you use it as usual. The engine takes care of all the details behind the background, automatically using the impostor when needed. By default, the physics impostor mash has the same name as the mesh on disk, but with a ".physics" added to the end, before the extension.

So with these two changes, let's add the a tree, first just an untextured trunk:

Oooo... What is this? After some digging, I determine that this is related to bones. My system is very poor at handling bones because I wrote it before I understood the concept. I still need to study bones and after this post I'll be sure to write a proper bone handling system, but for now I just hack it away by adding a rotation:

The next obstacle comes when I try to add the canopy. My system can't handle meshes made out of submeshes that have multiple meshparts. I edit the model in Blender, making each submesh have a single meshpart.

And now another obstacle appears. These tree meshes have vertex colors. I used vertex colors before, but my latest shaders don't support this and neither does the mesh importer. So I update it to ignore the colors. I also log the colors and notice something strange. The colors are all very dark, close to black. What kind of three will this turn out to be with black vertex colors? Maybe I need to use the negative color. Anyway, ignoring colors, we finally get the full mesh:

Next is to load up the textures:

Rendering the canopy requires a similar shader to the grass one, that does not render and does not update the depth buffer for transparent pixels. For grass to work properly with this technique, you need to render grass twice for maximal quality. Strangely for trees, rendering the canopy once or twice makes little difference. I can tells which one is from screenshots, but for the untrained eyes this makes no difference, so I might skip the second render pass for performance reasons. Here is the tree with transparency:

You can also see the trunk texture. Amusingly, this texture eats up 28 MiB on disk and is a total waste for this model :).

And the finals step is to disable face-culling for the canopy to get a richer looking tree. Here is a landscape:

Performance is not great. The main reason is that I decided for the canopy to not have physics. My engine is very physics based. Everything has physics. And when the engine finds the canopy without physics, it gets confused and it can not do frustum culling for the canopies. So all canopies are rendered two times even if not in view. And I am using antialiasing. And a high quality terrain renderer.

The results are very promising, but as you can see, there are tons of smaller issues to fix. I need to do this before I can continue.