108 – Summer cleanup!

It's still summer, am I right?

So the plan for this week was to get the container list going and implement inventories, together with persistence with them, but I realized that containers are not the simplest form of statics and I should get the foundation finished first before attempting something more complicated.

I updated the module system to support list, for now a single list: statics! As I mentioned, most gameplay rules can be expressed as lists, the implications items derive from the nature of the list and their properties. The static list is the simplest list and it has the simplest implications: if a mesh is added tot he static list, that mesh will be available to be added to any map/interior on the terrain/floor and it will persist. Items can be added to separate lists at the same time, so if you define an item in a static list, you can have it in another list that let's say implies that the objects is dynamic and you can interact with it, but will expire if left alone for 20 minutes.

But first let me describe the complete structure of game objects:

• Raw textures. These are stored on disk and they usually have the same name as their path on disk, but without extension, so that the engine can differentiate between multiple versions of the texture used for different things, like when running low on RAM the engine might decide to reduce the texture resolution. It could do this by loading only a lower mip-map, but alternatively it could load a specially prepared texture from disc meant to be used in this scenario. Right now textures can be loaded multiple times form the same path and you will get two independent copies, but if I determine that there is no need for this, the engine could only load one path as one texture in memory and use that for all the "copies". You do not have explicit control over textures as resources in the editor because they are governed by materials.
• Materials. These are basically a structure describing how material should look and they usually have a diffuse texture and a normal map texture, but also properties relating the the strength of the material channels, specular highlights, etc. This is what you change when you need to change the way an object looks, not the raw texture objects and there is full support in the editor for this.
• Meshes. These basically describe a 3D mesh as exported by a 3D modeling program, under a neutral stance (position, scale, rotation). Example: an upright barrel.  A mesh has multiple parts, and each can have a material. In the past a mesh object used to have properties related to its scale when introduced in the world and its physics properties, but starting with this version a mesh will be size and physics agnostic, just a raw data, similar to the raw textures. The old properties are moved to a new objects, that shares the same relation with meshes as the relationship between textures and materiel.
• Object descriptors. These take a mesh and add size and physic properties to it. Let's say your exported mesh is a cylinder with dimensions (200, 450, 200). This size is resulting from the scale that the 3D artist used. It would be great if all meshes would be created with the same scale, but with an army of modelers that I'll surely have soon, this is not very likely. This size of the mesh informs its use in the descriptor, so you could choose that the descriptor would use the exact size given by the mesh. But more likely you'll choose another object. This way you could have a static barrel in an object descriptor which specifies that the barrel mesh will be used with static physics and a size of (0,8, 1.2, 08). This item descriptor will be added to a lists and then used. Or you could specify a static physics model that uniformly scales you input mesh, but to height of maximum 1.5 units. Or some ranges. Or make it dynamic and force it to a convex wrap. Or mobile cylinder with some dimensions.
• A model instance. This is an actual instance of an object descriptor, inheriting all properties form it, but this will have it location in world and other properties, like content. The entire world is made up of model instances. Their specific properties are saved to disk to enable map loading.

Seem complicated, but here is the short version: a game object is an instance of an object descriptor, which is describes a mesh with ti physics properties and size. A mesh is a 3D model that can use multiple materials, which in turn will load multiple textures. Here is a screenshot with the mesh editor for the skybox, showing multiple materials:

So now we know what resources we can have and our final goal: use this information to populate our world with model instances. But what do you do with those instances? You add them to a slot. Currently there are two slots types:

• The master model pool. This can handle anything. It can handle static meshes with one or multiple sub-parts and any shape or dynamic objects, but only with one submesh, and currently forced to become a box or a cylinder. As the need arises, I'll add spheres, convex wraps and compound shapes as a valid shape choice. The master pool can handle any number of objects in any mix and added in any order with perfect batching. Or at least how I currently understand how perfect batching should work. It frustum culls and can further cull your objects based on distance, but it does have the downside of having to go though all the objects that you added to it every time it tries to determine what to render. This fact combined with the ease it handles mobile objects makes the master pool an ideal candidate for mobile objects, which you should have less of than static entities.
• The terrain chunk pool. Actually, there is one such pool per terrain chunk and it can theoretically hold the same things that the master pool can, but I never tested with mobile objects because this is meant for static objects. If you add a static object to the game world and choose the terrain pool as destination, only the chunk that contains the object will be updated. It has perfect batching, but only for the objects inside a chunk, chunk boundaries breaking perfect batching. This pool first frustum culls the entire pool as a single test as an optimization, so if you have a terrain chunk with 10000 objects and the entire chunk is outside your view frustum, with one test all 10000 objects are eliminated. After the chunk test, all objects inside are further tested if needed and there is distance based culling. So not all terrain pools must be fully traversed like in the case of the master pool, but it doesn't handle mobile objects well.

Again, to clarify, let me give you the short story on how objects should be added to the game world. If it is static and should be cull-able, add it to the terrain chunk pool. If it is mobile, add it to master model pool. The engine has default behaviors and since the object descriptor tells it how the object is meant to be used, you generally don't need to worry or know this stuff. Just select a descriptor and do level.Add(descriptor, position) as an example. The engine will select the pools as appropriate. And will do this efficiently, with constant cost and low overhead.

This is why I called this post summer cleanup. Most of these systems existed before, but now I formalized and cleanup everything related to them and added the new object descriptor system. This is what I wanted to say, so you can stop reading now.

but bellow I'll detail parts related to the development of this system and some distance based filtering. So after a first round of cleanup and creating support for object descriptors and adding list support to the module system and adding a first item to the static list, I needed to finish the support for adding such a descriptor. I started with the foundation dropping code, but made it use the new object descriptor system:

Above you can see this work, maybe a little bit too well. The pots are the size of foundations and terrain gets deformed and its texturing changed. I strip out the unneeded parts and I compare a pot added by the old system and the one that uses the object descriptors:

Great! thinks look the same. But they are not! Before the object descriptor system you needed to give some physics properties to a mesh directly, and if you did not, it defaulted to a box. The new descriptors coexist with the old properties, but since we selected an object from the static list, their physics behavior is quite different:

The one on the left is a static pot, that wills stay there until somebody or something removes it, while the one the right is a mobile one mapped to a box which you can just push around by walking. From this screenshot I can draw two conclusions: maybe, just maybe, the pot mesh is too high poly. But for sure, the physics mesh, which can be different from the render mesh is too high poly. I'll model a new physics mesh, that will have at most half as many vertices and use it as physics impostor.

The static mesh is in the terrain chunk pool, the dynamic one is in the master pool. They both have support for different ways of culling as described earlier, but also support for distance based filtering and actions. I created this very approximate grid of pots:

I intentionally set the distance on the pool to something very low, like 20 meters. See what happens as I move back:

 

And move back more:

Object further away that the 20 meters will get culled as I move about. This partially solves the problem of very ugly pop-in as you move around. Do you remember my old video where I did a 64 square kilometer map with hundreds of thousands of physics enabled objects and I ran diagonally from one corner to another with high speeds? Well, you could hardly tell because of the YouTube blurring that distant chunks would pop in all at once when running around. As in all the hundred of objects inside the boundaries of one chunk would appear as soon as you got close enough to the chunk. This was very immersion breaking and due to the chunk single frustum pre-test. Now, if I were to repeat the same test, internally the same pop-up would happen, but the engine would only render things in a set radius around you, so you would see object pop in one by one instead of the whole chunk. And in practice, I won't set the view distance to 20 meters.

I will set a generous radius and add other methods to enlarge the radius, like omitting very small objects and using meshes with LOD levels. In the above screenshots, instead of hiding the objects, I could switch to a lower LOD mesh. If I had one. I need to model that when I sit don to do the physics impostor.

Switching to a lower LOD will still have some pop in, but it will be minor. Full chunk pop in: VERY VERY BAD. Single object pop-in: PRETTY BAD. LOD transition and/or smart billboard: pretty good. That is what the AAA are using, so it will be good enough for me.

I couldn't test LOD transition, but I could test pixel shader complexity transition. I set the view distance to something very small, like 1 meter. If I am inside the radius, everything looks as normal:

If I take just one step back, the normal mapping effect is omitted:

Seeing this screenshot I realize that I did do a fair job at modeling and texturing this simple object, but I did a horrible job of normal mapping it. The pot looks a lot better without the normal map. I'll have to redo it.

Anyway, the change is very jarring. but as you increase the radius, the difference becomes less apparent, like at 5 meters:

At 10 meters, the change is barely noticeable:

So if I set a larger radius, the change won't be noticeable. But this won't give you any great performance benefit. The object will be only a few pixels wide and you will save a few pixel shaders instructions which in the long run will barely affect your framerate. The way you use this to actually get some benefit is use it to count the uses of the normal map based on the radius. If it is zero for a given raw texture, you can unload the texture and save on VRAM. As an example, if your object is so small that from 200 meters you can't tell the difference between high LOD and normal mapping and low LOD and no normal mapping, you make it so that at 220 meters it unloads the normal map and maybe replaces the full size diffuse texture with only 64x64 mip. The 20 meters extra units gives you a little bit of leeway if the texture streamer is very busy right now. It gives you a better change that by the time you reach the 200 meters mark , it is done streaming in the previously unloaded textures. And if it won't quite make it until 180 meters, you still won't notice the change, only when you are really looking for it.

Screens of the day 35 - Yeah, I can see why people hire graphics artists

Today's post will detail in relatively chronological order the last development session and updates resulting from it and another task I did in parallel: model a brand new metal cooking pot! As I said last time, modelling turned out to be very expensive even with such simple meshes that I need, so I'll have to supplement the outsourced meshes with my own or risk to be stalled development wise, waiting on meshes. I'll try to add two meshes per week, one outsourced, one made by me.

I started by updating the engine boot up process and now for the first time, if the engine can't find the bare minimum resources to run, it will fail gracefully and give you a message box, instead of crashing.

Then I modeled the metal pot. After I was done with a fair looking version, I introduced it into the engine. This helped me test the GUI in a real life scenario and I fixed a bunch of bugs and made minor improvements. Then I updated the importer and made it much more flexible. If your input mesh has position, normals and UV coordinates, it can be imported. The missing parameters can be generated. The generation was done in the past, but the engine would still refuse meshes that didn't have all the necessary attributes, even if they would be regenerated as soon as the import was done.

This is how the mesh looked in game:

The general shape is good, but the shading is not great. I experimented some with other kinds of shading:

The handles here have flat shading. Then I tried to add some loop cuts to the handles and use smooth shading, but the results are disappointing, yet the model is more complex:

Then I textured it:

OK, maybe it is too large:

Finally, I went with a counter-intuitive shading scheme that still gave better results, but they are based on compromise:

Here is a larger and more quality render:

OK, I"m done with it!

Only kidding. There is a lot more to go. I tried some AO:

I tried another texture and experimented further with AO:

OK, this is the final result:

I also updated the model editor with a zoom bar. I still need to make it auto-detect a reasonable zoom level, but it works manually well: 

And I updated the category selection options with a new filter for clutter objects.

The updated model editor is a good tool because it helped me track down some bugs I introduced in rendering in the last snapshot. 

I tried to make the mesh import completely dynamic, without the need of XNA projects, but this turned out to be problematic dependency wise. I'm afraid the mesh importer will need to be an external windows application used only to import meshes. After this is done you can use the in-game editors for everything else.

Next step will be to implement static persistence and then use the new meshes that I have for some static cluttering. While most physics entities will be dynamic, some will remain static. Most columns will actually be completely static, and maybe even pots on top of fireplaces.

Tworenas Snapshot 08

Is out! On time this time!

It contains the fixed up column texturing:

And I thought that I'd start doing art asset videos:

OK, maybe next time I'll do it without v-sync on because the recording dropped the framerate to 30 FPS and movement is jerky.

For art assets I am working with someone so I expect regular new assets to be introduced. On the other hand, he is more expensive than I would like, so I'll also start to regularly model new assets. This is to speed up asset creation, with two people working on it. AS a secondary reason, it is a personal test to see how I hold up to the professionals.

This snapshot has largely the features that I spoke about in the last post, except that I figured out that I'll need to add static persistence soon, so I did dome major enhancements to saving/loading and the map format. I wanted to explain in detail, but I'm really not in the mood today to talk about binary formats and buffer offsets, but someday I'll give the exact format of the saved games.

BY AGREEING TO DOWNLOAD THIS SOFTWARE, YOU TAKE TOTAL RESPONSIBILITY FOR ANY CONSEQUENCE RESULTING FROM THE USE OR MISUSE OF THE SOFTWARE. I AM NOT TO BE HELD RESPONSIBLE FOR ANY POSSIBLE HARM THAT CAN BE CAUSED BY THE SOFTWARE NOR DO I GUARANTEE THAT IT IS FIT FOR ANY PARTICULAR PURPOSE OR THAT THE INTEGRITY OF THE SOFTWARE WILL BE PRESERVED WHILE BEING TRANSFERRED ON ANY AND ALL MEANS OF COMMUNICATION.

ALL RESOURCES INCLUDED FALL UNDER MY COPYRIGHT AND ARE ORIGINAL CREATIONS, EXCEPT FOR THE TERRAIN, GRASS AND SKYBOX TEXTURES, WHICH WILL BE REPLACED SOON WITH ORIGINAL OR OWNED CREATIONS. FULL LIST OF BORROWED ASSETS:

• data\textures\bush: grass01.c.dds, grass02.c.dds, grass03.c.dds, grass04.c.dds
• data\textures\terrain: region1.c.dds, region1.n.dds, region2.c.dds, region2.n.dds, region3.c.dds, region3.n.dds, region4.c.dds, region4.n.dds
• data\textures\skybox: clouds-back.c.dds, clouds-front.c.dds, clouds-left.c.dds, clouds-right.c.dds, clouds-top.c.dds

Link: Snapshot 008


Changelog:

• New version based save format with sanity checks and meant to be robust on mod change.
• Full player stats saved. Stats are cheeked for availability and if a non existing skill is found, the character is issued a skill refund.
• Day-night cycle saved.
• Saves moved to "My Documents\My Games\Tworenas" or "Documents\My Games\Tworenas" or whatever your personal documents folder is inside your user folder.
• Eliminated last remnants of the previous singleton terrain: there is no longer a global maximal terrain height stored. Each terrain holds this data now separatelly.
• Light-shaft strength fades with day-night cycle.

107 – M-m-m-m-m-m-multi Mod

Hey, I'm done with Snapshot 8. I'll package it after I write this post and I'll start working on Snapshot 9 tomorrow. I need to have something cool  for snapshot 10, the half point of this development cycle. But I'm not making it available today, it is only Saturday.

So, new features: multiple mod support! Take a look at the new editors:

Tee new design language and background structure intended for code reuse is at work here, but the features are the things worth noticing. First, we have a few buttons for filtering the content. Right now we have only a handful of items so it is not that useful, but once we'll have hundreds, this will help navigate the list. The "MSC" filter is already very useful, since when I turn it off I am left only with 3 items, 2 furniture pieces and one container, getting rid of terrain, skybox and grass, things you will rarely edit. The list of filter buttons is closed, but I'll add all the useful ones.

Then we have the list that shows first what mod the item belongs to. There is now support for adding an item to a given mod, renaming, removing and the disabled button is used for moving items from one mod to another, but it is not done yet.

On the right side of the material window you have a redesigned panel, this time only with two textures: diffuse and normal. The specular map is gone to save video memory and sampler calls, and there is an always one check box that allows you to use the alpha channel of the diffuse map for the specular map. I don't see a good scenario for having it turned off, but it is there for foresight.

The models window got the same treatment, but the right side panel is still underdeveloped.

To stress test the mod system, from now on all new content will come in separate mods. The final product I want to ship will of course feature a singular mod, but until then the list will just grow.

So from today, there are two mods available out of the box, "main" and the new one, "column". Column just defines a material and mesh for an asset that you may saw before, but it is newly available, the column:

This model was created by BrewStew but was never textured, so now it is textured. The look is not quite final and in the snapshot you'll get a texture with a bit of AO on top of it and some more tweaking.

The editors are very important, but they are not gameplay editors. One major part of gameplay will be building, and these are not building gameplay related editors. You need to have materials and meshes, but gameplay wise it is the rules that count, not the assets. You could theoretically use only colored boxes and gameplay rules to achieve similar results. The skill editor can be considered a gameplay editor, but next week the first building related editor will be introduced, the third major editor: the list editor. Most rules can be described as lists.

The first list has been added to game already, but there is no GUI for it: the grass list. Using this list you can control the meshes that you want to use for grass. A rudimentary list editor will come next week, allowing you to change the grass mesh and introduce the first major list: the containers. An item in there will allow you to choose a mesh that will be used as a container, together with what size should it have (independent from the original mesh size/scale), what physics shape it can have, how many items can it hold, the maximum weight, what kind of items it will accept, etc. Using this list I'll add the first two real containers, a usable barrel and a chest and implement player inventory with moving items from the container to the player and back. Heck, if I have time, I'll introduce a cooking pot too, where you can only add food. Not promising anything on the cooking pot. And it won't be functional for cooking.

I also changed the terrain response to dropping a foundation with some smoothing. An original terrain like this:

Will become like this:

The shape is a little bit more natural and rounded:

You'll see it sometimes in a new video, when I repair and reintroduce house body building.

I don't know if I ever mentioned the goals for the 20 week development period, but one of them was to take any idea or primordial feature I have shown of before, and if it was a good idea, no mater how half assed and non-viable the initial implementation was, to turn it full featured, stable and integrate it smoothly into the engine in the hopes that what will turn out will begin to resemble a real game.

So, next feature. Hmmmm. Ah yes! Remember how I said that the alpha channel will be used for speculator maps? I updated the shaders for this. In the process, I created a new shader file and copied over all the good content, while leaving out all the bloat. The new leaner shader is a lot easier to edit, so I added a bunch of new debug modes to make sure all the assets are on par. This is the raw textured debug mode, perfect for noticing seams in the texturing:

This is the raw normal map debug mode, perfect for noticing seams in the normal map:

And this one is the raw debug mode for speculator mapping:

It is working properly, trust me. The reason you see white is that the tools I am using are really shit and are bad at creating mip-mapped files with an alpha channel. I had to manually assign an alpha channel to each mip level and I'm looking for better tools:

Ah, pipeline woes!

Tworenas Snapshot 7

Here it is, Snapshot 7, late as usual. I was not able to get it out on Monday, but for snapshot 8 I'll do a feature freeze on Friday, package it and on Saturday I'll start working on Snapshot 9, so I hope that next time thing will be out on schedule. Anyway, the fixed release schedule is just for me, so that I can get used to some release discipline.

The first big feature that nobody will notice was the finishing of the Level class. This is pretty much the last step in my major internal structure refactoring. There is no such thing as a perfect design, but I'll get a great millage out of this one: generic renderer, game specific renderer, standalone terrain and grass components, level, rule set and modules. The introduction of the Level class was very disruptive and I spent a couple of hours fixing null pointer exceptions. Hopefully I fixed them all.

For GUI, I introduced a new set of conventions, both in code and GUI design. I'm going with a more compact and panel based representation, inspired by some other game engines. Here is a sample:

Looks about the same, but in the background things are different. Until snapshot 10 I need to finish both the refactoring and the fundamental mod GUI.

Then I uploaded this video:

I don't think that I'll go with the dropping from the sky them. I'll try to go with an organic build/progressive update scheme, and if this does not work, I'll try the Assasssin's Creed III Animus Simulation fade in. Anyway, most of the systems will be the same, so I had to fix this. The foundation button has been disabled for quite a few snapshots. That was no placeholder, dropping and terrain deformation worked, only it was far too slow. Setting the foundation is extremely difficult and time consuming, both as development and CPU time. You need to:

• Determine if this is a good place to drop. Currently, I'm always dropping, but there are some places where it is not a good idea, like on an almost vertical wall or on a patch of terrain with huge height differences. You need to sample the terrain in a wide radius and determine somehow how disruptive the terrain change would be and reject spots above some threshold.
• Deform the terrain. Very time consuming on such a large area. I''ll try to animate it in the future, not just for the looks, but for spreading out the task over time. The deformation is particularly hard because of chunked nature of terrain and the myriad of caches and pools it uses. They all must be updated, or else you get memory leaks or the terrain will disappear under you feet.
• Remove the grass. Again, same story: chunks and caches.
• Do something about objects that are under the falling platform. I did not do anything with this. In the future they will be destroyed.
• Do something with objects near the point the impact that don't get crushed. I did not do anything with this. I'll apply some explosion like effect, making the objects get thrown in a direction based on their angle to the point of impact, simulating a shock-wave.

So it is hard and I'm not done. The last two points are not implemented. And you also get ugly results when putting platforms very close one to another. And you can drop them on your head. It won't kill you, but It might get you stuck in the terrain. So still a lot to do.

Oh, and persistence. When loading the map the platforms should be still there. Unfortunately, I did not have enough time for persistence. I did not want to half ass it.

BY AGREEING TO DOWNLOAD THIS SOFTWARE, YOU TAKE TOTAL RESPONSIBILITY FOR ANY CONSEQUENCE RESULTING FROM THE USE OR MISUSE OF THE SOFTWARE. I AM NOT TO BE HELD RESPONSIBLE FOR ANY POSSIBLE HARM THAT CAN BE CAUSED BY THE SOFTWARE NOR DO I GUARANTEE THAT IT IS FIT FOR ANY PARTICULAR PURPOSE OR THAT THE INTEGRITY OF THE SOFTWARE WILL BE PRESERVED WHILE BEING TRANSFERRED ON ANY AND ALL MEANS OF COMMUNICATION.

Link: Snapshot 7

Tworenas Snapshot 6

I am just realizing it right now: am I repeating the unfortunate naming incident of the planet Uranus? How do you native English speaking folks pronounce "tworenas". Is there any hidden "anus" in it?

So, snapshot 6 is out! Like I mentioned, this is half bug fixes, half graphics.

I made a ton of screenshot detailing the performance of the engine with the new tweaks, but I realized this is no place to show them all. So in words: on an Intel HD chip, performance was bad at 640x480: 30 fps. So I optimized the new effects and got a good improvement, but performance is still between 30 and 39 fps with light shafts:

So I added an option to turn them off:

Still, this did not greatly increase performance, The high quality terrain  terrain with multiple light support is a performance hog. So I went in and wrote the new multi light support shaders for low quality. High:

Low:

This helped a bit, but even the low quality is heavy because of the light processing, so I added a special variant to the shaders that is used when no lights are present and this shader is used on the appropriate cells of terrain.

So eventually I managed to squeeze out around 60 fps with light shafts:

Now, the default settings are on high and 1280p. I did some testing and setting them to low while maintaining resolution and light shafts gives you between 32 and 45 fps. Not ideal, but this is Intel onboard. Dedicated chips have an easier time. I can run the engine at 200-400 fps :).

Snapshot 7 or 8 might bring something really interesting, but I'm not sure I can pull it off yet, so not telling you what it is or making any promises.

BY AGREEING TO DOWNLOAD THIS SOFTWARE, YOU TAKE TOTAL RESPONSIBILITY FOR ANY CONSEQUENCE RESULTING FROM THE USE OR MISUSE OF THE SOFTWARE. I AM NOT TO BE HELD RESPONSIBLE FOR ANY POSSIBLE HARM THAT CAN BE CAUSED BY THE SOFTWARE NOR DO I GUARANTEE THAT IT IS FIT FOR ANY PARTICULAR PURPOSE OR THAT THE INTEGRITY OF THE SOFTWARE WILL BE PRESERVED WHILE BEING TRANSFERRED ON ANY AND ALL MEANS OF COMMUNICATION.

Link:  Snapshot 6

Changelog:

• Extended the skill tree and introduced the skill editor.
• Skill choices are now saved. (There will be a new option in the world manager to reset the skill points.)
• Grass system optimized to use advanced pools and caches while avoiding C# GC.
• Grass looks updated, with slope coloring and dynamic light support, with a maximum of 5 lights/cell.
• New Renderer component to isolate rendering from game logic.
• New ModRuleSet component to isolate mods and game rules from the rest of the game logic.
• New Grass class that abstract away the entire grass system into a simple to use component. The Grass class is part of the game library.
• Entire rendering and content management system refactored!
• Light shaft!
• Debug visualization support for vertex normals, pixel normals, tangents, binormals, depth.
• Mesh tangent seam correction at all UV seam included into to pipeline.
• Low quality terrain profile is back. Medium is still M.I.A.

Bugfixes:

• Fixed so incorrect tangents all over the place. In consequence, normal mapping pops more than usual and no longer changes direction based on angle.
• Some stability fixes on device change
• Grass lighting bugs fixed. Grass light can still sometimes be just a bit too neony, but this is not a bug, just a light scaling factor that is too high in the rendering profile. I will properly scale the lights when a GUI is introduced so you can control live the strength of illumination for the entire world.
• Fixed a major and quite old terrain bug. How tor reproduce: raise or lower terrain near your feet, then walk so far away that you can't see the spot you edited anymore. I mean that the spot should be outside of the fog barrier. Then walk more, to be absolutely sure that that region of the terrain has been removed from RAM and all caches. Then return to the original position and notice that a chunk of terrain around the spot you edited is invisible. You cans till walk on it and edit it as usual, and the changes are saved, the chunk is just invisible. Reloading the save fixes this. Because of the huge distance you needed to traverse from  the edited piece of terrain and return, I did not notice this bug before. The fix was one line of code.

106 – Trinity

The plan for snapshot 6 did not go 100% as planned. I wanted to do a full bug-fixing run, but halfway though it tuned into a full graphics and graphics bug fixing run. Anyway, the changes are so substantial that I'm bumping up the version number in snapshot 6.

The first reason for the version bump is major internal refactoring that while only about 60% done, it has already payed off in spades. As the project size increases it becomes harder and harder to maintain. Back in the day I split the project in two, a general purpose game library and the actual game and this helped a lot to maintain things under control. But lately, the Game class was getting too large. Splitting it up into a hierarchy of classes only helped spread the complexity over multiple files, not reduce it.

So I introduced the holly trinity:

• The Rederer. This component babysits all the almost 20 classes responsible for pushing pixels to the screen and takes care of loading resources, render targets, rendering profiles, post-processing, etc. Delegating all the rendering tasks to this class helps make the game render engine agnostic. The migration will be done in two phases. Phase one is already done and consists of creating the class and making it sure that it offers all the rendering services. The Game class uses these services to render the world. For phase two, I will inherit from the Renderer class and create a child class that renders my world using the services of the parent, thus achieving true decoupling between game and rendering. Phase two is not done yet.
• The ModRuleSet. This is a collection of modules and rules that define how a world is set up and what assets can be used to build what. You give it the list of modules that you want and it will try to merge them into a master rule set. It also handles loading materials and other assets that might be needed. And it also includes some system variables, like the way the wold looks under lighting, how long the day is, what is the run speed, etc. While I have no intention to vary these from world to world, the engine is completely capable of it. The ModRuleSet is fully implemented.
• The Level. I named it level to be more generic, but it is actually a world. This class holds the real data related to a world, like your character, terrain, grass, placed objects physics, etc. Think of an instance of ModRuleSet as the generic class for a world, and an instance of Level being a specific instance of that class. The Level class also handles loading and saving. This class is not yet fully implemented.

So with these classes, the Game class need to instantiate a Renderer and tell it in broad strokes what it wants to do, then you create a ModRuleSet and give it a list of modules and finally you load a Level from disk. And you tell these classes do their thing! And they do. Now, the Game class is only responsible of input, GUI, managing your interaction with the world and the game mechanics.

The second reason for the version bump is the major improvement in mesh tangent space continuity and everything that depends on this.

Implementation wise, I have three problems with my lighting scheme. First sometimes the normal mapping effect gets inverted and looks very strange. This is particularly bad with the barrel and you can see this in a lot of screenshots and videos. Heck, you can see it in some videos I did in other engines using the same assets, which lead me to believe that maybe not my shaders are to blame. And you can see it in last post's screenshots. Anyway, I'm getting ahead of myself. Second, there is nasty bug with point light attenuation. I have a fix for this which I used since day one, but it is hack. I would like to find a real solution for it. Third, there are some nasty seams in the meshes where the UV mapping continuity breaks. These seams combined with normal mapping create very ugly results. This is partially due to the textures not being seamless, but also there is something else wrong. Something very bad!

So I went forth and tried to fix as much as possible. The new debug mode visualizations helped a lot. Also, I started being active on the GameDev forums (mostly asking for help :P ). Using the vertex normal visualization I determined that vertex normals are correct on the meshes. Pixel normals are not, but the normal map is fine. You compute the pixel normals using the normal, tangent and binormal of the vertex, and since the bi-normal is computed, it was a safe bet to try and inspect the tangents:

Bingo! You can't see from this angle, but the barrel also has a discontinuity in the tangent space on the back side. These meshed have some craptastic tangents.

I googled it and I found dozens of threads on several forums. It seem that having normal mapping seams is an extremely common problem and those forums were mostly without a proper solution. The problem is caused by the way 3D modeling programs have a continuous vertex list across smoothing groups, even if there is a break in the UV mapping continuity. If you export this mesh without tangents, since the vertices at UV continuity breaks will be saved as two different vertices, with the same normals (if the exporter is any good) but with two different UV coordinates. Then when you compute the tangents, you get this result. Most formats do not get exported with tangents. So problems abound. Ideally you would export normals, tangents and binormals, to make sure that the mesh looks identical in your renderer and the 3D modeling program. These bad tangents were the reason why I got the same problems with the rings across different XNA engines.

The first step was to get rid of the crappy computed values provided by XNA and compute my own:

The seam is still there of course. And it will remain while there is an UV continuity is there. It is about taking control away from XNA when generating tangents and taking over that. Special care can be had when modelling the mesh to avoid such problems, but it is not always possible. And I'm a programmer, not a modeler. So now I'm in control of the tangent generation. Let's improve this generation a little:

Tangents are now just a little bit smother. But besides the obvious seam, the tangents should be theoretically correct, so maybe they can help with my barrel ring normal mapping problem. Let's find out!

Before:

After:

OK, take a look at those rings! I tested from different angles and the rings now extrude the right way. The tangents finally point the right way! I've had this problem for over a year without knowing it. Here is a closeup of the normal mapping with correct tangents:

The normals can still look a little bit weird, especially around the rings, where it seems that the wood is popping out, not the metal, but this is due to my poor auto-generated normal maps. With some proper normal maps, this can be fixed.

So let's see what we did. We fixed the tangent space and thus fixed the normal mapping issues. This has done nothing for the seams nor for the attenuation bug.

I have no idea how to fix the attenuation bug, but I write a code to detect the seams and a debug visualization mode to render the results:

This might look like the normal render of the tangents with the seams, but it actually highlights the seam that it detected procedurally. Now that I know where the seam is, I try to fix it. The fix is not 100% mathematically sound and I did pull some epsilon values in some comparisons straight out of my ass, but it seems to work just fine. I won't upload the full bunch of screenshots that I have, they can be found on the forum. The major very annoying seam is gone and I am left with a minor seam. Let's study this minor seam:

It can be seen if you look for it, but otherwise you probably won't notice it. The old seam was dire an immediately noticeable. But what causes it? Is it the textures that are not seamless or the fix is not good enough? Well, this is the corrected tangent view of the mesh:

Looks pretty smooth to me. Maybe in the future the seam fixing algorithm can do a better job, but I'm happy for now with it. Here is the same shot only with the normal mapping effect:

A minor seam, at least partially due to the texture work. And now the diffuse texture:

OK, I think it is clear that most of the seam is caused by the diffuse texture. If the diffuse texture and normal map get fixed, I think the seam will go away now that tangents are proper. I still need to test my algorithm on more complex meshes.

So to reiterate, fixing the tangent space fixed normal mapping direction, except on the seams. And now I fixed the seam. All imported meshes get their seams automatically corrected.

This is definitely the high point of graphics correctness in my engine! I'm looking forward to the day I'm done with the graphics once and for all.

But not today!

Another thing I tried is adding light shafts:

I based my implementation on J. Coluna's method, but I had to heavily modify it to get it to work and my results are not as good as his. Still, not bad. Even with that post and the working code in front of me, it took me almost 6 hours to get this thing to work.

So lighting is mostly fixed and we have light shafts. What next? I tried my hand at SSAO, but this one did not turn out that good:

It is dark, grainy, full of artifacts, there is an ugly fixed pattern on the screen that is very disturbing when moving the camera around and the distant lands look even worse.

I did try and fix this and came up with possibly the most ghetto ass SSAO implementation out there:

Needless to say, I don't like how it looks and I won't be including it in snapshot 6.