101 – Snapshot 1 almost done

I order to make good progress on the snapshots, I am trying to give each one a theme or some major feature to keep me focused. It is very easy to go all over the place, do a lot there, but end up with nothing to show.

The theme for the first two snapshots (at least) is "a mad scramble to get the basics done". Snapshot 0 in particular was super mad, trying to get a bare minimum of systems working good and giving the engine a major smoothness overhaul. Snapshot 1 shares this theme, with again a major focus on smoothness. I'm not expecting any downloads on the snapshots, each one being more like a personal milestone, but if somebody does download one of the early snapshots, I want them to say "This has barely any features". I don't want them to say "This is an unpolished pile of shit". In the above scenario, that person has above average objectivity, because this is the Internet after all. No mater what quality you achieve, people are going to hate.

What I'm trying to say is that each snapshots needs to have one fairly polished and very smooth feature rather than a bunch of small incremental changes to 5 or more features. For the first two snapshot this feature was the terrain.

And I achieved major speed up on everything related to terrain. The change is so significant like replacing a 3 year old computer with a new modern one.

On top of snapshot 0's changes, I did a ton of new optimizations that resulted in added smoothness. But the major improvement was moving the tangent and binormal calculations over to the GPU. Not having to update these on the CPU has freed up a ton of resources. One small caveat is that the method by which tangents are computed is different, thus the terrain is just a little bit brighter. And I have encountered exactly one visual artifact. This can't be true and I'm sure there a tons of such smaller artifacts, yet this one is the only one I managed to find. Before screenshot:

After:

So the artifact is minor and I kind of like it. Makes the terrain just a little bit less ordered there.

Another thing I changed is the quality levels of rendering. I had a slightly outlandish and somewhat more polarizing shader set in use to render terrain. Initially it was a response to normal mapping not being strong enough, but eventually it got enhanced with a ton of stuff, like spherical harmonics and fancy detail mapping, so it kind of lost its original goal. Also, in the meantime I grew fond of a more traditional look for terrains. So I did a "back to basics" on the shaders, removed some of the bells and whistles and came up with a more comprehensive set of 3 quality levels: low, medium and high. The previous 5 were too much, especially since they defied a simple naming scheme. Quality 4 was not lower in quality as Quality 5, but it was faster. Now low has no normal mapping, medium has normal mapping and high has some extra effects for better contrast.

Low:

Medium:

High:

Now that the world is smooth enough, one question remains? How do you get a world? I could package a small world inside the snapshot. But that is just one world, plus worlds are really big. I can't put out weekly  300+ MiB package. Eventually snapshots will get over 1 GiB and by then I'll need to create an incrementally updating patch system. But I am getting ahead of myself. So I need a way to manage worlds. Introducing, the world manager:

I decided to go with a persistent world set. A world remains in your list until you explicitly delete it. But you can do anything with that world without deleting it, including deleting all the world content, regenerating it, doing major automated editing on it. So there is a major difference between deleting a world and deleting the data inside it. If you delete the world, it is gone for good and without a trace, together with you character, skills and inventory. Deleting the data just gets rid of your maps, so you can adventure on a new set of maps. Your character, inventory and anything that you have acquired remains. It is like moving to a new area of the world. In the future this might even be implemented and a world might have any number of regions.

The world manger can be brought up on its own or when exploring a loaded map and can load a new map asynchronously:

After first creating a world, or after you cleared the data from it, you are given an option to generate a new terrain:

The import option is disabled and not implemented, but it will allow you to load a heightmap bitmap in a number of popular formats in order to create a map. I will add support for normal grayscale bitmaps and also for floating point textures, in raw format of standard ones.

Map generation takes ages so it is good that it is asynchronous. OK, maybe not ages, but it takes a lot of time. My final hopes for this engine includes very complex procedural worlds and that might take up to 20 minutes of generation time for large ones. Currently the generation is slow because diamond-square algorithm is slow. I need to dig up my old Perlin implementation and use MT with it.

The world manager is mostly done, yet feature poor. It will evolve over time.

The main problem right now is that I am having some major memory leak issues.

Anyway, snapshot 1 is mostly done. I need to test it until Monday and maybe I can even get rid of some of the memory leaks. Snapshot 1 is going to buggy as hell, maybe even crash, but it is stable enough given the time frame.

On Monday I'll post again with the download link.

The theme for snapshot 2 is going to be "RPG". Snapshot 2 will introduce the concept of channel rate, mana, overcast, stamina, exhaustion, rest, a fully functioning (but for now quite small) skill tree, player stats and two implemented skills.

Changelog:

• World manager!
• Tangents and binormals are now computed in the vertex shader. This reduced memory consumption and greatly improved terrain editing and loading snappiness. A few minor visual changes resulted from this.
• With the new performance levels, terrain chunk streaming wait time has been reduced 10 times, making streaming far more responsive while keeping walking at all speeds snag free. On very strong computers, the wait time could be reduced, halved or even quartered.
• The number of terrain quality levels have been reduced from 5 to a more manageable and easier to understand 3. These quality levels avoid the more outlandish shaders and give a pretty "standard" look to the terrain.
• Massive speedup of map loading from disk. On my computer, for a 64 square kilometer map, it went down form 2.7 seconds to about 0.5 seconds. Reading floats from disk in C# is slow. Who knew? Still, physics creation in memory was yet not improved and it still takes about 2.5 seconds, so 3 seconds for the first load. After the first load, live streaming kicks in so you don't need to wait for load times.
• Using the above trick and applying it to terrain texturing streaming, I improved chunk stream in time from about 4.0 ms to 3.4.
• The chunk allocator is now capable of pre-initializing the buffers that will be in us, so every time a chunk that was in use before is reused, some computations can be skipped. And with an additional optimization round for those calculations, I reduced the average chunk load time from 3.4 to 2.8 ms.
• Changed the way MSAA is set up. It should give a minor performance gain for all levels of MSAA. If you are just about at the minimum specs to run the game without MSAA, this boost might be enough to allow you to run at around 60 FPS with 2x and rarely 4x MSAA.
• The game window restores mouse position after returning from another window.

Caveats:

• C003: Partially fixed, since stream delay time has been greatly reduced and chunk load speed increased.
• C004: Partially fixed, again by the improvements that helped C003. As a new addition, the streamer is now partially biased towards the camera position and will first try to bring in a set radius around the camera and then proceed to go from corner to corner. The radius is not yet large enough to cover the entire streaming area. Need to be made as big and ten I need to add frustum biasing. Once these two things are done, this caveat is fully solved.

Bugfixes:

• B001: Turning on physics debug information at he same time with SMAA causes render to fail. Only an one pixel thin screen diagonal gets some color rendered. Temporary fix: turn off SMAA when using physics debug information. Fixed. Was rendering the SMAA full screen quad in wireframe because debug physics sets the render mode to wireframe.

Bugs:

• B003: Massive memory leaks in the new word manager. Every time you use it to load or generate a new world, some resources remain in use. Eventually you will run out of RAM.
• B004: Loading a map right after generating one is inexplicably slow. A second load, loading a map generated some while ago or loading after a game restart loads at normal speed.