25 – That’s no stone explosion… that’s a spoon!

This is a stone explosion!

I totally underestimated the complexity of stones and walls. Adding all the stone types in the editor with layering information, preparing the graphics and doing the coding took forever and a day. I also took two days off.

More data added to stones in the editor

The current count is 91. This includes 4 types of soil and 5 types of sand. The number is not going to increase, but maybe it will decrease. There is now detailed layering information available for stones.

Stone layering implemented

There is now a basic stone layering implemented. It is not as smart as the data available for it in the editor, but it is enough for now. Layers are randomly selected and they have pockets of different stones. There is also a top layer of soil, which is also randomly selected. The soil cuts into stone forming a nice pattern. It is hard to find a representative image that gives the same impression as having the game in front of you and scrolling the map, but I’ll try.

Stone cutting added

All new feature. Select a number of stones and select what you want to carve out of them. While masons need a workshop for doing fine stonework, they can still go out and cut out a huge stone where it stands. Hauling such stones would be out of the question anyway. This is a city builder, not a give-hernia-to-all-your-dwarves simulator. You can carve the following things: fortifications, tables, chairs, cabinets, chests, columns and of course, the most difficult but valuable result: statues. Contrast could be improved for some types of stone, but here is a sample:

New options for floors

In the last version I added the tasks of smoothing or engraving walls. Now, the same is available for floors.

Interface tweaks

As you probably saw in the screenshots, the interface is changing. I am going to focus more on task categories which open up additional options. Here are the menus for walls and floor, which also indicate the number of items on which the operation can be performed:

Skills overhaul

I may have touched upon the number of skills in the past and gave my fair assessment of the situation. Quoting myself: “that's a $%^$^$@@@@!!# lot of skills”. But reducing the number of skills is not that easy. To solve this problem, I have taken two steps. The first one is to remove all skills that are not used right now. This is a temporary solution at best, because as new features are added the skills will come back and I’ll end up where I started. On the other hand, this way I can make sure no token skills are added. The second step is to combine a few skills into one where it makes sense, but keeping their associated labors as separate actions. So gem setting is out as a skill and has been merged with engraver. Those guys did not do that much anyway and it makes sense for engravers to sometimes use gems in their field of work. So they will now handle all gen setting related tasks. Gem cutting is still a separate skill. Herbalists now take care of both plant gathering and plant growing, including farming. Farmers are just less adventurous herbalists. Butchers now take care of wish cleaning also, not just land animals. Here is the new skill dialog. As tasks get added, there will be more and more options, but less than 89 skills:

U++/Irlicht bridge changes

My U++/Irrlicht bridge has served me well but I grew tired of maintaining it. Every time I needed a new control from Irrlicht I had to write a new wrapper and make sure it has the same interface as U++. So I removed the heavy wrapper and replaced it with a very simple one. This means that now I’m using Irrlicht API for GUI. The bridge now only adds callback support to Irrlicht controls. I would rather give up completely on Irrlicht than use the painfully outdated and just painful ID system.

And plenty more

There are a lot of extra tweaks that I am not going to describe here. Both small stuff and a huge freaking memory corruption bug that took me 4 hours to fix! I am too lazy to do full statistics each post.

6205 total lines of code

Ready fo ma vid

So now I think I have all the features in place for the video I promised last time after adding some features. So I’ll get on to that.

24 – Random stone explosion

A lot has happened since last post.

I managed to get in contact with the Stonesense crew and they gave their permission for me to use the tiles. While as time goes by it is becoming more and more apparent that 3D is the way to go, until that point it is good to know that I don’t have to worry about getting sued! All joking aside, I would like to thank them and wish them the best.

Also, special thanks to Jadael, who hooked me up with a sizeable set of graphics for all kinds of stones and stone objects. Now, I can’t let these resources go to waste, even if this means that I need to change my schedule a little.

And even though I said that I shouldn’t rewrite existing systems anymore, I really needed a new item system. So, on with the show!

New item system

I have a new item system that is faster, cleaner, more elegant, general and powerful that ever. The old was quite cumbersome and the new system also has the advantage of being a fully OOP system. So is it possible that I could have a new system that has only huge advantages and no disadvantages when compared to the old one? Well, if I was an idiot the first time maybe. Unfortunately I wasn’t, and as is often the case, you need to compromise and the gains of the new system come at the price of more memory consumption. Don’t worry, it is still under control and you will be able to run on a wide range of machines.

Another side effect is that workshops are broken, but they were placeholder anyway and they’ll be back.

Stone added to the editor

The editor has a new tab for stones. I added 80 kinds of stones, all with unique graphics. I’ll take some time and finish the basic stone system before I move on with what I had scheduled.

Maps now use stones randomly

Well, if stones are present in the editor, they must be present in the world too, right? For now, they are just randomly placed, but by next version I should have some basic layering done.

Stones can be smoothed and then engraved

That’s right! Two new actions for your dwarves! They are not that useful yet, but once you get the ability to carve out furniture (next post), you will be almost at the point where you can create a real fortress, not just an empty tunnel in the ground. All smoothed and all engraved stone kinds have unique graphics.

Plants have seeds

The plant editor has some extra fields, where you can edit what plants have seeds, how they are called and if they are available for purchase by new expeditions.

Expedition supplies customization dialog extended

The new expedition supplies customization dialog has been extended with seeds, but the big new feature is that it now uses the new item system allowing for more control and it now also track and groups together identical items, giving you a better overview.

Better cell inspect feature

Again, thanks to the new item system, the little panel that shows you what is under your mouse just got a lot better. Let me show you a montage of different information it can provide:

New mining scheduling algorithm

This new algorithm is quite experimental, but the old one had one big problem. If you would designate an area with unreachable tiles, these tiles would be removed from your designation and thus ignored. Not cool. The new system determines if the tile is reachable, and if not, it ignores it only for the moment. As soon as it becomes reachable, the dwarves will carry on with their given task. If the system proves itself a success, I will apply it to all tasks, not just mining.

Things to come

For my next post, I’ll do my best to give the final polish to some existing features, add stone layering and stone carving and create a tutorial video on YouTube, which will walk you thought the beginning phase of a new game and the existing features. Existing videos are old and not quite representative anymore.

23 – Going pro

In an attempt o not bore myself or my very few readers, I will be periodically switching format over here.  I changed the layout and theme of the blog, but the bigger change is going to be in format. Up to this moment, my posts were free format free flowing block of text. Starting with this post, I’ll try to organize it into sections. This way it going to be easier to figure out what the post is about and will also serve as a quick change log.

New world creation dialog

When selecting a new game, users will be greeted by a dialog that will allow them to select a few details, like the general layout and content of their respective slice of the world, the size of the slice, biome, alignment and savagery of the region. Item properties of wet and dry are ignored for now.

This dialog is quite ugly, but this is intentional. A massive user interface overhaul is planned, and any effort to align stuff, decorate or make it beautiful would be a waste until then. So I am going to focus on quick, sparse and to the point dialogs.

This dialog is the first one in a series that will allow you to customize your world and expedition parameters. The next one is the “Expedition customizer” dialog.

Expedition customization dialog is back

In case you were wondering why you haven’t seen this dialog since my third post, this was one of the first pieces of code I have written and was well before the Irrlicht transition and ever since it has been dead code. But now it is back and converted to Irrlicht.

The conversion was quick and painless, with very few features lost, but now it is ugly and doesn’t really fit on the screen unless I increase the resolution. It will have to do for now. I’ll have to rethink the way skills work, an especially the large number of them. I’ll have to condense it as much as possible without losing depth.

Expedition supplies dialog is back

Yet another dialog back from the dead? Well, zombies are very popular nowadays.  The revival of this dialog was less straight forward and is more of a rewrite. Item availability is governed by a new property of items called avail. The editor has been updated to support this value.

DHEditor has new layout and properties

The tab based layout was not going to be long lived, since the editor will eventually have a large number of tabs. The new layout is based on menus. The editor also supports a few extra properties and has a new “tab”.

New butchery editor

As you have seen when in the section about expedition supplies, you can choose not meat before embarking. The results of butchering an animal are now entered in the editor, using a new tab. As you can see, I am very fond of the spreadsheet look. Did you know that dwarven engineers were the first sentient being to use spreadsheets?

Serialization validation

All operations of serialization to streams or XML are now governed by a validation process. Should it encounter an invalid value, it will correct it and log the occurrence. Theoretically it should be impossible to crash DwarvesH now with invalid data. But I am looking forward on you proving me wrong.

Statistics

DHCore                                1823 / 43.3 / 10

DH                                        3093 / 77.4 / 18

DHEditor                              780 / 20.3 / 15

Total lines                           5696 lines

22 – Happy New Year!

Have a great new year! I sure plan to.

Here we are in January 2011, a fresh new year. So it is time to leave the not very productive December behind and let the keyboard roll. Or burn.  Still, I managed to get to a computer in some lazy afternoons and work just a little bit. Here is a sorry excuse of a main menu, but it works fine as a proof of concept:

I also worked a little on implementing animals. In an exercise of futility, I also added support for animations. Since I still don’t have my own tile set, these animations will be replaced, but they look good for now, not that you can see the idle movements of the animals if I only link pictures:

These animals are courtesy of a more complex animal editor:

There are unique tiles and names for all gender/age combinations. And that’s about it for new features.

But before doing this, I also did a lot of testing with the new rendering engine. I greatly increased map height for testing purposes and frame rate was smooth, but scrolling speed was very low. But after leaving for holiday, I did no longer have access to any of my computers and the ones I used were a lot less powerful. Here, scrolling has extremely slow, even with normal map heights. So I wrote a new scroller.

I seem to be doing this way too much: greatly improving existing features rather than adding new ones. I plan to change this. With the new renderer and scroller, I have a great isometric engine that does not care about map horizontal sizes and cares very little about map height. Scrolling is constant cost and near zero. Performance is surprisingly uniform on all machines with hardware acceleration and pretty predictable without. So I’m calling it done.

For the immediate future, I am going to try and finish all partially implemented features (not perfect them) and then get a version together and make a ten minute demo of solid gameplay. Also, need to do something about graphics.

Statistics:

DHCore:  1866 lines / 44.3 KiB / 11 files

DH: 2680 lines / 67.2 KiB / 15 files

DHEditor: 618 lines / 15.9 KiB / 13 files

Total:  5164 lines / 127.4 KiB / 39 files

21 – Hollow Earth, part two

After the lengthy explanation from last time let us jump right in, and only draw front-left and front-right sides for floors only when they are needed. The world becomes quite black:

Turning on multi-level rendering we can see how few tiles need to get rendered. Keep in mind that there is on layer that covers everything like a sheet on a bed that is not visible right now:

Then, for the first time ever, we draw the top side of floors:

And then of walls:

Dwarftastic!!! Everything looks like it looked before and the frame rate is pretty much the same. But, if we turn on multi-layer rendering and scroll to the side of the map where it is the thickest, we get this result:

The thing to note is the FPS in the title bar: 91! This used to be around 20. And with single-layer rendering, we get 108. The actual numbers are not important, since they change a little based on the content of the map and a lot based on the hardware that you are running it on. What is important is the relative difference between them. Going from 100 to 20 is very bad. Going from 100 to 90 is negligible. So it seems that all the effort put into this algorithm has paid off.

One thing that you will notice is that there are black squares where I dug out the floor. This is because in the past, it was only an illusion that we are not on level 0 but on level 1. I used some tricks and faked and extra level when needed so that I didn’t have to render another floor. Now I removed this hack and we are on level zero, so there is nothing below it when digging.

Since we now have this great engine that is capable of drawing multiple levels fast, let us change from level zero to level 10 for now. That is 10 extra levels. The problem is that I have hard-coded zeroes everywhere in code. After changing that in a long and arduous process that I am not going to talk about any more, we get this:

Perfect! And this is with levels zero through 10 drawn. There is no longer the distinction between single- layer and multi-layer rendering. Now you can choose between levels 0 and your current level and rendering of the full height of the map. This also means that you can lower your current level to underground, and become completely lost because everything is uniform rock. This is also a problem in Dwarf Fortress. You can get lost so bad that you need to zoom over to your expedition drop point. You do know the keyboard shortcut for that, don’t you? Fun times...

So let us scroll over to the side of the map, enable full-map rendering and check it out:

Pretty thick. That’s what she said! The new rendering engine is not quite map height agnostic, but it is lot faster than before. And there are a lot of performance tweaks to be made yet, especially since I was quite in a hurry with the implementation. Anyway, this is not a game in which you will need hundreds of FPS. Going up to you screen refresh rate is fine, but there isn’t a lot of benefit in going further than that. The tricky part is getting the required FPS on old hardware. Right now I am only rendering worlds of 300x300x20. I would like to increase that if possible. Horizontal dimensions are easy to increase, they only eat up memory. Vertical dimension increases have negative effects on FPS.

This concludes my two part mini-feature on the new rendering engine. The code is written in such a way that later I can replace the drawing of a side with the placement of four vertexes in a vertex buffer and I already have all the visibility factors computed for a relatively painless transition to a true 3D engine when the time calls for it.

20 – Hollow Earth, part one

As said last time, the only way to speed up rendering still left is to draw fewer tiles. But in a multi-layered 2D isometric game, where a wall tile is a single sprite, determining when something covers something is both difficult and costly in CPU power.

So let us suppose that we have a 3D world. While this world is not made up entirely by cubes, floors and walls are made up by rectangular 3D shapes that we can abstract as cubes. So how would we proceed in a 3D game? We would use one of the well-used and documented methods, like octrees. Octrees are basically cubes that are divided recursively into 8 cubes. These have interesting properties that are useful in 3D rendering engines. Or so I’m told. I never used them and I can’t write an octree implementation of the top of my head. I could with a little bit of study. But let us try something different.

Coming back to our 2D game, the idea is to split up the single tile of the walls into multiple “polygons” like you would do in 3D, and only draw the ones that are adjacent to a free space. We only draw the “borders” of the world. Hollow out the world. This should greatly speed up full multi-layer rendering, where all Z levels are drawn and which currently is unplayable, having with all optimizations a FPS of less than 20 while on the same computer single layer rendering is well over 100.

I’ll proceed and visually walk you through the process since this is not a technical blog. So let us start with a normal world, where walls are drawn by a single sprite:

Now, since the “camera angle” is fixed, we don’t have to worry about all sides of our cubes. At most, only three sides are visible, that is the ones in the front. We have one on the left, on the right and one on the top. So let us draw only all the ones on the left: 

Where is the gain in this? Well, first we need to only draw the sides that would actually be visible, not all of them. If we do that, everything will disappear. While all other sides are not drawn yet, they still would cover the ones on the left. So I cut out a small corner where two of these sides are seen:

Actually, let us cut out a few tunnels, so we can get a better look. All black parts are not rendered and should be invisible since they are covered by top layers. When enabling single layer rendering, the top sides of walls will be forced to render, but we’ll talk about that another time. This is the result:

Something you may notice is that floors seem to be drawn in a strange manner. This is due to a previous optimization in the rendering engine, which would enable some floor tiles that are covered to not be rendered. So don’t mind these. Now let us enable all right sides of the walls:

Then we will do the same thing, that is draw only the ones that should be visible. To better illustrate this, I managed to cut out a few extra tunnels:

If it is not clear what I am doing, maybe it will become clear when I apply the same process to top sides of walls and floors. Basically, the black part is and should be invisible. Theoretically no processing power is wasted on them. This only applies to multi-level rendering, but single level is fast enough.

But this post is way too long. I even broke the five pictures per post rule J. So I’ll split up this post…

19 – Not dead

Good news everybody. I am not dead and neither is this project. The huge break in posting is due to me having a busy period right now. It is December and in December you spend half the month on holyday and the other half preparing to go on holyday. Also, there was a launch of a high profile game that I am not going to name here that had occupied a lot of my free time.

But the main reason for having a hard time getting work done on the project is that I’ve gotten stuck on the scheduler. In the beginning it was simple but it got very complicated. I could stop here and formulate the entire task in a formal language so you can see how complicated this was, but that would take a lot of space and I am way behind. Enough to say that I hate it when a game is perfectly fine, but it has some serious but not game breaking flaws. A lot of games have camera problems for example. Why did not the developers fix it? More often than not the answer is that it was very hard. There is not a fixed formula that you can apply and have a perfect game camera that never obstructs your view. Making it perfect would require maybe huge resources. But still, I’ll have a perfect scheduler and I am getting close.

So now let us talk about the new features that were almost done at the beginning of December, but yada yada yada things written above.

So workshops? Yes! Quite a few of them:

Selecting a building type will result in a marking on your map. Here your building will be built. As with stockpiles, these markers are not physical objects. They are just something to help you in the game:

After that, a single dwarf will go and build the workshop tile by tile. While the process is animated and scheduled fully, the resulting building is underwhelming:

In the future you will be able to tell all building apart. Right now only the frame is placed. As this will be only a cosmetic feature, I will not rush to implement it.

This is all I promised for next post last time, but luckily I have more given the huge break. Now, when harvesting plants they will appear on the map in a small sack. These sacks are free. Just consider that your dwarves love sacks and brought 3000 of them on this expedition:

Creating food stockpiles will result in your dwarves hauling them there, like they do with logs, only this time their movement speed will not be affected. Logs are big and heavy, but a sack of food is not that much for a sturdy dwarf. Here they’ll combine the content of sacks so they take up less place and left over empty sacks will magically disappear only to reappear when needed. Here you can se the dwarves as they haul food and fill the stockpile: 

So let us say this is two posts worth of features? Not really…

Luckily, I have one more: map saving and loading. This is way over due. Now I can continue to develop the same map and I am not forced to start from scratch each time. It is maybe a little bit buggy, but I’ll fix it. Saved maps are huge, almost 3 MiB, so I use Z compression to get them down to a few KiB.

And there was also the last round of rendering speed improvements. This is it. Things can’t get any faster than this. The only way is to draw less. I have some ideas about “hallowing” out the map, ideas that could lead nicely to a future 3D engine.

For the rest of December, I am not going to have so much time, but I’ll still squeeze out a post or two. But in January everything should get back to normal.

Statistics S45:

DHCore:  1875 lines / 44.8 KiB / 11 files

DH:  2533 lines / 63.8 KiB / 13 files

DHEditor: 612 lines / 16.1 KiB / 13 files

Total: 5020 lines / 127.4 / 37 files