Monday, August 29, 2011

53 – Bedthorns!

Hey! Guess who's back? If you are thinking Nana, then you would be right. I am also back!

I've been working on finalizing the plant and biome lists. Right now my plant system is so powerful, that it could handle all plants uniformly. Using such a system, a tree is just a normal plant and it could have an extra property to tell you that it generates logs when cut down. But after some serious design effort, I decided to not pursue a very realistic modeling of plants and trees and to keep plants more diverse, only for gameplay reasons. Sure, you could play a game where everything is a highly realistic simulation, with all plants going through all the phases, transition and life stages. I'm sure some biologists out there would love something like that. But I decided to create my own world, with my own rules that make sense inside that world, where everything is still heavily inspired by nature and realism, but where said realism does not get into the way of fun or you focusing on more important things.

Plants come in two varieties: bushes and trees. Of course, there are the organic components, which again fall into different categories, but I am talking about the grand picture. Plants all inherit from the Organic class, which offers the base properties for all such items, but in practice each subtype only cares about a fixed subset of those properties based on the way I wanted to design the game. In the above mentioned theoretical system that can fully simulate everything, every subtype cares about all properties, but I am not doing that. These key properties can be seen in the editor:


Let me explain the properties of the plants:
  • Id: this is a unique string identifier for the item. I use the convention that can be seen in the picture, but mod authors are free to use a different system. In the same mod, two items can not share the same id (not checked right now). If the same id is shared across mods, this is where automatic mod merging sets in. I will talk about this another time, but by the time the editor is done, it will be capable of overwriting only specific properties through hierarchies of mods.
  • Tile: this is the index of the graphic for the plant. Soon, this index will change to tilesheet index and index in said tilesheet, to better accommodate for changing graphics and modding. Plants have a single tile. Other entities can have multiple tiles.
  • Name and plural: all items have these properties. The engine can also handle adjective forms, but this is currently not used and maybe never will be.
  • Value: the multiplier for the object that uses this material. If a bush has a value of let's say 10 and a bag 2, then these values will be multiplied by this property. Currently all plants have the basic value of 1, because they are all equally as common, but I will adjust this.
  • Seasons: the seasons in which one can plant these plants from seeds. It also influences natural plant surges. And yes, right now you can cheat and plant something on the final day from the season it can grow in, thus getting one extra harvest.
  • Grow: the duration it days it takes the planted seed to mature into a full grown plant, with all the produce.
As you can see, a lot of the properties have been removed, like edibility and possibility of attack by vermin. These properties still exist, but they do not matter for bushes. You will never eat a bush and neither will vermin attack a healthy bush. Instead, these properties define the produce of the bushes.

Bellow the plant list, you can see the organics you will get by harvesting a plant, and their properties. These items are actually defined in the Organics tab, here you only assign an organic to a plant. In the Organics tab you get a lot more properties, but here you only see the one that are relevant for organics produced by bushes. If you take the same organic, and change it from a bush produce to a tree produce, the set of important properties also changes.

As said, I decided to keep bushes behaving differently from trees. When you harvest a mature bush, you get all the produce. This way I can keep bushes fast yield and low on the micromanagement, while trees are more durable and offer a yearly yield. It makes sense for crops to plant them and harvest them after, but for bushes it does not make that much sense. A little abstraction is required here, but I think the gameplay will benefit. The actual bush is of no use and will be discarded and once I have refuse stockpiles, moved there. Produce does not decay while still attached to the plant, but does die together with the plant.

I also changed the edibility property, from a boolean to a nutrition real value. If something is edible, it will have a non-negative nutrition value. Zero is for a few things, like spices, which offer very little nutrition, but still have uses. -1 means that something is not edible, and in the future I may use other negative values for fun stuff, like poison and disease. Cookability property has been removed. You can't just flip a switch and "foo" can be converted into "cooked foo" anymore. Once the new cooking system is done, and item is cookable if it appears in a cooking recipe.

Let me illustrate this by telling you about one of the new plants. Since I have realistically named and behaving trees, I'll compensate this by using only made up bushes. Also, the list of 3 plants is only from one biome (more on this another time) and there are not that many plants because trees also have edible produce.

Every biome will have one jack-of-all trades plants, that is very common, grows in all seasons, has theoretically a lot of uses but practically only a few are created, leaving the rest as back up plan if balancing though the rest of the plants fails. For my first biome, this noble spot of great but not that exciting work horse and sometimes desperation survival source is the bedthorn. It is a rather short thorn bush, that thrives in forests in all seasons (but probably not under snow). It has bright red plum sized berries that have saved the lives of many adventurers lost without rations in deep forests. Children especially enjoy these berries because of their strong red pigment, that can be used for painting. But the most interesting aspect is the name: nobody alive today remembers the origin of this strange name. Why would you make a bed out of thorn bushes?

Harvesting the bedthorn yields fruit, seeds and thorns. The fruit is edible raw and will be part of several cooking recipes, even staying around as snacks and "exotic" spicing when your society has moved on from consuming berries in large quantities. It has a fairly normal nutritious value and can be attacked by vermin. The seeds are small and similar to apple seeds, only larger and can even be eaten, but they are not too filling. They can not be attacked by vermin. And finally we have the inedible thorns, that will serve for decoration purposes and maybe make shift traps.

The graphics are placeholder.

The one thing that is missing form my first biome is the specific animals.

I also decided to handle the design of biomes differently. Land biomes rarely if ever meet on the same map section (because it is too small). So because I was designing all the biomes at once, I was actually designing the whole game at once, making my job a lot harder without a need for it. From now on, I'll design all biomes in isolation, only stopping when needed to think about the overall cross-biome balance. This also means that from now on every biome will have its own mod. I'll have a single core mod that gives a lot of general content, but all biome specific stuff will be in its own separate mod. I'll also add the possibility of specifying the rock types for each biome, to give better diversity to maps.

You know what this means: payed biome DLC! CHA$$ING!!!! Like 50$ a pop! I'll be rich! No, actually at least all the biomes and plants that have crossed my mind up to this moment will be free forever. I would rather see some third party mods with new biomes.

Here is the biome listing, without stones and animals:


Once this biome gets play tested and I have a few extra ones, I would like to add one extra plant for each allignemnt, so you have even more diversity in your biome. If I say biome one more time, probably Xzibit will burst though my wall and reenact the Internet Meme I'm sure he loves: "Yo dawg, I herd you like biomes, so...".

I also updated the in game dialog where you choose the map creation parameters, stripping away all DF inspired stuff (except for the alignment system; I like the two axes alignment system and would not like tot remove it) and replaced this with the condensed information loaded from the mods:


Another dialog that has been given the final round of cleanup was the initial expedition customization dialog:


It may look the same, but behind the scenes it no longer uses a semi harcoded heavily DF inspired skill system, but loads the skills from the editor. Actually, the entire DF like skill system has been striped out of DwarvesH.

I am trying to obtain a game where the actual implementation does not define the experience. I am trying to make a platform, a mod player, that is capable of running a wide range of similar games. The editor is starting to define an ever increasing amount of the content available.

Just to summarize, the entire plants, trees, biomes and skill system have now evolved past their DF roots. I'm sure there are some people out there who will be happy to hear that. Sure, the general idea is still similar in some cases, but I think I have given my own interpretation to a lot of these elements.

Almost a year ago, in my enthusiasm of getting the basics of the engine going, I copied a few things from DF. For nostalgia's sake, I'll add here an extract of a code I have written for one of the first posts, when implementing the first version of the skills dialog. Code that no longer exists:

enum {
dsMining = 0,
dsEngraver,
dsMason,
dsBowyer,
dsCarpenter,
dsWoodCutter,
dsArmorsmith,
dsFurnaceOperator,
dsMetalCrafter,
dsMetalsmith,
dsWeaponsmith,
dsAlchemist,
dsBoneCarver,
dsClothier,
dsGlassmaker,
dsLeatherworker,
dsStoneCrafter,
dsWeaver,
dsWoodCrafter,
dsStrandExtractor,
dsBrewer,
dsButcher,
dsCheeseMaker,
dsCook,
dsDyer,
dsGrower,
dsHerbalist,
dsLyeMaker,
dsMilker,
dsMiller,
dsPotashMaker,
dsSoaper,
dsTanner,
dsThresher,
dsWoodBurner,
dsAmbusher,
dsAnimalCaretaker,
dsAnimalDissector,
dsAnimalTrainer,
dsTrapper,
dsFishCleaner,
dsFishDissector,
dsFisherdwarf,
dsMechanic,
dsPumpOperator,
dsSiegeEngineer,
dsSiegeOperator,
dsGemCutter,
dsGemSetter,
dsArchery,
dsArmorUser,
dsAxedwarf,
dsBiter,
dsMarksdwarf,
dsDodger,
dsFighter,
dsHammerdwarf,
dsKicker,
dsMacedwarf,
dsShieldUser,
dsSpeardwarf,
dsStriker,
dsSwordsdwarf,
dsWrestler,
dsBoneDoctor,
dsCrutchWalker,
dsDiagnostician,
dsSurgeon,
dsSuturer,
dsWoundDresser,
dsAppraiser,
dsBuildingDesigner,
dsOrganizer,
dsRecordKeeper,
dsCleaning,
dsComedian,
dsConversationalist,
dsFlatterer,
dsIntimidator,
dsJudgeOfIntent,
dsLiar,
dsNegotiator,
dsPersuader,
dsConcentration,
dsConsoler,
dsObserver,
dsPacifier,
dsStudent,
dsTeacher,
dsLAST_SKILL = dsTeacher,
};
That's a $%^$^$@@@@!!# lot of skills!

Thursday, August 18, 2011

pre-alpha-3 – A* updates

Phil was kind enough to point out a link on the forums about A* and I also did some extra research, and I realized that obtaining the fastest A* algorithm is a very complicated task that depends on your exact needs. While the core of the algorithm is always the same, the small implementation details and data structures that are used count a lot. So I would have to rewrite everything form scratch and I don't want to do that right now, since my A* optimization was just so that I could fix the issues I talked about last time.

But I did do one more round of optimizations: I optimized the central node container. The recommendation is to use a priority queue but I do not think there is a name for the thing I am using. Anyway, I won't bore you with all the numbers. Suffice to say, that in release mode, with 700x700 maps, the algorithm finds a path in 45-50 ms with any kind of map: empty, maze or just random. Nice improvement. I'll leave it at that. At normal map sizes, it takes at most 10 ms and the average time over the usual distances is 3-4 ms.

This is when I realized that there is a huge bug in my algorithm that has been there from the start. I tried to fix it, but the fixed A* is about 8 times as slow as the broken one. My algorithm shouldn't even work. After some testing and experiments, I think my algorithm degrades to a very quirky version of Dijkstra's algorithm. This is probably why the performance degraded so much when the explorable region was big with my less then optimal container implementation.

I really should prove my conjuncture that my A* degrades to a Dijkstra's, but with the current performance I won't bother with it now. I'll leave it to when I do not have anything else to do.

Wednesday, August 17, 2011

pre-alpha-3 – Day 17 - More Stone Power

The polishing process continues.

I started by tackling the floor operations, fine tuning panels, adjusting skills, balancing, testing filters, etc.. This is the floor operations panel:


And this is the tool tip:


Short as always, but hopefully informative and enough to give you a starting point if this is the first time you pick up the game.

You may notice that the beatify option is not present for floors. I added the beautify options for walls to combat the frequent forgetting to issue an engrave action after a smooth action when engraving is the end result you are after. But I realized that this system is silly and adds complexity without adding depth. Combating such poor interface choices is one of the reasons why this project exists after all! I can already imagine the conversation between the expedition leader and a worker:
"Hello Myorlin! Do you see that wall over there?"
"Sure boss!"
"I want you to engrave it!"
"Can't do that boss!"
"Why?"
"Because it is not smoothed yet. You see, that wall has a very rough surface. A fellow dwarf was in a hurry to dig out a passage. I need a smooth surface to carve into, smooth as my bottle, that be the way how smooth it be must."
"Can't you smooth it first and then engrave it?"
"Sure I can! I can do anything! But you need to ask me to beatify the wall! You asked me to engrave it!"
"Beautify? What does that mean?"
"I take the wall, I engrave it, making sure that I smooth it out where needed. It is really efficient! It will even take me less time to do than first having to smooth it all and then engrave it! Alas, you asked me to engrave that wall. And listen to me: there is no way I can engrave it without it being smooth. If only you asked me to beatify it!" 
"You are an idiot!"
As you can see, any justification or rationalization can be thrown out the window: why can't you just give an engrave order, and if the wall is smooth the dwarf will start engraving, and if it is not, he will adjust the time needed to engrave dynamically based on the amount of work needed for the smoothing operation. This is what I did for floors and also applied the same principle to walls. So now engrave works on all non engraved walls, having a duration of 100%-130% based on now fast that dwarf could finish the operation. This is a perfect case where thinking stuff through and streamlining without loosing depth really pays of. I think the new system is even deeper.

But you may ask: if the engrave operation does not take a lot longer than a smooth, why would anyone smooth? Because smoothing and engraving have different outcomes. It is hard to see, but here we have a sample of some smooth floors arranged in a rectangular pattern:


And here we see the result of engraving:


Ah yes! Remember these patterns? These engraving patterns are actually the content of my first two third party mods. A while ago I asked for some engravings and this is what I got. It is a little bit hard to tell, but I used engravings to create the text "HELLO FROM DH!". The official engraving patterns will be slightly better suited for the visuals of the game, but the important thing is that I am running my main mod and two third party ones.

Another thing I have noticed is that the walls/floors duality inherited form DF can seem awkward at some moments. In DF it is less apparent, because floors are abstract and have no real height. But in an engine like mine, where floors have actual height, taking a short chunk of rock and calling it a floor and taking a higher chunk of stone and calling it a wall does not make that much sense. I'll think long and hard about it, but I can imagine a system where these chunks have different heights, and a floor is nothing more than shorter chunk. In this system different height blocks could be freely mixed.

Now on to bigger fish! With most systems greatly improved, there is but one system that I did not touch upon: A*. Now my A* is probably not the fastest out there, but it is fast enough. But it does have one huge disadvantage: the performance degrades direct proportionally not only to the length of the path (which is normal), but also based on the number of obstacles the algorithm can handle. These obstacles don't even have to be around the path. It would be great if I could improve this situation.

In order to properly test this, I'll create some custom worlds specially for A* testing:


Ohhh, look how cute and small it is! A single dwarf starts in one corner and I will give an order to dig something in an opposite corner. I just created a small map so you can see the shape.

I will be testing with 700x700 worlds. At such high map sizes A* is certainly a resource hog. Let's see some results where I measure the time it takes for the algorithm to do the path finding, in debug mode:

387.2084301 ms
388.3904938 ms
386.1530586 ms

So around 387 ms. It may be a resource hog, but it reasonably fast. It is going to be hard to improve it. I probably need to do tests on  a slower computer. I'll try a more complicated layout, forcing the dwarf to walk a long path though a simple maze:


Let's see the results for this pattern, but with a 700x700 map:
141.3050402 ms
141.6633958 ms
147.1755488 ms

OK, OK... I may be fast enough. It is even faster this way, because while the path may be longer, it does not have enough place to explore. A* depends a lot more on the total space it explores until it finds its target than on the length of the path. Still, let's try and optimize it, but using the simple straight forward empty map, on which path finding takes around 387 ms.

The result is around 350 ms and for the maze map around 125. An OK improvement and now the engine can handle any number of obstacle types without affecting the performance. In case you are wondering, in release mode, on a 700x700 map going from one corner to the other on empty map the time is 210 ms and on maze map around 45 ms. I am really curious what the performance of the top A* algorithms out there is. Any hint would be appreciated!

These tests also revealed a few big bugs, like some visibility functions being able to go outside of the map. I hopefully fixed these bugs and I added some extra asserts to these functions and added a new map sanity check pass after map generation.

The A* chapter is not finished, but today's experiments certainly helped make DwarvesH a lot more stable.

Tuesday, August 16, 2011

pre-alpha-3 – Day 16 - Stone Power

Cheers to another very fruitful day of polishing and bug fixing!

*CHUGS DOWN A BOTTLE OF DWARVEN ALE*

I have slightly redesigned the editor: using the menus was very cumbersome because you had to swap a lot between all the tabs. I removed the menu and added a vertical toolbar that nicely takes advantage of today's widescreen predominance:


The icons are not that pretty, but they are fairly self-explanatory, have tool tips and are diverse enough to be easily memorized if all else fails. Now I can easily switch between the tabs and I'm thinking of assigning keyboard shortcuts to all the tabs.

This was a usability/cosmetics change. Now for the real deal: two of the action groups are now fully realized, being 100% parameterized with the appropriate skills and pretty much ready for QA. In my previous examples explaining task filters, you may have noticed that tasks used to depend on some field from a class called EngineParams. But there is a better place to define these constants: in the skill editor!


Here you can see that the mining labor from the miner skill gives 100 experience points, has a duration of 7200 seconds and consumes 0.01666666667 task specific energy (a second). Skills now gain XP, but I'll wait with actually implementing leveling up until I balance the game. Speaking of balance, the game is pretty poorly balanced right now. It will take a lot of play testing on my behalf and others until the balance improves. Just to give an example: two hours raw dig time is not a good value. The difference in energy and time between a novice and master miner is pretty much negligible. If you are not under attack you have plenty of time to use an inefficient dwarf for digging or take another dwarf from another task to finish quicker. With two hours, the game does not incentivize you enough to use the best dwarf for the job and thus you won't appreciate and get attached to your top workers. And you won't miss them when they die gruesome deaths. After a little play testing, I think that a duration of 6-8 hours is more aproriate for a skill 5 dwarf, and every point could knock off 20-30 minutes from that. This way your master miner will do a lot more wok than a novice. What do you think?

Here is another sample skill:


Stone detailing is still physical work, so you get an energy penalty, while in the case of gem setting you do not. The durations will get increased.

I also redesigned the skill system and now, for the first time, it is capable of handling any skill, not just a few that have their logic implemented in the game engine. There is one caveat though: these skills must be production skills, i.e. take any number of items, any workshop you wish (official or from a mod) and must produce some goods. With this new system, it is possible to create a mod that adds a new skill and a new industry to the game, and your dwarves will be able to learn and level up that skill.

With the skill system in place and task filters from the first two task categories using the system to its fullest, I went in and updated the interface for these categories.

This is the new "wall actions" panel:


Oooooooooops!!! This is the new "wall actions" panel:


Carve options have been removed and we have two new actions. These actions are not planned for 0.1, but they will be part of the final game, so I though I should add them now. But what do they do? Let's see if the popup helps a little:


"Fortify wall" is an operation with which you use a wooden log to better support the wall and the ceiling. In the future I'll implement wall and roof collapsing. I don't want this to be to stressful, so for average size rooms you won't need to fortify walls. But for huge rooms, you will probably need to. But you will still be able to use it anywhere, even if it is only cosmetic.

"Decorate walls" will allow you to place things on walls: wall carpets, metal plates, paintings, flags and what not. And in the far future, when steam punk mechanism, logical gates and "circuit boards" will be implemented, you will be able to place triggers on walls with this action. Until then, this option will serve only a cosmetic purpose. The difference between this and engraving is that engraving is a action that carves out small pieces out of the wall to form a pattern, while decorate takes a finished item and nails, glues or otherwise attaches it to a wall.

Carve options have been moved to their own panel:


The entire carving system has been greatly cleaned up. Only chairs and "cests" are passable. Some spell checking is still pending! And a couple of option are still missing. In a future version I'll replace these buttons with an icon grid, because some items can have multiple angles, like stairs.

And finally, the quick dwarf inspector now uses obscure three letter words to show you what dwarves are doing:


I hope I did not forget anything, like I did in the last post. I forgot to document how map caching now takes up to two multiplication less per tile, bringing the grand total to zero multiplication per tile. This should improve performance a little.

Monday, August 15, 2011

pre-alpha-3 – Day 15

Let me start by thanking again everyone who donated! Your help is greatly appreciated!

You may have noticed that this pre-alpha is longer that the previous ones and the number of altered subsystems and overall polish is higher than usual. One of the reasons why it is taking longer is graphics. During the last 10 days I spent a lot of time trying to find the best way to give DwarvesH a new and original look. During this process I tried several things, including outsourcing, and right now I'm not exactly sure with which artistic direction to continue, but soon you'll see some new samples.

But the main reason is that I have changed the way I work and what I do in such a phase. The initial plan was to go over the 0.1 feature list and make sure that as much as possible is working well and then to adjust the next feature set plan. But I changed this approach to one in which I try to make sure that every single feature that is on track to make it into the final design of the game is perfect and fully implemented, without using any placeholders. I have a vision for most of the features. The implementation of said features varies from zero (combat as an example) to placeholder implementation that will probably be drastically changed (weapons and weapon skills are already in the game, but I am not advertising this because I know that the system will be changed) to good implementation that might lack some depth and polish to finished features that will probably make it 1.0. If one one these finished but lacking polish features is something that I want to have in the final game, during this pre-alpha it will get finished and extensively tested.

In consequence, version 0.1 will have less features, but almost everything will be near final implementation and bug free.

So next time you will see the final shape of the plant system (and a little bit even today).

I am in a hurry to write this post because Firefox is driving me crazy with its Blogger bugs. Using the WYSIWYG editor has been lately a huge pain. I can barely insert an image without everything getting messed up and I can't use bullet points.



Added error reporting
During game start up, if something goes wrong, the game will print some messages on screen. This dialog is probably not the best looking thing out there, but it sure beats the game crashing, freezing or otherwise misbehaving:




Visually countable harvested plants
Now you can tell by looking at them how many harvested plants are left in a square. The graphics are just placeholder right now, and food stockpiles do not yet have this feature:



IPS improved
The Incremental Proxy Storage is my home brewed method of storing maps in a relatively very small space without sacrificing speed. Unlike other methods of squeezing out every single possible byte, this method still maintains an OOP interface to map access and thus is still pleasant to use. But it does have some disadvantages too: it is fairly verbose to create an object inside IPS for the first time, it is not too easy to change properties, it is very hard to change structure and it is vulnerable to order and count changes. The last two points relate to saving the map as a save game. I tried to fix these vulnerabilities: creating items is now very short, changing properties remains at the same difficulty, but now this difficulty applies also to changing structure. For the last two, I need to explain the save system.

I have separated the save file into two files: the first is the ruleset and IPS for your game and the second is your raw map, saved and compressed. DwarvesH is a complex game that is getting more and more complex, thus there is no way for me provide compatibility between save games from different versions. This is where the split into two components helps. If you want to migrate to a new version, you need to create a new map under the new version, copy over the new saved IPS together with only your old raw map from the previous version to a new location, and with a bit of luck, everything should work. This is the maximum level of compatibility that I can provide. Making a save game importer for all versions would be an extremely difficult task.

IPS has an item order vulnerability when doing the above method of updating a save game from a previous version. If you change the order of some items from the same category, this order will cascade and apply to all your derivative items creatable from the original items involved that have changed order. So let's say you have a version 0.3 save game on which you played for weeks. Version 0.4 comes, the IPS migration happens, but version 0.4 has swapped the order of blackberries with a mushroom. When you load, all blackberries will be mushrooms and the other way around. So too will be all the items that have as a base material one of the two items. There is no way around this and this misfeature is not that bad: it is a lot better to be able to load your fortress on which you have worked a lot and risk a few small changes then having to either restart the fortress on the new version or be forced to keep both versions installed on your disk. It is perfectly fine if some people choose two keep more versions, but you shouldn't have to be forced to. So this vulnerability of IPS can not be fixed.

The count vulnerability is similar to the previous one, but this time the number of items changes, either because the new version added some new items, or because you are trying to change the mod loadout on a saved fortress. This vulnerability is fixed, but it does have some limits. IPS now intentionally has "holes" to accommodate more items. I added generous padding, but it is still possible to grow over its limits. This is a mechanism that protects fairly well against accidents, not against intention.



Reimplemented world generator
The old "incline hill" map generator was one of the first things I did with world generation. The code was old and not that easy to change, so I redid it in a way that hopefully bridges the gap between it and the random landscape generator. I still need some testing before I can start using the random one regularly. The new implementation also fixed an old and ugly bug.



New reachability algorithm
Ah yes: this is the star change of this post! As said in the past, A* is a poor choice for determining if two points can be connected by a path. If you know that at least a path exists between those two points, then A* is great not only at finding a path, but the shortest one. Maybe too good. It is sometimes weird to to see dwarves move with near clairvoyant path finding on an optimal path. Back in the day I added a little noise to it to provide a less "robotic" and a more "human" feel to it. Here I go again, ranting. So I use another algorithm that can tell me if two points are reachable and which caches this information for the entire map. I changed the algorithm itself to be faster and limited to the area that dwarves can explore at the current moments, so you can have any number of hidden enemy caves that have yet not been discovered without pathfinding taking a hit. Then I replaced the backbone worked of this algorithm: I went from a "vanilla" flood fill algorithm to a very fast scan line fill algorithm. The end result is that the new system uses barely any memory (while the old one ate it like crazy), and the speed in the reference interval for normal maps went from 35 ms to 8 ms.

As a test, I tried it out on a 6000 square meters map: again, barely any memory used and 270 ms speed. Pretty good! Wait... 6000 meters? That is more that last time. Well, all the performance, memory use and stability improvements that went into this pre-alpha had an indirect effect on the game, and the old limit of 4500 square meters is now exactly 10440 square meters! Again, I do not want to create such large maps. This is only a stress test. But it is good to know that without actively working to achieve this goal, the engine can now run maps with an area 5.3824 times as large as before. And the best part is that while last time I could not go over 4500 square meters area because of a weird segfault, this time the engine fails with a simple out of memory error. This error can traditionally be fixed by installing more RAM, but my case is different because I am near the limits of a 32 bit executable.

Thursday, August 11, 2011

Donations

I think the time has come to accept donations. In the future, when I have a rich set of features, my own original and hopefully pretty graphics and a community interested in this game, I would like to release a retail version of the game. Probably on Steam. It is a nice and painless online platform. Maybe, a few months before that I can pull a "minecraft": sell the unfinished but fairly complete beta version at a big discount. Both scenarios are still far away, so I need a potential way to cover some costs and make it easier to invest into the game. I did invest some money in it, but I can't say it had a real budget before.

The first solution could be ads. But I hate ads! Are you with me on this one? Who here enjoys ads? If you do, tell me in the comments, but I suspect the number of genuinely positive attitudes toward ads is extremely low. I could add ads to the blog, post an ads post once a week, add ads to the game, etc. I do not want to do either of these. But since I hate ads and probably would get annoyed by my own ads, not to mention that it would be hypocritical in this situation to use them, I will do my best to avoid ads. I can't promise that 5 years from now I won't have a single ad, but when I will have to use them, I will do it sporadically and they won't be in you face. Plus, there are multiple consumer friendly tools to hide these ads from you, so it would be quite the inefficient method of gathering some funds.

The second solution would be donations. This is what I am going to try: pure optional donations. Before I do this, there a few things that must be explained. Even though the donations link will appear on this blog, you will not be donating to this blog. You will be donating to support the development of the game.

On the 3rd of September, I will be doing the official launching of DwarvesH, under the new name, on IndieDB. I probably won't be able to replace all the graphics by then, but after a few weeks from launch I would like to have a new original tile set, partially procedurally generated. Isometric engines are really a thing of the past. Today, even if the game looks isometric, it is probably still using 3D behind the scenes. I am working hard to create the best truly isometric engine, a swan song of the isometric engines if you will.

On 3rd of October, I would like to release the editor under a public beta. The official site, forum, mod repository and others are on their way.

So you see why I must provide some way to reduce the costs. I know that you can't make a lot of money from donations, and most of the costs will be covered from my pockets, but still.

I could ramble on forever on the subjects, but I think it will be a lot clearer to list a bullet point disclaimer:
  • Donations are for supporting the development of the game.
  • Donations are not for supporting this blog. This blog will go on for as long as I enjoy writing it and may or may not change content in the future, without donations having any effect on it.
  • Donating are purely optional and development is going to continue even if there isn't a single donation. I am not trying to do emotional blackmail here: "Donate now or this puppy will die!". On the other hand, anytime an expense comes up, donations can help a lot.
  • Because donations are optional and a way for you to support the development of the game, you are actually not buying any product or service. You are not entitled to anything legally. I had to add this here to avoid any legal issues when somebody misunderstands the donations concept and actually expects to receive a copy of the game.
  • Donating does not guarantee me finishing this project. I may abandon it, but consider this: I worked for a year on it and I enjoy what I am doing. Abandoning it is very very unlikely. If an event comes up that makes me abandon this project, this event would be of extreme magnitude and importance. I won't stop working out of some trivial reasons. Such and extreme circumstance that would make me abandon this project is stronger than any donations. Well, not any. Huge sums of money have been known to do a lot of incredible things :).
  • Donations are done though PayPal. My account is in Euros, because I am from Europe. Depending on you geographic location, there may be different conversion rates and PayPal taxes. Logging into your PayPal account may give you all this information and more.
I'll just add these few main points. I have a lot of smaller ones that I'll skip and I tried to avoid legal talk.

So if you like what I am doing, would like to support this project, have some spare money, agree with the above bullet points and don't mind that in the far future I will stop accepting donations and go retail (you can't really legally sell retail versions and accept donations at the same time), then please consider donating. Any amount helps!

Thank you!




PS: Please let me know if there is something wrong with the button!


Edit: button removed!

Wednesday, August 10, 2011

pre-alpha-3 – Day 10

Not every post can have something exciting in it. This goes double for posts in pre-alpha phases, when often you must do the low level fine tuning. The devil is in the detail, as they say.

Changelog: 

  • Coordinate system fully 3D now. Regular readers may be familiar with the fact that I already switched over to a 3D coordinate system for tasks. Well, this is a long and tricky process, and last time I made sure that interfaces were using 3D points and also the implementation was, when needed, but the really low level internals would still drop the Z coordinate. Now, the entire game uses 3D coordinates! Even the A* implementation takes such coordinates and returns a 3D path, but the actual path finding is still limited to a 2D plane. Trust me, having the game fully take advantage of all the Z levels will be a major announcement, not a small point in a changelog.
  • Task lists now actually use lists. Another pending internal change. This should be more efficient and better suited for task lists, where items constantly jump around.
  • Cleanup of DHCore continues. This package is starting to look pretty good and soon will be better suited as a general purpose library, rather than a single purpose (serve my game) library.
  • New action: beautify walls. Even when testing this has happened to me: I want to engrave some walls, give the smoothing command, get side tracked and realize later that I forgot to give the actual engrave command when smoothing was finished. This will undoubtedly happen in a real game for a lot of people. So I added the beatify command, that can only be applied on natural non-smoothed non-engraved walls and is basically a smooth combined with an engrave action, but executed as a single task. It also costs only 80% of the time it would take to first smooth and then engrave. Just a little quality of life thing. PS: The wall option panel is getting very crowded. It is probably time to separate the carve options to their own menu.


  • Started modularizing tasks resolution. Gah, I can't find a way to explain that in the language of mere mortals.
  • Implemented a high resolution timer. I have been having this problem for a long time: most timer objects, including Irrlicht if I am not mistaken, use this little thing: GetTickCount. A very useful function, but as said also in its documentation, it has a resolution of 10-16 milliseconds. This is great for most tasks, but sometimes it makes this function unusable. Let's say you have an A* implementation that takes 30 ms to find a path, and you would like to optimize it to only take 20. With a 10-16 ms resolution, you are utterly incapable of measuring such an improvement. After scouring the Internet, I put together a high resolution timer that is even less portable than the above Windows API function. This timer is even capable of using multiple CPU computers to improve the precision of the measurement. In theory. In practice I have no way of telling that. I ran a few artificial benchmarks to test it. In these tests I ran a test 10 times: wait 30 ms and measure the precision of the timer, logging the absolute difference between the 30 ms known interval and the measurement, and determining the total and average deviation. Here are the results, first for a average test:


As you can see, the high resolution times is extremely precise, while the normal timer, during ten tests, accumulated a  total error almost equal to the duration of a single test. Once in a while, you get lucky and GetTickcount returns quite an accurate results:


But the HR timer still has it beat and is very predictable. I do not know what resolution the sleep function I am using has, but if the HR timer is to be believed, it is quite accurate. Using this timer I am going to try and greatly improve the A* and reachability calculation algorithms one last time during this pre-alpha phase.
A long long time ago, in a galaxy far far away, somebody on the Internet thought that such massive changes to the task system went over without causing any bugs. And he was wrong! Surely there are a few new bugs that will get squashed. On the other hand, the new system also probably fixed a few existing bugs, like when you give the same order on two different Z levels.

And if you still want something exciting, later today I should be introducing the donation system!

Saturday, August 6, 2011

52 – This is for the n00bs

Here is another half new features half pre-alpha polish post.



Map renederer moved to its own widget
The map handling class was getting very cluttered, so I isolated all the map rendering, cursor management and mouse collision detection code into its own class and widget. This was done mostly with copy and pasting, but with a slight redesign of the interface for this widget, I will have a very good general map rendering component that can be dropped anywhere in the UI. This way I will be able to create multiple windows with different functionality and GUI, but which all show the same map, by simply inheriting from this base class. I will be creating an "AI Playground" mode soon with this facility.



Mining code given the final polish
I talked about how I finished the mining skill and mining task last post. I went in again and made sure that it is well designed and bug free. During this pre alpha all the skills that I have in the game will start to affect their associated jobs. The reason that I am bringing this up again is that I am going to show the code for this task again. I usually don't show code here because I want to keep the blog non technical, but I have already shown this part of the code once and this is good opportunity to show how its complexity has changed over time:


Task scheduling is basically the most important part of a game like this (together with path-finding). Nevermind the performance. If your dwarves are doing something stupid you won't care that you have 300 FPS. Creating one task can be hard, and there are a lot of tasks, several of which interact and have multiple phases. So I came up with the above design to handle these complex tasks. Every tasks has such a class. The implementation is fairly simple for this task which defines the wall dig operation and all similarly complex tasks have such implementations. Other tasks like hauling and building of structures area lot more complex, but the idea is to have one centralized place where the task is defined and if something is wrong to know where to search for a bug. There is one more phase, tasks resolution, which must be defined for each task and is not visible in the screenshot, because it is not part of that class. Task resolution and task filter for each task is the only thing that separates one task from another and all tasks are handled 100% uniformly.

Let me explain in two words the task again. The constructor of the task tell it to use my own super smart task finding algorithm and bypass A* when the engine feels like this would be a benefit. Then we have "TestDwarf", which tells us if a dwarf is capable of doing this job. Other low level bookkeeping, things that are general for all tasks, like if a dwarf is idle or has enough energy and other conditions are handled outside of the task filter. In this case the test boils down to the mining skill having the mining labor enabled. Then we have "IsValidSource", which is called for each candidate task coordinate, to make sure that nothing unwanted slips in. We test and make sure we only dig walls, ramps and built walls. This check is mostly for the UI and also a failsafe. The final method, "OnSuccess" handles the actual adding of the task to the dwarf. It is only called if we have found a worthy dwarf and a task that he can reach. We test again the task type with an ASSERT this time. At this stage there is no way this test will fail. If it does, the entire scheduler had a big bug somewhere and the assertion will warn me about it. Needless to say, it never goes off. I am too good. We calculate the energy consumption and adjust it by the mining skill of the dwarf. We do the same for the duration of the task, taking into account the material we dig though and the skill of the dwarf. And we are done! In the task resolution section we have again simple things, like clearing out the cell and placing boulders if we dig though rock.

I promise not to have a lot of these code sections.



Skill system redesigned
I redesigned the skill system to be more flexible and rely lesson hard coding. Dwarves can have any number of skills and any skill can have any number of labors. In theory. In practice, the skill set is well established and created with the help of the editor:


You can use the editor to create a completely different set of skills, but the game won't function with them because I would have to also move the skill logic to the editor. But it is still a good tool to do all non-structural changes and maybe in the future provide internationalization. Together with the special tile editor, the skill editor is designed to be used only by people who know what they are doing. All other panels can be used by anyone and won't mess up the game too much.



Basic help system
This is the feature that gave the name for this post: I implemented a little hint system that will tell you why you don't have access to the task designation buttons if you have selected something wrong. I'll let the video do the explanation:

Wednesday, August 3, 2011

51 – NRGY

The subject of this post is the result on today's pre-alpha work, but since I have a lot of new and updated features, I decided to make it a feature post. And stay tuned for an Irrlicht rant.

First I must explain the four core needs of a dwarf. While dwarves haves a lot of statistics, nothing is more important than: energy, food, alcohol and water. All these core needs are numerical values that express how much said need is satisfied, so the higher they are, the more time the dwarf can do other stuff before they must replenish these needs. Every second of in game time, these values decrease, roughly with a value equal to the unit. The good old 1! But each dwarf has his/her own rate of needs decay, generally withing the range of +/-5%. So needs decay with values between 0.95 and 1.05.

Energy tells you how much a dwarf can work. Even idling tires a dwarf slowly and eventually the dwarf will have to go to sleep. Every dwarf has his own energy decay rate. Doing physically tiring tasks will deplete energy faster. So every second, the dwarf's energy will decay by personal decay rate + task difficulty penalty (adjusted by the skill level and perks). Like all needs, reaching the value of zero does not mean game over. Zero energy simply means that the dwarf's body has spent the energy reserves it is used to and would normally stop working. But the dwarf can go into negative if needed. Zero energy is you, if you had a physically challenging job, and you just got home after a particularly hard day. A moderate amount of negative energy is still you from above, but after getting home you went to the gym and did a killer work out. A high amount of negative energy need is you, if after the work out, you had to fight an army of ninjas and came out victorious. The dwarf will certainly feel that he is in negative the next day, being sore, grumpy, having to sleep longer and still being less productive. But the system is not designed to be punishing. It is designed for the case when the immediate survival of your fortress depends on the execution of a long task. You give the order, the dwarf checks out his energy levels and says to you "I would really like to go to bed in 5 minutes. Is this task going to take me less than 5 minutes? No? Well, see you tomorrow!". And then you die. But with this system dwarves can go into negative, but almost always will just go a little bit in that direction. In case of emergency, they will work their ass off. If a dwarf still has positive energy when he starts a task, his honor will dictate that he finishes it, even if he falls over when it is done. After a full night's of sleep, a dwarf will always be at 100% energy, i.e. enough energy to idle for about 20 hours.

This is the model for energy that I will be using in 0.1, and if I do not get a better idea, for several versions from now on. It is a system for the ages!

Food need is similar but won't make it in 0.1. Just to give a few details, the food need behaves similar to energy. Once we get to negative values and the dwarf is not very busy doing something urgent, he will take a break and go eat something. Dwarves can either eat when they are hungry, or if you designate an official group meal time in your great dinning room, dwarves will wait until then, unless they are starving. Having official eating events will cause the dwarves to plan their tasks around these hours (thus being slightly less productive), will take longer eating (thus being slightly less productive), but will socialize, make friends and their morale will rise. If no food is available, once the dwarf gets desperate, will try to eat raw a plant from nature, even if it is not edible. The plant will be destroyed and the dwarf may get sick. If no plant is available, the dwarf will try to kill a pet. Or even a fellow dwarf. And different foods have different nutritious values. With high quality cooked food, a dwarf will eat twice a day and be happier, while with raw/low quality food, he will eat 3-4 times a day and have average happiness. Not eating for about 4 days will kill the dwarf. Since they are so resilient physically, they also need a lot of food. There is chance that the dwarf will wander off in search of food if starving, leaving the fortress forever.

Water need is identical to food, except that you can not designate official drinking times in the great dinning room and eating will often cause the dwarf to replenish his water and alcohol needs too, without taking extra time. As long as the dwarf took a break to eat, he might as well take a drink too. Water need has desperation actions similar to food need, but with natural water sources. I'm not sure about desperation kills and drinking of blood. A dwarf will dehydrate in about 2 days and die. Water need is planned for 0.2.

The more interesting need that is tightly related to water need is the alcohol need. After so many generation of alcohol consumption, the dwarf's need for alcohol is passed down genetically from one generation to the next. While a dwarf may seem tipsy or even drunk, especially to other races, they actually can't get drunk in the traditional sense of the word. Their clarity of though is only mildly reduced by alcohol and their hand eye coordination only improves. There is a critical point where this improvement stops, drastically reverses and the dwarf is practically completely smashed, but they have long since fallen asleep by then. On the other hand, not drinking alcohol regularly causes them to become slow, clumsy and depressed. These effects will worsen, up to a 60% acquired slowness (tasks will take 60% longer to finish), but a dwarf can't die from alcohol withdrawal as long as they don't die from thirst. Satisfying at least 75% of you water need with alcohol will keep your alcohol need at 100%. 

So this is the theory. Let's get to the features!



Energy system reworked
The above described energy system has been fully implemented. The other three needs were added, but they don't decay. I had energy system and decay in place for a while: see the videos with dwarves going to bed after they dig. The old system had some caveats though. Only dwarves who did something suffered energy decay, while idle dwarves would never need to go to bed. And energy was taken in one huge chunk at the start of the task. This is fixed now: all dwarves suffer form energy decay and the decay is gradual. Every dwarf has his own personal energy decay rate (which will be passed down to offsprings).



Digging operation fully realized energy wise
When doing a difficult task, to the personal energy decay rate you add a penalty based on the difficulty of the task. This penalty is zero for all tasks except digging. The reason for this is I only bothered implementing the new system for wall digging. Other actions will follow as the pre-alpha progresses. The dwarf's mining skill now has an impact on digging. Skills range from 0 to 20, but all dwarves start out with a skill of 5, meaning they are dabbling. I'll detail the dwarven education system some other time and explain why they all start out at level 5. Skill 5 is essentially skill zero, giving zero advantages. But as skill level increases, the mining skill will reduce the penalty energy decay (only when mining). At skill level 20 the penalty will be zero. The skill level also influences the time needed to finish an operation. Digging though soil takes one hour and has only a 25% penalthy. Digging though rock takes two hours and has 100% penalty. A skill of 5 does not effect this time and a skill of 20 reduces the time by 15%, with 1% per skill level. A dwarf with skill under 5 can not dig.




Dwarf inspection panel
With all these needs and a lot of stuff going on at once, it can be hard to keep focused. So I added this small panel:


This panel will grow based on the number of dwarves you have, but never be large enough to cover the small button in the corner that opens the options panel. It lists all dwarves, with their image, nickname, and four bars. Each bar represents a different need: energy, food, alcohol and water, in this order. They are color coded. These bars will grow or shrink in real time:


The vertical line is the zero level, so you can see when a need is starting to go bellow zero:


And in the corner, besides each dwarf, you will get a little status string. Right now it only shows which dwarf is sleeping:


I will extend this status to show what each dwarf is doing.



Skinned panels
You may have noticed that some panels now have textures. This is phase one of bringing the panels on par with the concept art I never got to show.



Fixed sleep scheduling bug
There was a long standing bug in the sleep scheduling algorithm. I suspected the presence of the bug for a while now. I think even in the videos you may notice a sleeping dwarf changing beds. But the bug was more severe than that. The game checks the list of idle dwarfs to determine if there is a good candidate to put to sleep and if there is, proceeds to schedule the choosing of beds, path finding, occupying and other book keeping operations. During this process, due to a very subtle and hard to find bug, the candidate dwarf was lost and replaced with the first idle dwarf.



Fixed mouse selection bug
When interacting with the map, you update the current selection with the mouse. I found a new bug here. First, I was scared that during the Irrlicht GUI elimination I broke mouse support, but actually the problem was caused by the fact that I am allocating a larger map than it is used. Bug fixed now, but the fix is not ideal. Added task to fix this properly: planned task #1.



Automatic nicknames
During world creation (manual or automatic), all dwarves without nicknames will get a random one. The game usually presents you information about the dwarves using their nickname, so having an empty string instead of a nickname does not help. Nicknames will be changeable at any point in the game. Normal names are long and hard to pronounce, so having you dwarves called Mike and Joe helps a lot. Or if you are pragmatic, M1 and M2 for your miners.




Never wait for sleepers
Imagine the following scenario: all you dwarves go to bed at roughly the same time. You are stuck waiting for minutes until the in game hours have passed and your dwarves are awake again. Not a problem! If the current time compression is x1 and all your dwarves are sleeping, time compression will change to x180. As soon as one of them wakes up, time compression returns to x1. This way you don't miss anything and you don't have to grab a sandwich. Alternatively, I could pop up a window: "All your dwarves are sleeping. Would you like to skip tot he next event?". And clicking yes would warp you to the moment the first dwarf wakes up. Using time compression seems more natural, for now at least.



Reworked status bar
The status bar now displays the number of idle dwarves. If this is more than zero, it will use the color red. I shrunk the pause button, since most people will be using the keyboard shortcut to pause (space). The button looks ugly because I am using ASCII art (|> and ||) to represent "play" and "pause". My buttons do not support icons yet.



Here is a video showing the panel and energy decay in action: