DX(t)UT - 01 - Empty on the inside

Hi and welcome to my new and understated series! As promised quite a while ago, I will try and share some of the knowledge I have acquired in the fascinating but daunting domain of 3D engine writing. I have acquired a lot of knowledge, some useful, some that I will never use again and created a few shaders that would be a shame to waste. So I'll create a few tutorials based on DXUT to introduce some basic concepts and show how these shaders can be put into use. I have named the series DX(t)UT. See what I did there? I r wordsmith. Think of it LL3DLGLD version 2.0, with the exception that all the presented material will be fully explained, together with source snippets and even full compileable source code when needed.

Since this is the first post in the series I must stop and focus on some introductory points. First off all the tutorials will be targeting C++ with DirectX. Not the best choice. DirectX, while the best at what it does, even after multiple interface clean-ups is still quite obtuse and unpleasant to use. Before you can use DirectX you need to set up a basic window and event handling and since I am targeting low level APIs, this would end up being done directly with Windows API, which is even more unpleasant to use and understand. This is where DXUT comes in. It is a very thin wrapper over some parts of the API that takes care of initializing the main window, event handling and the nitty gritty of DirectX initialization. DXUT is an utility library and after you use it to initialize all that must be initialized you still end up using full DirectX. So you will both learn to use DirectX and generally be able to copy and paste DirectX snippets into your DXUT application without problems. We will put this to the test around lesson 5, where I'll convert some very useful DirectX samples to our DXUT application framework.

As said before, DXUT with C++ is a pretty poor choice. I think this series would be more useful if it targeted XNA 4.0. As a famous graphics programmer before me said, "that's where the shiznit is at, dawg"! This series is dual purposed. While I want to record some stuff and make it available for posterity, I will also try to teach you a few concepts and make you understand them. In consequence, a primary audience would be collage students into computing and graphics who are not taking a course on this subject. Or maybe high school students. I did a lot of graphics related programming in high school and if you are as smart as I was you'll have no problems understanding any of it. So a young and skilled but lacking experience individual, who likes games and would like to create and engine might find some very useful information here. No, you won't be available to create competitive engine. Get it out of your head. Not because the content here will be lacking (which it will), but because the number of man hours and shear effort needed to create an engine as powerful as the ones the big boys are using for commercial games is too much even for a small team to handle, yet alone one individual. But you can learn a lot of small stuff. And if this is what you want to do, I recommend XNA. It is a great platform, well documented, everybody is using it, the Internet is full of information regarding it and it can be quite pleasant to use. There is no good reason to use DirectX/DXUT directly if you want to create a game or even a high level engine capable of multiple games. But if you are interested in the low level stuff and would like to have total control of every single bit, create everything by yourself and have a playground, read on.

Boy, I sure am rusty at writing and teaching. The above paragraphs are all over the place. I promise to improve upon this in the future. So I won’t stall and dive right into it at a moment’s notice, but there is one more thing that I want to mention. I’ll start out with a few introductory lessons, but the main focus of the series is shaders! This is why I don't mind that much that I am targeting the lesser platform here. The way graphics work today, you'll end up relying on the GPU and shaders for most tasks. In a few years even more so. Even today you can do tons of stuff without invoking the CPU in a meaningful way (like fully GPU based terrains that do not create vertex buffers) and once you understand the GPU part you can easily adapt if from DXUT to XNA or some high level engine.

The shaders will be written in HLSL (duh, DirectX). I will not investigate Cg as a "cross platform" shader language and in the future I'll explain why I am not targeting OpenGL and GLSL. I will also try to make full use of shaders version 2.0, trying to push them to the limits, but some things cannot be done without version 3.0 or even 4.0. I will also be mainly targeting DirectX 9.

That's it for the introduction. Let's dive right into it!

For the first lesson I will analyze and explain one of the DirectX SDK samples: EmptyProject. This is the shortest official example that does absolutely nothing so it is well suited for explaining how DXUT works. Even so, it is relatively long and verbose for a program that does nothing except to show an empty window. But is a lot shorter that the classical DirectX/Windows API equivalent.

A huge reminder here in the middle of the post: I expect you to be comfortable with C++ and Visual Studio. In the future I'll distance myself from it, but for starting lessons this offers the path of least resistance. I won't explain step by step how to set up your environment. Try to figure it out on your own. But here are the main steps:

• Download the Windows 7 SDK free from the Microsoft site. This includes the compiler, header files and necessary libraries. This is fully command line, so you'll need and IDE too. You only need the C++ parts, but you can fully install it to avoid problems if you don't know what you are doing. 
• Download Visual Studio 2010 Express free from the Microsoft site. The Express edition is pretty good editor. It won't get you too far because the libraries that it ships with are very limited. This is where step one comes in. The Windows 7 SDK will have all the platform specific headers and libraries that the Express edition is missing. Or at least this is how the 2008 version worked. 
• Download the Direct X SDK free from the Microsoft site. 

Obviously, Microsoft is paying me. I am a corporate sell out and... DIE GNU! No, seriously, these are just a few free tools that are easy to set up and have a large user base so you can Google stuff if it is broken. I have a setup where I can use MINGW as an alternative for compilation with the press of a button and have not used Visual Studio in years. But for this series, if you followed all the steps, this will get you going a lot faster.

Now that you environment is all set up (bonus points if you already had one set up or you managed to get it right on your first try) you should have all the DirectX samples installed. For simplicity I'll consider that you installed under the default paths, so navigate to "c:\Program Files\Microsoft DirectX SDK (June 2010)\Samples\C++\Direct3D" (your path may be slightly different). Here you will find all the samples. Go to the "Bin/x86" folder. Here you will find all the samples compiled. Run a few of them to see what they do and to make sure that your GPU can handle them. Up to date DirectX and drivers are a good idea.

Do the samples work? Good. Now go back to the first path and go into the "EmptyProject" folder. Double click on "EmptyProject_2010.sln" and after Visual Studio boots up, compile and run it. If everything went without an incident, you will be greeted by this lovely window:

Lovely, just lovely! I bet you are all excited and tingly on the inside about becoming a real graphics programmer!

This application is perfect to show off how DXUT works. Let's start at the end of the program, where the "main" method is located:

Above is the code for the main method. Since Blogger is not that great at formatting and spacing text, I'll use screenshots. You can find the code in the folder I mentioned above, so you don't need to type it out. When I'll post original code I'll provide download links for the full project.

So let's analyze this "main" function. Since this is Windows API application, the entry point is somewhat different from the standard main function, but is serves the same role. The first 3 lines (#if... #endif) set up some debugging utility checks which I'll leave in for now.

The next "block" sets up all the callbacks. DXUT is callback based. You need to call a set of "DXUTSetCallback..." functions. These callbacks have very specific role and meaning and do a nice job of abstracting away the glue code so you can focus on the actual part you are interested in. DXUT supports DirectX 9 and 10 callbacks (and 11, but I won't talk about 11). You have a set of functions that set up the callbacks for DirectX 9 and equivalents for other versions. As an example, the DXUTSetCallbackD3D9DeviceAcceptable function has a DXUTSetCallbackD3D10DeviceAcceptable counterpart for DirectX 10. You should set all the callbacks for the version you are targeting. If you want to support multiple versions, you should fully specify all callbacks for all versions. DXUT will pick the highest version that is available, but there are methods that give you full control.

Let's skip this block for now and see how we create our application. The first method is DXUTInit. This does the main initialization and set up tasks so that DXUT can work and is the first thing you should call after having the callbacks set up. If you do not call it, DXUT will call it when you try create the device with default parameters. The first parameter is a boolean that tells DXUT to parse the command line. DXUT supports quite a few command line arguments. You could pass "-fullscreen" as an example to any DXUT app to force fullscreen mode. These parameters work as overrides, but only the first time. Let’s say you have your application set up to start in windowed mode. If you run it with a command line to force fullscreen, it will be run in full screen. If the user than goes into a possible setting dialog your application provides and disables fullscreen, the command line argument will not override it a second time. You can find the full list of parameters in the SDK documentation. The second parameter tells DXUT if it should display error messages for possible errors it has found. This function takes more parameters that I will not describe here.

The next function that we call is DXUTSetHotkeyHandling. This sets up a few basic keyboard shortcuts. The first parameter will enable Alt-Enter as a fullscreen toggle. The second will allow Esc to be used to exit the application. The third will allow Pause to be used to pause the frame processing loop. As you can see, these are very basic features that are at most a useful little feature that you can use while prototyping something. Later you will probably enhance/replace these small touches. This is very much in line with the design philosophy of DXUT.

Next we have DXUTSetCursorSettings, the third useful features set up function. By default the application will display a cursor when in windowed mode. Setting the first parameter to true will keep that cursor visible when you switch to fullscreen mode. If you want fullscreen windows without a cursor, use false. The second parameter clips the mouse movement to the window boundaries in fullscreen mode. To be quite honest, I don't get this flag. It does not seem that useful unless you have a multiple monitor setup. On the other hand, a toggleable mouse clip for windowed mode would be very much appreciated. Get on it, DXUT authors!

Together with the callbacks, this concluded the set up step of our application. We told it to parse a few default commands line arguments, display error messages, show cursor, handle a few shortcuts and gave the window a title. All this without using Windows API or even DirectX API!

The final step is to create the device: DXUTCreateDevice. This is the first function that actually does something "important" directly. The first parameter controls windowed mode, with true meaning that the application will not run in fullscreen mode. The second parameters control the suggested width and height of the window. DXUT might adjust these values depending on your system. In fullscreen mode the width and height should describe a resolution your adapter supports for optimal functionality. Based on what callbacks are set up and what your system supports, this function will choose a DirectX version. It will take all the options your provided (and if you did not call some of the setup functions, they should be called automatically with default parameters), gather them and after some steps it will call the callback you set with DXUTSetCallbackDeviceChanging so you can inspect and review these setting before they are applied. And then it will create everything and your window should be ready to go.

The main function has still two lines left. As customary for Windows applications and games too, they have a main loop that pools events and does game logic and rendering. In DXUT you use DXUTMainLoop to run this loop. 

And finally, you return with the value provided by DXUTGetExitCode. DXUTMainLoop will exit from the main loop when your application finalizes correctly or when an unrecoverable error has occurred. DXUTGetExitCode will return zero if the application finalized as expected and a positive value if an error is encountered. I won't list the values here, but most can be described with "an error occurred when creating/initializing...".

So, that's about it for the main function. I did not go into great detail, yet still the post is quite lengthy. And it will get even longer because I did not go into the callback functions. Still, this is a lot shorter and abstract than the equivalent plain DirectX application.

After we get used to all this, I'll try and isolate the "glue code" core and only have smaller parts that you need to change from application to application. So on to the callbacks!

First, we have IsD3D9DeviceAcceptable (set by DXUTSetCallbackD3D9DeviceAcceptable):

This callback is used to test the capabilities of your GPU in order to determine if it is capable of running your application. Here you will test hardware features, shader support and other major features. You won't test parameters that are unique to you current execution of the program, like resolution or colors. In this sample it only checks to see if the device supports alpha blending and rejects it otherwise. I won't explain that since you will probably copy & paste that everywhere. If your device does not support basic blending, you probably don't have any worth-while 3D acceleration. In future samples we'll keep this check and also verify the shader versions that are supported.

Then we have ModifyDeviceSettings (set by DXUTSetCallbackDeviceChanging):

This function is called before each device settings change. DXUTCreateDevice gathers all the settings and presents them to you before it creates the device so you can review and change them. Every time you change the settings this function will be called. If the function returns true, the DXUT will try to apply the settings. If it returns false, it will keep the current settings. So it makes sense that you return true the first time, so at least you have a window here something can be displayed. More specific applications might refuse some resolutions and other combination of settings, but generally an application should be flexible because you have no way to predict what setup the user will use to run it. In this skeleton application we simply return true and accept anything.

Then we have OnD3D9CreateDevice (set by DXUTSetCallbackD3D9DeviceCreated), which initializes the resources that our application might need for rendering. There are two kinds of resources. There are persistent resources that once created can be used until the application exits (and frees the resources before the exit). Other resources do not live such a long life. These resources can depend on the buffer size or other variables. The back buffer is such a resource, that get's reset every time the size changes. Or they can get reset because of other reasons. A texture might get freed when you Alt-Tab away from your game and run another application that requires that memory. That resource needs to be reset when you switch back to your application. OnD3D9CreateDevice is used to create only persistent resources and in our empty sample we simply return success:

The counterpart to this function is OnD3D9ResetDevice (set by DXUTSetCallbackD3D9DeviceReset), which creates resources that are no persistent. It is called "reset" because the line between having to initialize a non persistent resource and having to reset it due to some event is a blurry one. In general you'll just call a "reset" on all the appropriate resources here and this is mostly a filler function. One case where you actually need it is when the reset is caused by you changing the device settings. Each time you do that this function will be called (only if the setting were accepted by ModifyDeviceSettings ) and here you will probably do some computation based on the device size and reposition you camera and GUI. Again, we return just the OK value:

The resources that are reset here and were created in OnD3D9CreateDevice must also be released somehow. These creation functions have they counterpart that releases them. As a general rule, what you create somewhere you destroy in the counterpart function. These functions can be quite confusing if you are a beginner and since the empty project example only returns a success value if needed, I can't really explain them here. I'll try to do so in the second example. Here is the body for the two functions:

This concludes my description of the second phase of the project. Phase one was initializing everything and phase two was adding the bodies for the resource creating/freeing functions. The final phase is handling frames, rendering content and handling events. This is done again by callbacks and yet again they are very simple in this example.

The first one is OnFrameMove (set by DXUTSetCallbackFrameMove). This function is called every frame before it is rendered and here you will have the bulk of your game's logic. You will update counters, timers, run animation, collision, path finding, A.I. etc. here. As long as it is a short enough task. You don't want to have a long task because this is called in single threaded mode and no new frames will be rendered before this function exits:

Next we render the frame with OnD3D9FrameRender (set by DXUTSetCallbackD3D9FrameRender):

This is the first function we actually do something that is not initialization/book keeping/glue code. Here we clear the buffer and the Z-buffer and we call BeginScene and EndScene. Between these two calls the actual 3D rendering calls would be put. "D3DCOLOR_ARGB( 0, 45, 50, 170 )" is a macro that specifies the color in ARGB format using integers. And I'll leave as homework the presence of the apparently unused "hr" local variable. Hint: it is not a type/oversight.

You are not allowed to do logic in this function. While normally for each OnFrameMove OnD3D9FrameRender will be called only once, this is not guaranteed. If you include logic here and it changes some internal state, two frames that were supposed to be rendered identically might turn out differently.

And finally, we have MsgProc (set by DXUTSetCallbackMsgProc), used as the callback for event handling. Empty as usual:

Wow! That was long and hard! Avoiding obvious default joke. But bringing attention to me avoiding it!

Putting all this together, you will find that you have 6 KiB of code that create an empty blue window that I have spent pages describing. I tried to keep it as short as possible, but giving basic information on all the elements is a must so we can get on to real task and lessons on shaders.

If you came from a Unity/other drag and drop engine background you probably left the moment you saw the WinMain function. If you came from an OpenGL/GLUT background you probably find this long and verbose. I hope you are not patting yourself on the back and saying "LOL, FAIL M$!!!". If you come from Ogre3D then...

Anyway, this is the end of part one. In part two we'll go over another simple skeleton app that does nothing (but this one is going to be a lot more complicated) from the DirectX SDK samples. In part 3 we'll try and abstract away all the glue code so we can have a nice little OOP playground for playing around with DirectX, while adding some parts from another sample application. In part 4 we'll completely recreate the before mentioned sample application using this mini-framework. In part 5 we'll take some very useful DirectX snippets as they are from the Internet and see how easy is it to integrate them into a normal DXUT app and then how easy it is to integrate them in our test framework. By lesson 7 we'll load meshes from external files and render them with lighting and CSAA antialiasing.

And if any aspiring DirectX-er reads this, please let me know what you think!

Oh, and stay tuned for status updates on Dwarves & Holes, the engine and the future of this blog.