Cs345-2010

Porting Intel's Smoke Game Demo to Linux

N.B. - If you edit by section (using the "edit" links next to the section heading) you will be much less likely to cause a Save conflict.

Known Bugs

• Smoke.cpp does not recognize Framework.cpp unless it is explicitly included.
  • Even if Framework.h is included

Smoke Wiki Pages

• Raw List of Smoke Changes

---

Background Information

• BASH quick reference card - Media:bash-quick-ref.pdf
• Intel Software Network article
• FMOD Audio Engine
• Havok Physics Engine
• Ogre Rendering Engine Wiki
• TinyXML Main Site
• Intel's Thread Building Blocks
• (Post Other Links Here)
• Notes from the Intel Software Network article on porting Smoke
  • No one has tried in on OSX. I would be very interested to know if you can get it to work. The biggest dependencies on Windows are DirectX and .NET. DirectX is used by Ogre3D... however, you should be able to use OpenGL instead of DirectX (OpenGL is just a separate plug for Ogre). The scripting system uses .NET... but you can feel free to disable the scripting project (it's not used in the demo at the moment). In the \bin folder you will find Smoke.gdf... delete the reference to the scripting system (you might need to remove it from Demo.cdf too).
  • Now here are the big challenges. There are a few systems that also depend on DX. These is SystemInput and SystemFire. SystemInput uses DirectX's input support. But that's easy enough to replace (or disable). SystemFire uses DirectX for some of the math calculations. Our math library should support all the needed functionality... we just didn’t have time to get all the DX out of there.

---

Near Term Tasks

1. Examine all of the documentation and build an annotated index of it in the wiki. Materials that are on-line already, e.g. the ones listed above, can have URLs and annotations. Ones that come with the source kit should just have titles, file system paths, and annotations. We don't want to upload those to the wiki without first checking with Intel about re-publishing their materials. (Nick)(Done!)
2. Examine all of the source kit's directory tree and build an annotated map of it in the wiki. (Nick)(Done!)
3. Examine all of the #includes in Smoke.cpp and propose solutions for porting/converting/removing each of them. Remember that #includes nest, we need to follow each thread starting with the ones in Smoke.cpp. (Sam)(Done!)
4. Develop a Perl script/command line that converts back slashes to front slashes in directory paths. Use the feature that creates a backup of each changed file as it goes. (Sam)(Done!)
5. Figure-out roughly how each of these works, what the dependencies are, etc. and propose a solution for porting/converting/removing each of them:
   • Logging (Gil)
     • Internal messages
     • Debug messages, add the ability to
   • Debugging (Gil)
     • getopt()
   • EngineExecuteGDF (Gil)
   • main() in Smoke.cpp (Gil)(Done)
   • Argument processing in Smoke.cpp (Gil)(Done)
6. Develop an ordered plan in the wiki (collective, in class on Friday)
   • Modules
   • Libraries
     • Ogre/OpenGL replacing DirectX (Sam)
     • Find and guard/eliminate .NET stuff (Nick)
     • DirectX -> ? conversion for SystemInput and SystemFire (To Do before 2/26)
     • Havoc (Nick)(Done(ish))(See Note*)
   • Functionality
7. Figure-out Doxygen and build a doc set (Sam)(Done!)
8. Figure-out Make and construct the first simple Makefile that builds Smoke (To Do before 2/26)

• Note: Upon examining the code, I find that Havok is called, but the source for Havok is not given. We may have to inquire about and purchase the Linux Havok Libraries, as only the PC version Libraries are available for free (and only in binary).

---

Documentation

• Smoke Main Page - Outlines primary benefits of multi-threading in Games, and possibilities of use.
• Smoke Overview Video - A Short video that explains the purpose of the demo, the tools used, and some design choices.
• Tech Overview Video - A longer video which gives examples of each of the listed calculations shown in the demo, and shows the difference between 8 threads and 1 thread.
• Procedural Fire Overview - Article that explains how the Procedural Fire decides where to burn, and when hit with water, where to be put out.
• Parallel Game Engine Framework - Article explaining the decisions made when designing the demo as it pertains to an overall framework for threading a game engine.
• Fall IDF Presentation - Smoke/docs/Presentations/Fall IDF/ - Initial introduction of Smoke, outlines the challenges multi-core has presented to game development, and how Smoke solves those challenges.
• Gamefest Smoke Presentation - Smoke/docs/Presentations/GameFest/ - Explains how Smoke alleviates problems in Game Development, and how the Change Control Manager manages threads.
• GDC 08 Presentation - Smoke/docs/Presentations/GDC 2008/ - 7 zip files demonstrating games that have successfully used threading at the time, including the successes and failures of each.
• Doxygen Documentation

---

Directory Tree

• bin - Contains Subdirectories which contain DLL files for both Release and Debug Versions, Also contains various .cdf, .odf, .gdf, .sdf files
  • debug - Contains DLL files (Microsoft's Shared Libraries)
  • logs - Contains Log files
  • profile - More DLL files
  • release - DLL files used in the Live Version
• code - All files needed to build the Smoke program on its native platform
  • build info - Instructions for building under Windows
  • extern - Files for external tools used, such as Ogre, the Havok Physics Engine, etc.
    • FMOD - The FMOD Audio Engine
    • Havok - The Havok Physics Engine
    • Ogre1_9 - The Ogre Rendering Engine (Can be used with either OpenGL or DirectX, files for both included)
    • PagedGeometry - Ogre Add-on that allows Rendering of Massive Dense forests
    • SystemGraphicsOgre - Source files for Graphics produced directly by Ogre (Wire meshes, text)
    • TBB - Thread Building Blocks, Intel's Open Source Runtime Library for Paralellism in C++
    • TinyXML - A C++ XML Parser
    • Trees - Files pertaining to the Size, Shape, Type, and Growth of the Procedural Trees
  • src - Source code files for the program
• docs - Documents pertaining to the Smoke project. Presentations, etc.
• media - Audio, Video, and other files associated with output.

Note: Ogre subdirectories already include many things pertaining to Linux and OpenGL, so rendering differently may be easier than expected.
Note 2: A full directory tree, including files, has been uploaded to the repo.

---

Smoke.cpp #include Files

• windows.h - REMOVE: A Windows-exclusive header file.
  • Files it includes - Three standard headers and thirty-six Windows-specific headers
    • List of Windows.h files
• tchar.h - REMOVE: Defines generic text types for easy switching between ASCII and Unicode
• ..\BaseTypes\BaseTypes.h - The basic types on which the program is built
  • "Defines.h" - REMOVE : This includes a safe_delete, which could be replaced with some programming awareness
  • "DataTypes.h" - REMOVE:Just a bunch of definitions that make things more confusing.
  • "Debug.h" - PORT: Useful info for debugging
  • "Errors.h" - CONVERT: More useful debugging - some low level stuff might need some close work
  • "Assert.h" - PORT: debugging
  • "Math.h" - PORT: Useful math functions
  • "MathX.h" - PORT: Useful math functions
  • "Singleton.h" - PORT: defines a "Singleton," or a class that will be instantiated no more than once in each run.
  • "UnitAllocator.h" - CONVERT: custom memory management. We might want to be sure this works as expected in Linux
  • "ArrayList.h" - PORT: custom list handling.
  • "List.h" - PORT: more custom list handling.
  • "RedBlackTree.h" - PORT: Tree handling.
  • "Synchronization.h" - CONVERT: Thread handling

• ..\Interfaces\Interface.h -
  • SystemTypes.h - CONVERT: This is all about systems, so we'll want to look closely at it.
  • Property.h - CONVERT: This appears to be another interface between the program and the system.
  • Platform.h - CONVERT: Specifically for working with the underlying OS
  • ChangeControl.h - CONVERT: The idea behind this is to be a low-overhead way of making sure everything is using the right information
  • TaskManager.h - CONVERT: The scheduler
  • Environment.h - PORT: This defines some stuff to make it easier to work with the environment
  • Service.h - PORT: Interface for providing collision tests
    • Services\CollisionAPI.h - PORT: Also for collision testing
  • System.h - CONVERT: For working with "systems" (probably Havok)
  • Generic.h - CONVERT: For defining a generic "scene" (probably related to Havok)
  • Geometry.h - PORT: More math functions
  • Graphics.h - PORT: Assuming the ogre port works, this should be a cinch
  • GUI.h - PORT: This is also for working with Ogre
  • Contact.h - PORT: This is for providing "contact information" which seems more like hit tests to me
  • Behavior.h - PORT: Behaviors for objects like chickens
  • Area.h - PORT: "An interface for providing Area related functionality" Gee, thanks.
  • Move.h - PORT: For changing objects' positions based on their velocities
  • Target.h - PORT: modifies the "target" of an object

• ..\Framework\FrameworkAPI.h - PORT: Runs EngineExecuteGDF

• ..\Framework\EnvironmentManager.cpp - PORT: Handles the environment, which seems to just be a set of variables that keeps track of things.

• atlbase.h - REMOVE: Part of the Active Template Library. Microsoft-specific. Seems to do a number of different things.

---

System Path #include files

Format is: (A file that includes the header) #include <(include file)>

• Tbb is an intel threading library
  • BaseTypes/TbbSpinMutex.h #include <tbb/spin_mutex.h> -
  • Framework/TaskManagerTBB.cpp #include <tbb/parallel_for.h>
  • Framework/TaskManagerTBB.cpp #include <tbb/blocked_range.h>
  • Framework/TaskManagerTBB.cpp #include <tbb/partitioner.h>
  • Framework/TaskManagerTBB.h #include <tbb/task.h>
  • Framework/TaskManagerTBB.h #include <tbb/task_scheduler_init.h>
  • Framework/TaskManagerTBB.h #include <tbb/tbb_thread.h>
  • SystemProceduralFire/FireObject.cpp #include <tbb/scalable_allocator.h>
  • Framework/IttNotify.h #include <libittnotify.h>

• This is just a collection of windows-specific functions. It contains stdarg, ctype and string standard header functions
  • BaseTypes/Synchronization.cpp #include <windows.h>
  • Framework/ThreadPool.cpp #include <Windows.h>

• Standard Headers. All of these should work fine sans modification
  • BaseTypes/Math.cpp #include <math.h>
  • BaseTypes/Debug.h #include <stdarg.h>
  • Interfaces/Property.cpp #include <StdArg.h>
  • BaseTypes/BaseTypes.h #include <queue>
  • BaseTypes/BaseTypes.h #include <list>
  • BaseTypes/BaseTypes.h #include <map>
  • BaseTypes/BaseTypes.h #include <string>
  • BaseTypes/dbnew.cpp #include <new>
  • Framework/ChangeControlManager.h #include <set>
  • BaseTypes/Debug.cpp #include <stdio.h>
  • BaseTypes/Debug.cpp #include <time.h>
  • Framework/Instrumentation.cpp #include <sstream>
  • SystemAudioFMOD/Object.h #include <ctime>
  • SystemAudioFMOD/System.cpp #include <vector>
  • SystemAudioFMOD/ObjectAI.cpp #include <cstring>
  • BaseTypes/Debug.cpp #include <direct.h>
  • Framework/TaskManagerTBB.cpp #include <process.h>

• OgreGraphics. This hopefully is easily reengineerable
  • SystemGraphicsOgre/ObjectWindow.cpp #include <OgreBorderPanelOverlayElement.h>
  • SystemGraphicsOgre/ObjectWindow.cpp #include <OgreHardwarePixelBuffer.h>
  • SystemGraphicsOgre/Scene.cpp #include <OgreFontManager.h>
  • SystemGraphicsOgre/ObjectWindow.h #include <OgrePanelOverlayElement.h>
  • SystemGraphicsOgre/ObjectWindow.h #include <OgreTextAreaOverlayElement.h>
  • SystemGraphicsOgre/DynamicBuffer.h #include <OgreSimpleRenderable.h>
  • SystemGraphicsOgre/DynamicBuffer.cpp #include <Ogre.h>
  • SystemPhysicsCollisionHavok/Scene.cpp #include <Common/Visualize/hkVisualDebugger.h>
  • SystemPhysicsCollisionHavok/Scene.cpp #include <Physics/Utilities/VisualDebugger/hkpPhysicsContext.h>

• Other
  • BaseTypes/MathX.h #include <intrin.h> - Not sure what this all means http://parasol.tamu.edu/~yuriys/polaris/intrin_8h-source.html
  • Framework/DebugWindow.cpp #include <commctrl.h> - Looks like a Windows-specific directory
  • Smoke/Smoke.cpp #include <tchar.h> - Microsoft specific library for character mapping (lets you easily switch between Ascii, Unicode etc.)
  • Smoke/Smoke.cpp #include <atlbase.h> - Looks Microsoft specific

Sub-Systems

Logging

• Purpose and operation
• Dependencies
• Proposed adapt/convert/remove plan

Debugging

• Purpose and operation
• Dependencies
• Proposed adapt/convert/remove plan

EngineExecuteGDF

• What is it calling?
• Purpose and operation
• Dependencies
• Proposed adapt/convert/remove plan

Ogre/OpenGL replacing Ogre/DirectX

• Purpose and operation

High-level 3D graphics engine

• Dependencies

OpenGL, DirectX, or Cg

• Proposed adapt/convert/remove plan

Figure out how DirectX is being used and switch out DirectX functionality (HLSL in Ogre), with OpenGL (GLSL).

? replacing .NET

• Where is it used?

 code/src/SystemScripting/ScriptEngine.cpp

• Purpose and operation
  • using namespace System; - System Namespace
  • using namespace System::Reflection; - Reflection Namespace
  • using namespace System::Runtime::InteropServices; - Interop Services Namespace

• Dependencies
• Proposed adapt/convert/remove plan

DirectX -> ? conversion for SystemInput and SystemFire

• Where is it used?
• Purpose and operation
• Dependencies
• Proposed adapt/convert/remove plan

SSE

Streaming SIMD Extensions (SSE) is a system for speeding operations required for audio and video processing. These were implemented as extensions to the IA32 (i.e. x86) instruction set.

Resources:

• Intro to SSE
• SSE in Linux