Section 1...

Heres what I have, I hope this helps. I'm using the framework I suggested and Realm Forge GDK as a model for this but, again, there are some gaps in here and various technical issues that need to be fixed but thats where the real hard work comes in. You'll probably spend as much time working on the framework itself and various standards as you would have spent working on any two middle ware components.

Sorry for the eye strain, this is another long post. If you hit something you don't understand just ask and I'll tell you everything I know about it or at the very least give you a link to a site that explains it in detail. Also, if some of these 'blocks' of text are too much I can repost those sections with different formatting and rewrite parts to make it easier to read and hopefully understand.

Part of the problem here is I'm heavily summarizing stuff, glazing over things, and in some cases skipping things completely. Take OpenGL for example, I have a 597 page page that just covers OpenGL, thats it, plus theres another 1,000+ pages of free books and references on the OpenGL website but I only wrote a couple lines and added a few links. Please keep that in mind, this information is by no means complete but should scratch the surface at least a little. Also there is some stuff in here that I'm near clueless about so if it sounds really vague thats probably why but I'll still try to research it enough to answer any questions you might have. My object here is to make sure this project is off to a running start before I get back to my other projects and if I have spend another week on here answering your questions that is exactly what I'll do!

Assumptions: Rereading this it kind of dawned on me that there are a few good examples of the differences between C++ and VB in here. I still don't know much about C# but if you read or even skim an older programming book on C++ that covers basic program structure you'll see some commonalities here that just do not apply to VB. Also I'm assuming that you know by module I mean system or a collection of very similar things within the program (e.g. similar data types, ex: music modules, its just a folder(s) full of MP3/Ogg files, nothing seriously complex about that!). I'm also assuming this thread will be met with some understanding, a little appreciation, and probably a few dozen questions. Do not call me insane or dismiss my efforts or I'll delete the whole damn thing!

Thread Organization: Framework summary with required support systems, framework outline, detailed explanation of the framework itself, a summary of the “fastest functional engine build”, and framework notes. Some of the module descriptions includes the links to various open source software projects so if you need more information about a particular module it might be better to check those sites before asking me for more detailed information (I'll add more links after I post this).

Framework Organization: The problem with forum threads and flat text is it can't really relay the framework's tree-like or flowchart-like structure. I clumped the different modules together in levels but its kind of hard to visualize the connections between them, so it might be useful to create a diagram. I suggest using Visio or OpenOffice to create a chart to follow, but thats up to you (charts can be very useful in designing complex programs or any complex system for that matter).

Operating System Independence: Someone requested OS independence. The only things that are truly OS dependent here are noted, everything else should be OS independent. So the guy who wanted that feature can port your core program over to another OS and then everything that is OS independent should work fine. This is a very attractive feature to have since there aren't many great games on non-windows platforms. The process of porting the core modules to other operating systems needs to be noted in the source code documentation under its own chapter. This is not something you will actually have to worry about doing yourself but through this framework it will only take a little extra planning, some attention to detail, and a few special programming measure to support. Too, I'm sure if you take the time to increase the game engine's OS portability someone will come along and port it.



Edited by - MegaBurn on 12/20/2005 1:49:53 PM

Section 2...

Modular Game Engine Framework:
To implement this you will have to create a few systems:
- A framework core program or system that ties everything together. Most of these items only need a little work to “fit together” but others need relays, abstraction layers, translation lists/tables, and other things to properly implement. Fitting everything together is really the hard part in developing the first alpha build of the game engine. Once thats done you'll really have something solid to build on.

- A set of standards for all modules such that if another module were plugged into its place you could easily create a sort of abstraction layer instead of having to heavily modify the rest of the framework. This is important for a number of reasons but now that you're building on Realm Forge GDK you can center most of the standards around that framework.

- Tools to help with building new modules: This is more planning for the future so it could be done last but it can be easier to create some of this stuff while your working on the code rather than months later when you might have forgotten parts of it. A middle ground would be to create a set of extremely detailed notes while you're working on the code, that will make fully documenting the code and building tools much easier in the future (it could also allow someone else to come in and do that part of it for you). The actual tools are more code systems that can be used repeatedly or smaller code snippets, in either case its just intended to help with integrating new modules, systems, engines, middle-ware, etc... into the framework and thus the game engine much more rapidly than if you have to sit there and stare at the code for a few hours to figure out where to begin.

- Framework and module documentation. In short this is a ton of technical writing and a critical part of making the game engine open to future improvement. If you take a 'middle ground type of route' (could also call it the fast route) then the notes you take during development to create the tools to build new modules can be used here to create full documentation.

- And a system to plan the development of the different modules. This is where the short cuts I suggested come in. You need to plan for the future but the best way to do that is to literally say: “Alright, this is the idealistic way to make this module perfect (with all sorts of details) BUT this is how to make the module as fast as possible so we can meet our 3 month alpha goal (with all sorts of details) AND this is how someone might go about upgrading our “fast route” module into the “perfect module” (with a some details, idea, and theory).” Then maybe a year or so down the road you should have your “perfect” or “idealistic” game engine.


Suggested Framework Structure:
- Framework Core: client executable
- Framework MP Server Extended Core: server executable
- Framework Low Level Modules: included stuff installed into the OS
- Framework Middle Ware Modules: Realm Forge GDK and other 3rd party libraries.
- Framework High Level Modules: stuff built on or around the middle ware
- Framework Top Level Modules: scripts
- Framework Game Play Management Modules: user interfaces
- Framework Data Level Modules: game data
- Framework Mod Modules: user space modules and mods
- Framework Back End Programs: external programs
- Framework Internet Modules: web server and MP community systems



Edited by - MegaBurn on 12/20/2005 1:56:51 PM

Section 3...

Framework Core:
The framework core is the main game engine executable program, this is the stuff in the EXE file that the players run to start the game. This entire section is all compiled into a single program. It's really only modular in the source code itself and in special developer builds of the game engine. Most of this is OS dependent.

Rules: Anything that accesses operating system resources or low level modules is routed through the OS Access or Low Level Access Modules. This will help prevent or reduce future headaches even if it is a headache now, also helps to keep the code as portable as possible.

- Client Game Engine Core Executable: This isn't a module, its the actual program itself and most of it is included in the Realm Forge GDK.

- OS Access Module: (operating system access module) When you write the core program there are a few ways to call functions and resources from windows, the short way is just to call the function or resource directly, the long way is to create a reference function that includes the windows function you want to use. The reason for doing this is to centralize everything that is dependent on windows to one small part of the program, that way if you have to make a small change to that part of the program later you only have to make that change in a single part of the code rather than dozens of different sections of code. The other important aspect is if someone wants to get this game engine running on an operating system other than windows (e.g. Linux) all they will have to do is rewrite the core program, this OS Access Module, and any of the other OS dependent modules and that saves them from having to go over every and probably rewrite huge sections of program code manually. This module includes stuff like OS specific libraries, hardware identification, ActiveX controls (if you use any), and anything else along those lines that is not part of the Low Level Modules. As I'm writing this I'm trying to route as much as possible through this module to centralize everything that needs to access the OS.

- Low Level Access Module: This is the actual code that loads and references the low level modules. All of the low level modules are third party libraries like DirectX and Open Libraries (e.g. OpenGL, OpenAL, SDL, etc...). The player is required to have these libraries as part of the minimum system requirements for the game and you are not required to included them with the game download but you should include links to them in the game's website, readme, and manual. Note that this module is OS dependent and does basically the same thing as the OS Access Module. So any references in Realm Forge GDK to DirectX or OpenGL would be rerouted through this module. I haven't looked at the Axion code yet but I think parts of it could be moved to this module such that the player can easily change from Direct3D, to OpenGL, to proprietary library, to software render, to null in the game settings.

- Middle Ware Access Module: This module loads and accesses Realm Forge 3D and all other middle ware. Everything that uses those libraries is routed through this module. This setup will make the game a little slower but more flexible. It might be worth while to try to figure out exactly how much slower this is compared to loading those modules directly. Point being by defining all of this stuff here if some part of Realm Forge changes or you pick a different engine, then you can create an abstraction layer here that allows you to literally plug the new libraries in without having to change anything other than this module. This is also a bit of “check stop” for anything thats OS dependent, it would make porting the game to another operating system a little easier. Again this will slow down the game, maybe not by much but if you tried to plug-in a new graphics engine by adding an abstraction layer to this module there would be a big performance decrease. Another thing to note here is you can use this to test different engines, if you can calculate the rough performance loss in the abstraction layer you can get a feel for how fast another engine is when you plug it into the rest of the game engine (could be very useful in the future if you want to try out different render, audio, or other engines).

- Error Control and Debug Module: This is fairly self explanatory and there should be snippets of code or even full libraries included with Visual Studio to do most of this work for you. The error control stuff can range in complexity from really simple (e.g. logging the error so it can be fixed manually) to extremely complex (e.g. trying to get the program to actually fix the error on the fly). If you take the time to define this module within the source code then upgrading it from a very simple system to a rather complex system will be much easier in the future but again, for starters just use a very simple system. Another thing to consider is breaking this up into multiple modules, one module to handle the top level modules, another module to handle the high level module, another module to handle the low level modules, and a main module that both binds everything together and handles the core level debug/error control. Keep in mind that all/most of the middle ware comes with its own internal error control and debug systems, so rerouting the output from all of those little debug/error control systems to a few different modules and then rerouting it again to a centralized module might make tracking down the errors a little easier (kind of like “do you want to spend more time developing a complex error control system now or spend more time hunting for errors later”.

- Engine Console Module: This allows you to enter commands directly to the engine, for starters this should include a few shortcuts to make entering commands a little easier and a “help” command. Then later on this can be expanded into a full command line interface with support for scripts (batch files) and all sorts of other things. Some cheat controls need to be added here for multiplayer but should be fully functional in single player.



Edited by - MegaBurn on 12/20/2005 2:03:12 PM

Section 4...

Framework MP Server Extended Core:
The extended server core includes everything in the main core (above) plus everything in this section, this is the multiplayer server program. Note that some of the server modules are a little different so this is not just redundancy, its special considerations for the server. Along those lines most of Realm Forge GDK needs to be ripped out, the server doesn't need probably 75% of the GDK and leaving that extra stuff in there will only slow it down. This entire section is compiled into a single program except for special developer builds of the game engine that have each module in its own DLL.

Rules: Anything that connects to the network is routed through the Server Security Module. As with the core module anything that accesses the operating system or low level modules is routed through the OS Access or Low Level Access Modules. This will help prevent or reduce future headaches even if it is a head now.

- Server Game Engine Core Executable: This is not a module, it is the executable program that people run to start the multiplayer server. It replaces Client Game Engine Core Executable. This is OS dependent.

- Server Security Module: A special security system that protects the server against hacking, cracking, and viruses. Everything sent or received from the network must be rerouted through this module (even then security wholes will be a problem), this module connects to the Low Level Access Module to gain network access. All multiplayer servers run firewalls, some run multiple firewalls, but none of the firewalls can reliably inspect the data being sent to and from the server, so we need to include a system to give server admins that extra level of protection. This isn't critical to making multiplayer work so it can be fully implemented later but we need to create a “pass through firewall” from the very beginning, thats basically just a relay that blindly sends and receives anything sent to it without inspecting the data. Not including this data relay system now will make adding a full security system much harder in the future. Server wide passwords, player username/password accounts, and player ID authentication keys can be managed by this module too.

- Server Network Error Control Module: This is a very small module that warns the server and the player about lag and is a feature in Freelancer (that little red lag icon that pops up at the top of the screen). The code for this is fairly simple for starters but plan on expanding it in the future to include more advanced features like logging or even fixing errors in the data.

- Administrative Link Module: A program link via the OS Access Module to the framework back end admin modules for server management. It also links to the server monitor program. The best way to explain this is to look at how Ioncross works, they added a DLL to Freelancer that allows the server manager program to control the server. That setup has major limitations because it was developed after Freelancer was released. By including it in the server version of the game engine you can get around all of those problems and give OL one of the best server management systems of any game out there (except maybe MMOG's). For starters you can leave this module mostly empty until there is enough game content to actually run a public MP server. Another option is to just leave this section open for the people who created the server management tools for Freelancer, they should be able to rewrite their existing management programs to work with OL. In the future a link to the Server MP Network Module can be added so admins can connect to the server as “admin players” with some rather extreme in game (“god-like” abilities. Note this module does not handle network connections directly, an external program with a self contained web server and SSL wrapper handles all network connections with access to administrative functions.

- Administrative Management Module: Handles all administrator functions. I moved this from an external module to an internal module because that should reduce system load on the server (if only by a faction of a percent). This module handles all admin functions for the Administrative Link Module, Server Link Module, Cheat Control Module, Server Security Module, Server Internet Network Module, Server MP Network Module, and the Automated Administrator Module.

- Automated Administrator Module: An admin system that runs on top of the administrative management module. This is kind of out of place in this section and while it seems like should be part of the administrative management module or the external admin modules, it would be better to have it as another module so it can be developed in future version. For starters this is just a series of scripts that can kick cheaters, flash global server messages (console messages on FL), track server stats, and handle a limited amount of general server business while the admin is offline. As an interface to a scripting system all script functions would be defined here. It also needs to link to the Administrative Link Module so admins can enable, disable, and write scripts. In the future we can replace this with a more dynamic system built on one or more AI bots. These bots can be used as programmable server police NPC's and other things during server events but can also semi-intelligently deal with normal admin actions. The link to the Administrative Link Module will allow admins to connect to the server as “admin players” and run more complex external programs.

- Server Link Module: For connecting multiple servers together, it connects to the network via the Server Security Module. For starters this module can be left mostly empty. Then after a dozen or so servers are online you can go back and create this module. I'm still not sure what the best method is to make this work but there is a lot of software available for small server banks and we might be able to use that or maybe a virtual desktop type of system. It also depends on how we want the servers to be able to link together. The options are splitting solar systems up among different servers (fixed load balancing), making the servers run in parallel with each other (parallel processing), splitting the players up between different servers (dynamic load balancing), splitting the game engine modules up between different servers (distributed processing), or making the servers do all four automatically (dynamic load management). The players/clients will be doing most of the really processor intensive game play stuff so there is little to be gained with some of these server link options. Beyond that a lot more research needs to be done on this module.

- Cheat Control Module: Basically does what it says, tries to stop people from cheating. This will be an issue as soon as the first MP server goes online, theres just something about some people that makes them cheat if they can possibly find a way to cheat, and it will probably be an on going problem during the entire life of the game. The real technical problem and the reason we can't use the old hash system, is that hash codes don't work with items that are created dynamically on the client side and I think any attempt to make it work or implement encryption would be way too processor intensive. So we need to use this module to develop a cheat control system thats heavily tied to the top level Dynamic Creation Engine Module. I'm still researching this but off the top of my head I think some sort of a variable control system might work, since the server must keep tract of the player character's inventory it would have access to most of the variables used during the item creation process, so it could send that data to the player. Then the player's dynamic creation engine would create the item and return the final item information to the server with all the random variables used to create the item, if the random variables are consistently statistically too high to be truly random it would flag the player as a possible cheater. Thats simple enough for the dynamic stuff but will be defeated or at least exploited in time. For starters you can just use the old hash code system in this module but plan to upgrade to something that supports dynamic data and possibly open source middle-ware. All this module has to do is identify the cheater and flag their account as a possible cheater with an explanation for the flag, like some sort of cheat ID code, then the administrative management module can take action.

- Server Internet Network Module: This is the SQL client module of the SQL stuff I was talking about before. It connects to a SQL server running on another computer, 3rd party web server, or on the same computer as the MP server. This should be rather easy to implement, its just building an open source client into the server with a few modifications. The data its handling is stuff like server stats, new news/rumors, and other stuff. For starts just include the SQL client and worry about actually using it later. There are a few future expansion options here and on high power servers with a lot of bandwidth a full SQL server could be built into replace this module. This module connects to the Server Security Module for network access, it doesn't connect directly to the network.

- Server MP Network Module: This handles sending and receiving data with the players, it connects to the Server Security Module. Its a straight up high speed client/server system and its rather complex. In short this module needs to take all dynamic (real time) game data being sent out to players and encode it into a very abbreviated set of codes and numbers and vise versa with data coming in from players. It also needs to handle error controls, resending some data, some higher level data flow control (data volume), and a number of other things. Links (same as MP client): HawkNL, OpenTNL, RakNet, GNE. Out of those RakNet and OpenTNL look the best, both are used in commercial games, you guys will have to research this in more detail.

- Game Data Server Module: This is split off of the Server MP Network Module because they do two different things. This module serves game data to players and transfers incoming game data to the Game Data Interface Module. It runs between those two modules. So for example, when a player enters a new system this module would send the player everything they need to get in 'sync' with the real time activity in that system. This can be rather simple with the INI system but there are some technical headaches with the possible volume of data involve in the dynamic modules.

- Game Data Server Cache Module: High speed memory cache, needs to stay in RAM and maintain a buffer of whats happening in each system. The cache won't be very big at first because there isn't a lot of data being produced for each system, just object ID's and locations (mainly ships and missiles/mines). Once the dynamic systems are enabled this could turn into an issue as it will eat up massive amounts of memory.

- Game Data Interface Module: The interface to all game data, including the INI system your working on and contains all code to read the INI files. A blind relay for graphics and audio data should work for the first few versions. In the future this can be converted into an advanced database system so that dynamically generated graphics data can both accessed and stored from this module. This sits directly on top of the Game Data Server Module on the server version of the engine since it'll be sending as much data to the players as the server is using itself.

- Game Data Server Data Module: This is split off of the Game Data Interface Module, but it sits directly on top of the Data Server Module. It is a database system that stores any data that should not (or really cannot) enter the normal flow of game data. This includes all player account information and anything else that needs to remain securely stored within the server. At the very least some encryption or hashing should be done for all passwords. If security ever becomes an issue, for server admins or the players, we could implement a rather high security setup modeled after online e-commerce systems (like just about anything that sounds excessive or just makes you say “why?”. Another thing to think about is system and base specific passwords or access codes and user/group access control systems, these can be used by admins, players, and factions (clans) to restrict access to systems or bases. Example: A secure base could have a set of access codes required to dock, these codes could be available for sale from information dealing NPC's or as a reward for a mission but can also be shared openly between players. A faction could set a username/password on their home base that can be shared between players. An access control system could restrict docking access to only specific players with permission to dock. This is basically the same sort of security a typical forum system has. Anyway, point being, that data should not enter the normal stream of game data and should be stored in this database.

- VoIP MP Server Module: The MP server's VoIP client. You can used this for voice mail messages and all sorts of other things. There are a large number of options here but beyond just including the VoIP client the server needs some scripts to manage it. HawkVoice should be able to handle most of the complex stuff and allow you to build script functions. Then you can build on that in future versions with all sorts of other features.



Edited by - MegaBurn on 12/20/2005 2:09:04 PM

Section 6...

Framework Middle Ware Modules:
Everything in this section is loaded by the core program or runs on top of the low level library interface modules. This is all built around Realm Forge GDK so anything included should be considered any add-on/plug-in to Realm Forge GDK or something that operates independently of Realm Forge GDK (depending on how you compile it).

- Realm Forge GDK: I think we covered this in the other forum thread but this is now the guts of the game engine. Covers almost everything but leaves plenty of room for more stuff. Links: Realm Forge, 3rd party libraries used in the GDK

- Control Interface Module: Handles the “response side” of user input from HID's (human interface devices). Basically what happens what you press a certain button or move the joystick around. Mainly triggers user interface functions/actions, camera movement, and script functions. Realm Forge GDK covers some of this stuff but I'm really not sure how much, the references to Direct Input and SDL HID are buried in their wiki, so this may need some development. Freelancer uses an excessively complex 3 INI system, a simpler single INI would be much better. Support for multiple devices is critical. Also support for some more “free form” or custom device configurations would be nice. Force Feedback is another option but I'm not really sure how to make that work, might be able to do something simple with the physics engine but regardless the libraries are freely available directly from the companies that make the hardware so it shouldn't be hard to figure out.

- General Physics Engine: The engine in Realm Forge GDK should suffice for most things but from what I read the ODE physics has some speed and accuracy issues. Take a look at the possibility of upgrading to the Newton Dynamics physics engine for more realistic and probably faster physics.

- Math Libraries: Just stuff to help with general math and/or extend the physics engine. ORSA , its a scientific level orbital mechanics library used to model the path of real objects by real astronomers, the problem is speed and I have no clue if it would even be possible to use it (its also extreme overkill). Another option is libnova but it contains specific data for our solar system, since its GPL that could be removed if the algorithms/formula are useful (or just ignore the extra data). Sharp3D Math covers some basic stuff that could make coding math intensive functions must easier.

- Text-to-Voice Engine Module: A data converter engine that converts text into audio. This should be possible to implement and these systems have gotten much, much better in recent years. It'll still sound like a computer in long conversations. To make this work a translation matrix might have to be added that works like a spell check, when a word comes along that isn't pronounced in the same way it is spelled the word will be replaced with a special spelling of the word that the text to voice engine can read (or in short it'll make the voice sound more natural or less like a computer). This can be added later when the rest of the dynamic systems are developed.

- Dynamic Dialog Engine: A centralized text dialog generator that produces dynamic dialog based on in game conditions. This can include radio chatter, normal NPC misc dialog (e.g. yapping with NPC's), and merchant dialog. This text is then sent to the text-to-voice engine for conversion into audio. To implement this you'll have to look at some of the AI chatter bot engines available and pick one that works off of scripted dialog templates. The dialog templates allow the engine to pull data from the rest of the game data to make the dialog seem like the AI knows more about what its talking about than it actually does (e.g. seemingly intelligent dynamic and detailed dialog). Another option is to include both a scripted/template driven engine and a neural network based engine. The neural network takes an extremely long time to train into something that can carry on a normal conversion but it would allow NPC's to have free form conversions that do not conform to any template but can use templates for certain subjects (or lines of dialog). This can be added later when the rest of the dynamic systems are developed.

- Game Data Interface Module: Manages all game data. While idealistically all game data should pass through this module but that adds a noticeable performance hit for stuff thats just being blindly relayed. This will start off as your INI system and then switch to a multilayer database system. Keep in mind when deciding which formats of data to route this module that you can store small bits of graphics, audio, and other data in the INI files and later in the databases. That would be well suited for storing the lego-like “building blocks” of any dynamically created items/objects, it may be faster to store short/small sound effects in there too.

- MP Client VoIP Module: The player to player voice over IP communication system with related codecs, filters, and scripts. The HawkVoice library should handle most of the hard/complex stuff so you can just add script functions and a mappable interface function to make it work (e.g. the button you press to talk). Note: Take a very close look at RakNet, it also includes a VoIP system that uses the same codecs as Team Speak and seems like it would be far easier to implement.

- MP Client Messaging Module: Handles all communication between the players. This is a simple text messaging system but it also needs to include a way to send commands to the server and to other players. If you want to enable player to player music play-list sharing as a sort of radio feature, this is where you would add the function that actually sends the play-lists.

- Client MP Network Module: Realm Forge includes parts of a client/server system but I'm searching for something better. This is the client side of the multiplayer system, if sends and receives game data with the MP server. Links (same as MP server): HawkNL, OpenTNL, RakNet, GNE. Out of those RakNet and OpenTNL look the best, both are used in commercial games, you guys will have to research this in more detail.

- Client Internet Network Module: The SQL client I was talking about. This allows users to pull and post data from forums and other systems. Using an SQL client alone should be possible but it will require a display system. The display system should be added to the Neural Net Programs and format data so it looks like a normal forum setup, news/rumor submission form, community news page, etc... Different Neural Net Programs could support just about any text formatting. It also needs to read game data and the users account information for personal logs and other information to be posted in the SQL databases so people can read it online. Including a full blown web browser is an option but it will bloat the engine and probably have problems than its worth.



Edited by - MegaBurn on 12/20/2005 2:20:38 PM

Section 5...

Framework Low level Modules:
This is the low level stuff that pretty much makes everything else work. It is all routed through the Low Level Access module so don't call anything directly from these libraries, put the reference in the access module so if you have to swap out one of these libraries in the future you don't have to change all references to it.

Rules: Everything is relayed through the Low Level Access Module and nothing is called directly from any place else in any of the code.

- Local Data Access Module: This is what actually does the local data access and is operating system specific. The only reason this isn't lumped into the OS Access Module is because in future versions it could be replaced with things like a RAM disk or maybe even a distributed network storage systems. For starters you can use the VS.net data access class, SDL file access class, DirectX file access, something directly from windows, or maybe Realm Forge has something else included. Allowing the player to switch between these would be a nice feature.

- Network Access Module: Like the Local Data Access Module this is just a network access library. Should be included in VS.net resources to access window's network devices. In the future this could be expanded to include virtual network interfaces or any number of other things...but for now you can use a class included with VS.net, SDL net class, Direct Play, maybe something included with Realm Forge, or something included with windows. Allowing the player to switch between these would be a nice feature.

- DirectX Modules: Basically a hardware abstraction layer driver package. It creates a standardized platform on top of PC hardware so that games don't have to access that hardware directly. The term “direct” is kind of a misnomer because that is the exact opposite of what “Direct”X really does. The DirectX SDK is now advanced enough that it can be used as a graphics engine by itself, if that doesn't say bloated drivers I don't know what does. Anyway, make sure you download the DirectX SDK from micro$oft's website if you haven't already.

- Open Library Interface Modules: Roughly the same thing as DirectX but distributed across a dozen different organizations. OpenGL is the best known of the libraries but it includes about a dozen different system files. Technically OpenAL is not included but I like to think it is for some reason. Links: OpenGL (lots of resource links including links to full free PDF development books). Simple Direct Media Layer ( SDL Wiki ). OpenAL.

- Proprietary Library Interface Modules: While DirectX and Open Libraries are designed to create a hardware abstraction layer you can bypass them to either increase speed or add support for hardware not supported by either abstraction system. This is by no means a requirement but you should include place holder for this in the Low Level Access Module, maybe with a comment saying something like “// proprietary code goes here, see coding news for details”. Then add a code section to the development docs with a template for adding more interface libraries. Always remember that abstraction of any kind, be it in hardware or libraries or data, will always be slower than direct access but is much easier to manage if you have to change what you are accessing directly. In the early days of game engine development developers were forced to write code for every single hardware product they wanted to support and then maintain that code via game patches as new hardware became available, just be glad you don't have to do that now...that is a prime example of the power of abstraction layers and why I'm stressing the OS Access and Low Level Access Modules. Anyway...Links: nvidia developer center (tools, SDK's, dev docs), ATI developer center (they have some useful stuff if you can find it in that mess of a site). Intel Game Developers Center, Intel Open Source Developers Center. AMD Developers Center. Creative Labs Developer Center (SDK's, tools, docs). Logitech Developers Program (tools and docs).



Edited by - MegaBurn on 12/20/2005 2:14:44 PM

Section 7...

Framework High Level Modules:
Stuff that runs on top of everything else.

- Script Function Modules: Defines all scripted actions/functions that can be called. This would just include the program code side of the script functions, not script functions that are themselves scripts (e.g. does not include “class scripts” that are intended to be called by other scripts as script functions, just real C#/VB code). Creating this module will require finding all functions in Realm Forge GDK and all add-on libraries and then compiling them or references to them into a single library. This is like many other pain in the ass 'bulk work' items that in the long run will make life easier. With all of your script functions centralized in a single library you can more easily build on that library and it should make documenting the code a little easier. After that you can create new functions that do really what ever you want them to do. The scripts are game data and in the Scripts Module.

- Dynamic Economy Engine Module: Manages and tracks the economy. It can adjust prices, tracks the prices of each item, tracks the production rates of each item, and can add/remove both items and money from the economy. If you include a company and/or faction stock market that would also be handled in this module. For starts just use a static economy like Freelancer and worry about adding this later. I'll post a more detailed dynamic economic model and suggestions for both building and controlling it.

- Dynamic Creation Engine Module: Manages production of all dynamically created items, including commodities, equipment, character bodies, vehicles, ships, bases, planets, and systems. I strongly suggest you start with a non-dynamic system that uses the same INI system as Freelancer and then worry about the dynamic stuff later. Because of the large amounts of data generated by this system a database game data storage and management system will have to be in place before the dynamic systems are enabled. I'll post more detailed descriptions of how the creation engine could work.

- Dynamic Mission Engine Module: This uses dynamic faction AI to create a mission and/or mini campaign. This can get extremely complex. I suggest you use scripted missions for starters and worry about enabling the dynamic stuff later. I'll describe this in more detail in another thread.



Edited by - MegaBurn on 12/19/2005 12:33:49 PM

Section 8...

Framework Game Play Management Modules
The game play can be radically changed by changing user interfaces. Existing interfaces defined in the design doc are flight mode, character mode, and a few menu interfaces (market, item creation, etc...). I'm expanding on this with a system to manage a wide range of different interfaces. Remember that the user interface has two main parts: the on screen heads up display (HUD) also known as a graphical user interface (GUI), and the human interface device control map (HID map) also known as a control or key map (keymap). I'll continue to note these two items to stress their importance (also because I wrote everything below before writing this paragraph and I don't feel like deleting anything). Finally, this is all just HUD changes, control map changes, and camera changes – or in other words it is very easy to setup but sounds far more complex than it really is to implement.

- Main User Interface Management Module: The “master control” for all under interfaces or game play modes below. This allows the user to manage global HID maps (keymaps) and at the same time handles the controls for the F1-F12 function keys, print screen, pause/break, and any other keys set to use global functions that are always available regardless of which game play/user interface mode the player uses. Note: Heres the home page for “Crazy Eddie's GUI system” Realm Forge GDK. That site includes a large number of useful resources (really a must read).

- HUD Mod Interface Module: This is the interface for the player defined “widgets” that display various bits of information on their screen. They're just like any desktop widget with a few limitations, they cannot display game data not in their Neural Net because that might give them an unfair advantage over other players. They can display their system time, server time, another computer's time, SQL data (forum activity, server rankings, user activity, etc...), RSS data (literally anything from local weather to a stock ticker to headline news), and maybe a few other things. The display method can include a wide range of options but probably something like allowing a dynamic box and/or text to be displayed anywhere on the screen even if it covers up another HUD/GUI object. The data going into the box/text displayed would come from a script in their profile and could be connected to a Neural Net Program added by the player. These programs and HUD/GUI widgets could also be used to allow the player to display Neural Net information on another screen (like a 2nd monitor or HID/PC-case LCD), display the state of their caps lock/scroll lock/num lock keys, or to display the mode of their HID device (e.g. the high quality keyboards, mice, and joysticks designed just for gamers who aren't worried about money). Note: This module is intended to support future growth and modding of OpenLancer and (probably) will not be used in the standalone main game, it can be left out until an SDK is released and modding is fully supported.

- Dynamic User Interface Mode Management Module: A special user interface mode that allows the different game objects to determine how the interface looks and functions. Basically with each user controllable game object definition (e.g. ships) it will reference a file that contains the settings for that game object's user interface including its HUD (GUI) and HID (keymap) settings. The SDK for this module should include templates for small car-like vehicles, medium size APC-like vehicles, large tank-like vehicles, straight aircraft (non-spacecraft), VTOL aircraft (helicopters), and a copy of the normal flight mode. Unlike normal flight mode and the other game play modes the player will be locked to a specific main mode but will be able to use the sub-modes (e.g. they cannot switch between flight and turret modes but can use the sniper or remote sub-modes of game play). The main point here is to allow modders the most flexibility possible without forcing them to edit the game engine. Ultimately it would be very cool if each ship could have a completely customized user interface profile but to keep it easy to play these interfaces could be designed around common themes (e.g. different houses could have a different graphic style/theme HUD/GUI while different classes of fighter could have a different HID/keymap). I will add templates for each weight class of BattleMech (utility/ultra-light, light, medium, heavy, assault, and quad). This is part of and sits directly on top of the user interface module in Realm Forge GDK. Note: This module is intended to support future growth and modding of OpenLancer and (probably) will not be used in the standalone main game, it can be left out until an SDK is released and modding is fully supported.

- Dynamic User Interface Sub-Mode Management Module: Like the Dynamic User Interface Mode Management Module this is a highly customizable interface system to allow modder the most flexibility possible without forcing them to edit the game engine. As a sub-mode of game play this can be used for custom made mini-games, custom weapon controls, custom remote controls, custom command modes, custom turret modes, or just about anything else. Each dynamic sub-mode will have to be defined by the dynamic main mode that uses it or by a game object that uses it or maybe by a script function that can trigger it (or in other words it must be linked something so players can access it).. Templates for turret mode and the sniper, remote, and tactical command sub-modes should be included with the SDK for this module. This is part of and sits directly on top of the user interface module in Realm Forge GDK. Note: This module is intended to support future growth and modding of OpenLancer and (probably) will not be used in the standalone main game, it can be left out until an SDK is released and modding is fully supported.

- Flight Mode Management Module: A user interface plug-in that displays and manages the flight or cockpit mode of game play. It defines the graphical user interface (GUI), HUD functions (on screen data display and input controls), keymap/HID controls, and anything else that the player will only see while they're in flight mode. This is basically modeled after Freelancer's controls, screen, and look & feel (except it supports joysticks). This is part of and sits directly on top of the user interface module in Realm Forge GDK.

- Character Mode Management Module: A user interface plug-in that displays and manages first/third person mode game play. It could also be called the first or third person shooter mode (FPS/TPS mode). When switching from flight mode, it would change the GUI/HUD, keymap/controls, and other game play mechanics to fit FPS/TPS mode. The thing to remember here is that the player's body is just another vehicle, it has its own engine, equipment mounts, cargo hold, and anything else that a ship has. The difference is it has more complex animation that changes in speed depending on the speed of the player. The physics engine can be used to determine most of the speed variables in the animation. The character creation system is used to allow the player to build their character's body just like they would build a new ship. The player's inventory or cargo hold would need special mount slots that allow the player to 'mount' backpacks or other storage containers for a larger cargo hold. Weapons also need a special handler in the animation, when the player's weapons are offline they need to be shown as being holstered. Finally, like ships, they need to have special upgrades for armor suites, space suites, and other things that change the body's model and stats (e.g. stats meaning armor, power, speed, cargo capacity, damage filters, weapon mount point types, maybe personal shields). Mount points are dependent on the body type and different armor/power assisted suites, like plain clothing has a base line mount point for a normal small arms item (pistol/rifle) but a power armor suite could mount a heavier weapon up to and including a class 1 ship weapon. Do this right and the game will attract a wide range of FPS/TPS fans, not to mention it'll seriously kick ass! This is part of and sits directly on top of the user interface module in Realm Forge GDK.

- Strategic Command Mode Management Module: Manages game play during “RTS mode”, this will take some work to create and perfect. Basically you need to be able to use a special enhanced nav map that shows the location of units with the ability to give them orders. At the same time you need to be able to order around the command vehicle. I think this should be modeled after the command interface for Home World 2 or Nexus but without any of the construction support. So players will be able to go to the bridge of a capital ship, enter a holo-tank or command terminal, and then order their ship around, order fighters around, and generally do what ever else they want. The really freakishly cool part of this is its realistic to how being a capital ship captain would be. This is part of and sits directly on top of the user interface module in Realm Forge GDK.

- Turret Mode Management Module: The turret mode counter part to the Flight Mode Management Module. The only difference is the flight characteristics, like turret view in Freelancer, they would have to use a keyboard or joystick to move the ship while they're using the mouse to move/fire the turrets. Some special functions can be added to make this more useful than turret mode ever was in Freelancer. For one, the ability to zoom the camera in and out, switch the camera to different centered positions (so it rotates around a different part of the ship), and a more direct turret control that allows the player to use the capital class main cannons like a huge sniper rifle (see weapon mode below). Add a detailed diagram of the ship, each ship section, each ship system, and each weapon/turret including a status icon, damage bar, power bar, and maybe other little details. Also add a details box that allows the player to click on any ship section, system, or weapon to get full details about that item. This will require some complex user interface design and camera management but isn't nearly as difficult as it sounds. The end result will make capship players extremely happy (myself included)! This is part of and sits directly on top of the user interface module in Realm Forge GDK.

- Sniper Sub-Mode Management Module: This is a special camera mode accessed by the each of the other game play modes. It allows the player to zoom in/out and fire the selected weapon. On fighters is can be used to fire 'big guns' like Evo's Long Beams or Gauss Rifles (or railguns, or other such weapons). In capital ships it can be used on the main guns like the Liberty Cruiser Forward Cannon or other massive long range guns. In character mode (FPS/TPS mode) it'll be used for any weapon that has a scope. It will require a weapon setting that says the weapon can or cannot be used in sniper mode (like the INI settings for “auto-turret = true/false” in Freelancer). Any weapon that has this INI setting set to true will need a special GUI/HUD file that displays the target reticle (cross hairs), range to target, IFF (friend or foe), and any other details about the weapon (e.g. weapon status, power level, ammo level, heat level, etc...). It also needs to include a camera range setting for movement range, movement speed, and zoom range. This is part of and sits directly on top of the user interface module in Realm Forge GDK.

- Remote Sub-Mode Management Module: Like the Sniper sub-mode, this is another special camera mode intended to be used on top of other game play modes. This mode is special as it gives the player camera control, movement control, and a few input controls on remote game objects. These objects can include missiles, mines, robot drones, weapon platforms, robots, and maybe other normal game objects. This would require a few extra settings for the game object the player would be controlling but its basically the same stuff found in normal ship settings, like camera control and a switch that enables/disables remote control (like the auto-turret true/false INI setting in Freelancer). Most of the movement range and other settings should already be there, another setting might be remote control range that would limit the distance the controlled object can travel from the player before they lose control of the game object (e.g. radio range or something like that). There should also be a self destruct control in addition to the minimal movement controls. I think manually guiding long range missiles, controlling weapon platforms, and little robots would be extremely cool. Another option is to allow the player to control normal game objects like ships by hacking into them, this would require some extra settings used to create a small hacking mini-game. This is part of and sits directly on top of the user interface module in Realm Forge GDK.

- Tactical Command Sub-Mode Management Module: Another user interface add-on that allows the player to issue commands to other characters (NPC or PC) based on mission or faction commands. Like the strategic command mode it would have to allow the player to issue movement orders based on a system, base, or planet surface map. But unlike the normal command mode they couldn't control their own movement or issue very complex orders. It should also have a toggle to allow the player to take control over remote control units around them, such that if a robot is in the area they could tell it to go to a certain location or do a certain simple task but then take manual control of the unit for more complex tasks. This could be tied into the hacking mini-game as well so that the player can order around robotic unit they have forcefully taken control of (don't want to get too carried away with adding robots though). This is part of and sits directly on top of the user interface module in Realm Forge GDK.

- Market Sub-Mode Management Module: Allows the user to buy and sell commodities, items, ships, etc... Basically a combination of the commodities, equipment, and ship dealer windows from Freelancer but would include a management module for bases the player owns. Allowing them to set prices for their goods or set conditions for other characters taking items that they own (e.g. faction store management). This module would be accessed any time the player talks to another character or visits a command terminal at a base that they own – or something to that effect, there are a large number of ways to implement it. This is part of and sits directly on top of the user interface module in Realm Forge GDK.

- Item Creation Sub-Mode Management Module: A user interface module for creating new items. This one is probably the most complex of all of the interface modules since it will have to handle accessing different item templates/designs, user created logos, user created paint jobs, and other just files. This includes building everything from new items to equipment/ship components to even their player character body. The actual player created art assets should be left to an external editor so they're not spending hours needlessly connected to the MP server tweaking logos, paint jobs, or designs (note that they can still be connected via a back end module so they can text message or VoIP chat with their friends on the server). Since items are created in game dynamically this is one of the two “front ends” to the Dynamic Creation Engine Module and also since that module will probably be left “under developed” in the first few versions of the game this user interface module can be left as more of a filler/place holder until the dynamic systems are fully implemented. This is part of and sits directly on top of the user interface module in Realm Forge GDK.

- Neutral Net Sub-Mode Management Module: The user interface module that contains the Neural Net as defined in the design document. This is a special user interface module that can load user created programs. It also contains the user's log/journal, SQL/RSS display system, other internet data display systems, game data displays, and various other windows. To streamline access this should be universally mapped to a single access key, like F12, on all game play modes. Internally the fully range of function keys F1-F12 can be used as short cuts to the different windows.

- Communication Sub-Mode Management Module: The user interface module used to manage communication between players during multiplayer game play. This must include text entry box, text display box, server text/VoIP channel display box, faction/team text/VoIP channel display box, an options panel, and security panel. The server channels can be determined by the admin in server back end management system but should include a few defaults: global, systems, player group and private player channels (player and group channels should be automatically created by the server). The options panel is used to control VoIP codecs, VoIP volume, ignore players, set “buddies”, text display formats, and change various other settings (all of these settings should be mirrored in the game options). The security panel is used for dockable game objects that require a username/password, security/authorization code, or other security check before the player can dock with that game object. A text, voice, and video (render) mail system could be added to another panel or included with the Neural Net Sub-Mode Management Module.



Edited by - MegaBurn on 12/19/2005 6:22:43 PM

Section 9...

Framework Data Level Modules:
This stuff is accessed by the Game Data Module, Audio Engine, and Graphics Engine. Most of this is rather vague organizational suggestions and can be implemented in any of a dozen different ways. How ever you choose to handle data management there is one thing to keep in mind, the data storage system will change in time so be very consistent with the different data types and formats. Try not to add too many (if any) special formats for INI's, models, or audio files (e.g. don't add a radically different INI format thats only used in a few spots). Just be consistent and converting everything from the current file formats to databases will be much, much easier in the future.

- Script Modules: A collection of all standard/global scripts used in various parts of the game and can be called by missions, campaigns, and objects with special functionality. Its like a central script repository or library, just a single folder full of INI or XML files. Later all of this can be compiled into a database for much easier and faster access.

- File Lists Module: Like Freelancer any time you have to reference the location of a file in the program code use this list and nicknames/ID numbers to enter the location. It should be an index of all game data files in the game. The reason for doing this is if the program code points to this list it can be more easily redirected to a database or other data source later on. Also if a file name changes then you only have to change the file name in this list rather than in the program code (e.g. if a modder changes the name they'll have to note that change here, without this they would have to note the change in the program code and recompile it).

- Game Balance Data Set Module: A database of the legal minimum and maximum values permitted in multiplayer. This is used by the server to determine if something is “legal or illegal” (illegal meaning the player is cheating). A simple way to easily enable a cheat control system is just sync this data set with the server so that the player/client side is actually checking if the items are legal or not but the server controls the data used in that check. The data set can be put into INI or some other format but will later be put into a database.

- User Interface Data Module: Contains all data used in the user interface. Defining this data here is only important if you want to worry about allowing the game to be translated to other languages. I strongly suggest you include this as a general feature but that does add a lot of extra “busy work” to the project. I'm sure you can find a few international TRL members who would be more than happy to help with this part of it so that other people can do the translation.

- Name Data Module: The replacement to the IDS system. While the IDS system in Freelancer was rather ugly to work with it did have one strong point, it made translation to different languages easier. You can use DLL's again to recreate the same IDS system or a simple name, nickname pairing system where instead of IDS numbers you have nicknames. Example: Instead of ids_info you would have info_nickname and ids_info1 becomes info1_nickanem, ids_info2 becomes info2_nickname. The ids_name entry would be removed completely and you would just use the normal nickname entry. Then in this Name Data Module would be a file for names with lines like: “nickname = name string” (ex: “li_elite = Defender”. The info entries would be the same with lines like: “info_nickname = info string” (ex: info_li_elite = Liberty's Heavy Fighter blah blah blah....”. This will be pretty easy to both translate into different languages and convert/import into a database later on.

- Campaign Modules: All campaign related data for story missions. Basically the same setup as Freelancer, except one level higher on the file tree. So instead of having a folder named missions with each mission in a sub-folder, have a folder named campaigns with each campaign in a sub-folder, and each mission in a sub-folder (e.g. game folder -> data -> campaigns -> campaign folders -> mission folders).

- Mission Modules: Scripted missions that are too complex for a dynamic/random mission generator. This could be a loose clone of the random mission system in Freelancer until we can get a real dynamic mission engine built. After the dynamic mission system is built it could be used for missions that are too complex to be dynamically generated in real time.

- Text-to-Voice Pronunciation Data Modules: Used by the text-to-voice engine to help it pronounce words more naturally or rather so it sounds less like a computer talking. Basically its word pairing, when a word comes along that isn't pronounced like its spelled it will replace the word with a string value that “sounds out the word” for the computer (e.g. dropping silent letters, ex: light = lite, chris = cris). This should be freely available for download (I'll add a link if I can find one). This can also be used to add an accent to words for different language dialects (e.g. Bretonians sound English and Rheinlanders sound German). This actually goes way beyond word conversion but thats the only thing we have to worry about here, the full pronunciation data should be included with the text-to-voice engine and includes all language rules and anything/everything else to convert the letters into sounds. This module should include all of this data and be setup so people can add other languages if they want to (suggested folder structure: game folder -> data -> ttv -> language folder (e.g. en for english) -> language/pronunciation files).

- Dynamic Dialog Data: Used by the dialog engine as a word bank, anything in this module can be spoken by the NPC's. If a neural network driven AI dialog engine is included the data matrix for it would also be placed here. This should be broken up by subject. (suggested folder structure: game folder -> data -> dialog -> word data files).

- Audio FX Modules: Shorter/small audio files including weapon sounds and all other audio FX. Later on most of these could be moved to a database for easier/faster access than reading from the normal file system. (suggested folder structure: game folder -> data -> audio -> FX -> FX type (e.g. gun, engine, etc...) -> audio files).

- Audio Modules: Longer/big audio files including radio chatter and anything else that can be used just about anywhere and is not specific to a certain mission or campaign. Kind of a misc audio dumping spot. These files should be encoded with something like MP3 or Ogg. (suggested folder structure: game folder -> data -> audio -> audio type (e.g. dialog, noises, etc...) -> audio type (e.g. specific NPC, noise type, etc...) -> audio files)

- Music Modules: Audio files played as music. Could lump everything into a single folder or break it up by flight, danger, and battle music types. If its split up you should add a folders for bars, bases, and special/misc music. I'm sure you guys know what you're doing here but there is a lot you can do with scripted music and music zones, both in systems and on bases. You might want to consider adding a “discovery” music type for when the player finds something special (e.g. wrecks, rare loot, new system). Another special music type could be good to set the mood for when they find something or get something new, like when they buy a base, the music zone around it could be set to something that sort of adds to the “wow, thats my base” feeling the player has (or should have). It could be scripted to play once.

- Player Profile Module: The player's profile folder. Contains all settings and related data, including HID control maps (key map), display settings, network settings, music play-lists, chat logs, buddy lists, etc... Should also contains a programs folder for their Neural Net programs, a logo folder, and a faction folder. The faction folder will contain a faction settings file, logos, and other data that auto updates to their Neural Net so clans can easily get someone up to speed with their faction data (rules, base seucirty data, logos, etc...). By putting everything about their profile in here people can have multiple profiles. By putting it in their My Documents folder it will automatically stay with their user account, this makes it portable on networks and allows multiple users to run the game on the same computer without being able to access either others profiles. It also makes it far easier for them to move their profile between computers. It should be noted in the documentation that on Linux they'll have to change this from My Documents to the user's home folder (although a number of distributions now come with a My Documents folder that uses the windows ID tag...).

- Model Modules: Big models go here, should be most of the MS3D files.

- Block Model Modules: Models used in the in game/real time building of game objects go here. They should all be fairly small, enough so that the entire folder can be converted into a high speed database later on. That'll eat system memory like candy but it'll be much faster. Putting the different block types in different folders will make it much easier to manage and then they can be put into different databases in the future (databases that don't all have to be loaded at the same time).

- FX Data Modules: Graphics FX goes here, starts as INI's, then later databases.

- Modding Data Resources: A misc data dumping ground for anything produced while the game is being made. Then later it can be added to a special download of bulk items for a mod resource kit of sorts to be included with a normal SDK.

Dynamic Data Template Modules: Below are the item designs I was talking about before, they are used in the dynamic creation of items. The templates tell the item creation engine what is being created, what values it needs to create the item, and any modifiers used in calculating the final stat values for the item. They are a special subset of game data because they aren't normal data, they're more like instructions for how to use game data. Another way to think about them is they define item types, if you add a new template you are in effect adding every possible combination of item that can be produced by that template rather than a single item. All of this can be ignored until the dynamic systems are developed and then they can be added directly to databases. There is no difference between item templates and item design or item blueprints, except the word template is used in the game data folder and the words blueprint or design is used in game.

Note: This is a complex system and it may be hard to see the relationship between the different items. I will explain this in detail later but for now remember I'm suggesting five types of commodities: low-level, mid-level, high-level, supplies, and equipment. Low-level items are used to make mid-level items and supply items. Mid-level items are used to make high-level items, supply items, and equipment items. High-level items are used to make game objects. Again, this is the system I'm suggesting you use, you can don't have to use it but I think its a fine system, you can also build on it or change it how ever you like. I will explain this later so these different template types are by no means excessive but the descriptions are vague.

- Dynamic Low-Level Commodity Template Modules: Used to make low-level items.
- Dynamic Mid-Level Commodity Template Modules: Used to make mid-level items.
- Dynamic High-Level Commodity Template Modules: Used to make high-level items.
- Dynamic Supply Template Modules: Used to make supply items.
- Dynamic Equipment Template Modules: Used to make equipment items.
- Dynamic Ships Template Modules: Used to make ships.
- Dynamic Base Template Modules: Used to make bases.
- Dynamic Object Template Modules: Used to make all other game objects.
- Dynamic System Template Modules: Used to create systems.
- Dynamic Character Template Modules: Used to create characters.
- Dynamic Planet Template Modules: Used to create planets.
- Dynamic Solar Object Template Modules: Used to make all other solar objects.

Section 10...

Framework Mod Modules
This section is open ended within the framework, anyone can create anything they want and use it during MP so long as the server admin approves (e.g. verifies that it won't effect game play). I put the downloadable SDK's here since its all mod related stuff (although it goes well beyond normal modding).

- Neural Net Program Modules: Player programs installed into the Neural Net as defined in the design document. This can include a pretty wide range of different programs so there is little need to get into the details here. A player SDK and manual should be released as another download for players who cannot mod but might be able to figure out how to write a Neural Net Program. Various templates should be included in the SDK and maybe a half dozen simple programs should be included with the main game (kind of how FLMM includes a few simple mods).

- HUD Mod Modules: Player created widgets for their HUD/GUI. There is a pretty wide range of possible widgets so there is little need to get into the details (read the HUD Mod Interface Module description for more details). This should be included with the player SDK for non-modders to create little widgets of their own. A half dozen or so simple HUD Mods should be included with the game itself.

- OpenLancer Player SDK: A independent download/product that includes all of the documentation, files, code samples, and simple examples of Neural Net Programs, HUD Mods, and other player/client side mods.
- OpenLancer Modder SDK: A independent download/product that includes all of the documentation, files, code, samples, and simple examples of game data mods. This can include the game data resources modder resources I mentioned in the data modules section.
- OpenLancer Admin SDK: A independent download/product that includes all of the documentation, files, code, samples, and simple examples of server side mods. This can include extra administration scripts and other resources for server administration.
- OpenLancer Webmaster SDK: A independent download/product that includes all of the documentation, files, code, samples, and simple examples of internet game engine modules, LAMP based CMS plug-ins, and other website related development. This can include extra PHP scripts, SQL databases, and RSS feed templates for webmasters.
- OpenLancer Developer SDK: A independent download/product that includes all of the documentation, files, code, samples, and simple examples of game engine mods, back end programs, and program interfaces. This can include extra source code snippets and other resources for programmers.
- OpenLancer Full SDK: Download that contains the Player SDK, Modder SDK, Admin SDK, Webmaster SDK, Developer SDK, and maybe a few extras.
- OpenLancer Game Source Code: The full source code for the main player/client side game. Does not include source code for the server, internet modules, or most of the back end modules.
- OpenLancer Server Source Code: The full source code for the server side game and most of the back end modules.
- OpenLancer Internet Source Code: The full source code for all LAMP based systems. Includes no real .NET code, just PHP scripts and SQL databases.
- OpenLancer Full Source Code: All source code, everything including the main game source, server source, internet source, back end source, etc...
- OpenLancer Full: Everything in a single huge compressed archive. Might want to offer an ISO CD image too (or maybe more than one if this really gets big, in that case maybe a DVD image).


Summary Note: There isn't must point in listing all of this stuff now but that gives you a good idea of how it should be broken up. For the downloads I was going to list compiled versions of the game for players/clients, servers, internet, server support programs, and back end programs/modules. Point being more specific download packages will get you more efficient use of bandwidth and probably happier players. I doubt most will want to spend an extended period of time downloading everything when all they want is the main game without the source code, server stuff, internet stuff, SDK's, etc... Likewise, I doubt someone looking to download maybe just the Player SDK will want to download anything else with it. I know the game is still a long, long ways from that point but its something to think about now and probably someone will ask about sooner than later. In short, serve your “customers” well and they will be happier with the overall project and as such more likely to both help you and maybe donate money to support your efforts.

Section 11...

Framework Back End Programs:
Another two modular programs to manage Open Lancer and two more to manage the MP server. If you're serious about support modders this will be a key area to focus your attention on after version 1.0 is released. The modules are DLL's or scripts that can be dynamically loaded and unloaded on the fly and can be coded in any number of languages, I suggest avoiding VB.net for OS portability but thats up to you.

OL Back End Executables: These programs help to support the game and are the same kinds of tools developers release to help modders. While these are OS dependent the modules should be coded to be OS independent.

- OpenLancer Module Runner Executable: Loads a single module at a time from drop down list and has command line option to start with a specific module so you can create direct short cuts to a specific module. Idea here is to make accessing the back end modules as easy as possible and at the same time reduce system load so if someone has this program open while they're playing it will not effect game play performance. Coding wise this is just a heavily stripped down version of the OpenLnacer Manager Executable.

- OpenLancer Manager Executable: Loads different modules and/or groups of modules from a drop download or from a command line option such that shortcuts can be created to load a group of modules at a time. The different groups of modules can be defined with an INI file, something like “[group ”, “group_name = modes”, “group_config = player_module.ini”, etc... Then if player_modules.ini exists it'll include those modules from the list. I think the best layout would menu bar across the top, then a drop down list of all modules with icons to load/upload the module, then either tabs down the left size of the window to switch modules or tabs across the top to switch modules. The menu bar could include an icon to launch OpenLancer, an icon to launch the server, a program menu to launch one of the other back end programs (could be listed in an INI), file menu to load modules manually (and exit), options menu, help menu, exit button. Note: Windows already has a system for modular programs like this but I suggest you don't use it to help maintain OS portability, thats your call but it'll be far easier to port it to another OS if there is as little windows specific garbage in there as possible (and I mean that, windows sucks, I don't know enough to port this beast to Linux but I'm sure there are a few people on TLR that do).

- OpenLancer Server Monitor: A program that links to the Administrative Link Module in the MP Server Extended Core. That module sends this program information about the status of the server. This can start out as a very simple system where the server tells the monitor its “I'm still alive” every 30 seconds or so. If the server doesn't send out its “I'm still alive” message after 40 seconds the monitor program will end the server program and restart it. Then in future game engine versions more features can be added to allow the program to have more control over the server. Example: It should be adapted to tell the server to shutdown before windows is shutdown, these kinds of system shutdown warnings are sent out by server management programs that come with uninterruptible power supplies (UPS) and can prevent data loss and player character or game data corruption (in other words if the power goes out on a server that has a battery backup and the battery backup tells the PC to shutdown no data will be lost). This is a mostly passive program so it could just be a little systray icon for an interface with the actual settings being managed from the MP Administrative Module.

- Open Lancer Remote: A generalized remote server administration web server with SSL that allows a server administrator to control the server from any web browser (e.g. Webmin for OL). This should also include a relay to allow the administrator to connect to the server as an “admin player” with near god-like powers. The way I defined the links between the extended server core modules it should be near impossible for someone to connect to the server remotely as an “admin player”, this program will bypass that restriction and allow admins to securely play on their server without having to worry about someone else gaining access to the admin powers. These powers can be extended to the admin via a number of the admin specific modules but also include using the editor in Realm Forge GDK to literally do anything they want and using the network managers to kill players by safely kicking them from the server rather than shooting them or brute force disconnection (which might lead to data loss). For the web server just use Apache and a SSL wrapper, later on add the multiplayer relay by pulling/copy code from the MP server network modules and security modules, then pick 2 ports and assign one to apache and another to the MP relay. Then add a systray applet (VS.net code snippets) and put the configuration stuff into the MP Administrative Module. Finally you'll need to find a perl programmer to either port Webmin with some new modules or write something decent from scratch (or use PHP and I can try to create a Webmin-ish system but I think perl is better suited for this).


Mod and Module Management Module Set:
Modules that most players will probably want to use but that shouldn't be required to player the game or be included with the game download. Again, all of this is optional for the player, if they don't know enough to download this they won't know enough to download or safely use mods.

- Mod Manager Module: Front end to backup and restore module, its like the FLMM GUI. Should be a detailed list of mods (like the detailed file view in windows explorer), like a mod name, mod author, developer team/studio name, mod version, mod data, mod size, etc... You can use the same color formatting as FLMM, black for inactive, green for active, orange for may not activate correctly, red for cannot be activated, and add flashing for errors. Could also add a little status icon and build in error checking, so if the mod contains an known possible error it could display a little “X” icon but if there are no known problems it could display a check mark icon. Should include buttons to activate, deactivate, and analyze mods.

- Backup and Restore Module: In short, a comprehensive backup and restore utility, supporting the full game, game data, and mods. In long, it should be able to: Backup, restore, and/or move the full game to a another folder, local drive, network drive, compressed archive, or CD/DVD image file. Backup and/or restore all game data to another folder, local drive, network drive, compressed archive, or CD/DVD image file. Backup, restore, and/or move one or more player profiles to another folder, local drive, network drive, compressed archive, or CD/DVD image file. Backup and/or restore one or more inactive mods to another folder, local drive, network drive, compressed archive, or CD/DVD image file. And do a differential backup, restore, activation, and/or deactivation of one or more mods. By network drive I mean either a mapped drive or non-mapped FTP server. This is the guts behind the Mod Manager Module and acts as an abstraction layer between the operating system and the various libraries that actually do the real work. These functions are used by other modules so I listed it as a separate module. Most of the required libraries can be found on Source Forge and pulled from the OS itself.

- Game Update Module: This is a CVS, SQL, FTP, and HTTP client to download program, module, mod, and game data updates. The updates are then processed by the Backup and Restore Module. Like the backup/restore module this is really just an abstraction layer for probably a half dozen libraries that do all the real work. Most of the required libraries can be found on Source Forge and pulled from the OS itself (a database client and download manager should do it).


MP Administration Module Set:
Stuff admins will need, should come with the server download.

- MP Administrative Module: Manages the server settings, server security settings, faction settings, economy, and player accounts. Basically everything you can think of for day to day server management. I know a few server admins that could give you a list of everything they could possible want in a server management tool (Dazzla99, if your reading this, care to comment?). The Realm Forge GDK editor needs to be modified to allow them to do the same while playing on the server as an admin.

- MP Game Data Manager Module: Manages game data in near real time. This won't do much until the dynamic systems are built and enabled but once they are... This needs to allow admins to tweak global idem creation modifiers (or rather “tilt” the random number generators to produce better or worse quality items). Also needs to be able to create, edit, or destroy any item in the game data databases. So they can use their “god-like” powers to do almost anything they want. The Realm Forge GDK editor needs to be modified to allow them to do the same while playing on the server as an admin.

- MP Network Manager: Manages server links, network security, and connections to internet servers. This won't do much until you build up the server link system and/or enable the game data database system. This should include a wide range of options including setting up VPN/SSL wrappers around the server to server connections (that may be critical to slow down hackers who like to cheat). Should also include text a few status messages if a CVS or SQL based service goes offline. If a CVS server goes offline it will also have to tell the server to send players the game data updates, otherwise they'll be missing static game data when they connect and might get kicked for being out of sync with the server. This could also be used to manage remote administration links, including which addresses are allowed to connect to the MP relay as an “admin player” or use the web based “OLmin” system.

- MP Log Manager Module: A series of server log filters so admins can quickly find the log entries they need to read or search for log entries they want to read. Should include a system to backup logs into packages and also load log packages so the same filters can be used, this would be routed through the Backup and Restore Module.

- MP Server Stats Exporter: Exports log and character data to a variety of different formats. Should allow for transferring log and character data to different servers, FTP servers, websites, etc... The Update Manager and Backup and Restore Modules can be used here, in addition to libraries to convert the data to HTML, XML, SQL batch, and other formats.


Mod Development Module Set:
Stuff modder and developers will need, should come with the SDK's for both groups.

- Game Data Editor Module: INI editor for starters, then both INI and database editor. The INI editor should be pretty straight forward, just a INI parser with fields defined for different INI section types (probably time consuming to code too). The database editor should be a little easier to setup, there are a few open source database data management programs available and most of this will be the same stuff used in the MP Game Data Manager Module. Should include a special editor for creating/editing dynamic item creation templates.

- System Editor Module: Basically Freelancer Explorer, if you can get the source code you might be able to use it. For that to work you'd have to overhaul the INI parser to read/write the updated system formats, plus I think the coordinate grid will be different but its still a starting point... This could also be tied into the dynamic item creation engine to generate systems for modders to use as a starting point.

- Game Graphics Editor Module: I'm not sure what you has in mind for this but it sounds like you want to include this as a modder tool. Might be able to use the Realm Forge GDK editor and then find some plug-in's for editing different graphics formats for both models and textures. Could do the same for audio. Theres a lot of powerful code in the game engine and a fair bit of it could be put to work in this module but at the same time people will probably want to use 3rd party tools.

- Game Data Analyzer Module: For starters this will analyze the INI files, it should be possible to update one of the existing FL INI error checkers to use as a starting point. Then with the databases this will become exceedingly complex and be more centered around data integrity (data structure errors), as well as looking for data value errors. Can also be used to check for data structure efficiency, large mods will probably run into problems with stuff being inefficiently scattered all over the place, this could stream line the data structures and would be useful on both INI and database systems. As with most things there is a lot of open source code and libraries available that will do most of this but it'll take work to narrow the number and type of specific data checks done so the error checking is as useful as possible.

- Data Import Module: Imports data and converts it to INI format, should be possible to use this to convert Freelancer INI format to OpenLancer INI format. It would also be extremely useful for importing game data sets from tables (thats actually a problem I'm having now, I have a large amount of data in spreadsheets and I need to generate FL INI code using that data, including about 45,000 market entries). So, being able to read a large number of different spreadsheet, CSV, and SQL batch formats and then generate INI code would be very helpful. With the database system this will be even easier, seems like you can convert just about any data type (including INI's) into something a database can read, but sadly it doesn't always work going the other way.

- Data Export Module: Exports game data to a variety of different formats, the counter part of the Data Import Module. Should be able to export to Freelancer INI format to port mods backwards to Freelancer. It needs to support all the same formats it can import from or really just as many formats as possible.

- Game Data Upgrade Module: Converts game data from one format to another to support switching from INI data storage to database data storage. This is the key to upgrading from INI data storage to database data storage. If the Data Import and Export Modules are fully developed this shouldn't be a problem, just use the existing code in a loop back setup with added error checking. This is critical and must be flawless or there will be some serious headaches down the road.

Section 12...

Framework Internet Modules:
This is mainly here as reference for future expansion. All of this is Linux, Apache, MySQL, PHP (LAMP) based. Most if not all of this should work on “run of the mill” LAMP servers and as such almost any commercial web host. Free web hosts may or may not work, depends on how they manage forced advertising. Most hosts block direct access to their SQL servers so there could be a few issues with getting open SQL access but that can be rerouted via PHP (in which case their Apache server is accessing the SQL server, thus getting around any SQL restrictions). This stuff should be pretty straight forward to most webmasters and isn't that hard to develop if you know your way around PHP (knowing SQL batch commands and Linux in general certainly doesn't hurt either). Some of it might seem excessive to some people or just not enough to other people, just keep in mind the main object with this is to build community access into the game engine and most server communities are heavily reliant on forums. I make heavy use of *nuke type (PHP-Nuke, CPG-Nuke, and PNP) content management systems (CMS), its what I know and strongly suggest server communities use. I have 5 years of experience as a part time webmaster and collectively maybe a year and a half of PHP coding experience, I should be able to code all of the PHP scripts and setup the databases.

Important Note: Most of this is possible with Freelancer, or specifically an automated system to maintain and update all INI based game data on a SQL or CVS server with a web based interface to automatically edit the INI's and a MP server side system to automatically download the updated files. Then have the web/CVS server automatically generate player mod updates. I discussed (or kind of yapped at) my clan about the possibility of doing this on Haven 24/7 (Dazzla99's server) but nothing ever came of it. I was going to send the concept to EOA or Alcander but just didn't get around to fully documenting it. So with that in mind, some of the stuff below is possible with Freelancer, its just a matter of setting up the system (something I wish I had time to do but that would take a month or more). The only thing I have no clue how to do is automate the process of adding IDS entries for news and rumors. Adding or changing the INI files automatically or even manually via a CVS server is easy but time consuming to setup. If anyone knows how to do this (and I'm assuming to EOA this would be cake work) please by all means do it, document it, and then do it again for OpenLancer.

- CMS User Interface Modules: Allows users to setup RSS feeds, forum access (read/post), and a massive amount of other possible data sources to access from Neural Net Programs and HUD Mod widgets during game play. This will require a special Neural Net Program to handle the data display and formatting, pretty much one per “feature” item or a huge program that does everything. The important aspect of setting this up is to make sure the Neural Net Program works with as many different systems as possible so that players don't have any compatibility headaches or risk downloading the wrong Neural Net Program, same for HUD Mod widgets.

- CMS Admin Interface Modules: Allows admins to manage users via the CMS's user/group access control system and access an admin Neural Net Program for CMS status checking from in game. The user/group access control system in most CMS's can be mirror with player groups and player run factions. Status checking is just that, who's online, what's been posted recently, poll results, etc... This could easily be extended to general CMS administration functions but that might be a security risk of the admin is connected as a normal player, as an “admin player” they would be connected either locally or via SSL (not to mention with that data volume packet sniffing would be a real pain in the ass for anyone who tried) – even still if the admin needs CMS admin access its probably easier to use the tools included with the CMS then anything that could be built into a Neural Net Program.

- CMS Dev Interface Modules: Allows developers to work on game data from a web browser. This is front end to the CVS server, not the MP game server, so it only covers static game data. This can be extended to support editing INI's since the CVS server probably can't tell the difference between C++ and INI files (at least not in content, they do have different file extensions). So a PHP INI parsing system would be needed to read, edit, and save INI's to the CVS server. Then later this could be upgraded to support editing databases, probably use a heavily overhauled version of phpMyAdmin and a few dozen scripts to make stuff easier and ensure the values entered at checked against the Game Balance Data Set (e.g. that any items created are legal).

- CMS Data Access Modules: CMS functions for serving data to the game in a format it can read. A not so simple abstraction layer. I needs to take multiple different data formats and convert them into something the Neural Net Program can read without having to do any conversion. This leave the processor load on the web server rather than slowing down the game (I'm a gamer first, webmaster and server admin second, and yes I'm sure some admins will ***** about the extra server load on busy servers).

- CMS Abstraction Layer Modules: Enables support for different CMS's starting with PNP, PHP-Nuke, and CPG-Nuke, This may or may not be realistic, most of these types of setups end up being too slow and have security issues. If it is possible or even practical, it will be a serious pain in the ass to code (not to mention an even higher load on the web server).

- Forum Interface Plug-ins: An abstraction layer and read/post user interface for phpBB, maybe Invision, and maybe others. Despite having used dozens of different forum systems I've only ever worked on phpBB code, it's a nice little forum package and comes built into nearly all *nuke CMS's (for that matter most LAMP based CMS's).

- Game Data Submission System: An overhauled LAMP based trouble ticket system for submitting news, rumors, and other stuff into the game data. With INI's it'll have to pull files from the CVS server and use a script to update the INI, then send it back to the CVS server for later download by the server and players (e.g. on restarts). With the databases this process can be massively faster, its basically the same system but the MP server's RSS feed reader and SQL client can be used to grab the news, rumors, and other stuff straight from the SQL server or CMS RSS relay. Then the news and rumors gets added to the Variable Game Data so gets automatically sent to the player when they change systems or dock. Note: This is possible in Freelancer! But I have no idea how to handle the IDS entries on a web/CVS server (it just can't be done in PHP).

- OLmin: Generalized remote server administration from any web browser (e.g. Webmin for OL), runs on the OpenLancer Remote over SSL. Might even use Webmin as a basis for this system, except I really don't care for perl. Should work fine in PHP and can be extended to manage other MP servers, CVS server, SQL server, CMS community sites, forums and include all of the other web based game data editing stuff. It will take a long time to build but will be a very powerful administration system when its finished.

- CVS System: I'm still not sure about the CVS system for handling static game data but that would go into this section. I know of a couple system that could work but I need to do a little research. For that matter I think I know of a database system that might work for the Variable Game Data and a means to easily convert the INI data into a database format such that nothing has to be reentered manually if and when that actually happens. Just plan for the future man, thats all I can say.

Section 13...

Modular Game Engine Framework:
Heres a quick run down of what is critical to build the first alpha release as fast as possible. Remember that there is a fair amount of items in here that just need a small bit of code to setup a place holder (none of that is noted here). Most of the “important” modules are little more than bulk work to adapt to a major change or make it easier for others to join the project. I'm deleting the modules you should be able to “patch over” with a place holder. For multiplayer, get the engine running and worry about adding multiplayer in a later alpha build.

Framework Core:
- Client Game Engine Core Executable: Critical.
- OS Access Module: Important.
- Low Level Access Module: Important.
- Middle Ware Access Module: Important.
- Error Control and Debug Module: Critical.
- Engine Console Module: Critical.

Framework Middle Ware Modules:
- Realm Forge GDK: Critical.
- Control Interface Module: Covered by Direct Input and/or SDL.
- General Physics Engine: Optional.
- Math Libraries: Useful.
- Game Data Interface Module: Critical.

Framework High Level Modules:
- Script Function Modules: Critical.

Framework Game Play Management Modules
- Main User Interface Management Module: Critical.
- Flight Mode Management Module: Critical.

Framework Data Level Modules:
- Script Modules: Critical for testing.
- File Lists Module: Important.
- User Interface Data Module: Important.
- Audio FX Modules: Optional.
- Audio Modules: Optional.
- Music Modules: Optional.
- Player Profile Module: Optional.
- Model Modules: Critical.
- FX Data Modules: Optional.

I'll add to this later...
I'm researching this now, like I said I know virtually nothing about Realm Forge GDK beyond its feature list. For that matter I know virtually nothing about the .NET framework other than it includes C# which is based on C++ which is a language I use to know (yeah, not much help there!). All I've really found so far is a number of references to the readme and other documents that come with the GDK download but none of these files is posted on the Realm Forge wiki, so if I can't find more details some where I'll just download the docs and try to answer weapons from those. This probably isn't useful but I did find a simple demo called Realm Forge From Scratch.



Edited by - MegaBurn on 12/19/2005 3:40:35 PM

Section 14...

Framework Notes:

- Framework Updates: I'm updating this entire thread so keep an eye on the edit tags at the end of each post. I'm also moving posts around to get those 2 empty posts to the bottom of the thread. I added this section 15 too. Thats not nearly the end of it either, I'll continue to add new links, refine the design, clarify confusing module descriptions, and add detail to the vague spots. I want this to be as useful as possible.

- Network Middle Ware Packages: I found two good network middle ware packages RakNet Package and OpenTNL. RakNet should cover most of the networking stuff. Including both client and server side, VoIP, server security, chat, and a bunch of other stuff. Its free, includes coding examples, code snippets, is generally well documented, and its used in commercial games. This one package would replace both HawkVoice and HawkNL. The VoIP codecs are the same as Team Speak so that aspect is pretty solid. The networking stuff, again, you just need to read the feature list and skim the manual to know thats probably the best system to use. Still, OpenTNL is also a free package that claims they won awards for network features and is used in commercial games (Tribes franchise and others). I think RakNet is the better of the two but I'll leave that up to you guys to figure out. I left HawkNL and GNE in the middle ware section for reference. Then theres also the libraries included with Realm Forge GDK, I have no idea what that includes so you'll have to sort through the GDK and then pick one of the three: GDK net libraries, RakNet, or OpenTNL.

- I'm not sure which IDE's you guys are using but m$ is offering Visual Studio Express 2005 for free if anyone needs it. Its six different downloads: Visual Web Developer 2005 Express, SQL Server 2005 Express, Visual Basic 2005 Express, Visual C# 2005 Express, Visual C++ 2005 Express, and Visual J# 2005 Express. You can download one or all. Probably requires a genuine windows check. I'm going to see if I can get my hands on Visual Studio 2005 Professional.


I'll add more later...



Edited by - MegaBurn on 12/19/2005 8:44:30 PM