/>Thread #85 - January 27, 2012:
Here is an official explanation by DICE employee Mikael Kalms:
Frostbite 1.5 consists of these components:
- The game runtime
- The editor runtime
- The content processing runtime (aka "the pipeline")
- and some plugins for Maya
The game runtime is distributed outside of EA, but the editor + pipeline + Maya plugins are not.
So let's take a look at some things that would need to be solved before we'd be ready to distribute the editor + pipeline.
Pipeline operation
Let's say that you tell the pipeline to build level MP_003.
MP_003 is represented by an XML file, which references a bunch of other
files. These in turn reference other files. If you follow this graph of
references, you will find the level layout, heightmap, characters,
weapons, vehicles, and all the content that you can see in-game. (The
in-game HUD and related stuff might also be in the graph.)
When the pipeline is about to build MP_003, it will first perform a
consistency check on all content, and yell if any file that is
referenced by any other is not present.
If all files are present, the pipeline will attempt to convert all files
referenced by MP_003. It uses the file system journal to determine
which files have changed on-disk. Also, and any files that have already
been converted have info on which files depend on it (so it has info
like: "if file X changes, then files Y,Z,W will also need to be
rebuilt").
Building all content for BC2 from scratch takes something like 48-72
hours on a normal workstation. Half that time is spent building common
content (such as character animations), half builds level-specific
content.
In addition, there's a caching mechanism: if the pipeline wants to build
a specific bit of content, it will first check if the pre-built content
is already available on a cache server and take the result directly
from the cache server instead. The pipeline can also populate the cache
if it builds something new.
Pipeline issues
So how does this work in practice? It's not ideal, but it's good enough for us to ship games on it.
The pipeline is a bit overzealous with regards to rebuilding assets - sometimes it rebuilds stuff that it shouldn't need to.
The pipeline will normally crash about 2-3 times during a full rebuild.
You need to have Maya 8.5 (32-bit version) installed in order to convert any meshes.
Any content in the cache expires after 3 weeks. After 3 weeks have
passed, that content will need to be rebuilt and re-uploaded by a
machine running the pipeline. The effect that this has on day-to-day
development is minimized by having one or two machines dedicated to
running the pipeline every time any content change is done. By running
the pipeline, those machines will populate the cache, thereby speeding
up the build process for everyone else. (The output form those content
build steps is discarded.)
In short: the pipeline + cache setup works better the more people are using it simultaneously.
If there are content errors, you need to know a lot about the internals of the game engine to figure out what's wrong.
Finally, in its current form, the pipeline + editor expects some
specific IT infrastructure in place (most notably the cache server and a
Perforce server).
If it's not there then the pipeline + editor will behave strangely.
The first time I tried, it took me about one week to get the full editor
+ pipeline setup to work properly outside of the DICE office. And that
was when I had the option to call any of the other developers to ask for
help.
... does this sound bad to you?
Truth be told, this is approximately where the industry average is at
for game studios' internal game engines. One of FB 1.5's weaknesses is
specifically that its content processing is flaky, and the flakiness
gets more problematic as the amount of content goes up. FB 2.0 is much
improved in this regard, but FB 1.5 is what we're using for BC2 and
that's what relevant in the current discussion (or monologue if you
prefer).
Content
Both the pipeline and the editor takes in all content in its raw,
original form. Anyone who is to build any content needs the full 80GB of
raw data on their machine. We are not comfortable giving out all our
animations, meshes etc in raw form.
We are comfortable giving out the processed data - after all, that's
what on the game disc - but that data does not plug into the
editor/pipeline at all.
Licenses
The game, editor and pipeline all use commercial middleware. It is developed by Havok and several other companies.
The licensing agreement for the middleware allows us to use that code in specific products, on specific platforms.
If we want to release editor + pipeline, we need to license the
middleware specifically for this. How much would that be? Perhaps
$1M-$3M. I'm guessing wildly here.
Stripping out that middleware would seriously hamper the functionality
especially of the pipeline. We use Havok Physics, for instance. Without
Havok Physics, the pipeline wouldn't be able to convert any of the
physics meshes. We also use Granny. Without Granny, the pipeline will
not be able to convert any of the character animations. Etc.
Re-implementing the necessary functionality of the middleware ourselves
("let's make our own physics engine / let's plug in an open-source
physics engine") would take literally man-years. Licensing is cheaper in
pure $ cost and faster (it works now instead of by 2012).
The pipeline also uses some code that is under GPL. Given that we do not
want to release the full source code for the editor + pipeline, we
would need to replace the GPLed code with other implementations.
The GPLed code is less of a problem than the proprietary middleware.
Editor
The editor itself is reasonably stable and well-behaving. It is far from
obvious how to set up the game logic for a level, but that is easily
covered by releasing some example levels which contain the logic setup
for the common gamemodes.
Test-running levels
First the level needs to be successfully processed by the pipeline. Then
you'd want to be able to test it locally. That involves having a listen
server around. We don't have a listen server neatly packaged. There's
probably a piracy angle here too but I'm not going to discuss that.
Distribution of levels
Getting levels onto the RSPs server machines would likely not be any
problem. However there's need for checksumming levels, so that game
clients can know whether or not they have the correct version of level X
on their machines. There's a whole bunch of other things (mainly
UI-related) which will need cleaning up as well. Not difficult to do,
just takes time and I'm listing it for the sake of completeness.
Also, there are some complications wrt when we release patches that
affect the base game's content. Whenever we release a patch, all
existing levels will need to be rebuilt with a new set of original data.
This is because some level-common data is stored inside of the level
archives. I'm not sure at the time of writing, but that probably means
that the only manageable way for us would be to invalidate any user-made
levels when we release a patch of that form.
Then creators of any user-generated levels would be required to run
their levels again through the pipeline with the new base content
supplied.
So how about just a map editor?
If it doesn't plug into the ecosystem above, then getting it to work
involves some serious wrangling. Either it is a light-weight replacement
for our existing editor - in which case all the challenges with the
pipeline still remain - or it is a separate mode (think Forge for Halo).
Developing an extra mod-layer that is sandwiched into the game would
easily take 6-12 months.
Synergy effects between FB 1.5 and FB 2.0
So let's say that we would go through the procedure of making mod tools
for FB 1.5. How much of that work would be reusable for FB 2.0?
I don't have any firm figures, but the differences between FB 1.5 and FB
2.0 are pretty large by now. Given this and the fact that a fair bit of
the FB 1.5-specific problems (where the devil often is in the details)
don't apply to FB 2.0, I'd guess that less than half of the work would
port over to FB 2.0.
Conclusion
In conclusion, my recommendation to the rest of DICE is not to develop
mod tools for BC2 PC. There are too many hurdles to overcome. That
energy is better spent elsewhere, be that on BC2 or other titles.
You can found the original thread in this link:
Explanation of No Mod Tools for FB 1.5 & FB 2.0
>/End of Thread\<
