Source

The Source engine is a game engine developed by Valve Corporation. Its unique features include a large degree of modularity and flexibility, an artist-driven, shader-based renderer, industry-leading lip sync and facial expression technology, and a powerful, efficient and completely network-enabled physics system.

Source supports both 32-bit and 64-bit environments and the Microsoft Windows, Xbox, Xbox 360 and PlayStation 3 platforms. Also it has been unofficially ported to Macintosh using cider technology. It debuted in October 2004 with Half-Life 2 and Counter-Strike: Source.

Technology overview



 * For a full overview, see Source Engine Features on the Valve Developer Community Wiki.


 * Rendering
 * Shader-based
 * DirectX 9 with Shader model 3.0 and OpenGL
 * 3D skybox; an area can be displayed as a skybox at up to 16x its actual size with full 3D parallax.
 * High dynamic range rendering


 * Animation
 * Any animation can merge seamlessly with any other animation at any time.
 * Inverse Kinematics to ensure that characters' limbs react to environments


 * Networked physics
 * Originate from Havok 2 physics engine
 * Highly-tuned by Valve
 * Processor-efficient
 * Software only
 * Fully networked with low bandwidth requirements
 * Vehicle physics including torque, power, gears, tire material, suspension and mode.


 * Audio
 * Full DSP system
 * Full Surround sound support
 * Software only
 * Low and high frequency components merged depending on surrounding area and relative position of origin


 * Scalability
 * Supports DirectX 6 and upwards
 * Modularity allows for all current and future Source projects to scale back to DX6 if they desire


 * Facial expressions
 * Full range of human and non-human facial movements
 * Based on Professor Paul Ekman's Facial Action Coding System
 * Over eighty-four "digital muscles"


 * Lip-sync
 * Works in tandem with facial expressions
 * Auto-generated, but completely configurable
 * Stored in sound file itself for localization

Modularity and notable upgrades
Source is designed from the ground up to be highly modular. This allows for the easy upgrade and modification of certain features without breaking other areas of the engine, or breaking engine continuity (that is to say, there need be no 'version jumps' from 1.0 to 2.0). When coupled with Steam, these updates can be distributed retroactively and automatically. For instance, if Source is upgraded to support hardware physics, every Source title on Steam will instantly benefit. Entirely new features such as High Dynamic Range (HDR) Rendering have been shown to require developer input, however.

High dynamic range rendering
First seen in Day of Defeat: Source, then shortly after in Half-Life 2: Lost Coast, HDR was the first major instance of Source's modularity in use. However, whilst in theory all Source engine games and mods were able to use HDR immediately after its release, the game code required to 'hook in' to the new system was not made available to modders until eleven months later, on August 4 2006. Official licensees and Valve themselves have all made use of the technology since its release.



Facial animation 2
When Half-Life 2: Episode One was released on 1 June 2006, it introduced the second version of Valve's proprietary facial animation system. Ken Birdwell explains the upgrade's features in the game's commentary track:

"When we designed the Half-Life 2 facial system back in 2000, our goal was to get a natural-looking performance at a moderate distance. For Episode One, we wanted to extend the characters' facial systems to support more intense performances with a wider range of facial expressions, that would hold up better at close range. These facial improvements included increasing the detail around the eyes and mouth, increasing the number of facial shape targets – think of these as movements of muscle groups – by about 50%, rewriting the rules that control how these shapes blend, and increasing the intensity of many of our existing shapes."

Dynamic lighting and shadowing 2
A new dynamic lighting and shadow mapping system was released for Source, replacing the somewhat limp existing system. It launched with the various other new Source features with Half-Life 2: Episode Two.

Next-gen renderer
An upgraded rendering path is in development for future Source engine games on PC, Xbox 360, and presumably PlayStation 3. For PC, the new renderer has been confirmed to support DirectX 10.

Landscape and Flora Rendering


Large, open natural environments with heavy foliage, traditionally a weakness for the Source engine, have been supported as of Half-Life 2: Episode Two. The updates were also made available for MOD teams to use shortly after the release.

Soft-Particle system
During the July 2006 Electronic Arts Summer Showcase press conference, Gabe Newell mentioned that a new soft-particle system would be introduced into the Source Engine in the upcoming title Half-Life 2: Episode Two. It was first demonstrated in the July 19 Team Fortress 2 teaser, which showed a remarkably realistic flamethrower in its closing moments.

Cinematic physics


During the July 2006 Electronic Arts Summer Showcase press conference, it was revealed that former Weta Digital employee Gray Horsfield, special effects destruction lead on The Return of the King and King Kong among other roles, was building a "Cinematic Physics" system for Source. GameSpy described the new system in their conference report:

"The idea behind this is to give players the opportunity to experience in-game physics in action on a grander scale. As an example of Cinematic Physics in action, a clip from Half-Life 2: Episode Two was shown of a huge bridge collapsing across a vast ravine."

The system adds the following features to Source's physics simulator:


 * Deforming objects — before, physics models could not be modified except through animation
 * Dynamic crumbling of brush geometry — before, lines of separation had to be specified by the mapper

Cinematic Physics supports a keyframe system, but its exact nature is currently unclear. It could be that an animator creates a largely complete but low-detail sequence which then sees details added by the physics system, or it could be that an animator creates a handful of single-frame states which are then used as motion targets for the ensuing simulation (in a manner not dissimilar to the Endorphin NaturalMotion technology).

Either method results in a drastic reduction of developer input, thus allowing the creation of far more complex scenes than before with the same budget. It is currently unclear both whether or not keyframes are strictly required, and what number are needed to create a scene as complex as the bridge collapse demonstration.

Multiprocessor optimizations
As a part of the Source engine's transition to next-generation consoles, multiprocessor optimizations have been added, resulting in faster processing on PC hardware with dual or quad core systems and the Xbox 360 and PlayStation 3 consoles. Gabe Newell:

Yes. We definitely think that content needs to move forward. For example, one of the things we're reacting to is the speed at which microprocessors are coming out. So, Intel has very aggressively moved up delivery of desktop processors with four different cores; we'll have support for that in Episode Two, and we'll definitely go back to affect, Episode One or Half-Life 2 or Counter-Strike Source, so they can take advantage of that. We'll definitely try to keep the existing games - especially the multiplayer games - current as technology evolves.

Valve has demonstrated the new multi core optimizations which use a multi-threading style they dub "hybrid threading." A Source multi-threading update and benchmark are expected to be released before Half-Life 2: Episode Two, though a date for either and the content of the benchmark are currently unknown.

Cinematic effects


With color correction and film grain already released, Valve intends to add other cinematic effects such as motion blur and depth of field to Source when hardware is able to render it to their satisfaction. The effects are accomplished with an accumulation buffer for quality, creating enormous overhead; for instance, twenty to thirty motion blur frames need to be rendered for every one frame that the user sees. For a constant frame rate of thirty frames per second, a video card is required to produce between six hundred and nine hundred frames per second. This causes late 2005-era hardware to require a full two seconds to render each frame.

Motion blur and depth of field can be seen in several of Valve's promotional videos, including:


 * The Day of Defeat: Source video Prelude to Victory
 * The initial Portal trailer.

Image-Based Rendering
Image-Based Rendering is a technique in which 2D elements are manipulated to appear in a 3D world. In the context of a 3D game, it delivers a significant performance boost by replacing 3D geometry that is far enough away for the transition to be imperceptible with a 2D image. Implementation of the technology in this role can be found applied to Rome: Total War 's soldiers, Far Cry 's forests, and various objects in Guild Wars' environments, such as buildings and flora.

The technology had been in development for Half-Life 2, as a 2003 interview with Driverheaven.net shows, but was cut. It was mentioned again by Gabe Newell during 1UP's 'Valve week':

"There's this technology that was really exciting that I’d like to see us get into production, which is a different approach to rendering complexity: Moving things into and out of an image domain and then seamlessly interpolating between those motions as the player moves around. So that everything close to the player is physical and geometry, and everything really far away from the player is an image, but the player has no way of telling that if the player does it properly and things can fly out and come back."

The June 2006 Day of Defeat: Source update included an "image-based texture blending shader", and Half-Life 2: Episode Two 's expansive environments seem ideally suited to the technology, but it has yet to be dated or even officially announced.

File streaming
One of the technologies developed for Half-Life 2's Xbox release was file streaming, wherein a map's resources could be loaded as the player moved around in it rather than in one operation before playability. With the system in place, loading times were reduced to as little as fifteen seconds. The system expanded on the caching system already implemented. There is no time frame for its release, as implementing such a system on the potentially infinite variations of PC hardware setups in use poses serious performance problems (see Stutter section).

Origins
Although Valve has explicitly stated that the Source engine has been built internally from the ground up, rumors and myths persist that it is instead merely derived from the original Quake codebase via Valve's GoldSrc offshoot. The primary reasons for this are the manner in which the engine uses similar development interfaces to GoldSrc (to aid transitioning developers), and John Carmack's comment on his blog that "there are still bits of early Quake code in Half-Life 2", expanded through hearsay to be a confirmation that large swathes of code are identical, when no such conclusion can be drawn from the statement. There remains no solid proof that Source is derived from the GoldSrc codebase &mdash; and indeed, given the fact that the 2003 code leak did not produce any such claims it can only be assumed that no incriminating evidence was to be found.

However, it is known that Source was developed part-by-part, slowly replacing the GoldSrc engine in Valve's internal projects. This explains its modular nature, and suggests that, even if Source was not derived from GoldSrc, GoldSrc was at the very least modified to plug into it during development.

Stutter
The Source Engine uses a caching system, whereby the loading of certain resources is handled and managed on the fly, rather than in a single operation behind a traditional loading screen. Texture and sound data are the primary areas in which this occurs. Textures are loaded to memory but only moved to the system's video card when needed and audio files are loaded with an unusual "soundcache" system: only the first 0.125 seconds of each file are pre-cached, and the clip is used to cover the asynchronous buffering of the full sound file in the background when it is first requested.

Both systems keep data in the heap until there is no more room and older resources are flushed out, and when either is held up or otherwise slowed down the engine will either freeze or go into a temporary loop until the data arrives. 'Stuttering', or 'hitching' as it is sometimes known, is the result of these pauses.

While stutter can be caused by poor system performance, it has also been noted on hardware setups that should be more than powerful enough to cope with the data rate, and despite many theories, the precise cause remains unknown to the public even over two years since the engine's debut. Most solutions that have been found involve bypassing the caching system, as it cannot be directly disabled, or system-specific optimizations (e.g. driver updates).

When Half-Life 2 was first released and stuttering became a widely-known problem, community member Mark McWilliams set up a page covering the issue and Valve's communication and work on resolving it. Several updates were released by Valve, the effects of which varied from complete fixes for some users to previously smooth systems becoming "infected" with the problem.


 * Example (.WMV), showing a relatively low level of stutter. The hitches coincide with the drawing of new areas (a process that is by design not visible without debugging tools).

Most recently (February 7 2006), changes to the Source engine were introduced alongside a beta test of Steam's Friends chat service, with the aim of 'narrowing down' the problem. The update featured a limited implementation of Source's file streaming system (see above). Generally, the response was very positive.

Looping audio
The Source Engine suffers from an error whereby the asynchronous loading (see Stutter, above) of a new sound file will cause the engine to lock up with looping audio. Because of the nature of DirectX, once the engine enters such a state it will remain on the screen unless the user can blindly terminate the program, or reboots their computer. The error occurs in a standard Windows library associated with on-board sound cards, and in some cases can be resolved by decreasing DirectSound Acceleration.

Reports of looping audio crashes increased around the release of Episode One. While it is likely that the spike was simply because there was an unusually high number of people playing the game, changes to the engine, of which there were many for the game, cannot be ruled out. It has been noted that people who have been able to play Half-Life 2 without any crashes and audio errors in fact do find Episode 1 to be more prone to that problem. A steampowered.com forum thread is dedicated to discussing the problem and attempting to work out solutions, although the experience of the thread contributors indicates that most suggested workarounds and fixes do not remedy the problem.

Valve Developer Community
On 28 June 2005, Valve opened the Valve Developer Community Wiki. The VDC replaced the previously available Source SDK documentation with a full MediaWiki-powered community site. Within a matter of days Valve reported that "the number of useful articles [had] nearly doubled". These new articles covered the previously undocumented Counter-Strike: Source bot (added by the bot's author, Mike Booth) and Half-Life 2 AI, source control for Source engine mods, and more.

Licensed Source games

 * Valve
 * Counter-Strike: Source
 * Day of Defeat: Source
 * Half-Life: Source
 * Half-Life 2
 * Half-Life 2 episodic series (One, Two and Three)
 * Left 4 Dead
 * Portal
 * Team Fortress 2


 * Other developers
 * Alien Swarm: Infested
 * The Crossing
 * Dark Messiah of Might and Magic
 * Deep 6
 * DinoHunters
 * Garry's Mod
 * The History Channel's ShootOut! The Game
 * Kuma\War 2
 * Natural Selection 2
 * Postal 3 (PC and Xbox 360)
 * Prime
 * SiN Episodes
 * The Ship
 * They Hunger: Lost Souls
 * Twilight War: After the Fall (production resumed)
 * Vampire: The Masquerade - Bloodlines
 * Mabinogi Heroes
 * Untitled Threewave Software game