Often when I listen to game produced audio (either when playing an audio game, or looking over the shoulder of a sighted player), it struck me how sounds resulting from consecutive similar actions sound exactly the same. A good example of this is the sound of an opening door. In most games one can open a door 10 times, and each time it will sound exactly like the other times. This effect can also be noticed with most musical keyboards, especially with the wind and string instruments. In real life the sound of an action is dependent on many factors, including the environment in which the action occurs, the velocity of the action, and whether a random cosmic disturbance took place that very moment. Some games try to simulate this variation by having more than 1 sound assigned to an action, playing a sound at random each time the action occurs. Pitch variation is also used by both games and musical keyboards to simulate differences in sound, but the observant listener will still detect the similarities.
This problem is do to the fact that sound effects are mostly produced by playing sound waves from storage. A possible alternative to this approach is to use sound synthesis for some action sounds. This will enable more variety in sound effects, and will also save storage space — which may be especially useful in a mobile setting. This was indeed the method employed on the early game consoles, when the sound chip consisted of nothing more than 3 tone generators and a noise generator. Some people may be concerned about the artificial nature of synthesized audio for sound effects, but it is important to remember that many sounds in both games and movies are already artificial. The classical “dish!” sound often employed during fist fights is one example. As any one who has ever witnessed a bar fight can attest, fists don’t sound that way in real life. This sound was traditionally produced by hitting the end of an empty toilet roll close to a microphone.
There exist a number of sound synthesis methods, each with it’s strengths and weaknesses. Additive sound synthesis works by the principal that all periodical wave forms can be produced by adding sine waves of different frequency’s and amplitudes together. This proved to be time consuming to do with hardware synthesizers, and hence this synthesis method went out of favor. This is however becoming more popular again as it is easy to do in code. A program demonstrating this synthesis method is Spear (www.klingbeil.com/spear/). Frequency modulation synthesis (FM synthesis) is another synthesis method that is similar to FM radio in principal. The most basic setup involves a carrier sine wave with a frequency determined by the amplitude of another sine wave called the modulator. More complex setups involve different wave shapes, or modulators chained together. FM synthesis can be used to create approximations of existing sounds and was used extensively in MIDI synthesis chips of early cellphones. Subtractive synthesis is yet another synthesis method by which an initial wave form is shaped by subtracting wave forms from it. It can for instance be used to generate the “dish!” sound by starting with white noise, subtracting all sound but a narrow frequency band, and applying an envelope to it (varying it’s amplitude over time to produce an attack at the beginning of the sound, and a fast decay towards the end).
Programming languages exist which are specifically designed for sound synthesis and the researching of synthesis methods. The oldest language still in use is the CSound language (www.csounds.com), which attempts to be the “C” of synthesis languages. The following is an example of CSound code — the definition of a basic FM synthesizer.
instr 1 ; start of an instrument definition
amod oscils 300, 880, 0 ; modulator sine oscillator — arguments are amplitude, frequency, and phase
acar oscils 10000, amod+440, 0 ; Carrier oscillator with frequency equal to 440 (A above middle C) added to modulator
out acar ; send sound to output (sound card or file)
endin ; end instrument definition
CSound contains over 1000 unit generators (routines that produce sound samples). Super Collider (supercollider.sf.net) is another synthesis language, but designed for real-time sound synthesis and live coding. Live coding involves writing code live as to produce sound while a sound performance is happening.
Sound synthesis can not replace all sound effects, many sounds are very hard to produce with current synthesis methods. It can, however, be used for many sound effects which are currently prerecorded and played from storage. It is especially suitable for sounds which are artificial by nature (the “dish!” mentioned above, or the sound of electric equipment). It can also be used to process recorded sound, as to add more variation in sound effects. I hope to see more sound synthesis being employed in the future, specifically in games.