Adaptive Soundtracks

This is something that I’ve thought about for quite awhile and would like to share with everyone. While I’m not a musician, composer or have any professional qualifications in sound design, I feel I do have relevant views on these subjects since I am part of the audience.

The evolution of the video game soundtrack differs greatly from what can be heard at the movies… at least that’s how it used to be. The electronic beeps, boops, buzzes and obvious synthetic effects are becoming a thing of the past as most video games become more complex and tend to become a lot more cinematic. Of course, the obvious difference between a cinematic movie and a cinematic video game is that a movie is viewed in a linear form, from start to finish with no ability to change the events of the movie which in turn isn’t able to effect any changes in the soundtrack. Continuing on with stating the obvious, the interactive nature of a video game opens up many opportunities to create soundtracks which adapt and change depending on how we as the player interact with the game environment. Adaptive soundtracks have evolved to a certain point with video games, but there is room for growth that I’m sure most people haven’t really thought of.

For the most part, the level of interactivity we hear with most current video game soundtracks amounts to scripted events. When Solid Snake in Metal Gear Solid is sneaking around Shadow Moses Island, the music is subdued and suggests to the player to remain cautious as they proceed with absolute stealth. However, as soon as Snake is discovered, the alarms go off, the guards go nuts and the music, in a split second, becomes action packed to suit the current situation. After awhile, when the guards haven’t found and killed Snake, they go back to their rounds but remain cautious with the music changing yet again to reflect this new mood. Finally, the guards are less cautious and the mood returns to normal with the original music playing again.

Many games follow that formula for their soundtracks, and why not? It is a very effective way of creating a cinematic scene using linear music elements which seems like it’s adapting to the way the player is interacting with the game environment. Some games do this more effectively than others depending on how much time and effort the developers are willing to place into their sound design. And, of course, a game doesn’t have to rely on cinematic story telling to use this method. Scripted events work well in racers as well, allowing the last lap to feel faster and more intense as the music changes accordingly no matter what time you pass the line.

Thanks to the introduction of larger storage devices, we’ve moved on from using MIDI based music to using pre-recorded tracks that give a much richer audio experience. However, the use of pre-recorded media will, at some point in the future, be overtaken by MIDI… I’m sure many will disagree with me on that point, but hear me out. When people think of MIDI, they immediately assume it to sound synthetic and unnatural. While I agree that a lot of MIDI based audio does sound that way, that doesn’t mean that they all do. In fact, it is possible to create very convincing music tracks that sound just as natural and organic as a live orchestra. The increasing capabilities of video game consoles are making this level of quality a reality.

But why would we want to go back to using MIDI in the first place and why is it so important in terms of adaptive music? I’d like to point out The Legend of Zelda: Twilight Princess which has some interesting effects made possible by its MIDI based music.

When you’re in Hyrule Field, the music will differ whether you’re on foot or on your horse. On foot, the music doesn’t seem very grand and the number of instruments is quite few, however as soon as you get on your horse, the music changes to something which sounds a lot more epic and you discover that the instruments have changed and more have been added as you gallop full speed across Hyrule. The transition in the music all occurs without skipping a beat and can revert back as soon as you get off the horse. With the same theme playing, if you move close to an aggressive creature or enemy, the music will begin to change again in real-time to include more instruments that signify the danger.

It is possible with the use of high quality MIDI soundtracks and some pretty advanced programming to create a cinematic game where the music plays to your every motion as if the music was composed to follow your actions as they happened. I imagine that same scene I mentioned earlier in Metal Gear Solid with a difference.

As Snake sneaks around, a guard approaches his position. Snake slides himself up against the wall and waits. As he does and the guard gets close, the music changes from its standard beat, becoming slower which adds to the suspense. The guard stops for a moment and aims his weapon into the darkness causing the music stops for a moment in reaction the guards actions. The guard then sighs and turns back and as he does the music resumes, naturally flowing from the silence back into its normal stride. Snake sneaks up behind the guard who has his back turned to him and as he does the music changes slightly to highlight the risky nature of Snake’s moves…

I guess you get the idea… If you still don’t, just watch any movie where the music seems like it’s being played to the actions of the characters on-screen and you’ll realise what I mean. The composer of a video game doesn’t know exactly what the player is going to do in exact terms. All they can do is create pieces of music which fit certain moods with their most dynamic compositions saved for the cut scenes where all of the actions have already been scripted in advance. It is possible to create a piece of music that will dynamically change and adapt to the players actions as they happen, the question is… when are we going to hear this level of interaction occur in our video games?