Hey Newgrounds, we're back with another devlog/guide that we hope you find interesting!

The other week, we sat down with Stone Story's sound designer, Rafael Langoni Smith. Rafael has a long and storied career with music, and he brings an incredible spread of talent and experience to the project. We picked his brain about the role of music in video games, how to compose for games, and the evolution of game music at large.

If you'd like to hear some of the music that was composed for Stone Story, check out this playlist on our YouTube channel: https://www.youtube.com/watch?v=JmlwzBg6Z9g&list=PLuSmtYLrtlFfCcPqxwctGLdFD4czjeek2

Additionally, if you'd rather listen to this talk instead of reading it, we've uploaded the Q&A on our channel: https://youtu.be/iQiqPNWP--o

---

Who you make your music for

You don’t have to be a composer to make a soundtrack, you have to be more of a dramatist. You have to manipulate music in a way that it reminds people of things, where it makes them associate with something they lived in the past. It’s not about doing your music or making your print in the world. It’s not about self-expression, as much as it is expressing whatever the listener has to feel. When you make soundtracks, you work with the cultural references of whoever is listening. Not yours.

As a sound designer you cultivate an emotional, evocative experience in the player by tapping into shared cultural contexts. Stone Story has these anchor points that much of the soundtrack is based in.

The beautiful thing about making music for our generation is that we have a pool of shared knowledge that’s very deep. Everyone shares a group of influences that comes from the start of video game music, and we have this very emotional, nostalgic connection to each other. We can listen to the same things and communicate; video game music is a very universal means of expression.

Exploring a shared cultural connection

Composing music is somewhat like playing a game withyour audience. It’s true of a lot of creative processes, where you have to imagine how the listener is going to think and feel. 

It’s all based on expectations. When you write music and it’s easy to understand, when it’s very referential, when it’s easy to apprehend, it’s like you’re losing the game. You’re letting the listener win, because the music is so easy to “get” and the audience is in a superior position to you as the composer. When you up the bar and make it not so easy to understand, you’re in a more balanced game. You’re dealing with expectations, you’re surprising the person at times, you’re presenting new stuff. 

Be confident in your shared cultural background with your listeners in order to explore the language you have from all the games you’ve played throughout your life. You can be pretty sure that everyone who listens to the soundtrack will understand everything that you did. It won’t be easy for them, but that challenge can excite and entice them. The role of sound and music in video games can be thought of as opportunities to make people listen to different stuff and widen their musical horizons.

That might sound contradictory when we say that we’re trying to get people to listen to new things while at the same time tapping into this shared body of knowledge. But it’s like this: everyone that’s playing Stone Story, presumably, will have part of the culture that the development team has. You don’t really get to our game as the first game in your life. It’s likely that if you’ve looked into playing Stone Story it’s because the fact that the graphics are in ASCII mean something to you. 

So this person that’s going to play Stone Story has lived throughout decades of video game music that are meaningful. When creating the soundtrack, that can be an advantage. It’s like talking to old friends: you have so much shared history. You can tell them new things, but they’ll have enough in common with you to care about it. Music, when it comes to video games, has developed into a pretty universal language as a way to communicate feelings and thoughts. It’s beautiful to be able to be verbose in that language by bringing new things.

Creating a soundtrack based around the concept of “progression”

One suggestion that came up in the process was each location having seven different incarnations, corresponding to appropriate levels of difficulty. When you start a location in-game, it starts at difficulty 0. Each time you beat it, it gets harder and gains a star. Each time the location gets harder, the music gets more complex. Initial thoughts were “That’s crazy”, as there are eight levels in the game, each with eleven difficulties.

It was a pretty big undertaking that was harder than expected. The original tracks were fully realized, and there wasn’t much to take or put in. They were effectively “full packages” that couldn’t just have instruments removed or added at will. The soundtrack ended up being multiplied. 

When the decision was made to create a progression-based soundtrack, most of the music was already done. This made for a bit of a problem, as the music that already been composed was made with equipment that had been setup in a specific way. Once a track was recorded it was near impossible to replicate that exact sound.

(Video showcasing Rafael's gear setup - https://youtu.be/pkd5w3wMZTI)

Some of the music had to be redone in order to simulate what was done before because Rafael, the sound designer, didn’t note down how sounds were created or recorded. With his gear setup, that was rather difficult to do, as it’s basically an old telephone switchboard: lots of jacks and cables that you can connect to and meters to adjust.

There are so many moving parts to the music that you can’t exactly keep static and have on-hand to reference. They all interact with each other in a way; they’re all distorting together, modulating together, etc. It’s never an accumulation of parts, it’s more of an engine that runs with different parts that all play off of each other.

Stone Story’s Temple location is one of the best examples for this concept of soundtrack progression. At the lowest level of difficulty, the Temple’s music probably has the least amount of action. It begins with these ambient tones that convey an ominous feeling of dread, where the soundtrack has taken a much more suspenseful, dramatic turn. Over time it evolves into a huge, bombastic, overwhelming track.

(Supercut of the Temple's music, showing off track progression - https://youtu.be/bOu0fT7IYWY)

The style of Stone Story’s music and “post-chiptune”

The style of the music in Stone Story is considered to be "post-chiptune": it's what happens after chiptune has gotten through all its cultural consequences. Chiptune has done its original phase, where the technical limitations of the game would dictate what was and wasn’t possible. Back then, people were trying to transcend the limitations of chiptune. They were trying to make the hardware do what it couldn’t do musically.

A few years ago there was a revival of the chiptune style. Everybody started to simulate the limitations again. They made music as if it was for a chiptune platform. What Rafael is calling “post-chiptune” is going past this phase of simulating the limitations and trying to use the style for what it was aesthetically, but with no limits.

It's the same for the visual style of the game, which is ASCII art without the limitations. It’s true to the style in a certain way, but we can go beyond ASCII with things like unicode characters or special effects where it makes sense to do so. Things that weren’t possible on DOS back in the day.

Influences and how to make use of them

As far as influences go, Stone Story’s music draws upon several different cultural contexts in order to cultivate a specific kind of mood. The Temple, for example, features Indonesian gamelan, which is a pretty new set of sounds to a lot of people. However, Rafael was able to use those sounds and have it not be foreign to whoever’s listening. The kind of rhythmic scheme used for the Temple is very tense, and the fact that it’s a foreign sound adds to the atmosphere of something bizarre going on.

There are many other influences playing into Stone Story’s soundtrack, but it mostly boils down to this idea of a shared culture: what all of us have played in the past and what we can use to talk between ourselves in a musical way. Sometimes you have an influence and you don’t even realize it. Self-analysis is tough! It takes a lot of thought to sort out where those inspirations can come from.

Synchronizing sound cues for the player experience

Some scenes in Stone Story were done like a movie: video was captured, then Rafael scored it like he would a film. Most of the music in the game, however, consists of loops and doesn’t synchronize with the action in any way.

What’s nice is that when you do get to those moments in-game where the music syncs up with the action on-screen, it’s pretty dramatic. You’re not expecting it to go that way. They create these beautiful moments, these revelations when someone is playing Stone Story for the first time. 

In some of these scenes, there’s a certain amount of cinematography, a kind of visual poetry that works independent of the game. These scenes, in conjunction with an accompanying score, create powerful moments because you need to reward the player for spending their time. This is an important part of the game design as well: Stone Story is a game that sometimes makes you wait and have patience, which, in this day and age, is something that people may not be used to. When you do that you’re rewarded with beautiful visuals and an awesome soundtrack!

Creating sound effects to make the world come alive

The thing about Stone Story’s sound design is that early on in the process, it was decided that the characters were living in a naturalistic world with a veneer of fantasy added on. As a result, that necessitated a sort of realistic sound design in a way where the wind would sound like the wind, grass would like grass, and things hitting things would sound like things hitting things! 

There are some synthesized sounds happening on top of the realistic stuff, quite a lot of reverb, and it’s used to simulate the game’s environments. Lots of soundscapes were done for the game’s locations; there’s music, but there’s also this underlying bed of sound that represents the natural world around the character.

For example, one of the scenes in-game involves a massive, rushing waterfall. In Rafael’s design process for what this waterfall’s audio design, he honed in on the unique sensation of seeing ASCII characters falling down and what that would sound like. You look at this very unreal scene and you obviously can’t fool yourself to make believe it’s a natural environment, but you listen to it as if it was.

The original direction for the game’s ASCII art was to represent what’s in peoples’ subconscious; how the mind deconstructs what it sees. It’s sort of a cartoon expression of the world. The role of audio is to add the realism that the visual doesn’t deliver. It’s a juxtaposition, which is important. 

When a sword hits an enemy, the audio needs to convey what material the sword is made of. You can’t show that, there’s no texture on the sword; visually you can’t tell! But is it wood, is it metal: what are these things made of? When a chest bounces on the ground, you can hear what’s inside the chest.

As you play Stone Story, you’re accumulating resources and thinking about stockpiling things like wood, stone, and tar. It’s all stuff that you can’t really see in the game, since it’s represented by symbols. You’re always looking at the game and forcing yourself to think of reality within the game that is realistic and naturalistic: it’s a departure from what it looks like.

While this approach to sound design may not be necessary for the game, it does make some things easier and helps to keep the immersion. The main function of the audio in Stone Story is to add this layer of gravity, emotion, tension, and realism to the visuals in order to draw the player in even more.