At the Game Audio Network Guild (G.A.N.G.) Awards 2024, our SIDE team based in Montreal took home the prestigious award for Best Sound Design for an Indie Game for their recent collaboration on SEASON: A letter to the future, developed and published by our friends at Scavengers Studio.

The team’s work is considered a massive technical achievement, and the result is a truly one-of-a-kind immersive audio experience. That’s why we’re excited to give you an in-depth look at their process developing the sound system for the game, in this article filled with audio-visual excerpts and fun facts, as originally published on the Audiokinetic blog.

Grab your headphones and let’s dive in!

About SEASON: A letter to the future

SEASON: A letter to the future is an atmospheric journey filled with soul-searching, contemplation and reflexive mechanics. In Season, you play as Estelle, a young woman leaving her home village for the first time. She embarks on a bicycle road trip to explore the world, documenting and recording memories in a special journal, before a mysterious cataclysm washes everything away.

In the game, you flounder to grasp the enigmatic world around you and capture its essence before it is too late.

The game has a very unique way of giving weight to your actions: instead of looking for audio logs lying around, you are encouraged to take pictures, document your encounters with the world’s inhabitants and record various elements of Season’s extremely detailed soundscapes. All this in order to fill the protagonist’s journal and reveal the story of this wavering world.

The field recorder

As sound professionals, this game felt tailored to one of our most beloved practices: collecting interesting sounds. The feature appeared first in the 2020 Game Awards Trailer:

The video created a strong buzz in the gaming community and led Scavengers Studio to give the feature an even bigger role in the story and game design.

The final design arose out of a process of collaborative iteration between the SIDE Montreal audio team and members of the design, programming, and narrative teams of Scavengers Studio. Using a series of various “gyms” in Unreal, we tested out a bunch of different ideas for how the concept could be approached, and how it would in turn work within the mechanics of the journaling system (and, in turn, the unfolding of the narrative itself). We landed on what was probably the most ambitious of the possible approaches, using not only in-engine audio capture but having the recordings trigger journal entries that would reveal and deepen the storyline depending on what was recorded. The idea was essentially taking what the concept of “photo mode” did for in-engine photography and applying it to audio.

We wanted it to be as “realistic” as possible, allowing the player to record any location and store those recordings in their journal. When playing back the file from your journal, you are indeed hearing your actual recording, and not just a polished ‘pre-baked’ version of what you attempted to record.

The mere presence of this feature greatly influenced the way audio was designed and implemented—it required certain bespoke tech to be developed by our team.

The recorder in action:

Playback of recordings in your journal:

Creating the soundscape

As hinted in the previous paragraph, one of the biggest challenges in designing the game’s soundscapes was that every location and object needed to be “recorder friendly,” in a harmonious and homogeneous manner.

Usually, the ambiances of a game are a combination of three different techniques:

1. 2D sounds linked to location;

2. 3D sounds linked to objects (on Unreal Engine Blueprints);

3. 3D sounds hand-placed in the environments.

Since 3D sounds add randomness and uniqueness as opposed to 2D ambient pads, we relied heavily on the former. After some research, we decided to hand-place sounds on pretty much every element of the landscape. Each tree, each bush has its own sound source. There are literally thousands of them!

We chose this solution over the use of Blueprints to have the freedom to fine-tune individual locations, which was a time-consuming but worthy endeavor. We used the same process to fake the continuity of a river or a cliff, placing sound sources all along its path.

This system wasn’t without its drawbacks, however. We had to use a lot of randomization to avoid phase issues on sounds occurring simultaneously, along with a pretty heavy priority system based on proximity to keep the voice count and performance under control.

How does it work?

The recorder itself takes the form of a custom plugin coded directly in Unreal in C++ using the Wwise API. The master output of Wwise is recorded as an audio file in the app’s save data. When triggered from the journal, this audio file is sent back in the middleware using the Wwise audio input plugin, which allows it to be played back in the actual gameplay. These recordings can also be brought back into the diegetic world of the story, as they can be played back to some of the NPCs to have them elaborate on their meaning within the sonic landscape and its interwoven narrative.

How does it feel to record sounds in-game?

There are tremendous changes in sonic perspective once you put the headphones on and try to record something. They say that humans listen, but a microphone simply hears. Monitoring the sound of a microphone, you suddenly get to hear all the details that your ears usually filter out and feel how skewed our perception of sounds truly is. We wanted to recreate this precise sensation.

Working across multiple departments, we workshopped a number of different methods to try to “focus” the listening. A reference mentioned was Hildegard Westerkamp’s sound art piece “Kits Beach Soundwalk”, in which she narrates focusing in on the “tiny” sound of hissing barnacles on Kitsilano Beach during a recording session in Vancouver Bay and recreates the psychological experience of close listening using studio post-production techniques.

We wanted this feeling of close or “deep” listening to a single sound object, but we didn’t want to lose the background environments as it would make the recording feel too staged and artificial when it was played back (and the environmental context around each sound was a big part of what evoked its sense of place). We also wanted the player to have the ability to record the more general environmental ambience, not just the sounds that were staged as being narratively important. As with the camera, it was essential to give the player as much freedom as possible with its use.

To achieve this, three different states influence the real-time game’s mix and focus in relation to the use of the recorder.

1. Recorder off: normal gameplay

2. Recorder in hand: monitoring

3. Recorder in hand: recording in progress

State 1: Normal gameplay

In this state, the focus on 3D elements emanating from the soundscape is quite narrow, meaning that you hear primarily what is in front of you (except for keepsakes, which are described below). No other mix modifiers are used.

State 2: Recorder in hand, monitoring

When drawing the recorder out, music and voice-overs are muted. The focus on 3D elements is much more pronounced. 2D ambiences are lowered by 6dB. Furthermore, « Keepsakes » (see below) are boosted by 5dB to get even more presence. Finally, compression, changes in the frequency prevalence (EQ) and a slight distortion are added to the overall mix. This simulates listening through headphones connected to a vintage recorder. For the purpose of the game, the microphone is deemed stereo.

State 3: Recording in progress

State 3 is pretty close to state 2, except the changes in the frequency prevalence (EQ) and distortion are bypassed/removed in order to get a clean recording file. The compressor also has slightly different settings.

The PS5 headphone mix uses Sony’s Tempest system for 3D audio, which makes all these mixing variations even more obvious and perceptible when you switch to your stereo recorder as the spatial placement of the sounds is more emphasized and directional.

Keepsakes

“Keepsakes”, or “collectibles”, are standout 3D elements we want the player to record and document. In many cases they were first designed by our SIDE Montreal team, before being submitted to the game’s writers and designers, who would then include them in the story and level design. This is one of those precious times where the audio influences the level design!

We fondly remember a certain character snoring which was so iconic on the audio side that they decided to add it to the keepsakes. It might or might not be one of SIDE Montreal’s team snoring in real life!

Keepsakes have 2 different states: before and after being recorded.

Before being recorded

Unrecorded keepsakes are louder and heard more often. They sidechain (reduce the volume of) other environment sounds to attract attention. They also have their own, longer attenuation curves so that we hear them from a distance. Finally, they have a much broader focus so they can be heard even when the player is looking away.

After being recorded

Documented keepsakes don’t need to stand out as much. Their frequency of occurrence decreases; for instance, cicadas will emit their characteristic sounds less often. The sounds will also be softer, and their sidechain is disabled. They also switch to the same attenuation curves and focus settings as other regular 3D elements.

Additionally, successfully recording a keepsake prompts specific lines of dialogue offering some insight into the game’s story, further encouraging the player to document all the collectibles they come across.

Memory flowers

Flowers are special keepsakes which, in contrast with the others, have a different sound depending on the state of the recorder. In normal mode, you hear magical ethereal sounds with a glimpse of unintelligible reversed and scrambled voices (which were cut up into pre-baked snippets using granular synthesis and placed in randomizable containers). Recorder in hand, the processed voices become prominent, with a touch of magic aura. Finally, when being recorded, you can hear and understand the vocal memory kept in the flower, though they still carry a trance of the “ethereal” effect using some subtle vocoding on the voice lines. Audio Director and Composer Spencer Doran designed these different stages with Kevin Sullivan, the game’s narrative and creative director, and mocked up a few different prototypes in his DAW to explain the direction.

Haptic feedback

Keepsakes

Whenever you hold the tape recorder in your hand, nearby keepsakes cause the controller to vibrate. The closer you are, the more intense the vibrations. The effect is particularly sophisticated on PS5 where the controller’s vibration also hints at the direction of the sound by panning it between the two handles of the controllers based on the source location. Haptic feedback is therefore used as another way to guide the players toward their goals, and there are various ways to increase the intensity of effect in the settings for deaf or hard-of-hearing players.

Also, as sound is such a central element within the recording process, we wanted all players to have an experience of it in some manner, so the controller haptics also act as a way to transfer (or translate) sound to the sense of touch. We made a system where the audio of the collectible sound sources is also routed to the haptics bus (as the DualSense uses a Left/Right audio feed to control the vibrations with sub-bass range low frequencies) and pitched into the haptic vibration range if necessary (on the DualSense, this is below 500Hz) so it becomes a physical sensation as well as an auditory one, and the “feeling” of recording becomes multisensory.

All sounds are not created equal in the world of haptics, though. Some keepsake sounds naturally created interesting vibrations. Others needed to be processed in Wwise with various effects (like EQs or distortion) before being sent to the Wwise Motion bus to be effective and readable. Furthermore, other sounds didn’t work at all and were actually replaced by a Wwise Motion source. The sound always needed to make the controller vibrate in an interesting way.

Bike riding

Haptic feedback was also used to enhance the biking experience. The first few pedal strikes send a strong signal to the controller. Once biking, each surface has a specific haptic feel, created by its own Wwise Motion source and modulated by the bike’s direction and speed. We thus needed to make a number of effect haptic “textures” to match the different kinds of surfaces in the level design which in turn also matched the sound design.

Conclusion

Working on SEASON: A letter to the future, and particularly on the unique mechanic of the tape recorder, was a sound designer’s dream. The game’s soundscape makes the immersion feel authentic and organic, and the gameplay elevates the sound design – crafted lovingly by the team at SIDE Montreal, along with the work and vision of Spencer Doran – and gives it total center stage. It goes to show that truly innovative and creative sound design doesn't always mean flashy sounds born of quasi-infinite layering, and that sometimes the simple and peaceful hum of the wind or water running down the valley can be all you really need.

This article, like everything produced by SIDE Montreal, is the result of the collaboration of a multidisciplinary team. Nikola Viel and Manuel Silva wrote the core of the text, but notable contributions were made by Dylan Escalona, who acted as a Lead Audio on the project and designed systems in Wwise; Alexandre Choinière, audio programmer who wrote the recorder plugin; Félix Leblanc, who with helped with editing; and FX Dupas, who made it all even better with his insights and inputs. Thanks also to Spencer Doran for his inspiring music and audio direction, and his help in contributing to this article.

*Bonus content*

The SIDE Montreal (formerly Vibe Avenue) team joined Audiokinetic with Audio Director & Composer Spencer Doran for a special episode of Wwise Up On Air – further talking through the ambitious and thoughtful music systems of SEASON: A letter to the future, the binaural in-game field recorder, haptic feedback, and more. Watch the livestream replay below!