Can reported soundscapes foster deeper audience connections?
In this regular column, Ryerson University School of Journalism professor Adrian Ma explores emerging trends, technological developments and compelling work in the burgeoning field of immersive storytelling, including 360 video and virtual, augmented and mixed reality content.
When it comes to immersive storytelling, virtual reality and augmented reality generally gets the lion’s share of the attention. But interest is rapidly growing in “360” or “3D” sound, not only as being an essential aspect of an immersive video experience, but as an intriguing form of storytelling in its own right.
You may have heard the terms “binaural audio” and “ambisonics” which are different types of immersive, multidirectional spatial audio sound production. If you’re not familiar with the term, here’s a quick (and rather un-sciency) explanation.
Binaural audio is created by using two microphones to record audio in a way that essentially mimics the way human ears are positioned on the head. Indeed, many popular models of binaural microphone devices tuck the microphones behind plastic ears to replicate a human head, which can be kind of adorable or downright creepy depending on how you feel about ears growing out of your devices.
When we hear things, soundwaves make their way to our respective ears at different times and at different volumes. Conventional stereo also utilizes two channels of audio, but the sound is localized to the left or right, with no ability to hear above or below. Binaural audio offers the listener a greater sense of depth as the microphones capture each sound source in relation to the listener’s position in the environment. BBC Click has a video demonstrating how binaural sound works.
Ambisonic audio takes this to another level by recording sound using multiple microphone capsules arranged in a tetrahedral configuration. This allows for a complete 360 representation of audio which also responds and changes to the user’s head movements. Ambisonic audio is a vital aspect of making experiences like virtual reality gaming and 360 movies more realistic and immersive. The Berklee music school has a video primer about ambisonic recording here:
News outlets like NPR and the BBC have been increasingly experimenting with binaural sound in different ways, from concert performances to radio documentaries. Also exciting are the recent wave of podcasts incorporating spatial audio, many of which are audio dramas/non-fiction. Darkest Night, for one, is a trippy horror podcast that features narration and voice work from a pretty star-studded cast, including Lee Pace, Denis O’Hare and Keith David. Marvel has utilized ambisonic audio recording techniques for its acclaimed podcast “Wolverine: The Lost Trail,” which has also received high praise for its intense realism, sonic depth and innovative approach.
With major companies like Sony recently launching 360 audio platforms and a slew of new smartphones and smart speakers expected to be outfitted with spatial audio features, we could see a major boom in immersive audio storytelling.
As spatial audio recording gear and software becomes more affordable and accessible, I would be really curious whether radio reporters and non-fiction podcast producers could use this technology to add more experiential dimensions to their work. Naturally, spatial audio has found its way into audio documentaries because of how well it can reproduce environmental sound and foster a deeper sense of immersion. But could even an intimate conversation between a host and an interview subject be more compelling if the audience felt like they were in the room with them? Could the sounds of a riot or protest in a field report be more engaging if they were captured and presented in a 360 manner? While the jury is still out on how much (if at all) audiences prefer spatial audio over conventional stereo, as the widespread availability of 3D audio playback grows, immersive audio definitely offers intriguing potential for storytellers.
Adrian Ma is an award-winning journalist and an assistant professor at the Ryerson School of Journalism in Toronto. He has spent most of his journalism career as a reporter and digital producer and specializes in teaching multimedia storytelling and personal branding.