There are many applications and approaches, but the ultimate goal of the fast-growing VR industry is to deliver a truly immersive experience to the end user. Companies are working hard to develop the visual technology needed to provide these experiences, but there is still one major piece missing from the puzzle – audio.
Human eyes are fairly powerful, but they can only see 200 degrees, at most, of the view in front of them at any given time. Human ears, on the other hand, listen to full 3D space—front, sides, behind, above, and below—at all times. This is why synchronizing a user’s viewing and listening experience with 3D audio can take the experience to a whole new level. The interest in achieving that level of sophistication can now be seen across all segments of the industry.
Increased Consumer Interest and A Higher Standard for Immersion
When people watch content via a traditional medium like a TV or a theater, the screen in front of them represents a virtual world. There’s a separation between the real and virtual world, which makes it difficult for some viewers to remain engaged with the content. Through VR, however, people can isolate themselves completely from the real world, and when that happens, the entire adventure becomes incredibly visceral. “Being there” is the whole allure of VR, so sounds must be 3D to support the immersive nature of the experience.
Sound spatialization consists of placing positional data to sound sources and gives the listener the impression of a sound source within a 3D environment. When the team at G’Audio Lab, a leading spatial audio company, showcases two different demos — one with spatialized audio and one without — they notice a common pattern of reactions. After taking off the headset, people profess “The spatialized one felt so real” or “I didn’t know that audio made such a big difference.” The hunger for spatial audio becomes insatiable once consumers experience it at least once. There’s no going back.
Powerful New Tools and Revenue Streams for Content Creators
When delivered correctly, sound becomes a storytelling cue and can help lead the narrative in a powerful way. Since the audience can look or walk in every direction in VR, using audio cues is especially critical in designing affordance. Light or haptic signals can also act as directional tools, but sound provides a more provocative, immediate, and natural response.
One exciting opportunity for 3D audio centers around live concerts and sporting events. With VR, content creators can overcome physical limitations to serve the audience by reproducing and redistributing live events. But the true seduction of VR is the power to create experiences that could never exist in the real world. Why sit in the nosebleeds of a concert when you can be on-stage in VR, inches away from your favorite artist? As more fans engage with performances that way, it’s not crazy to think that 3D audio could lead to entirely new musical styles, potentially disrupting the music industry.
When music is delivered in stereo formats, the sonic experience is shackled by the finite quantity of loudspeaker placement. 3D audio can be delivered via an unlimited number of virtual speakers if the music sources are in an object-based format, or structured as individual mono signals. This allows content creators to experiment with new methods of sound design and produce projects that are totally different from the typical music consumption experience. Music is about to leap past traditional techniques, and performances can now take place all around you, not just in front of you.
Binaural Rendering Requires Talent and Technology
The combination of consumer interest and creator need has given birth to a thriving 3D audio market with high demand, but there are plenty of challenges when it comes to delivering the technology on a practical level. A personal and mobile VR experience inherently requires a pair of headphones for audio output, and delivering 3D audio through that two-channel format instead of layers of loudspeakers is no easy task. That’s why binaural rendering plays such a critical role in the VR listening experience. This signal processing technology synthesizes 3D sound scenes comprised of object, channel and/or Ambisonics into two-channel outputs.The corresponding signals should then be rendered in a way that reflects where the sound is actually coming from, taking into account the relative direction and distance between the sound source and the listener.
What’s even more challenging with VR is that 3D audio has to be interactive 3D audio. It should take into account the actions of the user, not just the sound source’s movements in the scene. This complex rendering process can severely burden hardware, so using minimal computation power while maintaining premium sound quality is key. It is a puzzle that could potentially be solved with hardware, but whoever can do it with effective software will have the best chance to win early market share. VR experiences without 3D sound are almost already obsolete, so continuous efforts to incorporate better binaural rendering technology are here to stay.
Old Barriers Are Breaking Down
In the past, audio engineers required an expensive studio with multichannel loudspeakers, amplifiers and mixers for monitoring sound outputs to work on 3D audio projects. This created a significant barrier to entry for small studios trying to break into the 3D audio world. However, with virtual reality today, the output is simply a two-channel format because VR adventures are always consumed with a pair of headphones. Studios now just need a computer, some decent headphones and a head-mounted display (HMD) device (for monitoring purposes) to master 3D sound.
Also, because of the growing interest for accurately capturing 3D sound, microphone industries will likely go through major changes. The quality of evolving equipment like Ambisonics microphones still leaves a lot to be desired, highlighting the need for more research and development. That same equipment is still very expensive, and there are already efforts to disrupt this market utilizing more efficient technological advancements. 360 degree cameras are becoming a more common sight in the hands of consumers, and hopefully, with affordable prices, 360 degree microphones will follow closely behind.
Since best practices aren’t fully established and no major player has already conquered the market, nearly anyone can excel in this field by taking advantage of first-mover effect. We may see a small yet agile studio rise to prominence.
About G’Audio Lab
G’Audio Lab is a spatial audio company dedicated to developing immersive, state-of-the-art audio production software solutions for creative professionals. Combining 3D sound with interactivity, G’Audio’s suite of tools, including G’Audio Works, G’Audio Sol and G’Audio Craft, offer full stereophonic sounds and seamless integration for cinematic VR, 360° video and VR gaming projects.
G’Audio’s team of experienced sound experts hold numerous patents and the company’s core binaural rendering technology was adopted as part of next generation international 3D audio standard MPEG-H 3D Audio.
For more information on G’Audio, its sophisticated 3D audio technology and premium suite of products, visit http://www.gaudiolab.com and follow G’Audio on Facebook, Twitter and LinkedIn.
Co-founder and CEO of G’Audio Lab
