Sound design in VR is a staple of what makes a deep and immersive environment able of captivating an audience and also for that environment to express emotion and spatialization relevant to the scene it describes. In providing entertainment in VR, often the senses that are able to be utilised are limited to sight and hearing and therefore audio design is important in providing the end user a great experience.
There are many forms of audio design that can be used to effectively used to convey genre and emotion. One VR game that does this well is a game called “Phasmophobia”. The general idea of the game is that of an investigative horror game played from a first person perspective where the player works alone or in a group to complete a contract in which they must identify the type of ghost haunting the specified site. Players can use various basic equipment such as spirit boxes, video cameras and UV flashlights for example in order to complete this goal.
Phasmophobia, although not next generation in it’s visuals, graphics or narrative, is elite in conveying fear and horror using it’s well designed audio soundscape and the developers have not overlooked audio as element that is essential in delivering that heightened sense of immersion.
One technique they utilise well is the implementation of wide spectrum sources and a borad spectrum of sounds including wind sounds and high frequencies to spatialize effectively and provide a lot of frequences for the HRTF (Head Related Transfer Function) to work with, also helping mask audible glitches resulting from pan and attenuation. The high frequencies are heavily used by humans for sound localization.
The developers have also leveraged 3D audio spatialization which provides much more accurate spatial cues, including height, and with this improved accuracy, in addition to volumetric sources, these audio techniques truly create a terrifying space when you are in the game. With the environment emanating such eery noises that lock you into the immersion of your surroundings and aware of the genre/emotions the game is trying to express, the developers also add ghosts with their own immersive sound design. The use of the doppler effect that changes a sound’s pitch as the source approaches or recedes makes the movement of the ghost (when heard) keep you always attentive.
The influence of an ambient nature soundscape and movement-triggered step sounds are also used to further drive that feeling of presence. Using the ambience as an auditory stimuli is great in influencing presence and creates a virtual environment that feels alive and immersive. The sounds of walking interact with the environments in their sounds and tempo giving the user a feeling of direct interaction with the surroundings. As these audio cues and implementations get better, we move towards even greater presence where the user is under a more believable impression that they are in a virtual environment with less awareness of a mediating technology, such as a game.
As mentioned before, interactive sound is one of the most important things in audio design when trying to deliver a feeling of presence. The game does this really well with its constant reminders that you are in the environment as all interactions with the surroundings provide responsive dynamic audio, such as footsteps and opening doors for example.
In VR, hearing is the only sense able to provide full spatial information going beyond our field of view, including elevation, 360 degrees and depth, allowing us to guide our decisions and behaviors as well as understanding our virtual surroundings. Not only is audio unique in being able to provide spatial information for the brain to analyze but also tell a narrative.
For me, playing this game really highlighted the importance of audio in immersion in the virtual realty space. However much a visual experience can paint a narrative, the sound design is what really adds the dimension that makes you believe you are living that experience. It can create tension, emotion and a sense of spatialization that is an important channel for information about their environment and creating presence. It is not going unnoticed as realistic and responsive spatial audio is quickly become a cornerstone for investment and key to the development of the so called metaverse by big tech companies. I for one am curious about the future of audio in VR and hope many new immersive audio technologies are used to create a deeper more believable virtual reality.