Adi Kamaraj

Sound design in VR is a staple of what makes a deep and immersive environment able of captivating an audience and also for that environment to express emotion and spatialization relevant to the scene it describes. In providing entertainment in VR, often the senses that are able to be utilised are limited to sight and hearing and therefore audio design is important in providing the end user a great experience.

There are many forms of audio design that can be used to effectively used to convey genre and emotion. One VR game that does this well is a game called “Phasmophobia”. The general idea of the game is that of an investigative horror game played from a first person perspective where the player works alone or in a group to complete a contract in which they must identify the type of ghost haunting the specified site. Players can use various basic equipment such as spirit boxes, video cameras and UV flashlights for example in order to complete this goal.

Phasmophobia, although not next generation in it’s visuals, graphics or narrative, is elite in conveying fear and horror using it’s well designed audio soundscape and the developers have not overlooked audio as element that is essential in delivering that heightened sense of immersion.

One technique they utilise well is the implementation of wide spectrum sources and a borad spectrum of sounds including wind sounds and high frequencies to spatialize effectively and provide a lot of frequences for the HRTF (Head Related Transfer Function) to work with, also helping mask audible glitches resulting from pan and attenuation. The high frequencies are heavily used by humans for sound localization.

The developers have also leveraged 3D audio spatialization which provides much more accurate spatial cues, including height, and with this improved accuracy, in addition to volumetric sources, these audio techniques truly create a terrifying space when you are in the game. With the environment emanating such eery noises that lock you into the immersion of your surroundings and aware of the genre/emotions the game is trying to express, the developers also add ghosts with their own immersive sound design. The use of the doppler effect that changes a sound’s pitch as the source approaches or recedes makes the movement of the ghost (when heard) keep you always attentive.

The influence of an ambient nature soundscape and movement-triggered step sounds are also used to further drive that feeling of presence. Using the ambience as an auditory stimuli is great in influencing presence and creates a virtual environment that feels alive and immersive. The sounds of walking interact with the environments in their sounds and tempo giving the user a feeling of direct interaction with the surroundings. As these audio cues and implementations get better, we move towards even greater presence where the user is under a more believable impression that they are in a virtual environment with less awareness of a mediating technology, such as a game.

As mentioned before, interactive sound is one of the most important things in audio design when trying to deliver a feeling of presence. The game does this really well with its constant reminders that you are in the environment as all interactions with the surroundings provide responsive dynamic audio, such as footsteps and opening doors for example.

In VR, hearing is the only sense able to provide full spatial information going beyond our field of view, including elevation, 360 degrees and depth, allowing us to guide our decisions and behaviors as well as understanding our virtual surroundings. Not only is audio unique in being able to provide spatial information for the brain to analyze but also tell a narrative.

For me, playing this game really highlighted the importance of audio in immersion in the virtual realty space. However much a visual experience can paint a narrative, the sound design is what really adds the dimension that makes you believe you are living that experience. It can create tension, emotion and a sense of spatialization that is an important channel for information about their environment and creating presence. It is not going unnoticed as realistic and responsive spatial audio is quickly become a cornerstone for investment and key to the development of the so called metaverse by big tech companies. I for one am curious about the future of audio in VR and hope many new immersive audio technologies are used to create a deeper more believable virtual reality.

Skyrim VR

When I laid hands on the first fantasy game that I had ever played, The Elder Scrolls V: Skyrim, I was immediately hooked. The open-world game was rich with lore and diverse regions to explore. Coupled with great soundtracks and well-designed ambience, Skyrim was able to draw players in with its deep immersion – only limited by its technological interfaces at that time. After all, despite its best efforts, players could only interact with the game through a mouse and keyboard, and see the expansive world only through a 2D screen. However, all this changes with the introduction of virtual reality.

For the first time, players were able to truly walk around and explore this world with new eyes. They could stand in the middle of a wide open plain and “look around” at the scenic views the game had become well known for.

Combat was enhanced in particular through the use of VR. Gone were the days of button mashing and remembering the keyboard shortcuts for combat. Now, players get to hold weapons and shields in their hands, or actually wield magic should they choose to. They now come face to face with their enemies and feel as though they are truly in battle – presented with a level of immersion impossible without VR.

However, intense immersion may come with a few drawbacks in such an immersive combat fantasy game. Experiences that were once far removed, may now actually feel “lived” by the player.

Draugr depicted in the image above, are zombie like creatures in the game. Imagine the scare players may experience when they feel like they are actually placed in the world. They go about minding their business and find themselves stumbling right into such a creature. Such experiences may feel much more vivid in VR, and game designers will have to take this into account.

Blade and Sorcery VR

Another VR game that I really like is Blade and Sorcery. Unlike the story and immersion centred Skyrim, Blade and Sorcery is a multi-player combat game that allows players to duke it out in an arena. While realistic combat is difficult to come by in games, Blade and Sorcery is at the top of its class in providing it. Combat and magic are realistic, and allows players to relive their childhood fantasies with its wide array of mods. Players can battle with lightsabers, or in medieval gear and with magic. The game also allows players to perform “trickshots” in combat, and have become an increasingly popular sub-goal in playing the game.

Despite its ability to bring realistic combat to the player, its visuals do seem clunky, and movement may not always be very natural. Players have complained of weird perspectives in game regarding their in-game limbs, which do break immersion for some players. Overall, it is an enjoyable game to play and sets the foundation for VR games of the future.

About the APP

SkyView is a lightweight stargazing app available on iOS and Android devices. The augmented reality technology allows users explore different constellations, planets, star clusters, stars and other celestial bodies in the night sky through their smartphone’s camera. Simply pointing the smartphone to the sky, then SkyView will identify the stars and other objects in the view.

Why do I like it & Why it is engaging?

Have you ever imagined a date with your boyfriend or girlfriend, you guys take a walk along the beach, and then she or he points to the sky and tells you what constellation that is? Romantic, isn’t it?

I never imagined that until I met a boy who loves boys, we walked on the west coast park and identified all the constellations we could see. Still romantic. Hence, I find I can have such romantic experience without a boyfriend or girlfriend. I can do this by myself, with SkyView.

I downloaded the App months ago, and I found it is easy to set up and use. It brings a fascinating view of night sky to users with the stylish UI and the well designed graphics. The main function and interaction of the application are quite simple, so that many parents also use this as the stargazing education for their kids.

What features are well done & What can be improved?

The main function of the app, the AR stargazing camera, has simply interactions. A short description of the star or other objects will be shown on the side of the screen when you click on the object. This helps users connect the star on the screen and the star they see in the sky quickly.

Users are also allowed to increase and decrease the magnitude visibility of planets and stars, this make it easier to see the stars and the brightness of them for users who are in a region of high light pollution.

However, the night mode is not clear as the usual mode. The filter is not very user friendly and covers some views in the camera. The same planet and star size settings lead to the less brightness of stars. The color of the filter can be changed to green or other light colors instead of red. The size setting of objects could be larger so that it will be clearer in the dark.

Overall, SkyView is a fascinating and romantic AR stargazing application which is easy to start and use.

There are so many amazing places that one would like to visit during one’s lifetime. With a bit of luck and resources, one may be able to travel to a few of them. However, to travel to all of them would be impossible… or would it?

With Google Earth VR this could be possible.

The industry of Virtual Reality (VR) is growing at a fast pace. Its many different applications are not only recreational but may also be educational and be more than just a distraction. The Google Earth VR application allows the user to strap on a VR headset and start exploring all the wonders of the world from the comfort of one’s room.

Why they are engaging?

During the rough times of COVID-19, the possibilities of travelling have been severely limited. With the use of VR technology, the world is opened up again without the risk of transmitting the virus. Google Earth VR uses the 3D model of Google Earth, which has mapped 98% of the World. However, it is not just Google Earth ported to VR, it is a completely new experience. All the interactions have been redesigned and the VR application allows flight as the mode of travel, both at different locations but also in-between them.

This allows a very engaging experience for the user. The application is fast-paced and allows for many different amazing sights in just a couple of minutes. But it is not only that the user gets to tick off locations from their bucket lists at a pace not possible without this technology. While travelling in VR, the user also gets to experience the location in a brand new way, from many different perspectives which would not be possible if physically present.

With the application, one can fly over any city and experience them in impressive VR, stand at the highest mountain peaks of the world and appreciate the view in all directions, walk along streets in remote destinations you never thought you would get to visit, and much more.

What features are well done?

The Google Earth VR application is impressive in many ways. The sheer scale of the project is unimaginable, as Google Earth covers almost all parts of the world on a real-life scale. The controls and the straightforwardness of the application allow the user to effortlessly explore the world in an immersive way as if one was there in person.

What features can be improved and how?

However, the features of Google Earth VR are not all perfect. Due to the scale of the project, Google has been required to do a lot of optimization for the experience to be enjoyable. The 3D models are procured from combining enormous amounts of 2D images and layering everything onto 3D space. As a consequence of this, the detailing is not very accurate and sometimes certain graphics appear as blocky smudges of colours.

Improving this while still maintaining the scale of the project is hard. For the user, going to ground level (which activates Google Street View’s POV) results in much more details than the approximations in the bird’s view. As for what Google could do, this seems to be a project that does not get that much care anymore. However, with the use of Artificial Intelligence, Google could aim to do some photogrammetry repairs of textures, for example by combining the detailed Street Views with the textures in the Google Earth view.

To summarize, Google Earth VR allows the user to travel to incredible places across the world. At its current state, it may not be detailed enough in order to replace real traveling. It does however offer an immersive sandbox experience with new sights to explore anytime.

SketchAR is a mobile app developed in 2017 which revolutionised the way we learn how to draw. It projects AR stencils on the user’s drawing paper, and all the user has to do is to follow the stencils and draw accordingly. I like this app because it is very beginner-friendly. Users with no background in drawing can pick up a pencil and paper and draw right away. It removes the technicality of drawing and allows beginners to quickly immerse themselves into drawing. While this app is targeted to novices, experienced artists can also use the app and try a new method of drawing with AR technology.

The main feature of this app are the drawing lessons, which are broken down into smaller steps – each consisting of an AR stencil. Upon completing the stencil for the current step, the user presses ‘Next’ and the stencil changes into the one in the next step. There are many lessons offered on the app, such as animals, food, vehicles, anime, and architecture, to suit the different interests of its users. Users are also given the option to disable the AR stencils and follow the steps from their phone instead. Here is a video review of the app by an artist:

However, a major drawback of using the app is that users are recommended to attach their phones to a tripod while drawing. Otherwise, it will be tiring for the user to hold up his phone with one hand and draw with the other. Perhaps the use of goggles could be used to solve this problem. The user will wear goggles to view the stencils and free both of his hands, which could help him to concentrate on his drawing better.

Furthermore, the AR stencils restricts the area on the paper which the user’s hand can occupy. If the user places too much of his hand on the paper, the AR stencils will become distorted. A possible solution is for the app to continue projecting parts of the stencil that is covered by the user’s hands. In that way, the user can continue to follow the stencil without having to worry about touching the paper.

Another concern expressed by artists with regards to using AR stencils to learn drawing is that it hinders one’s creativity. Budding artists may become used to strictly following instructions such that they do not engage their own imagination. They will also lack drawing fundamentals which is crucial in developing and refining their drawing skills.

However, for the average user who wishes to draw for their own entertainment, this app is great for helping them to dive right in. When users sign up for an account, they can choose categories which they are interested in, and their drawing lessons will be personalised according to their interests.

Besides the AR stencils, this app boasts other cool secondary features as well, such as sketching from stencils designed by other users, sketching photographs from the phone’s gallery, and even a colour-matching game that trains users to recognise colours of different shades.

All in all, I would say that SketchAR is a great app which allows anyone to start drawing almost immediately, without any prior experience in drawing. Although it does have some setbacks, I believe that they can be rectified quickly since the technology is already there (goggles etc). SketchAR utilises AR technology in a fun and innovative manner that can transform anyone into an artist!

One of my favourite VR applications is called “Bogo”, because it seems like a very active game where users can interact with a virtual pet, which is a very expressive character with many different emotions. It is very engaging because users are free to pet or touch the virtual character anytime, and the character will respond accordingly with clear expressions on its face.

There are features which were well done. Besides the character itself, there are many other things occuring in the background, which includes preparing food to feed the character and throwing sticks or balls to play games with the pet character.

• Despite the fact that the little alien pet character was evidently artificial, as it could feed on the food even if the user chooses not to give it directly to the pet but throws it on the ground instead, it seemed to have a mind of its own, which builds onto the level of subjective reality in the game.
• The character has many ways of communicating with the user. For example, feeding the character also causes the character to change its own colour and sometimes, new architectures will evolve in the environment, which is fascinating and evokes positive emotions in the user, and elevates a sense of immersion and presence.
• A good point is that the pet character is always close to the user and responds immediately to the user’s actions accordingly, which is a key and shining feature of the game as it has the potential to make users feel appreciated.
• This feature where the pet character changes its colour also allows the user to see traits of the character that are unique to it, and helps to build a sense of personal bond between the user and the character.
• Another element which I found was good, was that the character comes to the user, instead of the user having to walk over to the character, which can be an important feature in the game as it has the potential to be built on with some other traits of the character, to provide emotional support to the user and make the user feel much closer to the game character.

“Bogo” is an application that uses Oculus Quest Headset, along with Oculus Rift. It is developed by Oculus.

There were also some features which could be further improved to make the experience to the user much more pleasurable by further elevating the sense of presence of the user.

• I thought it could be improved by elevating the aspect of social communication. Since it is an application involving virtual pets, it could be more fun to include more opportunities for users to communicate with other humans in the VR game who could be taking their virtual pets on a camp or a walk in the environment.
• Alternatively, there could instead be a storyline on the virtual pet embarking on a quest to fulfil some missions, to help users immerse in the nature of the virtual pet in its natural environment and be able to see more facets of the traits of the virtual character, in order to build a much more immersive experience.