About the APP

SkyView is a lightweight stargazing app available on iOS and Android devices. The augmented reality technology allows users explore different constellations, planets, star clusters, stars and other celestial bodies in the night sky through their smartphone’s camera. Simply pointing the smartphone to the sky, then SkyView will identify the stars and other objects in the view.

Why do I like it & Why it is engaging?

Have you ever imagined a date with your boyfriend or girlfriend, you guys take a walk along the beach, and then she or he points to the sky and tells you what constellation that is? Romantic, isn’t it?

I never imagined that until I met a boy who loves boys, we walked on the west coast park and identified all the constellations we could see. Still romantic. Hence, I find I can have such romantic experience without a boyfriend or girlfriend. I can do this by myself, with SkyView.

I downloaded the App months ago, and I found it is easy to set up and use. It brings a fascinating view of night sky to users with the stylish UI and the well designed graphics. The main function and interaction of the application are quite simple, so that many parents also use this as the stargazing education for their kids.

What features are well done & What can be improved?

The main function of the app, the AR stargazing camera, has simply interactions. A short description of the star or other objects will be shown on the side of the screen when you click on the object. This helps users connect the star on the screen and the star they see in the sky quickly.

Users are also allowed to increase and decrease the magnitude visibility of planets and stars, this make it easier to see the stars and the brightness of them for users who are in a region of high light pollution.

However, the night mode is not clear as the usual mode. The filter is not very user friendly and covers some views in the camera. The same planet and star size settings lead to the less brightness of stars. The color of the filter can be changed to green or other light colors instead of red. The size setting of objects could be larger so that it will be clearer in the dark.

Overall, SkyView is a fascinating and romantic AR stargazing application which is easy to start and use.

There are so many amazing places that one would like to visit during one’s lifetime. With a bit of luck and resources, one may be able to travel to a few of them. However, to travel to all of them would be impossible… or would it?

With Google Earth VR this could be possible.

The industry of Virtual Reality (VR) is growing at a fast pace. Its many different applications are not only recreational but may also be educational and be more than just a distraction. The Google Earth VR application allows the user to strap on a VR headset and start exploring all the wonders of the world from the comfort of one’s room.

Why they are engaging?

During the rough times of COVID-19, the possibilities of travelling have been severely limited. With the use of VR technology, the world is opened up again without the risk of transmitting the virus. Google Earth VR uses the 3D model of Google Earth, which has mapped 98% of the World. However, it is not just Google Earth ported to VR, it is a completely new experience. All the interactions have been redesigned and the VR application allows flight as the mode of travel, both at different locations but also in-between them.

This allows a very engaging experience for the user. The application is fast-paced and allows for many different amazing sights in just a couple of minutes. But it is not only that the user gets to tick off locations from their bucket lists at a pace not possible without this technology. While travelling in VR, the user also gets to experience the location in a brand new way, from many different perspectives which would not be possible if physically present.

With the application, one can fly over any city and experience them in impressive VR, stand at the highest mountain peaks of the world and appreciate the view in all directions, walk along streets in remote destinations you never thought you would get to visit, and much more.

What features are well done?

The Google Earth VR application is impressive in many ways. The sheer scale of the project is unimaginable, as Google Earth covers almost all parts of the world on a real-life scale. The controls and the straightforwardness of the application allow the user to effortlessly explore the world in an immersive way as if one was there in person.

What features can be improved and how?

However, the features of Google Earth VR are not all perfect. Due to the scale of the project, Google has been required to do a lot of optimization for the experience to be enjoyable. The 3D models are procured from combining enormous amounts of 2D images and layering everything onto 3D space. As a consequence of this, the detailing is not very accurate and sometimes certain graphics appear as blocky smudges of colours.

Improving this while still maintaining the scale of the project is hard. For the user, going to ground level (which activates Google Street View’s POV) results in much more details than the approximations in the bird’s view. As for what Google could do, this seems to be a project that does not get that much care anymore. However, with the use of Artificial Intelligence, Google could aim to do some photogrammetry repairs of textures, for example by combining the detailed Street Views with the textures in the Google Earth view.

To summarize, Google Earth VR allows the user to travel to incredible places across the world. At its current state, it may not be detailed enough in order to replace real traveling. It does however offer an immersive sandbox experience with new sights to explore anytime.

SketchAR is a mobile app developed in 2017 which revolutionised the way we learn how to draw. It projects AR stencils on the user’s drawing paper, and all the user has to do is to follow the stencils and draw accordingly. I like this app because it is very beginner-friendly. Users with no background in drawing can pick up a pencil and paper and draw right away. It removes the technicality of drawing and allows beginners to quickly immerse themselves into drawing. While this app is targeted to novices, experienced artists can also use the app and try a new method of drawing with AR technology.

The main feature of this app are the drawing lessons, which are broken down into smaller steps – each consisting of an AR stencil. Upon completing the stencil for the current step, the user presses ‘Next’ and the stencil changes into the one in the next step. There are many lessons offered on the app, such as animals, food, vehicles, anime, and architecture, to suit the different interests of its users. Users are also given the option to disable the AR stencils and follow the steps from their phone instead. Here is a video review of the app by an artist:

However, a major drawback of using the app is that users are recommended to attach their phones to a tripod while drawing. Otherwise, it will be tiring for the user to hold up his phone with one hand and draw with the other. Perhaps the use of goggles could be used to solve this problem. The user will wear goggles to view the stencils and free both of his hands, which could help him to concentrate on his drawing better.

Furthermore, the AR stencils restricts the area on the paper which the user’s hand can occupy. If the user places too much of his hand on the paper, the AR stencils will become distorted. A possible solution is for the app to continue projecting parts of the stencil that is covered by the user’s hands. In that way, the user can continue to follow the stencil without having to worry about touching the paper.

Another concern expressed by artists with regards to using AR stencils to learn drawing is that it hinders one’s creativity. Budding artists may become used to strictly following instructions such that they do not engage their own imagination. They will also lack drawing fundamentals which is crucial in developing and refining their drawing skills.

However, for the average user who wishes to draw for their own entertainment, this app is great for helping them to dive right in. When users sign up for an account, they can choose categories which they are interested in, and their drawing lessons will be personalised according to their interests.

Besides the AR stencils, this app boasts other cool secondary features as well, such as sketching from stencils designed by other users, sketching photographs from the phone’s gallery, and even a colour-matching game that trains users to recognise colours of different shades.

All in all, I would say that SketchAR is a great app which allows anyone to start drawing almost immediately, without any prior experience in drawing. Although it does have some setbacks, I believe that they can be rectified quickly since the technology is already there (goggles etc). SketchAR utilises AR technology in a fun and innovative manner that can transform anyone into an artist!

One of my favourite VR applications is called “Bogo”, because it seems like a very active game where users can interact with a virtual pet, which is a very expressive character with many different emotions. It is very engaging because users are free to pet or touch the virtual character anytime, and the character will respond accordingly with clear expressions on its face.

There are features which were well done. Besides the character itself, there are many other things occuring in the background, which includes preparing food to feed the character and throwing sticks or balls to play games with the pet character.

• Despite the fact that the little alien pet character was evidently artificial, as it could feed on the food even if the user chooses not to give it directly to the pet but throws it on the ground instead, it seemed to have a mind of its own, which builds onto the level of subjective reality in the game.
• The character has many ways of communicating with the user. For example, feeding the character also causes the character to change its own colour and sometimes, new architectures will evolve in the environment, which is fascinating and evokes positive emotions in the user, and elevates a sense of immersion and presence.
• A good point is that the pet character is always close to the user and responds immediately to the user’s actions accordingly, which is a key and shining feature of the game as it has the potential to make users feel appreciated.
• This feature where the pet character changes its colour also allows the user to see traits of the character that are unique to it, and helps to build a sense of personal bond between the user and the character.
• Another element which I found was good, was that the character comes to the user, instead of the user having to walk over to the character, which can be an important feature in the game as it has the potential to be built on with some other traits of the character, to provide emotional support to the user and make the user feel much closer to the game character.

“Bogo” is an application that uses Oculus Quest Headset, along with Oculus Rift. It is developed by Oculus.

There were also some features which could be further improved to make the experience to the user much more pleasurable by further elevating the sense of presence of the user.

• I thought it could be improved by elevating the aspect of social communication. Since it is an application involving virtual pets, it could be more fun to include more opportunities for users to communicate with other humans in the VR game who could be taking their virtual pets on a camp or a walk in the environment.
• Alternatively, there could instead be a storyline on the virtual pet embarking on a quest to fulfil some missions, to help users immerse in the nature of the virtual pet in its natural environment and be able to see more facets of the traits of the virtual character, in order to build a much more immersive experience.

The concept of Omnidirectional Treadmill Simulator (for home use) first come into my sight in 2013 CES, when Virtuix showcase their prototype which was originated from a Kickstarter campaign.

The Omni-directional treadmill replaces the traditional use of joystick and allows players to walk and run in 360 degrees inside videos games using body movement, providing a higher level of immersion while gaming.

The technology was immature when it was first released 9 years ago as the devices only allow simply walking and running. The technology has matured over the year, the latest product allows the user to perform more complex movements such as crouch, kneel and jump. The following video shows the latest version of the treadmill.

Other companies have followed the trend. The following video shows a similar product by another company that is currently available for purchase.

Gaming is often associated with unhealthy behaviour as gamers spend most of their time sitting down in front of the screen. I like how the device will change this idea to “Some Gaming device is healthy as the player will have to move their body while playing the game”. providing a platform for the gamer to play healthy while moving.

When playing a VR game, the players will often minimize their lower body movement. Otherwise, they might end up being one of the people in the video.

The treadmill improves the safety of the player by fixing the player in one place. It also reduces the space required for the VR setup, as a larger space is usually required to avoid collision with furniture while playing VR games. It also allows the player to make intensive movements such as running, which is usually impossible for a home VR setup due to limited space. The devices also improve the immersion of the VR experience as the player will not need to consistently worry about their location, avoiding the fate of ending up as one of the people in the video.

The Omni-direction treadmill is a heavy source of noise pollution. The noise produced by the device may be unbearable to the player family/housemate, especially when the player is making intensive movements such as running. The device may not be suitable for player who often plays the game at midnight and might disturb their family/housemate when using the devices. One of the possible improvements is to make use of noise damping material to reduce the sound produced when interacting with the devices. It is also possible to provide a soundproof dome (which the player can buy as an add-on) that cover the entire devices to minimize noise pollution when using the devices.

The notion of “walking” is replicated by using specialized shoe or shoe covers and a surface that reduces friction. It gives a feeling of walking on ice, which isn’t that realistic and immersive. The devices may be improved by using an actual Omnidirection treadmill.

The current technology is current immature (and bulky). I will be looking forward to a future(decades?) where a true Omni-direction treadmill is available to home consumers.

With the recent trend of virtual reality, many games have tried to create a VR version of their world, and Minecraft is one of the more popular game that has succeeded in converting its world into virtual reality. Minecraft is a sandbox video game where the environment and its inhabitants are made up of blocks.

Being a classic game many people have played before, introducing the VR version brings a new sense of experience in exploring the virtual world. The activities from the original version could be done inside the virtual world, with a more feeling of closeness. One can feels more close to the world as compared to seeing it through the screen as the player movement is followed closely which bring a higher sense of control inside the world. Mining and killing enemies is also more exciting as hand movement is required to execute the order which make it more interactive as compared to clicking a button. Hence the VR version of Minecraft helps to create a more interactive experience for the players to enjoy inside the world.

However, some people might not be familiar with the experience that comes along with VR. Some might feels that they are too close to the world and might feels excessive emotion such as being scared when a explosive creeper appear out of nowhere inside the virtual world as compared to just appearing on the screen. Some might get VR motion sickness as their movement inside the world might be too excessive like when falling of a cliff. Therefore, not everyone would enjoy Minecraft VR as different people have different expectation of the virtual world and some experience are hard to be compromised for everybody. Nevertheless, some of the experience could be toned down as not everyone is well versed with virtual reality and some of them might be newcomers to virtual reality.

In conclusion, I feels that it is good that many games such as Minecraft are trying out VR version of their worlds to bring out a new gaming experience. However, not all games can be converted into virtual reality and some games should remain as it is as the VR version might be a disappointing experience in the end. Overall, Minecraft itself is a great game and the VR version attract people that want to experience more of its world.

The caption above is a pretty succinct explanation of the core mechanics of the game. The gameplay is simple: time moves only when you move, use this ability to take down numerous bad guys with a variety of weapons lying all around the map. Originally it started as an entry from 2013 7 day FPS challenge, it further expanded to a full game with funding from Kickstarter. This is one of the most engaging FPS titles that I’ve played and it adds a new dimension to FPS genre in which the genre has gotten pretty stale over the years (my honest opinion, don’t stone me if you enjoy FPS games).

Despite dying repeatedly in this game, I could not help but to keep retrying over and over again. Because time only moves when you move, everything around you slows down, allowing you to evade bullets and take your time to aim with utmost precision. It feels incredibly badass to be able to dodge incoming bullets similar to the famous bullet dodging scene in the Matrix. It is thrilling to see bullets whizz past you knowing that if one of these hits you, it’s game over.

Being able to dodge bullets in slow motion is a powerful selling point, but it’s not the only element that keeps me engaged in the application. The reason why I keep going back to the game is due to the high level of freedom, creativity and immersion offered. In VR, you can individually control each hand. This allows for more immersive gameplay as you feel directly involved in the middle of the gunfight itself. Instead of playing through WASD + mouse on PC, your hands are frantically searching for anything useful to fight back.

To make things more interesting, each hand can carry a different weapon. Other than firing guns, you can actually throw your weapons to kill, hit enemies in close range or even block bullets like in a Japanese anime fight. This allows for many creative approaches to the game. You can dual wield pistols like Lara Croft, throw bottles like in a bar fight or shank opponents up-close. Unlike most VR games that has clunky controls and being largely restrictive, freedom in Superhot VR is largely bounded by your wits, luck and the amount of space in your room.

The first feature that immediately caught my eye is the minimalist, abstract art style. I really liked the simple color palette of the game: white means environmental backdrop, red means bad guys, navy blue means weapons, only 3 crucial colours reminiscent of the popular “3 Colour Rule” in fashion. It is stunning to see the bad guys shatter like glass when they get killed. Other than simply being a unique treat for the eyes, this minimalist graphics serves an important function: allowing the players to see important elements clearly. In a game where you have to dodge bullets to survive, you have to be able to see the bullet trails, where the weapons are, where the bad guys are located. With all these things that the player have to look out for, I can’t see this game having a realistic-looking graphics replicating the Matrix as it would be too visually noisy to be immersed in evading bullets. Overall, this abstract art style is an excellent choice, both aesthetically-stylish and utilitarian.

The next nice feature that I would like to highlight is the game balance & direction. In this game, ammunition is scare. You run dry after firing 2-5 rounds. Made worse by the fact that you can’t reload, forcing you to scavenge weapons from fallen enemies. The lack of a hit-point bar makes it very clear that once you’re hit, you are dead. The combination of the inability to reload, death on hit and the slow motion gameplay makes the direction of the game very clear: Be careful, do not waste bullets and kill with well-timed precision and finesse. Not only does this make the game engaging in terms of difficulty, it reinforces the image that you don’t need overwhelming firepower to win. All you need are just your wits and using your foes’ weapons against them. This makes Superhot VR stand in the interesting middle ground between being an action and a strategy game.

However despite the game having nailed well in numerous aspects, there are still some shortcomings to be addressed.

The first shortcoming is insufficient levels and the lack of difficulty chokepoints (aka mini-bosses). The campaign (story mode) eventually becomes boring, and I ended up only playing endless mode, where I try my best to survive for as long as possible. The lack of bottlenecks and levels might make some users lose interest in the game experience gradually. Taking this game to a multiplayer realm is extremely daunting. Hence, my suggestion to improve replay-ability and engagement would be to make the game more rogue-like. This heavily ramps up the game’s unpredictability, making the experience more interesting for veterans.

The next is the ‘lobby’ of the game which I think is not very intuitive and have perspective scaling issues. To start the game, you have to insert a floppy disk (which some players might not even know what a floppy disk is!) representing a game mode, and then putting on a VR headset from above (yes, putting on a VR headset in-game while wearing a VR headset). I have to stand up physically to reach for the top to try to put on the virtual VR headset and it is difficult to grab. Not to mention the floppy disk representing different game modes are not the most easy to understand. This act of inserting a disk and then putting a VR headset looks cool, but to a first-timer with no prior experience of the game, this can be a bummer as they probably do not know what to do to start the game.

The last shortcoming is the limited freedom of movement. While the VR arm movement are well-executed, I felt that locomotion (movement from one place to another) still leaves a lot to be desired. You can move in the map to a limited extent as long as you have sufficient space in your room, but gameplay is still largely static. The developer addressed this by having the option to teleport to another spot in the room with a grab of a pyramid after slaying all/enough enemies.

I understand that this implementation of locomotion is easy for the user, but I really wished I could have the option of moving in all 4 directions in the room like your typical FPS shooter. Decimating an enemy stronghold on the offence is much more fun than defending myself from enemies.

Overall, this is still a fun and interesting VR application that is easy to learn and fun to play. I had so much fun doing numerous goofy antics, like lying down on the floor killing minions or waving my hands to speed up the game. All the while, my friends having a great laugh watching me play. I highly recommend this to anyone who wants a VR application that is easy to pickup and play while still wanting to feel like a badass.

With the advancements in VR, the applications now include helping people cope with grief at the loss of a loved one.

In 2020, a 2-part Korean television documentary, called Meeting You (or I Met You depending on translation) was released, that told the story of using VR to reunite a mother with her deceased daughter. Jang Ji-sung, mother of four, lost her seven-year-old daughter Nayeon due to a blood disease. After 3 years, Ji-sung keeps pictures of Nayeon and all her favourite toys around the house, promising to remember her forever. But as time passed, she feared she would start to forget her daughter as her memories become more distant. Hence, she hoped to see her daughter for one last time. 

To fulfil her wish, the production team worked to produce this meeting between mother and daughter in the virtual world. They recreated Nayeon to be as accurate as possible, from the face down to the voice and motions. With a set of VR HMD and gloves, Ji-sung was able to meet Nayeon once again and had an emotional reunion. Nayeon’s father and siblings watched tearfully as her mother finally gotten the closure she was seeking from this final meeting.

To make this demo possible, an HTC Vive Pro was used, along with Noitom’s VR gloves that allowed Ji-sung to interact with her daughter. The virtual Nayeon was created by Korean startup Vive Studios over the course of 8 months using various methods. High resolution photos of Nayeon’s younger sister were taken in a process called photogrammetry to serve as the basis for the 3D model, with further refinement done using old photos to better resemble the 7-year-old Nayeon.  In addition, numerous child actors were brought in to recreate Nayeon’s voice according to old recordings, and motion capture was used to record an adult actor emulating the motions and facial experience based on old videos and photos.

The demo itself was run in Unreal Engine 4, with an NPC using the animations and model of the recreated Nayeon. The model included different levels of detail, allowing the pores on Nayeon’s face to be seen when Ji-sung moved nearer, along with interactable hair and clothes. The short demo also included a basic AI, allowing Ji-sung to experience events with her daughter such as her birthday and putting her down on her bed for a nap. While these events were ultimately scripted, it still elicited a very emotional response from Ji-sung.

Although I have had personal experience with many VR applications such as Superhot and Beat Saber, as well as having developed a VR demo in UE4, it was this recreation of a passed loved one that truly struck a chord with me. It demonstrated the ability of VR to create immersive experiences that resonate on a truly visceral and emotional level.

However, there is the elephant in the room with regards to the ethics of such an application. Many see this as impeding the natural healing process of grief. But such accusations were rebutted by Ji-sung, who maintained that it was a positive expirence. Others criticised the experience as ‘fake’. While it is true, given the virtual Nayeon is not a truly robust AI and all the encounters within the demo are scripted, the emotional response from Ji-sung cannot be denied. And neither can the efforts of the developers in creating a virtual Nayeon that is sufficiently realistic while still avoiding the uncanny valley. For performance, capturing the right motions for Nayeon to move like a convincing 7-year-old and the voice to sound like her are also props I would give to the developers.

As to the question of how such a use case can be improved, I would say that only time will tell. If AI progresses to the point that we can accurately recreate personalities of the deceased and generate interactions in real time, we could truly remake a late loved one. And the latest improvements in creating realistic faces can already be seen in the trailer for The Matrix Awakens made in Unreal Engine 5 which features a highly realistic Keanu Reeves.

The viral video of Ji-sung’s final meeting with Nayeon has touched many hearts in its time on Youtube.

This application was experienced using a Lenovo Explorer VR headset with its Windows Mixed reality controllers.

Microsoft Flight Simulator(MSFS) 2020 , as its name suggests, is the latest iteration of one of the most well known and longest running flight simulator series. In one of its recent updates, it introduced a VR mode, supporting VR headsets using OpenXR. 

What I love about the new VR mode is the immersion it provides, where there is an increased sense of presence in the cockpit. Instead of having to use a controller’s joystick or a mouse to look around, there is head tracking available so that you can easily have a 360 degree view from the cockpit.

One of the biggest appeals of MSFS2020 is the ability to visit anywhere on the planet, enjoying views from high up in the air. A binocular display allows for depth and 3D, which enhances this aspect of the game by giving a greater sense of scale and speed as compared to its traditional flat screened counterpart.  Certain cities, such as New York City and Paris are captured through photogrammetry, allowing for incredibly realistic representation of the city in game.

It is also quite satisfying to relearn how to operate aircraft using the actual controls in the cockpit, and to sit in a pilot’s seat where the actual controls of the aircraft are faithfully recreated in the game. For example, to increase thrust of the aircraft, you would have to push the throttle physically up. It is incredibly satisfying to sit in a large aircraft such as the A320 with many different knobs and buttons and be able to interact with them to figure out what they do. After spending time exploring and learning, being able to set up an entire flight from take off, the different ways of controlling the aircraft in the air, whether it be through the autopilot systems or fly by wire, and landing the plane, provides an immense sense of accomplishment.

However, the interactions and controls using the VR controllers do leave some things to be desired. The VR controller acts like a glorified mouse pointer, which can be quite unintuitive. For example, when throttling up a twin engine plane, one would usually want to push down on both throttles, but with the VR controls, you have to look at the throttle, press down on the trigger to select it, and awkwardly attempt to slide it upward. Therefore, you have to take your vision away from the view in front of you to focus on the interior of the cockpit.

It would be perhaps more natural for the controllers to act more similarly to hands, where you would just place it over the throttle instead of aiming at it, and when holding down the trigger or side grip button, it would be considered as having selected the throttle, and pushing your controller forward and backward would control the thrust.

Another issue arises from the fact that I had a first gen windows mixed reality headset, which uses an inside out camera tracking system for controllers, which were not precise enough for small knobs requiring small motions to manipulate. Perhaps it could be designed such that we could select a control using the trigger button and after that, instead of using the motion controls, the joystick could be used, where the rotation of the joystick would be mapped to the rotation of the knob.

There is also the option of using a hands on throttle and stick (HOTAS) controller setup as well, which is a joystick with a throttle, which works quite well for VR. There are situations where certain controls are mapped to buttons which are hard to find with a headset, since you cannot see where the button is physically on the controller. Perhaps an ideal design could be a combination of having hands tracking such as in the oculus quest 2 for interactions with the different knobs and buttons in the cockpit, and the HOTAS being used for the main flight controls.

Overall, while Microsoft Flight Simulator provides an unique experience by putting you in a highly accurate recreation of aircraft, allowing you to fly anywhere in the world, due to the fact that it is designed as a flat screen experience first and then being ported over to a VR experience, its interactions and interface can be further enhanced to increase immersion as well as ease of use and convenience.