Microsoft Mesh – Metaverse as a social construct

The Metaverse is a social construct.
It only exists because we collectively deem it so.
We invent complex economic systems, written and spoken modes of communication, so that we can better cooperate as a species.
We construct laws, conform to social norms so that we require less overhead when collaborating with one another.
Simple acts like buying food from a restaurant requires a complex supply chain powered by collaboration that is only made possible by such systems.

At the heart of all such systems is connection, which this demo emphasise deeply on.

It paints a future where the Metaverse, expressed in various forms, lives closely in harmony with our work.
It focuses deeply on showing how it “gets out of the way” when it’s no longer required.
It shows how it can be additive, preferring mixed reality over completely immersive experiences frequently found in VR demos.

I really like how it makes use of a sentimental soundstage to further generate resonance.

While it helps to paint a harmonious future, it doesn’t actually educate the audience what Microsoft Mesh is.
I think that should be improved by adapting the scenes to directly show what role Mesh plays.

Jane’s Story: Living with a Mental Illness

For so long, the topic of mental health has been a taboo. When it comes to such a sensitive topic, many would rather tiptoe around or even disregard it altogether. Despite its prevalence, people with mental illnesses still face considerable stigma and discrimination1, forming a barrier for those who need to seek treatment. Efforts by mental health advocates and non-profit organizations have made headway in raising awareness of mental illness but combating the stigma remains no easy task. For the healthy, it can be difficult to empathize with the plight of those suffering from mental conditions. This lack of understanding could also bring about the wrong impression that psychiatric patients are violent and are nothing but trouble makers, a sentiment that is further perpetuated by the media.

Opening Screen of Jane’s Story

This is why I was especially intrigued when I came across Jane’s story during my internship at Hiverlab. Jane’s Story is a 360 degree VR video produced by Hiverlabs and TOUCH Community Services which puts you in the shoes of Jane, a student suffering from Depression. Users are put through the anxieties that are faced by Jane and is constantly bombarded by her doubts and insecurities which are conveyed through her thoughts in audio. This creates an intended effect that overwhelms the user as you drag your way through the story. The environment, the audio and the 360 degree viewing angle truly brings about an immersive experience that offers a glimpse of those plagued by mental illnesses.

Screencap of the In-Video Bully

While the video may not be able to fully capture the experience of those with a mental health condition, users would at least be able to understand and empathize what the former goes through. The video presents a great tool for the purposes of raising mental health awareness as it can potentially bridge the empathy gap for those watching it. In fact, social workers at TOUCH Community Services were able to utilize the video to conduct guided sessions with students.

A rather awkward model of the teacher

Despite its successes and utility, the experience in Jane’s Story could still be improved. Primarily, I feel as though the graphics could use more polish as certain models like the teacher as well the trees in the background are glaringly awkward. Obviously, these aren’t the main focus of the video but it has the potential to ruin the immersion for some.

Barring its minor hiccups, I do feel as though the project is a great entry for the utility of VR for portraying mental health conditions. Such efforts could go a long way in aiding the cause for removing the stigma faced by psychiatric patients. With news of more potential episodes, there is much to look forward to for Jane’s Story.

Link to video and how it was made here: https://www.xsens.com/cases/how-hiverlab-is-using-motion-capture-to-help-people-understand-anxiety?utm_campaign=Motion%20capture%20-%20character%20animation&utm_content=185793804&utm_medium=social&utm_source=linkedin&hss_channel=lcp-125725

  1. https://www.samhealth.org.sg/understanding-mental-health/what-is-mental-illness/
  2. https://www.imh.com.sg/wellness/page.aspx?id=2106

Underground – serious games

Underground was developed by Grendel Games for surgeons to practice their motor skills in a fun and challenging way.

In the game, the player controls a vehicle equipped with two large mechanical arms suited for underground mining operations, such as drilling into large rocks, picking up debris, moving around objects, and welding pieces of scrap metal together.

Underground is one of the many games that fall under the category of Serious Games, where entertainment is used to maintain players’ interest in the subject, especially since players’ interest tend to diminish when the subject is dry. Serious games transfer knowledge, teach skills, create awareness, change behaviour, and increase motivation of its players. Players are able to practice skills through simulations, or games with less realistic environments so long as the correct action is performed.

Previously, the surgeons at the University Medical Center Groningen (UMCG) made little use of the original simulator, despite having to practice an average of 200 hours a year to keep their motor skills up to standard. Practicing in a serious game provides the player with motivation to keep playing, through exciting characters, an imaginative game world, and competition with other players. Serious games make repetitive exercises a lot more enjoyable, making the player more intrinsically motivated to practice for a long time. Surgeons can practice using a laparoscopic controller that allows them to control the game using motions identical to those performed during laparoscopic surgery.

Underground teaches its players laparoscopic skill sets and knowledge in a more engaging manner. By combining game elements and learning strategies, players learn how to solve problems through challenges and rewards in the game. This makes it more appealing than traditional forms of learning and mediums knowledge is presented in.

Despite being a laparoscopic training tool, Underground has no official way of tracking the performance metrics related to laparoscopic surgery skills. It would have been better if players using the game for laparoscopic surgery practice could monitor their performance, and whether they have improved throughout their time practicing on the game.

Source credits:
http://undergroundthegame.com/press/
https://www.undergroundthegame.com/product/underground-controller/
https://www.nintendolife.com/news/2015/09/video_this_laparoscopic_wii_u_controller_is_teaching_medical_students_to_perform_keyhole_surgery

Mini Motorways – Small Game, Big Heart

Mini Motorways is a minimalistic puzzle strategy game by Dinosaur Polo club. You can find its website at https://dinopoloclub.com/games/mini-motorways/. In Mini Motorways, the player builds a transport network by connecting houses (the smaller buildings) to stores (the larger buildings) of the same colour using roads, roundabouts, tunnels, bridges, and motorways, which the player gains at the end of each in-game week. Stores generate pins, which cars must deliver from stores to houses. If a store is overloaded for too long, the player loses. As the game progresses, the map expands, and more houses and stores are spawned. This requires the player to build more complex road networks to sustain the stores’ needs, increasing the risk of traffic congestion.

In this post, I will be sharing my experience playing Mini Motorways, and analysing my experience with respect to the elemental tetrad conceptual framework (technology, mechanics, story & aesthetics), and five of Jesse Schell’s lenses from his book The Art of Game Design.

When the game first begins, I’m presented with two buildings with disconnected roads. Without providing any instructions, the task is intuitive: connect the two. The first thing I tried was to drag the house to the end of the road at the store, and it works! This is a great example of how Mini Motorways uses well-known affordances of real-life road networks and takes advantage of natural pattern recognition abilities to make its mechanics as intuitive as possible. This way of presenting mechanics (instead of a full-on tutorial) also makes the game slicker and cleaner, which supports its minimalistic aesthetic well.

As I continue playing the game, I find that I’m deriving enjoyment just by connecting buildings and developing my road network, as I feel like I’m creating something cool. Mini Motorway’s story was largely player-driven, or more accurately, player-imagined. I imagined that I was building an actual road network for a real city, and this gave the game life, while also making me emotionally invested in the game. The minimalistic aesthetic complements this well, as the game allows imagination to run wild by only providing plain coloured boxes. Of course, completing the elemental tetrad is the solid technological foundation of a grid-based system which was simple for players to understand and easier for developers to implement.

I found the intrinsic motivation of imagining a story to my road network more captivating than the extrinsic motivation of increasing my score, and this shows Mini Motorway’s success with respect to Lens 17 (Lens of the Toy). Even if Mini Motorways had no goal, being able to build my road network in a sandbox would be engaging and be a playable game. By being fun to play with, Mini Motorways managed to hold my attention and interest even after I’ve mastered its mechanics (after playing for weeks), and its goals are less challenging.

Although I felt that extrinsic motivation wasn’t too important, Mini Motorways doesn’t neglect Lens 55 (Lens of Visible Progress). Your score is based on the number of pins you have delivered from stores and is displayed prominently on the top right. Completing each in-game week also feels like an achievement, as I’m rewarded with additional road tiles and a choice of a helpful tool such as a bridge or motorway to develop my road network with.

While Mini Motorways managed to retain my interest over a few weeks with its intrinsic fun, at an individual game level, it does seem to have room for improvement with respect to the usual dramatic arc. When analysed with Lens 69 (Lens of the Interest Curve), Mini Motorways has gradually rising tension throughout a game as the difficulty slowly increases, but it lacks two vital elements: there’s no rest for the player (other than pausing the game), and there’s no grand finale – there isn’t a “final boss”. The player can usually tell when their game is about to end, and there’s usually not much they can do about it except perform some damage control to maximise their score and wait for the game to end. However, this shortcoming is what makes player planning important and rewarding and is a justifiable trade-off in my opinion.

Lying beneath the complex road networks that players build in Mini Motorways are surprisingly simple mechanics. Lens 48 (Lens of Simplicity/Complexity) is one where Mini Motorways shines. Each game element has very simple mechanics: stores produce pins, houses consume pins, bridges allow roads over water. But the emergent complexity from the decision-making on how to best use each element to create the most efficient road network is immense, and difficult to master. It’s impossible to tell if a road network I’ve created is the “best” road network, as the state of the network is in a constant state of flux as more stores and houses spawn. Many emergent strategies are formed, such as a colour-separation strategy I employed in the screenshot above. There are even somewhat cheat-y strategies such as placing roads in certain areas to prevent a bad store spawn!

Ultimately, Mini Motorways is a game about problem solving, the subject of Lens 8 (Lens of Problem Solving). Problem solving games tend to have an issue with freshness, as once the player knows how to solve a problem, they aren’t challenged anymore. Mini Motorways keeps things fresh by having random spawns (thus making every game different) and daily challenges that impose certain restrictions such as a limited number of motorways, so players are forced to adapt their strategies.

Overall, Mini Motorways is a gem of a game, and displays many excellent design choices for game designers to learn from. Do try it out!

An example of VR sports: The Climb

I have heard VR+Sports for a long time, but have never believed its practicability — we are living in the real world, why should we do sports in the virtual world? At the end of the day, doing sports virtually is not as convenient as in the real world.

I stuck to this idea stubbornly, until I got my shoulder injury. In a judo sparring, a non-standard Ippon-seoi-nage broke my left shoulder. “AC-joint separation, ” the doctor told me, “had better never do any competitive sports anymore. ” I stopped doing all kinds of sports. But to promote injury recovery and muscle soundstage, I needed to do lots of rehabilitative exercises. The so-called rehabilitative exercises were just some basic upper body movements, which were so boring that I hardly liked to do them.

Since I could not do sports outsides, it is at that time that I started playing VR games at home. Shortly, I found playing VR games can be an ideal substitute for rehabilitative exercises. Especially, The Climb fitted my situation perfectly. First, when playing The Climb, I needed to hold my hands overhead and do lots of overhead movements, which were especially good for my shoulder injury. Second, in The Climb, I did not need to do any quick movements, which prevents me from secondary injury. Third, I did not need to move my location, so I can play it in any place, including my small bedroom. Last but not least, the game scene was so awesome and realistic. I held myself by small pivots. Above me was the cliff. Behind me was the abyss. The wind in the canyon blew me. The eagle whistled while circling in the air. I am fully focused on my movement, looking for any protruding rock to grab, keeping my balance, and carefully moving to the goal, just like a real freestyle solo. Oh, God. What an immersive experience!

After three months of playing this game, I went to the hospital for a recheck. “What did you do?”, the doctor was shocked, “MRI result shows your left shoulder has fully recovered!” I told him about my experience in the virtual world. He said, “Amazing! VR should be a new kind of treatment. The Climb should be known by other patients of mine.”

Via this experience, I realize the practicability of VR+Sports. It is not a substitute but a complement for real-world sports. It gives a chance to experience sports for the people in rehabilitation and the disabled. It also gives a chance to experience extreme sports for normal people without risk.

Although many VR sports games have achieved good performance, there is still room to improve. Take The Climb as an example. After my recovery, I feel it is too easy to play it as an exercise. One simple but useful idea to improve it is, we can design a pair of trackers with removable counterweights. The more counterweights we add, the heavier the trackers are, and the harder the exercise is.

Tree VR: use VR to raise environmental awareness

In a previous project, our team became interested in increasing peoples’ awareness of environmental issues through games. Upon our research, we found that recently, immersive experience is widely used to raise the public’s awareness towards environmental issues. The Tree VR, introduced by New Reality Company and Rainforest Alliance, is a good example.

Tree is an award-winning VR experience where users take on the body and perspective of a rainforest tree and immerse themselves in the tragic fate of the tree. The entire game flow is only eight minutes. In the eight minutes, the player will grow from a seed to a majestic Kapok tree and witness a forest fire that destroys the forest. The Tree VR experience was open to visitors in Gardens by the Bay in 2019 with a ticket price of S$5.

Features:

The Tree VR provides a short but highly immersive experience. First of all, the graphics and art style used are very close to reality. The player can see details such as ants on the ground and wind blowing through the branches. In addition, what I liked a lot is the sound effect of the entire experience. There is no background music, and all the player can hear is the sound of the environment: the wind, the animals, the fire, etc. When people wave their controllers(branches), they can hear the slight noise made by the leaves.

Another interesting feature of Tree VR is that there are hardly any interactions throughout the entire game flow. In Tree VR, users will no longer see their hands. Instead, their hands are turned into the two branches of the tree. The player can only look around and wave their branches throughout the game. This design has both pros and cons.

Limiting users’ interactions actually makes the experience more immersive in this particular VR experience, because trees in real life are unable to interact with the environment on their own. When we see the forest fire as a tree, we cannot run away to seek help and have no choice but to wait for the fire to reach us. This is exactly what the game designers wanted to convey: “we must protect the forest for those who can’t speak for themselves such as the birds, animals, fish, and trees”.

However, I also agree that games with very few interactions are usually boring. Actually, I got bored for a while in Tree VR, as nothing is happening in my sight, and all I can do is wait for the next event to happen. If the designers decide to limit the interactions to make the experience more immersive, they should work harder not to make players bored.

Here I found a full playthrough video of the Tree VR, you may get a rough idea about what this VR experience is like after watching it 🙂

Minecraft VR

Minecraft has been one of the few games that remains popular even after its initial release and it is constantly being updated by the developers. It is one of the greatest gaming sandbox where players can build, craft, explore and play in their own world. Millions of players around the world would spend countless hours everyday getting lost in their own worlds. As it increases its reach and expands its support across various consoles, such as PC, Xbox and mobile, it is not surprising that Minecraft has tried to create support for VR headsets as well.

Why it is engaging:

Virtual reality is a refreshing change from the classic Minecraft gameplay on a screen. After spending so much effort into building and creating a world of your own in Minecraft, what would be better than to enjoy all of that work you spent in person? By using Minecraft VR, players are able to fully enjoy the world around them and see a whole new perspective of their Minecraft world.

Firstly, the sense of scale hits almost all players when they are new to Minecraft VR. As a two block tall character in the Minecraft world, standing among the craggy chasms, being able to look up at the looming mountains, seeing them extend into the sky far above you. One cannot deny that the experience is simply breathtaking. Due to its low-fidelity, block-based graphics, immersing into the Minecraft world gives a cartoon-like feel to players. The lack of fine details made the gameplay comfortable for players to enjoy their time in the virtual world.

Secondly, certain game actions such as swinging the arms to attack a mob, pulling the string of the bow to fire an arrow and hitting the block to mine it was very intuitive and realistic. This helps to make the game more immersive as the player has to mimic the actions in real life in order to complete a certain task in the game.

In addition, with the help of virtual reality, players are able to get up close with mobs such as dogs and cats to “pat” them. Encounters with hostile mobs in Minecraft becomes way scarier. Having to see a hostile mob approaching, attacking or even dropping from the top of the cave down to where the player is inflicts a greater fear in players as everything seems more real. This increases the challenge of the game where players get to fight all the scary mobs in first person, slashing away with their swords or blocking attacks using their shields.

Points of improvements:

Although being “inside” a Minecraft world is a novelty, it is a game where players spend a lot of time getting lost in their own virtual world with their arbitrary set of goals that they can chase for hours – building a city, exploring all the biomes, slaying the ender dragon. Despite efforts in making the game comfortable in VR such as the ability to toggle between theatre mode and immersive mode, Minecraft VR still faces the challenge of players experiencing vertigo after long hours of playing the game. Players can feel the strain in their eyes, the ache on their necks as well as the headset pressing into their face. Therefore, players would choose not to play Minecraft VR for for long hours unlike its other gaming platform counterparts.

Furthermore, as Minecraft have many controls for mining blocks, placing blocks, crouching, sprinting, jumping, eating as well as switching inventories, players who are new to VR may be overwhelmed by the shear amount of controls to familiarise. Chatting in multiplayer mode is difficult as well since the VR headsets do not provide keyboard inputs for text chats. Other modes of communication such as voice chats can help to alleviate this issue.

Nonetheless, I believe that VR has unlock a huge potential for the gaming industry to improve players’ gaming experience. Minecraft took the plunge to create their own VR experience for players and there will be many more gaming companies that will follow the trend to create VR options for their games in future.

Superhot VR: a reduced form of reality

Slowing down, and the world slows down with you. This is one of the most intriguing designs that I have seen in VR games, and you can experience it in Superhot VR.

Superhot VR is a first-person action game of solving puzzles via dodging bullets and killing enemies developed and published by Superhot Team. It invites the player to experience both motion and time at a more precise level than what people could do in real life. In the game, time only progress when the player moves. Just like the movie Matrix, you can catch each bullet path and take your time to decide how to keep your body intact while killing the opponents as the world freezes around you. That is nothing you can experience in real life.

Another reason why I am particularly fond about the game is its aesthetics. A lot of VR games strive to mimic the real-world environment for immersive gameplay. But Superhot is set in a rather surreal place. Everything in the environment is painted dull white from walls to tables. The enemies seem to be made of crystal red glass that shatter in crisp sounds when you break them. They are like statues devoid of human flesh, facial expression, and blood. I like this reductionist approach towards reconstructing our reality where things are no more than what they need to be. By removing the colour and texture of mundane items, it draws our attention to their form, shape, arrangement, and relationship as we watch the glassy figures jumps out of those constructs. There is definitely a psychological and artistic impact to this game.

Yet, the game feels real because of the flexibility given to players to interact with the tools they have. A gun can be used to shoot but also thrown at enemies when out of bullets. A knife can cut the bullets away in the air. Games often restricts the amount of freedom players have in order to prevent them from going off track, which could be intrusive to the immersion. Having less objects to play around but more freedom to decide what to do with them is a smart way to make the “reality” authentic and engaging.

There are also merits to the interaction design in the game. To proceed to the next section, the player is to hit, grab, or shoot some object with real actions as if one is still interacting with the virtual world. The instructional text is kept minimal and attached to the surrounding environment, making the game experience holistic and immersive. While the game requires a lot of body movement, no repositioning through the virtual environment is needed, avoiding motion sickness.

It is a pity that there is not much variation to the virtual environment since I really love the aesthetics and design techniques of the game and would love to see the game setting in dark or in the wild. In terms of interaction, one small issue is that although it is impossible for players to move around with their feet, the virtual space sometimes surrounds the player with edges that seems like one could fall off. I feel like a small indication of boundary could help the player to move more freely without having to worry about the edges.

Dance Central (VR 2019) : A Music Rhythm Experience

Dance Central VR is not just any game, it’s an experience – a lifestyle. You can now indulge in a virtual partying or clubbing environment at the convenience of being at home. Yet, it is also not just a place to gather and dance together, but to also interact with characters in the scene and feel safe to unleash your hidden dance talents. If you have played and enjoyed the past titles of non-VR Dance Central, this game takes your music immersion to the next level.

Seamless Flow from Previous Titles

Harmonix has released various versions on non-VR platforms prior to the 2019 VR version. They have made a noticeable effort to keep characters who were previously present in the older versions. For seasoned players of this series, it gives them a sense of instant connection with the game and it feels less foreign to be dancing with familiar appearances. For newer players, it gives them relevant insights on how other versions are like without having to play them. The game mechanics remain the same for the essential part – the dancing itself. Players still get to choose their favourite tracks and emulate dance moves shown on the screen. As an upgrade, what I really appreciate is the fact that the characters now are more diverse in culture, experiences and also proportions. This adaptation made it more relatable for users as it mirrors the real society’s current efforts to embrace differences in people across the globe

Usage of Characters

The characters are used as a way to provide dance tutorials and a guide for users to visually learn how to dance by copying what they see. I was impressed by the amount of thought placed into designing the characters, as the level of interaction with them is sufficiently deep to immerse you into a whole new social circle whenever you play. Each character has a story to tell, and they are highly relatable to real life scenarios (such as wealth and daddy issues). The mode of interaction requires you to be familiar with the environment and players can text them or even receive a voice message from them! What is really interesting is that despite having a fixed set of songs to dance with, the same song feels different when you dance with different people. In a sense, the game discreetly provides different game paths through the emotions and experience with the characters so that players will constantly find entertainment at their own will.

Ambience and Setting

Together with the presence of interactions within the game, the environment plays a very big significance in making the whole gameplay feel real. You will really feel as if you are partying you are the main character in the club. When you hit combos and the song gets more hyped up, the mood changes are further enhanced by animations and lighting around you and your dance partner. The loading of tracks are also seamless and does not take you out of the experience. Such subtle differences contribute to the emotions players may experience as they dance throughout the songs, thus intrinsically providing motivation to perform better.

Competition

Players can dance alone or with other players in battles. From my personal experiences as a dancer, dance battles in our society are seen more of an exchange, appreciating each other’s hard work and celebrating successes. In this case, the idea of incorporation battles as competition brings this concept over to the virtual world and makes the competitiveness more natural and welcoming. It feels more like a gathering – which links with the setting and story. Even when dancing alone, it brings positive emotions and players can treat it as a self competition, where they practice to be better versions of themselves. This approach provides entertainment and incentives without disturbing the core essence of what it is to enjoy dancing for yourself and with others.

Room for improvement

A possible downside is that despite the characters having strong interactions, the depth of story and plot is not consistent across characters. This means that players may tend to be closer to certain characters over time because they simply offer more. A viable option is to add more story or plot to the characters lacking in it. However, I feel that the connection you feel with someone may not necessarily be proportional to the amount of information you know about each other. Rather, perhaps a better way would be to improve on the frequency of initiative taken by the character(s) to contact the player during gameplay so that more meaningful impressions can be made.

Personal thoughts

Overall, this game gives players a highly stable and holistic VR rhythm experience. It even includes small details such as having a wardrobe of clothes to choose from and dress up before entering the club, having people welcome you as you navigate into the area and possessing a personal phone to play tracks or communicate with others. Fun fact : your interaction with one character affects how another character might perceive their interaction with you. Personally I would say there are no misses in this game, only hits (and combos).

AR application analysis – IKEA Place

How Ikea is using augmented reality - Digiday

What is it?

IKEA Place is an AR application developed by IKEA. It allows the user to browse and place furniture from IKEA into the environment, enabling users to design and preview how the furniture would look like in their house.

Why do I like it?

The app is relative easy to use, the gestures used to place, rotate, and enlarge/minimize the furniture are intuitive. There was a large pool of furniture to choose from and it was generally a fun experience.

Why is it engaging?

The app has a clean and minimalistic interface with an inconspicuous HUD which could be hidden. This heightened the immersive experience of the user by minimising distractions.

The furniture models are well made with great lighting. While it may feel somewhat out of place depending on the lighting conditions of the surroundings, the picturesque models made the experience pleasing to the eye.

What features are well done?

The AI seems to be able to recognize people and automatically crop the AR furniture to so they are never blocking people. This is useful when users want to see how their new set-up will look with people in the frame.

The app has a feature which allow the user to search for similar options in IKEA by taking a photo of a piece of existing furniture. I see this as a very helpful feature when the user wish to buy or replace furniture which fits the existing theme of the room.

What features can be improved and how?

The application is currently only available on iOS. It is possible that this is a deliberate design decision since iOS users are more well-verse with hand gestures compared to android users, since iOS has many more built-in functions that are accessed using hand gestures.

However, due to the minimalistic interface, the app can feel confusing to first time users or users who do not use the app often. There are no labels to what each button does or how to make adjustments to the furniture.

The app can be improved by adding an accessible tutorial button to show users a short demo of what they are able to do with the app.

Functionality wise, the AI is ineffective in recognizing existing furniture, often resulting in the AR models overlapping with actual furniture. Currently the app works best when used in an empty room.

There are 2 possible features that may be very useful for the user. One is for the user to select and remove existing furniture so that they can design on an empty canvas without having to clear the room physically. Another is for the user to save the designed layout. The app does not have a built-in save feature, so users need to painstakingly take screenshots of their design if they want to show them to others.