XR For Retail: Perfect Corp AR Diagnostics & Try-ons
Introduction
Perfect Corp is a company that specializes in AR for diagnostics and product try-ons for various consumer products on hands, the face and on the head. They have AR demos for a lot of applications, such as makeup try-ons, accessory try-ons for watches, earrings, rings, eyewear and even dermatological skin issue diagnostics. Their demo store is available here (and this link will bring you to a specific demo on earrings try-ons but there’s a lot more in their demo store).
Why do I like them?
Having worked in the consumer goods industry for the past few years, I like Perfect Corp’s focus on trying to bring about a very accessible alternative to users who may not find it convenient to physically go down to stores to try on the products they are interested in. This is also an interesting solution to business owners finding it difficult to stretch their resources across many locations to serve their entire consumer base across multiple geographies. By tailoring these technologies to create AR experiences for multiple brands, Perfect Corp’s AR try-ons also provide great value to both businesses and their consumers in how the technologies will be used to create coherent and strong brand experiences. Perfect Corp’s AR try-ons and diagnostics serve a big need which conventional product pack shots and even newer 3D product previews are unable to provide.
Top features which make it engaging
Real-time realistic movements and interactions
In the earring demo, you can really see how the earrings swing around when you move your head and it helps to really improve the realness of the experience. These are experiences which you simply cannot recreate with images or even having only the 3D model of the product. They require certain physics models in how the product will interact with human motion and these all help to elevate the consumer experience in the comfort of the consumers’ own location of choice.
In-depth and well-represented information reflecting the real world inputs
In the skin analysis demo , you can easily find out about your technical skin age, and have a report on the skin issues you’re facing. The application pinpoints exactly the visual features which it’s using to generate the scores, and they present all these visual features in a way which users can easily digest. They also prove that the information they retrieve from the images are highly correlated with actual dermatological assessments (Cook et al., 2022) which further prove that AR can not only be used for generating pleasant consumer experiences in try-out but also very effective communication for actual medical information about ourselves.
Smooth, intuitive controls
Perfect Corp has done a lot of groundwork in creating a seamless switch between what is real and what is virtual. If you look into their shade finder AR application, they automatically find the correct shade of makeup based on your facial skin shade and apply it to the face. The controls to slide between reality and the virtual face with makeup on are very intuitive to users and allows them to really see and compare what they can expect before and after applying makeup. This is a very powerful use of AR for try-ons which cannot easily be matched in the real world without spending a lot of time applying and removing the makeup.
What features can be improved and how?
Lighting
I find the textures and lighting of the try-on products a bit off from what is considered as realistic to consumers. It looks similar but you can easily tell that the objects are fake, not because of the geometry or the physics but mainly because of the lighting on the materials. I think with better lighting mechanisms this product can really be elevated to a whole new level. Especially if you try the nail AR, you’ll find that the coloured mask over the nails hardly changes even when you change the ambient lighting and direction of the light source. If we can incorporate lighting effects properly, the try-ons will definitely become even more engaging and believable.
Perception of objects
Although many of the try-ons are quick accurate and responsive, there are still times whereby the detections will flicker and disappear, such as in their nail AR. By not only implementing instance segmentation on each frame, but by also adding on object tracking algorithms (Apicharttrisorn et al., 2019) across the past x number of frames, it could be that we could have a better hold on keeping stable AR experiences which will allow for more realistic interactions.
Conclusion
I believe that AR for retail is going to be a huge field for further exploration, and Perfect Corp’s implementation is a frontrunner in this space. With improvements in real-time perception and lighting, I believe AR for retail will become a huge force for businesses and consumers that enables impressive and immersive brand experiences, through meaningful product interactions and discovery all in the comfort of the consumers’ location of choice.
References
Wang, Y., Ko, E., & Wang, H. (2021). Augmented reality (AR) app use in the beauty product industry and consumer purchase intention. Asia Pacific Journal of Marketing and Logistics.
Cook, M. K., Kaszycki, M. A., Richardson, I., Taylor, S. L., & Feldman, S. R. (2022). Initial validation of a new device for facial skin analysis. Journal of Dermatological Treatment, 33(8), 3150-3153.
Apicharttrisorn, K., Ran, X., Chen, J., Krishnamurthy, S. V., & Roy-Chowdhury, A. K. (2019, November). Frugal following: Power thrifty object detection and tracking for mobile augmented reality. In Proceedings of the 17th conference on embedded networked sensor systems (pp. 96-109).
XR for Rehabilitation: Neuro Rehab VR
Introduction
Neuro Rehab VR is a company using tailored VR experiences to facilitate patient-specific rehabilitation exercises. They provide rehabilitation across multiple functional areas, such as over the upper body, lower body, cervical function, for pain management, and cognitive functions.
They have certification from the American Physical Therapy Association for using VR as a therapeutic intervention, and is also an FDA-registered therapy system covered by insurance in the US. Videos on the use of this system can be found at their official website.
Why do I like them?
I think Neuro Rehab VR presents a new paradigm to rehabilitation. It’s no longer about repetitive movements and performing various exercises to build up our bodily functions. Rehabilitation is a very important and tailored process in which we have experts guiding us on how we can attain or regain certain key functions with our bodies. In the past, I’ve gone for pain management sessions for the lower back and the therapist would often prescribe me with certain exercises for me to perform regularly. However, these sessions were quite dry and it didn’t feel like something I would enjoy. Eventually, I stopped going for them. By creating experiences which achieve rehabilitation through games and activities, I like how Neuro Rehab VR turns things around to allow people to focus on spending more time on having fun and enjoying new experiences instead of focusing on what we would normally consider as therapy time.
Top features which make it engaging
Reduced monotony with targeted experiences using various equipment and scenarios
I think it is great that Neuro Rehab VR is able to create new scenes and activities which are tailored to different rehabilitation aims, which can help to motivate and energize users into putting in more effort into the rehabilitation, as well as maintain interest in regularly working the particular bodily function they are training for. They often also use additional things on top of the headset in order to create the right environment for rehabilitation, such as upper body harnesses, or elastic bands held by the therapist. Since they can easily adjust the difficulty or change the settings of the game, they are also able to tweak the experience for different users to maximize the results for them. From the videos, we are also able to see that the users are a lot more motivated and enthusiastic about putting in a lot more effort into achieving the game objectives. Patient video testimonials (such as this one) also stressed on how they were able to get a good workout as compared to the past sessions which may have been more mundane.
Safe environment
In some of their VR settings, they help users with common tasks like getting items from grocery shelves and these are safe environments for the users to practise and regain confidence in daily activities. Instead of going into a real physical environment where there may be fall risks and confidence issues, the VR setup together with safety harnesses provides a new environment where users can refamiliarize themselves with everyday tasks.
What features can be improved and how?
Realism of objects
Many of the games showed in the demos were largely based on 3D models which were very cartoonish and polygons with the same color shading across the entire object. If we are able to render more realistic scenarios that may help to increase the engagement of the users even more.
More varied sceneries
One of the key benefits of being in a virtual world is that you can create beautiful sceneries which are out of this world. Some users undergoing rehabilitation may have physical constraints which prevent them from visiting places with beautiful sceneries or being tourists in foreign lands. If we can create sceneries which are beautiful to these users, it could increase the level of immersion and possibly enhance the effectiveness of the experience on the goals. They could benefit from having better rendering for natural foliage or for indoor materials in order to achieve this.
Conclusion
VR-based rehabilitation is a very good use of VR for motivating users to achieve functional and important objectives. By improving further on the graphics and toolkits for creating great environments, this should be a very ripe space for widespread adoption and providing a great experience to users requiring rehabilitation.
References
Cameirão, M. S., Oller, E. D., & Verschure, P. F. (2010). Neurorehabilitation using the virtual reality based Rehabilitation Gaming System: methodology, design, psychometrics, usability and validation. Journal of neuroengineering and rehabilitation, 7(1), 1-14.
Burdea, G. C. (2003). Virtual rehabilitation–benefits and challenges. Methods of information in medicine, 42(05), 519-523.
Schultheis, M. T., & Rizzo, A. A. (2001). The application of virtual reality technology in rehabilitation. Rehabilitation psychology, 46(3), 296.