What is it?

The MediaScape XR experience offers users a way to visit a virtual museum and interact with 3D models of cultural artifacts. The users also get to interact with each other and enjoy the experience together. The demo was created by the partnership between Centrum Wiskunde & Informatica (CWI) and the Netherlands Institute for Sound & Vision.

What features make it engaging?

Highly realistic representation of yourself

The users will be captured in 3D and a photorealistic hologram of themselves will be projected into the virtual world [3]. This allows the users to see themselves in the museum, as well as other users nearby. The representation opens up ways to make the experience realistic as the visitors see how the artifacts can interact with their presence and also what others are doing.

Multi-user setup to allow social interactions

The highlight of the experience is the opportunity to learn more from the 3D recreation of a cultural artifact: a costume once worn by Jereney Kaagman, a member of the Dutch rock band Earth and Fire, for the performance of their song “Weekend” in the 1979 television program [4]. The user gets to observe the costume from different perspectives, and even relive the experience of wearing the costume and performing on stage with their friends. This is amazing because it goes beyond just visual appreciation of the cultural heritage. The users get to experience the historical setup and have their rendition/creative output in the same scene.

What features can be improved and how?

Virtualizing museums is one of the most natural applications of XR. We can take on museum tours without leaving our homes and the experience is often claimed to be more immersive. While I think that MediaScape XR has done a good job of showcasing what’s possible in the future, there is still a lot of room for improvement.

Some potential improvements that I can think of are:

• Transcending the physical limitations for the museum tour: while I know that XR applications tend to want to reproduce how people visit museums, I wonder if they can do so solving problems that can’t be eliminated in the real world. For example, as a rather lazy person, I would love to be able to navigate the museum without having to walk around. Seated, if possible. Other potential improvements include having the ability to interact with the exhibits with touch. The current MediaScape XR setup requires a significant room space and it is not possible to have the experience at home. Support for more than two users is also difficult due to the equipment needed. I would hope that in the future, the XR experience can truly be a way to explore without ever leaving one’s home.

• Ways to tune the crowd/environment at museums: just as how MediaScape XR refers to their application as social XR, museum tours can be a very social activity. This means it would be nice to see what others are looking at, have guided tours by museum staff, and exchange comments with other visitors.

• More immersive experience: museums as a way to store artifacts is a somewhat old concept that originated from an era without photos or films. I think that XR applications can do a better job if they don’t just think from the standpoint of how to make artifacts more alive, but also how to make the experience immersive in time and space. For example, allowing visitors to hold conversations with historical figures.

I look forward to further developments of museum tours by MediaScape XR and hope that such tours will become available for museums in Singapore.

References

[1] “DIS wins best demo award at ACM Multimedia 2022,” CWI, [Online]. Available: https://www.cwi.nl/en/news/dis-wins-best-demo-award-at-acm-multimedia-2022/

[2] “Earth & Fire – Weekend (Video),” YouTube, [Online]. Available: https://www.youtube.com/watch?v=4iK5e76auJA

[3] “Sneak peek into future of cultural heritage,” CWI and Netherlands Institute for Sound and Vision, [Online]. Available: https://www.cwi.nl/en/news/cwi-and-the-netherlands-institute-for-sound-vision-gave-a-sneek-peek-into-the-future-of-cultural-heritage/

[4] “With MediaScape XR, Cultural Heritage Gets The Virtual Reality Treatment,” Jingculturecrypto, [Online]. Available: https://jingculturecrypto.com/cwi-netherlands-sound-and-vision-mediascape-xr/