Jack on the Holodeck 3 (MEmories)

Introduction

"MEmories" is a revolutionary Virtual Reality (VR) experience that allows users to explore significant memories from the creator's life through interactive organ representations in a unique visualization of their bodies. It merges technical prowess in VR and design with deeply personal storytelling, aiming to evoke strong emotions and foster a profound connection between the creator and users.

Description

"MEmories" is a novel Virtual Reality (VR) project blending personal experiences with innovative technology, aimed at exploring the nuanced intersection of self and memory. The name cleverly combines "ME" and "memories," highlighting its focus on individual yet universal experiences of reminiscence and identity. Users are immersed in a virtual space where their avatars are rendered through particle systems and visual effects, revealing internal organs in a state of transparency. This not only showcases technical virtuosity in VR and design but also introduces an interactive element where each organ is tied to a vivid memory from my life. For example, a virtual interaction with the heart plunges the user into a scene of profound personal heartbreak.

The essence of "MEmories" is to forge a deep, meaningful bond between myself, the creator, and the participants, transforming personal stories into a collective narrative experience. It's designed to evoke strong emotions and offer an introspective journey through significant life moments that have shaped my identity. This project transcends typical VR applications, aiming to create a unique, emotionally charged experience that deepens the connection between the creator's stories and the user's perception, challenging the conventional boundaries of virtual reality storytelling.

Key Moments

Full Body VR

I used Animation Rigging and Inverse Kinematics to add physical body to the player. Players will be able to see their bodies in virtual spaces moving around with their real-world movements.

Body and Reflection

In each scene, players will see a hologram/reflection of their virtual bodies. I wish to, via this, explore how seeing one's body from different perspectives at the same time can create any unique feelings/vibes.

Interaction with Organs

Each visible organ within the user's virtual body is correlated to a significant memory from my own life.  Users have to interact with the organs to get to other scenes that are created based on the experience that left an indelible mark on my existence.

Technical Showcase

VFX for Body

The problem was that since I used animation rigging and skin mesh rendering to make the body meshed move with the player, SDF couldn't move with the animation in real time. The solution I found was to use a skin mesh render to set the values of VFX. As you can see, I created a subgraph to simplify the usage of SMR for my VFX.

Dissolve Shader and Smoke VFX

The main interaction that users can do with the organs is that users can pull their organs out of their bodies and through them into the air. Then the organs will dissolve and the scene will be changed to the one that is related to the thrown organ. The picture shows the dissolve shader I made with a shader graph.

Coding and Environment Building

I coded all the scripts that control and support the smooth running of the experience and used a combination of skybox, post-processing, lighting, and display of objects to create the feelings and emotions of each scene.

User Testing and Reflections

The installation was placed in an open exhibition for people to play with.

Testers were all excited about the experience and interactions. The biggest issue was that it was hard for users to find out the way to interact with the organs without the introduction and guidance from me.

Also, since this project is quite personal, how to make it more balanced and engaging cound be a good direction to go. My intention was to build a connection between me, the creator, and the audience, but right now the connection seems to be a one-way info delivery. Further development can focus more on how to integrate the audience's memories into the project.