Sense Me Move Me

Dream Simulation

A full room installation designed to displace user from reality and simulate the disconnectedness of a mixed-reality dream state.


Ever since entering my early twenties, I’ve had very vivid, often disturbing dreams. My subconscious frolics around in the night and goes places my conscious mind would never typically allow. Sometimes I wake up and am stuck somewhere between real life and my dream, being convinced that the imagery and world I just witnessed is still partially real. Sometimes, the dream will become so ridiculous or overwhelming that this person deep within pushes through and says “stop it, this is ridiculous, I’m waking up.” And then I wake up and I think, why didn’t I do that sooner?!

I’d like to recreate the feeling of a nonsensical dream, however, I don’t necessarily want to get into the horror of nightmares. Instead, I’d like people to almost forget about their body, and enter a head space where anything feels possible. I’d like the audience to feel as if they dove into the inner workings of the subconscious mind for a moment.

I want people to move with caution and curiosity. I think I will map their position in the room to either sound, volume, video projection or color projection. I’d like to work a lot with sound, and keep the visuals generally abstract.

I’m thinking to get a voice recording of one of their wildest dreams before they enter the room, and then somehow distort their voice or fragment the recording and have it come in and out overlayed with other sounds depending on where they move in the space.

Also thinking to just have one lit area of the room where their registered image will be projected (before they enter that space), it will just be a black projection. Maybe as they step to this region and notice the disconnectedness of their image, their dream overlay will play.

I was inspired by Mimi’s noise color mesh sketch today, and was thinking that would be an appropriate visual to project on the floor, while this person moves around. I would play around with it and see what I can do to make it fit the vibe. Not sure about floor projection though. But was still very inspired by that sketch.

The images below are documentation of the effect I was playing with last night. They are using the registered image of depth and rgb. The background is only drawn once, and the frame rate is brought down to 4fps.

What I really need to figure out is how to make the experience personalized. How can I make the user feel as if this installation is it is paying attention to the user? Maybe that means keeping everything abstract so their are no impositions on the workings of their subconscious mind. Also, how can I make it all an aesthetic and integrated representation of ethereality and and disconnectedness? Having a hard time finding the backbone here.

Also thinking about dividing the room into two spaces. One is a representation of a calm and centered mind, the other is one of being lost and disconnected. Argh. Having a hard time nailing this down.




Rethinking the Body

with Find Contours Sketch


General feedback from Tiri and Sebastion:

1. This sketch made them want to move slowly, but in a satisfying way

2. The sketch enabled a sense of time, do to the overlapping of frames that were only drawing their bodies’ outline each time

3. Three adjectives they used to describe this sketch were delicate, sensitive and self-aware

My Obersvations:

The most interesting observations I was able to make when watching users play-test, was the difference between testing the functionality of the code vs. creating a unique portrait.

When watching Tiri play test, she was interested in finding out how spatial parameters effected the sketch. Once she discovered the parameters, she began to play with that threshold, and discovered some lovely functionality that I hadn’t intended for when creating the sketch.

When watching Sebastion, he was interested in creating an image. He discovered a pattern that drew an image of him having wings, made out of the contours of his arms over time. This was also something I hadn’t intended for but was very pleasantly surprised to witness.

Interestingly, and perhaps because we both have a dance background, I identified with Mimi’s method of play testing most. It seemed that she was interested in using the body to create a unique collage of contours. The relationship between movement and drawing felt pretty even. In other words, the quality of the movement felt just as important, if not more, than the image itself. The image was almost a structured happenstance of the movement.


Study #2 Moving with Light

featuring dancers of The Academy of Dance Arts


Click here to watch on Vimeo

Study #1 Moving with Light

featuring dancers of The Academy of Dance Arts


Or click here to watch on Vimeo

Pathway Studies



  • What pathways did you see? Which ones did you predict and design for? Which were surprises?

When interacting with this visual sound installation, people were moving in large, expansive pathways, experimenting and trying to learn the rules of the sketch. Because the space was divided horizontally and the projector was the only source of light, users understood that their interaction should take place within the parameters of the light. This influenced random yet linear pathways across the horizontal space, as people ran from corner to corner. Interestingly, the users began to interact with each other, in attempts to find out if the sketch was sensitive to inter-user interaction. This created a really playful environment, that we had not anticipated.

  • What design choices did you make to influence the pathways people would take? What choices were not made? Left to chance?

We mapped the space so that it was divided on the x-axis into four quadrants. Each quadrant triggered a different sound, therefore we encouraged linear exploration through the space. Also, the changing lights that filled the space with a different color based on the location of the user encouraged a sense of freedom that we didn’t initially intend for.

  • What did people feel interacting with your piece? How big was the difference between what you intended and what actually happened?

People seemed to be moving very energetically and freely throughout the space, which was a pleasant surprise to us. Initially the sketch was tested without lights, only sound, and that caused our user testing to be more static, slow motions. It was a more careful exploration of the space. Once the lights were involved, this encouraged a sense of freedom.

Link to code