“Pool” is an interactive installation that investigates the cross-modal relationship between expectations and actual sensations.

The following scenario was conceptualized with the intention to manipulate the audiences’ expectations and then surpass them.

Visitors come into a dimly-lit room and see a water tank placed on a low table. They walk closer to the tank and look down into it. Suddenly, they notice mist rising from the surface of the water. They realize that the mist is glowing, illuminated by LEDs below the water’s surface. Then they notice that their portraits are being projected by the LEDs onto the mist. As they move around the front of the tank, the image matches their movement as it is captured in real time by a camera mounted on the edge of the tank. The visitors become curious and intrigued, and move even closer to the tank, perhaps even trying to touch the portrait or blowing the mist away. As they interact with the mist, they also interact with their own image.





I’ve been very interested in experimenting with bioplastics. On the other hand, I’m very curious about what else Kinect can do to detect human body. So I want to use bioplastics as a way to make a robot out of household ingredients, and use Kinect to vitalize it.


I started with corn starch, water, corn oil, glycerin, food colors, wax paper, and vinegar.


I tried different recipes based on the basic ingredients: 1 tsp corn starch + 4 tsp water + 1 tsp glycerin. The detailed recipe can be found in this slide. After trying all these recipes, I found that corn oil keeps it from drying when exposed to air and more important makes it more flexible, while glycerin makes it more gelatinous, and the proportion of corn starch and water decides the solidness of this material. This is the final look of all my experiments.


After knowing the recipe to the biobot, I started to make a fake body that Kinect can detect(don’t ask me why…). Following images are several tries of what bottom line Kinect is to detect human shape.




Now I need to explain why it is a human shape and why Kinect. Kinect is what I always want to use in projects, but what Kinect does is mainly detecting human body and body movements. So the first thought is to use bioplastic to make a human shape robot. But later I found even if I can detect this human shape with Kinect, there is nothing I can do with it. So I have to add Arduino to make it move. I originally want to make a biobot that can play computer games, particularly to wear this biobot a Kinect that can detect shape and color, which means being able to play the game that I made for the midterm… Now I realized the logic and expectations are too complex, so maybe just using bioplastic and Arduino is a good way to try. Also I want to make is a product that people with instructions can try themselves. So I think LittleBits can be a good replacement other than Arduino. Anyway, the experiment result is in the following video. The material turns out to be too thin and flexible to be put on servo.

[Additional Project]

I know the above experiment failed. So I narrowed it down to just use Kinect to make a pure Kinect game. I like matching games that let people complete tasks by matching either shapes, colors, or positions. So the result is a game that have people put their separated and messed up body parts back into the correct positions. In this way, people can both have fun by making various kinds of, especially weird/funny, body movements and at the same time experience a new way of matching game. The following image is a sketch of what this game looks like.



During this whole game, people would use their body movements to control the process of this game. As they go into further levels, there will be more body parts that they have to fix. The follow video shows how to play this game.

Final-Max, Hang


Weird motion fighter is a Kinect fight game. Players have to do weird motions to fight against each other during the game. The game is inspired from classic fight games like Street Fighters and King of Fighters. We found characters always do funny motions in fight games. So we decided to make fun of these games. The reason why we use Kinect is that we also found players always act like an idiot in front of Kincet. We are also trying to let players see themselves doing funny motions in in the game since they usually don’t notice that.



Code Repository:

Video Documentation:

Thought Bubbler Final Project – Julie Huynh

Final : From Dreams to Thought Bubbles – Julie Huynh


Sci-Fi Precedents:

Screen shot 2014-05-20 at 3.53.25 PM


Initial Project Statement:

Using a brainwave sensor as the next step in my immersive tools research to access the recesses of dreams because they can create the most enveloping environments in your mind tricking your body and senses to believe you are in another environment.



Science Fiction to Reality

Screen shot 2014-05-20 at 3.53.53 PM

Screen shot 2014-05-20 at 3.54.11 PM


Brainwave Sensor Options:

emotiv-headset imgres

Emotiv 14 Channel Sensor


OpenBCI 8 Channel Sensor


MUSE 4 Channel Sensor


Chosen Sensor:

Screen shot 2014-05-20 at 3.54.47 PM

NeuroSky Mindwave

1 Contact Affordable Sensor



Children and adults ages 13 and up because by approximately 13 years their Alpha and Theta waves are similar to adults, so at this age the code will work similarly for their brainwaves from 13 years and up.



Sensor Shortcomings

  • Abstract data hard to map
  • Some technology is inaccessible, for example Conor Russomanno and Lisa Park expressed that Emotiv’s Epoc technology is hack-blocked
  • Needs more scientific research on the complex roles of the waves detected
  • Brainwaves can be detected and data received, but large gap in the interpretation of data


First Digital Data to Physical Device Connection Issues:

Using openFrameworks to Arduino, initially I used firmata to make the connection, but finding glitches in my connection I switched to serial communication.

However I was unable to send strings of serial data because I did not use a PWM pin to receive analog data output, so I tried using an LED to  to photoresistor, but that issue was solved by using a PWM pin and getting serial communication to work sending from openFrameworks and receiving serial data in Arduino.



Revised Project Statement/Concept:

To create the best translation of digital brainwave data to physical play.

Originally hoping to find a more accurate way to interpret the data read from the thinkgear and translating it to aiming a water gun and shooting it, I ran into technical limitations that led me to use a bubble gun instead to better interpret the brainwave readings. By using the the data and mounting the physical translating onto the user’s head, it aids in creating the illusion that they are making actual thought bubbles out of their own head with the use of the sensor worn underneath to track their attention level.  It becomes a play on words to create actual bubbles with their mind allowing the user to have a more immersive experience.  The project concept allows the user to believe that their attention can be measured based on the amount of bubbles emitted from the thought bubbler headgear.  Moving the physical interface to become an attachment of the user’s head creates a physical extension of the user’s head, and allows them to believe that it is an extension of their mind with the aid of the thinkgear sensor.  The thought bubbler headgear becomes an embodied experienced through its physical interface and brainwave sensing technology.

Screen shot 2014-05-20 at 3.55.35 PM


Project Objectives:

  • Digital data to physical play
  • Create a digital sketch to control physical device
  • Prototypes 1-3: Parse out different waves, connect with CV object, use CV object as trigger, connect CV trigger to air pump
  • Prototype 4: Blow-up balloon with detection of attention level measured by Mindwave sensor
  • Prospective Prototype 5: Control water gun aim and water pressure with different wave channels (CANCELED)
  • Final Prototype 5: Bubble gun best translation of brainwave data within hardware limitations to create bubbles based on thought rate

Prototype 1: Parse out Channels

Screen shot 2014-04-07 at 7.32.31 AM

Prototype 2: Draw a CV object and move with Attention and Meditation levels

Screen shot 2014-04-07 at 7.32.43 AM

Video showing jumpiness of data received from Thinkgear

Prototype 3: Use Visualization to Trigger Air Pump

Serial Connection from openFrameworks to Arduino

Screen shot 2014-04-07 at 7.33.13 AM


Working Pump Triggered by Serial Communication from Thinkgear


Testing Pictures:





Playtech Testing:

Playtech Title: Don’t Think Too Hard

Kids will concentrate on openFrameworks sketch to start the signal connection, and try to reach an attention threshold to blow up the balloon connected to the air pump.

Screen shot 2014-05-20 at 3.58.14 PM


Thinkgear Experiments


New Physical Computing Challenges

Translating the new openFrameworks sketch and arduino sketch to the hacked motor of the bubblegun, and moving the h-bridge to a smaller board to mount onto headgear.

Screen shot 2014-05-20 at 3.55.39 PM


Thought Bubbler Headgear Diagram

Creating headgear to hold the thought bubbling device while also fitting the thinkgear sensor on the tester’s head.



Final Thought Bubbler Headgear:








Back H-Bridge Detail




Final Headgear Worn by Tester

Progression of the Thought Bubbler Video:



Thought Bubbler Code

Future Iterations:

  • Create 2 wireless Thought Bubblers and use 2 Mindwave sensors
  • Make wireless, using wifi or bluetooth to arduino
  • Connect with a game interface for users to compete with who can produce the most thought bubbles

Further Research:

  • Complete a digital to physical interaction to simulate an immersive environment
  • Explorations of other sensors that can simulate immersion
  • Other tools to access the change of embodiment created by dreams, possibly by sending in different brain wave patterns to users to recreate the visualizations stored in those patterns

Powerpoint Presentation:

PowerPoint Link


Final- visual

Using geometric shape to visualize weather data in an innovative way and have the shape changed dynamically over time

First of all, I want to explore more about api and openframesworks. Besides, I always see visual presentation as an important part in technology and wonder how this can be applied in different aspect such as data.

Is it data visualization?
This is my personal question. I am not sure whether I want people figure out the data information from the visualization or not. If not, do I only want people to enjoy the visual part? then how this can affect people or interest people?

For now, it is kind of failed. Neither the data part or the visual part can I achieve my goal or imagination.

openframeworks: yahoo weather api. of mesh
geometric shape: triangle
weather data to use: temperature, wind, humidity, time

Screen Shot 2014-05-08 at 上午3.13.04

use low opacity to create overlapping color
different location will have different geometric shape and color

Got lost try- temboo and processing

Screen Shot 2014-05-13 at 下午6.52.57

I have no idea of why I would try this maybe because ,for me, processing is an easier way to understand the logical and coding part…
In this iteration(?), the location indicate different country. Also, the shape and the color are controlled by temperature.
Problem: I can not grab other data I want such as wind speed and humidity.


yahoo weather api, of mesh, HSB color

Screen Shot 2014-05-13 at 下午7.02.33 Screen Shot 2014-05-13 at 下午7.02.54

the HSB color will change with time
Hue: hour
Saturation: minutes
brightness: seconds
the each vertex is controlled respectively by temperature, wind speed and humidity.

1. there will be a big jump of color change since the color is control by time, seconds and minutes are 60 a round.
2. the data are not refreshing over time, it only refresh once an hour, therefore, the shape would not keep changing


Seasons of Air

Seasons of Air is a data visualization project that draws the attention to the relationship between weather change and pollution. I am working with a data visualization developer Sam Thurman to push this project further.

How we are achieving this is by compare information from Weather underground and state air. SeasonsofAir5


We are pulling information from Weather Underground API for the weather information of Beijing, Shanghai and Guangzhou. This information includes temperature, wind speed, humidity and visibility. The most clear relationship is between temperature and air pollution, therefore we decided to focus on this compression.


Below is what the date visualization looks like. We are finishing some details and then pushing this web site live soon.

Screen Shot 2014-05-20 at 11.47.27 AM

VATS Final Project

Final Project -

For the final I cleaned up a lot of what I had in the midterm and got the color tracking to at least pick up. While this is a long way from being to the assisted targeting system I originally invisioned I believe i have made several leaps and bounds in the use of the software.

Mid-Term Project.

For my Mid-Term Project I worked on bringing a game element from Bethesda Soft Work’s Fallout Series to life. The V.A.T.s. or Vault-Tec Assisted Targeting System, is an in game targeting system that allows the player more exact targeting with a mid-level critical attack system.


To do this I choose to recreate the software and aiming interface using the Xbox Kinect and Open Frameworks, as opposed to recreating the actually device that allows this (the pip boy).

Eventually I had to restart this idea using Processing instead of Open Frameworks due to complications. Here are the source sites for the work that I did:

Color Tracking in Processing:

OpenNI for Processing


Meet The Kinect – E-Book on Kinect Programing in OpenFrameworks

Attached is my Presentation on the Project. And videos

VATs Project

Skeleton Capture Video


Final Project – Julie JuHyun Lee


iPhone App that teaches children about abduction. (coded/made with Xcode)

What is “Stranger”?

An interactive iPhone application that teaches children to be aware and be prepared for abduction.

The goal of this app is to let children interact with this application to learn about abduction and learn about how to prevent themselves from getting kidnapped.  The easiest but the most fun way to teach kids is having kids interact and play with animations and games to learn about the subject. Therefore, this will help them to be aware and to be prepared on what to do when they face bad situations, making this app ideal for them.



How did creating “Stranger” get started

My childhood/young adulthood was all about being with my family/my little cousins.  When my parents and my aunts were not around, I was the only one who could take care of my cousins who are much younger than me since I was the oldest among all my cousins.  I personally like being with kids anyways, but when I am with my cousins, I just feel happy and thrilled to be with them.  As my family was moving into the States from Korea, I’ve realized many different things, including the children laws.  For example, children under 13 can’t stay home alone in the United States, whereas children are left home by themselves in Korea, because there is no law for it.  Despite the fact that there are many laws here focusing on children’s safety, we often hear about abduction and kidnapping, which got me into study deeper into the statistics and facts about abduction.  I found many different articles and websites about facts and past events about different kinds of kidnapping and abduction for adults to read, but I could not find that kind of stuff for the kids; and because I figured that only informing the parents will not be enough to prevent abduction, I decided to make something for the children.  The researches have been shown that the AmberAlert has been successfully helping children to return home safely, it would have a great impact on people to have some sort of program in their mobile phones to carry around.  Making a mobile app for children wouldn’t be limited to edit or make changes unlike books or articles to constantly make it better whenever it needs changes.

An interactive piece that educates and helps people is something that I always wanted create.  After going through a lot of research and prototype tests from the first semester of Thesis, it has been clear to me that I wanted to build something fun but informative at the same time.

An iPhone app, “Stranger” is a game that explains what kidnapping/abduction is and helps kids to understand, get familiar with, be aware, and be prepared for the real-life situations of kidnapping.  Having interactions and watching video clips would help children learn about the matter faster and easier, and playing short game/quiz of what they just learned about watching the animation will be a fun way to be informed and prepared for the kidnapping.



Problem and the Audience

There are so many incidents occurring where kids and teenagers are getting abducted by strangers and/or by neighbors.  This is a worldwide problem that many people are going through, wether they personally are victims of it or not, and a lot of people go through pain and the suffering because of child abduction.  Many children get separated from their parents by force destroying the happiness of many families, and giving many different psychological disorders, including PTSD(Post-Traumatic Stress Disorder,) anxiety issues, hyperarousal, hyper vigilance, dissociation, and/or disorientation, to individuals which affects their social life for their lifetime.  Although many tried to come up with a way to resolve or reduce this problem through different methods, like having magazine articles, showing it on news, or developing movies about it, they have not been working out well.  The ones who really need to be informed are children themselves. However, reading an article or watching serious news are not really going to help kids to be informed enough about what abduction is, because those methods target adults, not kids.  When it comes to an actual event of abduction, children can’t really do anything(they would just freeze most of the times) simply because they are not aware of it.  Children are not informed enough about what abduction is, how it happens, and what they should do when it happens.  Although there are many movies and TV shows about abduction, they are very different from what actually happens and kids get wrong ideas about what really goes on because kids are more related to the action,  super heroes, and the fun of fiction movies.  Kids and teenagers are the ones who need to be informed and prepared. This is certainly a global issue that adults around the world need to solve.

Because research shows that kids learn better by playing games/having interactions rather than by memorizing lists or information, playing a game to learn how to prevent abduction would be very helpful for both kids and parents.  The app will teach kids about dangerous environments that are around them, targeting 6-8 year old kids who are starting elementary school and after school activities, in which they go out to hangout at playgrounds and friend’s houses.  This period is when kids are first exposed to the dangers of kidnapping and need to be taught about being careful of their surroundings and being on guard with strangers.  Although the app is targeted at 6-8 year old children, it will be suitable for younger/older kids to play as well because it will be fun and helpful for them to learn strategies to prevent getting kidnapped as well.  Once they get the basic information about what it means to be abducted, how it happens, and what kind of decisions they would have to make, they will be well informed about the subject and will be safer and less worried.

Background/Facts about abduction

The abduction rate has been increasing since 1990’s and this has been a big problem for society.  The reason why abduction and kidnapping is an important issue that needs to be resolved is that there have been so many incidents where kids are forcefully taken by strangers that wants to physically and sexually abuse the child, or wants vengeance or ransom.  Many incidents happen very near the child’s house or nearby his or her parents. The statistic says, “approximately 80 percent of child abductions by strangers occur within 1/4 of a mile of the child’s home. Roughly 75 percent of abduction murders occur within 3 hours after the child goes missing”(http://children-laws.laws.com/child-abduction/abduction-statistics) The fear and the worry felt by parents can only be reduced when they finally realize that their children are safe, and a way to do so is by educating and informing the children about abduction.  This way, children will be aware of the situations and they will be less likely to be exposed to be taken away from their family.

How does this app work?

The app itself has 3 different parts: informative description/statistics, watching animation to learn, and playing quiz game that leads into different endings depending on the player’s answers.  All three parts will have a little characters and illustrations describing each sub-part of the stories(School bus, Home alone, and Playground) for kids to have fun with the interactions in the app.

You can see the screen shot of the app here

You can watch the animations here


This project is still work in progress, and it will be keep updated and edited until the day when kids won’t need this kind of app to prevent abduction/kidnapping.


Final Project Proposal – Xinhe

Territory – A multiplayer mobile tapping speed game

How to play:

  1. Players go to a page
  2. Set up your color
  3. The host player tap “Play”
  4. Everyone tap to mark the land with their own color as quickly as possible (You can also tap on other people’s land)
  5. Once one player (out of N players) occupied more than 1/N*110% (e.g. 2players-55%; 4players-27.5%) this player wins.

Screen Shot 2014-05-06 at 4.08.16 PMScreen Shot 2014-05-06 at 4.08.30 PM Screen Shot 2014-05-06 at 4.08.44 PM Screen Shot 2014-05-06 at 4.14.06 PM

Will Use:

Html5 canvas


… I have no idea how to build it from scratch …