Playing games with no handheld controls: Just use your muscles!

By D. Yvette Wohn

When the Nintendo Wii came out, people were fascinated with how the game controller- the Wii-mote- could detect their motion. Although motion-detection devices had been available, the Wii made motion-detection control an accessible  mainstream phenomena.

But what if you didn’t need to hold something in your hand? What if you could just play games by moving your fingers? Scott Saponas, a PhD candidate in the Computer Science & Engineering department at the University of Washington, claims that games can be played without hand-held controllers. Instead, something as simple as an armband could detect minute movement in your fingers. After watching Saponas show a video at CHI of him playing “air Guitar Hero,” Play As Life caught up with the young computer scientist to talk about his research on physiological computing.

Q. Give us an elevator speech about what is it you study.

A. I’m interested in how we can continue to expand the use of computers beyond traditional situations. We can use laptops, desktops, and phones for work, but when the computer is not the primary task, traditional interfaces don’t work well. How can we use physiological mechanisms? How can we use body movement that is easy without touching buttons?

Q. Are you saying that we can use devices without touching things?

A. We can interact with devices without having  to push buttons by incorporating simple things like pinching fingers or squeezing your hand into a fist. Imagine going for a run and instead of pushing your iPod to change the song, you can do something simple like pinching two fingers together.

Q. How does that work?

A. The way we do it is sense muscles from the surface of skin through an armband. There is complex mechanism of tendons. They go from different fingers up your arm. There is electrical activity going on with motor neurons, so if you flex and hold a muscle, the muscle fibers will fire neurons at different rates. You can sense the voltage difference from the surface at your skin. If we place this band we’ve developed on your arm, we can train the system to recognize your gestures using signal processing.

Q. Do you play games? What games do you play?

A. I play Wii and Xbox360. I play most games, mostly actions and shooters.

A. How could we apply this technology to gaming?

B. From a gaming perspective, it provides opportunity to add new experience to games. Some things are good with a controller, which can reflect body movement. The opportunity for muscle computer interface is that something about your physiology state can be reflected in the game. For instance, in throwing a baseball, your grip could matter. We could use both body movement and muscle movement [for games].

A. You showed a demo about playing guitar hero without a guitar. How does that work?

B. The air guitar hero is similar to Natal, letting you use body movement as input to the game. It’s not gross body movement but fine body movement. It’s complementary to some of the stuff being done with Wii and Natal. I don’t think muscle computer interfaces would necessarily replace existing movement controls. If you’re playing Call of Duty, for example, you’re not trying to replace the controller. In games like that, the opportunity is more about having some cool interaction or bringing something like heartrate into the game. But in sports games, it could be additive or a replacement.

A. How feasible is this [arm band] technology for game implementation? Will we see a commercial version soon?

B. I can’t speculate about marketing plans [for the arm band], but it is not expensive to create.

*Download Saponas’ paper on muscle-computer interaction here.


One response to “Playing games with no handheld controls: Just use your muscles!

  1. Am working on, movements of the fingers and computer games.I don’t know,if you could be of help

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s