Hackszine.com: User Interface Archives

Archive: User Interface

December 23, 2008

Gravitron - ping pong ball LED toy

Marcus wrote in to tell us about his latest ping pong ball electronics project, the Gravitron. In half of a ping pong ball, he's crammed 12 LEDs, an accelerometer, and an ATmega168 for a brain. If you pick it up and tilt it, it will light the highest LED. It's sort of poetic.

If you're as impressed as I am with his efficiency of space, check out Marcus' other ping pong ball experiments on his blog.

Gravitron - Playing around with gravity

December 16, 2008

MusicBox: music collection visualization

For her Masters thesis at the MIT Media Lab, Anita Lillie created a music library visualization tool called MusicBox. The project will analyze the songs in a music library and group them based on a number of different dimensions - stuff like song length, tempo, ID3 genre information, and more advanced spectral properties of the audio.

This project is killer in so many ways, but my favorite feature is that you can use your mouse to draw a path through the song graph and and the tool will create a playlist that smoothly transitions between the different genres along that path. If this ever gets released, human DJs are going to have to step up their game.

MusicBox: Mapping and visualizing music collections

Previously:
Anita Lillie's Arduino sleep tracker project

December 2, 2008

Image processing in Mathematica

A clown fish, diced into 40 pixel squares.

Theodore Gray demonstrated some impressive image processing features of Mathematica 7 on the Wolfram blog. You can drag and drop an image right into a line of code and perform a number of image processing functions on it directly.

I'm not sure how many of you are Mathematica users—I haven't used it myself since college—but from a UI design perspective, it's very cool to see multimedia assets integrated so seamlessly into a programming language.

This function identifies patches that are similar in color, then connects them into a network. The parameter says how many neighbors to look at before building the network.

The same tools can be used to process through frames of video. In one example Theodore shows off about 15 lines of functional code that can separate flying ducks from a stream of video by examining the morphology of the image frame and identifying the unique objects. Pretty impressive stuff.

Mathematica Image Processing

November 15, 2008

Myvu Crystal as a wearable head mounted display

Ralf Ackermann sent us a tip on using the Myvu Crystal headset in conjunction with a wearable computer. The Myvu glasses were designed to block out the rest of the world for private iPod video watching, but its VGA resolution and device compatibility makes it pretty suitable for tearing apart.

The consumer myvu crystal HMD (sold as a nice though still somewhat "socially unacceptable" 2 eyepiece video output device for the ipod and other devices generating a PAL/NTSC signal can be modified into a much smaller 1 eyepiece version. This one works very well with a multitude of devices like a Parallax propeller, a Nokia N95 via TV out or a Archos PMA 430. It is thus well suited as the core of "another wearable computer".
For this purpose it might also be combined with the iphone / ipod touch
Xbee IO extension described earlier this week.

Ralf's project is still a work in progress, but it's a reminder that most of the hardware required for a wearable is now commonly available. Considering most of us already carry a sufficient computer (iPhone, N95, G1, etc.) around with us all the time anyway, it's only a matter of time before a HMD design is made cool enough to dodge the social stigma.

Myvu Crystal HMD Modification (Flickr Photo Set)

November 7, 2008

Hand gesture multitouch using only a webcam

Andy Wilson of MS Research—a name you may recognize from yesterday's $1 gesture recognition post—is responsible for a number of pretty unbelievable projects involving image processing and human computer interfaces. It's the sort of stuff that really blurs the boundaries between real and digital environments.

I was blown away by the video above, in which Andy demonstrates a multitouch-like hand gesture interface. Get this. It uses only a standard webcam.

The webcam is positioned to watch your keyboard and by simply making a pinching gesture with your thumb and index finger, you can grab and move objects on the screen, or rotate them by twisting your hand. Pinching with two hands, you can control two separate points on the screen, allowing you to easily perform more complex zoom and rotation actions by pulling your hands apart or moving them relative to each other.

I haven't seen source for this anywhere, but he does describe the technique, which is quite clever. By subtracting the background and examining the topology of the remaining image (just the solid background and your hands), you can easily determine how many shapes are made by the background.

With fingers unpinched, the background is a single shape, albeit with a hand shaped isthmus pushing into it. When you pinch and form a circle with your thumb and forefinger, things change. A little island is created in the middle of your fingers and the background becomes two distinct shapes. The position and rotation of the inner shape provides you enough information to control objects on the screen.

Hand Gesture Multitouch [via Procrastineering]
Andy Wilson

Previously:
Gesture recognition for Javascript and Flash

November 6, 2008

Gesture recognition for Javascript and Flash

The "$1 Recognizer" is a simple gesture recognition algorithm created by Andy Wilson from Microsoft Research and Jacob Wobbrock and Yang Li from the University of Washington.

By simple, I mean that it's under 100 lines of code that you can quickly add to your application to give it gesture recognition capabilities.

To enable novice programmers to incorporate gestures into their UI prototypes, we present a "$1 recognizer" that is easy, cheap, and usable almost anywhere in about 100 lines of code. In a study comparing our $1 recognizer, Dynamic Time Warping, and the Rubine classifier on user-supplied gestures, we found that $1 obtains over 97% accuracy with only 1 loaded template and 99% accuracy with 3+ loaded templates. These results were nearly identical to DTW and superior to Rubine.

It works by using a simple 4-step process, which basically amounts to:

Resampling the recorded path into a fixed number of points that are evenly spaced along the path
Rotating the path so that the first point is directly to the right of the path's center of mass
Scaling the path (non-uniformly) to a fixed height and width
For each reference path, calculating the average distance for the corresponding points in the input path. The path with the lowest average point distance is the match.

What's great is that the output of steps 1-3 is a reference path that can be added to the array of known gestures. This makes it extremely easy to give your application gesture support and create your own set of custom gestures, as you see fit.

Give the demo a try. I was pretty surprised at how accurate the results were, even with single-temple custom gestures that I quickly scribbled out.

$1 Gesture Recognizer - Examples and Source (Javascript, Actionscript, and C#)

Gestures without Libraries, Toolkits or Training:
A $1 Recognizer for User Interface Prototypes (PDF)

October 13, 2008

Microsoft Surface without a surface

Microsoft has released what's more or less a stripped-down version of their multitouch Microsoft Surface; in face, there's no surface included at all. Anyone with a webcam (and Windows) can try it out. From Geek.com:

Although many developers may be waiting for the release of the Surface SDK expected later this month, Microsoft have in fact released another touch-related SDK you can play with right now.

Called Touchless, this SDK allows you to experiment with multi-touch applications with little investment in hardware. In order to use Touchless all you need is a webcam as the video above demonstrates.

Microsoft release webcam multi-touch Touchless SDK