Multitouch Display - Part One
Multitouch video collection
Table of Contents
(last update November 13th 2007)
- Ok, I want to build one! What I need?
- Let's build it!
Have you seen the Jeff Han's video showing his incredible multitouch table? Me too, that's why I could not resist and I started to try building one of them!
Here on the left you can see a photo taken from the video of my first multitouch experience.
In this page you'll find some information about what is a multitouch display and how it can be built, how I built mine and, hopefully, the updates about the evolution of my new creature. This project is always work in progress.
What is a Multitouch Display?
Accordingly to Wikipedia: "Multi-touch is a human-computer interaction technique and the hardware devices that implement it. Multi-touch consists of a touch screen or touch tablet (touchpad) that recognizes multiple simultaneous touch points and software to interpret simultaneous touches". So a multitouch display is a display capable of multiple touch recogniction. Probably this definition is not giving you the right idea, so I suggest you to take a look at my multitouch video collection on YouTube, just clicking on the play button in the following video:
BlaXwan's multitouch videos
Just to point out one of the main big differences with other fancy devices that came out recently from Apple, multitouch means more than one touch, dozens, hundreds of touches. The Apple devices are actually capable of sensing two touches maximum.
The other big difference is that the kind of multitouch displays that I'm talking about in this page are based on a so simple principle that let them be really cheap and easy to build up. No hard industrial processes or sci-fi microelectronics, just a piece of plexiglas, a webcam and some LEDs, really!
How is this achieved? Well, basically this is done "looking" at the display. This technology is actually possible thanks to the high computational power of the modern PC. This power is used for image recogniction. The kind of multitouch display I'm dealing with in this page are based on computer vision so that a webcam is pointed to the display and a software running in the PC is able to recognize the points where the user is touching it.
Now, there are two main approaches when trying to grab the images from the display: the FTIR method and the DI method. In both cases infrared light (IR) is used to illuminate a sheet of plexiglass and the webcam used is modified to let it see just the infrared light and not the visible light. This is made to avoid that the visible light of the environment, and also emitted by the video projector, impacts the quality of the touch recogniction.
|Back to top|
DI stands for Diffused Illumination. In this kind of Multitouch setup approach, the panel of plexiglas is illuminated from the back, with an IR beamer.
The IR light from the beamer is diffused on the back of the plexiglas surface and this way the webcam is able to see what's on the plexiglas surface or very near to it. This method has some advantages and, unfortunately, some disadvantages. The good things are that it is relatively easy to setup and it does not require a compliant surface like the silicone rubber layer to work properly. It is also the method that may recognize the "fiducials". Fiducials are symbols printed on object so that the computer is able to recognize which symbol is placed where and how on the screen. This technique is used, for example, in the Reactable to use plastic cubes with fiducials painted on the back as a controls for the sound generation, the cubes may be used just like knobs. Microsoft Surface is actually based on this kind of approach.
As far as I know, the main problem with DI is that this kind of system are quite sensible to the ambient light. This is caused by the fact that they "sees" what's over the plexiglas surface and not just what hit it. When building DI systems at home, another problem that I've read about (I repeat: I didn't tried this multitouch approach) is about the correct positioning of the IR beamer in the multitouch box and about the material used to diffuse the IR light on the back of the plexiglas panel and to rear project the image onto.
Actually I'm considering to use this multitouch technique for my next multitouch display. The main reason will be for playing with the fiducials.
|Back to top|
FTIR (frustrated total internal reflection) is a very simple principle that basically says that if you light up a certain material with some light, the light will stay inside the material itself, boucing on it's external surfaces until the external surfaces is "frustrated", for example, with a finger touch. In this case the light is bounced back from the finger and exit the material. So, with FTIR multitouch display, this material is plexiglas and we use the usual IR-pass modified camera to track the finger's position. This principle works also with infrared light so that the tracking light will be not disturbed by visible light.
To resume: a plexiglas plate is acting as a multitouch surface. A bunch of infrared LEDs is placed on the plexiglas edge and light up it. When a finger touches the plexiglas the IR light is bounced back and get caught by a modified webcam: easy!!!
Some additional triks must be used: a standard webcam is not able to catch the IR because it comes with an IR-block filter, you should get rid of this filter then the webcam will be able to see IR. In addition, some layers of exposed photo films are used as IR-pass filter so the webcam will be no more sensible to the normal light.
The image is projected with a video projector on the plexiglass plate, this requires a projection surface. Someone is using tracing paper or mylar but the best effects in terms of image quality are given by plastic projection films.
The projection surface will give good quality image but the FTIR effect will be disturbed by this additional surface, requiring hard pressing on the display to activate the effect and let the webcam see something. To solve this issue a compliant surface is used as layer to be placed beetween the plexiglas and the projection screen. The compliant surface will help to better couple the two materials and ease the FTIR effect and the sensibility and responsiveness of the display. The silicone rubber seems to work really well as compliant surface.
|Back to top|