d'strict uxlab (en)

Few months ago, we've developed an interactive screen that recognizes multiple touches (MTS)
by using a methodology called Fourier Transform Infrared Spectroscopy (FTIR).



 Multi Touch Screen with Mobile and External Display

We are working on improving sensitivity of sensors and coupling it with other external devices.
As one can see from the movie above, we've coupled multi touch screen with a mobile phone
via Bluetooth and with an external display.

We keep working on small targets such as 'simply transfering photos by tossing them to a mobile
phone on the screen,' 'choosing a movie and wathing it on a big screen installed at a difference
location, and 'connecting MTS with a printer,' etc.

We will continue to make it better by upgrading software and hardware.

MTS Case


FTIR (Frustrated Total Internal Reflection)

Total Internal Reflection refers to an optical phenomenon that occurs when a ray of light hits from high refractive index material to low refractive index material at a certain angle, no light can pass through the boundary. When different refractive rate of air and acryl are utilized in a certain manner, the light ray entered into the acryl does not escape the acryl. When you touch the acryl, at this point of time, light ray around finger(s) scatter which then captured by IR camera inside of the system.



Captureing touches by using IR camera and LEDs

                 IR Camera Image                                                 Reprocessed in Flash

The whole deal of this project was to connect MS Surface and various wireless devices.

We planed to create an environment where you take a picture, it is wirelessly transferred to Surface and seleted photos are pritined, also wirelessly.




Sending a photo from a wireless digital camera


Printing a photo through a Polaroid POGO

Another thing we did is to aligning numbers of photos in Surface by applying Natural UI concept. When you circle your hand or sweep to a certain direction on Surface, photos are aligned along with the shape you just made. Photos would go back to their original positions when you rapidly rapidy flick left and right.


Aligning photos with Natural UIs

And we also successfully transferred photos from MS Surface to interactive wall, which was developed by other team.

In this research, we wanted to create an environment where text messages send by multiple users using their own mobiles are displayed on the display of interactive wall.




A user sends a text to interactive wall

Another cool thing is that the system fetches weather information of the day from Google and displays the most appropriate interaction based on user's gesture and the weather. There are five weather type available:
Sunny, Rainy, Cloudy, Windy and Snowy.




Interaction based on weather information



Thanks to Surface's control panel, you can search message sent by users in Surface. One of the core goal of this efforts of coupling MS Surface and interactive wall was to get rid of all the buttons. So in this system, users would bring up a menu by giving gestures to Surface.

Instead of entering numbers with a keyboard, users would draw numbers on the screen with their fingers. Want to find message you sent? Just draw a rectangular on the screen, draw your mobile numbers (only the last four) in it and again draw check at the end of the box. No button at all!


Available gestures in Surface