my final year project

Camera Based Interactive Wall Display With Hand Detection Using LED Lights

Abstract

There are many different ways to manipulate OS, we can use keyboard, mouse or touch screen. When doing presentation, it is inconvenient to control the OS and present at the same time. Our system, interactive wall, allows you to use hand to control OS on the projection screen but not touch screen. You can now experience a new way to control your cursor

Our system can be applied to the existing equipment, so we don’t need to purchase any expensive touch screen. The system requires one web camera to detect the hand and then you can start to experience our system. Also, three LED lights are required to put on the fingers to help the system to detect the location of the hand and the gesture performing.

The physical equipment is not restricted. Any size of projection screen can be set up for the system or you can use a wall or a table to project the screen. User can be more flexible to enjoy our interactive wall in any places.

Introduction Overview According to Wikipedia, interactive screen technology was invented in the second half of the 1960s. It emerged from academic and corporate research labs. After about 40 years’ development, interactive screens have become very common in daily life. For example, touch-screen applications, like the ticket selling system of the MTR, the portable video game NDS, etc., are very popular. These rely on heat and motion to interpret user input. The size of touch-screens in these examples is usually quite small and limited. Usually, the larger the size, the more expensive the product will be. Furthermore, some control methods are not very user-friendly. To solve this problem, many inventions have already used cameras to allow the same kind of user input as touch-screens do, but with a lower price and a larger screen. This market has good potential for further development. Therefore, the aim of this project is to design a web camera-based system to perform like an interactive screen but with a larger sized screen and using our own hands to control it. Objectives The goal of this project was to build an interactive wall display for presentation or classroom use. To achieve these using traditional methods, we would have had to spend around eighty thousand rupees to buy a touch screen or use other devices to control the system. However our target was to create a system that uses a projector instead of a monitor, so the computer simply projects a large image onto a white wall and uses a web camera device to recognize hand gestures as input and thus act like a touch-wall or touch-screen device. The system includes the functions of a mouse and executes commands by detecting and interpreting gestures of the hand. Since we use computer vision software to replace the video display, the keyboard and the mouse, the cost of the system hardware is reduced and it allows a larger size display that is also interactive. The only required hardware is a normal PC, an overhead projector, a web camera, a white wall and three LED lights.

Wiimote Finger Tracking -FAQ and Adv Discussion

A lot of very smart people have written in questions and suggestions and I think a lot of them are very clever and worth sharing... and I'd like to add some of my own expansions on what I showed in the video. Like they say, 2 brains are better than 1.... it turns out several thousand brains are even better. This post will be REGULARLY EDITTED to add new ideas as they come in. I know it breaks the RSS model, but I don't want to make a new post for every new idea. Sorry.
------------------------
LED arrays from security cameras - online stores sell IR LED arrays for night vision security cameras. These should work perfectly for the wiimote.

Gloves - gloves exactly like minority report are definitely do-able and probably a good idea for repeated use.

LEDs on fingers - instead of tape you can use LED lights and do this without the LED array. But, you effectively create IR pens similar to the WiimoteWhiteboard demo which means batteries on the glove too.

Pinching with thumb and forefinger - the wiimote does upto 4 points. So both left and right thumb and finger could be tagged with reflective tape or LEDs. The software needs to be smart about maintaining the correct association and recognizing pinches, but very do-able.

Tagging a wiimote and knunchuck - getting the spatial location of another wiimote and knunchuck using a wiimote and reflective tape is an idea VERY much worth exploring. Now you get bi-manual (two handed) positional input in addition to all the accelerometer data and buttons.

Tracking fingers on surfaces (instead of the air) - unless it's a rear projection surface, the tape will be facing the wrong way... though you might be able to put the tape on the back of the fingers and figure out some method of triggering input. That said, it'll probably work well on a rear projection surface. You may need to offset the view angle of the wiimote so the support glass/acrylic won't retroreflect the light back to the wiimote. Alternatively, use active LED points.

Tagging objects on tables and projection- combining this method of tracking with projection is VERY powerful. This is essentially a field of research called Augmented Reality and will likely the be subject of a future video.