More fun with the marvelous OpenCV library. While red is still a convenient color to track, I’m no longer using real vegetation. Instead, I found an old pair of gloves that do the job quite well.
(Edit: rather embarrassingly, I seem to have accidentally deleted the source code for this project. However, you should be able to figure out everything you need from camshiftdemo.cpp in the samples/ folder. I apologize for the inconvenience.)
A brief explanation: I’m no longer using the library’s object-tracking algorithms. Instead, the mouse is just tied to the position of the “average” red pixel, with some trivial logic for throwing away outliers. Once the bounding box of the glove drops below a certain width, a click event is triggered.
(Edit: I simulate clicks and mouse motion with the XTest library.)
(The interaction is not normally this laggy, but istanbul clobbers performance when compositing is enabled. Disable compositing, you say? I need my exposé: the window buttons on a taskbar would be too small to use reliably. The system’s not perfect. Yet.)
Some thoughts:
The click interaction really needs to be improved. There’s just enough “wobble” between the camera and an unsteady hand to make it very difficult to click without dragging, and perspective makes the rotate gesture almost impossible to execute when not dead center.
In the same vein, it would be good if I could get the program to even roughly judge the distance of the glove to set the threshold for click events dynamically.
Once I get the kinks ironed out of the current setup I’m going to try moving up to two gloves, but I think I have a way to go first. Glove-in-air interaction will never be a good model for desktop use, but I could see this being useful in applications (games in particular) that require only gross input.
TAylor Nelson
on Tuesday 29 July, 2008 at 7:21:42 pm
Awesome now you just need to map the right mouse button to your right eye left mouse button to the left eye and mouse movement to your head overall haha :O
TAylor Nelson
on Tuesday 29 July, 2008 at 7:48:36 pm
Also gnome looks really nice, I will have to give it a try.
TAylor Nelson
on Tuesday 29 July, 2008 at 7:49:30 pm
Also gnome looks really nice, I will have to give it a try.
P.S.: Since comcast+firefox2=crash here’s an interesting website I found a little while ago.
tony5m17h.net
MM
on Sunday 30 November, 2008 at 7:16:41 am
What do you use to change the co-ordinates of the mouse cursor, any insite into this would be a great help!!
Jacob
on Sunday 30 November, 2008 at 8:53:24 am
@MM: mouse interaction is accomplished with the XTest library.