GSoC Accelerometer-based Gestures Update

[Paul-Valentin Borza]

Mickey requested a bi-weekly report from all students, so here's mine
on the Accelerometer-based Gestures.

Problems: I need a Neo FreeRunner to have access to the
accelerometer... Because I don't have a FreeRunner at hand and looks
like I won't have one any time soon, I've continued my work with the
Wii Remote. My previous library was implemented on Windows and had to
re-implement the communication over bluetooth in Linux; which by the
way has a much much better bluetooth stack. Windows sucks at bluetooth
communication. Anyway, I've done this and now I'm working with the Wii
in Ubuntu.

Now, on accelerometer-based gestures. I've bought two books from
Amazon: Speech language processing: A Guide to Theory, Algorithm, and
System Development written at Microsoft Research and another one:
Statistical Methods for Speech Recognition written at IBM Continuous
Speech Recognition Group. Although it mainly covers speech
recognition, it also covers hidden Markov models in depth and how
continuous recognition works.
My previous work with the Wii on Windows already did isolated (push-
and release-to-make gesture) gesture recognition. But now, I wanted
continuous gesture recognition for gestures like auto-pick and answer
(when you wouldn't want to push any buttons). I always wondered how
can the framework know when to stop processing and decide that that's
the winning gesture and auto-answer the phone. Well, the book cleared
this out. It's called an end-point detector (smile :)) and is mainly a
two-class classifier (one class for walking/noise and one class for
gesture/motion). The classifier is based on multivariate gaussian
mixtures. I've written the discriminant functions and just finished
writing the estimation/training functions (yes, this classifier also
needs to be trained). The classifier is great because it can
adapt/learn new values on the fly while in use. So, when this
classifier classifies enough frames as gestures/motion, it passes
these frames to the recognizer that has a unigram grammar of hidden
Markov models.

I will have to test and train the classifier in the next two days and
then I'll check in the code to SVN.

By the way, it would be great if you guys would have ideas for
gestures. Although creating a new gesture will be trivial, it would be
good for me to know how many I have to create. The demo app will
enable auto-answer when picked up from the table, auto-hang-up and
snooze the alarm when hit. Any more ideas?