[SoC Project Idea] Extending the OpenMoko framework to have support for multimedia

[Sayamindu Dasgupta]

Hi everyone,
I have been lurking on this list for almost a month and a half now,
and following the discussions with a lot of interest  Recently, while
going through http://wiki.openmoko.org/wiki/OpenMokoFramework, I
noticed that there's almost no support for multimedia stuff like MP3
playback, etc.
As far as I understand, libmokoui has some basic stuff which lets you
play alarm tones/ring tones, etc, and since there is alsa-lib towards
the base of the framework, one can theoritically use that to write
one's audio player, and the same can be said about a video player,
using xlibs, etc (though I read somewhere in the mailing list that
probably the current hardware is not powerful enough to support video)
In such a scenario, I guess the best way to go forward is to have
multimedia support is to have Gstreamer[1] as a component of the
framework. Openembedded already has gstreamer in their package
repositories (Am I using the right term here ?), and Gstreamer already
runs on ARM (Nokia 770/800), so probably adding gstreamer to the
framework would not be a major issue.
So, I was wondering, if this is a good idea for a SoC project. The
deliveriables for the project would be, in short, as I see at the
moment:

1. Porting and integration of the core Gstreamer lib to the OpenMoko framework
2. Identification of plugins to be included in the basic framework
(will need to take into account performance issues, as well as
legal/patent stuff)
3. Porting gst-python to OpenMoko (assuming that the Python framework
is more or less ready at that stage)
4. If the timeframe allows, building of a media player on top of the
framework, following the UI specs at
http://wiki.openmoko.org/wiki/Music_Player, to serve as a proof of
concept, as well as example code, which other implementations can
follow later on.

Addition of Gstreamer to the framework would mean that,

a) Writing applications like audio/video players, camera related stuff
(if and when the hardware includes a camera) becomes much more easier.
For example, the C code for a nicely implemented command line
Ogg/Vorbis player looks like
http://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/chapter-helloworld.html#section-helloworld
Code to get video from a webcam, adding a text overlay, encoding it to
Ogg/Theora and sending it to a IceCast server looks like
http://hippopotamus-streamer.googlecode.com/svn/trunk/src/hippo_streamer.c
I remember implementing a command line tool to do video capture from a
webcam, in under half an hour at Freedel.in 2006 (I was speaking on
Gstreamer at that conference, and needed some example code)
There are also tools like gst-launch to prototype your concepts
quickly (here's a command which would add vertigo effects to a test
video source and show it on screen:  gst-launch videotestsrc !
vertigotv ! ffmpegcolorspace ! xvimagesink )

b) Since there is already very good bindings for gstreamer in Python,
porting that to OpenMoko would be pretty easy once the rest of the
Python stuff is in place.

c) Most of the multimedia devs in the FOSS world (especially those
working with GTK+/Maemo/GNOME) are familiar with Gstreamer nowadays,
and it is being used in a large number of projects. So that would
lower the "entry barrier" to OpenMoko developement (multimedia
related) to a large extent.

d) A number of companies like Fluendo is providing commercial
gstreamer support and development, which is nice to have.




..now of course, I might be crazy, and all of the above stuff might
not make any sense at all, and in that case feel free to flame me :-).
If it does make sense, and if it sounds like a nice SoC project, I
would love to hear your comments/suggestions, etc.

And if anyone is wondering, I am a engineering student from India (and
one of my favourite pastimes is to listen to MP3s in my Motorola
A780). I have been playing around with multimedia and GTK+/GNOME
related stuff for around three years (libxine, gstreamer, etc). I
developed Senpai, a distant education tool with multimedia
capabilities (http://www.l2c2.co.in/index.php?option=com_content&task=category&sectionid=6&id=16&Itemid=30),
and Hippopotamus Streamer, a tool to do live streaming from your
webcam (http://sayamindu.randomink.org/ramblings/hippopotamus/). I am
currently involved with the Exaile Media Player project
(www.exaile.org), and I'm currently implementing Equalizer support in
Exaile (the basic code is in SVN - beta testers are welcome :-)). I am
also involved with the Sabayon project
(http://www.gnome.org/projects/sabayon/), and I am a member of the
GNOME Foundation and a member of the national working group of
FSF-India. I also do FOSS localisation, but I don't want to bore you
to death :-).

Warm regards,
Sayamindu

PS: The gstreamer developers are eager to help/co-mentor any SoC
project which involves gstreamer, so we can look forward to their
support if this eventually becomes a SoC project.

[1] Gstreamer, in very simple terms, is a multimedia framework which
lets the developer construct pipelines to process multimedia content.
So for a mp3 player, the pipeline would look like

filesrc -> mp3decoder -> alsasink

Of course, there are higher level elements that can autodetect media
formats and decode them automagically.




-- 
Sayamindu Dasgupta
[http://sayamindu.randomink.org/ramblings]