Neo1973 video (was Re: What would be a realistic but challenging level for Bryce announced trophy money for video playback on the Neo1973? Re: h.264 format is now open?)

[Siarhei Siamashka]

On Saturday 03 February 2007 15:47, you wrote:

> Also, the 770 has a landscape display.  We have a portrait display.  The
> S3C2410 cannot rotate the image, so you would have to rotate every frame
> in software, too!

That's a good point. Anyway, rotation can be combined with scaling or color
format conversion and done in a single pass, so overhead should not be too
big. One more solution is rotation at video transcode stage as Mikko
suggested. 

> > Just for some experiment, I compiled mplayer for arm920t (not using
> > armv5te instructions), and benchmarked it with sdl video output (software
> > YUV->RGB conversion, generic nonoptimized scaling 320x240 => 640x480) and
> > libmad mp3 audio decoder.
>
> Please note that the LCM we use in the Neo1973 can do hardware scaling,
> e.g. theoretically you can software-reconfigure the LCM to behave as
> QVGA 240x320, and then change the s3c2410_fb kernel driver timings
> accordingly.

If I understand that correctly, it is not arbitrary scaling but support for
240x320 resolution? But as we should first aim at transcoded video 
support anyway, that would provide some performance improvement.

> This has not been tested or implemented by us, since we're mainly
> interested in getting a high-res phone UI working right now :)

I clearly understand that :) I guess it is one of the reasons, why you
announced early access to the device for open source developers. I hope 
that some of them would try implementing some video support.

> However, if somebody skilled wants to write, integrate and test code for
> this, I'm happy the help.   We cannot disclose the LCM data sheet, but I
> can add the software sequence for 240x320 / 480x640 switching to the
> [GPL licensed] kernel LCM driver at least after phase-1 (March 11).
>
> The S3C2410 user manual is publicly available from Samsung
> Semiconductors, so basically it's up to whoever is interested to
> implement it.  One interesting question is how to merge this into
> X11/Kdrive... As far as I understand,  only the very latest big
> (non-kdrive) X11 has randr capabilities for dynamically adding/removing
> viewports and changing resolutions.
>
> [But did I mention that Keith Packard gets a phase-0 phone? *g*]

As it seems to be not quite trivial to do, this part of work can wait a bit
until some initial video benchmarks are available (centered nonscaled
320x240 video playback vs. software scaled to fullscreen). 

About integration with x11/kdrive. Nokia 770 also uses kdrive, and implements
some extensions to access extra features (pixel doubling). Unfortunately it
does not provide access to hardware YUV colorspace (libxv exists, but it is
just a stub and does nothing). So right not mplayer on Nokia 770 uses some
kind of hack and accesses framebuffer concurrently with x11 server.
Synchronization is not perfect, so it is possible to get garbage on the screen
when you switch between different applications a lot while mplayer video
playback is active. A more clean solution would be to patch xserver to
interact with mplayer better to synchronize framebuffer access (kdrive on
Nokia 770 already has some patches to synchronize with DSP video 
decoder used by built-in video player). Maybe I'll try this later.

I'm just interested in improving video support for ARM based devices, 
that's why I posted to this openmoko mailing list . I'm currently trying to
integrate a fast scaler for ARM into ffmpeg library (the engine used 
by mplayer, vlc and the other video players for linux ):
http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-January/051209.html

So now I'm trying to collect information about what scaling/color conversion
functionality would be useful for various devices. Looks like Neo 1973 
may need rgb565 => rgb565 scaler, image rotation support and probably
yv12 => rgb565 (grayscale). I don't have much free time, so support for 
just Nokia internet tablets has the highest priority for me. But if anybody
else would like to work on the features needed for Neo 1973 video support, 
I would be glad to help.