Maemo Multimedia

Multimedia
Maemo offers lots of possibilities for multimedia applications. Maemo was built with
the Internet Tablet devices in mind - hardware in this form factor provides a big highresolution
touch screen, audio input and output, video and image capturing through
the camera, fast network connections for streaming and a digital signal processor for
efficient audio and video manipulation.
For developers there are open programming interfaces to make use of these features
apart from directly programming the DSP. Preferred way of doing video and audio
programming for maemo is using the GStreamer framework and for manipulating static
images there are several libraries. Interactive multimedia for applications like games
can be done using SDL framework. The Multimedia Architecture document[59] gives
more details on how the components are organized both on the physical device and on
the maemo SDK.
6.1 GStreamer - multimedia framework
The main multimedia framework in maemo is GStreamer. It is based on the concept of
pipelines which are made of multiple elements. Elements themselves can be practically
anything that do something with a data stream.
Maemo includes a variety of these elements to support audio and video effects, encoding
and decoding and interfacing with the device’s hardware. Using this approach,
developer does not have to bother with details like low level hardware management,
audio and video compression schemes etc. If the elements included with the OS are
not enough, more can be either compiled for maemo or new ones can be developed.
Developer documentation for GStreamer can be found at the project’s website[23].
6.2 Stream encoding and decoding
Maemo devices include a wide arsenal of video and audio codecs which enables the
maemo applications to read and write nearly all commonly used video and audio formats.
These are supported also by the GStreamer framework. For many purposes the
developer doesn’t even have to specify the used codecs implicitly since GStreamer automatically
detects the format and which codec to use. The playbin[22] GStreamer
base plugin provides a convenient abstraction layer for all audio and video content.

Due to non-technical reasons, most of the codecs are not distributed with the SDK
which is good to keep in mind when developing applications relying on such features.
6.3 Audio
For audio programming maemo has two main APIs: GStreamer and ESound. Usually
system sounds, such as sounds for notifying user for an event, e.g. battery low, is
played through ESound. More sophisticated operations, for example playing music
files or recording audio, should be generally done using GStreamer which provides
better performance and a much more flexible API. Most of the maemo’s computing
intensive GStreamer elements are implemented using the device’s DSP which greatly
enhances their performance.
Linux kernel also has two lower level audio interfaces: ALSA and OSS. From these
ALSA is supported through a plug-in package which is part of the SDK. The legacy
API OSS is not supported by the kernel, but ALSA has an OSS emulation system,
which works for most purposes.
For the audio APIs’ documentation, see the GStreamer web site[23], ESound white
paper[9] and ALSA project’s web site[1]. More in-depth material is available in the
Multimedia Architecture guide[59].
6.4 Video
Although the framework hides much of the implementation and hardware details, it’s
good for a developer to know what happens beneath the interfaces. Video capturing
is done via a Linux kernel’s Video4Linux API and graphics are displayed using the
X Window System. Practically all GNU/Linux applications rely on these components
for video tasks so porting existing applications should be quite effortless and finding
support easy.
Hands on instructions for using capture and output features are given in the Maemo
Camera How-to[30].
6.5 Digital Signal Processor
Under the hood Internet Tablets have a dedicated digital signal processor (DSP). It’s
design is optimized for tasks like stream coding and audio effects. Maemo has a high
level programming support for the DSP in form of GStreamer elements which can
decode most of the supported file formats. By using the DSP computing load is also
taken off from the main processor which greatly enhances system performance and
responsivity.
6.6 Graphics and Images
Maemo includes several libraries for working with images and graphics:
 Cairo is a library for doing 2D vector graphics. It features for example high quality
vector based graphics, support for importing and exporting variety of formats
such as SVG, PNG, PDF and Postscript and bindings for many programming
languages

 GDK and GdkPixbuf are the bitmap graphics libraries that GTK+ is built on.
They together provide opening and saving bitmap images in JPEG, PNG, TIFF,
ICO and BMP formats, tight integration with GTK+ and tools for drawing with
graphic primitives. Also e.g. loading of SVG images is supported
 For more detailed control over the different image formats, there are many more
specialized libraries, for example libpng, libjpeg and libwmf.
Visit Cairo website[5] for guides, tutorials and references. GDK’s and GdkPixbuf’s
documentation can be found at GTK+-project’s website[25].
6.7 Games
For game developers maemo offers a common startup and settings screen and a framework
called osso-games-startup. Using this library game developers and porters can
reduce the common startup screen coding and concentrate on the real game programming.
osso-games-startup’s operating system communication features also offer an
interface to make games behave correctly. For example it eases the management of full
screen mode and exceptional situations, such as battery low notification, in a unified
and user friendly manner. See Using Games Start-up Screen document[74] for more
in-depth description and API usage.