webOS and GStreamer

So we've learned from Chapter 1 of Palm webOS by O'Reilly that the webOS "media server" is based on GStreamer:

The media server is based upon gstreamer and includes support for numerous audio and video codecs, all mainstream image formats, and supports image capture through the built-in camera. Video and audio capture is not supported in the initial webOS products, but is inherently supported by the architecture. Video and audio playback supports both file and stream-based playback.

What exactly is GStreamer, and what does this mean for our applications? From the GStreamer website:

GStreamer is a library for constructing graphs of media-handling components. The applications it supports range from simple Ogg/Vorbis playback, audio/video streaming to complex audio (mixing) and video (non-linear editing) processing. Applications can take advantage of advances in codec and filter technology transparently. Developers can add new codecs and filters by writing a simple plugin with a clean, generic interface.

Basically, GStreamer is a framework for creating streaming multimedia applications, and one of the most common uses of the GStreamer framework is to build a media player, which is presumably what Palm has done for the Pre. One of the UI widgets mentioned in the first chapter of Palm webOS is a viewer widget, which allows you to "play or view content within your application, such as audio, pictures, video or web content". The viewer widget might invoke GStreamer components to display the first three types of media and webkit components to display web content.

So, if the media player is built on GStreamer, how can we as application developers leverage this open framework? Unfortunately, it's not clear at this point. GStreamer provides interfaces for both application programers and plugin programmers. Applications or plugins for GStreamer will have to be written in C or one of the languages for which there are bindings for GStreamer, currently Python, Perl, .NET, C++, Guile, Java or Ruby; none of which are currently supported by webOS.

So that leaves us with a few possibilities. One is that Palm has written a Javascript binding for GStreamer that would be exposed to webOS applications. This would be the most powerful option; potentially enabling the development of full-featured media player applications. Another is that Palm will expose the GStreamer API to compiled, binary GStreamer plugins that would then be included with webOS applications. This option, though unlikely (Palm has said nothing about any component of a webOS application being built in a compiled language), would enable developers to include additional codecs and other enhanced media-handling functionality in their applications. The third possibility is that the GStreamer implementation in webOS is closed to application developers, and that Palm only mentioned it to let us know that the webOS media player is built on an open, extensible platform for ease of future upgrades and add-ons (by the internal Palm development team).

If it's one of the first two possibilities, extending the default GStreamer functionality included with webOS will likely involve some serious coding skill. If you laugh in the face of danger (like carpal tunnel syndrome and near-sightedness), you can check out these docs for more info:

As soon as we hear from Palm, we'll come back to this article and hopefully be able to home in on the possibilities.