Natty video acceleration / kernel status

[Tyler Hall]

I've just started with the pandaboard running 11.04 and I'm trying to
figure out what level of functionality I should expect. I'm mainly
interested in video playback. I see that there are some interesting
packages in the omap-trunk PPA, but I can't figure out how to get them
to work for accelerated video decoding. Are they even supposed to
work at this point?

I would also be interested in doing kernel development, but I can't
find where the 2.6.38 Natty kernel work is taking place. Ubuntu's ti-
omap4 branch doesn't have any recent changes and the new work in TI's
repos seems to be on 2.6.35. Is the community mostly focused on
Android or am I looking in the wrong place?

[Nipuna Gunasekera]

If you're looking for 1080p playback support on ubuntu 11.04 it's not available yet and I believe should be functional in a few more weeks.  

Ubuntu 10.10 release should have those features supported using the TI OMAP public PPAs.  See here:  http://www.omappedia.org/wiki/Ubuntu_PPA

[Ricardo Salveti]

On Tue, May 17, 2011 at 11:07 AM, Tyler Hall <tyler...@gmail.com> wrote:
> I've just started with the pandaboard running 11.04 and I'm trying to
> figure out what level of functionality I should expect.  I'm mainly
> interested in video playback.  I see that there are some interesting
> packages in the omap-trunk PPA, but I can't figure out how to get them
> to work for accelerated video decoding.  Are they even supposed to
> work at this point?

Not yet available for Natty. TI said it should be releasing the new
packages in a few weeks.

> I would also be interested in doing kernel development, but I can't
> find where the 2.6.38 Natty kernel work is taking place.  Ubuntu's ti-
> omap4 branch doesn't have any recent changes and the new work in TI's
> repos seems to be on 2.6.35.  Is the community mostly focused on
> Android or am I looking in the wrong place?

Check http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-natty.git;a=shortlog;h=refs/heads/ti-omap4

Cheers,
--
Ricardo Salveti de Araujo

[haunma]

On May 17, 7:11 pm, Ricardo Salveti <rsalv...@rsalveti.net> wrote:
> Check http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-natty.git;a=shortlog;h=r...

That link seems to be broken.

Regarding video, I'm sure the OP is not the only one confused. Would
it be possible for someone knowledgeable to give us a birds-eye view
of the situation? An article for the wiki perhaps. The discussion
here is usually focused on specific use cases, getting some feature
"X" to work, rumors/promises of impending code releases from TI, and
so on. Admittedly I haven't been following it all, but I wonder

1) What *are* the different layers between X11 and the hardware
nowadays, including stuff needed for accel video decode and accel
graphics? It all used to be so simple...

2) Which pieces are available to hobbyists (in binary or source code
form), and which are not?

3) Which IP is owned by Imagination Tech. and which is owned by TI?
And what do either stand to gain by *not* giving it away for free?
(Ok, not really expecting an answer to the last part :)

4) Put this into the larger context of the PowerVR / Linux train wreck
(compare/contrast with the Intel GMA500 experience) and unpack it for
us. Should we expect a different outcome? Why or why not?

5) Is it likely that TI + the community will arrive at a solution
which enables long-term support (including video decode and graphics
acceleration) in the relevant upstream projects? Or is this likely to
be another case of "get this version of the kernel, this version of X,
this version of ..." with support forever tethered to legacy
packages?

In short, could someone explain to me, in a few well-written
paragraphs, why video support on Pandaboard is different from the
equivalent functionality on my Intel desktop PC?

(Reading over this, it seems a bit accusatory. Sorry. I'm not trying
to assign blame at all, just being frank in the hopes of a similarly
frank reply. I don't expect the corporate contributors here to share
my "world view," but I am honestly curious about theirs.)

Regards,

Mark

[Dechesne, Nicolas]

On Wed, May 18, 2011 at 7:37 PM, haunma <hau...@keteu.org> wrote:

> Check http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-natty.git;a=shortlog;h=r..

That link seems to be broken.

http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-natty.git, branch ti-omap4
 

Regarding video, I'm sure the OP is not the only one confused.  Would
it be possible for someone knowledgeable to give us a birds-eye view
of the situation?  An article for the wiki perhaps.  The discussion
here is usually focused on specific use cases, getting some feature
"X" to work, rumors/promises of impending code releases from TI, and
so on.  Admittedly I haven't been following it all, but I wonder

let me try to answer
 

1) What *are* the different layers between X11 and the hardware
nowadays, including stuff needed for accel video decode and accel
graphics?  It all used to be so simple...

- the video decoder algorithm runs on the Ducati h/w coprocessor that has a bunch of h/w accelerator related to video and image processing
- from the Linux standpoint, the decoder is exposed through OMX API and/or DCE API.
- TI provides GST elements for wrapping the OMX/DCE algorithms so that they integrate nicely into any GST based MM framework (such as the default one in Ubuntu)
- we have implemented the XVideo extension in the X11 OMAP driver using the OMAP4 DSS overlays. In essence when 'totem' plays a video, the totem window is displayed (borders, buttons, ...) but the video data is not displayed by 'totem'. Instead totem creates a transparent hole in the window, and GST will send the video data to the DSS overlay using another input pipe. The size and position of the 'hole' is controlled by the size and position of the totem window, and X uses this to configure the video overlay properly.
- SGX is not used at all during video playback. at least for now... we have some initial support for video texture streaming, but it's not integrated fully yet. in such case the video data (YUV) is provided to SGX which is doing all the rendering.

2) Which pieces are available to hobbyists (in binary or source code
form), and which are not?

in 11.04 what's missing to run video decoder use case:
 - ducati firmware (binary)
 - support for OMAP4 in the v4l2/dss driver (as we have moved away from the old .35 to the 'mainline' .38 kernel there is some functionalities which have not been ported yet.

The modified GST packages are available in the PPA, but this is a trivial problem anyways.
 

3) Which IP is owned by Imagination Tech. and which is owned by TI?
And what do either stand to gain by *not* giving it away for free?
(Ok, not really expecting an answer to the last part :)

SGX ==> IMG
Ducati ==> TI

4) Put this into the larger context of the PowerVR / Linux train wreck
(compare/contrast with the Intel GMA500 experience) and unpack it for
us.  Should we expect a different outcome?  Why or why not?

not sure i understand what the question is
 

5) Is it likely that TI + the community will arrive at a solution
which enables long-term support (including video decode and graphics
acceleration) in the relevant upstream projects?  Or is this likely to
be another case of "get this version of the kernel, this version of X,
this version of ..." with support forever tethered to legacy
packages?

TI recently released the video codecs for the IVAHD IP which is the IP behind the Ducati. see http://software-dl.ti.com/dsps/dsps_public_sw/codecs/HDVICP2/index_FDS.html
i am not 100% sure but I think it should be exactly what we are using today in our ducati binary blob. given that all the other s/w required (DSP Bios, CodecEngine, Syslink, ... ) are already available 'freely' (not open source...) that opens the door to create a ducati blob by anyone outside TI. i know that Rob is looking forward to making updates to DCE to use these codecs so that the ducati DCE binary blob can be a bit more 'open'...

once you have that you are close to be able to decode and display video.

In short, could someone explain to me, in a few well-written
paragraphs, why video support on Pandaboard is different from the
equivalent functionality on my Intel desktop PC?

i hope you will consider my few paragraphs as well written ;-)

(Reading over this, it seems a bit accusatory.  Sorry.  I'm not trying
to assign blame at all, just being frank in the hopes of a similarly
frank reply.  I don't expect the corporate contributors here to share
my "world view," but I am honestly curious about theirs.)

this is fine. we very well understand how painful it is.

[haunma]

Hi Nicolas,

Thanks, this helps a lot.

On Thursday, May 19, 2011 5:41:08 AM UTC-7, ndec wrote:

4) Put this into the larger context of the PowerVR / Linux train wreck
(compare/contrast with the Intel GMA500 experience) and unpack it for
us.  Should we expect a different outcome?  Why or why not?

not sure i understand what the question is

What I meant was that in general the experience with embedded GPUs on Linux, especially SGX, has been an unpleasant one.  (For a sample of the dialog among the main kernel developers, see this lengthy thread:  http://lists.freedesktop.org/archives/dri-devel/2010-December/006227.html)  The situation with GMA500 on Intel Atom is the most famous, because it affected a lot of people.  To summarize the main problems:

a)  IP was so restricted that only a small team inside Imag Tech could do driver development, hence progress was very slow, and QA and accountability almost non-existent.  The quality was so poor that even the Windows (!) drivers were never completed to a high standard.

b)  By the time the [buggy, partially implemented] Linux drivers were available, the hardware development cycle was already so far along, that there was little motivation to fix things for a chipset already on its way toward obsolescence.

c)  Free-software APIs have changed, as they always do, leaving the original drivers unusable, because only Imag Tech could update them, and they have no further interest.

This last point is less important for a dev board like Panda, because most of us aren't likely to care if we can run the latest kernel and graphical environments on Panda five years from now.  But as OMAP gets into more mainstream devices, this becomes more of an issue.  Ideally, all of the vital bits would get into the upstream open-source projects.  This effectively makes the software immortal, so if TI gets nuked tomorrow, everything will still work years into the future.  The worry with closed-source blobs is that this forces TI into an ongoing support role.  Then, Linux video will only work on OMAP as long as TI actively maintains the blobs, and once TI loses interest the hardware platform also dies, because fresh code no longer runs on it.  (And, often as not, the blobs lag about one software generation behind the kernel/X/desktop, causing no end of headaches to the end user.)

I did not realize that OMAP4 video decode was independent of the GPU; that's good news.  A shame that they make such nice hardware, which ends up under-utilized most of the time.

Regards,

Mark

[Alison Chaiken]

Thanks to Nicholas for his helpful and detailed post. I also found
Rob Clark's talk from Gstreamer 2010 Conference helpful:
http://free-electrons.com/blog/gst-2010-videos/ I reaslize you
folks don't make IP choices for TI, let alone ImagTech. These
thoughts are fresh in my mind given my (for the moment!) employer's
decision to switch its platform from Linux to Windows.

haunma writes:
>Then, Linux video will only work on OMAP as long as TI actively maintains
>the blobs, and once TI loses interest the hardware platform also dies,
>because fresh code no longer runs on it. (And, often as not, the blobs lag
>about one software generation behind the kernel/X/desktop, causing no end of
>headaches to the end user.)

Indeed, yesterday MeeGo released version 1.2 which requires hardfp
support. My application (IVI) requires HW-accelerated graphics,
which Beagleboard's SGX530 and Nvidia's Tegra 2 already provide. So
I may be forced to change to one of these other platforms unless the
hardfp blobs for SGX540 appear soon.

Questions: Given the discussion above, are there hardfp blobs for
Ducati M3 instead which might address the problem and which could be
invoked in GST via OpenMax? And is Ducati used for video encode
(sourcing of an RTP/HTTP stream) as well as decoding? Thanks again.

--
Alison Chaiken
MeeGo Technical consultant
Nokia Mobility Solutions
Sunnyvale CA
650-279-5600
alison....@nokia.com
http://exerciseforthereader.org/

[Rob Clark]

On Fri, May 20, 2011 at 5:47 PM, Alison Chaiken <alch...@gmail.com> wrote:
>
> Questions: Given the discussion above, are there hardfp blobs for
> Ducati M3 instead which might address the problem and which could be
> invoked in GST via OpenMax?   And is Ducati used for video encode
> (sourcing of an RTP/HTTP stream) as well as decoding?    Thanks again.
>

The ducati firmware (.xem3 files) would not need to be recompiled for
a9 side to use hardfp.. you aren't directly calling code on m3 side
from a9, but instead there is an RPC layer that turns the call into a
message to the other side.. so the two sides could even be using
completely different instruction sets (such as dsp vs a9)... And
anyways, m3 doesn't have vfp/neon.

And yes, ducati (or more precisely ivahd, managed by ducati) is used
both for video encode and decode. Muxing/demuxing and those sorts of
mm framework layer stuff happens on the a9 side, but the heavy lifting
for decoding/encoding a bitstream are on ivahd/ducati.

BR,
-R

[JuglansNigra]

I can confirm that 10.10 works with Open GL ES and 1080p playback (mp4
and h.264). You need to enable universe and multiverse
and then install all the OMAP4 packages. Then things will start to
happen..

11.04 is in pretty ragged shape right now for ARM - I am thinking of
downgrading.
I just updated all my packages in 11.04 and some broken initramfs
package made the board unbootable. I've needed to reflash it twice.

Let the dust settle for a while with 11.04.


On May 17, 6:52 pm, Nipuna Gunasekera <nipuna.gunasek...@gmail.com>
wrote:

> If you're looking for 1080p playback support on ubuntu 11.04 it's not
> available yet and I believe should be functional in a few more weeks.
> Ubuntu 10.10 release should have those features supported using the TI OMAP
> public PPAs.  See here:  http://www.omappedia.org/wiki/Ubuntu_PPA
>

[Vladimir Pantelic]

On 05/21/2011 12:47 AM, Alison Chaiken wrote:
>
> Indeed, yesterday MeeGo released version 1.2 which requires hardfp
> support. My application (IVI) requires HW-accelerated graphics,
> which Beagleboard's SGX530 and Nvidia's Tegra 2 already provide. So
> I may be forced to change to one of these other platforms unless the
> hardfp blobs for SGX540 appear soon.

does the sgx API use floating point at all? if not, why does hard
vs soft fp calling convention matter?

> Questions: Given the discussion above, are there hardfp blobs for
> Ducati M3 instead which might address the problem and which could be
> invoked in GST via OpenMax? And is Ducati used for video encode
> (sourcing of an RTP/HTTP stream) as well as decoding? Thanks again.

The ducati "blobs" run on the M3s which are fixed point only...

[Alison Chaiken]

JuglansNigra writes:
> I can confirm that 10.10 works with Open GL ES and 1080p playback (mp4
> and h.264).

SGX demos from OGLES2 SDK from Imagination Technology all run great
with 2.6.35 kernel from TI's Ubuntu repo, as previously reported.
The demos work with the MeeGo 1.1 userland as well as the Ubuntu
10.10.

Vladimir Pantelic

> does the sgx API use floating point at all?

Someone who has the source code may know for sure . . . I agree
that it wouldn't make too much sense for the GPU (or DSP, or Ducati)
to use the NEON instruction set of the FPU. However, that doesn't
mean that those drivers will work both with libraries compiled with
-hardfp as they do with drivers compiled with -softfp.

> if not, why does hard vs soft fp calling convention matter?

TI's response that they will make pvr_omap drivers available for Natty
in a few months that are compiled with -hardfp suggests to me that
there's more to solving the problem than "make clean; make -hardfp."

--
Alison Chaiken
(650) 279-5600  (cell)
             http://www.exerciseforthereader.org/
A career in Silicon Valley is just like a chess game, only players can
move all the pieces every turn and some of the pawns bite.