change android graphics backend from linxu framebuffer to direcfb

[Akio]

Dear All:

Right now we are thinking about the possibility to change
the android graphics backend from linux framebuffer to directfb. is
this idea work or not? btw, we also have to know the relateded
libraries have to modify or add new function to handle the original
linux frame buffer function. is this work is worth or not to do that?
the another problem is the graphics memory limitation. because the
display size could be very large and the graphics driver had enabled
2D accelaration function under directfb. OpenGL / ES also has the
hardware enabled driver to be called. If it is worth, what the other
thing do we have to consider? thank you very much.

Best Regards,
Akio

[FranckLefevre]

This sounds a very interesting idea indeed.

We ahve already investigated this recently and we think that such an
approach is possible. We also think that this could lead to dramatic
performance improvments when a proper 2D acceleration is offered
through the DirectFB port. Note that a GPU can be directly handled by
SGL but using DiretcFB clearly allows to directly take advantage of a
large range of already integrated hardware/drivers.

How do you plan to go forward ?


Franck

[Akio Lin]

Dear Franck:

after some study the source code of framebuffer related parts,
I think this is possible. the following file lists is using grep
to search "/dev/fb" or "/dev/graphics/" in android source tree.

./bootable/recovery/minui/graphics.c
./development/simulator/wrapsim/FakeDev.c
./system/extras/tests/framebuffer/fb_test.c
./system/extras/tests/framebuffer/refresh.c
./system/core/adb/framebuffer_service.c
./system/core/init/logo.c
./system/core/init/devices.c
./system/core/toolbox/rotatefb.c
./frameworks/base/libs/ui/EGLDisplaySurface.cpp
./frameworks/base/opengl/tests/sfsim/egl_surface.cpp

for openGL/ES part, the code seems only involved in
only one file(maybe more)
for system core or other parts, the related parts is
not very clear.
hope I don't miss any other one. ^_^

basically, the whole android system and linux kernel can be
separated. in other words, the os kernel could be other than
linux kernel, like xBSD. the whole missing part is the hal
layer(hardware abstraction layer). if the graphics part can
have a clear hal layer, it will be more easier to adapt ohter
platform other than linux.

BTW, after tracing the code, I found that the fb operation is
via mmap to map the fb memory to user space. maybe in directfb,
the framebuffer memory mapping can be adapted and replace the
original mmap operation with some modification. just thinking
and discussion.

for performance issue, I think the directfb will improve the x86
version's performance very dramatically. not only 2D performance,
but also 3D performance.

Am I wrong? if yes, please correct it.

Thank you.

Best Regards,
Akio

[FranckLefevre]

Dear Akio,

Yes, you are right.

The graphic work is actually "splitted" into several parts:
-1- the management of the graphic 2D surfaces (including surface
creation, decoding of bitmaps,...). This is mainly implemented by the
class SkBitmap and the associated classes... We have already
investigated this deeply, and I think that we have a decent knowledge
of this part.
-2- the management of the composition and the bitblits of these
surfaces (SurfaceFlinger,...) to the layers.
-3- the actual blit of the layers to the LCD.

>BTW, after tracing the code, I found that the fb operation is
>via mmap to map the fb memory to user space. maybe in directfb,
>the framebuffer memory mapping can be adapted and replace the
>original mmap operation with some modification. just thinking
>and discussion.

This mainly regards 2 & 3. If we want to fully take advantage of 2D
accelerations, 1 and 2 are certainly the most relevant parts. If the
surface can actually be managed DirectFB, the acceleration will be
used to:
- accelerate the decoding of images, as well as blitting, alpha
blending, resizing,...
- optimize the graphic memory used by offscreen buffers, minimize
the copying of buffers
The concerned classes are mainly part of the graphic toolkit itself,
that is not related to the kernel. ;-)

FYI, the profiling that we are used to do for many distinct embedded
platforms shows that more than 85% of the processor cycles are used by
decoding and bitblitting while using a software graphic toolkit.

The graphic user interfaces that are currently offered by Androïd
applications are quite "static", and mechanisms such as partial
refresh allows to take advantage of partial refresh and dirty surface
clipping, keeping decent (and even very good, on the G1) performances.
But if we want to create more dynamic experiences (with mobiles
objects, semi-transparency surfaces, animated backgrounds...), partial
refresh is no more applicable, and a hardware composer can be very
useful. In this context, DirectFB is a perfect model to mask the
hardware work !

I think that the main part of the performance improvements are there
for the moment.

>for performance issue, I think the directfb will improve the x86
>version's performance very dramatically. not only 2D performance,
>but also 3D performance.

You are right, but I think that focusing on 2D, taking care of fully
keeping the excellent (IMHO) architecture of SGL can immediately allow
us to taste the best part of the cake.


Note that another interesting point is the link between Multimedia and
DirectFB, but this is another story...;-)


Regards.


Franck

> >> Akio- Hide quoted text -
>
> - Show quoted text -

[Akio Lin]

Dear Franck:

Thank you very much.
It seems we all see almost the same target :-)
actually, I'm thing about the android over new x86 platform,
and the directfb is the only way to get 2d fully performance.
iptv stb is a major trend for multimedia application.
if android can be a general framework, the application will
be more rich then ever.

Best Regards,
Akio

[FranckLefevre]

Dear Akio,

I feel ready to spend some resources to continue this work: what about
you ?

Just for my information: are you doing this for fun or do you have a
business target right now ?

Did you already build a x86 version ?


Best regards.

Franck

On Jan 5, 4:23 pm, Akio Lin <akioo...@gmail.com> wrote:
> Dear Franck:
>

>         Thank you very much.
>         It seems we all see almost the same target :-)
>         actually, I'm thing about the android over new x86 platform,

>         and thedirectfbis the only way to get 2d fully performance.

>         iptv stb is a major trend for multimedia application.
>         if android can be a general framework, the application will
>         be more rich then ever.
>
> Best Regards,
> Akio
>
>
>
>
>
> > Dear Akio,
>
> > Yes, you are right.
>
> > The graphic work is actually "splitted" into several parts:
> >   -1- the management of the graphic 2D surfaces (including surface
> > creation, decoding of bitmaps,...). This is mainly implemented by the
> > class SkBitmap and the associated classes... We have already
> > investigated this deeply, and I think that we have a decent knowledge
> > of this part.
> >   -2- the management of the composition and the bitblits of these
> > surfaces (SurfaceFlinger,...) to the layers.
> >   -3- the actual blit of the layers to the LCD.
>
> >> BTW, after tracing the code, I found that the fb operation is
> >> via mmap to map the fb memory to user space. maybe indirectfb,
> >> the framebuffer memory mapping can be adapted and replace the
> >> original mmap operation with some modification. just thinking
> >> and discussion.
>
> > This mainly regards 2 & 3. If we want to fully take advantage of 2D
> > accelerations, 1 and 2 are certainly the most relevant parts. If the

> > surface can actually be managedDirectFB, the acceleration will be

> > used to:
> >     - accelerate the decoding of images, as well as blitting, alpha
> > blending, resizing,...
> >     - optimize the graphic memory used by offscreen buffers, minimize
> > the copying of buffers
> > The concerned classes are mainly part of the graphic toolkit itself,
> > that is not related to the kernel.  ;-)
>
> > FYI, the profiling that we are used to do for many distinct embedded
> > platforms shows that more than 85% of the processor cycles are used by
> > decoding and bitblitting while using a software graphic toolkit.
>
> > The graphic user interfaces that are currently offered by Androïd
> > applications are quite "static", and mechanisms such as partial
> > refresh allows to take advantage of partial refresh and dirty surface
> > clipping, keeping decent (and even very good, on the G1) performances.
> > But if we want to create more dynamic experiences (with mobiles
> > objects, semi-transparency surfaces, animated backgrounds...), partial
> > refresh is no more applicable, and a hardware composer can be very

> > useful. In this context,DirectFBis a perfect model to mask the

> > hardware work !
>
> > I think that the main part of the performance improvements are there
> > for the moment.
>

> >> for performance issue, I think thedirectfbwill improve the x86

> >>         for performance issue, I think thedirectfbwill improve the x86

> >>         version's performance very dramatically. not only 2D performance,
> >>         but also 3D performance.
>
> >>         Am I wrong? if yes, please correct it.
>
> >>         Thank you.
>
> >> Best Regards,
> >> Akio
>
> >>> This sounds a very interesting idea indeed.
> >>> We ahve already investigated this recently and we think that such an
> >>> approach is possible. We also think that this could lead to dramatic
> >>> performance improvments when a proper 2D acceleration is offered

> >>> through theDirectFBport. Note that a GPU can be directly handled by

> >>> SGL but using DiretcFB clearly allows to directly take advantage of a
> >>> large range of already integrated hardware/drivers.
> >>> How do you plan to go forward ?
> >>> Franck
> >>> On Jan 2, 2:15 pm, Akio <akioo...@gmail.com> wrote:
> >>>> Dear All:
> >>>>           Right now we are thinking about the possibility to change

> >>>> the android graphics backend from linux framebuffer todirectfb. is

> >>>> this idea work or not? btw, we also have to know the relateded
> >>>> libraries have to modify or add new function to handle the original
> >>>> linux frame buffer function. is this work is worth or not to do that?
> >>>> the another problem is the graphics memory limitation. because the
> >>>> display size could be very large and the graphics driver had enabled

> >>>> 2D accelaration function underdirectfb. OpenGL / ES also has the

> >>>> hardware enabled driver to be called. If it is worth, what the other
> >>>> thing do we have to consider? thank you very much.
> >>>> Best Regards,
> >>>> Akio- Hide quoted text -

> >> - Show quoted text -- Hide quoted text -

[Dejun Liu]

Dear Franck,Akio,

  I have another question,does the input module should be changed?

 

 

Steven

2009/1/6 FranckLefevre <fla...@gmail.com>

[Akio Lin]

Dear Steven:

this is a good question.
right now, the graphics part is the major problem.
for input side, in my opinion, using directfb only using the
part of graphics, not all of directfb. if you find some thing
missing, please let me know. thank you very much.

Best Regards,
Akio

> Dear Franck,Akio,
>
> I have another question,does the input module should be changed?
>
>
> Steven

> 2009/1/6 FranckLefevre <fla...@gmail.com <mailto:fla...@gmail.com>>

>
>
>
>
> Dear Akio,
>
> I feel ready to spend some resources to continue this work: what about
> you ?
>
> Just for my information: are you doing this for fun or do you have a
> business target right now ?
>
> Did you already build a x86 version ?
>
>
> Best regards.
>
> Franck
>
>
>
>
> On Jan 5, 4:23 pm, Akio Lin <akioo...@gmail.com

[FranckLefevre]

Dear Steven,

Reg. the performances, graphic rendering is THE place where dramatic
improvments can be obtained.

I think that there is no significant improvment to hope by modifying
the input system, as far as you don't try to use multitouch screens.
If you want to use a multitouch screen (but you need one do to so !)
the current windows mgr may be an issue. This is not a real problem as
the current Android MMI - that is very good regarding ergonomics - is
not that dynamic.


Franck

On Jan 6, 3:03 pm, "Dejun Liu" <dejun.liu1...@gmail.com> wrote:
> Dear Franck,Akio,
>
>   I have another question,does the input module should be changed?
>
> Steven

> 2009/1/6 FranckLefevre <flas...@gmail.com>

[Dejun Liu]

Dear Franck, Akio:
          thks a lot,i understand.
          and sorry for this later email.

best regards.
Steven Liu

2009/1/6 Akio Lin <akio...@gmail.com>