GPU Emulation plans

[David Turner]

GPU emulation as implemented in the Android emulator currently works ok, but not great, due to several factors. Given that it is becoming a requirement to decently emulate recent Android devices with insane pixel counts, we're going to fix it. Here's how.

1) Fixing implementation bugs.

We've started aggressively fixing bugs in our implementation, based on the feedback of the dEQP compliance test suite, which was developed by drawElements, a company recently acquired by Google.

These fixes are both on the emulator-side decoder and translator libraries, as well as the guest-side encoder one. All changes go directly into the AOSP source tree, but the latter fixes will need to be back-ported to the SDK system images of previous releases.

2) Using ANGLE to work-around buggy OpenGL drivers on Windows.

Experience taught us that many of our users are using the emulator on Windows machines with poor or buggy OpenGL drivers. The result is often bad rendering, and more rarely crashes. To work-around this issue, we plan to follow Chrome's steps and use the ANGLE library to translate GLES2 calls directly into DirectX ones on this platform (since DirectX has more mature drivers on Windows).

Note that, since ANGLE only supports GLESv2, this would require a GLESv1 -> GLESv2 translator at a some point. Note that currently GLESv1 is only used to display the boot animation in AOSP, and to support really old apps, so that might be less of a priority.

For the record, there is a MIT-licensed GLESv1 -> Desktop GL translator at https://github.com/vonture/fixie which could be modified to do this.

3) Software-based rendering to support headless emulation.

There are cases where the machine running the emulator has no GPU and/or no display connection (typically when running on a server, in a unit-test farm). To support these, we plan to provide a purely software-based rendering backend).

Another benefit of software-based rendering is the ability to support snapshotting (which is currently mutually exclusive with GPU emulation).

We started experimenting with Offscreen Mesa with decent results (though its current code still has some limitations), and also want to have the option to use alternative backends like SwiftShader, which seems to perform much better.

Note that these changes should minimally affect the emulator codebase itself, i.e. they should mostly impact the host-side GPU emulation libraries.

More details are available in this public design document, however feel free to comment on this thread.

- Digit

[Christoffer Dall]

Hi Digit,

On Mon, Nov 3, 2014 at 12:28 PM, David Turner <di...@android.com> wrote:

[...]

>
> More details are available in this public design document, however feel free
> to comment on this thread.
>

I get a "No permission error" when clicking the public document ;)

When I tried with my Linaro account, Google docs tells me the document
doesn't exist...

Can you have a look?

Thanks,
-Christoffer

[David Turner]

It looks like a Drive/Docs failure. I've copied it to this new document that should be viewable by anyone without requiring a sign-in, see:

Sorry for the annoyance

- Digit

[Rishi Ranjan]

Hi Digit,

             I had mentioned in another thread that ANGLE is not going to support ES 3.1 and AEP. Is there any plan to change ANGLE to support ES 3.1 and AEP? 

Thanks,

Rishi

[David Turner]

On Mon, Dec 29, 2014 at 4:18 PM, Rishi Ranjan <rishi....@gmail.com> wrote:

Hi Digit,

             I had mentioned in another thread that ANGLE is not going to support ES 3.1 and AEP. Is there any plan to change ANGLE to support ES 3.1 and AEP? 

Not that I know of (I'm not in the Angle team :-)), a good idea would be to ask on chromium-dev@ which is the public mailing list where to contact them. For now, we'll be happy getting decent 2.x support.

 

Thanks,

Rishi

On Monday, November 3, 2014 6:53:29 AM UTC-8, David Turner wrote:

It looks like a Drive/Docs failure. I've copied it to this new document that should be viewable by anyone without requiring a sign-in, see:

Sorry for the annoyance

- Digit

On Mon, Nov 3, 2014 at 12:45 PM, Christoffer Dall <christof...@linaro.org> wrote:

Hi Digit,

On Mon, Nov 3, 2014 at 12:28 PM, David Turner <di...@android.com> wrote:

[...]

>
> More details are available in this public design document, however feel free
> to comment on this thread.
>
I get a "No permission error" when clicking the public document ;)

When I tried with my Linaro account, Google docs tells me the document
doesn't exist...

Can you have a look?

Thanks,
-Christoffer

--
You received this message because you are subscribed to the Google Groups "Android emulator development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-emulator...@googlegroups.com.
To post to this group, send email to android-em...@googlegroups.com.
Visit this group at http://groups.google.com/group/android-emulator-dev.
For more options, visit https://groups.google.com/d/optout.

[Rishi Ranjan]

Hi David,

              Do you have something to try for us? I am trying to make this work with ANGLE but there are too many changes needed in current emulator.

ANGLE team will support Opengl 3.1 on desktop GL in future but not on DirectX. 

Thanks,
Rishi

To unsubscribe from this group and stop receiving emails from it, send an email to android-emulator-dev+unsub...@googlegroups.com.

[Rishi Ranjan]

I don't need the changes now. I was able to change the emulator locally to use ANGLE and run Instancing using ES 3.0. 

Thanks,

Rishi

[Chih-Wei Huang]

Hi David,
Glad to see the plan to improve GPU emulation.

The Android-x86 group is also working to enable
GPU acceleration on QEMU via the Virgil 3D project.
(ref: https://virgil3d.github.io/ )
The plans include:

* kernel 4.4 with virtio-gpu enabled
* GLES stack by mesa 11.2-devel (with virgl patches)
* AOSP's drm_gralloc & drm_hwcomposer
* Qemu 2.5 (all related patches upstreamed)

You may check the progress via
https://groups.google.com/d/msg/android-x86-devel/RErWaXk3b7g/aLd1Fh-kAQAJ

Related code:
https://github.com/openthos/

How do you think about the plans?

[David Turner]

Hello Chih-Wei,

Sorry for taking this long to answer this post.

The short version of my answer is that we're taking a look at virgil, but don't plan to use it for now.

The longer version is that using virgil would require a lot of work from various angles:

• Upgrading to a newer kernel (we're still on 3.10).
  
• Upgrading to a newer version fo QEMU2 (we're still on 2.2).
  
• Either port back guest virgil code to all previous Android SDK system images (probably a massive task),
  or support both images with virgil and without it (simpler, but probably a bit complicated).
• Other considerations regarding ANGLE support on Windows hosts, GPU snapshotting, GLES 3.x support, and Vulkan.

It's likely we're going to take another deep look at Virgil in a distant future, since most of the issues above are being worked on, but for now, our priorities are different.

Regards and good luck :)

- Digit

--

You received this message because you are subscribed to the Google Groups "Android emulator development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to android-emulator...@googlegroups.com.

To post to this group, send email to android-em...@googlegroups.com.

Visit this group at https://groups.google.com/group/android-emulator-dev.

[Alex Bennée]

'David Turner' via Android emulator development <android-em...@googlegroups.com> writes:

> Hello Chih-Wei,
>
> Sorry for taking this long to answer this post.
>
> The short version of my answer is that we're taking a look at virgil, but
> don't plan to use it for now.
>
> The longer version is that using virgil would require a lot of work from
> various angles:
>

> - Upgrading to a newer kernel (we're still on 3.10).
> - Upgrading to a newer version fo QEMU2 (we're still on 2.2).
> - Either port back guest virgil code to all previous Android SDK system

> images (probably a massive task),
> or support both images with virgil and without it (simpler, but probably
> a bit complicated).

I should mention a bunch of the Linaro android developers have been
working on alternate builds with a) vanilla kernels, b) QEMU mainline
and c) MESA based graphics stack in userspace.

https://plus.google.com/u/1/104877287288155269055/posts/YqRFghfwBKB

> - Other considerations regarding ANGLE support on Windows hosts, GPU

> snapshotting, GLES 3.x support, and Vulkan.

I think these are the things that get in the way of any potential
up-streaming to the Android project.

--
Alex Bennée

[Chih-Wei Huang]

Hi David,
Thank you for the reply.

2016-03-01 5:32 GMT+08:00 David Turner <di...@google.com>:
> Hello Chih-Wei,
>
> Sorry for taking this long to answer this post.
>
> The short version of my answer is that we're taking a look at virgil, but
> don't plan to use it for now.
>
> The longer version is that using virgil would require a lot of work from
> various angles:
>
> Upgrading to a newer kernel (we're still on 3.10).

Not a big problem. We have provided the kernel 4.4 android porting.
(based on AOSP's kernel/common android-4.4 branch)

> Upgrading to a newer version fo QEMU2 (we're still on 2.2).

Good to upgrade.

> Either port back guest virgil code to all previous Android SDK system images
> (probably a massive task),
> or support both images with virgil and without it (simpler, but probably a
> bit complicated).
> Other considerations regarding ANGLE support on Windows hosts, GPU
> snapshotting, GLES 3.x support, and Vulkan.

I think the most difficult parts are the virglrenderer
only support Linux host now.
Porting it to other platforms that the SDK supports
is a big challenge.

Anyway, we have integrated the virgl supports
into android-x86 codebse.
(most of the hard works are done by Rob Herring)
For those who are interesting, please see

https://groups.google.com/d/msg/android-x86-devel/RErWaXk3b7g/XTakX5X8EwAJ
https://github.com/openthos/openthos/wiki/virtio-gpu

> It's likely we're going to take another deep look at Virgil in a distant
> future, since most of the issues above are being worked on, but for now, our
> priorities are different.

That's fine.

Another task we are doing is to enable GLES2 software rendering
via mesa llvmpipe driver. That may also benefit the SDK emualtor.

Do you have any plan to upgrade AOSP's mesa3d
to the latest upstream? (like we have done)

> On Thu, Jan 14, 2016 at 1:34 AM, Chih-Wei Huang <cwh...@android-x86.org>
> wrote:
>>
>> Hi David,
>> Glad to see the plan to improve GPU emulation.
>>
>> The Android-x86 group is also working to enable
>> GPU acceleration on QEMU via the Virgil 3D project.
>> (ref: https://virgil3d.github.io/ )
>> The plans include:
>>
>> * kernel 4.4 with virtio-gpu enabled
>> * GLES stack by mesa 11.2-devel (with virgl patches)
>> * AOSP's drm_gralloc & drm_hwcomposer
>> * Qemu 2.5 (all related patches upstreamed)
>>
>> You may check the progress via
>> https://groups.google.com/d/msg/android-x86-devel/RErWaXk3b7g/aLd1Fh-kAQAJ
>>
>> Related code:
>> https://github.com/openthos/
>>
>> How do you think about the plans?


--

Chih-Wei
Android-x86 project
http://www.android-x86.org

[Marc-André Lureau]

Hi

On Wed, Mar 2, 2016 at 3:33 AM, Chih-Wei Huang <cwh...@android-x86.org> wrote:
> I think the most difficult parts are the virglrenderer
> only support Linux host now.
> Porting it to other platforms that the SDK supports
> is a big challenge.

virgl is fairly portable, since it merely needs opengl. As a proof of
concept, I fixed the build for win32 of virglrenderer, and checked
qemu crossbuilds with virtio-vga & virgl. I haven't tested it though.

--
Marc-André Lureau

[David Turner]

Hello Marc-André,

Thanks for participating to the discussion. While I don't know enough about virglrenderer

to assess its portability, I must admit a phrase like "merely needs opengl" is a little too

scary due to past experiences dealing with platform differences, and the joy of GLX / WGL / AGL :)

More seriously, we only care about EGL / GLES support. We have an EGL/GLES -> Desktop GL

translation layer running on the host, and considering using ANGLE for EGL/GLES -> DirectX

translation on Windows (where unfortunately many OpenGL drivers are seriously buggy and crashy).

Can you tell us if virglrenderer supports running on top of EGL/GLES?

 

Thanks

- Digit

--
Marc-André Lureau

[David Turner]

On Tue, Mar 1, 2016 at 6:33 PM, Chih-Wei Huang <cwh...@android-x86.org> wrote:

Hi David,

Thank you for the reply.

2016-03-01 5:32 GMT+08:00 David Turner <di...@google.com>:
> Upgrading to a newer kernel (we're still on 3.10).

Not a big problem. We have provided the kernel 4.4 android porting.
(based on AOSP's kernel/common android-4.4 branch)

 

> Upgrading to a newer version fo QEMU2 (we're still on 2.2).

Good to upgrade.

I'm happy to say we started working on rebasing our codebase on top of QEMU 2.5

I'll post more details about this soon on the list to discuss the best approach to do this, including upstreaming considerations.

 

> Either port back guest virgil code to all previous Android SDK system images
> (probably a massive task),
> or support both images with virgil and without it (simpler, but probably a
> bit complicated).
> Other considerations regarding ANGLE support on Windows hosts, GPU
> snapshotting, GLES 3.x support, and Vulkan.

I think the most difficult parts are the virglrenderer
only support Linux host now.
Porting it to other platforms that the SDK supports
is a big challenge.

Aaaargh. Does it run on top of EGL/GLES on the host? That would be the best option for us, since we can deal with the rest of portability issues.

 

Anyway, we have integrated the virgl supports
into android-x86 codebse.
(most of the hard works are done by Rob Herring)
For those who are interesting, please see

https://groups.google.com/d/msg/android-x86-devel/RErWaXk3b7g/XTakX5X8EwAJ

Can I ask you to invite me (di...@android.com) to the mailing list? I can't see this post.

 

https://github.com/openthos/openthos/wiki/virtio-gpu

The page mentions that KVM is required for this to work? Is there any reason why this cannot work with non-accelerated emulation? That would be a huge blocker for us :-/

 

> It's likely we're going to take another deep look at Virgil in a distant
> future, since most of the issues above are being worked on, but for now, our
> priorities are different.

That's fine.

Another task we are doing is to enable GLES2 software rendering
via mesa llvmpipe driver. That may also benefit the SDK emualtor.

Do you have any plan to upgrade AOSP's mesa3d
to the latest upstream? (like we have done)

We're interested in that as well. I just sent an email to the system team to query the state of this directory.
My recollection, though dated, is that these sources are not used anymore by any of our builds, so updating these on AOSP shouldn't be a problem at all. I'll let you know when I have more info.

 

> On Thu, Jan 14, 2016 at 1:34 AM, Chih-Wei Huang <cwh...@android-x86.org>
> wrote:
>>
>> Hi David,
>> Glad to see the plan to improve GPU emulation.
>>
>> The Android-x86 group is also working to enable
>> GPU acceleration on QEMU via the Virgil 3D project.
>> (ref: https://virgil3d.github.io/ )
>> The plans include:
>>
>> * kernel 4.4 with virtio-gpu enabled
>> * GLES stack by mesa 11.2-devel (with virgl patches)
>> * AOSP's drm_gralloc & drm_hwcomposer
>> * Qemu 2.5 (all related patches upstreamed)
>>
>> You may check the progress via
>> https://groups.google.com/d/msg/android-x86-devel/RErWaXk3b7g/aLd1Fh-kAQAJ
>>
>> Related code:
>> https://github.com/openthos/
>>
>> How do you think about the plans?


--
Chih-Wei
Android-x86 project
http://www.android-x86.org

[Marc-André Lureau]

Hi

On Tue, Mar 8, 2016 at 8:39 PM, David Turner <di...@android.com> wrote:
> Thanks for participating to the discussion. While I don't know enough about
> virglrenderer
> to assess its portability, I must admit a phrase like "merely needs opengl"
> is a little too
> scary due to past experiences dealing with platform differences, and the joy
> of GLX / WGL / AGL :)

I admit, that was a bit simplistic, just wanted to say quickly it
doesn't rely on much more than an opengl context. It currently
requires desktop opengl 3.1. It is up to caller to implement callback
to create/destroy contexts, or use the provided egl backend. Thus is
has been made to work succesfully with glx and egl. I don't think
anyone tried with WGL/AGL yet, although it can be tried with qemu SDL
virgl backends (I suppose SDL has WGL/AGL support, I know GTK is a bit
behind, WGL being worked on).

> More seriously, we only care about EGL / GLES support. We have an EGL/GLES
> -> Desktop GL
> translation layer running on the host, and considering using ANGLE for
> EGL/GLES -> DirectX
> translation on Windows (where unfortunately many OpenGL drivers are
> seriously buggy and crashy).

That makes sense to workaround buggy drivers on windows. But what's
the solution for other OS (MacOS/Linux etc)? But I suppose you also
have to handle guest side drivers etc. If you run virgl (guest&host),
then you can decide to use ANGLE if the driver is buggy, or the native
GL driver on any OS. (although it may be a lot of translation for the
ANGLE case: GLES->TGSI->GL->DirectX)

> Can you tell us if virglrenderer supports running on top of EGL/GLES?

EGL, yes. GLES no. I am not familiar enough with the differences
between the GL/GLES to say if this would be doable. Dave?

Hmm, I don't understand why you would want to run on top of GLES, you
said you have a GLES->DesktopGL translation already. Sorry if I
misunderstand, I am not familiar enough with the android emulator.

--
Marc-André Lureau

[David Turner]

On Tue, Mar 8, 2016 at 12:19 PM, Marc-André Lureau <marcandr...@gmail.com> wrote:

> More seriously, we only care about EGL / GLES support. We have an EGL/GLES
> -> Desktop GL
> translation layer running on the host, and considering using ANGLE for
> EGL/GLES -> DirectX
> translation on Windows (where unfortunately many OpenGL drivers are
> seriously buggy and crashy).

That makes sense to workaround buggy drivers on windows. But what's
the solution for other OS (MacOS/Linux etc)?

Most of our users are on Windows, and the vast majority of GPU crashes are on Windows too.

Generally speaking, OS X and Linux are much more stable in this regards.

We have crash analytics now, which allows us to better understand the issues, and build a "blacklist" of GPU drivers, depending on the platform. When one is detected, we force software-based rendering.

 

But I suppose you also
have to handle guest side drivers etc. If you run virgl (guest&host),
then you can decide to use ANGLE if the driver is buggy, or the native
GL driver on any OS. (although it may be a lot of translation for the
ANGLE case: GLES->TGSI->GL->DirectX)

I'm not sure I understand what you're describing here. We only have a single set of
guest (Android) drivers to support. All problems we have are in the host-side drivers at this point.

 

> Can you tell us if virglrenderer supports running on top of EGL/GLES?

EGL, yes. GLES no. I am not familiar enough with the differences
between the GL/GLES to say if this would be doable. Dave?

For simplicity, consider GLES to be a strict subset of Desktop GL, but keep in mind that both are _rendering_ APIs, while the APIs to manage graphics surface are platform-specific (GLX/WGL/AGL for desktop GL, and EGL for embedded systems).

Because the guest Android system only relies on GLES/EGL, we pipe all commands through a very simple but very fast wire protocol, and perform the translation of EGL/GLES commands to the native graphics system directly on the host.

There is also the added complexity of "gralloc", which is a very low-level Android system API to manage graphics buffers (including camera ones). It's probably on par with Linux's DRM / KMS APIs, but very different in scope and features. These commands are few but also sent through the wire protocol (and implemented on top of "desktop EGL").

In other words, can virgl at this point be used to send EGL + GLES commands from the guest to the host. I guess so, since TGSI seems to have been designed for this.

[Dave Airlie]

I'm a bit confused on some of this.

so I'll try to clarify.

in the guest virgl provides a mesa GL driver, which can do desktop GL
or GLES2 or 3 etc
(depending on supported features). Mesa abstracts all that stuff from
the guest driver.
So virgl doesn't care about guest GLES or GL.

Virgl isn't like your current solution so I think comparing it is
confusing me, virgl
isn't passthrough, it emulates a graphics card that talks gallium
state packets and TGSI shaders.

Now on the host, we currently run on Desktop GL only. Using epoxy as
the GL API wrapper
library. We talk to the host GL stack via whatever method qemu sets
up. So if using SDL
it should hide the context creation stuff, and the renderer just uses
a shared context.

Now I'm not sure if people are asking if on the host side we could
talk to GLES instead
of OpenGL, so far I haven't considered doing that and I don't see a
need for it, from what I can
see most use cases for virgl are on top of desktop GL systems.

Dave.

[David Turner]

On Tue, Mar 8, 2016 at 4:44 PM, Dave Airlie <air...@gmail.com> wrote:

I'm a bit confused on some of this.

I'm not surprised, these things are quite sophisticated, aren't they ? :)

 

so I'll try to clarify.

in the guest virgl provides a mesa GL driver, which can do desktop GL
or GLES2 or 3 etc
(depending on supported features). Mesa abstracts all that stuff from
the guest driver.
So virgl doesn't care about guest GLES or GL.

Virgl isn't like your current solution so I think comparing it is
confusing me, virgl
isn't passthrough, it emulates a graphics card that talks gallium
state packets and TGSI shaders.

Ah, this helps a lot. IIUC, it is possible to build libEGL.so / libGLES_CM.so and libGLESv2.so guest libraries that all use the virgl backend, which sends said state packets / TGSI shaders to QEMU through a dedicated virtio-gpu channel.

In essence, this is similar to our solution, except that the wire protocol is very different (i.e. lower-level for virgl).

 

Now on the host, we currently run on Desktop GL only. Using epoxy as
the GL API wrapper
library. We talk to the host GL stack via whatever method qemu sets
up. So if using SDL
it should hide the context creation stuff, and the renderer just uses
a shared context.

That's probably where we differ most. We really rely on desktop EGL/GLES APIs only.
We already have the necessary translation to use desktop GL, but we're moving to
DirectX11 for Windows due to reliability issues. We also use GLES directly in our UI to render
3D models.

It looks like epoxy supports GLES though, so modifying the virglrenderer to do the same seems possible. Probably something we will look at in the future.

 

Now I'm not sure if people are asking if on the host side we could
talk to GLES instead
of OpenGL, so far I haven't considered doing that and I don't see a
need for it, from what I can
see most use cases for virgl are on top of desktop GL systems.

Yes, I understand completely, and we're not asking you to do this :)

Thanks a lot for the clarification, this helps a lot.

- Digit

 

Dave.

[Huihong Luo]

Juts for your information, we have posted a gles v2 version of the boot animation code here:

https://github.com/huisinro/BootAnimation-GLESv2

We need to use this code inside our newly released LeapDroid android virtual machine (http://www.leapdroid.com)