Use swiftshader on physical android device

[edward yuki]

Hi list:

Is it possible to use swiftshader to act as a replacement for render libraries for apps on android, without using the vendor provided library? 

I tested the emulator (API 24/25 with swiftshader as GLES/GL library), and it seems apps are running in software rendering mode 

6-14 13:13:57.572  3775  3838 E libEGL  : validate_display:99 error 3008 (EGL_BAD_DISPLAY)

And isHardwareAccelerated returns false.

I know it sound wired, but for some reasons I'm trying to not use the vendor provided library/GPU device for certain apps, and that's why I'm testing swiftshader.

Thanks!

[edward yuki]

Besides, I was also wondering how swiftshader implements eglSwapBuffers on android?

[Alexis Hétu]

SwiftShader currently only targets desktop platforms (Windows, Linux, MacOS). It is indeed used in the Android Emulator, but not on Android, at the moment.

--
You received this message because you are subscribed to the Google Groups "swiftshader" group.
To unsubscribe from this group and stop receiving emails from it, send an email to swiftshader+unsubscribe@googlegroups.com.
To post to this group, send email to swift...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/swiftshader/4f845dff-8a9f-4649-8373-caa265167d95%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[edward yuki]

Hi:

Thanks for your reply. Actually I didn't quite understand this because if it's running in android emulator, it should contains an eglSwapBuffers impl for android.

Since it's used internally in the android emulator, (/vendor/lib/egl/libGL*swiftshader*.so), If I compile the ARM version of swiftshader, it should also work on arm android devices, is it the case?

Or it's still lacking something for the ARM version?

Thanks.

On Wednesday, June 14, 2017 at 11:01:46 PM UTC+8, Alexis Hétu wrote:

SwiftShader currently only targets desktop platforms (Windows, Linux, MacOS). It is indeed used in the Android Emulator, but not on Android, at the moment.

2017-06-14 10:53 GMT-04:00 edward yuki <edward...@gmail.com>:

Besides, I was also wondering how swiftshader implements eglSwapBuffers on android?

On Wednesday, June 14, 2017 at 10:27:59 PM UTC+8, edward yuki wrote:

Hi list:

Is it possible to use swiftshader to act as a replacement for render libraries for apps on android, without using the vendor provided library? 

I tested the emulator (API 24/25 with swiftshader as GLES/GL library), and it seems apps are running in software rendering mode 

6-14 13:13:57.572  3775  3838 E libEGL  : validate_display:99 error 3008 (EGL_BAD_DISPLAY)

And isHardwareAccelerated returns false.

I know it sound wired, but for some reasons I'm trying to not use the vendor provided library/GPU device for certain apps, and that's why I'm testing swiftshader.

Thanks!

--
You received this message because you are subscribed to the Google Groups "swiftshader" group.

To unsubscribe from this group and stop receiving emails from it, send an email to swiftshader...@googlegroups.com.

To post to this group, send email to swift...@googlegroups.com.

[edward yuki]

Maybe i can describe my question as, what's the difference for using swiftshader in emulator and on real android?

[Alexis Hétu]

2017-06-14 11:58 GMT-04:00 edward yuki <edward...@gmail.com>:

Hi:

Thanks for your reply. Actually I didn't quite understand this because if it's running in android emulator, it should contains an eglSwapBuffers impl for android.

I haven't worked on this personally, but I think there's an abstraction layer between the emulated Android app and the SwiftShader renderer.

 

Since it's used internally in the android emulator, (/vendor/lib/egl/libGL*swiftshader*.so), If I compile the ARM version of swiftshader, it should also work on arm android devices, is it the case?

ARM support is a work in progress. It's not complete yet and the emulator uses the regular x86 version of SwiftShader, AFAIK.

 

Or it's still lacking something for the ARM version?

Thanks.

On Wednesday, June 14, 2017 at 11:01:46 PM UTC+8, Alexis Hétu wrote:

SwiftShader currently only targets desktop platforms (Windows, Linux, MacOS). It is indeed used in the Android Emulator, but not on Android, at the moment.

2017-06-14 10:53 GMT-04:00 edward yuki <edward...@gmail.com>:

Besides, I was also wondering how swiftshader implements eglSwapBuffers on android?

On Wednesday, June 14, 2017 at 10:27:59 PM UTC+8, edward yuki wrote:

Hi list:

Is it possible to use swiftshader to act as a replacement for render libraries for apps on android, without using the vendor provided library? 

I tested the emulator (API 24/25 with swiftshader as GLES/GL library), and it seems apps are running in software rendering mode 

6-14 13:13:57.572  3775  3838 E libEGL  : validate_display:99 error 3008 (EGL_BAD_DISPLAY)

And isHardwareAccelerated returns false.

I know it sound wired, but for some reasons I'm trying to not use the vendor provided library/GPU device for certain apps, and that's why I'm testing swiftshader.

Thanks!

--
You received this message because you are subscribed to the Google Groups "swiftshader" group.
To unsubscribe from this group and stop receiving emails from it, send an email to swiftshader...@googlegroups.com.
To post to this group, send email to swift...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/swiftshader/4f845dff-8a9f-4649-8373-caa265167d95%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "swiftshader" group.

To unsubscribe from this group and stop receiving emails from it, send an email to swiftshader+unsubscribe@googlegroups.com.

To post to this group, send email to swift...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/swiftshader/d96c46d4-d9bd-4459-91db-0f968794a6c6%40googlegroups.com.

[edward yuki]

Thanks. What work is currently ongoing on the ARM side ? I'm happy to contribute to make it fully support ARM, if I may.

[Nicolas Capens]

Hi Edward,

I'm working on ARM support for Android Things devices that don't have an (adequate) GPU. A work-in-progress patch that enabled us to boot such a device is here: https://swiftshader-review.googlesource.com/9588. It's still very slow due to not yet using NEON instructions everywhere we should, but that work is underway.

That said, Android has a very tight integration between the EGL implementation, the system's framebuffer, and the 'gralloc' module. You can read about the architecture here: https://source.android.com/devices/graphics/. Essentially it means that the Android Emulator has SwiftShader-specific changes in some of this other system-level code. Note that this is quite typical for a Linux-based operating system. Unfortunately this means that it's not quite possible to take an OpenGL ES and EGL implementation built for one system, and run it on another.

However, SwiftShader does support "headless" rendering. You can create an EGL pbuffer surface, render to it, and read the result back with glReadPixels() without using any system-specific graphics buffers. We don't have any proof of concept of this on Android though, especially on ARM since that's still under active development and will take a while to finish.

I hope this helps.

Cheers,

Nicolas

To unsubscribe from this group and stop receiving emails from it, send an email to swiftshader+unsubscribe@googlegroups.com.

To post to this group, send email to swift...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/swiftshader/2a3c9de5-8cc9-43de-8ddf-c96f53fc69e7%40googlegroups.com.

[edward yuki]

OK, sorry for my dumb mistake. It's a path error.

Sorry again, thanks for this great project!

On Thursday, June 15, 2017 at 11:04:47 PM UTC+8, edward yuki wrote:

Hi Nicolas:

However, SwiftShader does support "headless" rendering. You can create an EGL pbuffer surface, render to it, and read the result back with glReadPixels() without using any system-specific graphics buffers. We don't have any proof of concept of this on Android though, especially on ARM since that's still under active development and will take a while to finish.

I just tried render with eglCreatePbufferSurface and do not render on system graphics buffer, but the pixels read back using glReadPixels are all zero. The same code works fine with adreno EGL/GLES library.

Main code is:

 

  EGLDisplay display = eglGetDisplay(EGL_DEFAULT_DISPLAY);
    eglInitialize(display, 0, 0);
    eglChooseConfig(display, attribs, &config, 1, &numConfigs);
    surface = eglCreatePbufferSurface(display, config, surfaceAttr);
    //surface = eglCreateWindowSurface(display, config, s.get(), NULL);
    context = eglCreateContext(display, config, NULL, NULL);
    eglQuerySurface(display, surface, EGL_WIDTH, &w);
    eglQuerySurface(display, surface, EGL_HEIGHT, &h);
    if (eglMakeCurrent(display, surface, surface, context) == EGL_FALSE)
        return NO_INIT;

and

   char *buf = new char[mWidth*mHeight*4];
    printf("mwidth %d height%d\n", mWidth, mHeight);
    glViewport(0, 0, mWidth, mHeight);
     eglSwapBuffers(mDisplay, mSurface);
 
    {
    
        glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
        glClearColor(0.5, 0.5, 0.5, 0);
        glFlush();
        EGLBoolean res = eglSwapBuffers(mDisplay, mSurface);
        glReadPixels(0,0,mWidth,mHeight,GL_RGBA, GL_UNSIGNED_BYTE, buf);
        FILE* f = fopen("/data/local/tmp/1.raw", "w");
        fwrite(buf, mWidth*mHeight*4, 1, f);
        fclose(f);
    } 

 I've also attached the source.

Maybe I'm missing something on arm version? Seems the rendering result is totally unwritten to buffer, even with offline surface.

Thanks! Wishing to hear back from you.

On Thursday, June 15, 2017 at 8:34:16 PM UTC+8, edward yuki wrote:

The drawing related statements in my sample are only these simple ones :

bool BootAnimation::android()

{

//...

    glViewport(0, 0, mWidth, mHeight);

     eglSwapBuffers(mDisplay, mSurface);

//...

    while(1){

    

        glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

        glClearColor(0.5, 0.5, 0.5, 0);

        EGLBoolean res = eglSwapBuffers(mDisplay, mSurface);

    } 

    return false;

}

On Thursday, June 15, 2017 at 8:29:00 PM UTC+8, edward yuki wrote:

HI Nicolas:

Thanks for your reply, I really appreciate it.

Actually I did not try to replace the physical device's EGL/GL library(e.g. adreno on Nexus6P), I was trying to use swiftshader as a per-process replacement, given the single process itself does not have access to GPU, but still want to draw use OpenGL

The rendering target is still ANaitveWindow(or, GraphicBuffer) handled back by SurfaceFlinger (actually on android I think normally only surfaceflinger will call gralloc functions, and apps & normal processes get back GraphicBuffer from SurfaceFlinger. Surfaceflinger still uses default vendor provided/hardware backed render library, only the single client process  use SwiftShader). 

For example, a typical render process on Android

ANativeWindow* window = ANativeWindow_fromSurface(env, javaSurface);

ANativeWindow_Buffer buffer;
if (ANativeWindow_lock(window, &buffer, NULL) == 0) {
  //DRAW: draw pixel on buffer 
  ANativeWindow_unlockAndPost(window);
}

After applying the patch you mentioned (https://swiftshader-review.googlesource.com/9588) and compiled an ARM version of swiftshader, I crafted a quick demo which requests Surface from surfaceflinger on nexus6p and use EGL/GL to draw on it. It's similar to android's bootanimation binary. The binary runs fine with stock libraries, However I replaces stock EGL/GLES library with SwiftShader's and it seems the drawing commands are not in effect. The surface stays black.

I didn't see any error log though, seems the EGL functions are working correctly.

I have minimized the OpenGL functions used to simple glClear and glClearColor and no shader source is involved. But the surface still stays black.

Is this an intended behavior or I'm just missing something? In my understanding, since the surface/GraphicBuffer has already been successfully requested, SwiftShader should be able to draw on the buffer just like the normal Android software renderer using Skia, what's the lacking piece?

I've attached my test demo source/binary. I changed the loading path of libGLESv1_CM_swiftshader and use LD_LIBRARY_PATH to load swiftshader libraries instead of system ones. My command line is like:

//copy swiftshader arm libraries to /data/local/tmp/

//cd /data/local/tmp/

//LD_LIBRARY_PATH=. ./animtest

Also attached the debug log. animtest.tar.gz contains the test binary's source and compiled binary tested on Android 6.0.1. 

I would be grateful if you could shed some light on this. Thanks!

[Cheng-Wei Lee]

Hi Edward,

I am going to build an ARM version of swiftshader, and use it as software GLES implement on android device.

However, I can not compile an ARM version of swiftshader successfully. There is always a lack of file or library.

As you mentioned, you could compile an ARM version of swiftshader. Could you share your details of compiling procedure with me?

Thank you

edward yuki於 2017年6月16日星期五 UTC+8下午5時33分44秒寫道：

[edward yuki]

Hi Cheng-Wei:

I did some hacks mainly in build files to let it compile.

1. add local_32_bit_only and LOCAL_MODULE_TARGET_ARCH=arm to force arm build in all Android.mk files. You can write a quick python script to do that

2. modify src/Android.mk to force use_subzero=true

After these steps you should be able to build swiftshader in an AOSP build environment. 

[Cheng-Wei Lee]

Hi Edward,

Thank you for your reply

about hack 1, I have some questions

1. I should add LOCAL_32_BIT_ONLY=true and  LOCAL_MODULE_TARGET_ARCH=arm in all Android.mk, should I?

2. Which Android.mk is your starting(compiling) point?

3. I only need the OpenGL libraries of Swiftshader in ARM version. Should I Build all the files in Swiftshader? or I can skip some files like files in third_party.

edward yuki於 2017年7月18日星期二 UTC+8下午2時34分19秒寫道：

[edward yuki]

Hi Cheng-Wei:

1. Yes. I know there should be other better ways to force a 32bit ARM build in AOSP env, but currently this will work.

2. Just start with the root directory.  put the source folder, make changes on .mk files, run `mm` under $AOSP_ROOT_SRC/external/swiftshader, and grab the outcome dynamic libraries under out/target/product/{}/system/lib/

[Cheng-Wei Lee]

Hi Edward,

Thank you for your nice guide. it really helps.

I add LOCAL_32_BIT_ONLY=true and LOCAL_MODULE_TARGET_ARCH=arm at the front of every module (after include $(CLEAR_VARS)), and It seems to build successfully.

When I run "mm" under the root of swiftshader, there is only one step to run.

So I compile /src and /third_party separately. Both of compiling procedures are successful.

Is it okay? of it will impact the execution of swiftshader. 

My outcome dynamic libraries are under out/target/product/{}/system/vendor/lib(lib64)/egl

After building swiftshader successfully, have you tried to run swiftshader on physical android device?

edward yuki於 2017年7月24日星期一 UTC+8下午4時16分53秒寫道：

[edward yuki]

Hi Cheng-Wei:

Well I previously just called mm in the root directory.

xxx@xxx-desktop:/mnt/hdd/aosp/external/SwiftShader$ mm -B

make: Entering directory '/mnt/hdd/aosp'

============================================

PLATFORM_VERSION_CODENAME=REL

PLATFORM_VERSION=6.0.1

TARGET_PRODUCT=aosp_angler

TARGET_BUILD_VARIANT=userdebug

TARGET_BUILD_TYPE=release

TARGET_BUILD_APPS=

TARGET_ARCH=arm64

TARGET_ARCH_VARIANT=armv8-a

TARGET_CPU_VARIANT=cortex-a53

TARGET_2ND_ARCH=arm

TARGET_2ND_ARCH_VARIANT=armv7-a-neon

TARGET_2ND_CPU_VARIANT=cortex-a7

HOST_ARCH=x86_64

HOST_OS=linux

HOST_OS_EXTRA=Linux-4.4.0-36-generic-x86_64-with-Ubuntu-16.04-xenial

HOST_BUILD_TYPE=release

BUILD_ID=MTC20F

OUT_DIR=out

============================================

PRODUCT_COPY_FILES device/generic/goldfish/data/etc/apns-conf.xml:system/etc/apns-conf.xml ignored.

No private recovery resources for TARGET_DEVICE angler

Import includes file: out/target/product/angler/obj_arm/STATIC_LIBRARIES/swiftshader_top_release_intermediates/import_includes

target thumb C++: swiftshader_top_release_32 <= external/SwiftShader/src/Common/CPUID.cpp

target thumb C++: swiftshader_top_release_32 <= external/SwiftShader/src/Common/Configurator.cpp

...

However since you successfully produced the binary files, then I think it should be OK.

Yes I've tried the 32bit version (64bit is not working because lack of subzero support) on physical android. It does work even on complex UI like webview, but since the lack of neon instructions, the speed is quite slow, compared to swiftshader on x86 emulator with SSE optimization on and the native software renderer built-in on android.

[Cheng-Wei Lee]

Hi Edward,

Thank you for sharing information!

Performance is not a big problem to me.

My physical device does not have GPU. That is why I need swiftshader as the implement of OpenGLES 2.0 on Android in order to render without GPU support.

Did you have the same condition like mine?

edward yuki於 2017年7月25日星期二 UTC+8下午9時22分16秒寫道：

[Nicolas Capens]

Hi folks,

I've landed all the changes for ARMv7 support in SwiftShader's master branch now, so you no longer need to apply the work-in-progress patch. It still uses 'emulation' of many vector operations which could be optimized with NEON instructions, but we've taken the first steps towards improving performance.

I'm not sure I understand the need for you to set LOCAL_32_BIT_ONLY. It looks like your 'lunch' setting is for 64-bit ARMv8 as primary arch, and 32-bit as secondary. Could you set 32-bit as primary? Or is this for a 64-bit device with 32-bit user space?

Cheers,

Nicolas

To unsubscribe from this group and stop receiving emails from it, send an email to swiftshader+unsubscribe@googlegroups.com.

To post to this group, send email to swift...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/swiftshader/e9650a15-ab70-404b-abf2-0cf61fbe9e2b%40googlegroups.com.

[edward yuki]

Hi Nicolas:

Great work as always :)

Yes I was building on a 64bit device with 32bit userspace support, which is the status for almost all android devices nowadays.

[saga]

hi Nicolas~  i got an error when i use swiftshader on physical android device.  i find out that it is because the function TexImage2D in "libGLESv2.cpp"  return error when the parameter "internalformat != (GLint)format"。how could i solve this problem?  thanks

在 2017年6月15日星期四 UTC+8上午1:57:38，Nicolas Capens写道：

[Nicolas Capens]

Hi saga,

The 'format' and 'internalformat' parameters must match on OpenGL ES 2.0 contexts, as per the spec: https://www.khronos.org/registry/OpenGL-Refpages/es2.0/xhtml/glTexImage2D.xml

If you have a scenario where this shouldn't be the case and SwiftShader validates it incorrectly, please file a bug at https://g.co/swiftshaderbugs

Thanks,

Nicolas

To view this discussion on the web visit https://groups.google.com/d/msgid/swiftshader/4b5f6234-75b2-4d82-bf62-08531a17d22a%40googlegroups.com.