resize a layer in surfaceflinger

[Tez]

How do I resize a layer in the surface flinger?

Basically, I want to simulate smaller screen sizes for certain layers.

I am looking at the glScalef function but I'm not sure where I can apply it.

I had come across Jeff sharkey;s post on setting tints for layers to save power but the file he modified (LayerBase.cpp)

does not seem to be available in android 4.4.2. 

The function he modified was called drawWithOpenGL but now it only exists in Layer.cpp and I see calls to something

called RenderEngine.

Can anyone help me out?

[Tez]

So I managed to resize ALL layers (thereby artificially shrinking the size of the entire phone's display)

by using a glScalef call to the drawWithOpenGL function inside Layer.

I verified that the code works on the emulator (I see a shrunken version of the entire display) but when I compile for my

target device and run it, the phone boots fine and is perfectly normal but I don't see any of the changes I saw when I

booted on the emulator.

I'm using 4.4.2 and a nexus 4. Any ideas why?

The code I added was to the RenderEngine. Its a method that contains the following lines:

void RenderEngine::compressLayer()

{   

    glMatrixMode(GL_MODELVIEW);

    glLoadIdentity();

    glScalef(1.0f, 0.80f, 1.0f);

}

I then call it from within Layer->drawWithOpenGL.

right after the call to engine.disableBlending(); (the last line of code in that function).

[Tez]

after inspecting logs on the device,

I noticed 3 lines 

I/libEGL(74)    Called unimplemented OpenGL API

Now how is that possible? How can something work on the emu but not on a real device??

[Tez]

So it seems that android 4.4 uses openGL ES 2.0 in the surface flinger.

I looked thru the GLES2/gl2.h functions and there's no glScalef function any more.

How do I achieve the same effect now?

-

[Tez]

so upon further study, it seems that gles 2.0 only uses shaders and the other glScale kind functions are disabled.

so I have to modify one of the vertex shaders that exist in the surface flinger to acheve the same effect.

[Tez]

more updates. I've gotten something to work. But it flickers. I see a scaled image on the phone display (basically entire phone UI is shrunk down to 80% of original)

but its intermittent. I see the shrunk version for a time, and then the normal scale (1.0f) version for sometime. And this goes back and forth.

I have modified the vertex shader as Follows (ProgramCache.cpp)

String8 ProgramCache::generateVertexShader(const Key& needs) {

    Formatter vs;

    if (needs.isTexturing()) {

        vs  << "attribute vec4 texCoords;"

            << "varying vec2 outTexCoords;";

    }

    vs << "attribute vec4 position;"

       << "uniform mat4 projection;"

       << "uniform mat4 texture;"

       << "uniform mat4 modelView;"

       << "void main(void) {" << indent

       << "gl_Position = projection * modelView * position;";

    if (needs.isTexturing()) {

        vs << "outTexCoords = (texture * texCoords).st;";

    }

    vs << dedent << "}";

    return vs.getString();

}

so basically I have created a new 4x4 modelview matrix.

I have also create the setter function similar to the one for the existing projection matrix.

In the rendering cycle, just before engine.drawMesh, I call a function to set the modelView uniform.

Towards the end of Layer::drawWithOpenGL (also in clearWithOpenGL)

RenderEngine& engine(mFlinger->getRenderEngine());

    engine.compressLayer(); // <------------------------------------- my addition, basically sets the modelView matrix (in mState which seems to be further updated in a call from ProgramCache->setUniforms())

    engine.setupLayerBlending(mPremultipliedAlpha, isOpaque(), s.alpha);

    engine.drawMesh(mMesh);

    engine.disableBlending();

Can anyone tell me why it is intermittently (seemingly random) scaling down and scaling back to normal?

[Tez]

Since my device is a nexus 4 running android 4.4.2, The surfaceflinger and hence I am using Open GLES 2.0.

All the other EGL library versions are the same for a standard system of that level.

here is a dump of my surfaceflinger for completeness.

I/SurfaceFlinger(  174): EGL informations:

I/SurfaceFlinger(  174): vendor    : Android

I/SurfaceFlinger(  174): version   : 1.4 Android META-EGL

I/SurfaceFlinger(  174): extensions: EGL_KHR_get_all_proc_addresses EGL_ANDROID_presentation_time EGL_KHR_image EGL_KHR_image_base EGL_KHR_lock_surface EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_gl_renderbuffer_image EGL_KHR_fence_sync EGL_KHR_create_context EGL_EXT_create_context_robustness EGL_ANDROID_image_native_buffer EGL_KHR_wait_sync EGL_ANDROID_recordable 

I/SurfaceFlinger(  174): Client API: OpenGL_ES

I/SurfaceFlinger(  174): EGLSurface: 8-8-8-8, config=0x5

I/SurfaceFlinger(  174): OpenGL ES informations:

I/SurfaceFlinger(  174): vendor    : Qualcomm

I/SurfaceFlinger(  174): renderer  : Adreno (TM) 320

I/SurfaceFlinger(  174): version   : OpenGL ES 3.0 V@53.0 AU@  (CL@)

I/SurfaceFlinger(  174): extensions: GL_AMD_compressed_ATC_texture GL_AMD_performance_monitor GL_AMD_program_binary_Z400 GL_EXT_debug_label GL_EXT_debug_marker GL_EXT_discard_framebuffer GL_EXT_robustness GL_EXT_texture_format_BGRA8888 GL_EXT_texture_type_2_10_10_10_REV GL_NV_fence GL_OES_compressed_ETC1_RGB8_texture GL_OES_depth_texture GL_OES_depth24 GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_element_index_uint GL_OES_fbo_render_mipmap GL_OES_fragment_precision_high GL_OES_get_program_binary GL_OES_packed_depth_stencil GL_OES_depth_texture_cube_map GL_OES_rgb8_rgba8 GL_OES_standard_derivatives GL_OES_texture_3D GL_OES_texture_float GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_OES_texture_npot GL_OES_vertex_half_float GL_OES_vertex_type_10_10_10_2 GL_OES_vertex_array_object GL_QCOM_alpha_test GL_QCOM_binning_control GL_QCOM_driver_control GL_QCOM_perfmon_global_mode GL_QCOM_extended_get GL_QCOM_extended_get2 GL_QCOM_tiled_rendering GL_QCOM_writeonly_rendering GL_EXT_sRGB GL_EXT_color_buffer_float GL_EXT_colo

-Earlence

[Hak Matsuda]

Hello Tez,

You could take a look into drawScaled in LayerOrientationAnimRotate, https://android.googlesource.com/platform/frameworks/native/+/076b1cc3a9b90aa5b381a1ed268ca0b548444c9b/libs/surfaceflinger/LayerOrientationAnimRotate.cpp

Hope that helps.

Thanks!
hak

[Tez]

Thanks Hak!

The code you pointed me to seems to link to an earlier version of Android, but I figured that this code exists in some form in the current version.

Turns out that the Transform class does scaling.

So in SurfaceFlinger::doComposeSurfaces, after the clipped region was computed,

I created a new Transform object and did:

Transform& tr2(const_cast<Transform&>(tr));

            tr2.set(1, 0, 0, 0.95f);

        const Region clip(tr2.transform(layer->visibleRegion));

This transforms all layers on the screen into scaled versions.

So there's no need to fiddle with the vertex shader.

I am scaling by a small amount (95% of original) and I am now noticing that the input events still follow the old co-ord system.

Is there a way for me to scale the input events accordingly as well?

-Earlence

[Andy McFadden]

I take it "adb shell wm size 1024x768" doesn't do what you want (or isn't available on your version of the OS)?

[Tez]

I think the wm size command changes the entire system's display size. I want to do it more selectively, on a per app basis.

[Daniel Doron]

Hi Tez,

Can you please share the exact code changes you made in SurfaceFlinger::doComposeSurfaces()? You say you wanted to make it more selectively, as in per app basis. how was that achieved ?

Daniel. 

[balazs...@accedo.tv]

Hi Tez,

Are u able to share ur project with us?
I 'm just wondering that u can do that, Im not really familiar with android NDK and surfaceflinger usage, but as I search in google there is nothing to find.

No exaple project or help page or any tutorial so it will be very very cool if u can share ur stuff to check it and try it on real device. 

Thanks,

Balázs