Guarantees of SurfaceProcessor callbacks.

[Anton Ivanov]

Hi guys,

I'm implementing camera blur effect using CameraEffect API.

The current problem is that on some devices surfaceOutput.getSurface 's callback which should notify about end of life of SurfaceOutput gets called when the app gets hidden before next onFrameAvailable is called and on other devices this order gets violated and onFrameAvailable gets called before surfaceOutput.getSurface 's callback. This results in further OpenGl calls like glUniformMatrix4fv return errors(0x505). Is it an expected behaviour? What are the guarantees of those callbacks?

Thanks,

Anton.

[Xi Zhang (张熹)]

Hi Anton,

In theory, CameraX only releases the output Surface until you call SurfaceOutput#close. So if you are drawing with OpenGL, you should wait until after the drawing is done before calling SurfaceOutput#close. One way to implement this is posting all the related operations to a single thread so they won't be triggered concurrently.

Please let us know if this doesn't fix the issue for you because in practice, it's possible for certain output Surface releases itself before CameraX can do anything. e.g. if the output is connected to a SurfaceView. It's a possibility but I haven't run into this issue yet.

Thank you,

Xi

CONFIDENTIAL

This  e-mail  and  any  attachments  are  confidential  and  intended  solely  for  the  addressee  and  may  also  be privileged or exempt from disclosure under applicable law. If you are not the addressee, or have received this e-mail in error, please notify the sender immediately, delete it from your system and do not copy, disclose or otherwise act upon any part of this e-mail or its attachments. Internet communications are not guaranteed to be secure or virus-free. Revolut does not accept responsibility for any loss arising from unauthorised access to, or interference with, any Internet communications by any third party, or from the transmission of any viruses. Replies to this e-mail may be monitored by Revolut for operational or business reasons. Any  opinion  or  other  information  in  this  e-mail  or  its  attachments  that  does  not  relate  to  the  business  of Revolut is personal to the sender and is not given or endorsed by Revolut. Registered  Office: 7  Westferry  Circus,  Canary  Wharf, London,  E14  4HD,  United  Kingdom.  Main  Office: 7 Westferry  Circus,  Canary  Wharf, London,  E14  4HD, United  Kingdom. Revolut  Ltd  is  authorised  and regulated by the Financial Conduct Authority under the Electronic Money Regulations 2011, Firm Reference 900562.

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.
To view this discussion visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/578d9647-2df8-4b1a-a3ba-8de7fac63be1n%40android.com.

[Anton Ivanov]

Hi Xi,

"One way to implement this is posting all the related operations to a single thread so they won't be triggered concurrently." - that's exactly what I'm doing and I'm getting this problem anyways.
The problem is reproducible in about 90% of cases on certain device. On another device I have it works as expected.

[Anton Ivanov]

Just tested the code on Pixel 8 and it seems that SurfaceOutput.getSurface  callback not getting called on the phone.

[Xi Zhang (张熹)]

If you simply hide the foreground activity, the SurfaceOutput.getSurface callback  won't get triggered since the Surface is not released. I tested the scenario with our testing app but couldn't reproduce it. Our source code is here: https://github.com/androidx/androidx/blob/androidx-main/camera/integration-tests/viewtestapp/src/main/java/androidx/camera/integration/view/ToneMappingSurfaceProcessor.kt 

It would be great if you could provide a minimal sample code that can reproduce the issue.

[Anton Ivanov]

Hey Xi,

Sure, please find the files attached(I could not attach zip archive with the project). On Pixel 8 it crashes once the app is getting hidden with:
java.lang.IllegalStateException: glGetUniformLocation: GL error: 0x505
at com.example.testcameraxapp.GLUtils.checkGlErrorOrThrow(GLUtils.kt:21)

On my other device it works without errors and SurfaceOutput.getSurface callback is getting called.

[Xi Zhang (张熹)]

Could you try setting the PreviewView's mode to PreviewView.ImplementationMode.PERFORMANCE and see if it changes anything? Also, does it happen on devices other than Pixel 8?

[Anton Ivanov]

I think that default mode is performance according to 

@NonNull
ImplementationMode mImplementationMode = DEFAULT_IMPL_MODE; 

However I tried both PERFORMANCE and COMPATIBLE.
Results are that in explicitly set PERFORMANCE mode behavior is exactly the same(crash in debug check). In COMPATIBLE mode it seems that everything works fine(no crash), however SurfaceOutput.getSurface not getting called. Id didn't test it on Pixel 8 as I don't have it on me right now(will do tomorrow) but on other devices that I have right now COMPATIBLE mode works fine.
"Also, does it happen on devices other than Pixel 8?" - yes, there are at least two devices we discovered where it happens(Pixel 8 and SM-G390F). On other device(SM-A057G) both COMPATIBLE and PERFORMANCE mode work fine.

[Xi Zhang (张熹)]

Thanks for the feedback. For the time being, you can use COMPATIBLE as a workaround. We will investigate the issue with PERFORMANCE later. It's likely related to the behavior of SurfaceView.

[Anton Ivanov]

Thanks for the advice,

will switch to COMPATIBLE mode for now.