Dead-lock situation in CreateBrowserSync with Linux/Aura

[michael....@gmail.com]

[Moved from private mail to have it publicly archived. Initial mail, then Czarek's responses]

Hi,

I'm working on a projecting consisting of porting CEF to linux/aura and use it from cefpython. After some hacking, I could get a working version of CEF for linux/aura and cefclient works fine. However, when I tried to use it from cefpython I got a dead-lock, mainly when creating a browser window with a startup URL.

After some debugging, I think the scenario is the following:
1) the python app executes cefpython.CreateBrowserSync with a startup URL
2) the startup URL loading triggers the creation of the RenderViewHost, and the start of the renderer process
3) the renderer process sends the synchronous CefProcessHostMsg_GetNewRenderThreadInfo message to the browser process (in CefContentRendererClient::RenderThreadStarted)
4) the IO thread of the browser process receives that message and executes its handler; eventually it executes CefPythonApp::OnRenderProcessThreadCreated and BrowserProcessHandler_OnRenderProcessThreadCreated (the latter being executed in the python interpreter)
5) in the UI thread of the browser process, the initialization of the RenderViewHost also triggers a (synchronous) IPC message to the GPU process (from RenderWidgetHostViewAura::GetCompositingSurface)

At this point, everything is dead-locked:
- the running cefpython.CreateBrowserSync locks the python interpreter (in the UI thread)
- the renderer is waiting for the response to CefProcessHostMsg_GetNewRenderThreadInfo
- the IO thread of the browser is locked, because it waits for the python interpreter
- the UI thread of the browser is locked, because it waits for the IO thread (to send its message to the GPU process)

Do you think this is a plausible explanation?

For the time being, I worked around the problem by not calling BrowserProcessHandler_OnRenderProcessThreadCreated, as it seems it is only used to get debug status to the renderer process. I can live without having debug enable in the renderer.

============================================================

Hi Michael,

I have created Issue 101 [1] with some useful links in regards to Aura on Linux in CEF/Chromium.

Which branch did you use when porting CEF to aura? In issue 101 there is a link to the CEF issue which is to port Aura to CEF, you might share your implementation with Marshall. Also, Marshall will definitely know some more about possible cause of the deadlock.

Yes, the BrowserProcessHandler_OnRenderProcessThreadCreated() callback is currently used only for debugging purposes. I even plan on getting rid of this python call, see Issue 98 [2].

If removing this call works for you then great. But there are more callbacks from various client handlers that are running on the IO thread, so there is some chance that you might get into similar trouble again.

Have you tried calling calling the asynchronous version of the function that creates browser? There is no api for calling it in cefpython, but you can find it in CEF, see the CefBrowser::CreateBrowser() method - without the "Async" postfix. The async version does not return the browser object. But you could obtain that object from client handler callbacks. One of the first callbacks that are executed after browser creation are probably LifeSpanHandler::OnAfterCreated() and V8ContextHandler_OnContextCreated(). When calling CreateBrowser(), I would release the python global interpreter lock (GIL), see the syntax "with nogil" in cefpython.pyx/MessageLoop().

Best regards,
Czarek

[1] https://code.google.com/p/cefpython/issues/detail?id=101
[2] https://code.google.com/p/cefpython/issues/detail?id=98

=============================================================

I have created Issue 102 [1] that is to remove the GIL lock when calling some of the CEF functions, to avoid deadlocks. Revision ec1ce788373b [2] removed the lock when calling CreateBrowserSync().

-Czarek

[1] https://code.google.com/p/cefpython/issues/detail?id=102
[2] https://code.google.com/p/cefpython/source/detail?r=ec1ce788373b

[michael....@gmail.com]

The port is based on CEF at revision 1556 (with chromium at 34.0.1768.0).

I'm not that optimistic that Linux/Aura will make it in M33. Although it was briefly enabled in November, it was disabled again a few days later. AFAIK it's still disabled. Marshall is waiting linux/aura stabilizes before integrating it into CEF [1]. Obviously, I'll share my implementation if he's interested in it.

Thanks for your changes, I'll test them today, and it looks less hacky than disabling BrowserProcessHandler_OnRenderProcessThreadCreated.

Using CefBrowser::CreateBrowser() is another option, but as you said, it's not exposed yet by cefpython, so I didn't experiment with it yet (it was already late last night :)). Getting the browser object would then be done through LifeSpanHandler::OnAfterCreated. I don't think V8ContextHandler_OnContextCreated is a valid candidate as it is only fired when you actually load a URL in the browser. If you don't specify a startup URL, the renderer process will not be created.

Let me know if you are interested in my current patches.

Michael.

[1] http://www.magpcss.org/ceforum/viewtopic.php?f=6&t=11300

[Czarek Tomczak]

Your patches to cefpython could be useful. As I remember from looking at CEF revisions, there were quite a few changes in CEF that will require modifications in cefpython as well. I use CEF stable releases, the latest branch in CEF is 1650 (Chrome 31) that I was planning to upgrade. I assume that you're using trunk. Is CEF trunk using the beta channel (Chrome 32) or the dev channel (Chrome 33)? With your changes (Chrome 34) you're probably a few steps further.

-Czarek

[Czarek Tomczak]

Your patches to the cefpython could be useful. But I don't plan on using an unofficial version of CEF with the Aura support. I think I will wait until it gets to the official CEF release. But changes to cefpython that prepare us for the Aura migration are welcome, as long as this can harmonize with cefpython that is to work with CEF stable branches.

-Czarek

[Michael Goffioul]

I attached the current patch against cefpython. It's mainly about updating CEF headers, using Fedora-x86_64 paths, and a few API changes in CEF. Please note that include/internal/cef_types_linux.h contains changes that are specific to my port of CEF to linux/aura.

I'm using CEF from trunk at revision 1556. Its has been recently updated to chromium 242756, which corresponds to Chrome 34.

Michael.

[Czarek Tomczak]

What was guiding you to create the CEF Aura port? Is it the touch screen support? I don't know too much about Aura.

Thanks for the patch. I see new includes for Fedora in makefiles, will add them. I see that you just pass NULL for the sandbox and CefRequestContext. So this still needs to be implemented when upgrading to the latest CEF. The sandbox is a nice security feature. The CefRequestContext will fix the cookie isolation in different browsers, maybe it will also fix some other known bugs when embedding multiple browsers in tabs, so it's worth implementing.

-Czarek

[Michael Goffioul]

On Wednesday, January 8, 2014 10:35:20 AM UTC-5, Czarek Tomczak wrote:

What was guiding you to create the CEF Aura port? Is it the touch screen support? I don't know too much about Aura.

The reason is indeed to get multitouch support under Linux.

 

Thanks for the patch. I see new includes for Fedora in makefiles, will add them. I see that you just pass NULL for the sandbox and CefRequestContext. So this still needs to be implemented when upgrading to the latest CEF. The sandbox is a nice security feature.

The additional argument for sandbox information is only used on Windows. In fact, the Chromium API doesn't even expose that argument on non-Windows platform (see content/public/app/content_main.h and content/public/app/content_main_runner.h). CEF exposes the argument on all platforms, but only uses it on Windows.

 

The CefRequestContext will fix the cookie isolation in different browsers, maybe it will also fix some other known bugs when embedding multiple browsers in tabs, so it's worth implementing.

Yes, it needs further support in cefpython. But as I didn't use it, I simply passed NULL.

Michael.

[Michael Goffioul]

On Wednesday, January 8, 2014 10:35:20 AM UTC-5, Czarek Tomczak wrote:

Thanks for the patch. I see new includes for Fedora in makefiles, will add them.

You may also want to add the 32 bits paths: /usr/lib/glib-2.0/include and /usr/lib/gtk-2.0/include

A more portable way to handle this would be to use pkg-config.

Michael.

[Czarek Tomczak]

Michael, Was the dead-lock reproducible every time or was it intermittent? I am having some other issue and am wondering if it could also be related to the GIL lock. In my case it occurs occasionally.

[Michael Goffioul]

It was reproducible, but only in a specific scenario: when a startup URL was specified. If no startup URL was specified, then the browser showed up fine. I could then create a menu item in the interface and load a URL from the menu item handler, and there was no dead-lock (the renderer process was only created when loading the first URL in the browser component).