Inquiry: chances for non-blocking shouldInterceptRequest

[Pavel Zdeněk]

Dear group,

For reasons too complicated to explain here, i would like to have a
result of WebViewClient.shouldInterceptRequest be based on a result of
Javascript call in another WebView instance. As shouldInterceptRequest
is being invoked on a non-Main thread, while
WebView.evaluateJavascript is expected to be called on Main, i
actually tried it. Miracles don't happen: the shouldInterceptRequest
IOThread gets as far as enqueing a main thread task and waiting itself
on a condition. Main thread keeps running after evaluateJavascript
execution, but its ValueCallback never hits. Chrome Inspection of the
"another" WebView instance shows it frozen, expectedly deadlocked on
IOThread somewhere inside WebKit.

My skimming of the Chromium sources shows that
shouldInterceptRequest starts its life as an asynchronous IOThread
task, so my question is: how insane it would be to try propagating
that asynchronicity through the JNI interface, so that it won't block
IOThread on return value?

FYI, the Android PoC is here
https://github.com/pavel-zdenek/android-testbrowser
which works out of the box because i shortcutted the JS eval. It will
deadlock on uncommenting
https://github.com/pavel-zdenek/android-testbrowser/blob/master/app/src/main/java/my/mybrowser/background/ScriptContainer.java#L111
(and removing the following line)

P.S. While fulltext searching this group for relevant terms, i found an intriguing pointer to ServiceWorker. How suitable is that for the intended usage pattern? Does it intercept every single request from the given page? Is it reasonable to route all requests out of the JS context via injected Object interface?

Best regards,

Pavel Zdenek

[Mikhail Naganov]

Hi Pavel,

You can achieve your goal by returning from shouldInterceptRequest an instance of your own implementation of WebResourceResponse. Think of WebResourceResponse as your "promise" to WebView to return web contents later. WebView will be calling `getData` of WebResourceResponse at the point when it actually needs the data -- this will be happening on a pooled thread, thus blocking there will not affect loading of other resources.

The only downside of this approach is that since you will be returning your implementation of WebResourceResponse for every request, you will have to implement fetching of web contents yourself. On the plus side, you are also getting an opportunity to modify the loaded content any way you like before passing to WebView. 

See this code for an example of a custom implementation of WebResourceResponse:

https://github.com/mobilyzer/Mobilyzer/blob/master/Mobilyzer/src/com/mobilyzer/util/AndroidWebView.java 

As for Service Worker, we are only in the process of adding support for it into WebView.

--
You received this message because you are subscribed to the Google Groups "android-webview-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-webview...@chromium.org.
To post to this group, send email to android-w...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/android-webview-dev/55df3e0b-0e83-437b-89bb-e29e4124a6a1%40chromium.org.

[Pavel Zdeněk]

Hi Mikhail,

thanks for your suggestion. Unfortunately it won't work for me. The "promise" is not the WebResourceResponse but the wrapped InputStream. WebResourceResponse is resolved immediately - AwWebResourceResponse is made from it, expecting headers and http code and mime type and all be ready.

https://chromium.googlesource.com/chromium/src.git/+/master/android_webview/glue/java/src/com/android/webview/chromium/WebViewContentsClientAdapter.java#315

On the other hand, InputStream poses as a promise indeed - not resolved until bubbled all the way through to native code async task RunShouldInterceptRequest.

So the first thing called on the WebResourceResponse reimplementation, immediately after returning shouldInterceptRequest, is getResponseHeaders. Which Mobilyzer does not reimplement in MyWebResourceResponse, so it supposedly returns null. It may work for Mobilyzer, but it won't work for generic web traffic interception. At the moment of getResponseHeaders call, i already need to know the decision of the other WebView JS code, and know the actual server response in case that JS code decides that it's ok. getData() could be deffered thanks to the nature of InputStream, but i need a full response, not just the body (which is what Mobilyzer code returns).

The only chance for me is to propagate the asynchronicity of RunShouldInterceptRequest task all the way through JNI. Put another way, i am trying to get on par with iOS NSURLProtocol, which is fully asynchronous traffic interceptor. So how insane is that, please? :-)

Cheers,

Pavel

Dne pondělí 8. února 2016 18:16:39 UTC+1 Mikhail Naganov napsal(a):

[Mikhail Naganov]

Yes, but getResponseHeaders should already be called on the other thread (from the pool of threads), where you can block as long as you wish to. The reason why you can't block inside shouldInterceptRequest is because it is called on the IO thread, which is used for all interactions in Chrome.

What you can do in order to speed things up is besides returning your customized WebResourceResponse from shouldInterceptRequest, post an async task that will start gathering data for making a decision. Then at the moment getResponseHeaders gets called, you may already have the answer, or you can just wait there until you get it. Basically, this is how network loading in Chrome works -- the request jobs are created on the IO thread, then the actual loading is happening on pooled threads where jobs can spend their time waiting for the server to respond.

[Pavel Zdeněk]

Thanks for your ongoing interest. Tell me one thing: do you know whether the Mobilyzer project needs the real actual deferred response headers? Because i’m damn sure it can’t have it. Chromium up to v37 (API 22/Android 5.1) is DEEP COPYING responseHeaders map right in the AwWebResourceResponse constructor, i.e. at the moment of shouldInterceptRequest return, still on IOThread.

https://chromium.googlesource.com/chromium/src.git/+/37.0.2062.96/android_webview/java/src/org/chromium/android_webview/AwWebResourceResponse.java#43

getResponseHeaderNames/Values is indeed called later on a worker thread, but it is called on this copied internal AwWebResourceResponse, not the WebViewClient created WebResourceResponse. The Map which i feed to WebResourceResponse is not queried anymore.

Yes the deep copy was lazied (moved to header-specific getters) in the latest Chromium head. But the worker thread still queries an internal copy of the response, not the original client response. I can subclass data InputStream. I can subclass the header Map - which i did to confirm the above finding. But i can’t subclass String. Even if i could, there is still statusCode primitive int. No chance to influence it after AwWebResourceResponse constructor, which happens immediately on IOThread after shouldInterceptRequest return.

It would be so great if i just overlooked something. But the immediate creation of AwWebResourceResponse copy looks like a show stopper to me. I updated the sources with WebResourceResponse subclass

https://github.com/pavel-zdenek/android-testbrowser

The only function called from a worker thread is FutureResponse$FutureInputStream.read. FutureMap and the primary getters are called only immediately on IOThread and never later.

[Mikhail Naganov]

Pavel, thanks for your persistence! I indeed was wrong -- both getResponseHeaders and getData of WebResourceResponse are called on the UI thread, thus blocking there hurts loading speed and can lead to deadlock. It's only the InputStream's read method that is called on a pooled worker thread. I don't actually know how Mobilyzer works or even what it does, I just found it to be an example of providing your own implementation of WebResourceRequest.

Let's return to our original problem though. You need to call `WebView.evaluateJavaScript` (of another WebView) while serving a network request, and the difficulty is that most of the action happens on the IO thread where you can't block waiting on something that happens inside WebView.

What I would do in this case is trying to eliminate the dependency on the second WebView. If your logic is in JS, and this is a strong requirement, try running a JS VM that's not from WebView, e.g. Rhino (http://lifeofcoding.com/2015/04/05/Execute-JavaScript-in-Android-without-WebView/), or a standalone V8 (that would require you to use NDK though). I know, sounds heavy, but I don't have any better idea for now, sorry. 

[Pavel Zdeněk]

Hi Mikhail,

that worked perfectly. Rhino is somewhat abandoned and reportedly slow, so i was looking around for unobtrusive ways of integrating V8 and found awesome https://github.com/eclipsesource/J2V8. So your wisdom was definitely pointing in the right direction. Now i’m down to a strange uncooperativeness of shouldInterceptRequest when http redirection comes to play, but i will be writing another inquiry on that.

Thanks for the assistance,

Pavel