Adding instrumentation to Webview to collect traffic stats using Android's Traffic Stats API

[Selim Gurun]

Hello,

Webview team is frequently asked whether there is a mechanism to measure WebView's data consumption. At present WebView does not expose an API for this purpose.

Android supports a Traffic Stats API for tagging sockets with some arbitrary tag and then collecting statistics for the given tag (the traffic for tags is visible via adb diagnostics commands or via  queryDetailsForUidTag()). When using this, apps set an active tag for the traffic originating from a thread using setThreadStatsTag() and call tagSocket() to tag each socket with the active tag for the current thread. Tags are integers and some range is reserved for platform use (for example for DownloadManager). I am thinking WebView could either use a number from this range or could provide a way to applications to set their own tag.

This API above is in Java but Android team seems to be willing to provide an NDK API and they opened b/29761778 to track this. We want to understand what Chrome network stack team thinks about it. Specifically, is the current design for Traffic stats reasonable for chrome network stack use or do you want to see any changes? One option that was recommended was changing tagSocket() to always require a tag.

thanks,

-Selim

[Ryan Sleevi]

On Tue, Jul 12, 2016 at 3:55 PM, 'Selim Gurun' via net-dev <net...@chromium.org> wrote:

Hello,

Webview team is frequently asked whether there is a mechanism to measure WebView's data consumption. At present WebView does not expose an API for this purpose.

Android supports a Traffic Stats API for tagging sockets with some arbitrary tag and then collecting statistics for the given tag (the traffic for tags is visible via adb diagnostics commands or via  queryDetailsForUidTag()). When using this, apps set an active tag for the traffic originating from a thread using setThreadStatsTag() and call tagSocket() to tag each socket with the active tag for the current thread. Tags are integers and some range is reserved for platform use (for example for DownloadManager). I am thinking WebView could either use a number from this range or could provide a way to applications to set their own tag.

Having a single tag for WebView seems to be counter to the Android security model of per-app isolation, in that it allows inferrence and analysis based on other applications' activity if they share the same tag.

 

This API above is in Java but Android team seems to be willing to provide an NDK API and they opened b/29761778 to track this.

Note: Public mailing list, internal bug :)

 

We want to understand what Chrome network stack team thinks about it. Specifically, is the current design for Traffic stats reasonable for chrome network stack use or do you want to see any changes? One option that was recommended was changing tagSocket() to always require a tag.

 

Given the security and privacy concerns fundamentally intrinsic to WebView, my understanding was that solutions like Chrome Custom Tabs represented the better, more secure path for users, similar to iOS and the Safari View Controller. Has this view changed?

The reason I ask is such an API is fundamentally incompatible with Chrome Custom Tabs, by more or less intentional design, and has been discussed at length within the Chrome team and Chrome senior leadership. As such, it seems at odds to offer a powerful diagnostic API to users who use WebView, when it fundamentally is problematic for Custom Tabs, as that would seem to create incentives app developers to providing a less than ideal experience to users, via WebView.

The networking team has explored, at various times, the tagSocket APIs and use cases, and rejected them every time due to the problem either being ill-defined or the use cases proposed fundamentally unable to be met to being at odds with the web platform. I don't see anything on this thread that changes that conclusion.

[Selim Gurun]

Thanks very much for the valuable comments. A few answers below:

On Tue, Jul 12, 2016 at 4:08 PM, Ryan Sleevi <rsl...@chromium.org> wrote:

On Tue, Jul 12, 2016 at 3:55 PM, 'Selim Gurun' via net-dev <net...@chromium.org> wrote:

Hello,

Webview team is frequently asked whether there is a mechanism to measure WebView's data consumption. At present WebView does not expose an API for this purpose.

Android supports a Traffic Stats API for tagging sockets with some arbitrary tag and then collecting statistics for the given tag (the traffic for tags is visible via adb diagnostics commands or via  queryDetailsForUidTag()). When using this, apps set an active tag for the traffic originating from a thread using setThreadStatsTag() and call tagSocket() to tag each socket with the active tag for the current thread. Tags are integers and some range is reserved for platform use (for example for DownloadManager). I am thinking WebView could either use a number from this range or could provide a way to applications to set their own tag.

Having a single tag for WebView seems to be counter to the Android security model of per-app isolation, in that it allows inferrence and analysis based on other applications' activity if they share the same tag.

I think each UID has its own tag space. WebView is part of the application and part of the UID so I don't think this scenario is possible. However I will investigate if the platform reserved range is somehow special. Note that we don't have to use this range.

 

 

This API above is in Java but Android team seems to be willing to provide an NDK API and they opened b/29761778 to track this.

Note: Public mailing list, internal bug :)

 

We want to understand what Chrome network stack team thinks about it. Specifically, is the current design for Traffic stats reasonable for chrome network stack use or do you want to see any changes? One option that was recommended was changing tagSocket() to always require a tag.

 

Given the security and privacy concerns fundamentally intrinsic to WebView, my understanding was that solutions like Chrome Custom Tabs represented the better, more secure path for users, similar to iOS and the Safari View Controller. Has this view changed?

No, apps should use CCT whenever they can, but WebView is still used by a large number of apps and we will continue to improve it. We are not deprecating webview :). 

 

The reason I ask is such an API is fundamentally incompatible with Chrome Custom Tabs, by more or less intentional design, and has been discussed at length within the Chrome team and Chrome senior leadership. As such, it seems at odds to offer a powerful diagnostic API to users who use WebView, when it fundamentally is problematic for Custom Tabs, as that would seem to create incentives app developers to providing a less than ideal experience to users, via WebView.

The networking team has explored, at various times, the tagSocket APIs and use cases, and rejected them every time due to the problem either being ill-defined or the use cases proposed fundamentally unable to be met to being at odds with the web platform. I don't see anything on this thread that changes that conclusion.

I am not aware of any previous work net stack team did. Are there any bugs/threads you can provide.

thanks!

[Ryan Sleevi]

On Tue, Jul 12, 2016 at 4:34 PM, Selim Gurun <sgu...@google.com> wrote:

I am not aware of any previous work net stack team did. Are there any bugs/threads you can provide.

There were some Google-internal discussions, so I sent those off-list.

To be clear: I'm not trying to reject it outright, just that the use cases so far haven't been aligned with something that works well with the Web Platform. If there are use cases/desires that can be articulated more, especially publicly, I'm sure it's a worthwhile discussion to have. Even if it rehashes past discussions, getting those out in the open sounds good. 

[Misha Efimov]

There is similar requests to add traffic tag support to Cronet: https://bugs.chromium.org/p/chromium/issues/detail?id=520198

Tagging threads or sockets doesn't seem to match well the network stack which uses one thread, pools sockets and advanced protocols like HTTP/2 and QUIC that multiplex requests over single connection.

One possible approach which may work is to tag entire UrlRequestContext with single tag, as it owns all socket pools, etc. 

In case of Cronet it could be even easier as each CronetEngine owns exactly one UrlRequestContext and associated networking thread.

--
You received this message because you are subscribed to the Google Groups "net-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to net-dev+u...@chromium.org.
To post to this group, send email to net...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/CACvaWvaLNvgOgQnkga_tBn9ViTzoFYYaPmpyKN9iPxGuW%3Doe2g%40mail.gmail.com.

[Ryan Sleevi]

On Wed, Jul 13, 2016 at 8:42 AM, Misha Efimov <m...@chromium.org> wrote:

There is similar requests to add traffic tag support to Cronet: https://bugs.chromium.org/p/chromium/issues/detail?id=520198

Tagging threads or sockets doesn't seem to match well the network stack which uses one thread, pools sockets and advanced protocols like HTTP/2 and QUIC that multiplex requests over single connection.

One possible approach which may work is to tag entire UrlRequestContext with single tag, as it owns all socket pools, etc. 

In case of Cronet it could be even easier as each CronetEngine owns exactly one UrlRequestContext and associated networking thread.

Right, for Cronet, this could work. The problem with WebView is there exist a variety of URLRequestContexts (or at least, there did). But any change in the process model for WebView, or any sharing of context/state with any other process/use case/application, it doesn't really work well.

I suppose put differently, it could work if everything is in-process and viewed in aggregate (and not trying to be specific). But there's also policy questions about whether that'd be a good thing (again, Web View vs Chrome Plate, feature alignment, etc). I think there's also a question of the use cases trying to be met. I totally understand the argument that it's for debugging, but couldn't that use case be met through other means (such as recording it as device egress)?

I'm hesitant and skeptical, but that's just because we'd need to flesh out more of the use case to explore the SHOULD, and if we SHOULD, then to explore the API of HOW.

[Selim Gurun]

On Wed, Jul 13, 2016 at 2:26 PM, Ryan Sleevi <rsl...@chromium.org> wrote:

On Wed, Jul 13, 2016 at 8:42 AM, Misha Efimov <m...@chromium.org> wrote:

There is similar requests to add traffic tag support to Cronet: https://bugs.chromium.org/p/chromium/issues/detail?id=520198

Tagging threads or sockets doesn't seem to match well the network stack which uses one thread, pools sockets and advanced protocols like HTTP/2 and QUIC that multiplex requests over single connection.

One possible approach which may work is to tag entire UrlRequestContext with single tag, as it owns all socket pools, etc. 

In case of Cronet it could be even easier as each CronetEngine owns exactly one UrlRequestContext and associated networking thread.

Right, for Cronet, this could work. The problem with WebView is there exist a variety of URLRequestContexts (or at least, there did). But any change in the process model for WebView, or any sharing of context/state with any other process/use case/application, it doesn't really work well.

WebView actually has only one BrowserContext, which owns the single URLRequestContext. 

https://cs.chromium.org/chromium/src/android_webview/browser/aw_browser_context.h?q=awbrowsercontext&sq=package:chromium&dr=CSs&l=66

and line 83

Perhaps in the long term we can think of having more. but this would probably be a very long term.

 

I suppose put differently, it could work if everything is in-process and viewed in aggregate (and not trying to be specific). But there's also policy questions about whether that'd be a good thing (again, Web View vs Chrome Plate, feature alignment, etc). I think there's also a question of the use cases trying to be met. I totally understand the argument that it's for debugging, but couldn't that use case be met through other means (such as recording it as device egress)?

 

I'm hesitant and skeptical, but that's just because we'd need to flesh out more of the use case to explore the SHOULD, and if we SHOULD, then to explore the API of HOW.

I agree that we need to look at the use cases more carefully and also it seems like if we decide to go ahead with something, Android is willing to design their NDK API around it.

[Ryan Sleevi]

On Wed, Jul 13, 2016 at 2:50 PM, Selim Gurun <sgu...@google.com> wrote:

WebView actually has only one BrowserContext, which owns the single URLRequestContext. 

https://cs.chromium.org/chromium/src/android_webview/browser/aw_browser_context.h?q=awbrowsercontext&sq=package:chromium&dr=CSs&l=66

and line 83

Perhaps in the long term we can think of having more. but this would probably be a very long term.

Apologies for not being clearer. There are a number of places in Chrome where URLRequestContexts are minted-on-the-fly or copied, and inconsistently. We're still investigating crashes in some, but certainly, a number of //components do.

For example, for 'normal' Chrome, there's a requestcontext for a profile, an off the record profile, and background requests like the system. Components like sync and spellcheck mint their own contexts, as does sign-in. While I have no doubt that the AWBrowserContext begins as a single context, depending on what other components are brought in, there may be more than one. Anyways, I don't want to derail too much, but I did want to share why it's 'not enough' to just look at the //content interfaces that might be used, because anything else that's brought in (higher or lower in the stack) may be minting contexts from it, and would have to be considered.

Anyways, I suppose next steps would be capturing up a design-doc and we could explore the open questions?

[Selim Gurun]

Sounds reasonable. Misha should we write one that would work both Webview and Cronet? or do you have something from the earlier bug (I see it was opened a year ago). 

[Misha Efimov]

I would be great to figure out solution that works for both WebView and Cronet. 

The bug was open a year ago, but its priority got lowered once we found a work-around.

We still need a proper answer for Cronet, although changes to NDK API may not help as we need to support older versions of Android.

--
You received this message because you are subscribed to the Google Groups "net-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to net-dev+u...@chromium.org.
To post to this group, send email to net...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/CAA5teAJiSj0UQTX9jht8PiYi9XUeESZG%3DxB8bGsdmSx5-%3DA0Sw%40mail.gmail.com.