Should we make net::NetworkDelegate available over mojo?

33 views
Skip to first unread message

cho...@chromium.org

unread,
Mar 12, 2018, 5:12:47 PM3/12/18
to network-service-dev
Background:
I'm looking at failed browser test "TaskManagerBrowserTest.SentDataObserved" and noticed that |net::URLRequestContext| doesn't have proper |network_delegate_| setup when Network Service is enabled.
e.g. It only has a dummy |BasicNetworkDelegate| constructed from |URLRequestContextBuilder::Build():407|.

My question is:
* Should we make |net::NetworkDelegate| a mojo class, and pass the interface pointer along with |mojom::NetworkContextParams|?

Or are we replacing |net::NetworkDelegate| with something else?

Thanks!
Chong

John Abd-El-Malek

unread,
Mar 12, 2018, 5:16:00 PM3/12/18
to Chong Zhang, network-service-dev
net::NetworkDelegate is specific to using net in-process, similar to net::URLRequestContextDelegate. We have alternatives in mojo, i.e. if we want something from the network process globally to the browser we can use NetworkServiceClient. However if it's per request, we prefer URLLoaderClient. The renderer can send data to the renderer, it already gives it notifications such as WebContentsObserver::SubresourceResponseStarted.

Can you describe what data is needed?

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.
To post to this group, send email to network-service-dev@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/network-service-dev/333d3297-be1f-4c03-836d-53bc310cb76e%40chromium.org.

Chong Zhang

unread,
Mar 12, 2018, 5:29:38 PM3/12/18
to John Abd-El-Malek, network-service-dev
Thanks for the information! The data I need is |NetworkDelegate::NotifyNetworkBytesReceived()| and |NetworkDelegate::NotifyNetworkBytesSent()| so the |TaskManager| can track network traffic per tab.

Will check if I can get such info from |WebContentsObserver|.

Thanks!

On Mon, Mar 12, 2018 at 2:16 PM John Abd-El-Malek <j...@chromium.org> wrote:
net::NetworkDelegate is specific to using net in-process, similar to net::URLRequestContextDelegate. We have alternatives in mojo, i.e. if we want something from the network process globally to the browser we can use NetworkServiceClient. However if it's per request, we prefer URLLoaderClient. The renderer can send data to the renderer, it already gives it notifications such as WebContentsObserver::SubresourceResponseStarted.

Can you describe what data is needed?
On Mon, Mar 12, 2018 at 2:12 PM, <cho...@chromium.org> wrote:
Background:
I'm looking at failed browser test "TaskManagerBrowserTest.SentDataObserved" and noticed that |net::URLRequestContext| doesn't have proper |network_delegate_| setup when Network Service is enabled.
e.g. It only has a dummy |BasicNetworkDelegate| constructed from |URLRequestContextBuilder::Build():407|.

My question is:
* Should we make |net::NetworkDelegate| a mojo class, and pass the interface pointer along with |mojom::NetworkContextParams|?

Or are we replacing |net::NetworkDelegate| with something else?

Thanks!
Chong

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service...@chromium.org.
To post to this group, send email to network-s...@chromium.org.

John Abd-El-Malek

unread,
Mar 12, 2018, 5:37:58 PM3/12/18
to Chong Zhang, cc...@chromium.org, Jay Civelli, network-service-dev
Conley and Jay are adding this information because it's needed for page load metrics, so I think you can piggy back off that.

On Mon, Mar 12, 2018 at 2:29 PM, Chong Zhang <cho...@chromium.org> wrote:
Thanks for the information! The data I need is |NetworkDelegate::NotifyNetworkBytesReceived()| and |NetworkDelegate::NotifyNetworkBytesSent()| so the |TaskManager| can track network traffic per tab.

Will check if I can get such info from |WebContentsObserver|.

Thanks!
On Mon, Mar 12, 2018 at 2:16 PM John Abd-El-Malek <j...@chromium.org> wrote:
net::NetworkDelegate is specific to using net in-process, similar to net::URLRequestContextDelegate. We have alternatives in mojo, i.e. if we want something from the network process globally to the browser we can use NetworkServiceClient. However if it's per request, we prefer URLLoaderClient. The renderer can send data to the renderer, it already gives it notifications such as WebContentsObserver::SubresourceResponseStarted.

Can you describe what data is needed?
On Mon, Mar 12, 2018 at 2:12 PM, <cho...@chromium.org> wrote:
Background:
I'm looking at failed browser test "TaskManagerBrowserTest.SentDataObserved" and noticed that |net::URLRequestContext| doesn't have proper |network_delegate_| setup when Network Service is enabled.
e.g. It only has a dummy |BasicNetworkDelegate| constructed from |URLRequestContextBuilder::Build():407|.

My question is:
* Should we make |net::NetworkDelegate| a mojo class, and pass the interface pointer along with |mojom::NetworkContextParams|?

Or are we replacing |net::NetworkDelegate| with something else?

Thanks!
Chong

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.
To post to this group, send email to network-service-dev@chromium.org.

Conley Owens

unread,
Mar 12, 2018, 5:43:38 PM3/12/18
to John Abd-El-Malek, Chong Zhang, Jay Civelli, network-service-dev
See this class:
https://cs.chromium.org/chromium/src/content/public/common/subresource_load_info.mojom

It ultimately gets sent through to WebContentsObserver::SubresourceResponseStarted

On Mon, Mar 12, 2018 at 2:37 PM, John Abd-El-Malek <j...@chromium.org> wrote:
Conley and Jay are adding this information because it's needed for page load metrics, so I think you can piggy back off that.
On Mon, Mar 12, 2018 at 2:29 PM, Chong Zhang <cho...@chromium.org> wrote:
Thanks for the information! The data I need is |NetworkDelegate::NotifyNetworkBytesReceived()| and |NetworkDelegate::NotifyNetworkBytesSent()| so the |TaskManager| can track network traffic per tab.

Will check if I can get such info from |WebContentsObserver|.

Thanks!
On Mon, Mar 12, 2018 at 2:16 PM John Abd-El-Malek <j...@chromium.org> wrote:
net::NetworkDelegate is specific to using net in-process, similar to net::URLRequestContextDelegate. We have alternatives in mojo, i.e. if we want something from the network process globally to the browser we can use NetworkServiceClient. However if it's per request, we prefer URLLoaderClient. The renderer can send data to the renderer, it already gives it notifications such as WebContentsObserver::SubresourceResponseStarted.

Can you describe what data is needed?
On Mon, Mar 12, 2018 at 2:12 PM, <cho...@chromium.org> wrote:
Background:
I'm looking at failed browser test "TaskManagerBrowserTest.SentDataObserved" and noticed that |net::URLRequestContext| doesn't have proper |network_delegate_| setup when Network Service is enabled.
e.g. It only has a dummy |BasicNetworkDelegate| constructed from |URLRequestContextBuilder::Build():407|.

My question is:
* Should we make |net::NetworkDelegate| a mojo class, and pass the interface pointer along with |mojom::NetworkContextParams|?

Or are we replacing |net::NetworkDelegate| with something else?

Thanks!
Chong

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsubscribe...@chromium.org.

Chong Zhang

unread,
Mar 12, 2018, 5:51:24 PM3/12/18
to Conley Owens, John Abd-El-Malek, Jay Civelli, network-service-dev
Looks like something I can use, thanks for the help!
Will get back if I went into other troubles.

On Mon, Mar 12, 2018 at 2:43 PM Conley Owens <cc...@chromium.org> wrote:
See this class:
https://cs.chromium.org/chromium/src/content/public/common/subresource_load_info.mojom

It ultimately gets sent through to WebContentsObserver::SubresourceResponseStarted
On Mon, Mar 12, 2018 at 2:37 PM, John Abd-El-Malek <j...@chromium.org> wrote:
Conley and Jay are adding this information because it's needed for page load metrics, so I think you can piggy back off that.
On Mon, Mar 12, 2018 at 2:29 PM, Chong Zhang <cho...@chromium.org> wrote:
Thanks for the information! The data I need is |NetworkDelegate::NotifyNetworkBytesReceived()| and |NetworkDelegate::NotifyNetworkBytesSent()| so the |TaskManager| can track network traffic per tab.

Will check if I can get such info from |WebContentsObserver|.

Thanks!
On Mon, Mar 12, 2018 at 2:16 PM John Abd-El-Malek <j...@chromium.org> wrote:
net::NetworkDelegate is specific to using net in-process, similar to net::URLRequestContextDelegate. We have alternatives in mojo, i.e. if we want something from the network process globally to the browser we can use NetworkServiceClient. However if it's per request, we prefer URLLoaderClient. The renderer can send data to the renderer, it already gives it notifications such as WebContentsObserver::SubresourceResponseStarted.

Can you describe what data is needed?
On Mon, Mar 12, 2018 at 2:12 PM, <cho...@chromium.org> wrote:
Background:
I'm looking at failed browser test "TaskManagerBrowserTest.SentDataObserved" and noticed that |net::URLRequestContext| doesn't have proper |network_delegate_| setup when Network Service is enabled.
e.g. It only has a dummy |BasicNetworkDelegate| constructed from |URLRequestContextBuilder::Build():407|.

My question is:
* Should we make |net::NetworkDelegate| a mojo class, and pass the interface pointer along with |mojom::NetworkContextParams|?

Or are we replacing |net::NetworkDelegate| with something else?

Thanks!
Chong

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service...@chromium.org.
To post to this group, send email to network-s...@chromium.org.

cho...@chromium.org

unread,
Mar 14, 2018, 8:31:46 PM3/14/18
to network-service-dev, cc...@chromium.org, j...@chromium.org, jciv...@chromium.org
Looked at |WebContentsObserver| more and my feeling is that it probably doesn't fit the use case for |TaskManager| here.
  * More specifically: |TaskManager| (on chrome - browser process) wants to track network bytes sent and received per (child_id, route_id).

Reason 1: It adds an additional process hop.
e.g. Network Process -> Renderer Process -> Browser Process (TaskManager).

Reason 2: |TaskManager| might want to be notified more frequently other than response started / completed.
e.g. According to the call site in |URLRequestJob::RecordBytesRead(...)::MaybeNotifyNetworkBytes()| the notification could be sent multiple times before the request completes.

---
To minimize the logic change does that make sense if I add the following API in |NetworkServiceClient|, and then send the data to chrome through |ContentBrowserClient| or a separate observer interface on 'network_service_instance.h'?
```
interface NetworkServiceClient {
  // ...
  OnRawBytesTransferred(uint32 process_id,
                                           uint32 routing_id,
                                           int64 bytes_read,
                                           int64 bytes_sent);
};
```

Or do we still prefer the |WebContentsObserver| path?

Thanks!
Chong
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.
To post to this group, send email to network-service-dev@chromium.org.

Jay Civelli

unread,
Mar 15, 2018, 1:01:37 AM3/15/18
to cho...@chromium.org, network-service-dev, cc...@chromium.org, John Abd-El-Malek
If we go the WebContentsObserver route, how will we report the network usage for non tab processes? (extensions, utility services...)

To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsubscribe...@chromium.org.

John Abd-El-Malek

unread,
Mar 15, 2018, 9:51:46 AM3/15/18
to Jay Civelli, Chong Zhang, network-service-dev, cc...@chromium.org
On Wed, Mar 14, 2018 at 10:01 PM, Jay Civelli <jciv...@chromium.org> wrote:
If we go the WebContentsObserver route, how will we report the network usage for non tab processes? (extensions, utility services...)

Extensions have WebContents, so it would be the same I thought?

Which utility processes make network requests?
 

On Wed, Mar 14, 2018 at 5:31 PM, <cho...@chromium.org> wrote:
Looked at |WebContentsObserver| more and my feeling is that it probably doesn't fit the use case for |TaskManager| here.
  * More specifically: |TaskManager| (on chrome - browser process) wants to track network bytes sent and received per (child_id, route_id).

Reason 1: It adds an additional process hop.
e.g. Network Process -> Renderer Process -> Browser Process (TaskManager).

Is your concern worry or extra hops?

Most of this data is being sent from the network process to the renderer already for subresources. So the question is whether to add a new api from the network process to the browser, vs using the existing one and having renderer code send it.
 

Reason 2: |TaskManager| might want to be notified more frequently other than response started / completed.
e.g. According to the call site in |URLRequestJob::RecordBytesRead(...)::MaybeNotifyNetworkBytes()| the notification could be sent multiple times before the request completes.

Do we need to maintain this behavior? The task manager is something that most users never open, so a small delay (or conceivably can be large for very lage resources) might be acceptable?

cho...@chromium.org

unread,
Mar 15, 2018, 1:52:26 PM3/15/18
to network-service-dev, jciv...@chromium.org, cho...@chromium.org, cc...@chromium.org
Thanks for the comments! Please see responses below.


On Thursday, March 15, 2018 at 6:51:46 AM UTC-7, John wrote:


On Wed, Mar 14, 2018 at 10:01 PM, Jay Civelli <jciv...@chromium.org> wrote:
If we go the WebContentsObserver route, how will we report the network usage for non tab processes? (extensions, utility services...)

Extensions have WebContents, so it would be the same I thought?

Which utility processes make network requests?

Will defer the question to Jay as I don't have a big picture of WebContents...
 
 

On Wed, Mar 14, 2018 at 5:31 PM, <cho...@chromium.org> wrote:
Looked at |WebContentsObserver| more and my feeling is that it probably doesn't fit the use case for |TaskManager| here.
  * More specifically: |TaskManager| (on chrome - browser process) wants to track network bytes sent and received per (child_id, route_id).

Reason 1: It adds an additional process hop.
e.g. Network Process -> Renderer Process -> Browser Process (TaskManager).

Is your concern worry or extra hops?

Most of this data is being sent from the network process to the renderer already for subresources. So the question is whether to add a new api from the network process to the browser, vs using the existing one and having renderer code send it.

Yes that's my concern, and I'm worried about the potential side effects of the logic change. e.g.
Before: TaskManager get raw data from network code directly.
After: TaskManager get data from renderer, which should be considered as the consumer of the network code.
(I'm a little bit worried even though we can make sure that the data was not affected by middleware such as data reduction, safe browsing, and decode/encode.)
 
 

Reason 2: |TaskManager| might want to be notified more frequently other than response started / completed.
e.g. According to the call site in |URLRequestJob::RecordBytesRead(...)::MaybeNotifyNetworkBytes()| the notification could be sent multiple times before the request completes.

Do we need to maintain this behavior? The task manager is something that most users never open, so a small delay (or conceivably can be large for very lage resources) might be acceptable?

Hmmm if that's the case we should be fine using WebContentsObserver. I will file a bug since the change is not trivial.
 

To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.

Jay Civelli

unread,
Mar 15, 2018, 1:54:13 PM3/15/18
to cho...@chromium.org, network-service-dev, cc...@chromium.org
On Thu, Mar 15, 2018 at 10:52 AM <cho...@chromium.org> wrote:
Thanks for the comments! Please see responses below.

On Thursday, March 15, 2018 at 6:51:46 AM UTC-7, John wrote:


On Wed, Mar 14, 2018 at 10:01 PM, Jay Civelli <jciv...@chromium.org> wrote:
If we go the WebContentsObserver route, how will we report the network usage for non tab processes? (extensions, utility services...)

Extensions have WebContents, so it would be the same I thought?

Which utility processes make network requests?

Will defer the question to Jay as I don't have a big picture of WebContents...
John is right, I don't think any utility process does network requests, so this shouldn't be an issue. 
To unsubscribe from this group and stop receiving emails from it, send an email to network-service...@chromium.org.
To post to this group, send email to network-s...@chromium.org.

Matt Menke

unread,
Mar 15, 2018, 1:57:36 PM3/15/18
to Jay Civelli, cho...@chromium.org, network-service-dev, cc...@chromium.org
The "utility process" (Which is used by cloud print?) does make network requests.  Currently it has its own URLRequestContext, but I assume that will need to change.

To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.
To post to this group, send email to network-service-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.
To post to this group, send email to network-service-dev@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/network-service-dev/CAJ6%3Dx6%3DCY1CTYJgHqa0LmcpDQ6ArZy%3DuuDfj6QKF11%3DxT5CLQw%40mail.gmail.com.

Jay Civelli

unread,
Mar 15, 2018, 2:00:27 PM3/15/18
to mme...@chromium.org, cho...@chromium.org, network-service-dev, cc...@chromium.org
Do you mean the one under chrome/service?
I was under the impression that this was built as a different exe and not launched by Chrome (so it wouldn't be reported today anyway in the task manager).


To unsubscribe from this group and stop receiving emails from it, send an email to network-service...@chromium.org.
To post to this group, send email to network-s...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service...@chromium.org.
To post to this group, send email to network-s...@chromium.org.

Matt Menke

unread,
Mar 15, 2018, 2:05:04 PM3/15/18
to Jay Civelli, cho...@chromium.org, network-service-dev, cc...@chromium.org
Erm.... I guess?  I'm not remotely familiar with it, beyond the fact that it creates its own URLRequestContext (As I own URLRequestContext construction/configuration).

To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.
To post to this group, send email to network-service-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.
To post to this group, send email to network-service-dev@chromium.org.

John Abd-El-Malek

unread,
Mar 15, 2018, 3:27:20 PM3/15/18
to Matthew Menke, Jay Civelli, Chong Zhang, network-service-dev, cc...@chromium.org
Yeah the cloud print service is something separate (i.e. none of the parent code runs in the browser process), so it doesn't show up in the task manager.


To unsubscribe from this group and stop receiving emails from it, send an email to network-service...@chromium.org.
To post to this group, send email to network-s...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service...@chromium.org.
To post to this group, send email to network-s...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service...@chromium.org.
To post to this group, send email to network-s...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/network-service-dev/CAEK7mvpFVq36H_2LokQQfnmtGPxKBOQCUMU-_QucS%2B%3D4YET1xA%40mail.gmail.com.

cho...@chromium.org

unread,
Mar 15, 2018, 4:15:31 PM3/15/18
to network-service-dev, mme...@chromium.org, jciv...@chromium.org, cho...@chromium.org, cc...@chromium.org
Thanks for all the information and suggestions! I've filed crbug.com/822415 and will start working on the |WebContentsObserver| route.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.
To post to this group, send email to network-service-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.
To post to this group, send email to network-service-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.
To post to this group, send email to network-service-dev@chromium.org.

cho...@chromium.org

unread,
Apr 2, 2018, 8:40:18 PM4/2/18
to network-service-dev, mme...@chromium.org, jciv...@chromium.org, cho...@chromium.org, cc...@chromium.org, Nick Carter, John Abd-El-Malek
Moving the discussion from https://crrev.com/c/978607/11//COMMIT_MSG#18 for more input.

> > Plans for Other Coverages:
> > Utility processes: They shouldn't be making network requests, see discussion below:
> 
> Isn't the network service a utility process? Doesn't the network service have to support requests that don't have a renderer process as consumer by the time they finish (e.g. navigator.sendBeacon(), or other opaque/detached requests)?

Yes network service should be the only utility process making network requests, or do you mean we want to report network service's usage in TaskManager?

I'm not very familiar with |navigator.sendBeacon()| though, but with the |WebContentsObserver| approach:
1. Detached requests won't be tracked (but the case should be rare).
2. We can increase the accuracy by sending more events during the request.

I'm not sure but what's our tolerance on the accuracy of TaskManager?

> 
> > Service Workers
> 
> The problem I'm anticipating is that ServiceWorkers are not affiliated with any one WebContents, so WebContentsObserver. However, they'll be able to get a ResourceLoadComplete type message. Currently there's no content/public observer interface that task manager can hook into to observe what's happening in a service worker (and as a result). https://bugs.chromium.org/p/chromium/issues/detail?id=716609#c8 is a sketch of what we'd have to do to plumb serviceworkers into the taskmanager. Currently if a process has a serviceworker, but no WebContents, it doesn't show up in the taskmanager. However this is a somewhat rare case -- usually the serviceworker runs in the same process as a webpage.

That's surprising to know, thanks for the info!

network-service-dev@ Is this an expected behavior that ServiceWorkers could have no WebContents?

> 
> > Extensions
> 
> For extensions, I don't think there's any follow-up work. Extensions are already WebContents-based, and they should (AFAIK) already be solved by your approach here. Are you aware of any extensions-specific problems?

Nope, I'm just referring to the discussion here.

> 
> > Browser Process
> 
> It's less clear to me what this will look like -- under the network service, I think the browser process is sometimes instantiating network::URLLoaders directly. I'm not excessively worried about being able to find a feasible solution -- we just need a hook function somewhere --, but the question I'd ask of any solution is: how easy will it be to add a new kind of browserprocess-initated network request, and forget to plumb it into the TaskManager?

Most consumers in the browser process are expected to use |StoragePartition::GetURLLoaderFactoryForBrowserProcess/IOThread()| to make requests, and we can add some hooks in, for example, |StoragePartitionImpl::URLLoaderFactoryForBrowserProcess::CreateLoaderAndStart()|.
There are a few other use cases like |SystemNetworkContextManager|, but in general it should be rare.

So I'd say the risk is low, but I'm not sure if it's introducing any additional mojo costs.

Matt Menke

unread,
Apr 2, 2018, 9:47:49 PM4/2/18
to cho...@chromium.org, network-service-dev, Jay Civelli, cc...@chromium.org, Nick Carter, John Abd-El-Malek
On Mon, Apr 2, 2018 at 8:40 PM, <cho...@chromium.org> wrote:
Moving the discussion from https://crrev.com/c/978607/11//COMMIT_MSG#18 for more input.

> > Plans for Other Coverages:
> > Utility processes: They shouldn't be making network requests, see discussion below:
> 
> Isn't the network service a utility process? Doesn't the network service have to support requests that don't have a renderer process as consumer by the time they finish (e.g. navigator.sendBeacon(), or other opaque/detached requests)?

Yes network service should be the only utility process making network requests, or do you mean we want to report network service's usage in TaskManager?

I'm not very familiar with |navigator.sendBeacon()| though, but with the |WebContentsObserver| approach:
1. Detached requests won't be tracked (but the case should be rare).
2. We can increase the accuracy by sending more events during the request.

I'm not sure but what's our tolerance on the accuracy of TaskManager?

> 
> > Service Workers
> 
> The problem I'm anticipating is that ServiceWorkers are not affiliated with any one WebContents, so WebContentsObserver. However, they'll be able to get a ResourceLoadComplete type message. Currently there's no content/public observer interface that task manager can hook into to observe what's happening in a service worker (and as a result). https://bugs.chromium.org/p/chromium/issues/detail?id=716609#c8 is a sketch of what we'd have to do to plumb serviceworkers into the taskmanager. Currently if a process has a serviceworker, but no WebContents, it doesn't show up in the taskmanager. However this is a somewhat rare case -- usually the serviceworker runs in the same process as a webpage.

That's surprising to know, thanks for the info!

network-service-dev@ Is this an expected behavior that ServiceWorkers could have no WebContents?

Yes.  A single ServiceWorker may be servicing multiple WebContents.  Or no WebContents at all, I believe.
 
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsubscribe...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsubscribe...@chromium.org.

To post to this group, send email to network-service-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsubscribe...@chromium.org.

John Abd-El-Malek

unread,
Apr 3, 2018, 1:15:03 PM4/3/18
to Chong Zhang, network-service-dev, Matthew Menke, Jay Civelli, cc...@chromium.org, Nick Carter
On Mon, Apr 2, 2018 at 5:40 PM, <cho...@chromium.org> wrote:
Moving the discussion from https://crrev.com/c/978607/11//COMMIT_MSG#18 for more input.

> > Plans for Other Coverages:
> > Utility processes: They shouldn't be making network requests, see discussion below:
> 
> Isn't the network service a utility process? Doesn't the network service have to support requests that don't have a renderer process as consumer by the time they finish (e.g. navigator.sendBeacon(), or other opaque/detached requests)?

Yes network service should be the only utility process making network requests, or do you mean we want to report network service's usage in TaskManager?

I'm not very familiar with |navigator.sendBeacon()| though, but with the |WebContentsObserver| approach:
1. Detached requests won't be tracked (but the case should be rare).
2. We can increase the accuracy by sending more events during the request.

If the detached request finishes while the tab is there, it would be shown right?

Also I'm not sure how the cl changes things, I don't believe detached requests show up now.


I'm not sure but what's our tolerance on the accuracy of TaskManager?

> 
> > Service Workers
> 
> The problem I'm anticipating is that ServiceWorkers are not affiliated with any one WebContents, so WebContentsObserver. However, they'll be able to get a ResourceLoadComplete type message. Currently there's no content/public observer interface that task manager can hook into to observe what's happening in a service worker (and as a result). https://bugs.chromium.org/p/chromium/issues/detail?id=716609#c8 is a sketch of what we'd have to do to plumb serviceworkers into the taskmanager. Currently if a process has a serviceworker, but no WebContents, it doesn't show up in the taskmanager. However this is a somewhat rare case -- usually the serviceworker runs in the same process as a webpage.

That's surprising to know, thanks for the info!

network-service-dev@ Is this an expected behavior that ServiceWorkers could have no WebContents?

Ditto re above, this change doesn't make things regress. When we care to show SWs in the task manager, we can send new IPCs that aren't tied to the WebContents.
 
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsubscribe...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsubscribe...@chromium.org.

To post to this group, send email to network-service-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsubscribe...@chromium.org.

nca...@gmail.com

unread,
Apr 3, 2018, 6:47:11 PM4/3/18
to network-service-dev, cho...@chromium.org, mme...@chromium.org, jciv...@chromium.org, cc...@chromium.org, ni...@chromium.org


On Tuesday, April 3, 2018 at 10:15:03 AM UTC-7, John wrote:


On Mon, Apr 2, 2018 at 5:40 PM, <cho...@chromium.org> wrote:
Moving the discussion from https://crrev.com/c/978607/11//COMMIT_MSG#18 for more input.

> > Plans for Other Coverages:
> > Utility processes: They shouldn't be making network requests, see discussion below:
> 
> Isn't the network service a utility process? Doesn't the network service have to support requests that don't have a renderer process as consumer by the time they finish (e.g. navigator.sendBeacon(), or other opaque/detached requests)?

Yes network service should be the only utility process making network requests, or do you mean we want to report network service's usage in TaskManager?

I'm not very familiar with |navigator.sendBeacon()| though, but with the |WebContentsObserver| approach:
1. Detached requests won't be tracked (but the case should be rare).
2. We can increase the accuracy by sending more events during the request.

If the detached request finishes while the tab is there, it would be shown right?

Also I'm not sure how the cl changes things, I don't believe detached requests show up now.

Do detached requests not go through ChromeNetworkDelegate today? At TOT, anything not attributable to a frame or child_id, gets credited to the browser process.



I'm not sure but what's our tolerance on the accuracy of TaskManager?

> 
> > Service Workers
> 
> The problem I'm anticipating is that ServiceWorkers are not affiliated with any one WebContents, so WebContentsObserver. However, they'll be able to get a ResourceLoadComplete type message. Currently there's no content/public observer interface that task manager can hook into to observe what's happening in a service worker (and as a result). https://bugs.chromium.org/p/chromium/issues/detail?id=716609#c8 is a sketch of what we'd have to do to plumb serviceworkers into the taskmanager. Currently if a process has a serviceworker, but no WebContents, it doesn't show up in the taskmanager. However this is a somewhat rare case -- usually the serviceworker runs in the same process as a webpage.

That's surprising to know, thanks for the info!

network-service-dev@ Is this an expected behavior that ServiceWorkers could have no WebContents?

Ditto re above, this change doesn't make things regress. When we care to show SWs in the task manager, we can send new IPCs that aren't tied to the WebContents.

Ditto re above -- doesn't ServiceWorker activity flow through ChromeNetworkDelegate::OnNetworkBytesReceived? 

 
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.
To post to this group, send email to network-s...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.
To post to this group, send email to network-s...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.
To post to this group, send email to network-s...@chromium.org.

Chong Zhang

unread,
Apr 4, 2018, 2:14:57 PM4/4/18
to nca...@gmail.com, network-service-dev, mme...@chromium.org, Jay Civelli, cc...@chromium.org, ni...@chromium.org, John Abd-El-Malek
Just to give a summary to current/possible implementations:

Current Implementation:
AFAIK 100% of the HTTP traffic will be tracked at |ChromeNetworkDelegate::OnNetworkBytesReceived()| since it's hooked into |net::URLRequestJob| directly, even though we may not sure about the source frame (will credit to browser).

Possible Implementations (with NS):
1. Report network traffic through |NetworkServiceClient|:
Pros:
  * Preserves existing behavior, small code change.
Cons:
  * Needs the extra API on |NetworkServiceClient|.

2. Report network traffic through |URLLoaderClient| => |WebContentsObserver|
Pros:
  * Can reuse existing APIs (e.g. |URLLoaderClient::OnComplete()| and |WebContentsObserver::ResourceLoadComplete()|).
Cons:
  * Requires non-trivial amount of work ensure the coverage and maintenance, and probably cannot give a guarantee to 100% coverage.
  * e.g. Need to plug something into all |URLLoader| consumers, and handle |ServiceWorker| specially since it may not have |WebContents|.

3. Use the |URLLoaderClient| => |ResourceUsageReporter| route, where the latter is per-process and is already used to get memory info:
Pros:
  * Less work compared to 2. since we don't need extra work for non-WebContents processes.
Cons:
  * Still need to modify all |URLLoader| consumers and cannot guarantee 100% coverage.


In other words the current trade off is between:
A. Additional |NetworkServiceClient| API.
B. Less HTTP traffic coverage and possible maintenance cost.


IMO I'd prefer 1. since it's much easier to implement and less likely to cause a regression, but I trust John on architecture decisions as I may easily overlook stuff.


To unsubscribe from this group and stop receiving emails from it, send an email to network-service...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service...@chromium.org.

Matt Menke

unread,
Apr 4, 2018, 2:17:05 PM4/4/18
to nca...@gmail.com, network-service-dev, cho...@chromium.org, Jay Civelli, cc...@chromium.org, Nick Carter
On Tue, Apr 3, 2018 at 6:47 PM, <nca...@gmail.com> wrote:


On Tuesday, April 3, 2018 at 10:15:03 AM UTC-7, John wrote:


On Mon, Apr 2, 2018 at 5:40 PM, <cho...@chromium.org> wrote:
Moving the discussion from https://crrev.com/c/978607/11//COMMIT_MSG#18 for more input.

> > Plans for Other Coverages:
> > Utility processes: They shouldn't be making network requests, see discussion below:
> 
> Isn't the network service a utility process? Doesn't the network service have to support requests that don't have a renderer process as consumer by the time they finish (e.g. navigator.sendBeacon(), or other opaque/detached requests)?

Yes network service should be the only utility process making network requests, or do you mean we want to report network service's usage in TaskManager?

I'm not very familiar with |navigator.sendBeacon()| though, but with the |WebContentsObserver| approach:
1. Detached requests won't be tracked (but the case should be rare).
2. We can increase the accuracy by sending more events during the request.

If the detached request finishes while the tab is there, it would be shown right?

Also I'm not sure how the cl changes things, I don't believe detached requests show up now.

Do detached requests not go through ChromeNetworkDelegate today? At TOT, anything not attributable to a frame or child_id, gets credited to the browser process.

All requests (Possibly modulo ones through SafeBrowsing's URLRequestContext - not sure) go through a ChromeNetworkDelegate.  This includes requests not associated with a profile as well.
 
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsubscribe...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsubscribe...@chromium.org.
To post to this group, send email to network-s...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsubscribe...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.

John Abd-El-Malek

unread,
Apr 4, 2018, 3:38:49 PM4/4/18
to Chong Zhang, nca...@gmail.com, network-service-dev, Matthew Menke, Jay Civelli, cc...@chromium.org, Nick Carter
Thanks for the detailed summary and investigation. I originally thought that it wouldn't be much more work to do 2, but it seems there are a bunch of corner cases. I defer to you, so since you prefer 1 let's go with that. However can we try to keep the api minimal (i.e. 1 method to get sent/received data) and only send this when the browser asks (e.g. because task manager is up)?

To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service-dev+unsub...@chromium.org.

Chong Zhang

unread,
Apr 4, 2018, 4:38:25 PM4/4/18
to John Abd-El-Malek, nca...@gmail.com, network-service-dev, Matthew Menke, Jay Civelli, cc...@chromium.org, Nick Carter
Thanks for the comments! I will keep the API minimal and send to your review later.

To unsubscribe from this group and stop receiving emails from it, send an email to network-service...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "network-service-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to network-service...@chromium.org.
Reply all
Reply to author
Forward
0 new messages