Proposal: ResourceStats API for NetworkStats API and PowerStats API integration

[Borting Chen]

Hi all:

As we have discussed in [1], we need a new API to integrate the PowerStats API [2] with the existing NetworkStats API [3]. After an internal discussion at Taipei office (with Gene Lian, Vincent Chang, Ken Chang, and John Shih), we want to propose a 'ResourceStats API' [4] to merge these two stats APIs into one.

The proposed ResourceStats API supports two primary functionality:
(1) resource statistics (provide resource usage information, such as power consumption)
(2) resource control (notify user when resource usage exceeds a defined threshold).

The ResourceStats API categorizes a system into several types of resource. Currently, we plan to support statistics and control for two types of resource: 'network' & 'power', which focus on network usage and power consumption, respectively.

The following is our proposal, note that:
(1) getStats() & clearStats() are designed to support the query & delete for statistics. The |resourceStatsOptions| argument provides the detail of the resource we want to obtain/delete, such as the Wi-Fi usage of the browser App.
(2) addAlarm(), getAlarm(), and removeAlarm() are designed for resource control. These functions are retrieved from existing NetworkStats API directly. [5]

== Proposal Start ==
dictionary NetworkStatsOptions
{
/**
* |type| is network type, 0 for Wi-Fi and 1 for Mobile.
*
* |id| is '0' for Wi-Fi, or the iccid for mobile (SIM).
*
* |manifestURL| is used to retrieve network usage of a specified application.
* If null, system network usage is returned.
*/
unsigned long type;
DOMString id;
DOMString manifestURL;
};

dictionary PowerStatsOptions
{
/**
* |component| specifies which component's power consumption will be returned.
* If null, total power consumption is returned.
*
* |manifestURL| is used to retrieve power consumption of a specified
* application. If null, all applications' power consumption is returned.
*/
DOMString component;
DOMString manifestURL;
};

dictionary ResourceStatsAlarmOptions
{
unsigned long long startTime; // time in milliseconds since Epoch
any data;
};

[JSImplementation="@mozilla.org/resourceStatsAlarm;1", Pref="dom.mozResourceStats.enabled"]
interface MozResourceStatsAlarm
{
readonly attribute unsigned long alarmId;
readonly attribute DOMString resourceType;
readonly attribute any resourceStatsOptions;
readonly attribute any threshold;
readonly attribute any data;
};

[JSImplementation="@mozilla.org/resourceStatsManager;1", NavigatorProperty="mozResourceStats", Pref="dom.mozResourceStats.enabled"]
interface ResourceStatsManager
{
const unsigned long WIFI = 0;
const unsigned long MOBILE = 1;

/**
* Query resource statistics.
*
* |resourceType| specifies the type of resource statistics we want to
* obtain. This value should be in the return of resourceTypes attribute.
*
* |start| and |end| specifies the interest time range, both included.
*
* |resourceStatsOptions| specifies the detial of resource in which we are
* are interested. Its value depends on which |resourceType| is specified.
* For 'network' resource, the options should be a NetworkStatsOptions object.
* For 'power' resource, the options should be a PowerStatsOptions object.
*
* If success, the |result| field keeps statistics. For 'network' resource
* the |result| is an array of NetworkStats object. For 'power' resource
* the result is an array of PowerStats object.
*/
Promise getStats(DOMString resourceType,
unsigned long long start,
unsigned long long end,
optional any resourceStatsOptions);

/**
* Clear resource statistics stored in database.
*
* |resourceType| specifies the type of resource statistics we want to clear.
*
* If |resourceStatsOptions| is provided, then only the statistics associated
* to that |resourceStatsOptions| is deleted. Otherwise, all statistics of
* the specified |resourceType| is deleted.
*/
Promise clearStats(DOMString resourceType,
optional any resourceStatsOptions);

/**
* Install an alarm to monitor resource usage.
*
* |resourceType| specifies the type of resource we want to monitor.
*
* The |threshold| is an object specified the limit of resource usage. When
* resource usage reaches the threshold, a "resourceStats-alarm" system
* message is sent to the application.
*
* |resourceStatsOptions| specifies the detial of resource in which we are
* are interested.
*
* |alarmOptions| is a ResourceStatsAlarmOptions object.
*
* If success, the |result| field keeps the alarm Id.
*/
Promise addAlarm(DOMString resourceType,
any threshold,
optional any resourceStatsOptions,
optional ResourceStatsAlarmOptions alarmOptions);

/**
* Obtain alarms set for the specified resource type.
*
* |resourceType| specifies the type of resource.
*
* If |resourceStatsOptions| is provided, then only the alarms monitoring
* that resource are returned. Otherwise, all alarms set for the specified
* |resourceType| is returned.
*
* If success, the |result| is an array of ResourceStatsAlarm.
*/
Promise getAlarms(DOMString resourceType, optional any resourceStatsOptions);

/**
* Remove all alarms.
*
* If an |alarmId| is provided, then only that alarm is removed.
*/
Promise removeAlarm(optional long alarmId);

/**
* Return available resource stats options stored in database.
*
* If success, the |result| is an array of resourceStatsOptions.
*/
Promise getResourceStatsOptions(DOMString resourceType);

/**
* Return the name of supported resource statistics types.
* Currently, we support 'network' and 'power'.
*/
readonly attribute any resourceTypes; // array of DOMString

/**
* Time in milliseconds between statistics stored in database.
*/
readonly attribute unsigned long long sampleRate;

/**
* Time in milliseconds recorded by the API until present time. All
* statistics older than maxStorageAge from now are deleted.
*/
readonly attribute unsigned long long maxStorageAge;
};
== Proposal End ==

We also filed a bug (Bug 951976) on Bugzilla [4] to discuss this API. Please feel free to provide any feedback you think is valuable on this thread or Bugzilla.

Thanks for your time and cheers!

--borting


[1] https://bugzilla.mozilla.org/show_bug.cgi?id=854200#c76
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=854200
[3] https://bugzilla.mozilla.org/show_bug.cgi?id=858003
[4] https://bugzilla.mozilla.org/show_bug.cgi?id=951976
[5] https://bugzilla.mozilla.org/show_bug.cgi?id=858005

--
Borting Chen
Intern of Mozilla Taiwan

[JOSE MANUEL CANTERA FONSECA]

El 20/12/13 10:44, "Borting Chen" <boc...@mozilla.com> escribió:

>Hi all:
>
>As we have discussed in [1], we need a new API to integrate the
>PowerStats API [2] with the existing NetworkStats

As these two dictionaries are quite similar it could be a good idea to
have only one dictionary for the two kind of resources.

>
>
>--borting
>
>
>[1] https://bugzilla.mozilla.org/show_bug.cgi?id=854200#c76
>[2] https://bugzilla.mozilla.org/show_bug.cgi?id=854200
>[3] https://bugzilla.mozilla.org/show_bug.cgi?id=858003
>[4] https://bugzilla.mozilla.org/show_bug.cgi?id=951976
>[5] https://bugzilla.mozilla.org/show_bug.cgi?id=858005
>
>--
>Borting Chen
>Intern of Mozilla Taiwan
>

>_______________________________________________
>dev-webapi mailing list
>dev-w...@lists.mozilla.org
>https://lists.mozilla.org/listinfo/dev-webapi
>



________________________________

Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo.
This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at:
http://www.tid.es/ES/PAGINAS/disclaimer.aspx

[Doug Turner]

On APIs that return urls and require you to go fetch data -- Why did you
take this approach? Where/How is that content result going to be defined?

Borting Chen wrote:
> Hi all:
>
> As we have discussed in [1], we need a new API to integrate the PowerStats API [2] with the existing NetworkStats API [3]. After an internal discussion at Taipei office (with Gene Lian, Vincent Chang, Ken Chang, and John Shih), we want to propose a 'ResourceStats API' [4] to merge these two stats APIs into one.
>
> The proposed ResourceStats API supports two primary functionality:
> (1) resource statistics (provide resource usage information, such as power consumption)
> (2) resource control (notify user when resource usage exceeds a defined threshold).
>
> The ResourceStats API categorizes a system into several types of resource. Currently, we plan to support statistics and control for two types of resource: 'network'& 'power', which focus on network usage and power consumption, respectively.
>
> The following is our proposal, note that:

> (1) getStats()& clearStats() are designed to support the query& delete for statistics. The |resourceStatsOptions| argument provides the detail of the resource we want to obtain/delete, such as the Wi-Fi usage of the browser App.

[Doug Turner]

On APIs that return urls and require you to go fetch data -- Why did you
take this approach? Where/How is that content result going to be defined?

Borting Chen wrote:

> Hi all:
>
> As we have discussed in [1], we need a new API to integrate the PowerStats API [2] with the existing NetworkStats API [3]. After an internal discussion at Taipei office (with Gene Lian, Vincent Chang, Ken Chang, and John Shih), we want to propose a 'ResourceStats API' [4] to merge these two stats APIs into one.
>
> The proposed ResourceStats API supports two primary functionality:
> (1) resource statistics (provide resource usage information, such as power consumption)
> (2) resource control (notify user when resource usage exceeds a defined threshold).
>
> The ResourceStats API categorizes a system into several types of resource. Currently, we plan to support statistics and control for two types of resource: 'network'& 'power', which focus on network usage and power consumption, respectively.
>
> The following is our proposal, note that:

> (1) getStats()& clearStats() are designed to support the query& delete for statistics. The |resourceStatsOptions| argument provides the detail of the resource we want to obtain/delete, such as the Wi-Fi usage of the browser App.

[Doug Turner]

On APIs that return urls and require you to go fetch data -- Why did you
take this approach? Where/How is that content result going to be defined?

Borting Chen wrote:

> Hi all:
>
> As we have discussed in [1], we need a new API to integrate the
> PowerStats API [2] with the existing NetworkStats API [3]. After an
> internal discussion at Taipei office (with Gene Lian, Vincent Chang,
> Ken Chang, and John Shih), we want to propose a 'ResourceStats API'
> [4] to merge these two stats APIs into one.
>
> The proposed ResourceStats API supports two primary functionality:
> (1) resource statistics (provide resource usage information, such as
> power consumption)
> (2) resource control (notify user when resource usage exceeds a
> defined threshold).
>
> The ResourceStats API categorizes a system into several types of
> resource. Currently, we plan to support statistics and control for two
> types of resource: 'network'& 'power', which focus on network usage
> and power consumption, respectively.
>
> The following is our proposal, note that:

> (1) getStats()& clearStats() are designed to support the query& delete

[Ehsan Akhgari]

On 12/20/2013, 7:40 AM, JOSE MANUEL CANTERA FONSECA wrote:
> El 20/12/13 10:44, "Borting Chen" <boc...@mozilla.com> escribi�:

>
>> Hi all:
>>
>> As we have discussed in [1], we need a new API to integrate the
>> PowerStats API [2] with the existing NetworkStats

> As these two dictionaries are quite similar it could be a good idea to
> have only one dictionary for the two kind of resources.

Yeah, perhaps something like:

enum ResourceType { "network", "power" };
enum PowerComponent { "CPU", /* other components go here */ };
dictionary ResourceOptions {
ResourceType type;
(PowerComponent or DOMString) component; // for network, "wifi" means
wifi, anything else is an iccid.
DOMString manifestURL;
};

Also, is ResourceStatsAlarmOptions.data a generic user data argument or
is it intended to be used for something else?

Thanks!
Ehsan

[Borting Chen]

Hi Doug:

Sorry, I don't understand your question. which APIs did you mean?

Thanks,

-- borting

--
Borting Chen
Intern of Mozilla Taiwan

----- Original Message -----
From: "Doug Turner" <do...@mozilla.com>
To: "Borting Chen" <boc...@mozilla.com>, mozilla-d...@lists.mozilla.org
Sent: Saturday, December 21, 2013 12:51:27 AM
Subject: Re: Proposal: ResourceStats API for NetworkStats API and PowerStats API integration

On APIs that return urls and require you to go fetch data -- Why did you
take this approach? Where/How is that content result going to be defined?

Borting Chen wrote:

> Hi all:
>
> As we have discussed in [1], we need a new API to integrate the
> PowerStats API [2] with the existing NetworkStats API [3]. After an
> internal discussion at Taipei office (with Gene Lian, Vincent Chang,
> Ken Chang, and John Shih), we want to propose a 'ResourceStats API'
> [4] to merge these two stats APIs into one.
>
> The proposed ResourceStats API supports two primary functionality:
> (1) resource statistics (provide resource usage information, such as
> power consumption)
> (2) resource control (notify user when resource usage exceeds a
> defined threshold).
>
> The ResourceStats API categorizes a system into several types of
> resource. Currently, we plan to support statistics and control for two
> types of resource: 'network'& 'power', which focus on network usage
> and power consumption, respectively.
>
> The following is our proposal, note that:

> (1) getStats()& clearStats() are designed to support the query& delete

[Borting Chen]

Hi Ehsan and Jose:

> As these two dictionaries are quite similar it could be a good idea to
> have only one dictionary for the two kind of resources.

enum ResourceType { "network", "power" };
enum PowerComponent { "CPU", /* other components go here */ };
dictionary ResourceOptions {
ResourceType type;
(PowerComponent or DOMString) component; // for network, "wifi" means
wifi, anything else is an iccid.
DOMString manifestURL;
};

Yeah, enum is a good idea, thanks.

I think provide a single ResourceOptions dictionary would be more generic, but there are some reasons we separate it as two:
(1) As we are going to add "serviceType" in the NetworkStatsOptions to query system-only network usage (plz refer Bug 992926 [1]), I am afraid the additional argument might confuse a user who want to obtain power stats if we provide ResourceOptions dictionary only.
(2) If we are going to support another types of resource stats (such as "memory") which requires different options, it might enlarge the ResourceOptions dictionary. So, I would like to provide each resource type a specific dictionary for option settings.

Also, is ResourceStatsAlarmOptions.data a generic user data argument or
is it intended to be used for something else?

Yes, the ResourceStatsAlarmOptions.data is a generic user data argument.

Thanks,

-- borting

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=922926

[Salvador de la Puente González]

Hello people

First of all, thank you John for adding Alberto and me to this IDL
proposal. I'm very happy about seeing NetworkStats API 2.0 as the base
for this new ResourceStats API. :)

Second, sorry for the delay. I was eager for contribute to this but my
workload has been huge until now. I'm on holidays but I would want to
provide some feedback before disconnecting completely.

Third my comments about the API inline and some conclusions and
alternative designs at the end:

On 20/12/13 10:44, Borting Chen wrote:

I need some clarification about the 'resource model'. As I understand so
far, it is this way:

The device is seen as a set of `resources` consumed through a set of
`components` by the execution of `applications`. I mean, `system app`
could be using `network` for FOTA through the `mobile component`.
Another example: `contacts app` could be spending `power` through the
use of `CPU`.

I assume there are more components compounding the power resource. Can
you cite some of them?
Am I correct, Am I missing something?

On 20/12/13 10:44, Borting Chen wrote:
> Hi all:
>
> As we have discussed in [1], we need a new API to integrate the PowerStats API [2] with the existing NetworkStats API [3]. After an internal discussion at Taipei office (with Gene Lian, Vincent Chang, Ken Chang, and John Shih), we want to propose a 'ResourceStats API' [4] to merge these two stats APIs into one.
>
> The proposed ResourceStats API supports two primary functionality:
> (1) resource statistics (provide resource usage information, such as power consumption)
> (2) resource control (notify user when resource usage exceeds a defined threshold).
>
> The ResourceStats API categorizes a system into several types of resource. Currently, we plan to support statistics and control for two types of resource: 'network' & 'power', which focus on network usage and power consumption, respectively.
>
> The following is our proposal, note that:

> (1) getStats() & clearStats() are designed to support the query & delete for statistics. The |resourceStatsOptions| argument provides the detail of the resource we want to obtain/delete, such as the Wi-Fi usage of the browser App.

> (2) addAlarm(), getAlarm(), and removeAlarm() are designed for resource control. These functions are retrieved from existing NetworkStats API directly. [5]
>
> == Proposal Start ==

> dictionary NetworkStatsOptions
> {
> /**
> * |type| is network type, 0 for Wi-Fi and 1 for Mobile.
> *
> * |id| is '0' for Wi-Fi, or the iccid for mobile (SIM).
> *
> * |manifestURL| is used to retrieve network usage of a specified application.
> * If null, system network usage is returned.
> */
> unsigned long type;
> DOMString id;
> DOMString manifestURL;
> };
>
> dictionary PowerStatsOptions
> {
> /**
> * |component| specifies which component's power consumption will be returned.
> * If null, total power consumption is returned.
> *
> * |manifestURL| is used to retrieve power consumption of a specified
> * application. If null, all applications' power consumption is returned.
> */
> DOMString component;
> DOMString manifestURL;
> };

As José Manuel already noticed, the two dictionaries are so similar that
they could be only one. If we want to equal both of them we could
provide a DOMString to refer the Wi-Fi/mobile interfaces and SIM
combinations instead of type and id. I'm thinking in some like:
'wifi:0' and 'mobile:<iccid>'.

This way, it's easy to parse and help to unify both dictionaries and
same schema can be easy applied to other multiplexed resources as CPU
for multi-core devices.

>
> dictionary ResourceStatsAlarmOptions
> {
> unsigned long long startTime; // time in milliseconds since Epoch
> any data;
> };

Just to clarify, `startTime` is a convenient way to place an offset from
which perform the measurements (i.e: count power consumption of CPU
since now) and `data` play the same role as in the Alarm API: to be
reflected in the alarm object when the alarm is triggered.

>
> [JSImplementation="@mozilla.org/resourceStatsAlarm;1", Pref="dom.mozResourceStats.enabled"]
> interface MozResourceStatsAlarm
> {
> readonly attribute unsigned long alarmId;
> readonly attribute DOMString resourceType;
> readonly attribute any resourceStatsOptions;
> readonly attribute any threshold;
> readonly attribute any data;
> };
>
> [JSImplementation="@mozilla.org/resourceStatsManager;1", NavigatorProperty="mozResourceStats", Pref="dom.mozResourceStats.enabled"]
> interface ResourceStatsManager
> {
> const unsigned long WIFI = 0;
> const unsigned long MOBILE = 1;

Note we have these constants here completely unrelated to power resource. To
remove this coupling, read my alternative proposal at the end of this
document.

>
> /**
> * Query resource statistics.
> *
> * |resourceType| specifies the type of resource statistics we want to
> * obtain. This value should be in the return of resourceTypes attribute.
> *
> * |start| and |end| specifies the interest time range, both included.
> *
> * |resourceStatsOptions| specifies the detial of resource in which we are
> * are interested. Its value depends on which |resourceType| is specified.
> * For 'network' resource, the options should be a NetworkStatsOptions object.
> * For 'power' resource, the options should be a PowerStatsOptions object.
> *
> * If success, the |result| field keeps statistics. For 'network' resource
> * the |result| is an array of NetworkStats object. For 'power' resource
> * the result is an array of PowerStats object.
> */
> Promise getStats(DOMString resourceType,
> unsigned long long start,
> unsigned long long end,
> optional any resourceStatsOptions);

It could be very convenient to allow `null` or `undefined` for `start`
and `end` to retrieve statistics from the beginning of measurements or
until the end of them.

>
> /**
> * Clear resource statistics stored in database.
> *
> * |resourceType| specifies the type of resource statistics we want to clear.
> *
> * If |resourceStatsOptions| is provided, then only the statistics associated
> * to that |resourceStatsOptions| is deleted. Otherwise, all statistics of
> * the specified |resourceType| is deleted.
> */
> Promise clearStats(DOMString resourceType,
> optional any resourceStatsOptions);

IMHO, it is safer to provide something like `clearAllStats()` if we
really want to clear all statistics. These kind of so much homogeneous
API lead the developer to perform a lot of automatisms which can result
in an unintended undefined object. If this is passed to
`clearAllStats()` the result could be fatale. Furthermore, this forces
you to test the second argument before using this method safely
introducing a lot of noisy `if` statements everywhere.

To be consistent with my last comment, I recommend the same approach
here: to have a `removeAllAlarms()`.

>
> /**
> * Return available resource stats options stored in database.
> *
> * If success, the |result| is an array of resourceStatsOptions.
> */
> Promise getResourceStatsOptions(DOMString resourceType);

I don't understand this very well, could you explain me, please?

>
> /**
> * Return the name of supported resource statistics types.
> * Currently, we support 'network' and 'power'.
> */
> readonly attribute any resourceTypes; // array of DOMString
>
> /**
> * Time in milliseconds between statistics stored in database.
> */
> readonly attribute unsigned long long sampleRate;

Maybe we should transform this into a function or a map as we could be
measuring each resource at different rates.

>
> /**
> * Time in milliseconds recorded by the API until present time. All
> * statistics older than maxStorageAge from now are deleted.
> */
> readonly attribute unsigned long long maxStorageAge;
> };
>

Same here.

So, now, my alternative approach is to have to have a Manager Factory
and allow the developer to instantiate a specific Manager for a specific
resource. I.e.:

var networkStatsManager = new ResourceStatsManager('network');
var powerStatsManager = new ResourceStatsManager('power');
var resources = ResourceStatsManager.getResources();

We should try to keep the API as homogeneous as possible but this allow
us to provide very specific methods for some of them if needed or
particular constants like WIFI or MOBILE applying only where they are
required.

What do you think?

Thank you guys!

[Ehsan Akhgari]

On 12/20/2013, 7:24 PM, Borting Chen wrote:
> Hi Ehsan and Jose:

>> As these two dictionaries are quite similar it could be a good idea to
>> have only one dictionary for the two kind of resources.
>
> enum ResourceType { "network", "power" };
> enum PowerComponent { "CPU", /* other components go here */ };
> dictionary ResourceOptions {
> ResourceType type;
> (PowerComponent or DOMString) component; // for network, "wifi" means
> wifi, anything else is an iccid.
> DOMString manifestURL;
> };
>
> Yeah, enum is a good idea, thanks.
>
> I think provide a single ResourceOptions dictionary would be more generic, but there are some reasons we separate it as two:
> (1) As we are going to add "serviceType" in the NetworkStatsOptions to query system-only network usage (plz refer Bug 992926 [1]), I am afraid the additional argument might confuse a user who want to obtain power stats if we provide ResourceOptions dictionary only.

Hmm, why would we need to change the API for that purpose? Can't you
just pass null for manifestURL to get the system network/power usage?

> (2) If we are going to support another types of resource stats (such as "memory") which requires different options, it might enlarge the ResourceOptions dictionary. So, I would like to provide each resource type a specific dictionary for option settings.

Do you expect the memory stats API to require a lot of additional
dictionary members? (I don't.) If not, let's not worry about that case
for now. Using only a single dictionary here will hopefully make people
think twice before adding members specific to only one measurement to
this dictionary.

Cheers,
Ehsan

[Salvador de la Puente González]

Hello guys

>> (2) If we are going to support another types of resource stats (such
>> as "memory") which requires different options, it might enlarge the
>> ResourceOptions dictionary. So, I would like to provide each resource
>> type a specific dictionary for option settings.
>
> Do you expect the memory stats API to require a lot of additional
> dictionary members? (I don't.) If not, let's not worry about that
> case for now. Using only a single dictionary here will hopefully make
> people think twice before adding members specific to only one
> measurement to this dictionary.

Please, consider the option I gave you about having a Statistics factory
and specialize for each service because it could solve these problems.

Cheers.
>
> Cheers,
> Ehsan

>
> _______________________________________________
> dev-webapi mailing list
> dev-w...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-webapi
>
>

[Ehsan Akhgari]

On 1/3/2014, 11:08 AM, Salvador de la Puente Gonz�lez wrote:
> Hello guys
>>> (2) If we are going to support another types of resource stats (such
>>> as "memory") which requires different options, it might enlarge the
>>> ResourceOptions dictionary. So, I would like to provide each resource
>>> type a specific dictionary for option settings.
>>
>> Do you expect the memory stats API to require a lot of additional
>> dictionary members? (I don't.) If not, let's not worry about that
>> case for now. Using only a single dictionary here will hopefully make
>> people think twice before adding members specific to only one
>> measurement to this dictionary.
> Please, consider the option I gave you about having a Statistics factory
> and specialize for each service because it could solve these problems.

I assume you mean this proposal?

var networkStatsManager = new ResourceStatsManager('network');
var powerStatsManager = new ResourceStatsManager('power');
var resources = ResourceStatsManager.getResources();

I was waiting for Borting to reply to your post first, but as far as I
can understand, ResourceStatsManager would still accept the same
arguments and return the same type here for all resource types, so it
shold be no better or worse in terms of being future proof for other
resource types, unless I'm missing something.

Cheers,
Ehsan

[Borting Chen]

> I need some clarification about the 'resource model'. As I understand so
> far, it is this way:
> The device is seen as a set of `resources` consumed through a set of
> `components` by the execution of `applications`. I mean, `system app`
> could be using `network` for FOTA through the `mobile component`.
> Another example: `contacts app` could be spending `power` through the
> use of `CPU`.
> I assume there are more components compounding the power resource. Can
> you cite some of them?
> Am I correct, Am I missing something?

Yes, you are right.
Here is the list of components which we are interested:
- CPU
- Screen
- WiFi
- Mobile
- GPS
- Bluetooth
- Audio/Video

> As José Manuel already noticed, the two dictionaries are so similar that
> they could be only one. If we want to equal both of them we could
> provide a DOMString to refer the Wi-Fi/mobile interfaces and SIM
> combinations instead of type and id. I'm thinking in some like:
> 'wifi:0' and 'mobile:<iccid>'.

This is a good idea. Thanks for the suggestion.

> > Promise getStats(DOMString resourceType,
> > unsigned long long start,
> > unsigned long long end,
> > optional any resourceStatsOptions);
> It could be very convenient to allow `null` or `undefined` for `start`
> and `end` to retrieve statistics from the beginning of measurements or
> until the end of them.

okay, btw, is that possible to use 0 to indicate the begin and end of measurements?

> > Promise clearStats(DOMString resourceType,
> > optional any resourceStatsOptions);
> IMHO, it is safer to provide something like `clearAllStats()` if we
> really want to clear all statistics. These kind of so much homogeneous
> API lead the developer to perform a lot of automatisms which can result
> in an unintended undefined object. If this is passed to
> `clearAllStats()` the result could be fatale. Furthermore, this forces
> you to test the second argument before using this method safely
> introducing a lot of noisy `if` statements everywhere.

So, you meant that it would be better to provide the following APIs:

Promise clearStats(DOMString resourceType, optional any resourceStatsOptions);

Promise clearAllStats(DOMString resourceType);

Am I right?

> > Promise getResourceStatsOptions(DOMString resourceType);
> I don't understand this very well, could you explain me, please?

Oh, this API is created to support getAvailableNetworks() of NetworkStatsManager.

For network stats, it would return available networks stored in DB.

For power stats, it would return the component supporting power metering.

> > readonly attribute unsigned long long sampleRate;
> Maybe we should transform this into a function or a map as we could be
> measuring each resource at different rates.

I think it would become much easier for implementation if we can measure resources at different rates.
But, because calculating the power consumption of wifi and mobile is based on network stats results (such as Tx/RX bytes),
it would be more efficient if we can save the network and the power stats to DB at the same time.

> So, now, my alternative approach is to have to have a Manager Factory
> and allow the developer to instantiate a specific Manager for a specific
> resource. I.e.:
>

> var networkStatsManager = new ResourceStatsManager('network');
> var powerStatsManager = new ResourceStatsManager('power');
> var resources = ResourceStatsManager.getResources();
>

> We should try to keep the API as homogeneous as possible but this allow
> us to provide very specific methods for some of them if needed or
> particular constants like WIFI or MOBILE applying only where they are
> required.

Thanks for the suggestion.
So, we can remove the |resourceType| parameter from all APIs, am I right?

[Borting Chen]

> Hmm, why would we need to change the API for that purpose? Can't you
> just pass null for manifestURL to get the system network/power usage?

okay, I see.
btw, because we may need to support the query for different service type [1], could we define something like these for the |manifestURL| variable?
1. "app://..." represents an app's manifestURL
2. "sys://..." represents a system service type
3. null means get the total usage

Here are some example for querying system services:
"sys://FOTA" means query the usage for FOTA
"sys://Tethering" means query the usage for Tethering

> > (2) If we are going to support another types of resource stats (such as
> > "memory") which requires different options, it might enlarge the
> > ResourceOptions dictionary. So, I would like to provide each resource type
> > a specific dictionary for option settings.
>
> Do you expect the memory stats API to require a lot of additional
> dictionary members? (I don't.) If not, let's not worry about that case
> for now. Using only a single dictionary here will hopefully make people
> think twice before adding members specific to only one measurement to
> this dictionary.

Yak, you are right.
So I think the ResourceStatsOptions could be something like this:

dictionary ResourceStatsOptions {
/**
* For Power stats, |component| specifies which component's power consumption will be returned.

* If null, total power consumption is returned.
*

* For Network stats, |component| specifies the network interface. WiFi is repreented as "wifi:0"
* and mobile is represented as "mobile:<iccid>".
* If null, system network usage (wifi + mobile) is returned.
*/
DOMString component;

/**
* If |manifestURL| is "app://...", then the stats of an application is returned.
*
* If |manifestURL| is "sys://...", then the stats of a system service is returned.
*
* If |manifestURL| is null, then the total resource usage (app + system) is returned.
*/
DOMString manifestURL;
};

Does this look better?

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=922926

Cheers,

-- borting

[Ehsan Akhgari]

On 1/3/2014, 12:13 PM, Borting Chen wrote:
>> Hmm, why would we need to change the API for that purpose? Can't you
>> just pass null for manifestURL to get the system network/power usage?
>
> okay, I see.
> btw, because we may need to support the query for different service type [1], could we define something like these for the |manifestURL| variable?
> 1. "app://..." represents an app's manifestURL
> 2. "sys://..." represents a system service type
> 3. null means get the total usage

I would rather avoid inventing a new URI scheme for this purpose.

> Here are some example for querying system services:
> "sys://FOTA" means query the usage for FOTA
> "sys://Tethering" means query the usage for Tethering

Alternatively, we can add components for these purposes, such as:

enum NetworkComponent { "wifi", "tethering", "OTA" };
dictionary ResourceOptions {
ResourceType type;
(PowerComponent or NetworkComponent or DOMString) component; // for
network, arbitrary strings represent an iccid
DOMString manifestURL;
};

That way, you'll pass {type: "network", component: "tethering"} to
measure the tethering usage, for example. If you pass a manifestURL,
you'll measure the tethering usage for that particular app. If you pass
a manifestURL to an app that is not the system app and specify
component:"OTA" then you'll get zero usage results as non-system apps do
not handle updates, for example.

Do you think this proposal addresses your needs here?

>>> (2) If we are going to support another types of resource stats (such as
>>> "memory") which requires different options, it might enlarge the
>>> ResourceOptions dictionary. So, I would like to provide each resource type
>>> a specific dictionary for option settings.
>>
>> Do you expect the memory stats API to require a lot of additional
>> dictionary members? (I don't.) If not, let's not worry about that case
>> for now. Using only a single dictionary here will hopefully make people
>> think twice before adding members specific to only one measurement to
>> this dictionary.
>
> Yak, you are right.
> So I think the ResourceStatsOptions could be something like this:
>
> dictionary ResourceStatsOptions {
> /**
> * For Power stats, |component| specifies which component's power consumption will be returned.
> * If null, total power consumption is returned.
> *
> * For Network stats, |component| specifies the network interface. WiFi is repreented as "wifi:0"
> * and mobile is represented as "mobile:<iccid>".
> * If null, system network usage (wifi + mobile) is returned.
> */
> DOMString component;
>

I still prefer to use a union of enums and DOMStrings for the iccid case.

> /**
> * If |manifestURL| is "app://...", then the stats of an application is returned.
> *
> * If |manifestURL| is "sys://...", then the stats of a system service is returned.
> *
> * If |manifestURL| is null, then the total resource usage (app + system) is returned.
> */
> DOMString manifestURL;
> };

See above.

Cheers,
Ehsan

[Borting Chen]

> I would rather avoid inventing a new URI scheme for this purpose.
>
> > Here are some example for querying system services:
> > "sys://FOTA" means query the usage for FOTA
> > "sys://Tethering" means query the usage for Tethering
>
> Alternatively, we can add components for these purposes, such as:
>
> enum NetworkComponent { "wifi", "tethering", "OTA" };
> dictionary ResourceOptions {
> ResourceType type;
> (PowerComponent or NetworkComponent or DOMString) component; // for
> network, arbitrary strings represent an iccid
> DOMString manifestURL;
> };
>
> That way, you'll pass {type: "network", component: "tethering"} to
> measure the tethering usage, for example. If you pass a manifestURL,
> you'll measure the tethering usage for that particular app. If you pass
> a manifestURL to an app that is not the system app and specify
> component:"OTA" then you'll get zero usage results as non-system apps do
> not handle updates, for example.
>
> Do you think this proposal addresses your needs here?

Thank you for the suggestion, but I think it is a little weird for me to use the same variable to represent "tethering" and "wifi".
In my opinion, the wifi is a component which "provides" network resources,
but tethering (or OTA) is a service which "consumes" network resources (as well as power resources).
So, I think it could be better if we can use separated variables to represent those have different characteristics.
Even more, tethering may transmit packets to Internet via different SIMs (on a multi-SIM phone).
So, providing a single "component" variable is not sufficient if we want to know how many packets that a SIM transmits for tethering.

Hence, my alternative proposal is something like this:

enum PowerComponent {"CPU", "Scrren", "WiFi", "Mobile", "Bluetooth", "GPS", ...};
enum NetworkComponent {"WiFi", ...};
enum SystemService {"OTA", "Tethering", ...}; // represent system services

dictionary ResourceOptions {
ResourceType type; // this could be removed if we decided to use the factory design suggested by Salvador

(PowerComponent or NetworkComponent or DOMString) component; // for network, arbitrary strings represent an iccid

(SystermService or DOMString) task; // arbitrary strings represent an manifestURL
};

Does this look much better?



Cheers,

-- borting

[Gene Lian]

Please see the in-line response.

> Thank you for the suggestion, but I think it is a little weird for me to use
> the same variable to represent "tethering" and "wifi".

Agreed. In the long term, we can still query the network stats like:

"How much traffic passes through WiFi when doing the Tethering service?"

IMO, we shouldn't assume WiFi and Tethering are exclusive.

> enum PowerComponent {"CPU", "Scrren", "WiFi", "Mobile", "Bluetooth", "GPS",
> ...};
> enum NetworkComponent {"WiFi", ...};
> enum SystemService {"OTA", "Tethering", ...}; // represent system services
>
> dictionary ResourceOptions {
> ResourceType type; // this could be removed if we decided to use the
> factory design suggested by Salvador
> (PowerComponent or NetworkComponent or DOMString) component; // for
> network, arbitrary strings represent an iccid
> (SystermService or DOMString) task; // arbitrary strings represent an
> manifestURL

Here comes the similar issue. Querying the following combination is
still imaginable:

"How much traffic has App A used to do the Tethering?" and
"How much traffic has App B used to do the Tethering?"

which means it's inconsiderate to keep (SystemService or DOMString)
by one attribute.

Based on the previous idea, I'd suggest:

----------------------------------------
enum ResourceType {"power", "network"}
enum PowerComponent {"cpu", "screen", "wifi", "bluetooth", ...}
enum NetworkComponent {"wifi", ...};
enum SystemService {"unknown", "ota", "tethering", ...};

dictionary ResourceOptions {
ResourceType resourceType;
(PowerComponent or NetworkComponent or DOMString) component; [1]
SystemService systemService;
DOMString appManifestURL;
}

[1] DOMString is an ICC ID to specify SIM if we want to query
the mobile network.
----------------------------------------

Note that we make component/systemService/appManifestURL separate
so we're allowed to query the following user cases, which are
all common and reasonable:

Example 1: "How much traffic has App A used to do the Tethering
through the SIM-1's 3G?"

=> { resourceType: "network",
component: "89886920...",
systemService: "tethering",
appManifestURL: "app://A" }

Example 2: "How much traffic has App B used to do the OTA update
through the WiFi?"

=> { resourceType: "network",
component: "wifi",
systemService: "ota",
appManifestURL: "app://B" }

Example 3: "How much power has App C consumed to do the Bluetooth-
related connections?"

=> { resourceType: "power",
component: "bluetooth",
systemService: "unknown",
appManifestURL: "app://C" }


Hope it makes sense.

Cheers,
Gene

[Borting Chen]

yah, I think this can meet all possible queries, thanks a lot.

BTW, a little question. Why you suggest an "unknown" type in systemService?
Does this mean "misc" system services?



Cheers,
Borting

[Gene Lian]

----- Original Message -----
> From: "Borting Chen" <boc...@mozilla.com>
> To: "Gene Lian" <cl...@mozilla.com>
> Cc: "Ehsan Akhgari" <ehsan....@gmail.com>, "JOSE MANUEL CANTERA FONSECA" <jm...@tid.es>,
> dev-w...@lists.mozilla.org
> Sent: Thursday, January 9, 2014 11:47:42 AM
> Subject: Re: Proposal: ResourceStats API for NetworkStats API and PowerStats API integration
>

Please see the above-mentioned example 3: what if we don't care
about the system service? That is, we only want to query the power
consumption based on a certain app. So I define a "unknown" to
sum up all types of system service. TBH, I'm not sure if we really
need that. Maybe setting |systemService = null| is OK?

The same issue also occurs in |appManifestURL|. Since its a
DOMString, we can assign null to it to specify we want to query
the network traffic or power consumption for all apps.

Gene

[Borting Chen]

----- Original Message -----
> From: "Gene Lian" <cl...@mozilla.com>
> To: "Borting Chen" <boc...@mozilla.com>
> Cc: "Ehsan Akhgari" <ehsan....@gmail.com>, "JOSE MANUEL CANTERA FONSECA" <jm...@tid.es>,
> dev-w...@lists.mozilla.org
> Sent: Thursday, January 9, 2014 5:50:27 PM
> Subject: Re: Proposal: ResourceStats API for NetworkStats API and PowerStats API integration
>

> Please see the above-mentioned example 3: what if we don't care
> about the system service? That is, we only want to query the power
> consumption based on a certain app. So I define a "unknown" to
> sum up all types of system service. TBH, I'm not sure if we really
> need that. Maybe setting |systemService = null| is OK?
>
> The same issue also occurs in |appManifestURL|. Since its a
> DOMString, we can assign null to it to specify we want to query
> the network traffic or power consumption for all apps.

I see. I think maybe we can left |systemService| as undefined when querying app's resource stats.
I'll check this during implementation. thanks for the suggestion.



Cheers,
Borting

[Borting Chen]

Hi all:

Thank you guys for the great suggestions.
The revised ResourceStats API proposal is attached at the end of this mail.
Please left your comments if anyone has other thoughts.
Thanks a lot!

Modification note:
1. Use the factory design as suggested by Salvador. So, I separate the original interface to two:
-- ResourceStatsFactory is used to instantiate a ResourceStatsManger for a specific resource
-- ResourceStatsManager is used to query statistics and add resource alarms
2. Merge PowerStatsOptions and NetworkStatsOptions to one ResourceStatsOptions dictionary.
And, define several enums to represent resource types, components and system services.
3. Provide methods to get/clear "all" stats/alarms, which includes:
clearAllStats, getAllAlarms, and removeAllAlarm.

== Proposal Start ==

/**
* Supported resource statistics
*/

enum ResourceType {
"network",
"power"
};

/**
* List of components supporting power statistics
*/

enum PowerComponent {
"cpu",
"screen",

"mobile",
"wifi",
"bluetooth",
"gps"
};

/**
* List of components supporting network statistics.
*
* Note that a query for mobile network usage should use a DOMString to specify
* the ICC ID of the SIM card rather than specify a NetworkComponent enum type.
*/
enum NetworkComponent {
"wifi"
};

/**
* List of system services supporting resource statistics
*/
enum SystemService {
"ota",
"tethering"
};

typedef (PowerComponent or NetworkComponent or DOMString) ResourceStatsComponent;

dictionary ResourceStatsOptions
{
/**
* For power stats, |component| specifies which component's power consumption
* will be returned. If null, total power consumption is returned.
*
* For network stats, |component| specifies the network interface. DOMString
* is used to specify an ICC ID when querying the usage of mobile network.
* To query the stats of other network interfaces, a NetworkComponent enum
* type should be specified. If null, total network usage (Wi-Fi + mobile) is
* returned.
*/
ResourceStatsComponent component;

/**
* |manifestURL| specifies the manifestURL of an application.
* |systemService| specifies the system service.
*
* If both |systemService| and |manifestURL| are null, then a system-wide
* resource statistics is returned.
*
* If |manifestURL| is specified, then the resource statistics of the
* specified application is returned.
*
* If |systemService| is specified, then the resource statistics of the
* specified system service is returned.
*
* If both |systemService| and |manifestURL| are specified, then the return
* statistics indicates the resources that the system service consumed for
* the application.
*/
SystemService systemService;
DOMString manifestURL;
};

dictionary ResourceStatsAlarmOptions
{
/**
* |startTime| indicates the start time of counting the resource usage.
*
* |data| is used to reflect in the alarm object when the alarm is triggered.
*/

unsigned long long startTime; // time in milliseconds since Epoch
any data;
};

[JSImplementation="@mozilla.org/resourceStatsAlarm;1", Pref="dom.mozResourceStats.enabled"]
interface MozResourceStatsAlarm
{
readonly attribute unsigned long alarmId;

readonly attribute ResourceType type;
readonly attribute any statsOptions; // ResourceStatsOptions

readonly attribute any threshold;
readonly attribute any data;
};

[Constructor(ResourceType type), JSImplementation="@mozilla.org/resourceStatsManager;1", Pref="dom.mozResourceStats.enabled"]
interface ResourceStatsManager
{

/**
* Query resource statistics.
*

* |start| and |end| specifies the time range of interest, both included.
* If |start| is 0, retrieves the stats since measurements.
* If |end| is 0. retrieves the stats until the current time.
*
* |statsOptions| specifies the detial of statistics of interest.

*
* If success, the |result| field keeps statistics.

* For networkStatsManager, the |result| is an array of NetworkStats object.
* For powerStatsManager, the |result| is an array of PowerStats object.
*/
Promise getStats(unsigned long long start,
unsigned long long end,
optional ResourceStatsOptions statsOptions);

/**
* Clear resource statistics stored in database.
*

* If |statsOptions| is provided, then only the statistics associated to that
* |statsOptions| is deleted. Otherwise, all statistics of this resource type
* is deleted.
*/
Promise clearStats(optional ResourceStatsOptions statsOptions);

/**
* Clear all resource statistics stored in database.
*/
Promise clearAllStats();

/**
* Install an alarm to monitor resource usage.
*

* The |threshold| is an object specified the limit of resource usage. When
* resource usage reaches the threshold, a "resourceStats-alarm" system
* message is sent to the application.
*

* |statsOptions| specifies the detial of statistics of interest.

*
* |alarmOptions| is a ResourceStatsAlarmOptions object.
*
* If success, the |result| field keeps the alarm Id.
*/

Promise addAlarm(any threshold,
optional ResourceStatsOptions statsOptions,
optional ResourceStatsAlarmOptions alarmOptions);

/**
* Obtain alarms.
*
* If |statsOptions| is provided, then only the alarms monitoring that
* resource are returned. Otherwise, all alarms set for this resource type
* is returned.
*

* If success, the |result| is an array of ResourceStatsAlarm.
*/

Promise getAlarms(optional ResourceStatsOptions statsOptions);

/**
* Obtain all alarms.
*

* If success, the |result| is an array of ResourceStatsAlarm.
*/

Promise getAllAlarms();

/**
* Remove alarms.

*
* If an |alarmId| is provided, then only that alarm is removed.
*/
Promise removeAlarm(optional long alarmId);

/**
* Remove all alarms.

*/
Promise removeAllAlarm();

/**
* Return available resource stats options stored in database.
*

* If success, the |result| is an array of ResourceStatsOptions.
*/
Promise getResourceStatsOptions();

/**
* Return the type of resource statistics provided by this manager.
*/
readonly attribute DOMString resourceType;
};

[JSImplementation="@mozilla.org/resourceStatsFactory;1", NavigatorProperty="mozResourceStats", Pref="dom.mozResourceStats.enabled"]
interface ResourceStatsFactory
{
/**
* Get the resourceStatsManager for a resource type.
*
* |type| specifies the type of resourceStatsManager we want to obtain.
* This value should be one of those returned by resourceTypes attribute.
*
* If success, this method returns an object of ResourceStatsManager.
* Otherwise, null is resturned
*/
ResourceStatsManager getResourceStatsManager(ResourceType type);

/**
* Return the name of supported resource statistics types.

* Currently, 'network' and 'power' statistics are supported.

*/
readonly attribute any resourceTypes; // array of DOMString

/**
* Time in milliseconds between statistics stored in database.
*/

readonly attribute unsigned long long sampleRate;

/**
* Time in milliseconds recorded by the API until present time. All
* statistics older than maxStorageAge from now are deleted.
*/

readonly attribute unsigned long long maxStorageAge;
};

== Proposal End ==

Cheers,
Borting

[Salvador de la Puente González]

Hello Borting, Gene, Ehsan and all!

First, happy new year!

Second: Borting, you got all at first glance! (I was on vacations so I
was not able to answer before).

Third, I want to review the abstraction before commenting in-line:

I like how the API is evolving because we can model the device as a set
of resources (first dimension) to be consumed through components (a
sub-dimension of the resource) by a set of application (second
dimension) optionally using some system's service (third dimension).
Examples provided by Gene illustrate this metaphor very well.

The only coupling is between resources and components. By using the
factory we specialize the manager for a specific resource. Now each
dimension (component, application and system's service) is not dependant
of other and has a slot in the ResourceOptions so we have a very
orthogonal, homogeneous and easy to automatize API. The absence of any
of these dimension should be treated as the addition of all it. So, no
manifestURL means "all the applications"; no systemService means "any
system's service"; no component means "all components together" and so on.

Before commenting the new API proposal I want to remark we have two
resources: network and wi-fi but I would want to introduce another one
(telephony: counting calls, SMS, MMS...). I will give you the details in
another thread ,)

And now:

On 09/01/14 16:12, Borting Chen wrote:
> Hi all:
>
> Thank you guys for the great suggestions.
> The revised ResourceStats API proposal is attached at the end of this mail.
> Please left your comments if anyone has other thoughts.
> Thanks a lot!
>
> Modification note:
> 1. Use the factory design as suggested by Salvador. So, I separate the original interface to two:
> -- ResourceStatsFactory is used to instantiate a ResourceStatsManger for a specific resource
> -- ResourceStatsManager is used to query statistics and add resource alarms
> 2. Merge PowerStatsOptions and NetworkStatsOptions to one ResourceStatsOptions dictionary.
> And, define several enums to represent resource types, components and system services.
> 3. Provide methods to get/clear "all" stats/alarms, which includes:
> clearAllStats, getAllAlarms, and removeAllAlarm.
>
> == Proposal Start ==
>
> /**
> * Supported resource statistics
> */
> enum ResourceType {
> "network",
> "power"
> };

As I said before, I will send a proposal for counting telephony statistics.

>
> /**
> * List of components supporting power statistics
> */
> enum PowerComponent {
> "cpu",
> "screen",
> "mobile",
> "wifi",
> "bluetooth",
> "gps"
> };
>
> /**
> * List of components supporting network statistics.
> *
> * Note that a query for mobile network usage should use a DOMString to specify
> * the ICC ID of the SIM card rather than specify a NetworkComponent enum type.
> */
> enum NetworkComponent {
> "wifi"
> };

I would rather prefer to pass the string with the
<component>:<subcomponent> format as it is more generic. Or a complex
object combination of enum (wifi / mobile) and identifier (interface
name / iccid). Note now, in our devices there are no more than one Wi-Fi
interface but this could change. Just my opinion, no more.

>
> /**
> * List of system services supporting resource statistics
> */
> enum SystemService {
> "ota",
> "tethering"
> };
>
> typedef (PowerComponent or NetworkComponent or DOMString) ResourceStatsComponent;
>
> dictionary ResourceStatsOptions
> {
> /**
> * For power stats, |component| specifies which component's power consumption
> * will be returned. If null, total power consumption is returned.

I encourage to be null or undefined (may this is implicit, just to
remark) in order to avoid including it in the JS object.

I was thinking 0 is perfectly valid for the 'beginning of time' but 0
makes no sense meaning "till the end of time". Furthermore, [0,0] is the
set containing only 0 so it is a valid value. For the sake of
correctness, if we could accept -inf and +inf it would be perfect and
they are valid values JS' numbers. If not, I suggest to specify
enumerates such as BEGIN and END or, in the less semantic option: null /
undefined.

> *
> * |statsOptions| specifies the detial of statistics of interest.
> *
> * If success, the |result| field keeps statistics.
> * For networkStatsManager, the |result| is an array of NetworkStats object.
> * For powerStatsManager, the |result| is an array of PowerStats object.
> */
> Promise getStats(unsigned long long start,
> unsigned long long end,
> optional ResourceStatsOptions statsOptions);
>
> /**
> * Clear resource statistics stored in database.
> *
> * If |statsOptions| is provided, then only the statistics associated to that
> * |statsOptions| is deleted. Otherwise, all statistics of this resource type
> * is deleted.
> */
> Promise clearStats(optional ResourceStatsOptions statsOptions);

As we are providing the next `clearAllStats()` method, the parameter is
no longer optional and the explanation about it, when omitted should be
moved to the next method.

>
> /**
> * Clear all resource statistics stored in database.
> */
> Promise clearAllStats();

This clear all stats of the resource type as I said in my previous comment.

Although I like this, note getting is not a critical functionality (we
can not destroy any information with a get operation) so, according to
the getAlarms() documentation, we are providing here a redundant way to
get all the alarms.

>
> /**
> * Remove alarms.
> *
> * If an |alarmId| is provided, then only that alarm is removed.
> */
> Promise removeAlarm(optional long alarmId);

Here we have the opposite example. As we are destroying things, we
should make mandatory the parameter and use the next method,
removeAllAlarms(), if we want to short-cut the method and destroy all
the alarms.

May be we can reduce verbosity by renaming this method to getManager().

>
> /**
> * Return the name of supported resource statistics types.
> * Currently, 'network' and 'power' statistics are supported.
> */
> readonly attribute any resourceTypes; // array of DOMString
>
> /**
> * Time in milliseconds between statistics stored in database.
> */
> readonly attribute unsigned long long sampleRate;
>
> /**
> * Time in milliseconds recorded by the API until present time. All
> * statistics older than maxStorageAge from now are deleted.
> */
> readonly attribute unsigned long long maxStorageAge;
> };
>
> == Proposal End ==
>
> Cheers,
> Borting
>
>

Hope it makes sense!
Thank you very much.

[Ehsan Akhgari]

On 1/9/2014, 11:27 AM, Salvador de la Puente González wrote:
> Hello Borting, Gene, Ehsan and all!
>
> First, happy new year!

Same to you!

> Second: Borting, you got all at first glance! (I was on vacations so I
> was not able to answer before).
>
> Third, I want to review the abstraction before commenting in-line:
>
> I like how the API is evolving because we can model the device as a set
> of resources (first dimension) to be consumed through components (a
> sub-dimension of the resource) by a set of application (second
> dimension) optionally using some system's service (third dimension).
> Examples provided by Gene illustrate this metaphor very well.

Yes, agreed.

> The only coupling is between resources and components. By using the
> factory we specialize the manager for a specific resource. Now each
> dimension (component, application and system's service) is not dependant
> of other and has a slot in the ResourceOptions so we have a very
> orthogonal, homogeneous and easy to automatize API. The absence of any
> of these dimension should be treated as the addition of all it. So, no
> manifestURL means "all the applications"; no systemService means "any
> system's service"; no component means "all components together" and so on.
>
> Before commenting the new API proposal I want to remark we have two
> resources: network and wi-fi but I would want to introduce another one
> (telephony: counting calls, SMS, MMS...). I will give you the details in
> another thread ,)

That would be interesting!

> And now:
>
> On 09/01/14 16:12, Borting Chen wrote:
>> Hi all:
>>
>> Thank you guys for the great suggestions.
>> The revised ResourceStats API proposal is attached at the end of this
>> mail.
>> Please left your comments if anyone has other thoughts.
>> Thanks a lot!
>>
>> Modification note:
>> 1. Use the factory design as suggested by Salvador. So, I separate the
>> original interface to two:
>> -- ResourceStatsFactory is used to instantiate a
>> ResourceStatsManger for a specific resource
>> -- ResourceStatsManager is used to query statistics and add
>> resource alarms
>> 2. Merge PowerStatsOptions and NetworkStatsOptions to one
>> ResourceStatsOptions dictionary.
>> And, define several enums to represent resource types, components
>> and system services.
>> 3. Provide methods to get/clear "all" stats/alarms, which includes:
>> clearAllStats, getAllAlarms, and removeAllAlarm.

FWIW these modifications look good overall.

Yeah I think at this point, NetworkComponet doesn't make sense as a
separate enum any more, we might as well just use a DOMString instead.
I think a "<component>:<subcomponent>" identifier makes sense, or we can
choose to be more verbose and convert that into its own small
dictionary. Also, the multiple WiFi extensibility in the future is a
great point, so we should take that into account here as well.

>> /**
>> * List of system services supporting resource statistics
>> */
>> enum SystemService {
>> "ota",
>> "tethering"
>> };
>>
>> typedef (PowerComponent or NetworkComponent or DOMString)
>> ResourceStatsComponent;
>>
>> dictionary ResourceStatsOptions
>> {
>> /**
>> * For power stats, |component| specifies which component's power
>> consumption
>> * will be returned. If null, total power consumption is returned.
> I encourage to be null or undefined (may this is implicit, just to
> remark) in order to avoid including it in the JS object.

We should probably make this default to null, and take the null value or
not specifying at all to mean "all". Similarly for other cases. Of
course, that means that the type should be nullable too.

I think we can do better:

Promise getStats(optional unsigned long long start,
optional unsigned long long end,
optional ResourceStatsOptions statsOptions);

That should let us drop |end| when we want current time, and drop both
start and end when we want the full range. Does that make sense?

>> *
>> * |statsOptions| specifies the detial of statistics of interest.
>> *
>> * If success, the |result| field keeps statistics.
>> * For networkStatsManager, the |result| is an array of
>> NetworkStats object.
>> * For powerStatsManager, the |result| is an array of PowerStats
>> object.
>> */
>> Promise getStats(unsigned long long start,
>> unsigned long long end,
>> optional ResourceStatsOptions statsOptions);
>>
>> /**
>> * Clear resource statistics stored in database.
>> *
>> * If |statsOptions| is provided, then only the statistics
>> associated to that
>> * |statsOptions| is deleted. Otherwise, all statistics of this
>> resource type
>> * is deleted.
>> */
>> Promise clearStats(optional ResourceStatsOptions statsOptions);

Is it OK that this doesn't accept a time range?

> As we are providing the next `clearAllStats()` method, the parameter is
> no longer optional and the explanation about it, when omitted should be
> moved to the next method.

Wouldn't it be better if we just get rid of clearAllStats and make
mgr.clearStats() clear all stats? That would be consistent with the
semantics of getStats(), which is an advantage.

>> /**
>> * Clear all resource statistics stored in database.
>> */
>> Promise clearAllStats();
> This clear all stats of the resource type as I said in my previous comment.
>>
>> /**
>> * Install an alarm to monitor resource usage.
>> *
>> * The |threshold| is an object specified the limit of resource
>> usage. When
>> * resource usage reaches the threshold, a "resourceStats-alarm"
>> system
>> * message is sent to the application.
>> *
>> * |statsOptions| specifies the detial of statistics of interest.
>> *
>> * |alarmOptions| is a ResourceStatsAlarmOptions object.
>> *
>> * If success, the |result| field keeps the alarm Id.
>> */
>> Promise addAlarm(any threshold,
>> optional ResourceStatsOptions statsOptions,
>> optional ResourceStatsAlarmOptions alarmOptions);

Should we specify that the data argument to this API is cloned using the
Structured Clone algorithm? We can't accept any JS object here unless
I'm missing something.

>> /**
>> * Obtain alarms.
>> *
>> * If |statsOptions| is provided, then only the alarms monitoring that
>> * resource are returned. Otherwise, all alarms set for this
>> resource type
>> * is returned.
>> *
>> * If success, the |result| is an array of ResourceStatsAlarm.
>> */
>> Promise getAlarms(optional ResourceStatsOptions statsOptions);
>>
>> /**
>> * Obtain all alarms.
>> *
>> * If success, the |result| is an array of ResourceStatsAlarm.
>> */
>> Promise getAllAlarms();
> Although I like this, note getting is not a critical functionality (we
> can not destroy any information with a get operation) so, according to
> the getAlarms() documentation, we are providing here a redundant way to
> get all the alarms.

Yes, I agree. I think we should remove this function and just have
mgr.getAlarms() return all alarms.

>> /**
>> * Remove alarms.
>> *
>> * If an |alarmId| is provided, then only that alarm is removed.
>> */
>> Promise removeAlarm(optional long alarmId);
> Here we have the opposite example. As we are destroying things, we
> should make mandatory the parameter and use the next method,
> removeAllAlarms(), if we want to short-cut the method and destroy all
> the alarms.

Hmm, but if we did what we do for the rest of these similar concepts,
then we would benefit from the consistency of the usage of this API.
What do you think?

>> /**
>> * Remove all alarms.
>> */
>> Promise removeAllAlarm();
>>
>> /**
>> * Return available resource stats options stored in database.
>> *
>> * If success, the |result| is an array of ResourceStatsOptions.
>> */
>> Promise getResourceStatsOptions();

It's not exactly clear to me what this method is expected to return...

>> /**
>> * Return the type of resource statistics provided by this manager.
>> */
>> readonly attribute DOMString resourceType;
>> };
>>
>> [JSImplementation="@mozilla.org/resourceStatsFactory;1",
>> NavigatorProperty="mozResourceStats",
>> Pref="dom.mozResourceStats.enabled"]
>> interface ResourceStatsFactory

I'm not convinced that this is a good idea. We have
[Constructor(ResourceType)] on ResourceStatsManager which means that the
getResourceStatsManager function here is useless. We can have
resourceTypes, sampleRate and maxStorageAge be static read-only
attributes on ResourceStatsManager. That would mean that we can
eliminate ResourceStatsFactory completely, and not have this interface
tie to navigator at all, which would let you write code like:

var stats = new ResourceStatsManager("network");
stats.getStats(...).then(...);

>> {
>> /**
>> * Get the resourceStatsManager for a resource type.
>> *
>> * |type| specifies the type of resourceStatsManager we want to
>> obtain.
>> * This value should be one of those returned by resourceTypes
>> attribute.
>> *
>> * If success, this method returns an object of ResourceStatsManager.
>> * Otherwise, null is resturned
>> */
>> ResourceStatsManager getResourceStatsManager(ResourceType type);
> May be we can reduce verbosity by renaming this method to getManager().
>>
>> /**
>> * Return the name of supported resource statistics types.
>> * Currently, 'network' and 'power' statistics are supported.
>> */
>> readonly attribute any resourceTypes; // array of DOMString

Can't you use sequence<DOMString> instead of any?

>> /**
>> * Time in milliseconds between statistics stored in database.
>> */
>> readonly attribute unsigned long long sampleRate;

Why is this an unsigned long long? Do you expect to have to deal with
such large sampling rates?

>> /**
>> * Time in milliseconds recorded by the API until present time. All
>> * statistics older than maxStorageAge from now are deleted.
>> */
>> readonly attribute unsigned long long maxStorageAge;
>> };
>>
>> == Proposal End ==

Thanks for your work here so far!
Ehsan

[Borting Chen]

Hi Ehsan and Salvador:

Thanks for the suggestions. Please see my in-line response.

I am not sure that whether we are planning to count bluetooth traffic.
But, if so, we will have "bluetooth" in NetworkComponent.
Hence, I would prefer to use an enum plus an identifier to express the network components.

Another idea, how about use an "ResourceComponent" enum to integrate "PowerComponent" and "NetworkComponent"?
and provide an |id| variable for further identification (i.e. this is the iccid for mobile)

My approach would be something like this:

enum ResourceComponent {

"cpu",
"screen",
"mobile",
"wifi",
"bluetooth",
"gps"
}

DOMString id; // for mobile, this is the iccid

I think this could give a clear classification, as the ResourceComponent defines the component owned by a mobile phone and the |id| is used to specify a more detail description if necessary.
Here are some use cases:

Case 1: Query network packets transmit by a particular SIM

==> { type: "network",
component: "mobile",
id: "89886920...", // iccid
...
}

Case 2: Query network usage of mobile (i,e, sum of all SIMs)

==> { type: "network",
component: "mobile",
id: null, // or undefined
...
}

Case 3: Query network usage of wifi

==> { type: "network",
component: "wifi",
id: null, // or undefined
...
}

Case 4: Query power consumption of mobile

==> { type: "power",
component: "mobile",
id: null, // or undefined
...
}

And the following misuse cases would return an error or an exception:

Misuse 1: Query network usage of cpu

==> { type: "network",
component: "cpu",
id: null, // or undefined
...
}

Misuse 2: Query network usage of wifi, but specify an id (assume we have only one wifi interface now)

==> { type: "network",
component: "wifi",
id: "1",
...
}

Misuse 3: Query power consumption of mobile, but specify an iccid
(It is not necessary to distinguish the power consumption of different SIMs.)

==> { type: "power",
component: "mobile",
id: "89886920...",
...
}

Is this okay?

> >> /**
> >> * List of system services supporting resource statistics
> >> */
> >> enum SystemService {
> >> "ota",
> >> "tethering"
> >> };
> >>
> >> typedef (PowerComponent or NetworkComponent or DOMString)
> >> ResourceStatsComponent;
> >>
> >> dictionary ResourceStatsOptions
> >> {
> >> /**
> >> * For power stats, |component| specifies which component's power
> >> consumption
> >> * will be returned. If null, total power consumption is returned.
> > I encourage to be null or undefined (may this is implicit, just to
> > remark) in order to avoid including it in the JS object.
>
> We should probably make this default to null, and take the null value or
> not specifying at all to mean "all". Similarly for other cases. Of
> course, that means that the type should be nullable too.

okay, thanks for the suggestion.

Good idea. Thanks for the suggestion
I would like to change the order of the parameter as following:

Promise getStats(optional ResourceStatsOptions statsOptions,

optional unsigned long long start,

optional unsigned long long end);

So, we can use this method as following:

mgr.getStats(options, startTime, endTime); // query stats of the specified app/service in the specified time range
mgr.getStats(options, startTime); // query stats of the specified app/service in the range [startTime, Now]
mgr.getStats(options); // query stats of the specified app/service stored in DB
mgr.getStats(); // query all stats stored in DB

Is this okay?

> >> *
> >> * |statsOptions| specifies the detial of statistics of interest.
> >> *
> >> * If success, the |result| field keeps statistics.
> >> * For networkStatsManager, the |result| is an array of
> >> NetworkStats object.
> >> * For powerStatsManager, the |result| is an array of PowerStats
> >> object.
> >> */
> >> Promise getStats(unsigned long long start,
> >> unsigned long long end,
> >> optional ResourceStatsOptions statsOptions);
> >>
> >> /**
> >> * Clear resource statistics stored in database.
> >> *
> >> * If |statsOptions| is provided, then only the statistics
> >> associated to that
> >> * |statsOptions| is deleted. Otherwise, all statistics of this
> >> resource type
> >> * is deleted.
> >> */
> >> Promise clearStats(optional ResourceStatsOptions statsOptions);
>
> Is it OK that this doesn't accept a time range?

I guess most users will want to delete all records associated to an app or a service when calling this method.
But it is okay for me to provide something like:

Promise clearStats(optional ResourceStatsOptions statsOptions,

optional unsigned long long start,

optional unsigned long long end);

This makes clearStats() to be consistent with the getStats().

> > As we are providing the next `clearAllStats()` method, the parameter is
> > no longer optional and the explanation about it, when omitted should be
> > moved to the next method.

I don't know why but I need to keep the "optional"; or else, WebIDL.py would return the following error:
"error: Dictionary argument or union argument containing a dictionary not followed by a required argument must be optional"

My fault. The |threshold| should be defined as unsigned long.
So, this method should be:

Promise addAlarm(unsigned long threshold,

optional ResourceStatsOptions statsOptions,
optional ResourceStatsAlarmOptions alarmOptions);

> >> /**
> >> * Obtain alarms.
> >> *
> >> * If |statsOptions| is provided, then only the alarms monitoring that
> >> * resource are returned. Otherwise, all alarms set for this
> >> resource type
> >> * is returned.
> >> *
> >> * If success, the |result| is an array of ResourceStatsAlarm.
> >> */
> >> Promise getAlarms(optional ResourceStatsOptions statsOptions);
> >>
> >> /**
> >> * Obtain all alarms.
> >> *
> >> * If success, the |result| is an array of ResourceStatsAlarm.
> >> */
> >> Promise getAllAlarms();
> > Although I like this, note getting is not a critical functionality (we
> > can not destroy any information with a get operation) so, according to
> > the getAlarms() documentation, we are providing here a redundant way to
> > get all the alarms.
>
> Yes, I agree. I think we should remove this function and just have
> mgr.getAlarms() return all alarms.

Thanks for the suggestion. I will remove the getAllAlarms().

> >> /**
> >> * Remove alarms.
> >> *
> >> * If an |alarmId| is provided, then only that alarm is removed.
> >> */
> >> Promise removeAlarm(optional long alarmId);
> > Here we have the opposite example. As we are destroying things, we
> > should make mandatory the parameter and use the next method,
> > removeAllAlarms(), if we want to short-cut the method and destroy all
> > the alarms.
>
> Hmm, but if we did what we do for the rest of these similar concepts,
> then we would benefit from the consistency of the usage of this API.
> What do you think?



> >> /**
> >> * Remove all alarms.
> >> */
> >> Promise removeAllAlarm();
> >>
> >> /**
> >> * Return available resource stats options stored in database.
> >> *
> >> * If success, the |result| is an array of ResourceStatsOptions.
> >> */
> >> Promise getResourceStatsOptions();
>
> It's not exactly clear to me what this method is expected to return...

Actually, this API is created to support getAvailableNetworks() of NetworkStatsManager.
(please refer: https://bugzilla.mozilla.org/show_bug.cgi?id=922926)

For network stats, it would return available networks stored in DB.
For power stats, it would return the component supporting power metering.

> >> /**
> >> * Return the type of resource statistics provided by this manager.
> >> */
> >> readonly attribute DOMString resourceType;
> >> };
> >>
> >> [JSImplementation="@mozilla.org/resourceStatsFactory;1",
> >> NavigatorProperty="mozResourceStats",
> >> Pref="dom.mozResourceStats.enabled"]
> >> interface ResourceStatsFactory
>
> I'm not convinced that this is a good idea. We have
> [Constructor(ResourceType)] on ResourceStatsManager which means that the
> getResourceStatsManager function here is useless. We can have
> resourceTypes, sampleRate and maxStorageAge be static read-only
> attributes on ResourceStatsManager. That would mean that we can
> eliminate ResourceStatsFactory completely, and not have this interface
> tie to navigator at all, which would let you write code like:
>
> var stats = new ResourceStatsManager("network");
> stats.getStats(...).then(...);

You are right. I will merge this to ResourceStatsManager.

But, a little question here.
In original design, ResourceStatsFactory.resourceTypes returns the supported statistic types (i.e. ["network", "power"]).
And a manager returns the statistic type it supported, for example netStatsMgr.resourceType would return "network".
But, if we provide ResourceStatsManager only, there is no attribute can provide the info of supported statistic types.
Do we need to provide another attribute to support this?

> >> {
> >> /**
> >> * Get the resourceStatsManager for a resource type.
> >> *
> >> * |type| specifies the type of resourceStatsManager we want to
> >> obtain.
> >> * This value should be one of those returned by resourceTypes
> >> attribute.
> >> *
> >> * If success, this method returns an object of ResourceStatsManager.
> >> * Otherwise, null is resturned
> >> */
> >> ResourceStatsManager getResourceStatsManager(ResourceType type);
> > May be we can reduce verbosity by renaming this method to getManager().
> >>
> >> /**
> >> * Return the name of supported resource statistics types.
> >> * Currently, 'network' and 'power' statistics are supported.
> >> */
> >> readonly attribute any resourceTypes; // array of DOMString
>
> Can't you use sequence<DOMString> instead of any?

I never thought about it before. Thanks for the suggestion.

> >> /**
> >> * Time in milliseconds between statistics stored in database.
> >> */
> >> readonly attribute unsigned long long sampleRate;
>
> Why is this an unsigned long long? Do you expect to have to deal with
> such large sampling rates?

Oh, I just wanted to keep consistent with maxStorageAge.
It is okay for me to define sampleRate as unsigned long.
Thanks for the suggestion.

> >> /**
> >> * Time in milliseconds recorded by the API until present time. All
> >> * statistics older than maxStorageAge from now are deleted.
> >> */
> >> readonly attribute unsigned long long maxStorageAge;
> >> };
> >>
> >> == Proposal End ==

Cheers,
Borting

[Salvador de la Puente González]

Hello guys!

I would want to make a call for other developers (better if they are
pure front-end developers) to take a look to our proposals in order to
provide a higher perspective.

In addition, Albert, can you provide some technical feedback here?

After my calling, comments in line:

I would like to keep all component dimension related stuff together but
your idea lead me to a variation I think it could provide functionality
and expresivity. Instead of component and id keys, why not { component:
{ <component>:<id> }, ... } ?

If you want to not specify a component, use { component: '<component>', ...}

>
> I think this could give a clear classification, as the ResourceComponent defines the component owned by a mobile phone and the |id| is used to specify a more detail description if necessary.
> Here are some use cases:
>
> Case 1: Query network packets transmit by a particular SIM
>
> ==> { type: "network",
> component: "mobile",
> id: "89886920...", // iccid
> ...
> }

This would be:

==> { type: "network",
component: { "mobile":"89886920..."}
...
}

Anyway, I still bet for the DOMString approach since it is more flexible
and allow undefined levels of multiplexing.

> Case 2: Query network usage of mobile (i,e, sum of all SIMs)
>
> ==> { type: "network",
> component: "mobile",
> id: null, // or undefined
> ...
> }

This would end as:

==> { type: "network",
component: "mobile"

See my comments about getResourceStatsOptions()

I like it but I want to hear the opinion of Albert and José Manuel
because we designed the API of NetworkStatistics and, for some reason,
we moved from an approach based on an interval object to making these
parameters mandatory. Anyway, the approach based on a range object makes
a lot of sense to me and reduces the number of parameters. This is
something like a dictionary with start and end keys to represent
interval [start, end]. If some of them is omitted, it is assumed to be a
semi-open interval to the limit: no start -> (-inf, end]; no end ->
[start, +inf). This way the signature for the method is as follows:

getStats(range[, options]);

Based on my experience with Cost Control app it is quite common to ask
for big-picture statistics in a specific range so the order of
parameters seems more intuitive to me this way.

Adding the start and end options (or the range object proposed before)
to the signatures seems a great idea to me and yes, make it consistent
with getStats().

>
>>> As we are providing the next `clearAllStats()` method, the parameter is
>>> no longer optional and the explanation about it, when omitted should be
>>> moved to the next method.
> I don't know why but I need to keep the "optional"; or else, WebIDL.py would return the following error:
> "error: Dictionary argument or union argument containing a dictionary not followed by a required argument must be optional"
>
>> Wouldn't it be better if we just get rid of clearAllStats and make
>> mgr.clearStats() clear all stats? That would be consistent with the
>> semantics of getStats(), which is an advantage.

I totally agree with you that ripping out the "all" versions of the
methods provides a more homogeneous API but I simply think it's more
dangerous. Please, consider the risk of wiping out the complete database
because your optional objects become null / undefined due to a bugged
behaviour in production. If this could happen, I prefer to end with a
console error about incorrect types for parameters and explicitly call a
'all' version rather than silently loose al my data. Well though, the
developer could provide their own safe methods too so I will discuss no
more here. Hope it makes sense to you.

Yep, we should.

Same reasons as above.

>
>
>>>> /**
>>>> * Remove all alarms.
>>>> */
>>>> Promise removeAllAlarm();
>>>>
>>>> /**
>>>> * Return available resource stats options stored in database.
>>>> *
>>>> * If success, the |result| is an array of ResourceStatsOptions.
>>>> */
>>>> Promise getResourceStatsOptions();
>> It's not exactly clear to me what this method is expected to return...
> Actually, this API is created to support getAvailableNetworks() of NetworkStatsManager.
> (please refer: https://bugzilla.mozilla.org/show_bug.cgi?id=922926)
>
> For network stats, it would return available networks stored in DB.
> For power stats, it would return the component supporting power metering.

I think the point here is we need a function in the manager to enumerate
components. So the method could be renamed to
getAvailableComponents() and, depending in which specific manager we
are, this method return a set of components. This is why I prefer
DOMStrings instead of other kind of values for components. In the case
of network it will return ['wifi:0', 'mobile:<iccid1>', 'mobile:<iccid2>'].

>
>>>> /**
>>>> * Return the type of resource statistics provided by this manager.
>>>> */
>>>> readonly attribute DOMString resourceType;
>>>> };
>>>>
>>>> [JSImplementation="@mozilla.org/resourceStatsFactory;1",
>>>> NavigatorProperty="mozResourceStats",
>>>> Pref="dom.mozResourceStats.enabled"]
>>>> interface ResourceStatsFactory
>> I'm not convinced that this is a good idea. We have
>> [Constructor(ResourceType)] on ResourceStatsManager which means that the
>> getResourceStatsManager function here is useless. We can have
>> resourceTypes, sampleRate and maxStorageAge be static read-only
>> attributes on ResourceStatsManager. That would mean that we can
>> eliminate ResourceStatsFactory completely, and not have this interface
>> tie to navigator at all, which would let you write code like:
>>
>> var stats = new ResourceStatsManager("network");
>> stats.getStats(...).then(...);
> You are right. I will merge this to ResourceStatsManager.
>
> But, a little question here.
> In original design, ResourceStatsFactory.resourceTypes returns the supported statistic types (i.e. ["network", "power"]).
> And a manager returns the statistic type it supported, for example netStatsMgr.resourceType would return "network".
> But, if we provide ResourceStatsManager only, there is no attribute can provide the info of supported statistic types.
> Do we need to provide another attribute to support this?

I think you're right. The only remark here is I would avoid the `new`
notation. IMHO, some like
navigator.resourceStats.getManager('<resource>') is a better idea.
Consider now 'resourceStats' could contain other methods or properties
like the list of available 'resources'.

Hope all this makes sense to you.
Kind regards!

[Ehsan Akhgari]

On 1/10/2014, 6:48 AM, Salvador de la Puente González wrote:
> Hello guys!
>
> I would want to make a call for other developers (better if they are
> pure front-end developers) to take a look to our proposals in order to
> provide a higher perspective.
>
> In addition, Albert, can you provide some technical feedback here?

Great idea, thanks for getting them into the loop. :-)

Hmm, if I understand what you mean here correctly, I'm not sure if that
would be expressible in Web IDL, since the |{ <component>:<id> }| bit is
not really a dictionary. If that's not the case, do you mind suggesting
the Web IDL declaration of the construct you have in mind please?

[snipped]

I don't feel very strongly about this either way; the range object idea
also seems good. It seems worth noting that the (-inf, end] range would
not be expressible with my proposal. Let's wait to hear from Albert and
José Manuel on this.

[snipped]

>>>> As we are providing the next `clearAllStats()` method, the parameter is
>>>> no longer optional and the explanation about it, when omitted should be
>>>> moved to the next method.
>> I don't know why but I need to keep the "optional"; or else, WebIDL.py
>> would return the following error:
>> "error: Dictionary argument or union argument containing a dictionary
>> not followed by a required argument must be optional"
>>
>>> Wouldn't it be better if we just get rid of clearAllStats and make
>>> mgr.clearStats() clear all stats? That would be consistent with the
>>> semantics of getStats(), which is an advantage.
> I totally agree with you that ripping out the "all" versions of the
> methods provides a more homogeneous API but I simply think it's more
> dangerous. Please, consider the risk of wiping out the complete database
> because your optional objects become null / undefined due to a bugged
> behaviour in production. If this could happen, I prefer to end with a
> console error about incorrect types for parameters and explicitly call a
> 'all' version rather than silently loose al my data. Well though, the
> developer could provide their own safe methods too so I will discuss no
> more here. Hope it makes sense to you.

Hmm, yeah that is a convincing argument. As much as I like a more
consistent and smaller API surface, you're right here. Perhaps we
should keep clearAllStats() and clearAllAlarms() for this reason.

[snipped]

>>>>> /**
>>>>> * Remove all alarms.
>>>>> */
>>>>> Promise removeAllAlarm();
>>>>>
>>>>> /**
>>>>> * Return available resource stats options stored in database.
>>>>> *
>>>>> * If success, the |result| is an array of ResourceStatsOptions.
>>>>> */
>>>>> Promise getResourceStatsOptions();
>>> It's not exactly clear to me what this method is expected to return...
>> Actually, this API is created to support getAvailableNetworks() of
>> NetworkStatsManager.
>> (please refer: https://bugzilla.mozilla.org/show_bug.cgi?id=922926)
>>
>> For network stats, it would return available networks stored in DB.
>> For power stats, it would return the component supporting power metering.
> I think the point here is we need a function in the manager to enumerate
> components. So the method could be renamed to
> getAvailableComponents() and, depending in which specific manager we
> are, this method return a set of components. This is why I prefer
> DOMStrings instead of other kind of values for components. In the case
> of network it will return ['wifi:0', 'mobile:<iccid1>', 'mobile:<iccid2>'].

Hmm, getResourceStatsOptions() would also enumerate the components...
Am I missing something?

What I suggested was to move these to be static attributes of
ResourceStatsManager, so you would access them using
|ResourceStatsManager.resourceTypes| for example. Does that sound good?

Cheers,
Ehsan

[Ehsan Akhgari]

OK, fair enough!

> Another idea, how about use an "ResourceComponent" enum to integrate "PowerComponent" and "NetworkComponent"?
> and provide an |id| variable for further identification (i.e. this is the iccid for mobile)
>
> My approach would be something like this:
>
> enum ResourceComponent {
> "cpu",
> "screen",
> "mobile",
> "wifi",
> "bluetooth",
> "gps"
> }
>
> DOMString id; // for mobile, this is the iccid

I think this is a nice idea, but Salvador doesn't agree. I'll let you
guys try to come up with the best option for now. :-)

> I think this could give a clear classification, as the ResourceComponent defines the component owned by a mobile phone and the |id| is used to specify a more detail description if necessary.
> Here are some use cases:
>
> Case 1: Query network packets transmit by a particular SIM
>
> ==> { type: "network",
> component: "mobile",
> id: "89886920...", // iccid
> ...
> }
>

> Case 2: Query network usage of mobile (i,e, sum of all SIMs)
>
> ==> { type: "network",
> component: "mobile",
> id: null, // or undefined
> ...
> }
>

FWIW for these misuse cases, we can probably throw in the implementation.

[snipped]
>>> As we are providing the next `clearAllStats()` method, the parameter is
>>> no longer optional and the explanation about it, when omitted should be
>>> moved to the next method.
>
> I don't know why but I need to keep the "optional"; or else, WebIDL.py would return the following error:
> "error: Dictionary argument or union argument containing a dictionary not followed by a required argument must be optional"

Given what Salvador argued, this should be moot now!

>> Wouldn't it be better if we just get rid of clearAllStats and make
>> mgr.clearStats() clear all stats? That would be consistent with the
>> semantics of getStats(), which is an advantage.
>>

>>>> /**
>>>> * Clear all resource statistics stored in database.
>>>> */
>>>> Promise clearAllStats();
>>> This clear all stats of the resource type as I said in my previous comment.
>>>>
>>>> /**
>>>> * Install an alarm to monitor resource usage.
>>>> *
>>>> * The |threshold| is an object specified the limit of resource
>>>> usage. When
>>>> * resource usage reaches the threshold, a "resourceStats-alarm"
>>>> system
>>>> * message is sent to the application.
>>>> *
>>>> * |statsOptions| specifies the detial of statistics of interest.
>>>> *
>>>> * |alarmOptions| is a ResourceStatsAlarmOptions object.

>>>> *

>>>> * If success, the |result| field keeps the alarm Id.
>>>> */
>>>> Promise addAlarm(any threshold,
>>>> optional ResourceStatsOptions statsOptions,
>>>> optional ResourceStatsAlarmOptions alarmOptions);
>>
>> Should we specify that the data argument to this API is cloned using the
>> Structured Clone algorithm? We can't accept any JS object here unless
>> I'm missing something.
>

> My fault. The |threshold| should be defined as unsigned long.
> So, this method should be:
>
> Promise addAlarm(unsigned long threshold,
> optional ResourceStatsOptions statsOptions,
> optional ResourceStatsAlarmOptions alarmOptions);

Oh, good! But I was actually talking about
ResourceStatsAlarmOptions.id. :-) It can't just be |any|.

[snipped]

>>>> /**
>>>> * Return the type of resource statistics provided by this manager.
>>>> */
>>>> readonly attribute DOMString resourceType;
>>>> };
>>>>
>>>> [JSImplementation="@mozilla.org/resourceStatsFactory;1",
>>>> NavigatorProperty="mozResourceStats",
>>>> Pref="dom.mozResourceStats.enabled"]
>>>> interface ResourceStatsFactory
>>
>> I'm not convinced that this is a good idea. We have
>> [Constructor(ResourceType)] on ResourceStatsManager which means that the
>> getResourceStatsManager function here is useless. We can have
>> resourceTypes, sampleRate and maxStorageAge be static read-only
>> attributes on ResourceStatsManager. That would mean that we can
>> eliminate ResourceStatsFactory completely, and not have this interface
>> tie to navigator at all, which would let you write code like:
>>
>> var stats = new ResourceStatsManager("network");
>> stats.getStats(...).then(...);
>
> You are right. I will merge this to ResourceStatsManager.
>
> But, a little question here.
> In original design, ResourceStatsFactory.resourceTypes returns the supported statistic types (i.e. ["network", "power"]).
> And a manager returns the statistic type it supported, for example netStatsMgr.resourceType would return "network".
> But, if we provide ResourceStatsManager only, there is no attribute can provide the info of supported statistic types.
> Do we need to provide another attribute to support this?

I'm not suggesting to change any of this from your proposal, please just
move those attributes to be static attributes of ResourceStatsManager.

Cheers,
Ehsan

[Borting Chen]

Hi Ehsan:

> >> Should we specify that the data argument to this API is cloned using the
> >> Structured Clone algorithm? We can't accept any JS object here unless
> >> I'm missing something.
> >
> > My fault. The |threshold| should be defined as unsigned long.
> > So, this method should be:
> >
> > Promise addAlarm(unsigned long threshold,
> > optional ResourceStatsOptions statsOptions,
> > optional ResourceStatsAlarmOptions alarmOptions);
>
> Oh, good! But I was actually talking about
> ResourceStatsAlarmOptions.id. :-) It can't just be |any|.

Oops, I misunderstood.
Should I specify the data as an object type?
so, the ResourceStatsAlarmOptions would be something like this:

dictionary ResourceStatsAlarmOptions
{

unsigned long long startTime; // time in milliseconds since Epoch

object data;
};

[snipped]

> >>>> [JSImplementation="@mozilla.org/resourceStatsFactory;1",
> >>>> NavigatorProperty="mozResourceStats",
> >>>> Pref="dom.mozResourceStats.enabled"]
> >>>> interface ResourceStatsFactory
> >>
> >> I'm not convinced that this is a good idea. We have
> >> [Constructor(ResourceType)] on ResourceStatsManager which means that the
> >> getResourceStatsManager function here is useless. We can have
> >> resourceTypes, sampleRate and maxStorageAge be static read-only
> >> attributes on ResourceStatsManager. That would mean that we can
> >> eliminate ResourceStatsFactory completely, and not have this interface
> >> tie to navigator at all, which would let you write code like:
> >>
> >> var stats = new ResourceStatsManager("network");
> >> stats.getStats(...).then(...);
> >
> > You are right. I will merge this to ResourceStatsManager.
> >
> > But, a little question here.
> > In original design, ResourceStatsFactory.resourceTypes returns the
> > supported statistic types (i.e. ["network", "power"]).
> > And a manager returns the statistic type it supported, for example
> > netStatsMgr.resourceType would return "network".
> > But, if we provide ResourceStatsManager only, there is no attribute can
> > provide the info of supported statistic types.
> > Do we need to provide another attribute to support this?
>
> I'm not suggesting to change any of this from your proposal, please just
> move those attributes to be static attributes of ResourceStatsManager.

Sorry, I am a little confused.
What you suggested is removing the ResourceStatsFactory interface and providing a ResourceStatsManager interface like this:

[Constructor(ResourceType), NavigatorProperty="mozResourceStats", ...]
interface ResourceStatsManager{
...
readonly attribute sequence<DOMString> resourceTypes;

readonly attribute unsigned long long sampleRate;

readonly attribute unsigned long long maxStorageAge;
};

Am I right?



Thanks for your responses.

Cheers,
Borting

[Ehsan Akhgari]

On 1/10/2014, 12:58 PM, Borting Chen wrote:
> Hi Ehsan:
>
>>>> Should we specify that the data argument to this API is cloned using the
>>>> Structured Clone algorithm? We can't accept any JS object here unless
>>>> I'm missing something.
>>>
>>> My fault. The |threshold| should be defined as unsigned long.
>>> So, this method should be:
>>>
>>> Promise addAlarm(unsigned long threshold,
>>> optional ResourceStatsOptions statsOptions,
>>> optional ResourceStatsAlarmOptions alarmOptions);
>>
>> Oh, good! But I was actually talking about
>> ResourceStatsAlarmOptions.id. :-) It can't just be |any|.
>
> Oops, I misunderstood.
> Should I specify the data as an object type?
> so, the ResourceStatsAlarmOptions would be something like this:
>
> dictionary ResourceStatsAlarmOptions
> {
> unsigned long long startTime; // time in milliseconds since Epoch
> object data;
> };

Are you familiar with the structured clone algorithm?
<http://www.whatwg.org/specs/web-apps/current-work/multipage/common-dom-interfaces.html#structured-clone>
This is an algorithm used to safely pass structured data around. The
problem here is that if we specify the type of data as |any| without
further restructions, you can pass things that are not safely
transportable to other browsing contexts etc, and we need to prevent
that by calling out that the data argument needs to be cloned using
structured clone. Does that make sense?

Almost, I suggested these attributes to be static
<http://heycam.github.io/webidl/#idl-static-attributes-and-operations>
which means they can be accessed off of the interface object without
needing to instantiate any objects from that interface first. So, for
example, instead of writing |navigator.mozResourceStats.resourceTypes|,
you would write |ResourceStatsManager.resourceTypes|.

Cheers,
Ehsan

[Salvador de la Puente González]

Hello guys!

More comments in-line:

On 10/01/14 17:25, Ehsan Akhgari wrote:
> Hmm, if I understand what you mean here correctly, I'm not sure if
> that would be expressible in Web IDL, since the |{ <component>:<id> }|
> bit is not really a dictionary. If that's not the case, do you mind
> suggesting the Web IDL declaration of the construct you have in mind
> please?

Oh, sorry, I would mean { '<component>':'<id>' } as strings. That is a
valid dictionary / JS object. Anyway, I want to say I support the idea
of taking a separate 'id' key for specifying the specific component.
What I don't agree is to have this at the same level of 'component' key
because it is **part of the component** not a sibling of the component.

Just notice since negative dates are invalid, (-inf, end] === [0, end]
so don't worry, both alternatives have the same expressibility. I'm only
concerned about how much intuitive is to write `getStats(somePastDate,
null, options)` or getStats(null, somePastDate). Lets hear news from
José Manuel and Albert.

> [snipped]
>>>>>> /**
>>>>>> * Remove all alarms.
>>>>>> */
>>>>>> Promise removeAllAlarm();
>>>>>>
>>>>>> /**
>>>>>> * Return available resource stats options stored in database.

>>>>>> *

>>>>>> * If success, the |result| is an array of ResourceStatsOptions.
>>>>>> */
>>>>>> Promise getResourceStatsOptions();
>>>> It's not exactly clear to me what this method is expected to return...
>>> Actually, this API is created to support getAvailableNetworks() of
>>> NetworkStatsManager.
>>> (please refer: https://bugzilla.mozilla.org/show_bug.cgi?id=922926)
>>>
>>> For network stats, it would return available networks stored in DB.
>>> For power stats, it would return the component supporting power
>>> metering.
>> I think the point here is we need a function in the manager to enumerate
>> components. So the method could be renamed to
>> getAvailableComponents() and, depending in which specific manager we
>> are, this method return a set of components. This is why I prefer
>> DOMStrings instead of other kind of values for components. In the case
>> of network it will return ['wifi:0', 'mobile:<iccid1>',
>> 'mobile:<iccid2>'].
>
> Hmm, getResourceStatsOptions() would also enumerate the components...
> Am I missing something?

I'm suggesting only to rename the function as I don't understand what
more information is provided here. I mean, the resourceStatsOptions
dictionary is always the same. The only field that is semantically
specific to the resource you're querying is the 'component' field so it
makes sense to ask explicitly for those components and nothing more.

>
>>>>>> /**
>>>>>> * Return the type of resource statistics provided by this
>>>>>> manager.
>>>>>> */
>>>>>> readonly attribute DOMString resourceType;
>>>>>> };
>>>>>>

>>>>>> [JSImplementation="@mozilla.org/resourceStatsFactory;1",
>>>>>> NavigatorProperty="mozResourceStats",
>>>>>> Pref="dom.mozResourceStats.enabled"]
>>>>>> interface ResourceStatsFactory
>>>> I'm not convinced that this is a good idea. We have
>>>> [Constructor(ResourceType)] on ResourceStatsManager which means
>>>> that the
>>>> getResourceStatsManager function here is useless. We can have
>>>> resourceTypes, sampleRate and maxStorageAge be static read-only
>>>> attributes on ResourceStatsManager. That would mean that we can
>>>> eliminate ResourceStatsFactory completely, and not have this interface
>>>> tie to navigator at all, which would let you write code like:
>>>>
>>>> var stats = new ResourceStatsManager("network");
>>>> stats.getStats(...).then(...);
>>> You are right. I will merge this to ResourceStatsManager.
>>>
>>> But, a little question here.
>>> In original design, ResourceStatsFactory.resourceTypes returns the
>>> supported statistic types (i.e. ["network", "power"]).
>>> And a manager returns the statistic type it supported, for example
>>> netStatsMgr.resourceType would return "network".
>>> But, if we provide ResourceStatsManager only, there is no attribute
>>> can provide the info of supported statistic types.
>>> Do we need to provide another attribute to support this?

>> I think you're right. The only remark here is I would avoid the `new`
>> notation. IMHO, some like
>> navigator.resourceStats.getManager('<resource>') is a better idea.
>> Consider now 'resourceStats' could contain other methods or properties
>> like the list of available 'resources'.
>
> What I suggested was to move these to be static attributes of
> ResourceStatsManager, so you would access them using
> |ResourceStatsManager.resourceTypes| for example. Does that sound good?

Yes, I realized I added my comment in a wrong place. I agree on moving
resourceTypes as a static attribute of ResourceStatsManager but I what
I was suggesting is a factory strategy. Take a look to this example:

var specificResources = navigator.resourceStats.resources;

var specificResourceManager = navigator.resourceStats.getManager(specificResource[0]);


Does all this make sense to you?

Thank you.

[Ehsan Akhgari]

On 1/14/2014, 5:29 AM, Salvador de la Puente Gonz�lez wrote:
> Hello guys!
>
> More comments in-line:
>
> On 10/01/14 17:25, Ehsan Akhgari wrote:
>> Hmm, if I understand what you mean here correctly, I'm not sure if
>> that would be expressible in Web IDL, since the |{ <component>:<id> }|
>> bit is not really a dictionary. If that's not the case, do you mind
>> suggesting the Web IDL declaration of the construct you have in mind
>> please?
> Oh, sorry, I would mean { '<component>':'<id>' } as strings. That is a
> valid dictionary / JS object. Anyway, I want to say I support the idea
> of taking a separate 'id' key for specifying the specific component.
> What I don't agree is to have this at the same level of 'component' key
> because it is **part of the component** not a sibling of the component.

Yes, I think I understood you correctly, but like I said, what you're
describing is not expressible in WebIDL, and I prefer if we came up with
a syntax that does map to existing WebIDL constructs. I don't care very
strongly how it looks like here as long as it's in WebIDL. :-)

>> [snipped]
>>>>> I think we can do better:
>>>>>
>>>>> Promise getStats(optional unsigned long long start,
>>>>> optional unsigned long long end,
>>>>> optional ResourceStatsOptions statsOptions);
>>>>>
>>>>> That should let us drop |end| when we want current time, and drop both
>>>>> start and end when we want the full range. Does that make sense?
>>>> Good idea. Thanks for the suggestion
>>>> I would like to change the order of the parameter as following:
>>>>
>>>> Promise getStats(optional ResourceStatsOptions statsOptions,
>>>> optional unsigned long long start,
>>>> optional unsigned long long end);
>>>>
>>>> So, we can use this method as following:
>>>>
>>>> mgr.getStats(options, startTime, endTime); // query stats of the
>>>> specified app/service in the specified time range
>>>> mgr.getStats(options, startTime); // query stats of the specified
>>>> app/service in the range [startTime, Now]
>>>> mgr.getStats(options); // query stats of the specified app/service
>>>> stored in DB
>>>> mgr.getStats(); // query all stats stored in DB
>>>>
>>>> Is this okay?

>>> I like it but I want to hear the opinion of Albert and Jos� Manuel

>>> because we designed the API of NetworkStatistics and, for some reason,
>>> we moved from an approach based on an interval object to making these
>>> parameters mandatory. Anyway, the approach based on a range object makes
>>> a lot of sense to me and reduces the number of parameters. This is
>>> something like a dictionary with start and end keys to represent
>>> interval [start, end]. If some of them is omitted, it is assumed to be a
>>> semi-open interval to the limit: no start -> (-inf, end]; no end ->
>>> [start, +inf). This way the signature for the method is as follows:
>>>
>>> getStats(range[, options]);
>>>
>>> Based on my experience with Cost Control app it is quite common to ask
>>> for big-picture statistics in a specific range so the order of
>>> parameters seems more intuitive to me this way.
>>
>> I don't feel very strongly about this either way; the range object
>> idea also seems good. It seems worth noting that the (-inf, end]
>> range would not be expressible with my proposal. Let's wait to hear

>> from Albert and Jos� Manuel on this.

>>
> Just notice since negative dates are invalid, (-inf, end] === [0, end]
> so don't worry, both alternatives have the same expressibility. I'm only
> concerned about how much intuitive is to write `getStats(somePastDate,
> null, options)` or getStats(null, somePastDate). Lets hear news from

> Jos� Manuel and Albert.

Hmm yeah you're right (well, except that s/null/undefined/ in your example.)

Right, makes sense.

What I suggested gives you exactly that, without pinning anything off of
navigator:

var specificResources = ResourceStatsManager.resourceTypes;
var specificResourceManager = new
ResourceStatsManager(specificResources[0]);

The main difference between our proposals is that mine doesn't pollute
the navigator object at all, and just uses a constructor which acts as a
factory. No navigator pollution + less Java-ism == win! :-)

Cheers,
Ehsan

[Ehsan Akhgari]

On 1/9/2014, 10:12 AM, Borting Chen wrote:
[snipped]

> [Constructor(ResourceType type), JSImplementation="@mozilla.org/resourceStatsManager;1", Pref="dom.mozResourceStats.enabled"]
> interface ResourceStatsManager
> {
> /**
> * Query resource statistics.
> *
> * |start| and |end| specifies the time range of interest, both included.
> * If |start| is 0, retrieves the stats since measurements.
> * If |end| is 0. retrieves the stats until the current time.

> *
> * |statsOptions| specifies the detial of statistics of interest.

> *

> * If success, the |result| field keeps statistics.
> * For networkStatsManager, the |result| is an array of NetworkStats object.

> * For powerStatsManager, the |result| is an array of PowerStats object.

BTW, I just realized that you need to define the NetworkStats and
PowerStats interfaces in WebIDL as well.

Cheers,
Ehsan

[Salvador de la Puente González]

Hi,

Just a couple of comments:

On 14/01/14 16:47, Ehsan Akhgari wrote:

> On 1/14/2014, 5:29 AM, Salvador de la Puente González wrote:
>> Hello guys!
>>
>> More comments in-line:
>>
>> On 10/01/14 17:25, Ehsan Akhgari wrote:
>>> Hmm, if I understand what you mean here correctly, I'm not sure if
>>> that would be expressible in Web IDL, since the |{ <component>:<id> }|
>>> bit is not really a dictionary. If that's not the case, do you mind
>>> suggesting the Web IDL declaration of the construct you have in mind
>>> please?
>> Oh, sorry, I would mean { '<component>':'<id>' } as strings. That is a
>> valid dictionary / JS object. Anyway, I want to say I support the idea
>> of taking a separate 'id' key for specifying the specific component.
>> What I don't agree is to have this at the same level of 'component' key
>> because it is **part of the component** not a sibling of the component.
>
> Yes, I think I understood you correctly, but like I said, what you're
> describing is not expressible in WebIDL, and I prefer if we came up
> with a syntax that does map to existing WebIDL constructs. I don't
> care very strongly how it looks like here as long as it's in WebIDL. :-)

Yes, you're right. It is not impossible, it is just a mess to define
upon dictionaries but you can do:
{
...
jsval component,
...

}
> What I suggested gives you exactly that, without pinning anything off
> of navigator:
>
> var specificResources = ResourceStatsManager.resourceTypes;
> var specificResourceManager = new
> ResourceStatsManager(specificResources[0]);
>
> The main difference between our proposals is that mine doesn't pollute
> the navigator object at all, and just uses a constructor which acts as
> a factory. No navigator pollution + less Java-ism == win! :-)
>

Let's discuss if `new` is more or less JS face of face xD... I see the
advantages for a developer to simply make `new
ResourceStatsManager(...)` so being pragmatic, I agree on you and I wont
push on the alternative any more.
> Cheers,
> Ehsan
>
Thank you very much. I think we agree on almost everything.
Let's see what happen with the `component` field and the `getStats()`
signature.

Cheers!

[Ehsan Akhgari]

On 1/14/2014, 12:36 PM, Salvador de la Puente Gonz�lez wrote:
> Hi,
>
> Just a couple of comments:
>
> On 14/01/14 16:47, Ehsan Akhgari wrote:

>> On 1/14/2014, 5:29 AM, Salvador de la Puente Gonz�lez wrote:
>>> Hello guys!
>>>
>>> More comments in-line:
>>>
>>> On 10/01/14 17:25, Ehsan Akhgari wrote:
>>>> Hmm, if I understand what you mean here correctly, I'm not sure if
>>>> that would be expressible in Web IDL, since the |{ <component>:<id> }|
>>>> bit is not really a dictionary. If that's not the case, do you mind
>>>> suggesting the Web IDL declaration of the construct you have in mind
>>>> please?
>>> Oh, sorry, I would mean { '<component>':'<id>' } as strings. That is a
>>> valid dictionary / JS object. Anyway, I want to say I support the idea
>>> of taking a separate 'id' key for specifying the specific component.
>>> What I don't agree is to have this at the same level of 'component' key
>>> because it is **part of the component** not a sibling of the component.
>>
>> Yes, I think I understood you correctly, but like I said, what you're
>> describing is not expressible in WebIDL, and I prefer if we came up
>> with a syntax that does map to existing WebIDL constructs. I don't
>> care very strongly how it looks like here as long as it's in WebIDL. :-)
> Yes, you're right. It is not impossible, it is just a mess to define
> upon dictionaries but you can do:
> {
> ...
> jsval component,
> ...
> }

The issue is that WebIDL dictionaries allow you to specify any number of
the members of that dictionary. So we can't create a dictionary which
has one member per component for the purposes of your proposal. So I
think the only way to represent this in WebIDL is to just accept an
|any| argument, which is terrible.

Cheers,
Ehsan

[Ehsan Akhgari]

[Salvador de la Puente González]

Hello Ehsan

On 14/01/14 20:25, Ehsan Akhgari wrote:

> On 1/14/2014, 12:36 PM, Salvador de la Puente González wrote:
>> Hi,
>>
>> Just a couple of comments:
>>
>> On 14/01/14 16:47, Ehsan Akhgari wrote:

Several choices here:

1) A fragment of the IDL needed:
dictionary SimComponentSpec { DOMString sim }
dictionary CPUComponentSpec { number cpu }
...

{
...
(SimComponentSpec or CPUComponenSpec or ...) component;
...
}

Is this not a valid IDL?

2) We can check a jsval and iterate on the first key only.

3) We can adopt another simpler type as:

dictionary ComponentSpecification {
DOMString name,
DOMString id
}

{
...
(DOMString or ComponentSpecification) component;
...
}

And this only remarks that what I don't agree is to have the `id` at the
same level of `component` as I said in my previos e-mail.

4) Allow this...

dictionary ComponentsSepecs {
DOMString sim,
DOMString cpu,
DOMString network,
....
}

{
...
ComponentsSpecs component,
...
}

For the present implementation, we only accept one of the fields being
not null (the behaviour for other combinations is undefined). A future
implementation could aggregate the concepts into the `component` field
(and I suggest renaming it to `components`).

I insist, my only complain is this arrangement of fields:

DOMString component,
DOMString id

Because I think we have a very clear interface based on specifying what
you want to meter based on independent criteria (component, application,
systemService) and I would want to keep this independence. But If my
arguments does not convince you, I agree on the `id` option. There is
always room for improvement : )

Thank you guys.

[Ehsan Akhgari]

On 1/15/2014, 5:13 AM, Salvador de la Puente Gonz�lez wrote:
> Hello Ehsan
>
> On 14/01/14 20:25, Ehsan Akhgari wrote:

>> On 1/14/2014, 12:36 PM, Salvador de la Puente Gonz�lez wrote:
>>> Hi,
>>>
>>> Just a couple of comments:
>>>
>>> On 14/01/14 16:47, Ehsan Akhgari wrote:

Hmm it is, but it will let you write code like:

foo.component = { sim: "bar" };
foo.component = { cpu: 100 };

Is that what you were looking to achieve?

> 2) We can check a jsval and iterate on the first key only.

I'd rather adopt an API which doesn't require this.

> 3) We can adopt another simpler type as:
>
> dictionary ComponentSpecification {
> DOMString name,
> DOMString id
> }
>
> {
> ...
> (DOMString or ComponentSpecification) component;
> ...
> }
>
> And this only remarks that what I don't agree is to have the `id` at the
> same level of `component` as I said in my previos e-mail.
>
> 4) Allow this...
>
> dictionary ComponentsSepecs {
> DOMString sim,
> DOMString cpu,
> DOMString network,
> ....
> }
>
> {
> ...
> ComponentsSpecs component,
> ...
> }
>
> For the present implementation, we only accept one of the fields being
> not null (the behaviour for other combinations is undefined). A future
> implementation could aggregate the concepts into the `component` field
> (and I suggest renaming it to `components`).

This is kind of abusing WebIDL dictionaries.

> I insist, my only complain is this arrangement of fields:
>
> DOMString component,
> DOMString id
>
> Because I think we have a very clear interface based on specifying what
> you want to meter based on independent criteria (component, application,
> systemService) and I would want to keep this independence. But If my
> arguments does not convince you, I agree on the `id` option. There is
> always room for improvement : )

To be honest, I don't care much about the dimensionality of component
and the rest of the values here, so I'm happy to leave this up to you.
Based on the four alternatives you suggested, I guess the #1 is the
least bad option, but I'm concerned that it makes the API too verbose.
But that's something that we can get feedback on from the useers of the
API once we implement one of these alternatives. I guess I'm happy to
leave it to Borting which one he wants to choose here. :-)

Cheers,
Ehsan

[Borting Chen]

Hi Ehsan:

--
Borting Chen
Intern of Mozilla Taiwan

----- Original Message -----
> From: "Ehsan Akhgari" <ehsan....@gmail.com>
> To: "Borting Chen" <boc...@mozilla.com>
> Cc: "Salvador de la Puente González" <s...@tid.es>, dev-w...@lists.mozilla.org, "JOSE MANUEL CANTERA FONSECA"
> <jm...@tid.es>, "Gene Lian" <cl...@mozilla.com>
> Sent: Saturday, January 11, 2014 2:15:33 AM
> Subject: Re: Proposal: ResourceStats API for NetworkStats API and PowerStats API integration
>

Yah, I understood.
But I still have no idea about how to specify the data argument to be cloned using structured clone.
(Or this should be done in the implementation?)
Could you give me some hints? Thanks.

> > Sorry, I am a little confused.
> > What you suggested is removing the ResourceStatsFactory interface and
> > providing a ResourceStatsManager interface like this:
> >
> > [Constructor(ResourceType), NavigatorProperty="mozResourceStats", ...]
> > interface ResourceStatsManager{
> > ...
> > readonly attribute sequence<DOMString> resourceTypes;
> > readonly attribute unsigned long long sampleRate;
> > readonly attribute unsigned long long maxStorageAge;
> > };
> >
> > Am I right?
>
> Almost, I suggested these attributes to be static
> <http://heycam.github.io/webidl/#idl-static-attributes-and-operations>
> which means they can be accessed off of the interface object without
> needing to instantiate any objects from that interface first. So, for
> example, instead of writing |navigator.mozResourceStats.resourceTypes|,
> you would write |ResourceStatsManager.resourceTypes|.

Understood, leaned a lot from you, thanks. :-)

Cheers,
Borting

[Borting Chen]

> >>>>>> [JSImplementation="@mozilla.org/resourceStatsFactory;1",
> >>>>>> NavigatorProperty="mozResourceStats",
> >>>>>> Pref="dom.mozResourceStats.enabled"]
> >>>>>> interface ResourceStatsFactory
> >>>>
> >>>> I'm not convinced that this is a good idea. We have
> >>>> [Constructor(ResourceType)] on ResourceStatsManager which means that the
> >>>> getResourceStatsManager function here is useless. We can have
> >>>> resourceTypes, sampleRate and maxStorageAge be static read-only
> >>>> attributes on ResourceStatsManager. That would mean that we can
> >>>> eliminate ResourceStatsFactory completely, and not have this interface
> >>>> tie to navigator at all, which would let you write code like:
> >>>>
> >>>> var stats = new ResourceStatsManager("network");
> >>>> stats.getStats(...).then(...);
> >>>
> >>> You are right. I will merge this to ResourceStatsManager.
> >>>
> >>> But, a little question here.
> >>> In original design, ResourceStatsFactory.resourceTypes returns the
> >>> supported statistic types (i.e. ["network", "power"]).
> >>> And a manager returns the statistic type it supported, for example
> >>> netStatsMgr.resourceType would return "network".
> >>> But, if we provide ResourceStatsManager only, there is no attribute can
> >>> provide the info of supported statistic types.
> >>> Do we need to provide another attribute to support this?
> >>

> >> I'm not suggesting to change any of this from your proposal, please just

> >> move those attributes to be static attributes of ResourceStatsManager.

> >
> > Sorry, I am a little confused.
> > What you suggested is removing the ResourceStatsFactory interface and
> > providing a ResourceStatsManager interface like this:
> >
> > [Constructor(ResourceType), NavigatorProperty="mozResourceStats", ...]
> > interface ResourceStatsManager{
> > ...
> > readonly attribute sequence<DOMString> resourceTypes;
> > readonly attribute unsigned long long sampleRate;
> > readonly attribute unsigned long long maxStorageAge;
> > };
> >
> > Am I right?
>
> Almost, I suggested these attributes to be static
> <http://heycam.github.io/webidl/#idl-static-attributes-and-operations>
> which means they can be accessed off of the interface object without
> needing to instantiate any objects from that interface first. So, for
> example, instead of writing |navigator.mozResourceStats.resourceTypes|,
> you would write |ResourceStatsManager.resourceTypes|.

Understood. Leaned a lot from you, thanks. :-)

Cheers,
Borting

[Borting Chen]

Hi Ehsan:

Sorry, I missed this part.

The webidl would be something like this:

[JSImplementation="@mozilla.org/networkStatsData;1", Pref="dom.mozResourceStats.enabled"]
interface MozNetworkStatsData
{
readonly attribute unsigned long rxBytes; // Received bytes.
readonly attribute unsigned long txBytes; // Sent bytes
readonly attribute unsigned long long date;
};

[JSImplementation="@mozilla.org/powerStatsData;1", Pref="dom.mozResourceStats.enabled"]
interface MozPowerStatsData
{
readonly attribute unsigned long power; // power consumption
readonly attribute unsigned long long date;
};

[JSImplementation="@mozilla.org/resourceStats;1", Pref="dom.mozResourceStats.enabled"]
interface MozResourceStats
{
/**
* Type of statistics
*/
readonly attribute ResourceType type;

/**
* Manifest URL of the application.
*/
readonly attribute DOMString? manifestURL;

/**
* Type of system service
*/
readonly attribute SystemService? serviceType;

/**
* The component that the statistics belongs to.
* If null, sum of all components' statistics are returned.
*/
readonly attribute DOMString? component;

/**
* Statistics, one element per day
*/
readonly attribute (sequence<MozNetworkStatsData> or sequence<MozPowerStatsData>) data;

/**
* Date range
*/
readonly attribute unsigned long long start; // start date
readonly attribute unsigned long long end; // end date
};

Some notes:
1. This design is mainly refer the design of present NetworkStats interface.
2. The type of |component| would be changed according to the type we defined in resourceStatsOptions dictionary.
3. I am not quite sure the type of |data| could be defined as such a union, is that a valid webidl?

Cheers,
Borting

[Borting Chen]

Hi Ehsan and Salvador:

First, thank you guys for the great comments and suggestions.
I am really learned a lot from you guys, thanks.

About the resourceStatsOptions dictionary, I totally agree Salvador's concern
that we should not put "id" and "component" at the same level.
And, it also makes sense to me to have the three dimensions
{"component", "service", "application"} being separated independently.
So, I would like to vote Salvador's option (1).

Hi, Salvador.
Sorry, I still have a small question.
Why do you want to define a dictionary for each component in your option (1)?
Could we use a DOMString to express component and id, like "<component>:<id>" as you mentioned before?

I think both solutions are okay for me.
I am just curious about what benefit we can obtain from the design of option (1)? :-)

Thank you guys for the help!

> > On 14/01/14 20:25, Ehsan Akhgari wrote:

> >> On 1/14/2014, 12:36 PM, Salvador de la Puente González wrote:
> >>> On 14/01/14 16:47, Ehsan Akhgari wrote:

Borting

[Salvador de la Puente González]

Hello Borting

On 16/01/14 07:31, Borting Chen wrote:
> Hi Ehsan and Salvador:
>
> First, thank you guys for the great comments and suggestions.
> I am really learned a lot from you guys, thanks.
>
> About the resourceStatsOptions dictionary, I totally agree Salvador's concern
> that we should not put "id" and "component" at the same level.
> And, it also makes sense to me to have the three dimensions
> {"component", "service", "application"} being separated independently.
> So, I would like to vote Salvador's option (1).
>
> Hi, Salvador.
> Sorry, I still have a small question.
> Why do you want to define a dictionary for each component in your option (1)?
> Could we use a DOMString to express component and id, like "<component>:<id>" as you mentioned before?

I don't remember where or why I suggested the dictionary one but my
favourite solution was always **this** (the WebIDL solution is very
verbose as Ehsan noticed). So I totally agree!

>
> I think both solutions are okay for me.
> I am just curious about what benefit we can obtain from the design of option (1)? :-)
>
> Thank you guys for the help!

Thank you. I learn a lot with you too!
Hope we can discuss about other APIs cause I think the result is very
good. :)

[Ehsan Akhgari]

You can use the any type, and in the prose describing the API mention
that the data field will be copied using the structured clone algorithm.
It's not possible to declare this in WebIDL.

We're moving towards a good solution here, thanks to you! Keep up the
good work. :-)

Cheers,
Ehsan

[Ehsan Akhgari]

On 1/16/2014, 1:13 AM, Borting Chen wrote:
> Hi Ehsan:
>

>>> [Constructor(ResourceType type),
>>> JSImplementation="@mozilla.org/resourceStatsManager;1",
>>> Pref="dom.mozResourceStats.enabled"]
>>> interface ResourceStatsManager
>>> {
>>> /**
>>> * Query resource statistics.
>>> *
>>> * |start| and |end| specifies the time range of interest, both
>>> included.
>>> * If |start| is 0, retrieves the stats since measurements.
>>> * If |end| is 0. retrieves the stats until the current time.
>>> *
>>> * |statsOptions| specifies the detial of statistics of interest.
>>> *
>>> * If success, the |result| field keeps statistics.
>>> * For networkStatsManager, the |result| is an array of NetworkStats
>>> object.
>>> * For powerStatsManager, the |result| is an array of PowerStats object.
>>
>> BTW, I just realized that you need to define the NetworkStats and
>> PowerStats interfaces in WebIDL as well.
>
> Sorry, I missed this part.
>
> The webidl would be something like this:
>
> [JSImplementation="@mozilla.org/networkStatsData;1", Pref="dom.mozResourceStats.enabled"]
> interface MozNetworkStatsData

Nit: please don't use Moz prefixes.

> {
> readonly attribute unsigned long rxBytes; // Received bytes.
> readonly attribute unsigned long txBytes; // Sent bytes

Nit: why not name these receivedBytes and sentBytes respectively? Also,
shouldn't these be unsigned long long?

> readonly attribute unsigned long long date;

Nit: I think date is a little bit misleading here because this
represents more than just the date. How about calling it timestamp?

(I think you should use the DOMTimeStamp type for this stuff, please see
<http://www.w3.org/TR/DOM-Level-3-Core/core.html#Core-DOMTimeStamp>,
even though it's a typedef to unsigned long long, it will make things
clearer.)

> };
>
> [JSImplementation="@mozilla.org/powerStatsData;1", Pref="dom.mozResourceStats.enabled"]
> interface MozPowerStatsData
> {
> readonly attribute unsigned long power; // power consumption

You should specify the units of measurement here. Depending on that,
this might need to be unsigned long long as well.

> readonly attribute unsigned long long start; // start date
> readonly attribute unsigned long long end; // end date

Nit: DOMTimeStamp for these and all other similar things as well. Sorry
I forgot to mention this earlier.

> };
>
> Some notes:
> 1. This design is mainly refer the design of present NetworkStats interface.
> 2. The type of |component| would be changed according to the type we defined in resourceStatsOptions dictionary.
> 3. I am not quite sure the type of |data| could be defined as such a union, is that a valid webidl?

Yes, I don't see anything wrong with it.

Cheers,
Ehsan

[Borting Chen]

Hi Ehsan:

> > [JSImplementation="@mozilla.org/networkStatsData;1",
> > Pref="dom.mozResourceStats.enabled"]
> > interface MozNetworkStatsData
>
> Nit: please don't use Moz prefixes.
>
> > {
> > readonly attribute unsigned long rxBytes; // Received bytes.
> > readonly attribute unsigned long txBytes; // Sent bytes
>
> Nit: why not name these receivedBytes and sentBytes respectively? Also,
> shouldn't these be unsigned long long?

Yes, these should be unsigned long long, thanks.
And you are right, receivedBytes and sentBytes are more clear, I would change the naming. :P

> > readonly attribute unsigned long long date;
>
> Nit: I think date is a little bit misleading here because this
> represents more than just the date. How about calling it timestamp?
>
> (I think you should use the DOMTimeStamp type for this stuff, please see
> <http://www.w3.org/TR/DOM-Level-3-Core/core.html#Core-DOMTimeStamp>,
> even though it's a typedef to unsigned long long, it will make things
> clearer.)

Understood. Thanks.

> > };
> >
> > [JSImplementation="@mozilla.org/powerStatsData;1",
> > Pref="dom.mozResourceStats.enabled"]
> > interface MozPowerStatsData
> > {
> > readonly attribute unsigned long power; // power consumption
>
> You should specify the units of measurement here. Depending on that,
> this might need to be unsigned long long as well.

The unit is mW, so you are right, the data type should be unsigned long long.
And, in order to consist with the naming of NetworkStatsData, I would like to rename |power| to |consumedPower|.
What do you think?

Oh, because webidl.py said it cannot distinguish sequence<NetworkStatsData> from sequence<PowerStatsData>,
I am just wondering whether it is a valid definition.
(or, should I define it as sequence<any> ?)
BTW, could I rename |data| to |stats|? I think it could be more clear for users.



Thanks for your comments.


Cheers,
Borting

[Borting Chen]

Hi guys:

Thanks for your help on the design of ResourceStats API.
Since we almost down the design and we have a lot of codes now,
I sent the updated codes to the Bugzilla to avoid making a long mail.
Please visit Bug 951976 (https://bugzilla.mozilla.org/show_bug.cgi?id=951976) to see the updates.
If anyone has another thought, please reply this thread or leave your comments on the Bugzilla directly.
Thanks a lot.

Cheers,
Borting

[Ehsan Akhgari]

On 1/23/2014, 10:24 PM, Borting Chen wrote:
>>> };
>>>
>>> [JSImplementation="@mozilla.org/powerStatsData;1",
>>> Pref="dom.mozResourceStats.enabled"]
>>> interface MozPowerStatsData
>>> {
>>> readonly attribute unsigned long power; // power consumption
>>
>> You should specify the units of measurement here. Depending on that,
>> this might need to be unsigned long long as well.
>
> The unit is mW, so you are right, the data type should be unsigned long long.

Great, please document the unit in the comment as well.

> And, in order to consist with the naming of NetworkStatsData, I would like to rename |power| to |consumedPower|.
> What do you think?

Sounds good to me.

[snipped]

>>> };
>>>
>>> Some notes:
>>> 1. This design is mainly refer the design of present NetworkStats
>>> interface.
>>> 2. The type of |component| would be changed according to the type we
>>> defined in resourceStatsOptions dictionary.
>>> 3. I am not quite sure the type of |data| could be defined as such a union,
>>> is that a valid webidl?
>>
>> Yes, I don't see anything wrong with it.
>
> Oh, because webidl.py said it cannot distinguish sequence<NetworkStatsData> from sequence<PowerStatsData>,
> I am just wondering whether it is a valid definition.
> (or, should I define it as sequence<any> ?)
> BTW, could I rename |data| to |stats|? I think it could be more clear for users.

OK, I stand corrected! So yeah, according to
<http://heycam.github.io/webidl/#dfn-distinguishable>, sequence<T> and
sequence<U> are not distinguishable. I checked with Boris on IRC today
and in fact he pointed out that given the fact that the value of the
array can change based on factors that are not observable in the DOM
interface (i.e., a change in the values stored in the database) this
should not be an attribute at all, since people don't expect that value
from an attribute. I think that's a fair point, and we should use a
method here instead. Example:

sequence<(NetworkStatsData or PowerStatsData)> getStats();

I'm open to bikeshedding the name of the method. getStats() sounds
better than getData() to me, but I don't have strong opinions on that.

Cheers,
Ehsan

[Vicamo Yang]

Hi all,

Some post implementation reviews and wish lists:

[[ Variable Sample Rate ]]

We are to introduce new component "power" to ResourceStats API in bug
854200 [1]. That wonders me: what's the point of those daily based power
consumption statistics? A general mobile device may not survive a couple
days without the charger. So on the power statistics chart, one will see
the power curve goes down to zero within a couple of data points. As for
charging in action, mobile devices usually take only a few hours to
become fully charged. That falls in one day period for sure. So on the
same power chart, you'll probably see nothing but a almost perpendicular
line goes all the way up to 100%. Not so fancy.

The sample rate of each resource type has to vary. Although this is not
addressed in the patch set, it has been implied by "current" API.
However, there is a line of comment right above that read-only attribute
|sampleRate| stating that after bug 863952 it will be moved to a static
attribute, which follows we'll share the same sample rate between all
resource types. So, please go back to the first paragraph.

Sample rates of different resource types are meant to be different. One
sample rate just don't fit all the resource types.

But what about the sample rate of a certain resource type?

The constant NetworkStats sample rate leads our implementation to merge
data point daily. "Daily" is a dangerous term for people traveling
around the world often. When exactly does 00:00 start for people of
different timezones? The way we implement such merge results in data
lose and over-written when people travel back and forth between UK and
US. There are other potential risks such as writing cached data once per
day or on XPCOM shutdown observer event. It's an implementation issue,
but we'd still need a complete definition for "daily sample rate". "Past
24 hours" or "calendar day"?

The constant sample rate also brings a problem: we have only one sample
rate. What if I want to know the network traffic statistics of the past
24 hours? We get only one data record with present API design. We have
the whole design only to fulfill current Cost Control app's requests but
it may raise other requests any time then. We should really define a
more flexible API set and specialize it to fit our own need.

[[ System Services and WebApps ]]

The original design is to allow query "network traffic generated on
<component> by <app> due to doing <service>". So far the defined system
services are "ota" and "tethering".

Although not correctly implemented, there can be "<app> doing
<tethering>" because tethering involves an additional kernel network
interface. But the term "tethering" here is a bit confusing because
there can be traffic through FORWARD chain and INPUT/OUTPUT chain.

| INPUT | | OUTPUT |
| |
--| PREROUTE |-+---| FORWARD |---+-| POSTROUTE |--

"<app> doing <tethering>" implies that traffic must flow over INPUT or
OUTPUT chain. But what the "tethering" service really cares is the
traffic over FORWARD chain. Accounting "<app> doing <tethering>" traffic
mixes the two different semantics together and results in meaningless
data. We still want tethering statistics, but that should be a network
component instead.

So what about "ota"? How do I mark a certain http connection is
downloading OTA firmware? I don't have a known solution for this and I
don't think we may have a viable solution at all.

The system services thing doesn't seem to mean anything for me. I hereby
suggest their removals.

[[ Multiplicity of getSample call ]]

Bug 1042149 [2] wishes the ability to query resource stats of multiple
apps within one call to the API.

The desire to reduce the calls to the API is originated from the
inconvenience to plot a usage chart that contains multiple interested
apps. One may image the only working way by now is to retrieve a list of
currently installed apps, and call to getSample() for each of them.
That's a rather lengthy process. Depending on the apps you've installed,
probably dozens or hundreds, the loading time increases in a large scale
because every transaction takes additional IPC penalties.

To be even worse, most of the calls may just returns an empty set
because not every installed app generates network traffic, especially
for packaged apps. We're likely to waste more time in unnecessary calls.

By current design, the returned statistics data must belong to either
one exactly app (manifestURL != null), one exactly predefined system
service (serviceType != null), or the system sum up (both manifestURL
and serviceType == null). This has to be re-worked to address this issue.

I'd like to propose following modified ResourceStatsOptions dictionary,
which is passed in getStats(), clearStats(), addAlarm(), and getAlarms().

dictionary ResourceStatsOptions
{
(DOMString or sequence<DOMString>)? component = null;

(DOMString or sequence<DOMString>)? manifestURL = null;
};

The |component| field becomes an union type that accepts zero, one or
more components. With |component| equals to null/undefined/empty array,
it means this field is ignored. Note that in previous discussion, an
empty value in this field means "sum of all components' usage". By
"ignored" I mean all usage data belong to origins specified in the next
field are returned.

With one or more entries in the |component| sequence, it means any
component that matches either one of the entries should be taken into
account. In getStats(), |component = ["wifi:0", "mobile:<iccid>"]| means
all usage records of either wifi:0 or mobile:<iccid> are returned.

Use an explicit "all" for "sum of all components' usage". We may also
optionally have "wifi" for sum of all wifi traffics and "mobile" for all
mobile connection traffics.

The |serviceType| is removed. See previous section.

The |manifestURL| field becomes yet another union field. Same as
|component|, when |manifestURL| equals to either undefined, null, or an
empty array, that means this field is ignored and all stats data of all
apps are taking into consideration.

With one or more entries in the |manifestURL| sequence, it means any
apps with manifest URL that matches either one of the entries should be
taken into account. In getStats(), |manifestURL = ["url1", "url2"]|
means all usage records of either app1 or app2 are returned.

Use an explicit "all" for "sum of all the traffic ever generated".

[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=854200
[2]: https://bugzilla.mozilla.org/show_bug.cgi?id=1042149

[Vicamo Yang]

[Vicamo Yang]

[Jonas Sicking]

On Wed, Jul 30, 2014 at 9:04 PM, Vicamo Yang <vic...@gmail.com> wrote:
>
> We are to introduce new component "power" to ResourceStats API in bug
> 854200 [1]. That wonders me: what's the point of those daily based power
> consumption statistics?

The main thing that I'd like to get out of recording power statistics
is to enable users to see which of their applications consume a lot of
battery.

So if I feel that my phone has been running out of power a lot quicker
the last couple of weeks, I'd like to be able get information about
which applications have been using up a lot of battery.

What exactly the best UX is to enable users to see that I'll leave up
to someone else. But some form of graph showing power consumption per
app over time seems like a reasonable approach.

Note that this is different from a graph showing the amount of charge
is in the battery over time. The amount of power consumed for a given
app is generally going to be completely unaffected by the user
plugging the device in, or the battery being low.

In theory applications could use the battery API to change their
behavior, which means that the power consumption would be affected by
if the battery is low or the device is plugged in. So possibly we'll
want to also show the battery status in the same graph as the graph
showing app power usage. But I think that's less important.

It would be interesting to show is when applications do background
syncing of data. This traditionally is one of the main causes of power
consumption throughout the day. Once we ship and start using the
requestSync API [1], we will be able to track that more easily.

I agree that if these graphs only have a resolution of one data-point
per day, then it will likely be less useful to users. They can still
see which applications use up a lot of power, but it will be harder
for them to know if the consumption was correlated to particular
events like background sync, or using particular parts of the
application. So having more data-points per day sounds good. But I'll
defer to UX here.

[1] https://github.com/slightlyoff/BackgroundSync

/ Jonas

[Vicamo Yang]

I'm afraid that's the root cause of these all these problems.

We have one single sample rate now right because our UX wants a chart of
daily network statistics. This concept was also adopted in other
resource types, so we end up with one single common sample rate in
ResourceStats API design. When some UX requirements say otherwise, we'll
have to accommodate the API to those new requirements. Not so flexible.

Is it possible to provide an API:

getStats({ period: "day" }); // Return daily sum.
getStats({ period: "hour" }); // Return hourly sum.
getStats(); // Return finest raw records.

So for Firefox OS network statistics, the finest raw records we have are
actually daily sum. For battery stats, they might be those recorded
every 5 to 10 minutes.

--
Vicamo Yang

[Jonas Sicking]

On Sun, Aug 3, 2014 at 7:22 PM, Vicamo Yang <vic...@gmail.com> wrote:
> On 8/2/14, 8:13 AM, Jonas Sicking wrote:

>> But I'll defer to UX here.
>

> I'm afraid that's the root cause of these all these problems.
>
> We have one single sample rate now right because our UX wants a chart of
> daily network statistics. This concept was also adopted in other
> resource types, so we end up with one single common sample rate in
> ResourceStats API design. When some UX requirements say otherwise, we'll
> have to accommodate the API to those new requirements. Not so flexible.
>
> Is it possible to provide an API:
>
> getStats({ period: "day" }); // Return daily sum.
> getStats({ period: "hour" }); // Return hourly sum.
> getStats(); // Return finest raw records.
>
> So for Firefox OS network statistics, the finest raw records we have are
> actually daily sum. For battery stats, they might be those recorded
> every 5 to 10 minutes.

Yeah. Seems like the first change we need to do here is to the
internals of the API so that we can store records with different
granularity for different data types. So that we can store more fine
grained data for battery stats than we do for network stats, if that
is indeed what we need in the UX.

I'm not sure that it's worth adding getStats({ period: "day" }) to the
API though. It seems just as easy to add up data in the application as
in the API. Unless there's significant performance differences?

/ Jonas

[Jonas Sicking]

On Mon, Aug 4, 2014 at 1:37 AM, You-Sheng Yang <vic...@gmail.com> wrote:
>> Yeah. Seems like the first change we need to do here is to the
>> internals of the API so that we can store records with different
>> granularity for different data types.
>

> Yes. That's definitely the first thing we have to address before
> creating other stores for other resource types. Currently FxOS
> NetworkStats API implementation buries that daily samples concept deep
> into the database. We'll need some more clean ups before we may go any
> further.

>
>> So that we can store more fine
>> grained data for battery stats than we do for network stats, if that
>> is indeed what we need in the UX.
>>
>> I'm not sure that it's worth adding getStats({ period: "day" }) to the
>> API though.
>

> "We don't want one single common sample rate that applies to all
> resource types." May I take this as the conclusion so far?

I guess the most important point is "We need to use a sample rate that
is fine-grained enough that it can solve all our UX needs".

That could either be solved by simply using a higher sample rate than
daily samples for all resource types. Or it could be solved by using
different sample rates for different resource types.

It depends on if using a high sample rate for all resource types
results in too much disk-space usage I guess. And how much complexity
is involved with using different sample rates for different types.

I definitely agree that embedding "daily samples" deep into the
implementation is not good. That seems too coarse.

/ Jonas

[Ehsan Akhgari]

On 2014-08-04, 7:51 PM, Jonas Sicking wrote:
> On Mon, Aug 4, 2014 at 1:37 AM, You-Sheng Yang <vic...@gmail.com> wrote:

>>> Yeah. Seems like the first change we need to do here is to the
>>> internals of the API so that we can store records with different
>>> granularity for different data types.
>>

>> Yes. That's definitely the first thing we have to address before
>> creating other stores for other resource types. Currently FxOS
>> NetworkStats API implementation buries that daily samples concept deep
>> into the database. We'll need some more clean ups before we may go any
>> further.
>>

>>> So that we can store more fine
>>> grained data for battery stats than we do for network stats, if that
>>> is indeed what we need in the UX.
>>>
>>> I'm not sure that it's worth adding getStats({ period: "day" }) to the
>>> API though.
>>

>> "We don't want one single common sample rate that applies to all
>> resource types." May I take this as the conclusion so far?
>
> I guess the most important point is "We need to use a sample rate that
> is fine-grained enough that it can solve all our UX needs".
>
> That could either be solved by simply using a higher sample rate than
> daily samples for all resource types. Or it could be solved by using
> different sample rates for different resource types.
>
> It depends on if using a high sample rate for all resource types
> results in too much disk-space usage I guess. And how much complexity
> is involved with using different sample rates for different types.
>
> I definitely agree that embedding "daily samples" deep into the
> implementation is not good. That seems too coarse.

If having a per-resource sampling rate doesn't complicate things too
much, I think we should definitely do that!

Cheers,
Ehsan

[Vicamo Yang]

Any comments?

On 8/6/14, 11:02 AM, You-Sheng Yang wrote:
> Actually these two problems are more serious for me. Bug 1042149 is even
> a blocker for v2.1.
>
> For the system services thing, I don't have a correct way to implement
> what they were meant to.
>
> For "ota", until we have a way to tag each TCP/UDP transaction, we can
> never know which transaction is for OTA. Besides, such tag must come
> directly from the app itself because it's the only one who knows what's
> that transaction for. This also follows that statistics data may be
> easily faked and and become a security hole (trigger usage alarm?).
>
> For "tethering", "<app> doing/using <tethering> connection" means either
> the source or the destination interface is that tethering interface, and
> the traffic delivered from INPUT/to OUPUT chain. But for tethering, we
> care traffic passing through FORWARD chain.
>
> In order to keeping track of tethering traffic after removing the system
> services, we may have an additional component type in the "network"
> resource type. For example:
>
> wifi:0 // WiFi HotSpot
> mobile:<iccid> // 3G tethering over WiFi or USB
>
> This is a proposal that fit current design most. But I'd rather create
> yet another resource type for tethering if possible.
>
> Then the ResourceStatsOptions dictionary may be further simplified as
> shown below. The call getStats() then supports querying for stats
> induced by multiple applications and/or across multiple components.

[Salvador de la Puente González]

Hi guys!

Including Alberto who know more about the sampling limits imposed by the
original specification of NetworkStats API.

AFAIK, sampling limitations were originated due to the restriction of
keeping the DB as small as possible. This, in addition with a UX showing
day samples could imply the use of 1-day samples. But this was never a
solid imposition and the NetworkStats API, both versions 1 and 2, allow
to change this parameter at Gecko-build stage.

About |sampleRate| and |maxStorageAge| fields, I don't think they are
not informative. Indeed I think they are quite useful. Consider
|sampleRate|, it exposes the maximum rate at which the resource is
taking samples allowing to (for instance) dynamically set the max zoom
in in a general chart component.

The same happens for |maxStorageAge| which can act as a limit for the
min zoom. Furthermore, it can act as an indicator to the user about how
much information is being stored or for 3rd party application to build
and sync their own databases.

So, despite I support multiple sample rates (although I simply recommend
1hour - 30min samples), I don't agree with removing |sampleRate| and
|maxStorageAge|.

What do you think?

On 06/08/14 03:49, You-Sheng Yang wrote:

> "having a per-resource sampling rate" only means we're to remove the
> comment above the |sampleRate| field, which says:
>
> This should be specified as static attribute after Bug 863952
> is resolved.
>
> Are we going to take a step further and remove |sampleRate| completely?
> It doesn't mean much for me. The sampling rate thing can just be an
> implicit rule. "On Firefox OS, getStats() of a certain resource type
> always returns statistics data over a predefined period of that resource
> type." For network statistics, it's one day. Others may vary. Keeping it
> doesn't hurt anything, but it just seem useless for me.
>
> And what about that |maxStorageAge| field?
>


________________________________

Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción.

The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it.

Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição

[Salvador de la Puente González]

Sorry for the delay on this concern. My comments in-line.

On 31/07/14 06:04, Vicamo Yang wrote:
> [[ System Services and WebApps ]]
>
> The original design is to allow query "network traffic generated on
> <component> by <app> due to doing <service>". So far the defined system
> services are "ota" and "tethering".
>
> Although not correctly implemented, there can be "<app> doing
> <tethering>"

I don't think this is possible from a user perspective. Applications on
the device never produce tethering. They are consuming the regular
mobile data channel. I though tethering was referring to the traffic
from / to outside the device.

> because tethering involves an additional kernel network
> interface. But the term "tethering" here is a bit confusing because
> there can be traffic through FORWARD chain and INPUT/OUTPUT chain.
>
> | INPUT | | OUTPUT |
> | |
> --| PREROUTE |-+---| FORWARD |---+-| POSTROUTE |--
>
> "<app> doing <tethering>" implies that traffic must flow over INPUT or
> OUTPUT chain. But what the "tethering" service really cares is the
> traffic over FORWARD chain. Accounting "<app> doing <tethering>" traffic
> mixes the two different semantics together and results in meaningless
> data.

I think I got it. Querying semantics allow "<app> while doing
`tethering`" but this is not only unimportant but meaningless (for the
reasons said above).

> We still want tethering statistics, but that should be a network
> component instead.
>
> So what about "ota"? How do I mark a certain http connection is
> downloading OTA firmware? I don't have a known solution for this and I
> don't think we may have a viable solution at all.

The 'OTA' case is different and it's bound to interpretation. The system
could track how many data is being transmitted for an application update
(or even for all the system) so, in theory, accountability of this
"service" is possible. But, in reality this is no more than tagged
traffic. For <system service> to be a completely independent query
dimension, we should allow to ask for power consumption during OTA or
tethering which is even more weird.

So, my suggestion is to focus on what we want to measure. I think
external device traffic is very important and I like the idea Vicamo is
suggesting in another mail about having a component labelled
|tethering-*| but I will comment on that later.

About, OTA, I'm not very sure that should be here at all. This is simply
traffic produced by the system application or Gecko (I don't remember if
we finally reserved a special manifest to refer to Gecko and distinguish
it from system app). But I would add this traffic to the system app to
keep consistency. If we want how many data has been consumed by a
certain update, maybe we should treat this in another API focused only
in keeping track of application life-cycle but not here.

> The system services thing doesn't seem to mean anything for me. I hereby
> suggest their removals.

Hope it helps.

[Salvador de la Puente González]

Hello!

On 06/08/14 05:07, You-Sheng Yang wrote:
> On 8/6/14, 11:02 AM, You-Sheng Yang wrote:
>> In order to keeping track of tethering traffic after removing the system
>> services, we may have an additional component type in the "network"
>> resource type. For example:
>>
>> wifi:0 // WiFi HotSpot
>> mobile:<iccid> // 3G tethering over WiFi or USB

> typo:
>
> tethering-wifi:0
> tethering-mobile:<iccid>
I agree on removing the service type but I have doubts about what your
proposal is referring to. Can you elaborate on the differences between
|tethering-wifi| and |tethering-mobile|?

IMO, the routed traffic from outside the device (which is what really
matters for tethering) is produce by Gecko. As I said in other
occasions, I would derive this traffic to the system application. The
service should be network and the component will be simply |tethering|.
The API will answer 0 for any application distinct than system manifest
and the routed traffic for the system one. As ignoring the manifestURL
would cause to sum up all the application usage, this becomes a natural
way to ask for tethering traffic.

WDYT?

[Ehsan Akhgari]

On 2014-08-18, 5:50 AM, Salvador de la Puente González wrote:
> Hi guys!
>
> Including Alberto who know more about the sampling limits imposed by the
> original specification of NetworkStats API.
>
> AFAIK, sampling limitations were originated due to the restriction of
> keeping the DB as small as possible. This, in addition with a UX showing
> day samples could imply the use of 1-day samples. But this was never a
> solid imposition and the NetworkStats API, both versions 1 and 2, allow
> to change this parameter at Gecko-build stage.
>
> About |sampleRate| and |maxStorageAge| fields, I don't think they are
> not informative. Indeed I think they are quite useful. Consider
> |sampleRate|, it exposes the maximum rate at which the resource is
> taking samples allowing to (for instance) dynamically set the max zoom
> in in a general chart component.
>
> The same happens for |maxStorageAge| which can act as a limit for the
> min zoom. Furthermore, it can act as an indicator to the user about how
> much information is being stored or for 3rd party application to build
> and sync their own databases.
>
> So, despite I support multiple sample rates (although I simply recommend
> 1hour - 30min samples), I don't agree with removing |sampleRate| and

> |maxStorageAge|.
>
> What do you think?

Yeah, I don't think there is a good reason to remove them either.

Cheers,
Ehsan

[Ehsan Akhgari]

On 2014-08-18, 6:24 AM, Salvador de la Puente González wrote:
> Hello!
>
> On 06/08/14 05:07, You-Sheng Yang wrote:
>> On 8/6/14, 11:02 AM, You-Sheng Yang wrote:
>>> In order to keeping track of tethering traffic after removing the system
>>> services, we may have an additional component type in the "network"
>>> resource type. For example:
>>>
>>> wifi:0 // WiFi HotSpot
>>> mobile:<iccid> // 3G tethering over WiFi or USB
>> typo:
>>
>> tethering-wifi:0
>> tethering-mobile:<iccid>
> I agree on removing the service type

That sounds good to me too, but I have to admit that at this point I'm
quite confused about the logical topology of different classes of data
that we're trying to make available through this API...

> but I have doubts about what your
> proposal is referring to. Can you elaborate on the differences between
> |tethering-wifi| and |tethering-mobile|?
>
> IMO, the routed traffic from outside the device (which is what really
> matters for tethering) is produce by Gecko. As I said in other
> occasions, I would derive this traffic to the system application. The
> service should be network and the component will be simply |tethering|.
> The API will answer 0 for any application distinct than system manifest
> and the routed traffic for the system one. As ignoring the manifestURL
> would cause to sum up all the application usage, this becomes a natural
> way to ask for tethering traffic.

That seems like tying things to much to the implementation details?
What if for example the tethering data is directly transmitted to the
app itself without the involvement of the system app?

Cheers,
Ehsan

[Salvador de la Puente González]

Hi Eshan!

Correct me if I'm wrong, please. In tethering, your device acts like a
bridge connecting the mobile data network and other devices via Wi-Fi.
So the traffic can originate from the Wi-Fi network or inside the
device. Isn't it? Although it could be, when an application is consuming
data while tethering is on, it is using the "tethering-mobile:<iccid>"
component, IMHO, from a user point of view this "inner usage" is no more
than regular data usage, i.e, "mobile:<iccid>", that will increase per
app network stats (because originates **inside** the device, what is not
the purpose of tethering).

In the other hand, when I use the hotspot from a laptop, data is routed
to my laptop via System App or Gecko (this second, I suppose). To keep
consistency of which <app> is spending the resource, I think the best
option is the system app, i.e, the app Gecko loads as privileged one and
communicates with.

Do you get my point?

>
> Cheers,
> Ehsan

[Ehsan Akhgari]

As you're tethering, if you open the email app to check your email on
the device, do you really expect that usage to be accounted as part of
the tethering usage? More specifically, if you have a bad app on your
device that starts to consume large amounts of data without your
consent, it would appear as if an app on your laptop has consumed this
data. That sounds misleading to me.

> In the other hand, when I use the hotspot from a laptop, data is routed
> to my laptop via System App or Gecko (this second, I suppose). To keep
> consistency of which <app> is spending the resource, I think the best
> option is the system app, i.e, the app Gecko loads as privileged one and
> communicates with.

But how do we differentiate that with other potential data that the
system app might consume for other purposes?

Cheers,
Ehsan

[Salvador de la Puente González]

Hello Ehsan

I think we are saying almost the same. Let's clarify some points:

What I expect if I open the email app to check my email on the device is
more data usage (regular data usage, not tethering at all) to be
accounted for the email app. If I have a bad app that consumes tons of
data from my device, I don't expect it counts for tethering but I review
the data usage and see what is that bad app. Do you understand?

>> In the other hand, when I use the hotspot from a laptop, data is routed
>> to my laptop via System App or Gecko (this second, I suppose). To keep
>> consistency of which <app> is spending the resource, I think the best
>> option is the system app, i.e, the app Gecko loads as privileged one and
>> communicates with.
>
> But how do we differentiate that with other potential data that the
> system app might consume for other purposes?

Sorry, I don't get you. I was referring to the case I open the email
client on my laptop, surf the web or see some Vimeo's videos then I
expect the usage to be accounted into the System app or maybe some
"tethering entry" in the list of apps or something like this (but this a
UX concern). Example:

bad app: 700MiB

email: 100MiB (not distinguishing among regular use or use while tethering)

other app: 60MiB

...

...

-----------------

<tethering>: 1GiB

What do you think?

[Salvador de la Puente González]

Hi Vicamo, thank you for the clarifications.

On 20/08/14 13:36, You-Sheng Yang wrote:
>>>> Although it could be, when an application is consuming
>>>> data while tethering is on, it is using the "tethering-mobile:<iccid>"
>>>> component,

> False. I thought that should always be "mobile:iccid" no matter
> tethering is turned on or off. All traffic send and received by device
> WebApps are accounted on either "wifi:<N>" or "mobile:<iccid>".
Perfect then, we agree on this. I was not sure about my statement so I
said "it could be". I think it should always be "mobile:iccid" too, just
as you say.

>>> IMHO, from a user point of view this "inner usage" is no more
>>> than regular data usage, i.e, "mobile:<iccid>", that will increase per
>>> app network stats (because originates **inside** the device, what is not
>>> the purpose of tethering).

> I think I'm lost here. Let's say there are two interfaces when WiFi
> tethering is turned on: wlan0 for WiFi, rmnet0 for mobile data. rmnet0
> has the default route. So the traffic are accounted to:
>
> 1) on device apps browse internet => mobile:<iccid>
> 2) on device apps talk to laptop app => wifi:0
> 3) laptop browse internet => tethering-mobile:<iccid>
>
> Does this help?
A lot. I was only considering routes 1 and 3. What I was saying is the
user probably only sees two kinds of usage: a inner usage (your case 1)
and a external usage (your case 3). Now, I'm realizing it could be
another more when apps communicate via WiFi. Just as you say, case 1
ends in "mobile:<iccid>" and case 3 in "tethering-mobile:<iccid>" (we
were saying the same, \o/).

For the case 2, just notice we need some way to identify wlan0 is the
interface being used whe enabling WiFi hotspot.

>
>>> As you're tethering, if you open the email app to check your email on
>>> the device, do you really expect that usage to be accounted as part of
>>> the tethering usage? More specifically, if you have a bad app on your
>>> device that starts to consume large amounts of data without your
>>> consent, it would appear as if an app on your laptop has consumed this
>>> data. That sounds misleading to me.
>> What I expect if I open the email app to check my email on the device is
>> more data usage (regular data usage, not tethering at all) to be
>> accounted for the email app. If I have a bad app that consumes tons of
>> data from my device, I don't expect it counts for tethering but I review
>> the data usage and see what is that bad app. Do you understand?

> Sorry, but I don't see the connection between your previous reply and this.
Ehsan asked: "do you really expect that usage to be accounted as part of
the tethering usage?" and I answered that what I expect is data
accounting in regular data usage ("mobile:iccid") as we just agreed before.

>
>>>> In the other hand, when I use the hotspot from a laptop, data is routed
>>>> to my laptop via System App or Gecko (this second, I suppose). To keep
>>>> consistency of which <app> is spending the resource, I think the best
>>>> option is the system app, i.e, the app Gecko loads as privileged one and
>>>> communicates with.
>>> But how do we differentiate that with other potential data that the
>>> system app might consume for other purposes?
>> Sorry, I don't get you. I was referring to the case I open the email
>> client on my laptop, surf the web or see some Vimeo's videos then I
>> expect the usage to be accounted into the System app or maybe some
>> "tethering entry" in the list of apps or something like this (but this a
>> UX concern). Example:
>>
>> bad app: 700MiB
>>
>> email: 100MiB (not distinguishing among regular use or use while
>> tethering)
>>
>> other app: 60MiB

> All I want to know is the difference of:
>
> getStats({
> component: "tethering-mobile:<iccid>",
> manifestURL: "<system app>"
> });
>
> and
>
> getStats({
> component: "tethering-mobile:<iccid>",
> manifestURL: null
> });
>
> IMO, the first one should return an empty set because tethering traffic
> is not generated by any on device app,
You are completely right here. My proposal is a matter of abstraction.
Our API combines a resource dimension (split into a sub-dimension for
the component) with an application dimension, you can ask which <app> is
using which <resource> through an specific <component>. This case,
resource = network, component = tethering-mobile:iccid but what about app?
> so no one should be blamed for
> it. The second call returns the very records set you want to know about
> tethering traffic usage.
manifestURL = null seems to me "out of dominion". What is the "null"
app? What does it mean for a Gaia developer? Gaia environment is
compound by apps that include an special & privileged application acting
as System. But I can agree on allowing 'null' as a valid value. Just to
remark: it seems a little bit inconsistent with the current approach.
>
> getStats({
> component: "mobile:<iccid>",
> manifestURL: "<system app>"
> });
>
> In contrast, above call gives the traffic generated by system app in and
> out via interface "mobile:<iccid>".
Agree.

[Ehsan Akhgari]

On 2014-08-20, 10:18 AM, Salvador de la Puente González wrote:
> Hi Vicamo, thank you for the clarifications.
>
> On 20/08/14 13:36, You-Sheng Yang wrote:

>>>>> Although it could be, when an application is consuming
>>>>> data while tethering is on, it is using the "tethering-mobile:<iccid>"
>>>>> component,

>> False. I thought that should always be "mobile:iccid" no matter
>> tethering is turned on or off. All traffic send and received by device
>> WebApps are accounted on either "wifi:<N>" or "mobile:<iccid>".
> Perfect then, we agree on this. I was not sure about my statement so I
> said "it could be". I think it should always be "mobile:iccid" too, just
> as you say.

I agree too!

>>>> IMHO, from a user point of view this "inner usage" is no more
>>>> than regular data usage, i.e, "mobile:<iccid>", that will increase per
>>>> app network stats (because originates **inside** the device, what is
>>>> not
>>>> the purpose of tethering).

>> I think I'm lost here. Let's say there are two interfaces when WiFi
>> tethering is turned on: wlan0 for WiFi, rmnet0 for mobile data. rmnet0
>> has the default route. So the traffic are accounted to:
>>
>> 1) on device apps browse internet => mobile:<iccid>
>> 2) on device apps talk to laptop app => wifi:0
>> 3) laptop browse internet => tethering-mobile:<iccid>
>>
>> Does this help?
> A lot. I was only considering routes 1 and 3. What I was saying is the
> user probably only sees two kinds of usage: a inner usage (your case 1)
> and a external usage (your case 3). Now, I'm realizing it could be
> another more when apps communicate via WiFi. Just as you say, case 1
> ends in "mobile:<iccid>" and case 3 in "tethering-mobile:<iccid>" (we
> were saying the same, \o/).
>
> For the case 2, just notice we need some way to identify wlan0 is the
> interface being used whe enabling WiFi hotspot.

Yep. I don't know if that's possible or not, but I'm willing to believe
You-Sheng on this.

>>>> As you're tethering, if you open the email app to check your email on
>>>> the device, do you really expect that usage to be accounted as part of
>>>> the tethering usage? More specifically, if you have a bad app on your
>>>> device that starts to consume large amounts of data without your
>>>> consent, it would appear as if an app on your laptop has consumed this
>>>> data. That sounds misleading to me.
>>> What I expect if I open the email app to check my email on the device is
>>> more data usage (regular data usage, not tethering at all) to be
>>> accounted for the email app. If I have a bad app that consumes tons of
>>> data from my device, I don't expect it counts for tethering but I review
>>> the data usage and see what is that bad app. Do you understand?

>> Sorry, but I don't see the connection between your previous reply and
>> this.
> Ehsan asked: "do you really expect that usage to be accounted as part of
> the tethering usage?" and I answered that what I expect is data
> accounting in regular data usage ("mobile:iccid") as we just agreed before.

OK great so we agree on this too.

>>>>> In the other hand, when I use the hotspot from a laptop, data is
>>>>> routed
>>>>> to my laptop via System App or Gecko (this second, I suppose). To keep
>>>>> consistency of which <app> is spending the resource, I think the best
>>>>> option is the system app, i.e, the app Gecko loads as privileged
>>>>> one and
>>>>> communicates with.
>>>> But how do we differentiate that with other potential data that the
>>>> system app might consume for other purposes?
>>> Sorry, I don't get you. I was referring to the case I open the email
>>> client on my laptop, surf the web or see some Vimeo's videos then I
>>> expect the usage to be accounted into the System app or maybe some
>>> "tethering entry" in the list of apps or something like this (but this a
>>> UX concern). Example:
>>>
>>> bad app: 700MiB
>>>
>>> email: 100MiB (not distinguishing among regular use or use while
>>> tethering)
>>>
>>> other app: 60MiB

manifestURL == null in this API means "give me the overall usage, I
don't care about which app". You can basically assume null means "any"
here. I agree with You-Sheng that it makes sense to only return the
tethering data when you set the manifest URL to none, because
technically, none of your apps (and definitely not the system app) have
used any of the tethering data.

Do you think this makes sense?

Thanks!
Ehsan

[Salvador de la Puente González]

>>>>>> In the other hand, when I use the hotspot from a laptop, data is
>>>>>> routed
>>>>>> to my laptop via System App or Gecko (this second, I suppose). To
>>>>>> keep
>>>>>> consistency of which <app> is spending the resource, I think the
>>>>>> best
>>>>>> option is the system app, i.e, the app Gecko loads as privileged
>>>>>> one and
>>>>>> communicates with.
>>>>> But how do we differentiate that with other potential data that the
>>>>> system app might consume for other purposes?
>>>> Sorry, I don't get you. I was referring to the case I open the email
>>>> client on my laptop, surf the web or see some Vimeo's videos then I
>>>> expect the usage to be accounted into the System app or maybe some
>>>> "tethering entry" in the list of apps or something like this (but
>>>> this a
>>>> UX concern). Example:
>>>>
>>>> bad app: 700MiB
>>>>
>>>> email: 100MiB (not distinguishing among regular use or use while
>>>> tethering)
>>>>
>>>> other app: 60MiB

This is my only concern. If we are asking for the usage that certain app
is making and with |manifestURL=null| we say "all aggregated usage" and
there is no app using the component |tethering-mobile:iccid|, then the
aggregate should be 0 (a sum of empty applications). But we are not
returning 0, we are returning the traffic routed from Vicamo's case 3 by
Gecko (which is something distinct than an app).

To distinguish between aggregated (manifestURL undefined or null) and
usage from Gecko, we could use a special keyword 'system' (to no be tied
to Gecko) or 'navigator' for manifestURL to indicate the developer wants
the usage made by Gecko itself. This allow us to simply list the
navigator entry among the applications and label it |{{Brandname}}
internal usage| in the UI.

Thank you!

>
> Thanks!

[Salvador de la Puente González]

Hi Vicamo.

On 25/08/14 06:35, You-Sheng Yang wrote:
> On 8/21/14, 3:50 PM, Salvador de la Puente González wrote:
> Hi,
>
> Please see
> https://groups.google.com/d/msg/mozilla.dev.webapi/tWkgbD1v_Gg/5pMH3KweITYJ
>
> Bug 1042149 is open for per-app statistics. Currently we have to 1.
> retrieve the full installed app list from mozApp API, 2. for each
> installed app, query its statistics with the manifestURL assigned from
> ResourceStats API.
Yes. Indeed, I'm reviewing bug 1033549 where the pattern consisting on
querying for all apps and then, making a request is observable.
> Apparently if we may provide an "any" semantics that returns all but
> separated statistic data for each app, it will be much much easier for
> Gaia developers. "Any" here doesn't equal to "all", "sum", etc.
Yep, some shortcuts would be appreciated.
>
> "system" or "navigator" seem to bring me an implication that the
> returned traffic statistics were generated by system in the background
> or the navigator itself. So in my original post I used "all" or "sum" to
> avoid such implication.
But the problem is that tethering routing traffic is being originated by
something that is not an application. I agree on it should be took into
account inside an aggregated query but **who** is originating this
traffic? It is not an app for sure, at least not a regular one. Thus I
wanted to propose including some like a "virtual app" named system /
operating-system / navigator or whatever you think it fits well.
>
> In summary, maybe we may turn the manifestURL field into:
>
> (DOMString or sequence<DOMString>)? manifestURL;
>
> where one or more valid manifestURLs mean the caller wants separated
> statistic data for each app specified; literal string "any" means
> separated statistic data for each available apps (in the database);
> literal string "sum" means the aggregated sum of the specified component.
I agree on 'any' as a valid wildcard for manifestURL (or your `origin`
proposal) but `sum` does not sound like an origin so I would reword it
as `all`.
>
> So, in order to query stats data for tethering, one should have:
>
> getStats({
> component: "tethering-mobile:<iccid>"
> manifestURL: "sum" // any other string returns an empty set.
> });
I agree (changing sum by all) but notice you should be able of querying
about the separated components in the summation but in tethering and you
can not since there is no origin for Gecko.
>
> However, that makes me feel the attribute name "manifestURL" doesn't
> reflect all the meanings it may have. Maybe "origin"? Do I just bother
> too much?
The origin alternative feels the best to me.
>
> Actually, there has been a little demon in my mind yelling that
> "tethering-foo" thing is really really ugly. He'd like to know if a new
> resource type "tethering" would be better? Then we may the same name in
> that components field. With "manifestURL" renamed to "origin" as well,
> for querying stats data of "tethering" resource type, one may have:

>
> getStats({
> component: "mobile:<iccid>",

> origin: "wifi:0"
> });
This is just like having the virtual app `tethering`! The main
difference is that if you refer to wifi:0 you are expanding the
semantics for `origin` field to include "hardware interfaces" while if
you stuck on "virtual apps" you can refer to those origins in a more
homogeneous fashion.

Another use case. Consider count how much data usage you expend in
downloading files. You could use download-manager as the virtual app /
system service for that kind of data. Some simple method such as
`getVirtualApps()` or `getSystemServices()` could return the list of
available origins.

Notice virtual apps are very similar to the former system services but
the main difference is very important. New virtual apps / system
services are valid origins (special origins), not a different dimension
to asking for so it is not possibly to ask for the tethering usage made
by the application <app> but for the tethering usage originated from the
tethering service.
>
> which means tethering traffic with source interface "wifi:0" and
> destination interface "mobile:<iccid>". A "sum" in origin field still
> means the aggregated sum of tethering traffic specified by the component
> field. "any" will then returns stats data for each individual available
> origin.
>
What do you think?

[Ehsan Akhgari]

On 2014-08-25, 12:35 AM, You-Sheng Yang wrote:

> Hi,
>
> Please see
> https://groups.google.com/d/msg/mozilla.dev.webapi/tWkgbD1v_Gg/5pMH3KweITYJ
>
> Bug 1042149 is open for per-app statistics. Currently we have to 1.
> retrieve the full installed app list from mozApp API, 2. for each
> installed app, query its statistics with the manifestURL assigned from
> ResourceStats API.
>

> Apparently if we may provide an "any" semantics that returns all but
> separated statistic data for each app, it will be much much easier for
> Gaia developers. "Any" here doesn't equal to "all", "sum", etc.

Hmm, do you mean "any" should mean |the total data - sum(what getStats
would return for each manifest URL)|? I don't understand how that
simplifies what gaia needs to do. Can you please elaborate on that?

> "system" or "navigator" seem to bring me an implication that the
> returned traffic statistics were generated by system in the background
> or the navigator itself. So in my original post I used "all" or "sum" to
> avoid such implication.

I still don't understand why a null manifestURL is not enough. The way
I see it, there are two use cases here:

1) How much data has app X consumed?
2) How much data have I consumed?

The first use case is useful for knowing if you have certain apps that
consume more data than you would expect. The second use case is useful
for knowing how much data has been used on your device as far as the
network operator is concerned (to know if you are close to exceed your
bandwidth cap, etc.)

Current you can achieve (1) with passing a manifestURL, and (2) with
passing null.

What are the use cases for the "sum", "all", "system", etc.?

> In summary, maybe we may turn the manifestURL field into:
>
> (DOMString or sequence<DOMString>)? manifestURL;
>
> where one or more valid manifestURLs mean the caller wants separated
> statistic data for each app specified; literal string "any" means
> separated statistic data for each available apps (in the database);
> literal string "sum" means the aggregated sum of the specified component.

Accepting several manifest URLs in one go probably makes sense if that
is useful when using the API.

> So, in order to query stats data for tethering, one should have:
>
> getStats({
> component: "tethering-mobile:<iccid>"
> manifestURL: "sum" // any other string returns an empty set.
> });
>

> However, that makes me feel the attribute name "manifestURL" doesn't
> reflect all the meanings it may have. Maybe "origin"? Do I just bother
> too much?

Well, the name reflects all of the meanings it may have in the current
API. :-) The name "origin" is very misleading because origin means
something specific and very different to manifest URLs (which are
effectively app identifiers).

> Actually, there has been a little demon in my mind yelling that
> "tethering-foo" thing is really really ugly. He'd like to know if a new
> resource type "tethering" would be better? Then we may the same name in
> that components field. With "manifestURL" renamed to "origin" as well,
> for querying stats data of "tethering" resource type, one may have:
>
> getStats({
> component: "mobile:<iccid>",
> origin: "wifi:0"
> });
>

> which means tethering traffic with source interface "wifi:0" and
> destination interface "mobile:<iccid>". A "sum" in origin field still
> means the aggregated sum of tethering traffic specified by the component
> field. "any" will then returns stats data for each individual available
> origin.

What use case does this serve? If you had multiple wifi interfaces,
would you as a user care which one was used during tethering?

[Ehsan Akhgari]

On 2014-08-25, 9:15 AM, Salvador de la Puente Gonz�lez wrote:
>> Actually, there has been a little demon in my mind yelling that
>> "tethering-foo" thing is really really ugly. He'd like to know if a new
>> resource type "tethering" would be better? Then we may the same name in
>> that components field. With "manifestURL" renamed to "origin" as well,
>> for querying stats data of "tethering" resource type, one may have:
>>
>> getStats({
>> component: "mobile:<iccid>",
>> origin: "wifi:0"
>> });

> This is just like having the virtual app `tethering`! The main
> difference is that if you refer to wifi:0 you are expanding the
> semantics for `origin` field to include "hardware interfaces" while if
> you stuck on "virtual apps" you can refer to those origins in a more
> homogeneous fashion.

I don't think this is useful, I suggest going back to think about what
use case this is supporting.

> Another use case. Consider count how much data usage you expend in
> downloading files. You could use download-manager as the virtual app /
> system service for that kind of data. Some simple method such as
> `getVirtualApps()` or `getSystemServices()` could return the list of
> available origins.

Isn't that information already available through the download manager?
I'm not sure what use case only cares about downloaded files and not
other big things you can download in the browser (such as videos, music,
etc.)

> Notice virtual apps are very similar to the former system services but
> the main difference is very important. New virtual apps / system
> services are valid origins (special origins), not a different dimension
> to asking for so it is not possibly to ask for the tethering usage made
> by the application <app> but for the tethering usage originated from the
> tethering service.

These virtual apps seem to be things that we need to support
individually, so I think they complicate the implementation quite a bit,
so I'd like to avoid this idea unless there is a very strong use case
that cannot be solved in any other way.

[Salvador de la Puente González]

Hi Eshan

On 26/08/14 20:52, Ehsan Akhgari wrote:

> On 2014-08-25, 9:15 AM, Salvador de la Puente González wrote:
>>> Actually, there has been a little demon in my mind yelling that
>>> "tethering-foo" thing is really really ugly. He'd like to know if a new
>>> resource type "tethering" would be better? Then we may the same name in
>>> that components field. With "manifestURL" renamed to "origin" as well,
>>> for querying stats data of "tethering" resource type, one may have:
>>>
>>> getStats({
>>> component: "mobile:<iccid>",
>>> origin: "wifi:0"
>>> });
>> This is just like having the virtual app `tethering`! The main
>> difference is that if you refer to wifi:0 you are expanding the
>> semantics for `origin` field to include "hardware interfaces" while if
>> you stuck on "virtual apps" you can refer to those origins in a more
>> homogeneous fashion.
>
> I don't think this is useful, I suggest going back to think about what
> use case this is supporting.

This is supporting "get tethered traffic". In a nutshell:

1. Tethering traffic can represent a big chunk of data consumption.
2. The user want to know about which apps are consuming mobile data.
3. So the user would want to know about per-app usage and tethered
traffic.
4. When a Gaia developers want to show usage per-app, they use
ResourceAPI.
5. They can pass manifestURL to specify which app.
6. But the tethered traffic is not generated by any app.
7. It is generated by something inside Gecko.
8. This "something" can be modelled as a virtual app or system service.
9. If not, any extension in manifestURL would seem a hack as, as you
say, manifestURL are app identifiers and tethered traffic is not
generated by an app.

Do you understand?

>
>> Another use case. Consider count how much data usage you expend in
>> downloading files. You could use download-manager as the virtual app /
>> system service for that kind of data. Some simple method such as
>> `getVirtualApps()` or `getSystemServices()` could return the list of
>> available origins.
>
> Isn't that information already available through the download manager?
> I'm not sure what use case only cares about downloaded files and not
> other big things you can download in the browser (such as videos,
> music, etc.)

Yes, but we could provide it as well.

>
>> Notice virtual apps are very similar to the former system services but
>> the main difference is very important. New virtual apps / system
>> services are valid origins (special origins), not a different dimension
>> to asking for so it is not possibly to ask for the tethering usage made
>> by the application <app> but for the tethering usage originated from the
>> tethering service.
>
> These virtual apps seem to be things that we need to support
> individually, so I think they complicate the implementation quite a
> bit, so I'd like to avoid this idea unless there is a very strong use
> case that cannot be solved in any other way.

Ok. Read above about why I'm suggesting this strategy.

Cheers!

[Ehsan Akhgari]

On 2014-08-26, 4:11 PM, Salvador de la Puente Gonz�lez wrote:
> Hi Eshan
>
> On 26/08/14 20:52, Ehsan Akhgari wrote:

Yes, and I think the current API is able to satisfy this use case. That
is what a manifestURL of null is intended for (give me the data for
everything, I don't care which app specifically.)

Why is that not good enough?

[vic...@gmail.com]

There are two different things to be addressed -- all and any.

By 'all' I mean all the traffic ever generated, by or not by the device
itself. There should be only one group of stats records.

By 'any' I mean the separated traffic stats generated by each installed
webapp or any other enumerable origins. The returned should be multiple
groups of stats records.
On Aug 27, 2014 4:21 AM, "Ehsan Akhgari" <ehsan....@gmail.com> wrote:

> On 2014-08-26, 4:11 PM, Salvador de la Puente González wrote:
>
>> Hi Eshan
>>
>> On 26/08/14 20:52, Ehsan Akhgari wrote:
>>

[Ehsan Akhgari]

On 2014-08-26, 4:38 PM, vic...@gmail.com wrote:
> There are two different things to be addressed -- all and any.
>
> By 'all' I mean all the traffic ever generated, by or not by the device
> itself. There should be only one group of stats records.

This is manifestURL == null.

> By 'any' I mean the separated traffic stats generated by each installed
> webapp or any other enumerable origins. The returned should be multiple
> groups of stats records.

This is what you get when you call getStats for each one of those
manifest URLs.

It seems like both of of thee are possible with the current API.

> On Aug 27, 2014 4:21 AM, "Ehsan Akhgari" <ehsan....@gmail.com

[Salvador de la Puente González]

Simply because it is not consistent as tethered traffic is not
originated by an application.

So if you want "manifestURL:null" to mean "any application" and tethered
traffic is not originated by an application, then you should not see the
tethered traffic in the aggregated statistics.

Correct me if I'm wrong. If I want to ask for **tethering only**, should
I ask for "any" traffic (manifestURL:null) and substract the sum of all
app traffic (manifestURL:[manifest1, manifest2, ...])?

If so, this is quite weird because this subtraction should return 0.

If not, what value should I provide to manifestURL in order to get the
tethered traffic only?

[Vicamo Yang]

Good. Bug 1042149 closed.

[Ehsan Akhgari]

On 2014-08-26, 4:55 PM, Salvador de la Puente Gonz�lez wrote:
> On 26/08/14 22:21, Ehsan Akhgari wrote:

>> On 2014-08-26, 4:11 PM, Salvador de la Puente Gonz�lez wrote:
>>> Hi Eshan
>>>
>>> On 26/08/14 20:52, Ehsan Akhgari wrote:

No, manifestURL:null doesn't mean "any application", it means "I don't
care". Those are different. The latter means that the caller doesn't
care whether the data is consumed by an application at all. (I agree
that the difference is subtle.)

> Correct me if I'm wrong. If I want to ask for **tethering only**, should
> I ask for "any" traffic (manifestURL:null) and substract the sum of all
> app traffic (manifestURL:[manifest1, manifest2, ...])?

No. Please note that for tethering, if you set the manifestURL to *any*
valid manifest URL, you will receive 0, because no application on the
device has consumed the data. If you ask for the total amount of data
consumed by tethering however (though manifestURL: null) then you will
get the correct amount.

I think you're assuming that the return value of getStats with
manifestURL:null is equal to the sum of the return values if you
iterated over every single manifest URL and called getStats on it.
That's not a correct assumption though.

> If so, this is quite weird because this subtraction should return 0.

FWIW the sum would be 0, so even if you did the subtraction you would
still have the correct value.

> If not, what value should I provide to manifestURL in order to get the
> tethered traffic only?

null.

Cheers,
Ehsan

[Salvador de la Puente González]

Hi Eshan

A lot of confusion here but I think it's solved and, as you say, current
API is enough.

On 27/08/14 19:44, Ehsan Akhgari wrote:

> On 2014-08-26, 4:55 PM, Salvador de la Puente González wrote:
>> On 26/08/14 22:21, Ehsan Akhgari wrote:

>>> On 2014-08-26, 4:11 PM, Salvador de la Puente González wrote:
>>>> Hi Eshan
>>>>
>>>> On 26/08/14 20:52, Ehsan Akhgari wrote:

There are lot of definitions of `null` in this thread. Some lines ago
you say "That is what a manifestURL of null is intended for (give me the
data for everything, I don't care which app specifically.)" and now you
say "The latter means that the caller doesn't care whether the data is

consumed by an application at all".

I got confused: you don't care about **which app** or you don't care
about an application **at all**?

Anyway to make the current API enough we should deal with a clear reword
of the second definition: `null` means `not originated from an application`.

>
>> Correct me if I'm wrong. If I want to ask for **tethering only**, should
>> I ask for "any" traffic (manifestURL:null) and substract the sum of all
>> app traffic (manifestURL:[manifest1, manifest2, ...])?
>
> No. Please note that for tethering, if you set the manifestURL to
> *any* valid manifest URL, you will receive 0, because no application
> on the device has consumed the data. If you ask for the total amount
> of data consumed by tethering however (though manifestURL: null) then
> you will get the correct amount.

I forgot that for tethering, component would be set to
"tethering-mobile:<iccid>". Isn't it? Even so, the subtraction paradox
holds.

> I think you're assuming that the return value of getStats with
> manifestURL:null is equal to the sum of the return values if you
> iterated over every single manifest URL and called getStats on it.
> That's not a correct assumption though.

Yep. That was I though. Reasons above.

>
>> If so, this is quite weird because this subtraction should return 0.
>
> FWIW the sum would be 0, so even if you did the subtraction you would
> still have the correct value.

Considering the second definition of manifestURL: null, you are right.

>
>> If not, what value should I provide to manifestURL in order to get the
>> tethered traffic only?

> ld
> null.
Clarified this, I would explain manifestURL saying it does not apply for
some components that are "gecko-restricted" and I would make fail the
DOM request or reject the promise if trying to use a valid manifestURL
with one of these restricted components. But its ok to me if you want to
say app usage is 0 for tethering (after all, it is true) but what seems
weird to me is that I can ask for:

'All tethering'
manifestURL: null --> tetheringFrom(manifest_1) +
tetheringFrom(manifest_2) + ... + tetheringFrom(gecko) = 0 + 0 + ... + x

And for:
'Tethering from manifest_1'
manifestURL: manifest_1 --> tetheringFrom(manifest_1) = 0

And **not** for:
'Tethering from gecko specifically'
manifestURL: ??? --> tetheringFrom(gecko)

And I need to rely on tetheringFrom(any manifest) will return 0.

That is what all my concerns are about.

Cheers!

[Ehsan Akhgari]

On 2014-08-28, 6:20 AM, Salvador de la Puente Gonz�lez wrote:
> Hi Eshan
>
> A lot of confusion here but I think it's solved and, as you say, current
> API is enough.

Yeah, I think we're basically stuck on the terminology at this point. :-)

> On 27/08/14 19:44, Ehsan Akhgari wrote:

>> On 2014-08-26, 4:55 PM, Salvador de la Puente Gonz�lez wrote:
>>> On 26/08/14 22:21, Ehsan Akhgari wrote:

>>>> On 2014-08-26, 4:11 PM, Salvador de la Puente Gonz�lez wrote:
>>>>> Hi Eshan
>>>>>
>>>>> On 26/08/14 20:52, Ehsan Akhgari wrote:

No. I think I'm struggling to find alternative ways of saying this:

manifestURL: null means that the caller does not care about the entity
that consumed the data, whether it's an app, Gecko internally, other
daemons and whatnot that may be running on the device, services such as
tethering originated from other devices, etc. When you specify null,
the value that getStats returns may include any of the above sources,
and maybe others that I cannot think of right now as well.

Now it so happens that for tethering, there is ever only *one* source of
data, and that is an external device, so manifestURL: null will return
that value. _If_ the tethering could also be initiated from an app too
for the sake of argument, then that data would be included in the return
value of getStats as well.

I'm not sure how to clarify what I mean here further...

>>> Correct me if I'm wrong. If I want to ask for **tethering only**, should
>>> I ask for "any" traffic (manifestURL:null) and substract the sum of all
>>> app traffic (manifestURL:[manifest1, manifest2, ...])?
>>
>> No. Please note that for tethering, if you set the manifestURL to
>> *any* valid manifest URL, you will receive 0, because no application
>> on the device has consumed the data. If you ask for the total amount
>> of data consumed by tethering however (though manifestURL: null) then
>> you will get the correct amount.
> I forgot that for tethering, component would be set to
> "tethering-mobile:<iccid>". Isn't it?

Sure.

> Even so, the subtraction paradox holds.

There is no paradox. You have:

Data_From_Unknown_Sources = 10MB // unknown sources being an external
device here
Data_From_Apps = 0

and then you call getStats with "tethering-mobile:<iccid>" and
manifestURL: null, and you get 10MB.

getStats_ReturnValue - Data_From_Apps == Data_From_Unknown_Sources

>> I think you're assuming that the return value of getStats with
>> manifestURL:null is equal to the sum of the return values if you
>> iterated over every single manifest URL and called getStats on it.
>> That's not a correct assumption though.
> Yep. That was I though. Reasons above.
>>
>>> If so, this is quite weird because this subtraction should return 0.
>>
>> FWIW the sum would be 0, so even if you did the subtraction you would
>> still have the correct value.
> Considering the second definition of manifestURL: null, you are right.
>>
>>> If not, what value should I provide to manifestURL in order to get the
>>> tethered traffic only?
>> ld
>> null.
> Clarified this, I would explain manifestURL saying it does not apply for
> some components that are "gecko-restricted" and I would make fail the
> DOM request or reject the promise if trying to use a valid manifestURL
> with one of these restricted components.

Why? A 0 return value is a very valid response. "How much tethering
data did the email app consume on my device?" "None!"

We should only reject the DOMRequest to signal an error.

> But its ok to me if you want to
> say app usage is 0 for tethering (after all, it is true) but what seems
> weird to me is that I can ask for:
>
> 'All tethering'
> manifestURL: null --> tetheringFrom(manifest_1) +
> tetheringFrom(manifest_2) + ... + tetheringFrom(gecko) = 0 + 0 + ... + x
>
> And for:
> 'Tethering from manifest_1'
> manifestURL: manifest_1 --> tetheringFrom(manifest_1) = 0

Yep, these are correct.

> And **not** for:
> 'Tethering from gecko specifically'
> manifestURL: ??? --> tetheringFrom(gecko)

The current API does not provide any way of querying this, because we
have not had any use cases for getting some specific resource
consumption from Gecko itself.

> And I need to rely on tetheringFrom(any manifest) will return 0.

It will.

> That is what all my concerns are about.

I hope the above clarifies everything.

Cheers,
Ehsan

[Salvador de la Puente González]

Hi Eshan!

On 28/08/14 17:16, Ehsan Akhgari wrote:
> No. I think I'm struggling to find alternative ways of saying this:
>
> manifestURL: null means that the caller does not care about the entity
> that consumed the data, whether it's an app, Gecko internally, other
> daemons and whatnot that may be running on the device, services such
> as tethering originated from other devices, etc.

If you feel the name manifestURL is the best suited for indicating (by
passing a `null` value) "the sum of all sources: apps, Gecko internals,
daemons and whatever" then ok. I don't like it. You have why a couple of
paragraphs later.

> When you specify null, the value that getStats returns may include any
> of the above sources, and maybe others that I cannot think of right
> now as well.

As clarification. How they will be included? As added data?

> Now it so happens that for tethering, there is ever only *one* source
> of data, and that is an external device, so manifestURL: null will
> return that value. _If_ the tethering could also be initiated from an
> app too for the sake of argument, then that data would be included in
> the return value of getStats as well.
>
> I'm not sure how to clarify what I mean here further...

Don't worry, I get you, it's simply I don't like it.

>
> There is no paradox. You have:
>
> Data_From_Unknown_Sources = 10MB // unknown sources being an external
> device here
> Data_From_Apps = 0
>
> and then you call getStats with "tethering-mobile:<iccid>" and
> manifestURL: null, and you get 10MB.

Yep, but passing only valid values to manifestURL and adding them
together I will never reach that usage because I can not ask for the
unknown sources with a valid manifestURL. This, exactly, is what I don't
like.

> Yep, these are correct.
>
>> And **not** for:
>> 'Tethering from gecko specifically'
>> manifestURL: ??? --> tetheringFrom(gecko)
>
> The current API does not provide any way of querying this, because we
> have not had any use cases for getting some specific resource
> consumption from Gecko itself.

The user case is the tethered traffic, but you're relaying on the fact
that the rest of the sources will return 0.

>
>> And I need to rely on tetheringFrom(any manifest) will return 0.
>
> It will.

Yep. That's the hack allowing the current API to be enough.

>
>> That is what all my concerns are about.
>
> I hope the above clarifies everything.

Thank you Eshan, It's crystal clear now but I don't like it. The main
reason is that I gave you a couple of paragraphs before.

To be constructive, we could rename manifestURL by "origin" as Vicamo
was suggesting. Then, valid manifestURL as well as special keywords
standing for those unknown origins could be all valid fields. `null`
could be still indicating "I don't care" but in the sense of "I don't
care about the origin". At least you could tag the unknown sources and
allow now and future to ask only for them without relaying on being the
unique source of the data consumption.

Hope it helps!

[Ehsan Akhgari]

On 2014-08-28, 12:26 PM, Salvador de la Puente Gonz�lez wrote:
> Hi Eshan!
>
> On 28/08/14 17:16, Ehsan Akhgari wrote:
>> No. I think I'm struggling to find alternative ways of saying this:
>>
>> manifestURL: null means that the caller does not care about the entity
>> that consumed the data, whether it's an app, Gecko internally, other
>> daemons and whatnot that may be running on the device, services such
>> as tethering originated from other devices, etc.
> If you feel the name manifestURL is the best suited for indicating (by
> passing a `null` value) "the sum of all sources: apps, Gecko internals,
> daemons and whatever" then ok. I don't like it. You have why a couple of
> paragraphs later.

OK, that's fair!

>> When you specify null, the value that getStats returns may include any
>> of the above sources, and maybe others that I cannot think of right
>> now as well.
> As clarification. How they will be included? As added data?

No, just the normal return value of getStats.

>> Now it so happens that for tethering, there is ever only *one* source
>> of data, and that is an external device, so manifestURL: null will
>> return that value. _If_ the tethering could also be initiated from an
>> app too for the sake of argument, then that data would be included in
>> the return value of getStats as well.
>>
>> I'm not sure how to clarify what I mean here further...
> Don't worry, I get you, it's simply I don't like it.
>>
>> There is no paradox. You have:
>>
>> Data_From_Unknown_Sources = 10MB // unknown sources being an external
>> device here
>> Data_From_Apps = 0
>>
>> and then you call getStats with "tethering-mobile:<iccid>" and
>> manifestURL: null, and you get 10MB.
> Yep, but passing only valid values to manifestURL and adding them
> together I will never reach that usage because I can not ask for the
> unknown sources with a valid manifestURL. This, exactly, is what I don't
> like.

OK, I get that. But the reason why I think this is fine is that there
is no solid use case for the summation you mention and manifestURL: null
to return the same thing. That _does_ make sense if the semantics of
that null were what you expect, but they're not.

I agree that this is not ideal, but I cannot think of a better way to
improve things without making the API needlessly more complicated... If
you have suggestions, please let me know! But I do care a lot about the
API working well for the typical use cases that people have, and I just
don't understand why the above is an important use case.

Perhaps we can improve things by adding more documentation?

>> Yep, these are correct.
>>
>>> And **not** for:
>>> 'Tethering from gecko specifically'
>>> manifestURL: ??? --> tetheringFrom(gecko)
>>
>> The current API does not provide any way of querying this, because we
>> have not had any use cases for getting some specific resource
>> consumption from Gecko itself.
> The user case is the tethered traffic, but you're relaying on the fact
> that the rest of the sources will return 0.

Yes indeed.

>>> And I need to rely on tetheringFrom(any manifest) will return 0.
>>
>> It will.
> Yep. That's the hack allowing the current API to be enough.
>>
>>> That is what all my concerns are about.
>>
>> I hope the above clarifies everything.
> Thank you Eshan, It's crystal clear now but I don't like it. The main
> reason is that I gave you a couple of paragraphs before.
>
> To be constructive, we could rename manifestURL by "origin" as Vicamo
> was suggesting. Then, valid manifestURL as well as special keywords
> standing for those unknown origins could be all valid fields. `null`
> could be still indicating "I don't care" but in the sense of "I don't
> care about the origin". At least you could tag the unknown sources and
> allow now and future to ask only for them without relaying on being the
> unique source of the data consumption.

Like I said elsewhere in this thread, origin is a bad name to use
because it means something very different than a manifestURL, please see
<https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy#Definition_of_an_origin>.
To be more clear, if AwesomeApp redirects to facebook.com for
authentication, the data consumed there is from the facebook.com
"origin" but still counted within the usage of AwesomeApp. That is why
we should not conflate origins and manifests together.

The other issue is that I am not really keen on the idea of tagging all
of the additional sources because of two reasons. One is that I believe
the API in its current form is able to address all of the use cases that
we have had for it (including the tethering use case, albeit with weird
semantics that may be confusing), and introducing the notion of virtual
apps/services only complicates things more (we would have to provide
ways of obtaining those virtual apps/service names, make sure they make
sense, keep them updated, etc.) And all of that will effectively give
us different ways of solving the same use cases as we can solve with the
current API, and will not solve any new use cases. The second reason
why I don't like that idea is that it's very difficult to implement.
Gecko code running in the parent process, for example, can initiate
network activity for a variety of reasons that are very hard to classify
in the virtual apps/services buckets (and require manual coding so it's
hard to keep them working correctly over time, as opposed to the
resource usage of apps, which can be more easily queried because apps
run in OS level processes separate from each other). And there is
things such as power usage. Do we want to have one of these virtual
apps per background daemons, for example? What if a partner ships a
device with a background daemon that we are not aware of? As you can
see, adopting this idea opens a can of worms, and I'd really like to
avoid that if possible.

Cheers,
Ehsan

[Salvador de la Puente González]

Hi Eshan.

On 28/08/14 18:53, Ehsan Akhgari wrote:

> On 2014-08-28, 12:26 PM, Salvador de la Puente González wrote:
>> Hi Eshan!
>>

Yeah, but I will propose another thing later.

Here we have an overlapping of terms because same term can mean
different things for different level of abstraction and remember the
consumer of the API does not want to know about the guts of the
implementation. But I agree on you that origin is a well known term so I
suggest instead `source`. source is entirely under our control. If you
provide an app, you receive traffic for an app. If you provide (right
now) the keyword `other` (or `non-app` or whatever you think it fits)
you receive the other data. Now it's enough to covering all the cases.
In the future, when more clear user cases arise, we can add other types,
even objects as values without interfering with `source` semantics.

>
> The other issue is that I am not really keen on the idea of tagging
> all of the additional sources because of two reasons. One is that I
> believe the API in its current form is able to address all of the use
> cases that we have had for it (including the tethering use case,
> albeit with weird semantics that may be confusing), and introducing
> the notion of virtual apps/services only complicates things more (we
> would have to provide ways of obtaining those virtual apps/service
> names, make sure they make sense, keep them updated, etc.) And all of
> that will effectively give us different ways of solving the same use
> cases as we can solve with the current API, and will not solve any new
> use cases.

I agree on no adding more complexity if we have all use cases covered
but let's try to recover that unknown source in the same way we recover.
Just because an homogeneous API is better for automation and I would
want to keep the API open for extension.

> The second reason why I don't like that idea is that it's very
> difficult to implement. Gecko code running in the parent process, for
> example, can initiate network activity for a variety of reasons that
> are very hard to classify in the virtual apps/services buckets (and
> require manual coding so it's hard to keep them working correctly over
> time, as opposed to the resource usage of apps, which can be more
> easily queried because apps run in OS level processes separate from
> each other). And there is things such as power usage.

Yes. I agree. This is why I suggested to simply add all Gecko traffic to
the system app. It is not true when you are aware about the internals of
Gecko but its very accurate when you move one level abstraction above
and consider System as the collaboration of Gecko + System app. Do you
get my point? For instance, in a future, we could add the `label` field
to get only certain kind of tagged data.

> Do we want to have one of these virtual apps per background daemons,
> for example?

Only if they provide visible network activity from a Firefox OS'
developer point of view.

> What if a partner ships a device with a background daemon that we are
> not aware of? As you can see, adopting this idea opens a can of
> worms, and I'd really like to avoid that if possible.

The partners could include their services in a list and would provide
their specific measurement implementations. If we are aware about the
total amounts of data transmitter we always could have a special
(computed as subtraction) tag called `unknown` and explain what this
value stands for.

Notice I'm not saying this is easy and I agree on not adding unnecessary
complexity. I'm only saying it is possible from a designing point of
view and it's not a bad idea.
>
> Cheers,
> Ehsan
I'm pushing on changing the name for two reasons:

+ Because it's meaningful from a Firefox OS' developer point of view.
With `source` the developer can accept other supported sources distinct
than manifestURL. And it is completely under our control: we say in the
docs that current implementation only supports `other` or `non-app`
right now but this could change as new use cases are detected.
+ Because it's meaningless from a Gecko's developer point of view. As
it can be the same (now) to passing `null` when asking for tethering and
returning 0 for other resources.

This way we leave a consistent public API without adding extra
complexity in Gecko.

Next week I'll be PTO but I'll be aware about email. Let me know what
you think.

Cheers!