Origin Trials with third party token and usage limits for Attribution Reporting API

[Gregory Truchetet]

Hi!

(Greg Truchetet from Teads here, this is a repost of https://github.com/WICG/conversion-measurement-api/issues/118, thank you John!)

We are using Origin Trials (Third party) to test Attribution Reporting API and what we observe is a bit disturbing. We are not sure to have fully understood how the limits applied to trials work.
This kind of question has been probably raised elsewhere but I didn't find any reference, if it's the case, sorry for the noise. :)

Our third party script, embedded on a lot of websites, write the meta tag with our trial token on every page. Then it checks that the feature is effectively enabled on the user's browser with that piece of code:

function isConversionMeasurementEnabled() {

   var isChrome = !!window.chrome && (!!window.chrome.webstore || !!window.chrome.runtime);

  return isChrome && !!document.featurePolicy && !!document.featurePolicy.features() && document.featurePolicy.features().includes('conversion-measurement');

}

It appears that for Chrome instances with a version greater than 86, the feature seems to be effectively activated for only 0.5%-1% of the traffic.
From what we understood by reading the Origin Trials restrictions here, the feature should be activated for everybody unless too many people use this feature worldwide, which is not the case when checking the Chrome Status page

Are we missing something and this is the intended behavior?

Thanks for your help!

[John Delaney]

Hi Greg,

(Thanks for posting here!)

You are correct that in Chrome 86-88, the API was only available for a fraction of Chrome browsers. Features available through Origin Trials are normally available on all Chrome browsers.

This behavior was by design, but we acknowledge that this was undesirable and confusing for some developers. As of Chrome 89, the Conversion Measurement feature is available through Origin Trials by default. You should see an increase in measured traffic as 89 is released. This will take time, you can read more about the release cycle here. The rollout of the Conversion Measurement behavior may be slower than the 89 release as well.

For additional context here is a snippet from an email that was recently sent to Origin Trial participants denoting the change. This also includes guidance if you wish to continue using the previous behavior.

```

Change #3: Origin trial token default configuration [origin trial change]When is this changing?
What's changing?

Up until now, Chrome was automatically selectively enabling the API for a small percentage of Chrome instances. Now, by default, the API will be enabled for all Chrome instances.

What should you do?

If you want the API to be enabled for all users by default: you don't need to do anything. This is the new default behaviour, starting from Chrome 89.

Else, if you want Chrome to continue enabling the API only for a subset of users that visit your origin trial sites: update your token as follows (before Chrome 89 is rolled out):

• Navigate to your active origin trials page page and click on Conversion Measurement.
• Click the pen icon to edit your registration.
• In the dropdown How is usage controlled, Select Exclude a subset of users.
• Note: if you don't do this, i.e. if you only renew your old tokens without changing them: on the origins where your token is added, the API will be enabled for all users. This means that your experiment will continue working, but the volume of end-users for which the conversion measurement feature is activated will increase. So if you're tracking metrics as part of your experiment, we recommend you take action.

See details at https://web.dev/using-conversion-measurement/#register-for-the-origin-trial.

```

There were other changes in 89 as well, such as a change to the `impressiondata` encoding.

Let me know if anything is unclear.

[Gregory Truchetet]

Thank you for your quick answer John.

It's probably too soon but we don't see any increase since Chrome 89, we'll monitor that in the next days.

My personal Chrome is already updated to the last version 89.0.4389.82 and when I visit https://www.adidas.fr/ where our third party script is dropping our trial token, after executing document.featurePolicy.features() in my console, I don't see any entry for "conversion-measurement". 

If I understand correctly it can be related to "The rollout of the Conversion Measurement behavior may be slower than the 89 release as well.", am I right? Basically, having a Chrome 89 does not mean that the feature is 100% enabled, it also depends on another rollout not necessarily effective yet.

Again, thank you for your help, much appreciated!

[John Delaney]

> If I understand correctly it can be related to "The rollout of the Conversion Measurement behavior may be slower than the 89 release as well.", am I right? 

> Basically, having a Chrome 89 does not mean that the feature is 100% enabled, it also depends on another rollout not necessarily effective yet.

Yes that is correct, I would give it a few days. Users also need to restart their browsers to get the new behavior.

[Gregory Truchetet]

Hi John!

Getting back to you on this topic. Unfortunately, these past few days we've seen a decrease in the number of calls we receive from Chrome instances.

Here a screenshot representing this activity on our servers:

Quite weird, isn't it? I think we are doing something wrong but I'm not able to see what exactly.

This is the setup of our token:

Are we the only ones to see such weird behavior or can it be something global? :)

Cheers,

[John Delaney]

Hi Gregory,

Thank you for following up on this, and thank you for the attached data. 

After looking into this I can confirm this issue is on Chrome's end and was related to the aforementioned rollout. The issue been corrected, you should see the desired behavior moving forward (given all of the previously mentioned caveats on rollouts). Please let me know if you continue to have issues.

[Gregory Truchetet]

Hi John,

I can confirm that we see a huge increase in our incoming traffic since yesterday:

Great news, thank you! :)