Intent to Experiment: Federated Learning of Cohorts

[Yao Xiao‏]

Contact emails
yao...@chromium.org, jka...@chromium.org, kle...@google.com, m...@google.com

Explainer
https://github.com/WICG/floc

Summary
The FLoC API intends to provide callers (primarily ad-tech) with a "cohort label" shared by thousands of people with similar browsing habits to aid in interest-based online advertising. The browser will group people together into k-anonymous interest cohorts via purely on-device computation, and reveal the user’s cohort via script.

We would like to launch via an Origin Trial to start getting developer feedback.

Link to “Intent to Prototype” blink-dev discussion
https://groups.google.com/a/chromium.org/g/blink-dev/c/DpZZG5K1PWs/m/IJT1W69wCAAJ

Goals for experimentation
We hope to get feedback from ad tech suggesting whether FLoC is viable.

We’ll keep monitoring the anonymity and sensitivity properties based on UKM data, and will trigger the blockings of some cohorts or a full assignment / recomputation as necessary.

Experimental timeline
We expect to start the Origin Trial in M89 (Stable on Mar 2, 2021)

Risks

• Interoperability and Compatibility
  • No signals from other browsers.
• Activation
  
  • The API is straightforward to use but understanding how to interpret the meaning of an interest cohort may be complex.
    
• Privacy
  • The current implementation of FLoC for this Origin Trial creates a new fingerprinting surface in that it provides the same value across sites. However, it provides a far smaller fingerprinting surface than the APIs it’s intending to replace (third party storage). We’re looking at ways to reduce it further down the line (e.g., privacy budget).
    
  • The methods for ensuring that each cohort is large enough and not correlated with sensitive categories are described in the explainer.
  • The long-term intent is for FLoC calculation to be fully opt-in at the page level (e.g., pages that use the API would be included in FLoC calculation). For an Origin Trial where only a small sample of page loads will use the API, it will be necessary to include more sites in the cohort calculation in order to be useful. So, for the OT only, we will include pages in cohort calculation that we anticipate will use FLoC in the future, e.g., pages that have ad resources as determined by Chrome’s Ad Tagging.
  • We also provide an interest-cohort permissions policy for pages to opt out from FLoC completely.
  • End users who have disabled third-party cookies in or before M89 will not be included in the experiment. In M90 we intend to introduce a separate end user control.
  • This Origin Trial will be available in selected regions.
  • Please see the https://github.com/WICG/floc#privacy-and-security-considerations and https://github.com/WICG/floc/blob/master/security-and-privacy-self-review.md for more privacy questions & answers.

Any risks when the experiment finishes?
No. When the experiment finishes, the document.interestCohort API will stop being exposed, but the site is expected to handle that due to the OriginTrial only being enabled for some page visits.

Reason this experiment is being extended
N/A

Ongoing technical constraints
No

Will this feature be supported on all five Blink platforms supported by Origin Trials (Windows, Mac, Linux, Chrome OS, and Android)?
Yes

Link to entry on the feature dashboard
https://www.chromestatus.com/features/5710139774468096

[Abhinav Sinha‏]

As per the qualifying user's definition I do not see the feature & JS API (document.interestCohort) working in the latest chrome release 89.0.4389.82 ( Tested US and APAC region)
The chrome feature status also states this as behind developer flag. I was able to force enable API using blink feature but doesn't seem to be ideal for OT.

Can we please elaborate on the Origin Trial scope and coverage around following:

1. Is this FloC data available by default to only Chrome/Google at scale through account sync?
2. How can any publisher , ssp , advertiser analyse the floc id data at scale during OT phase?
3. Will this ID / API be available for all qualifying users for selected regions or there are additional sampling / selection criteria ?

Link to same query on GitHub - https://github.com/WICG/floc/issues/57

[Simon Lomax‏]

I am also unable to see document.interestCohort working in the latest release. 

Can you elaborate on how you force enabled this please?

[Abhinav Sinha‏]

I enabled it using following flag while starting chrome : --enable-blink-features=InterestCohortAPI 
MacOS : open -na Google\ Chrome --args --enable-blink-features=InterestCohortAPI

Windows append following in Chrome Shortcut : --enable-blink-features=InterestCohortAPI

- Abhinav

[Yao Xiao‏]

Hi,

The document.interestCohort API will be made available in a third-party Origin Trial (https://web.dev/third-party-origin-trials/). The Origin Trial hasn't started yet and we expect it to start in the middle of M89. When it starts, the API will be available on sites that sign up for the OT, subject to region restrictions and 5% user sampling.

For your questions specifically,
1. It’s available equally on all sites, for first or third parties that participate in the Origin Trial.
2. Ad tech are supposed to observe the browsing behavior of floc ids, and train their models based on it.
3. This API will initially be available for 5% of users in the selected regions. The user sampling is to ensure that the API usage doesn’t reach beyond OT thresholds.

On Tuesday, March 9, 2021 at 10:44:37 AM UTC-5 abhina...@pubmatic.com wrote:

[Rick Byers‏]

Thanks for the additional details. LGTM to experiment.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/2f56d608-9b1a-44e8-9390-8d1054d35166n%40chromium.org.

[Abhinav Sinha‏]

Thanks for the details.

Regards,

Abhinav

---

From: Yao Xiao <yao...@chromium.org>
Sent: Wednesday, 10 March, 2021, 2:38 am
To: blink-dev
Cc: Abhinav Sinha; Yao Xiao; Josh Karlin; Michael "Chromium" Kleber; Marshall Vale
Subject: Re: Intent to Experiment: Federated Learning of Cohorts

[Yao Xiao‏]

Hi API owners,

We'd like to request an increase from the standard “0.5% of page loads” origin trial usage target to 5%, because 0.5% of data would be too small for ad tech to train a sound ML model / to evaluate the effectiveness of the FLoC.

For whatever threshold that's finally allowed, we'll configure a user sampling to ensure it doesn't cross the OT threshold.

[Greg Truchetet‏]

Hi guys,

(not sure my previous message worked,  new try ;) )

We have noticed that FloC is now available on the Chrome trials page, but the third-party option is not present, unlike conversion-measurement.

Is it intended or is it because the rollout of the trial is not finished yet? :)

Thanks for your help

[Greg Truchetet‏]

Hi guys,

Not sure this is the right place to ask this question but we have noticed that FLoC is available on the Chrome Trials page but the third-party option is not available, unlike conversion-measurement.

Is it intended or is it because the rollout of the trial is not finished yet? :)

Thanks for your help

On Monday, March 15, 2021 at 10:38:15 PM UTC+1 yao...@chromium.org wrote:

[Josh Karlin‏]

Hi Greg. Thanks for pointing that out. Should be fixed now.

Josh

[Josh Karlin‏]

You're not missing anything. We need to be clearer on the rollout process here. 

Because there is strong interest in this OT, we're going to limit the population that has the API available to ensure that we don't exceed the page-load limit of the OT.  So if you have the OT token, you'll still need to feature detect the API. Right now we've enabled the API for 0% of the population. We'll eventually (looks like we're ~2 weeks away) raise it to either 0.5% or 5% of users depending on what gets approved in this thread.

We'll update this thread and the chromestatus entry when we start to roll out or if the timing changes.

Thanks,

Josh

On Fri, Mar 19, 2021 at 9:18 AM Greg Truchetet <minto...@gmail.com> wrote:

Thank you Josh, seems good :)

Another question, do you have a dedicated group for talking about the running trial or can we use this thread?

The feature activation with the token seems to work fine now but once activated, we get this error: document.interestCohort is not a function

Here is the concerned piece of code running on Chrome 89 instances:

const isChrome = !!window.chrome && (!!window.chrome.webstore || !!window.chrome.runtime);
if (isChrome && !!document.featurePolicy && !!document.featurePolicy.features() && document.featurePolicy.features().includes('interest-cohort')) {
  return await document.interestCohort() || {};
}

Are we missing something? :)

Thanks again for your help!

[Greg Truchetet‏]

Thank you Josh, seems good :)

Another question, do you have a dedicated group for talking about the running trial or can we use this thread?

The feature activation with the token seems to work fine now but once activated, we get this error: document.interestCohort is not a function

Here is the concerned piece of code running on Chrome 89 instances:

const isChrome = !!window.chrome && (!!window.chrome.webstore || !!window.chrome.runtime);
if (isChrome && !!document.featurePolicy && !!document.featurePolicy.features() && document.featurePolicy.features().includes('interest-cohort')) {
  return await document.interestCohort() || {};
}

Are we missing something? :)

Thanks again for your help!

On Friday, March 19, 2021 at 1:45:27 PM UTC+1 Josh Karlin wrote:

[Tom Jones‏]

I was left unsure about how we can see any impact. Can we enroll participants to be in the group? How can we measure the impact?

[Federico Nieves‏]

Hi everyone,

I would like to start testing flocs on my localhost, however I always receive the error message: "document.interestCohort is not a function". 

On about:flags there is no option to enable flocs, however as per Abhinav comment, I launched it with:

> MacOS : open -na Google\ Chrome --args --enable-blink-features=InterestCohortAPI

I also registered a new token for localhost.

Is there anything I'm missing here?

Thanks in advance.

[Michael Kleber‏]

The current version of https://web.dev/floc/ lists all the flags you would need to test on your own browser.

https://web.dev/floc/#as-a-web-developer-how-can-i-try-out-floc

You'll need:

--enable-blink-features=InterestCohortAPI
--enable-features="FederatedLearningOfCohorts:update_interval/10s/minimum_history_domain_size_required/1,FlocIdSortingLshBasedComputation,InterestCohortFeaturePolicy"

Once you're using those flags, you don't need an origin trial token; the flags take care of it all.

--Michael

--

Forewarned is worth an octopus in the bush.

[Yao Xiao‏]

Hi Federico,

I believe you want to test in a https page. The API additionally requires to be called in a "secure context", and http://localhost isn't considered to be one at this point.

[Federico Nieves‏]

Hey Yao/Michael,

Thanks it worked, the thing is that if you launch Chrome from the CLI with the parameters, you have to make sure Chrome is completely closed before. Otherwise the flags won't take effect.

For the community, this is a good URL to test if the browser has Flocs enabled: https://floc.glitch.me/

Best,

[Josh Karlin‏]

Note that the rollout has started. Currently, the interestCohort API will work for 50% of canary, dev, and beta channel users if you have an Origin Trial token. The plan is to ramp up stable to 0.5% if we don't come across any issues early next week. From there we're exploring options for ramping up to 5%. 

Again, once you have an Origin Trial token, only a subset of users will actually have the interestCohort() API enabled, so be sure to use feature detection. The reason that we limit the population is so that you don't have to worry about using the API on more pages than is permitted for Origin Trials (0.5% of page loads by default). 

Josh

[Josh Karlin‏]

I also wanted to mention a couple of other resources:

1) the web.dev article describing how one can go about using the API

2) the Privacy Sandbox page for FLoC which is where we will post information about the various versions of FLoC seen in the wild as well as FLoC sensitivity analysis and statistics.

Josh

[Gregory Truchetet‏]

Hi!

@Josh, you mentioned a ramp up 15 days ago, from what we see it didn't happen. The feature is still opened to chrome canary, dev, and beta only, am I right?

Do you know when we'll be able to have the feature activated on chrome stable as well? :)

Is there another place than here where we can follow the rollout and discuss it? Perhaps a dedicated group like the one for attribution-reporting-api? (https://groups.google.com/a/chromium.org/g/attribution-reporting-api-dev)

Thanks!

[Josh Karlin‏]

You're correct, there has been no change since my last message. We're still working on pushing out to the stable channel. I'll post any changes to the experiment's status to this thread.

Best,

Josh

[Francesco Calì‏]

Hi all,

I'm tring to test FLOC , so I added origin-trials token to a website, but calling the API I have the error "document.interestCohort is not a function" always.

Is it something related to the 0.5% of site users or about "Chrome detects that the page loads ads or ads-related resources"  ?

What kind of ads must be delivered in page ? It's enabled with ads delivery through GTM only ? 

Thanks,

Francesco.

[Sam Dutton‏]

Hi Francesco

• If you just want to try out document.interestCohort()for a single browser, you can run Chrome with flags: see floc.glitch.me.
• As you say, the origin trial only makes the API available for a small proportion of users. Also be aware that the initial origin trial is only taking place in certain regions.
• "Chrome detects that the page loads ads or ads-related resources" means that pages that use ads or ads-related resources (as well as pages that use document.interestCohort()) are included (during the current origin trial) in the browser's computation of its FLoC cohort. This does not affect whether document.interestCohort() is available.
• This issue discusses what 'ads or ads-related' means.
  

Sam

[Francesco Calì‏]

Hi Sam,

thank you for your reply.

We tested the interestCohort API starting browser with flags and all seems good.

Now we would like to test it adding  origin-trials token on meta page tags at runtime, as described on OriginTrials/developer-guide.md at gh-pages · GoogleChrome/OriginTrials · GitHub and with no flags.

In this scenario seems that almost 100% times we have the error  "document.interestCohort is not a function". I suppose it's related with 0,5% users capping. 

Could you give me more details about 0.5% calculation ? 

Is it calculated on the site traffic where token is added or on chrome users ? 

In the site traffic case, it calculates the percentage using as user base the amount of users that comes from countries where the experiment is enabled ?

Thanks in advance,

Francesco.

[Sam Dutton‏]

Hi Francesco

0,5% users capping.
Could you give me more details about 0.5% calculation ?
Is it calculated on the site traffic where token is added or on chrome users ? 

0.5% of all Chrome page loads. 

(Also, I'm sure you're aware, but the initial origin trial is only taking place in certain regions.)

[Sam Dutton‏]

The initial FLoC origin trial is taking place with a small percentage of users in these regions (currently Australia, Brazil, Canada, India, Indonesia, Japan, Mexico, New Zealand, Philippines and the U.S.)

On Thu, 20 May 2021 at 13:51, Дмитрий Шалашов <shal...@corp.mail.ru> wrote:

Hello everyone,

is it possible to provide a list of countries in which FLoC OT would be launched? Maybe even with timelines? I'm personally (and my employer, Mail.Ru) interested in a trial in Russia.

вторник, 18 мая 2021 г. в 12:06:50 UTC+2, Francesco Calì:

[Dmitry Shalashov‏]

Thanks, but I thought that this a current list of countries, with a rollout to more countries later. I was wrong? It is a final list?
 

Четверг, 20 мая 2021, 16:38 +03:00 от Sam Dutton <dut...@google.com>:

 

 

--
Dmitry Shalashov

 

[Дмитрий Шалашов‏]

Hello everyone,

is it possible to provide a list of countries in which FLoC OT would be launched? Maybe even with timelines? I'm personally (and my employer, Mail.Ru) interested in a trial in Russia.

вторник, 18 мая 2021 г. в 12:06:50 UTC+2, Francesco Calì:

Hi Sam,

[Sam Dutton‏]

Correct: this is the current list. As per the post linked to: 'We’ll expand to other regions as the trial expands globally'.

[Dmitry Shalashov‏]

Oh, that’s good. But then we get back to my initial request — is it possible to get a full list of countries where OT will be launched; even better, with info when it will happen.
 

Четверг, 20 мая 2021, 21:03 +03:00 от Sam Dutton <dut...@google.com>:

 

 

--
Dmitry Shalashov

 

[Francesco Calì‏]

Hi Sam,

I have another question.

I'm watching this report https://www.chromestatus.com/metrics/feature/timeline/popularity/3687 where I can see the user percentage, but no urls are visible. 

Using BigQuery I have no resultset too. What am I doing wrong ? 

There is another servece I can call to see url's list ?

Thanks in advance,

Francesco.

[Sam Dutton‏]

Hi Francesco

 I can see the user percentage, but no urls are visible. 

Others can correct me if I'm wrong, but I think this is because this feature is in origin trial. (You'll notice the same for other features in origin trial.)

[Rick Byers‏]

The sample URLs there come from the HTTP Archive crawl, which I believe only happens once a month. I'd expect this to include OT APIs (unless the crawler was somehow configured to disable OTs). But it's only the home pages of top sites and the APIs they use on load. We have an additional mechanism within Google to anonymously collect some public URLs from the field ("UKM") but there is no public dashboard or data set for that unfortunately. There are some more details here.

Rick

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAACxJ5pnZe5dGPnFmnjP7c9k-Rf9hmh%2Bw7fN8qQJY%2BpkZmpFOg%40mail.gmail.com.

[Gregory Truchetet‏]

Hi guys!

According to https://developer.chrome.com/origintrials/#/view_trial/213920982300098561 , this experiment is planned to end on July 14. Is there a plan to extend the experiment on Chrome 92 and beyond? Or must we acknowledge that the feature will (temporarily?) disappear in two weeks?

Thanks, 

[Josh Karlin‏]

Hi Gregory,

Thanks for asking. We’ve decided not to extend this initial Origin Trial. Instead, we’re hard at work on improving FLoC to incorporate the feedback we’ve heard from the community before advancing to further ecosystem testing. More information soon.

Cheers,

Josh

[Geoff Yaworski‏]

For historical sake.  Where did the ramp up max out at?  I don't see an update on this, so if there is an official place to look, please let me know.  By where this thread left out, sounds like it was at 0.5%.

[Josh Karlin‏]

Up to 0.5% max, and it was enabled for canary/dev/beta channels, but not stable.

[Geoff Yaworski‏]

Hey Josh,

Thanks for responding.   Why was the OT stopped before stable?  How many people are in each of the channels (canary/dev/beta/stable)?

Cheers,

Geoff

[Josh Karlin‏]

To close the loop on this, we've incorporated the feedback from the first experiment and are now iterating on the Topics API to address the same use case as FLoC.