Intent to Ship: Attribution Reporting Feature Bundle: Additional Verbose Debug Reports, Further Gating Source Verbose Debug Reports, Splitting the Attribution Rate Limit

[Akash Nadan]

Contact emails

akash...@google.com, lin...@chromium.org, john...@chromium.org

Explainer

Attribution Reporting with event-level reports

Attribution Reporting API with Aggregatable Reports

Aggregation Service for the Attribution Reporting API

Specification

https://wicg.github.io/attribution-reporting-api/

Blink component

Internals > AttributionReporting

Summary

We are landing the following changes to the Attribution Reporting API focused on:

• additional debugging support by supporting new verbose debug reports

• improving privacy & security by adding additional gating to source verbose debug reports

• improving rate limit accounting by separating the attribution count for the two ARA report types

Explainer/Spec changes

1. Spec verbose debug report for max channel capacity and max trigger states cardinality

2. Report source reporting origins per site limit explicitly

3. Gate all source verbose debug reports behind cookie access

4. Split attribution rate-limit into separate event & aggregate rate-limits

Risks
Interoperability and Compatibility

(1, 2) Additional verbose debug reports and (4) splitting the attribution rate limit into separate limits for event and aggregate are fully backwards compatible changes. Feature (3) gating all source verbose debug reports is a backwards incompatible change because now source-destination-limit and source-destination-rate-limit verbose debug reports now require the ar_debug cookie to be set at source registration time. This is not a major concern because all other current source verbose debug signals already require the ar_debug cookie to be set and most ad-techs would already be setting this cookie at source registration time.

              

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

The attribution reporting feature bundle will be supported on all platforms with the exception of  Android WebView

Is this feature fully tested by web-platform-tests?

Yes

Estimated milestones

This feature bundle is anticipated to ship as part of Chrome 125. 

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5146883686400000

Links to previous Intent discussions

Previous I2S: 

Intent to Ship: Attribution Reporting API

Intent to Ship: Attribution Reporting features M117

Intent to Ship: Attribution Reporting features M118

Intent to Ship: Attribution Reporting features M119

Intent to Ship: Attribution Reporting features M120

Intent to Ship: Attribution Reporting features M121

Intent to Ship: Attribution Reporting features M123

Intent to Ship: Attribution Reporting features M124

Thanks,

Akash

[Philip Jägenstedt]

Hi Akash,

Are https://wpt.fyi/results/attribution-reporting all of the tests for this feature and these changes? Do you expect that all of the tests there will be passing?

I see that 3 of the tests time out which make it harder to understand the status. Is it possible to make these tests fail fast with more details about why they fail?

Best regards,

Philip

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/194f8cc0-03b3-47e5-ad1c-0938c1a686ben%40chromium.org.

[Nan Lin]

Hi Philip,

These tests are for Attribution Reporting API, but not explicitly related to the features in this intent. We do expect that all the tests there will be passing.

Those 3 tests involve writing to and reading from the stash, which probably causes the timeout flakiness.

As currently the rate limits are not configurable in the web platform tests yet, we don't add web platform tests for the new features which depend on the rate limits.

Thanks,

Nan

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYdvBLuqLjqL5iv6s8GbpKhV1rr5XujoBd6d7F%3Dc4a2qhw%40mail.gmail.com.

[Mike Taylor]

On 4/24/24 9:40 AM, Nan Lin wrote:

> These tests are for Attribution Reporting API, but not
> explicitly related to the features in this intent. We do expect that
> all the tests there will be passing.
>
> Those 3 tests involve writing to and reading from the stash, which
> probably causes the timeout flakiness.
>
> As currently the rate limits are not configurable in the web platform
> tests yet, we don't add web platform tests for the new features which
> depend on the rate limits.

Has your team considered adding this functionality to WPT? I don't see a
strong argument for blocking shipping given this, but ultimately we do
want to make interop testing possible, if another engine decides to
implement ARA.

[Mike Taylor]

Apologies for the delay here - it's a bit challenging to review 4 features at once. 

(Aside: it seems like this particular intent could have been split into 2... one for 2 debug report features, and another for rate limits?)

On 4/19/24 12:51 PM, 'Akash Nadan' via blink-dev wrote:

Contact emails

akash...@google.com, lin...@chromium.org, john...@chromium.org

Explainer

Attribution Reporting with event-level reports

Attribution Reporting API with Aggregatable Reports

Aggregation Service for the Attribution Reporting API

Specification

https://wicg.github.io/attribution-reporting-api/

Blink component

Internals > AttributionReporting

Summary

We are landing the following changes to the Attribution Reporting API focused on:

• additional debugging support by supporting new verbose debug reports

• improving privacy & security by adding additional gating to source verbose debug reports

• improving rate limit accounting by separating the attribution count for the two ARA report types

Explainer/Spec changes

1. Spec verbose debug report for max channel capacity and max trigger states cardinality

2. Report source reporting origins per site limit explicitly

3. Gate all source verbose debug reports behind cookie access

4. Split attribution rate-limit into separate event & aggregate rate-limits

These are also challenging to review, as each PR doesn't have a corresponding issue, or given a _why_, just the what (or maybe I'm missing something). The diffs for the explainers also aren't super enlightening. Could you write a few sentences on the motivations for each of these changes?

Risks
Interoperability and Compatibility

(1, 2) Additional verbose debug reports and (4) splitting the attribution rate limit into separate limits for event and aggregate are fully backwards compatible changes. Feature (3) gating all source verbose debug reports is a backwards incompatible change because now source-destination-limit and source-destination-rate-limit verbose debug reports now require the ar_debug cookie to be set at source registration time. This is not a major concern because all other current source verbose debug signals already require the ar_debug cookie to be set and most ad-techs would already be setting this cookie at source registration time.

Is it reasonable to assume that there aren't sites only registering for source-destination-limit and source-destination-rate-limit reports? Do we have usecounters or UKM to verify?

              

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

The attribution reporting feature bundle will be supported on all platforms with the exception of  Android WebView

Is this feature fully tested by web-platform-tests?

Yes

Estimated milestones

This feature bundle is anticipated to ship as part of Chrome 125. 

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5146883686400000

Links to previous Intent discussions

Previous I2S: 

Intent to Ship: Attribution Reporting API

Intent to Ship: Attribution Reporting features M117

Intent to Ship: Attribution Reporting features M118

Intent to Ship: Attribution Reporting features M119

Intent to Ship: Attribution Reporting features M120

Intent to Ship: Attribution Reporting features M121

Intent to Ship: Attribution Reporting features M123

Intent to Ship: Attribution Reporting features M124

Thanks,

Akash

[Akash Nadan]

Hi Mike,

We have not fully considered adding this functionality to WPT and it may be challenging due to delays and noise added by the Attribution Reporting API, but we will evaluate what is possible here. 

Thanks for the suggestions regarding the features. We will make sure to break apart the I2S based on features that can be grouped together in the future.

Regarding the motivation for the features:

1. Spec verbose debug report for max channel capacity and max trigger states cardinality

For debugging completeness of potential errors that can occur. This is an important debug error signal for the flexible event level reporting feature.

2. Report source reporting origins per site limit explicitly
For debugging completeness of potential errors.

3. Gate all source verbose debug reports behind cookie access
For reducing complexity across all source verbose debug reports by applying the same requirement across all of them.

4. Split attribution rate-limit into separate event & aggregate rate-limits
For the ability to properly account for report deletions and support the flexible event level reporting feature (which may produce more than 1 event report per trigger registration, whereas aggregate reports would not).

Regarding the interoperability and compatibility question, it is currently not possible to register for a specific error signal. The API caller would register to receive a debug report and the API then provides the error that occurs (assuming an error occurs). Additionally, it would be extremely unlikely for sites to only be interested in the source-destination-limit and source-destination-rate-limit errors, since they would most likely be interested in also understanding any error signals that may be impacting their conversions as well. 

Thanks,

Akash

[Mike Taylor]

Thanks Akash.

This is not quite the level of detail I was hoping for (I've more or less grokked that from the commits themselves), but I'm satisfied with the compat implications of always requiring the ar_debug cookie.

LGTM1

On 4/26/24 5:10 PM, Akash Nadan wrote:

Hi Mike,

We have not fully considered adding this functionality to WPT and it may be challenging due to delays and noise added by the Attribution Reporting API, but we will evaluate what is possible here.

Thanks - even if challenging, this is super important for long-term interop. So whatever we can do to improve the status quo will be worthwhile, IMHO.

[Alex Russell]

Hey folks,

We've talked about this in API OWNERS again, and the presentation of this set of features is...frustrating. 

Several of these features lack any explanation, example code, or any outline of alternative approaches that were considered and discarded. Having multiple features presented in a low-quality way does not make it easier to evaluate them, particularly when the relationship between them is unclear and no example code has been produced to demonstrate how they work together to solve an important problem, let alone why it solves those problem(s) well.

I've flipped the bit in chromestatus to "Needs Work" and will hold shipping these features until such time as a full explanation is produced. A good way to do that would be to produce an Explainer document (in the usual format) for each change, highlighting considered alternatives and foregrounding end-to-end example code for both the selected solution and the alternatives.

Looking forward to working with y'all to get this unblocked.

Best,

Alex

[Akash Nadan]

Hi Alex,

Thanks for the feedback and sorry for the initial lack of detail! We hope the following explanations (below) make each of these proposed changes more clear.

We will keep the feedback in mind and make sure to include the level of detail similar to the Explainer format going forward.

1. Spec verbose debug report for max channel capacity and max trigger states cardinality

Currently the API supports a feature called Flexible Event-Level reporting, which gives API callers additional customization capabilities by allowing them to change the number of conversion reports, number and length of reporting windows, and trigger data cardinality per source that they register.

As part of Flexible Event-Level, the API only allows configurations that do not exceed our current privacy bar (i.e. max channel capacity) of 11.5 bits of information gain for clicks and 6.5 bits of information gain for views. So currently if an ad-tech using Flexible Event-Level chooses a configuration that exceeds the current privacy bar their registration will fail silently. With the addition of the “max channel capacity limit” verbose signal, ad-techs will now be able to receive a verbose debug report telling them that they have exceeded the “max channel capacity” and will allow them to know that they need to use a different configuration and allow them to identify why their source registrations may be failing. This new verbose debug report will also help with future features that may be taken into account when calculating the information gain for a given configuration.

Similarly, as part of ARA source registrations and Flexible Event-Level, an ad-tech can customize their trigger data cardinality up to a maximum of 32 trigger states. The “max trigger states cardinality limit” verbose signal, will allow ad-techs to know if they have set a source registration that exceeds this maximum threshold on the trigger data cardinality, making the API easier to debug for ad-techs.

2. Report source reporting origins per site limit explicitly

Currently the API has a rate limit that only allows a maximum of 1 reporting origin per {source site, reporting site, 1 day} per source registration. This limit is in place to help prevent ad-techs from cycling through multiple different reporting origins as a way to measure additional information on a user.

Currently if an ad-tech exceeds this limit, their source registration will fail silently. In order to make the debugging process easier and help ad-techs identify a potential cause of data loss, the API will now provide a verbose signal for this error any time an ad-tech’s registration exceeds the limit.

3. Gate all source verbose debug reports behind cookie access

Currently the API provides 9 different source verbose debug signals. All of these signals require that the ad-tech has set the ar_debug cookie on the source in order to receive the verbose debug signal, except for two errors: “source-destination-limit” and “source-destination-rate-limit”. 

In order to reduce complexity for ad-techs so that there is no difference in when an ar_debug cookie needs to be set in order to receive a debug report across the different error signals, this change will now require the ar_debug cookie to be set for all source verbose debug signals.

This change also has the added benefit of improving privacy by adding an additional check for the two rate limits listed above.

4. Split attribution rate-limit into separate event & aggregate rate-limits

Currently the API has a single attribution rate limit of 100 attributions per {source site, destination site, reporting} per 30 days, and this limit is across both report types supported in the API (i.e. event-level reports and aggregate reports). For example, currently if 1 trigger registration generates both an event-level report and aggregate report this will count as 1 towards the attribution limit. And if the attribution limit is hit, neither event-level report nor aggregatable report is created.

Now that the API supports a subset of Flexible event-level, there may be scenarios where 1 trigger registration can generate multiple event-level reports, but still only 1 aggregate report. Because of this new behavior it would not be accurate to count and may negatively impact ad-techs if the API tracks the rate limit across both report types.

With this change, the API will now track the attribution rate limit separately for each report type. So in the example above, where a trigger registration generates multiple event-level attribution reports, these will only count for the event-level report limit, and not impact the aggregate attribution rate limit, which will allow ad-techs to continue generating aggregate reports.

Additionally, this separation of rate limits will also provide better report deletion accounting in the API in scenarios where a pending event-level report is scheduled to be sent at a later time, but then gets replaced by a higher priority newly generated event-level report. In the current API this scenario counts as 2 attribution reports, when in reality the first report is never sent since it is replaced by the higher priority second report. With this change, the API will only count this as 1 report towards the attribution rate limit, and specifically the event-level report count, and not the aggregate report count.

Thanks,

Akash

[Chris Harrelson]

Thanks for these mini-explainers, they clarified what is changing for me!

LGTM2

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/2fb1a68d-bdc7-4428-ab3f-858b7cabd91en%40chromium.org.

[Rick Byers]

LGTM3