[ACTION NEEDED] (Firefox Desktop) Activity Stream datasets are DEPRECATED in favour of New Tab Telemetry
15 views
Skip to first unread message
Chris H-C
Dec 13, 2023, 1:03:20 PM12/13/23
to fx-da...@mozilla.org, data-org, activit...@mozilla.com
Hello,
As of time of writing, Activity Stream datasets (those in the `activity_stream.*` namespace) are now deprecated and should not be used to inform decisions.
WHO
If your work depends on decisions informed by data measured on Firefox Desktop's New Tab pages (about:newtab, about:home, etc.): this is for you. If you do not use `activity_stream.*` datasets, or have already moved your analyses to New Tab Telemetry, or don't deal with Firefox Desktop data, you can stop reading now. Thank you for your attention.
WHY
As discovered earlier this year, the system Activity Stream uses to report data (PingCentre) is subject to data loss. Between 4-5% of PingCentre-sent Activity Stream data never made it into those datasets. Thus, Mozprod Shared Infrastructure OKR 3.7 was created to replace PingCentre with a system not subject to this data loss.
HOW
The New Tab Telemetry (NTT) Project, unaware of this data loss, began reinstrumenting a lot of what Activity Stream was sending beginning with Firefox 103 in 2022. (It was actually in validating New Tab Telemetry that we found PingCentre's data loss in the first place.) As NTT was supporting many analysis use-cases already, the PingCentre Replacement Project decided to augment it with all remaining Activity Stream data to support all remaining analysis use cases. This gap-bridging instrumentation work completed in Firefox Desktop 122.
WHEN
Activity Stream datasets are deprecated as of right now. PingCentre-sent Activity Stream instrumentation will be removed from Firefox Desktop in either Firefox 122 or 123 (you know how code is. Follow bug 1868988 for specifics.). This means that, soon, data in `activity_stream.*` datasets will stop flowing from Nightly clients who have updated. Then Beta. Then Release. ESR, with its vast population of old Windows and old Mac users, will likely continue sending data for some time to come.
WE ARE NOT YET PLANNING TO REMOVE THESE DATASETS. We will want to, eventually. Maybe in late 2024, or 2025, or whenever the population on ESR115 drops enough. You can expect at least one more email before that happens.
HELP
Through the New Tab Telemetry Project we believe we've reached out to most consumers of Activity Stream data in Firefox, Data, and Pocket for both internal and partner-serving pipelines. But "most" is not "all".
If your decisions are informed by Activity Stream data and have questions or would like assistance moving to New Tab Telemetry, please come to the #new-tab-telemetry Slack channel. We'll be happy to help.
Thank you,
Rob Miller and :chutten
on behalf of the New Tab Telemetry and PingCentre Replacement Projects
As of time of writing, Activity Stream datasets (those in the `activity_stream.*` namespace) are now deprecated and should not be used to inform decisions.
WHO
If your work depends on decisions informed by data measured on Firefox Desktop's New Tab pages (about:newtab, about:home, etc.): this is for you. If you do not use `activity_stream.*` datasets, or have already moved your analyses to New Tab Telemetry, or don't deal with Firefox Desktop data, you can stop reading now. Thank you for your attention.
WHY
As discovered earlier this year, the system Activity Stream uses to report data (PingCentre) is subject to data loss. Between 4-5% of PingCentre-sent Activity Stream data never made it into those datasets. Thus, Mozprod Shared Infrastructure OKR 3.7 was created to replace PingCentre with a system not subject to this data loss.
HOW
The New Tab Telemetry (NTT) Project, unaware of this data loss, began reinstrumenting a lot of what Activity Stream was sending beginning with Firefox 103 in 2022. (It was actually in validating New Tab Telemetry that we found PingCentre's data loss in the first place.) As NTT was supporting many analysis use-cases already, the PingCentre Replacement Project decided to augment it with all remaining Activity Stream data to support all remaining analysis use cases. This gap-bridging instrumentation work completed in Firefox Desktop 122.
WHEN
Activity Stream datasets are deprecated as of right now. PingCentre-sent Activity Stream instrumentation will be removed from Firefox Desktop in either Firefox 122 or 123 (you know how code is. Follow bug 1868988 for specifics.). This means that, soon, data in `activity_stream.*` datasets will stop flowing from Nightly clients who have updated. Then Beta. Then Release. ESR, with its vast population of old Windows and old Mac users, will likely continue sending data for some time to come.
WE ARE NOT YET PLANNING TO REMOVE THESE DATASETS. We will want to, eventually. Maybe in late 2024, or 2025, or whenever the population on ESR115 drops enough. You can expect at least one more email before that happens.
HELP
Through the New Tab Telemetry Project we believe we've reached out to most consumers of Activity Stream data in Firefox, Data, and Pocket for both internal and partner-serving pipelines. But "most" is not "all".
If your decisions are informed by Activity Stream data and have questions or would like assistance moving to New Tab Telemetry, please come to the #new-tab-telemetry Slack channel. We'll be happy to help.
Thank you,
Rob Miller and :chutten
on behalf of the New Tab Telemetry and PingCentre Replacement Projects
Reply all
Reply to author
Forward
0 new messages