Per-Glean app ping views now combine data across all channels

[Anna Scholtz]

Starting 2023-02-01 per-Glean app ping views now combine data across all channels

tl;dr
Per-Glean app ping views now UNION ping data across channels (release, beta, nightly). Previously these views only queried data from release channels. BigQuery datasets that contain these views are, for example, fenix, firefox_ios or focus_android. Examples of such views are mozdata.fenix.metrics, mozdata.focus_android.events and many more.

What does this mean?Some of our Glean applications, for example Fenix, have different channels (release, beta, nightly). Telemetry ping data sent from each of these channels is stored in separate datasets in BigQuery. For example, in the case of Fenix, ping data sent from release is stored in the org_mozilla_firefox dataset while data sent from beta is stored in org_mozilla_firefox_beta in BigQuery. These datasets are also called per-app_id datasets, since each channel has it's own distinct app ID.In the past, BigQuery datasets have been created for each Glean application that contain tables and views that combine data across channels (these datasets are referred to as per-Glean app datasets). For Fenix, these tables are stored in the BigQuery dataset fenix. These datasets also contain views for each ping. Previously, these views only queried ping data from release channels. This new change will query data across all channels and UNION it.Why is this great?This greatly simplifies querying and comparing data across channels. It is no longer necessary to manually UNION data across different datasets and find the right datasets in the first place. It's all automatically done in a single view.Will this break anything?This shouldn't have an impact on Looker dashboards derived from the existing ping explores for Glean apps. At the moment these explores are still based on the per-app_id datasets. Once we update these explores to use the per-Glean app views the behaviour and data will remain unchanged. All of these explores have a required filter on channel already (with the default being set to release).Any custom queries (e.g. on Redash) that query per-Glean app ping views without a filter on normalized_channel will now also return beta and nightly data. Add a filter normalized_channel = "release" to keep getting release-only data.Since most of our Glean applications don't actually have multiple channels, this will only impact data sent from Fenix, Firefox iOS and Focus Android.

- Data Platform Working Group