P75 - what does it means in the field? How to interpret the data?

[Holmes_L]

Hi there,

The explanation of P75 is clear: This value corresponds to the origin's 75th percentile of user experiences. In other words, 75% of experiences are better than this value. 

However, when I create the dashboard as below, I got a little confused.

• In the right-hand chart: the blue line (P75) is telling that: 75% of experiences are approaching "Poor".
• In the left-hand chart, in all the devices, it's telling a different story: generally looks like the % of users is much less than 75% to "Poor". 
• Similar result can be seen on the right-hand chart - bars.

In a word, the horizontal bar chart on the left side and the vertical bars on the right side are seemed telling similar story, the blue line (P75) is telling another.

May I have your advice on this: In the field, what does P75 is actually telling? How to interpret the data? Or I wrongly read the charts?

Thanks.

H

[Barry Pollard]

The p75 CLS number is the value of CLS that 75% of your users achieve. Let's say you had 4 page views and they had CLS scores of 0, 0, 0.5, 0.8. What number is the lowest number that covers 75% of page views? In this example it's 0.5. That's what the p75 represents - the number under which 75% of page views would be included. It does not mean that 75% of page views get that number exactly (two of your 75% sample get a much better value that than of 0 in this example), or even get an average of that number (the mean average of 0, 0, and 0.5 would be 0.16667 but that's not the p75) - it just means 75% of page views got that number OR better. In this case 2 of them got much better than that, and 1 got exactly that.

p75 basically removes the worst 25% of page views (basically it assumes they are outliers that are not really representative of what your page experience is like for "most users"). It then looks at the worst value of those remaining. And that is the value that represents what most of your page views achieved - at least! Many will get better than that, but you can guarantee that 75% of your page views got at least as good as an experience as that.

The p75 number is useful to give the worst experience that the majority (75%) of your page views will experience. But it does not say if that number is representative of ALL those 75% of page views, or just the very last one. It could be 74% of them are fantastic with 0 CLS, and only 1% get that number - but to reach 75% of page views you need that high number from those 1% so that's your p75 score.

Here's another way of looking at it:

If 75% of your page views achieved a CLS of 0.24 (i.e. just under the 0.25 "poor" limit) or better then that means 75% of your page views are in the amber "Needs Improvement" category.

Similarly, If 75% of your page views achieved a CLS of 0.1000001 (i.e. well under the 0.25 "poor" limit, but above the 0.1 "good" limit) or better then that means 75% of your page views are in the amber "Needs Improvement" category.

So, while the bars might be the same in both these cases (you're in "Needs improvement"), the p75 number, allows you to see how close you are to the limit rather than just the percentage of page views that made that limit.

The p75 shows you how close you are for your worst score(s). The bars on the other hand, shows how close you are in terms of number page views.

So let's look at your statements:

• In the right-hand chart: the blue line (P75) is telling that: 75% of experiences are approaching "Poor".

Kind of. Like the 0.24 example above you are very close to the "Poor" limit. You are in Needs Improvement for now, but if you keep trending the way you are trending then you will flip into "Poor". What you can't tell is how many of those 75% of page views are close to that poor limit. Is it all of them? Or only a few outliers? p75 doesn't tell you that (you'd need to look at other percentiles p0 - p74 to see that), but just that you are very close to the edge.

So this does not mean all 75% of experiences are nearly poor. It means to get to 75% of experiences we need to look at ALL the "Good" experiences, and nearly all the "Needs Improvement" experiences, and pretty soon we'll need to start looking at some "Poor" experiences too to make up 75% of your users. Hence why it's approaching "Poor".

• In the left-hand chart, in all the devices, it's telling a different story: generally looks like the % of users is much less than 75% to "Poor". 

It's not a different story. As per above, neither views is telling you that 75% of your page views are poor. But to find a CLS figure that includes 75% of your page views you need all the Good page views (0.1 or under), and nearly all the Needs Improvement page views (between 0.1 and 0.25), and maybe soon, some poor page views.

> In a word, the horizontal bar chart on the left side and the vertical bars on the right side are seemed telling similar story, the blue line (P75) is telling another.

That's incorrect. All three are showing that, to get to 75% of experiences, you need to include all the Good experiences, and most of the Needs Improvement experiences - so overall the origin is in "Needs Improvement" category (though it does look like Phone is the majority of your users, and pulling up the Desktop and Tablet numbers as they would be in "Poor" when looking at just those, but when looking at All Devices in the right hand side you're in "Needs Improvement" so clearer your Phone experiences are having a bigger effect of the All devices view so must be a bigger proportion of your traffic - which is common these days).

Thanks,

Barry

[Holmes_L]

Very clear clarification to the definition of P75 with detail explanations.

Cannot find a word other than HERO for you, thanks Barry!