Data Completeness / Reporting Rate for EHR measures

[Jacob Makowski]

What is the exact calculation for Data Completeness / Reporting rate for Quality Measures submitted via EHR/QRDA-III?

 

On each of the Registry/Claims measures specifications, you can see a Data Completeness / Reporting Rate calculation outlined, for example:

 

This matches the documentation at https://cmsgov.github.io/qpp-submissions-docs/measurements for reporting rate:

 

No such calculation is given for EHR measures. Also, as “Performance Not Met” is not quantified in an EHR measure submitted via QRDA iii, the above calculation cannot be used for EHR data completeness.

 

The HL7 implementation guide for QRDA cat III (Release 2.1) indicates this for Reporting Rate:

 

5.14 Reporting Rate for Proportion Measure

[observation: identifier urn:oid:2.16.840.1.113883.10.20.27.3.15 (open)]

Table 54: Reporting Rate for Proportion Measure Contexts Contained By: Contains:

Measure Reference and Results (V3) (optional)

This template is only used with proportion measures.This reporting rate represents the percentage of patients in the denominator who fall into one of the other sub-populations. The Reporting Rate is calculated using this formula: Reporting Rate = (Numerator + Numerator Exclusions + Denominator Exclusions + Denominator Exceptions)/(Denominator). The predicted rate (based on the measure's risk-adjustment model) can be captured in the reference range.

 

However, if the above calculation is used in concert with the 60% threshold, this will punish clinicians submitting inverse measure and measures with low benchmarks. For example, Measure #1 submitted via EHR with Numerator = 1, Denominator = 110, Denominator Exclusions = 10 will have a performance rate of 1 % [1 /( 110-10)] and a reporting rate / data completeness of 10% [(1 +10) / 110]. This measure should earn 10 decile points, but will fall below the 60% data completeness threshold and earn 1 point (or 3 for small practices) if the above calculation is used. Similarly, with a non-inverse measure, if Measure #317 was submitted via EHR with Numerator = 50, Denominator = 102, Exclusions = 1, Exceptions = 1   [Performance rate = 50%, Reporting rate = 50.98%], it would fall below the data completeness threshold although it should earn 10 points.

 

 

 

Thanks

[Ivana Ng]

Hi Jacob - 

I have forwarded your inquiry to the product team for further discussion. We will get back to you soon.

This may be related to this Google Group thread - https://groups.google.com/d/msg/qpp-apis/Cxig9eSCuwQ/tcr41_diBgAJ. We are working on updating our documentation at https://cmsgov.github.io/qpp-submissions-docs/ to better communicate how the fields that the API accepts map to eCQM fields.

Thanks,

Ivana

Ivana Ng

Product Manager

Nava PBC

--
You received this message because you are subscribed to the Google Groups "Developer Group for QPP APIs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qpp-apis+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/qpp-apis.
To view this discussion on the web visit https://groups.google.com/d/msgid/qpp-apis/d8d106cd-e604-42dc-87d6-35005d8d8848%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ivana Ng]

Jacob -

For eCQMs, the expectation is that reporting rate will be 100%. It is important to refer back to the measure specification on how the population and other fields are populated. If reporting is expected to be 100%, the requirement to submit performanceNotMet is no longer applicable, as if performanceMet is not met, the assumption is that this scenario would result in a performanceNotMet. 

Below is a calculation showing the performance rate creation for an eCQM, where numerator is performanceMet and denominator is all patients then subtracted out any exclusions or exceptions.  This results in performanceMet as the numerator and all eligible patients for the measure as the denominator.

Ivana Ng

Product Manager

Nava PBC

[Jacob Makowski]

Thanks, Ivana

As a follow-up, how does the scoring api determine data completeness and the minimum cases (20) for a multi-strata/performance rate measure? If a measure has 3 strata/performance rates, and the denominator on each of them is 19, would you sum them to determine the case minimum, do they all need to be 20 or greater? Would the approach change depending on the measure? I'm assuming that for Simple Average measures, all denominators would need to be 20+, for weighted average, the sum would need to be 20+, and I'm not sure how measures where only 1 rate is compared to the benchmark would work.

I have essentially the same question for data completeness. Do all strata need to meet the threshold, is it an average?

Thanks

- Jacob

To unsubscribe from this group and stop receiving emails from it, send an email to qpp-apis+u...@googlegroups.com.

[steven....@semanticbits.com]

Hey Jacob,

For multi-strata measures, it is either a rate selection based on the strata, a weighted average, or simple average. Please refer to here for additional information: https://cmsgov.github.io/qpp-submissions-docs/measurements#multi-performance-rate-measurements. Rate selection will use the specific strata for reporting and performance and the algorithms for the other two types are listed within the documentation. Please let me know if you need any additional information. 

[Jacob Makowski]

Thanks, Steve!

I'm aware of the info at - https://cmsgov.github.io/qpp-submissions-docs/measurements#multi-performance-rate-measurements  - which is sort of where my question comes from. The stated calculation for reporting rate for multi-strata measures is the same as single-strata measures. But this isn't possible. 

Here's an example that could help clarify. If a measure has 3 strata (none are overall) and the overall algorithm is weighted average, what is the reporting rate for the below:

stratum 1: met - 90   not met - 10  denom - 100  exclu - 0  excep - 0

stratum 2  met - 30   not met - 10  denom - 100  exclu - 0  excep - 0

stratum 3  met - 5     not met - 0    denom - 15    exclu - 0  excep - 0

The reporting rates for each strata are 1 = 100%, 2 = 40%, and 3 = 33%. If we were using the 2018 threshold of 60%, stratum 1 is above the threshold and strata 2 and 3 are below. 

If the API is requiring that all of the strata meet the reporting rate in order to score more than 3 points, this measure would score 3 points. 

If the API is taking a simple average of the reporting rates, the calculation would = 57.77% and would get 3 points (below 2018 60% threshold)

if the API is taking a weighted average of the reporting rates, the calculation would = 67.44% and would be above threshold

Now let's say that this measure had an overall algorithm for performance rate of overallStratumOnly and the overall stratum was stratum 1. Would it matter in this case that the overall stratum was the only one meeting the reporting rate/data completeness requirement?

The additional question is essentially the same, but applies to the 20 case minimum for the measure denominators. Sometimes a multi-strata measure has the same denominator for all strata (usually the simpleAverage measures) and I'm assuming that in this case we wouldn't be summing the denominators to determine minimum case criteria. Other measures have completely different patient populations/denominators in each strata, and it might make sense to sum the denominators of all strata to determine minimum cases. For measures with an overall stratum, it would make sense to just check the denominator of the overall stratum for at least 20 cases. These are all assumptions on my part, however, since I can't find that documentation as it pertains to reporting rate or the minimum of 20 cases - I can only see that level of detail in the performance rate documentation. 

If I'm missing something, please point me in the right direction. Thanks!

- Jacob 

[Sarah White]

Hi Jacob,

The developer documentation is correct. For reportingRate the formula is sum(reportingNumerators) / sum(eligiblePopulation) where reportingNumerator = performanceMet + eligiblePopulationExclusion + eligiblePopulationException + performanceNotMet. Please note that this is only for the relevant strata and it won't include the values for the overallStratum if it's not supposed to. In your example above, the outcome would be 67.44%.

Hope this helps!

Sarah

[Jacob Makowski]

Thanks, Sarah!

Can you point me to where it says that exactly? I'm having trouble finding the formula you referenced. Thanks!

[Sarah White]

Hi Jacob, 

It's not written the way I wrote it "sum(reportingNumerators) / sum(eligiblePopulation)", but it's in the link you provided - https://cmsgov.github.io/qpp-submissions-docs/measurements#multi-performance-rate-measurements. Next to reportingRate, it states the expanded formula I stated above:

The reporting rate, ranging from zero to one-hundred and representing a percentage, is equal to ((performanceMet + eligiblePopulationExclusion + eligiblePopulationException + performanceNotMet) / eligiblePopulation) * 100.

Sarah White

Business Analyst

Nava

To unsubscribe from this group and stop receiving emails from it, send an email to qpp-apis+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/qpp-apis.

To view this discussion on the web visit https://groups.google.com/d/msgid/qpp-apis/76bab5af-d7bd-41ce-82d0-b155575ba50a%40googlegroups.com.