2018 Sandbox Scoring for Bonus Points

126 views
Skip to first unread message

Randal Roe (Alpha II Registry)

unread,
Jan 28, 2019, 12:24:09 PM1/28/19
to Developer Group for QPP APIs
The attached text file is the response we received after a recent sandbox submission.

Quality measure 217, which is an outcome measure, in the attached response was given no bonus points — as expected, since the requirement is at that the EC or group submit at least one outcome measure (or high-priority measure if no outcome measure is available). The second quality outcome measure, 218, was given 2 bonus points — again, as expected. And then high-priority measure 46 was given the expected 1 bonus point:

                  {
                    "name": "046",
                    "value": 4,
                    "detail": "Contributing 4",
                    "instrumentations": [
                      {
                        "reasonCode": "QUALITY-009",
                        "message": "Measurement meets the minimum reporting rate for this attestation year."
                      },
                      {
                        "reasonCode": "QUALITY-051",
                        "message": "Measure is a class 1 measure, Eligible population is >= 20 and reporting rate > 60 and has benchmarks"
                      },
                      {
                        "reasonCode": "QUALITY-076",
                        "message": "Applied minimum decile score."
                      },
                      {
                        "reasonCode": "QUALITY-091",
                        "message": "Class 1 measures receive a higher weight than the rest."
                      },
                      {
                        "reasonCode": "QUALITY-093",
                        "message": "Default end to end bonus score of 0."
                      },
                      {
                        "reasonCode": "QUALITY-105",
                        "message": "Measure eligible for high priority bonus with minimum performance rate and eligible populations reached."
                      },
                      {
                        "reasonCode": "QUALITY-110",
                        "message": "Allotted high priority bonus point."
                      },
                      {
                        "reasonCode": "QUALITY-111",
                        "message": "Measure contributed points to total score."
                      },
                      {
                        "reasonCode": "QUALITY-113",
                        "message": "Measure contributed bonus points to total score."
                      },
                      {
                        "reasonCode": "QUALITY-117",
                        "message": "Measurement points for PICKED measure include decile score with all bonus points."
                      }
                    ],
                    "metadata": {
                      "performanceRate": 83.33,
                      "reportingRate": 96.83,
                      "measurementSetId": "03d0cbf4-27d4-43db-8f57-65f6f734db5f",
                      "measurementId": "03d0cbf4-3894-46a8-9984-29efde01ea5b",
                      "measureClass": "Class 1",
                      "measureTitle": "Medication Reconciliation Post-Discharge",
                      "highPriorityBonusEligible": true,
                      "endToEndBonusEligible": false,
                      "endToEndBonus": 0,
                      "outcomeOrPatientExperienceBonus": 0,
                      "highPriorityBonus": 1,
                      "decileScore": 3,
                      "performanceDenominator": 60,
                      "performanceNumerator": 50,
                      "eligiblePopulation": 63,
                      "partialDecileScore": 0,
                      "partialPoints": 0,
                      "decile": -1,
                      "deciles": [
                        0,
                        96.15,
                        99.44,
                        100,
                        100,
                        100,
                        100,
                        100,
                        100
                      ],
                      "processingStatus": "PICKED",
                      "totalMeasurementPoints": 4,
                      "totalBonusPoints": 1,
                      "messages": {
                        "benchmarkType": "Did not override registry benchmark as eHR benchmark was not found",
                        "measurementClass": "Eligible population is greater than or equal to 20, reporting rate is greater than 60% and has benchmarks",
                        "decileScore": "Applied minimum decile score",
                        "e2eBonusScore": "Default E2E bonus score",
                        "highPriorityBonus": "Allotting 1 point to high priority bonus for measure type 'process'",
                        "measurementPicker": "Picked at 3",
                        "totalMeasurementPoints": "Measurement points for PICKED measure include decile score with all bonus points"
                      },
                      "noBenchmarks": false,
                      "benchmarkType": "registry",
                      "eMeasureId": null,
                      "toppedOut": true,
                      "isGhost": false,
                      "unroundedScoreValue": 4
                    }
                  },

However, measure 66, which is also a high-priority measure, did not receive the expected 1 bonus point:

                  {
                    "name": "066",
                    "value": 3,
                    "detail": "Contributing 3",
                    "instrumentations": [
                      {
                        "reasonCode": "QUALITY-009",
                        "message": "Measurement meets the minimum reporting rate for this attestation year."
                      },
                      {
                        "reasonCode": "QUALITY-052",
                        "message": "Measure is a class 2 measure because eligible population is >= 20 and reporting rate > 60 but has no benchmarks"
                      },
                      {
                        "reasonCode": "QUALITY-086",
                        "message": "For year 2018, apply maximum points for Class 2 measure meeting data completeness."
                      },
                      {
                        "reasonCode": "QUALITY-093",
                        "message": "Default end to end bonus score of 0."
                      },
                      {
                        "reasonCode": "QUALITY-105",
                        "message": "Measure eligible for high priority bonus with minimum performance rate and eligible populations reached."
                      },
                      {
                        "reasonCode": "QUALITY-111",
                        "message": "Measure contributed points to total score."
                      },
                      {
                        "reasonCode": "QUALITY-113",
                        "message": "Measure contributed bonus points to total score."
                      },
                      {
                        "reasonCode": "QUALITY-117",
                        "message": "Measurement points for PICKED measure include decile score with all bonus points."
                      },
                      {
                        "reasonCode": "QUALITY-005",
                        "message": "If measure is found to be in a clinical cluster the measure will be labeled with its EMA relationship"
                      }
                    ],
                    "metadata": {
                      "performanceRate": 83.33,
                      "reportingRate": 96,
                      "measurementSetId": "03d0cbf4-27d4-43db-8f57-65f6f734db5f",
                      "measurementId": "03d0cbf4-4c4c-45f3-a467-7eaec9bc33bf",
                      "measureClass": "Class 2",
                      "measureTitle": "Appropriate Testing for Children with Pharyngitis",
                      "highPriorityBonusEligible": false,
                      "endToEndBonusEligible": false,
                      "endToEndBonus": 0,
                      "outcomeOrPatientExperienceBonus": 0,
                      "highPriorityBonus": 0,
                      "decileScore": 3,
                      "performanceDenominator": 72,
                      "performanceNumerator": 60,
                      "eligiblePopulation": 75,
                      "partialDecileScore": 0,
                      "partialPoints": 0,
                      "decile": -1,
                      "deciles": null,
                      "processingStatus": "PICKED",
                      "totalMeasurementPoints": 3,
                      "totalBonusPoints": 0,
                      "messages": {
                        "benchmarkType": "Did not override registry benchmark because measure is missing EHR submission method or spec and not an eCQM measure",
                        "measurementClass": "Eligible population is greater than or equal to 20, reporting rate is greater than 60% but has no benchmarks",
                        "decileScore": "Applied maximum points for Class 2 measure meeting data completeness",
                        "e2eBonusScore": "Default E2E bonus score",
                        "measurementPicker": "Picked at 6",
                        "totalMeasurementPoints": "Measurement points for PICKED measure include decile score with all bonus points"
                      },
                      "noBenchmarks": true,
                      "benchmarkType": null,
                      "eMeasureId": "CMS146v6",
                      "toppedOut": null,
                      "isGhost": false,
                      "unroundedScoreValue": 3
                    }
                  },

Shouldn’t measure 66 have been given 1 bonus point? After all, 6 measures were picked, so we have a denominator of 60, which means up to a maximum of 6 bonus points. But only 3 were given. We’re thinking it should have been 4.

We appreciate your help and, if needed, correcting our understanding on bonus point scoring.

Thank you.

Randal Roe
Alpha II Registry
000106106_0125533694_ecdebced-abed-421e-a2f6-e123d667db8b_Score.txt

Steve Szeliga

unread,
Jan 30, 2019, 5:06:14 PM1/30/19
to Developer Group for QPP APIs
Hi, can you please provide the submission? 

Randal Roe (Alpha II Registry)

unread,
Jan 31, 2019, 10:38:46 AM1/31/19
to Developer Group for QPP APIs
Absolutely. Please see the attachment.

Thank you.

Randal Roe
Alpha II Registry
000106106_0125533694_ecdebced-abed-421e-a2f6-e123d667db8b_Quality.txt

Steve Szeliga

unread,
Jan 31, 2019, 11:11:31 AM1/31/19
to Developer Group for QPP APIs
Morning,

It appears 066 is not eligible for bonus as it is not a high priority measure. I listed the spec below, please let me know if I can provide any additional information.

{
    "title": "Appropriate Testing for Children with Pharyngitis",
    "eMeasureId": "CMS146v6",
    "nqfEMeasureId": null,
    "nqfId": null,
    "measureId": "066",
    "description": "Percentage of children 3-18 years of age who were diagnosed with pharyngitis, ordered an antibiotic and received a group A streptococcus (strep) test for the episode",
    "nationalQualityStrategyDomain": "Efficiency and Cost Reduction",
    "measureType": "process",
    "primarySteward": "National Committee for Quality Assurance",
    "metricType": "singlePerformanceRate",
    "firstPerformanceYear": 2017,
    "lastPerformanceYear": null,
    "isHighPriority": false,
    "isInverse": false,
    "isIcdImpacted": false,
    "isToppedOutByProgram": false,
    "category": "quality",
    "isRegistryMeasure": false,
    "isRiskAdjusted": false,
    "submissionMethods": [
      "electronicHealthRecord",
      "registry" 
], 

Randal Roe (Alpha II Registry)

unread,
Jan 31, 2019, 11:25:20 AM1/31/19
to Developer Group for QPP APIs
Steve:

Thank you for getting back so quickly on this. I see what you mean, that the measure spec (from measures-data.json) shows that measure 66 is not a high-priority measure. However, the attached spreadsheet, which is part of the 2018-Quality-Measure-Specifications-supporting-documents.zip file (freshly downloaded from https://qpp.cms.gov/about/resource-library, then https://qpp-cm-prod-content.s3.amazonaws.com/uploads/149/2018-Quality-Measure-Specifications-supporting-documents.zip) shows that measure 66 is a high-priority measure.

Also, the measures-data.json file for 2017 shows that this was a high-priority measure then, and the 2018_Registry_IndivMeasures_ReleaseNotes.pdf (also attached) shows the only change from 2017 to 2018 was “Updated Copyright.”

So, is it possible that the spec in measures-data.json is incorrect?

Thank you.

Randal Roe
Alpha II Registry
2018-Measure-List_v2.3.xlsx
2018_Registry_IndivMeasures_ReleaseNotes.pdf

Sarah White

unread,
Jan 31, 2019, 12:29:48 PM1/31/19
to Randal Roe (Alpha II Registry), Developer Group for QPP APIs
Hi Randal,

Thanks for flagging this. I've confirmed with the measures team this information is incorrect in the repository. We'll get it updated.

Sarah White
Business Analyst


--
You received this message because you are subscribed to the Google Groups "Developer Group for QPP APIs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qpp-apis+u...@googlegroups.com.
Visit this group at https://groups.google.com/group/qpp-apis.
To view this discussion on the web visit https://groups.google.com/d/msgid/qpp-apis/f255ce5d-cd0c-41d6-9fbd-eaab4c703a50%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Randal Roe (Alpha II Registry)

unread,
Jan 31, 2019, 3:28:59 PM1/31/19
to Developer Group for QPP APIs
Sarah:

Thank you for taking care of this. In the meantime, however, we have caught another discrepancy, this one for measure 320. From measures-data.json:

"title": "Appropriate Follow-Up Interval for Normal Colonoscopy in Average Risk Patients",

    "eMeasureId": null,

    "nqfEMeasureId": null,

    "nqfId": "0658",

    "measureId": "320",

    "description": "Percentage of patients aged 50 to 75 years of age receiving a screening colonoscopy without biopsy or polypectomy who had a recommended follow-up interval of at least 10 years for repeat colonoscopy documented in their colonoscopy report",

    "nationalQualityStrategyDomain": "Communication and Care Coordination",

    "measureType": "process",

    "primarySteward": "American Gastroenterological Association",

    "metricType": "singlePerformanceRate",

    "firstPerformanceYear": 2017,

    "lastPerformanceYear": null,

    "isHighPriority": false,

    "isInverse": false,

    "isIcdImpacted": false,

    "isToppedOutByProgram": false,

    "category": "quality",

    "isRegistryMeasure": false,

    "isRiskAdjusted": false,

    "submissionMethods": [

      "claims",

      "registry"


But the 2018-Measure-List_v2.3.xlsx spreadsheet that I attached to my last message shows that measure 320 is, in fact, a high-priority measure.

So, would you mind please taking a look at this one too?

Thank you.

Randal Roe
Alpha II Registry

Sarah White

unread,
Jan 31, 2019, 3:48:04 PM1/31/19
to Randal Roe (Alpha II Registry), Developer Group for QPP APIs
Hi Randal,

Thanks for this information. We're working on updating the repository. I'll post back here when these are changed to reflect the most recent document in the Resource Library.

Sarah White
Business Analyst

sa...@navapbc.com

unread,
Feb 22, 2019, 5:12:08 PM2/22/19
to Developer Group for QPP APIs
Hi Randal,

These measure changes should be reflected in the API with the release today. There's no need to re-submit anything as the most up to date version of the scoring engine is used each time you call GET /score on a submission.
To unsubscribe from this group and stop receiving emails from it, send an email to qpp-apis+unsubscribe@googlegroups.com.

Randal Roe (Alpha II Registry)

unread,
Feb 27, 2019, 11:16:02 AM2/27/19
to Developer Group for QPP APIs
Thanks, Sarah. We’ll retest.

Randal Roe
Alpha II Registry.
Hi Randal,

Reply all
Reply to author
Forward
0 new messages