Permissions in the Developer Preview

[iv...@navahq.com]

Hi all - 

If you're in the Developer Preview, you have an API key that is associated with an 'organization'. This affects what endpoints you are authorized to use, and what behavior each endpoint has. Below, I've detailed what endpoints you should use and how endpoints behave in regards to creating, reading, updating and deleting data.

Overview

The Submissions API has three types of objects: submissions, measurementSets, and measurements. 

• A submission object contains any performance data submitted on behalf of a single MIPS-eligible clinician or practice. 
• A measurementSet object represents a set of performance data related to one specific category (Quality, Improvement Activities, or Advancing Care Information), and is tied to a submission object. 
• A measurement object represents one single data point related to a specific measure in a given category, and is tied to a measurementSet object. 

With your organization API key, you can read, update and delete data that you yourself have submitted at the measurementSet- and measurement-level. However, only a user with the appropriate authorizations can view the entire submission object of a MIPS-eligible doctor or practice. 

Creating data

With your organization API key, you can create measurementSets and add them to any submission object. In practice, this means that with your organization API key, you are authorized to send POST requests to these two endpoints: https://qpp.cms.gov/api/submissions/measurement-sets and https://qpp.cms.gov/api/submissions/measurements. If you attempt to send a POST request to https://qpp.cms.gov/api/submissions/submissions, you will get the following ' 401Unauthorized' error message: 

{

    "error": {

        "type": "Unauthorized",

        "message": "Current user cannot create this object"

    }

}

Note: when you send a POST request to https://qpp.cms.gov/api/submissions/measurement-sets, the API will check if the submission object for that TIN/NPI/performanceYear/entityType/entityId already exists. 

• If the submission object already exists, the API will link your measurementSet object to the existing submission object.
• If the submission object doesn't exist, the API will create the submission object, and then link the measurementSet object as specified in your request payload to the submission object that the API just created.

Reading data

When you send a GET request to https://qpp.cms.gov/api/submissions/submissions, you will only see measurementSets that you've submitted. For example, if you send a GET request to https://qpp.cms.gov/api/submissions//submissions for a specific TIN, you'll only see measurementSets that you've created and submitted for that TIN, even though the submission object for that TIN may contain measurementSet objects created by other users.

Editing and deleting data

With your organization API key, you can update data that you yourself have submitted at the measurementSet- and measurement-level. In practice, this means that with your organization API key, you are authorized to send PUT, PATCH, DELETE requests to these two endpoints: https://qpp.cms.gov/api/submissions/measurement-sets and https://qpp.cms.gov/api/submissions/measurements for measurementSet objects and measurement objects that you have created.

Scoring data

When you send a GET request to https://qpp.cms.gov/api/submissions/submissions/{id}/score, you will see a score based on the data that you've submitted. For example, if you send a GET request to https://qpp.cms.gov/api/submissions//submissions for a specific TIN, you'll see a score that is calculated based only on the measurementSets that you've submitted for that TIN, even though the submission object for that TIN may contain measurementSets created by other users.

Additional notes:

• You may send a POST request to https://qpp.cms.gov/api/submissions/submissions/score-preview to see what your score would be like for a submission - without persisting it to the database. This is a good way to 'preview' what your score will look like if you submit some set of data - without officially submitting it. During the Developer Preview, no data at all is officially submitted to CMS, but this endpoint should be useful during the official reporting period.
• https://qpp.cms.gov/api/submissions/public/benchmarks is a public endpoint - which means you do not need to have an API key to send requests to this endpoint!
  

[Sam Yoder]

Hi Ivana,

Currently, our team was working on this process with the expectation that we'd be able to POST and PUT at the submissions level, but this post seems to indicate that POST/PUT are only applicable to the Measurement Set and Measurement endpoints.

Essentially our process is to construct the entire submission, and if we have that qpp-submission-id associated with this TIN/NPI combo, we make a PUT call to the submissions endpoint; otherwise we use POST.

Will we eventually be able to make these requests to the submissions endpoint?

Thanks,

Sam

[Mari Miyachi]

Hi Sam!

Authorization rules are still in flux but this is super helpful to hear your feedback on the workflow. We do want to make sure the rules we design fit the common use cases.

Note that the /measurement-sets endpoint can be called with either the submissionId or a submission object (which is composed of the identifying information of a submission like TIN/NPI). This means you could alternatively call PUT or POST to the measurement-sets endpoint one measurement set at a time. 

Mari

[Sam Yoder]

Hi Mari,

Is there any reason why you're disallowing full submissions to the submissions endpoint? We submit data on behalf of our providers, and the full submission is easily the most convenient route for us (if you can at all communicate that to the powers that be and potentially push for that functionality being allowed, we'd be really grateful). 

We are also pretty blindsided by this, I was also wondering why the sandbox (which is what we were using essentially as a test environment/test run for the beta) mentioned the ability to submit at a submission level but the beta environment doesn't? Will there be more changes like this in the future?

I was also wondering how you're keeping track of existing submissions if we're submitting on a measurement set level, will the initial call to submit a measurement set create the qpp-submission-id and we need to include that on subsequent measurement set POSTs?

Sam

[jeffi.ming...@gmail.com]

I'm surprised there isn't more feedback on this thread. I agree with Sam's comments & share the same concerns. We find this change in access to end points confusing and troublesome. There wasn't any indication until Friday that the preview access key authorization was limited to certain end points.

 

If the qualified registry doesn't have access who will be authorized & how? 

 

How will submission-level properties be set or updated without access to the submission end point? For example, I can see a use case where an entityType value needs to be changed.

 

Ivana's post mentioned measure-sets PUT and POST did not include DELETE. Are we authorized to delete?

 

I know this is a work in process, but anything you can do to provide more information proactively would be greatly appreciated. It's inefficient to encounter errors using documented functionality only later learn of undocumented restrictions.

On Friday, August 18, 2017 at 10:29:59 AM UTC-4, iv...@navahq.com wrote:

Hi all - 

If you're in the Developer Preview, you have an API key that is associated with an 'organization'. This affects what endpoints you are authorized to use, and what behavior each endpoint has. Below, I've detailed what endpoints you should use and how endpoints behave in regards to creating, reading, updating and deleting data.

Overview

The Submissions API has three types of objects: submissions, measurementSets, and measurements. 

• A submission object contains any performance data submitted on behalf of a single MIPS-eligible clinician or practice. 
• A measurementSet object represents a set of performance data related to one specific category (Quality, Improvement Activities, or Advancing Care Information), and is tied to a submission object. 
• A measurement object represents one single data point related to a specific measure in a given category, and is tied to a measurementSet object. 

With your organization API key, you can read, update and delete data that you yourself have submitted at the measurementSet- and measurement-level. However, only a user with the appropriate authorizations can view the entire submission object of a MIPS-eligible doctor or practice. 

Creating data

With your organization API key, you can create measurementSets and add them to any submission object. In practice, this means that with your organization API key, you are authorized to send POST requests to these two endpoints: https://qpp.cms.gov/api/submissions/measurement-sets and https://qpp.cms.gov/api/submissions/measurements. If you attempt to send a POST request to https://qpp.cms.gov/api/submissions/submissions, you will get the following ' 401Unauthorized' error message: 

{

    "error": {

        "type": "Unauthorized",

        "message": "Current user cannot create this object"

    }

}

Reading data

When you send a GET request to https://qpp.cms.gov/api/submissions/submissions, you will only see measurementSets that you've submitted. For example, if you send a GET request to https://qpp.cms.gov/api/submissions//submissions for a specific TIN, you'll only see measurementSets that you've created and submitted for that TIN, even though the submission object for that TIN may contain measurementSet objects created by other users.

Editing and deleting data

With your organization API key, you can update data that you yourself have submitted at the measurementSet- and measurement-level. In practice, this means that with your organization API key, you are authorized to send PUT and PATCH requests to these two endpoints: https://qpp.cms.gov/api/submissions/measurement-sets and https://qpp.cms.gov/api/submissions/measurements for measurementSet objects and measurement objects that you have created.

[Mari Miyachi]

Hi Sam and Jeffi,

Thanks for your feedback! We are actively considering alternatives and it helps a lot to understand your use cases. 

Sam - to answer some of your questions - because multiple submitters can submit on behalf of an individual or group, we made the decision to limit registry access to the /measurement-sets endpoint, which gives access to create and subsequently edit only the measurement sets that you as the submitter have created. That being said, we recognize that this means you can only submit one measurement set per request. You have two options when POSTing to the measurement-sets endpoint: (1) include the submissionId to identify the submission that you'd like the measurement set to be associated with (2) include a submission object in the payload to identify the submission you'd like the measurement set to be associated with.

Jeffi - these authorization rules are still in flux. The use case you bring up about entityType is interesting! Could you give me more details about when a registry would want to do this? And yes, you are authorized to delete measurement sets that you have created. 

Mari

[Aditya Deshpande - Head Product - FIGmd]

Mari,

I have already written an email to Ivana directly about the issue that may be faced.

The current developer preview does not help me in simulating a test condition like I was able to do for SEVT submissions in 2016.

The main issue is - how does the submission object even get created. We were specifically told that FIGmd API Key would be sufficient for testing submissions on the behalf different QR and QCDR we operate. However, we have no idea on how to create the submission object since the endpoint is not permitted.

Regards

Adi

Head - Product

FIGmd

Polaris

[Ivana Ng]

Thank you everyone for sharing your feedback and concerns with us! We want to make sure the API fits the needs of our customers, so we hope you’ll continue to work with us to improve the API.

Based on concerns that we're seeing in this thread, it is clear that we need to provide a better explanation of the purpose and intended use of the /submissions endpoint, and the authorization model we’ve designed for registries using the API.

In addition to answering some of the questions in this thread below, I’ve also made amendments to the original post above to clarify that registries can DELETE measurementSet objects they’ve submitted, and how POST /measurement-sets can create a submission object.

1. How does the submission object work? 
• The submission object is shared among many users. A submission object is tied to one single eligible-clinician or eligible-practice, and any number of users (e.g., EHRs, registries, etc.) can create measurementSet objects and link them to a specific submission object. This is why the authorization model we've created for Registries' API keys allows you to submit data on the /measurement-sets endpoint, as opposed to the /submissions endpoint.
  
  With your API key, you can create a measurementSet object of quality measures data for Dr. Smith, but you cannot create the submission object directly (see below). 
  
  
2. How is a submission object created in the first place? 
• When you send a POST request to https://qpp.cms.gov/api/submissions/measurement-sets, the API will check if the submission object for that TIN/NPI/performanceYear/entityType already exists. 

• If the submission object already exists, the API will link your measurementSet object to the existing submission object. 
• If the submission object doesn't exist, the API will create the submission object, and then link the measurementSet object as specified in your request payload to the submission object that the API just created.
  
  

1. How do I get access to the submissionId? 
• The response payload that you receive from a successful POST request to the /measurement-sets endpoint will return the submissionId. 
• You can also, with your API key, call GET /submissions, which would return all submission objects that you've created measurementSets for. (Note, however that you'll only be able to see the measurementSets that you've created for those submission objects). 
  
  
2. When I create a measurementSet object (i.e. send a POST request to /measurement-sets), how do I specify the submission object I want to link it to? 
• You have two options when POST-ing to the /measurement-sets endpoint: 
• (1) include the submissionId to identify the submission object that you'd like the measurementSet to be associated with 
• (2) include a submission object (with TIN/NPI/performanceYear/entityType) in the request payload to identify the submission you'd like the measurementSet to be associated with. 
  
  
3. Why is the sandbox different from the Developer Preview? 
• The sandbox serves as interactive developer documentation. The Developer Preview is meant to give you early access to the real deal. This means that authentication and authorization are in place – i.e. you must have an API key to send requests to the Developer Preview, and authorization is in place, just as it will be with the API that you would use to report performance data officially to CMS. 
  
  
4. If the registry doesn’t have access to the /submissions endpoint, who will – and how? 
• Internal systems, such as the QPP website, will have access to the /submissions endpoint. 
  
  
5. How will submissions-level properties be set or updated without access to the /submissions endpoint?
• This is an interesting scenario that we haven’t considered much. We’d like to better understand when a registry or other user may want to do this. 
• For context, the submissions-level properties are:
  

    "id": string,
    "createdAt": datetime,
    "updatedAt": datetime,
    "programName": string,
    "entityType": string,
    "taxpayerIdentificationNumber": string,
    "nationalProviderIdentifier": string,
    "entityId": string,
    "performanceYear": integer
  

• 
  
  

1. Are we authorized to delete?  

• Yes, you are authorized to delete measurementSet objects that you’ve created. 
  

[abhijeet mahajan]

Hi Ivana,

I actually ended up reading this post because I was researching about the DELETE and UPDATE workflows. After reading these posts following is my understanding. Could you please confirm.

Also, if you think I have missed anything request you to add more information to this topic.

On the production API, submitter organizations :

1. Will be able to create the submissions only by using /measurement-sets API. Every (first) time we create a measurement-set with a unique TIN-NPI-SubmissionMethod (for individual) and TIN-SubmissionMethod (for GPRO) , it will give us the submission-id. Subsequent measurement-sets for the same TIN-NPI or TIN will be appended to the same submission id.

2. Will NOT be able to delete this submission id. They can however DELETE the measurement set. 

    Question : if we delete all the measurement-sets that we have created, will that also delete the associated submission-id ? or it will still exist ? And we will just keep adding\removing to the SAME submission-id ?.

    I am asking from a specific use-case perspective. Discussions with my business team are moving towards deleting the whole submission instead of modifying it. What is the recommendation from the QPP API owners ?

3. Will be able to create individual measurements ONLY if they have measurement-set ids

4. Will be able to delete measurements they have created.

    Question : if we delete all the measurements that we have created, will that also delete the associated measurement-id and also if there are no more other measurement-sets, will it also delete the respective submission-id ? 

5. Will be able to update measurement-sets and measurements they have created.

Scenarios :

1) We submitted 10 measures and realized that we shouldn't have submitted few of them. 

     - My understanding is we will have to correct this by deleting the measurement(s) added by error.

2) We got score for a reporting period which we think is not good enough and we need to increase the reporting period.

     - Update the respective measurement-set with the desired reporting period and with the respective updated processing results.

I think I started reading this specific topic a bit late :( and now I am in confused state because of the differences in the developer-preview and actual production APIs. But thankfully it's not too late :-)

If you could answer this as early as possible it will help. Also, I have been referring to this so far : https://qpp-submissions-sandbox.navapbc.com/#/

But it doesn't say which API will be available in production and which one is preview-only. If you or the respective owners could get that updated, it will be extremely helpful. 

If there's already any other page\documentation which says that clearly, request you to share the link. 

Thanks,

Abhijeet

[abhijeet mahajan]

Hi Ivana,

One more important thing to confirm. So far, to initiate a submission , the request looks like this and the endpoint was /submissions. 

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<data>

    <programName>mips</programName>

    <entityType>individual</entityType>

    <taxpayerIdentificationNumber>000776259</taxpayerIdentificationNumber>

    <nationalProviderIdentifier>0718688679</nationalProviderIdentifier>

    <performanceYear>2017</performanceYear>

    <measurementSet>

        <category>quality</category>

        <submissionMethod>registry</submissionMethod>

        <performanceStart>2017-01-15</performanceStart>

        <performanceEnd>2017-12-19</performanceEnd>

        <measurement>

<measureId>093</measureId>

            <value>

                <isEndToEndReported>true</isEndToEndReported>

                    <performanceMet>4</performanceMet>

                    <performanceNotMet>1</performanceNotMet>

                    <eligiblePopulationExclusion>1</eligiblePopulationExclusion>

                    <eligiblePopulationException>0</eligiblePopulationException>

                    <eligiblePopulation>7</eligiblePopulation>

            </value>

        </measurement>

    </measurementSet>

<measurementSet>

      <category>ia</category>

      <submissionMethod>registry</submissionMethod>

      <performanceStart>2017-01-01</performanceStart>

      <performanceEnd>2017-06-01</performanceEnd>

      <measurement>

        <measureId>IA_EPA_4</measureId>

        <value>true</value>

      </measurement>

      <measurement>

        <measureId>IA_PM_2</measureId>

        <value>true</value>

      </measurement>

      <measurement>

        <measureId>IA_PSPA_5</measureId>

        <value>true</value>

      </measurement>

    </measurementSet>

</data>

But since we will not have access to /submissions, request will look like this: (creates measurement-set and eventually submission-id)

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<data>

<submission>

    <programName>mips</programName>

    <entityType>individual</entityType>

    <taxpayerIdentificationNumber>000776239</taxpayerIdentificationNumber>

    <nationalProviderIdentifier>0718688639</nationalProviderIdentifier>

    <performanceYear>2017</performanceYear>

    </submission>

    <category>quality</category>

<submissionMethod>registry</submissionMethod>

        <performanceStart>2017-01-15</performanceStart>

        <performanceEnd>2017-12-19</performanceEnd>

        <measurement>

<measureId>093</measureId>

            <value>

                <isEndToEndReported>true</isEndToEndReported>

                    <performanceMet>4</performanceMet>

                    <performanceNotMet>1</performanceNotMet>

                    <eligiblePopulationExclusion>1</eligiblePopulationExclusion>

                    <eligiblePopulationException>0</eligiblePopulationException>

                    <eligiblePopulation>7</eligiblePopulation>

            </value>

    </measurement>

        <measurement>

<measureId>091</measureId>

            <value>

                <isEndToEndReported>false</isEndToEndReported>

                    <performanceMet>4</performanceMet>

                    <performanceNotMet>1</performanceNotMet>

                    <eligiblePopulationExclusion>1</eligiblePopulationExclusion>

                    <eligiblePopulationException>0</eligiblePopulationException>

                    <eligiblePopulation>7</eligiblePopulation>

            </value>

        </measurement>

</data>

And since we don't have 'measurementSet' element in the production request while creating a submission, we can't submit Quality, ACI, IA in one request. We have to send multiple requests.

And this XML\JSON structure : https://cmsgov.github.io/qpp-submissions-docs/examples is not a valid request example.

Could you please confirm.

Thanks,

Abhijeet

[abhijeet mahajan]

In reference to my questions, i tried this use cases this morning and here are the results.

Question : if we delete all the measurement-sets that we have created, will that also delete the associated submission-id ? or it will still exist ? And we will just keep adding\removing to the SAME submission-id ?.

I am asking from a specific use-case perspective. Discussions with my business team are moving towards deleting the whole submission instead of modifying it. What is the recommendation from the QPP API owners ?

 - It doesn't delete submission. So once you create a submission, it will be there... forever

Question : if we delete all the measurements that we have created, will that also delete the associated measurement-id and also if there are no more other measurement-sets, will it also delete the respective submission-id ? 

 - It doesn't delete measurement-Set. You have to explicitly delete the measurement-set.

From the payment adjustment perspective, will there be any NEGATIVE impact  of this behavior ? 

For example: 

1. We submit a measurement-set

2. Realize that we made error and we don't wish to report this measurement-set

3. We delete the measurement-set and never submit anything again for this TIN-NPI. So there will be a submission-id with EMPTY measurement-sets.

Will there be any negative payment adjustment in that case ?

To me, not submitting and having a empty submission are the same thing. Is that the case ?

Further, there what if we will include that NPI in some other TIN and create a new submission ?

So we will have one empty submission with one TIN-NPI combination and another submission with new TIN but with same NPI combination.

Will there be any negative payment adjustment ?

In general, could you please explain the impact of not being able to delete the submissions in terms of payment adjustment ?

Thanks,

Abhijeet 

On Tuesday, 17 October 2017 16:38:02 UTC-4, abhijeet mahajan wrote:

[Ivana Ng]

Responses inline.

Ivana Ng

Product Manager

Nava

On Tue, Oct 17, 2017 at 4:38 PM, abhijeet mahajan <abhije...@gmail.com> wrote:

Hi Ivana,

I actually ended up reading this post because I was researching about the DELETE and UPDATE workflows. After reading these posts following is my understanding. Could you please confirm.

Also, if you think I have missed anything request you to add more information to this topic.

On the production API, submitter organizations :

1. Will be able to create the submissions only by using /measurement-sets API. Every (first) time we create a measurement-set with a unique TIN-NPI-SubmissionMethod (for individual) and TIN-SubmissionMethod (for GPRO) , it will give us the submission-id. Subsequent measurement-sets for the same TIN-NPI or TIN will be appended to the same submission id.

Yes this is correct.

 

2. Will NOT be able to delete this submission id. They can however DELETE the measurement set. Yes this is correct.

    Question : if we delete all the measurement-sets that we have created, will that also delete the associated submission-id ? or it will still exist ? And we will just keep adding\removing to the SAME submission-id ?. The associated submission-id will still exist.

    I am asking from a specific use-case perspective. Discussions with my business team are moving towards deleting the whole submission instead of modifying it. What is the recommendation from the QPP API owners ? You should delete your own measurement-sets (i.e. data you've submitted).

3. Will be able to create individual measurements ONLY if they have measurement-set ids Yes.

4. Will be able to delete measurements they have created.

    Question : if we delete all the measurements that we have created, will that also delete the associated measurement-id and also if there are no more other measurement-sets, will it also delete the respective submission-id ? Submission-id and measurementset-ids are not deleted when you delete measurements.

5. Will be able to update measurement-sets and measurements they have created. Yes you are able to update data that you've created.

Scenarios :

1) We submitted 10 measures and realized that we shouldn't have submitted few of them. 

     - My understanding is we will have to correct this by deleting the measurement(s) added by error. Yes this is correct. 

2) We got score for a reporting period which we think is not good enough and we need to increase the reporting period.

     - Update the respective measurement-set with the desired reporting period and with the respective updated processing results. Yes this is correct. 

I think I started reading this specific topic a bit late :( and now I am in confused state because of the differences in the developer-preview and actual production APIs. But thankfully it's not too late :-)

If you could answer this as early as possible it will help. Also, I have been referring to this so far : https://qpp-submissions-sandbox.navapbc.com/#/

But it doesn't say which API will be available in production and which one is preview-only. If you or the respective owners could get that updated, it will be extremely helpful. Agree! Adding a ticket to our backlog to add more descriptions to each endpoint on https://qpp-submissions-sandbox.navapbc.com to explain how it works in public sandbox vs production/developer preview. I don't have an ETA on when we will work on this ticket at this time.

--
You received this message because you are subscribed to the Google Groups "Developer Group for QPP APIs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qpp-apis+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/qpp-apis.

To view this discussion on the web visit https://groups.google.com/d/msgid/qpp-apis/7e46eb5b-85f1-4ee2-a242-1a2c1e4acd59%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Murdock]

So question, We now know that EHR will have to do file upload instead of using the API for 2017. So how are measurements-sets going to be created etc before the EHR file upload is file is submitted. And what would be the workflow for that. 

Thanks

Responses inline.

To unsubscribe from this group and stop receiving emails from it, send an email to qpp-apis+u...@googlegroups.com.

Visit this group at https://groups.google.com/group/qpp-apis.

[abhijeet mahajan]

Thanks Ivana ! One more request.. could you please reply my other question (submitted on 18th October) in this topic related to negative payment adjustment ? Or that's something we have to follow up with CMS.

Thanks,

Abhijeet

Responses inline.

To unsubscribe from this group and stop receiving emails from it, send an email to qpp-apis+u...@googlegroups.com.

Visit this group at https://groups.google.com/group/qpp-apis.