Issues we've encountered while migrating affiliations from V2.1 to V3.0

147 views
Skip to first unread message

Jason

unread,
Jun 16, 2019, 9:47:22 PM6/16/19
to ORCID API Users
Hello ORICD team, we’re in the midst of migrating our code from V2.1 to V3.0.  As we’re using the swagger as the seed for our code, we’ve discovered a range of issues with both the swagger defs, and changes to the values that the API accepts.  Some of these appear stylistic, others are core.  

  1. Backward incompatible or/and inconsistent valid value sets: in the earlier versions, ORCID used to require json to give categorical values as upper case, and with spaces indicated by underscores.  V3.0 has reversed this, and substituted hyphens for those underscores. For example, visibility had to be one of ["LIMITED", "REGISTERED_ONLY", "PUBLIC", "PRIVATE"], but is now ["public", "private", "limited", "registered-only"]. CURL example:

    curl -X POST --header 'Content-Type: application/vnd.orcid+json; qs=4' --header 'Accept: application/vnd.orcid+json' --header 'Authorization: Bearer c2e0d44e-0b0b-431f-9983-92a7112c09e2' -d '{"department-name":"dept", "role-title":"my invited position role", "start-date":{"year":{"value":"2017"}}, "end-date":{"year":{"value":"2021"}}, "organization":{"name":"The University of Auckland", "address":{ "city":"Auckland", "region":"Auckland", "country":"NZ"}, "disambiguated-organization":{ "disambiguated-organization-identifier":"3232", "disambiguation-source":"RINGGOLD"}}, "visibility":"public"}' 'https://api.sandbox.orcid.org/v3.0/0000-0001-6831-790X/invited-position'

    Response Body:

    {

     "response-code": 400,

     "developer-message": "400 Bad request: invalid JSON - Cannot deserialize value of type `org.orcid.jaxb.model.v3.release.common.Visibility` from String \"PUBLIC\": value not one of declared Enum instance names: [public, limited, private, system, registered-only]\n at [Source: (org.apache.catalina.connector.CoyoteInputStream); line: 46, column: 17] (through reference chain: org.orcid.jaxb.model.v3.release.record.InvitedPosition[\"visibility\"])",

     "user-message": "Invalid JSON.",

     "error-code": 9047,

     "more-info": "https://members.orcid.org/api/resources/troubleshooting"

    }

    This creates unnecessary inconsistency with the earlier versions.

  2. Error messages lack information cf V2.1: In the example of visibility, the error message explains what are the accepted values are; however, in some cases, the error message does not give this information.  For example in the previous V2.1, ORCID use to accept “external_id_relationship” as ["PART_OF", "SELF"]. Now it seems that the V3.0 API only accepts “external_id_relationship” in lowercase and the error shown in the ORCID response is not descriptive enough to know about the expected values. Response Example:

    { "response-code": 400, "developer-message": "400 Bad request: invalid JSON - (was org.orcid.jaxb.model.common.adapters.IllegalEnumValueException) (through reference chain: org.orcid.jaxb.model.v3.release.record.InvitedPosition[\"external-ids\"]->org.orcid.jaxb.model.v3.release.record.ExternalIDs[\"external-id\"]->java.util.ArrayList[0]->org.orcid.jaxb.model.v3.release.record.ExternalID[\"external-id-relationship\"])", "user-message": "Invalid JSON.", "error-code": 9047, "more-info": "https://members.orcid.org/api/resources/troubleshooting" }

    CURL example:

    curl -X POST --header 'Content-Type: application/vnd.orcid+json; qs=4' --header 'Accept: application/vnd.orcid+json' --header 'Authorization: Bearer c2e0d44e-0b0b-431f-9983-92a7112c09e2' -d '{"department-name":"dept", "role-title":"my invited position role", "start-date":{"year":{"value":"2017"}}, "end-date":{"year":{"value":"2021"}}, "organization":{"name":"The University of Auckland", "address":{ "city":"Auckland", "region":"Auckland", "country":"NZ"}, "disambiguated-organization":{ "disambiguated-organization-identifier":"3232", "disambiguation-source":"RINGGOLD"}},   "external-ids":{      "external-id":[         {            "external-id-type":"grant_number",            "external-id-value":"arbitraty_value",            "external-id-url":{"value":"https://www.grant-url.com"},            "external-id-relationship":"SELF"         }      ]   },   "visibility":"public" }' 'https://api.sandbox.orcid.org/v3.0/0000-0001-6831-790X/invited-position'

    Response Body:

    {

     "response-code": 400,

     "developer-message": "400 Bad request: invalid JSON - (was org.orcid.jaxb.model.common.adapters.IllegalEnumValueException) (through reference chain: org.orcid.jaxb.model.v3.release.record.InvitedPosition[\"external-ids\"]->org.orcid.jaxb.model.v3.release.record.ExternalIDs[\"external-id\"]->java.util.ArrayList[0]->org.orcid.jaxb.model.v3.release.record.ExternalID[\"external-id-relationship\"])",

     "user-message": "Invalid JSON.",

     "error-code": 9047,

     "more-info": "https://members.orcid.org/api/resources/troubleshooting"

    }

  3. Swagger defs not being maintained with care.   i) The swagger.json file at https://api.sandbox.orcid.org/resources/swagger.json still has the old uppercase versions of values

    Examples from the swagger.json file:


           "country" : {

             "xml" : {

               "namespace" : "http://www.orcid.org/ns/address"

             },

             "$ref" : "#/definitions/CountryV3_0"

           },

           "visibility" : {

             "type" : "string",

             "xml" : {

               "attribute" : true

             },

             "enum" : [ "LIMITED", "REGISTERED_ONLY", "PUBLIC", "PRIVATE" ]

           },


    and

        "external-id-url" : {

             "xml" : {

               "namespace" : "http://www.orcid.org/ns/common"

             },

             "$ref" : "#/definitions/UrlV3_0"

           },

           "external-id-relationship" : {

             "type" : "string",

             "xml" : {

               "namespace" : "http://www.orcid.org/ns/common"

             },

             "enum" : [ "PART_OF", "SELF", "VERSION_OF" ]

           }

         }

    ORICD either needs to update swagger.json, or do something related to case sensitivity.

    ii) start_date is mandatory for affiliations in the swagger.json, but in 3.0 reality it’s optional. So the swagger is enforcing a requirement that the schema doesn’t have.

    Reference from swagger.json file:

    "EducationV3_0" : {

         "type" : "object",

         "required" : [ "organization", "start-date" ],

    CURL example:

    curl -X POST --header 'Content-Type: application/vnd.orcid+json; qs=4' --header 'Accept: application/vnd.orcid+json' --header 'Authorization: Bearer c2e0d44e-0b0b-431f-9983-92a7112c09e2' -d '{"department-name":"dept", "role-title":"my invited position role", "organization":{"name":"The University of Auckland", "address":{ "city":"Auckland", "region":"Auckland", "country":"NZ"}, "disambiguated-organization":{ "disambiguated-organization-identifier":"3232", "disambiguation-source":"RINGGOLD"}}, "visibility":"public"}' 'https://api.sandbox.orcid.org/v3.0/0000-0001-6831-790X/employment'

    Response Body:

    no content

    Response Code:

    201

    Response Headers:

    {
     "pragma": "no-cache",
     "date": "Thu, 13 Jun 2019 04:16:43 GMT",
     "x-content-type-options": "nosniff",
     "server": "nginx/1.10.0",
     "location": "http://api.sandbox.orcid.org/v3.0/0000-0001-6831-790X/employment/35434",
     "x-frame-options": "DENY",
     "content-type": "application/vnd.orcid+json; qs=4;charset=UTF-8",
     "access-control-allow-origin": "*",
     "cache-control": "no-cache, no-store, max-age=0, must-revalidate",
     "connection": "keep-alive",
     "content-length": "0",
     "x-xss-protection": "1; mode=block",
     "expires": "0"
    }

    NB: We have already patched these issues at our end together with some other minor issues related to generated swagger code. (reference: https://github.com/Royal-Society-of-New-Zealand/NZ-ORCID-Hub/pull/707/file)

  4. An issue with the swagger schema stability: after adding a new API version spec, the V2.1 model object schema names were changed, thus making the generated code even for the previous versions backward incompatible. It was expected that the generated code for point releases wouldn’t change because of a subsequent release.

  5. An issue with versioning in the swagger schema: non-standard versioning is used for new API versions utilizing tags. While it works with entry points (since for each tag the code generator creates a separate namespace/package), it doesn’t work the same way with the model schema. This approach of using version suffixes produces backward incompatible code and required substantial changes in the consumer code. Our recommendation is that a separate OAS specification is produced for each version and following the OAS specification. In this way, it would be possible to maintain each version and generate the code for the versions independently from each other; this would also render issue #4 moot.

We hope raising these issues is useful to ORCID, and helps anyone whose approaching the ORCID API in a similar way to us. If we're mistaken in our understanding of behaviour that would be useful too.

Best wishes,

Rad, Roshan, and Jason.

Jason

unread,
Jun 16, 2019, 10:11:11 PM6/16/19
to ORCID API Users
NB: Testing mixup: 

curl -X POST --header 'Content-Type: application/vnd.orcid+json; qs=4' --header 'Accept: application/vnd.orcid+json' --header 'Authorization: Bearer c2e0d44e-0b0b-431f-9983-92a7112c09e2' -d '{"department-name":"dept", "role-title":"my invited position role", "start-date":{"year":{"value":"2017"}}, "end-date":{"year":{"value":"2021"}}, "organization":{"name":"The University of Auckland", "address":{ "city":"Auckland", "region":"Auckland", "country":"NZ"}, "disambiguated-organization":{ "disambiguated-organization-identifier":"3232", "disambiguation-source":"RINGGOLD"}},   "external-ids":{      "external-id":[         {            "external-id-type":"grant_number",            "external-id-value":"arbitraty_value",            "external-id-url":{"value":"https://www.grant-url.com"},            "external-id-relationship":"SELF"         }      ]   },   "visibility":"PUBLIC" }' 'https://api.sandbox.orcid.org/v3.0/0000-0001-6831-790X/invited-position'

Is the call that fails, having "visibility":"public" would/will be successful.

Rob Blackburn

unread,
Jun 19, 2019, 4:50:52 AM6/19/19
to ORCID API Users
Hi Jason , Rad and Roshan,

Thank you very much for your extensive feedback. It is really helpful for us to get feedback such as yours. The first point about JSON now being lower case and hyphenated is a move we made to bring the JSON in line with our XML and we noted this change along with the other changes in 3.0 in the guide here:  https://github.com/ORCID/ORCID-Source/blob/master/orcid-model/src/main/resources/record_3.0/README.md.

Thanks for your notes about error messages we will look into this more and see where inconsistencies lie. 
As to you points about swagger, these too are all very valid points and we will work to update them as soon as possible.

When I have added these issue to our Trello boards I will update this thread with links so you can track progress of these issues.

Thanks again for your feedback,


Regards,

Rob

Rob Blackburn

unread,
Jun 19, 2019, 6:11:36 AM6/19/19
to ORCID API Users
Hi all,

Further to this you can follow it's progress on our new member feedback board here https://trello.com/c/yrskPPDX .

Thanks

Rob

On Monday, June 17, 2019 at 2:47:22 AM UTC+1, Jason wrote:
Reply all
Reply to author
Forward
0 new messages