Actual intended model for Person External Identifiers?

[Jason]

Hi ORCID peeps, 

More woes tracking down documentation to point to the API's actual behaviour for what is likely a infrequently accessed part.

For person external identifiers the V3.0 API is enforcing that to be valid the relationship must be given as "self", and a URL must be present despite both the most recent swagger and the ORCID model having these as optional.

The only place this self relationship appears to be hinted at (other than in code) appears to be here: https://members.orcid.org/api/tutorial/reading-xml where it's noted that 'Like all identifiers, the personal external identifier is described using the for tags: type, value, url, and relationship, however, personal identifiers should always have the relationship set to "self."'  However that's not "must"; and this tutorial page can't be the only place that this behaviour is documented.  

On the other hand outside of code and behaviour, we can't find any note to say that the person external identifier url is a reference field that must not be empty.

What should the model be, and where can we find that actual reference?

Cheers,

Jason.

[Pedro Costa]

Hi Jason,

Thanks for posting about this.

We have a tutorial about personal identifiers where all the required fields are listed:

https://github.com/ORCID/ORCID-Source/blob/master/orcid-api-web/tutorial/personal_identifiers.md

The API requires the fields external-id-type, external-id-value, external-id-url, and  external-id-relationship to contain data. The field external-id-relationship must always be set to 'self'.

The ORCID-model is used for writing and reading data so it must account for situations where some data may not be present. Some fields in the model show as not required so that it is possible to read items that may have been created without those fields (at a time when those given fields where not mandatory). But for creating new items, there are then other validation mechanisms used further to the XSD validation. One of those mechanisms is the PersonValidator which you have already identified.

I hope this clarifies the situation. I'll be happy to answer any other questions you may have.

Regards,

Pedro

[Jason]

Wow thanks Pedro,

This is a resource we've been after for a while, so very pleased to be pointed at it.  Huh, two years old but we must've missed its release.

Yes, the relaxed .xsd makes sense I guess, tho now knowing this I'm a little surprized in retrospect that we were warned of the impending requirement in V3.0 for affiliation start dates by the xsd and which would have presumably broken in the way you're suggesting.

While the tutorial pages are very nice, and definitely better than nothing, shouldn't this kind of restriction 1/ be in the xsd annotation, and 2/ spelled out in the swagger which can accommodate different requirements in GET and POST (I believe).

Finally, we'll obviously treat the person external identifier as the model requires; however, can we get some insight in to why and when url became a requirement for the person external identifier?  One of the casualties of the model's shift to its own repository has been to wipe all of the history pre-2020.

Cheers,

Jason.

[Pedro Costa]

Hi Jason,

Thanks for replying.

We appreciate your feedback about our technical documentation. We're looking at ways to make it better!

You can see all our tutorials in GitHub here: https://github.com/ORCID/ORCID-Source/tree/master/orcid-api-web/tutorial

As to the requirements for person external identifiers, I note URL was already required in v2; it's not a new requirement.

Thanks,

Pedro

[Jason]

Thanks Pedro,

Yes, indeed the tutorials are rather nice; however what I was after was the reasoning behind the url being required.  

You're quite right that this is an old requirement (as evidenced by its presence in the 2018 tutorial), but as a part of the model that's likely only used by a handful of vendors and who must already be pushing urls, this didn't come to our attention.

1/ Can we get https://members.orcid.org/api/news/xsd-20-update allowed again as this is presumably where this change was notified as it's currently giving "403 - Access Denied" for me.

NB: the whole area appears to be quite a mess, i.e., for just the 41,751 records with a public NZ-based assertion that are found in the 2019 public data file, we see a wide range of ids and uses:

ID Type	ID URL "Domain(s)" in use for the ID Type	Instances of ID Type	Most recent modification in:
Authenticus	https://www.authenticus.pt/	7	2017
AuthenticusID	https://www.authenticus.pt/	37	2019
AuthID	https://www.authenticus.pt/	15	2019
Chalmers ID	None	2	2014
Ciência ID	http://www.cienciavitae.pt/, https://qa.cienciavitae.pt/	572	2019
Digital author ID	https://en.wikipedia.org/	13	2019
eScientist	https://www.escientist.cn/	2	2018
GitHub	https://github.com/	3	2019
GND	https://d-nb.info/	7	2019
Google Scholar	http://scholar.google.com/	3	2019
HKU ResearcherPage	http://hub.hku.hk/	13	2018
HKUST Profile	http://repository.ust.hk/	5	2019
iAuthor	http://iAuthor.las.ac.cn/	1	2016
ID Dialnet	http://dialnet.unirioja.es/	9	2019
ISNI	http://isni.org/	76	2019
Loop profile	http://loop.frontiersin.org/, https://loop.frontiersin.org/	4859	2019
Pitt ID	https://orcid.pitt.edu/	29	2018
Profile system identifier	http://A-123456	1	2018
Researcher ID	http://www.researcherid.com/	42	2019
Researcher Name Resolver ID	http://rns.nii.ac.jp/, https://nrid.nii.ac.jp/	172	2019
ResearcherID	http://www.researcherid.com/, https://publons.com/, G-4817-2010	17072	2019
Scopus Author ID	http://www.scopus.com/	23646	2019
Scopus Author ID:	http://www.scopus.com/	3	2019
Scopus Author ID:	http://www.scopus.com/	13	2019
Scopus ID	http://www.scopus.com/	10	2018
Smithsonian Profiles	http://profiles.si.edu/	1	2018
Technical University of Denmark CWIS	http://www.dtu.dk/	43	2019
UOW Scholars	http://uowvivo.uow.edu.au/, http://scholars.uow.edu.au/	23	2019
US EPA VIVO	https://vivo.epa.gov/	5	2018
中国科学家在线	http://iAuthor.las.ac.cn/	5	2017

So it's obvious that (albeit infrequently) some integrations are resorting to putting text content into this field purely in order to satisfy the requirement.  It also looks like some integrations might be posting ids for other services?

2/ ORCID Inc might also want to review the example record(s), i.e., the relevant part of the record-3.0.xml read sample https://github.com/ORCID/orcid-model/blob/9afc7c0a929b834a644a58fe6e6312407b8389eb/src/main/resources/record_3.0/samples/read_samples/record-3.0.xml#L166 gives the person external identifier the invalid relationship of "part-of".

Cheers,
Jason

[Pedro Costa]

Hi Jason,

Thanks for your questions.

The URL became a requirement for personal identifiers in 2.0_rc2. The URL was optional in API version 1.2; however, it was already normal for members to include it. You can see release notes for 2.0_rc2 here:

https://members.orcid.org/api/news/api-version-20rc2-released  

ORCID advocate for FAIR principles (https://www.go-fair.org/fair-principles/metadata-retrievable-identifier-standardised-communication-protocol/) -- in the case of personal identifiers, we believe both humans and machines should be able to know the location of a given external identifier. That is why an URL is required.

Thanks for pointing out the inconsistency with the record-3.0.xml read sample. I'll open a card so it can be fixed.

Kind regards,

Pedro

[Jason]

Thanks for following up Pedro,

I can see why personal identifiers must come with a greater degree of responsibility than inanimate ones for organisation, project, works, etc, although it would be great if ORCID Inc were explicit that personal external identifiers are being held to a higher bar.  

NB: https://members.orcid.org/api/news/api-version-20rc2-released does not appear to state that url has become a requirement for personal external identifiers, and instead suggests the opposite to the extent that identifiers are reported as sharing the same schema, i.e., :

• All identifiers will now use the same schema element, external-identifier, this affects how identifiers are recorded in the works, funding and peer-review sections
• The new external-identifier elements include external-id-url where the full URL the identifier links to can be recorded.

Is this really the reference you were intending?  Was it actually announced?

Cheers,
Jason.

[Will Simpson]

Hi Jason,

Pedro and I were able to pin down the change to 2.0_rc2 by looking through the code. The release notes mention some changes to external identifiers, but indeed not the URL requirement for person external identifiers.

If  I recall correctly, at the time we worked individually with members that were posting person external identifiers to make sure they were already doing so, or were easily able to add them.

Do you have a use case where it is not possible to include the URL?

Best regards,

Will

[Jason]

Hi Will,

Do we have a real use case?  Not at all; this simply came up as we build against, and attempt to document, all the parts of v3.0 that can be asserted via the API.  See our erroneously generated issue here:

https://github.com/Royal-Society-of-New-Zealand/NZ-ORCID-Hub/issues/1270

In practise, I can imagine a raft of personal identifiers that don't come with a resolution service but can't think of any instances where I'd want to have them posted to the registry; and possibly it's a good rule that if not resolvable then too private for the registry. 

NB: This is effectively ancient history, well predating my/our earlier rant on Undocumented API behaviour changes? No more naked end-dates or part-of identifiers?? : however "What these experiences have emphasized for us is that when ORCID says vX.X of the API is a release version, what is actually being referred to is the combination of xsd together with the release notes, and it's the release notes which have precedence." turns out to need to be supplemented by it actually being the tutorials that have preference :(.  

BTW The part-of part was another instance of ORCID Inc seeming to only test changes with those who were already writing to the affected sections, and I hope that consultation for v3.1+ will be more open than this.

Cheers,

Jason.

[Tom Demeranville]

Hi Jason,

The idea of a community consultation for v3.1 is definitely a good one.  We're planning to put together a list of proposed v3.1 changes, and then engage with the community before implementation.  Feedback on what we're doing and help deciding what we should prioritise would be very welcome.  I'll make sure you are invited

Cheers,

Tom Demeranville

[Will Simpson]

Thanks for your thoughts, Jason.

Yes, we can definitely do a better job of documenting validation rules in the future.

Also, I hope we can work with members in the future to clean up the bad URL data.

Best regards,

Will