[OpenSherlock] Gold Standards

[Jack Park]

In a response to one of my early posts here, Benjamin Good asked how I figured to evolve a Gold Standard for testing.  Ben is busy with crowd-sourced evolution of Gold Standards, but today, I discovered one that might fit an early bill, at least for biomedical literature-based harvesting:

http://skr3.nlm.nih.gov/SemMedDB/

A downloadable database of some 70million semantic triples against more than 23 million documents.

Open issues include these:
a) mapping between OpenSerlock's triple representation and that of the database
b) validation related to the fact that many documents, perhaps even those already harvested, could be from documents later retracted for varieties of reasons.

Nevertheless, that database appears to provide at least one approach to Gold Standard verification and validation of harvested results.