state of proposal

[Arlin Stoltzfus]

Hello all.  I'm back from vacation and just took a look at the proposal.  There are a lot of good ideas in there.   

I'd like to suggest a way to tie things together with a common focus and consistent language.   There are many aspects to representing and sharing taxonomic information, and some people have been doing this for 20 years, including at international meetings (TDWG), and in funded projects.  My sense is that a lot of this work has focused on developing and populating representations of expert taxonomic knowledge.   IMHO we need to build on that, but we really want to avoid sounding like we are going to do more of it.   We aren't focused on collecting and organizing names or concepts.  

Instead, our focus is on delivering that expert knowledge in the context of name-resolution.  Whether or not taxonomic concepts are involved in mediating the resolution of names, the goal is to deliver expert name-mappings to users doing data integration.   Previously, perhaps, experts built taxonomic knowledge systems for other experts to use, and so the problem of how to use the information wasn't as pressing.  But we want to bring this knowledge to the rest of the world, i.e., non-experts, including interactive users and automated clients.  

Whether name-resolution is automated or interactive, it has several dimensions including 

(1) a theory of name resolution fleshed out in the terms of the back-end data model, 

(2) a theory of what users are trying to accomplish (starting conditions, satisfaction), and 

(3) a strategy for solving the user's name-resolution problem given (1) and (2).  

Within that general strategy, we are headed in a particular direction, based on the composition of our working group, and our discussions so far:  

* we assume that there is not a single taxonomic authority; instead, multiple authorities will be queried by users

* we define users' needs (i.e., #2) by use-cases experienced in the context of OToL, Phylotastic, Map of Life, etc.  

* possibly, we assume the use of web services as part of the technology strategy (#3 above)

Users will query multiple authorities to find the best resolution for their names.  From the perspective of resource-providers, this means we want to facilitate aggregation and linking across multiple services or namebanks, so as to better serve users.  

I think all of this specificity is good.   We want to present NESCent with a clear picture of what we aim to do.   Overall, I think we can describe the goal of this working group as a multi-faceted attack on practical name resolution, in the context of use-cases where the starting condition is a set of 2 or more resources indexed by names (or concepts via names, if you will), and the goal is to align those resources by leveraging multiple sources of taxonomic information. 

Arlin

-------
Arlin Stoltzfus (ar...@umd.edu)
Fellow, IBBR; Adj. Assoc. Prof., UMCP; Research Biologist, NIST
IBBR, 9600 Gudelsky Drive, Rockville, MD, 20850
tel: 240 314 6208; web: www.molevol.org

[David Patterson]

Arlin

No disagreement. Just a little elaboration. I don't think you need
to worry too much about sounding as if we are repeating the efforts of
taxonomic aggregators (e.g. CoL, IRMNG, WoRMS).

a). None of them provide a full spectrum of services that you / we need
b). They provide singular points of view, and don't evolve quickly;
and there are many points of view that are forever changing, so we can
add richness
c). They do not embrace all content providers, so they are incomplete
d). Sources are heterogeneous
e). Are not consistent in emitting content in compliance with standards

f) certainly not targeting a role in data integration

So we can commit to working out solutions that will overcome those
problems and then apply reconciliation and resolution on top of
aggregation and integration

In addition to aggregation of taxonomic efforts, uBio and now Global
Names have been making progress on more names related stuff, there are
others working on vernacular names, and so on. We need to also ensure
that what we do is a significant step forward from those projects. I
concur that connecting the names work directly to data integration is
of very high value.

Conference call?

PAddy

> --
>
>



--
___________________________________
David J Patterson

Senior Scientist, Marine Biological Laboratory
7 MBL Street, Woods Hole, MASS 02543, USA.

Research Professor
School of Life Sciences, Arizona State University
Tempe, AZ 85287-4501

Professor (MBL) Ecology and Evolutionary Biology
Brown University, Providence, Rhode Island

Life Sciences Lead, Data Conservancy dataconservancy.org

globalnames.org

[Cody Hinchliff]

Just a comment: from the perspective of OToL, data aggregation and integration are of primary importance. We can make use of external TNRS services for some things, but we absolutely need to maintain an internal taxonomic hierarchy as well. At this point, we are cobbling together a very ugly chimeric taxonomy because a single, complete, dependable option does not seem to exist. If such a resource did exist, it would trivialize data integration among services such as OToL, Map of Life, Phylotastic, etc., including name/concept resolution. I see this as an important target for phyloinformatics in general. Not sure what others think about this.

--

[Matt Yoder]

I think what you want here is quality assurance levels, or a QA
ontology. If such a resource exists (it does, sort-of, see CoL
aggregation of 100 + database), how would you know whether any one
record is any good (you don't). Am I correct that you'd largely be
content if you could heat-map quality on to your cobbled tree? This
branch is rock solid, this other one, not-so-much.

M

> --
>
>

[Cody Hinchliff]

We need taxonomic names in order to map the names in phylogenetic trees to nodes in the tree of life db. So for us, confidence is essentially binary: good (accepted, valid, correct placement) or bad (illegitimate, incorrect placement, junior synonym, etc). Bad names should be excluded; if they are not, input phylogenies get mapped to the wrong places in the tree of life (resulting in utter chaos from the user's perspective).

[Cody Hinchliff]

Also, COL is one of the data sources we have used to make the chimera we are using now; from our experience, its deeper relationships are fairly inaccurate, and there are data integrity problems (I could get more specific if that would be helpful).

[Matt Yoder]

I guess I see this as the extreme opposite sides of the QA ontology
spectrum. We all know there are a myriad of considerations into what
makes a name good/bad, some of them logical, others idiosyncratic
("it's bad because I don't follow it" <- happens *all* the time). One
consequence of considering a APIs for everyone solution is your data
become even more variable. However, as part of a QA ontology, we could
include reasoning that would allow users to infer these extreme end
points from any internal starting point- i.e. have a special "path"
for making decisions like you need.

M

> --
>
>

[Matt Yoder]

Strange thing about CoL that I didn't know until very recently (and
it's hard to actually tell this)- they historically didn't care about
any higher relationships, so what is there now can a) only be gained
via hackish approaches and b) sucks, by their own admittance. The
goal for CoL is a checklist of species- i.e. if you resolve a
*species* there it is supposedly supposed to be good, or at least from
as authoritative source as exists (YRMV). Where that species actually
fits in is anybodies guess, it's just known to be "good". That said-
they have promised significant improvements in the next couple of
months, a concentrated effort was made to improve the higher levels
specifically because people were using this data even though it was
not intended to be used as such.

M

> --
>
>

[Matt Yoder]

Re data integrity- I'm not really the person to beat on since I have
no official ties. I'm largely playing devils advocate here, trying to
get folks to see what's happened historically when integrated
approaches were attempted, with the hope that mistakes can be
leapfrogged. Do contact me off-list with details and I'll pass them on
to David Eades, a chair with CoL, and part of our group. Think
though- the mechanisms by which you detected these problems might be
good general approaches to any data/this proposal?

M

> --
>
>

[Cody Hinchliff]

Right, that makes sense. I think this is actually a very good example of why the OToL project would benefit from this taxonomy/TNRS project you all are discussing: OToL neither has the interest nor the resources to tackle this "QA ontology spectrum" as you phrase it. We need (at a minimum) a set of "good" taxonomic names, in a working (i.e. acyclic) taxonomic hierarchy, that is vetted by the community and used as the default internal OToL taxonomy. We can additionally have options for users to "prefer alternative taxonomic relationships, and even have mechanisms for these user-supplied alternatives to be made available to other users or incorporated into the default (e.g. in the case of a taxonomic expert uploading their own taxonomy). It would be ideal if the QA stuff, the vetting, the selection of "accepted" or "preferred" names could happen externally, in a taxonomic resources that could be shared by things that need it but don't curate it, such as OToL and other services. 

[Cody Hinchliff]

I imagine that could work well. We definitely need a mechanism to address taxonomic conflicts, such as homonyms, which have been a nasty problem in our chimera.

For instance, I can imagine that depending on the structure of data and the types of queryable-information, it could still be possible to get conflicting results. For example, perhaps a genus named "A" has been described in both insects and crustaceans. One of these is taxonomically invalid, but if that information hasn't been included in the db, then both could conceivably be returned by even a fairly restrictive query. From the perspective of the OToL machinery, this would be seen as a cycle in the taxonomy connecting insects and crustaceans through the node "A" (not good). These types of things are abundant in the current data sources, and we have had to curate many such conflicts by hand, which is good enough for a first pass but otherwise unsustainable. Presumably, these types of conflicts could be broken algorithmically with the right kind of data. Do you envision the query system being able to do things like this, or would this kind of thing be up to clients?

On Mon, Nov 26, 2012 at 3:22 PM, Matt Yoder <diap...@gmail.com> wrote:

Don't know if you wanted to send this off list.

Yes, exactly, essentially imagine an annotation on any given name that
links to a QA class- you just have to parse this annotation to get the
+/- result you want.  Providers would classify the data.

M

On Mon, Nov 26, 2012 at 2:20 PM, Cody Hinchliff <codif...@gmail.com> wrote:
> Right, that makes sense. I think this is actually a very good example of why
> the OToL project would benefit from this taxonomy/TNRS project you all are
> discussing: OToL neither has the interest nor the resources to tackle this
> "QA ontology spectrum" as you phrase it. We need (at a minimum) a set of
> "good" taxonomic names, in a working (i.e. acyclic) taxonomic hierarchy,
> that is vetted by the community and used as the default internal OToL
> taxonomy. We can additionally have options for users to "prefer alternative
> taxonomic relationships, and even have mechanisms for these user-supplied
> alternatives to be made available to other users or incorporated into the
> default (e.g. in the case of a taxonomic expert uploading their own
> taxonomy). It would be ideal if the QA stuff, the vetting, the selection of
> "accepted" or "preferred" names could happen externally, in a taxonomic
> resources that could be shared by things that need it but don't curate it,
> such as OToL and other services.
>
>

[Cody Hinchliff]

I realize this is moving toward nitty gritty stuff that is beyond the scope of the current discussion, just bringing these things up for the sake of thinking about the problems that we are already dealing with.

[Matt Yoder]

On Mon, Nov 26, 2012 at 2:36 PM, Cody Hinchliff <codif...@gmail.com> wrote:
> I imagine that could work well. We definitely need a mechanism to address
> taxonomic conflicts, such as homonyms, which have been a nasty problem in
> our chimera.
>
> For instance, I can imagine that depending on the structure of data and the
> types of queryable-information, it could still be possible to get
> conflicting results. For example, perhaps a genus named "A" has been
> described in both insects and crustaceans. One of these is taxonomically
> invalid, but if that information hasn't been included in the db, then both
> could conceivably be returned by even a fairly restrictive query. From the
> perspective of the OToL machinery, this would be seen as a cycle in the
> taxonomy connecting insects and crustaceans through the node "A" (not good).
> These types of things are abundant in the current data sources, and we have
> had to curate many such conflicts by hand, which is good enough for a first
> pass but otherwise unsustainable. Presumably, these types of conflicts could
> be broken algorithmically with the right kind of data. Do you envision the
> query system being able to do things like this, or would this kind of thing
> be up to clients?

I was thinking client level annotation- however the QA ontology might
help clients in the discovery process through other TNRS- i.e.
homonyms exist because not enough data has been cross referenced. If
your QA ontology determines that you have a low quality name because
you have not tied your name to references, say, or specimens, you
might then leverage this information to find out more about the name
elsewhere- in the process determining homonomy. This in turn takes
time to publish on to reconcile (though there are in fact people
mindlessly going through this process making terrible decisions
semi-automagically).

Luckily homonyms should be relatively rare, i.e. we can meet QA in
other ways for the "80%" as Arlin likes to point out.

In your usecase, from an application standpoint- you might detect
homonyms and because you know nothing about the quality of either one
(let's assume), you are safe to arbitrarily add the node wherever you
want. Without a published act that fixes the names there really isn't
anything you can do to make it better.

M

[Nicolson, David]

All,

To be clear, as Matt has noted, the CoL has historically been focused primarily on species and the nesting within the "sectors" (a "sector" in CoL being a taxonomic chunk of data from one provider, typically including the top named node, and the main taxonomic ranks within, down to species and in some cases below), with relatively less attention to the hierarchy above, upon which these sectors are hung. To some extent this will change, though.

There has been an ongoing project within the Species 2000 & ITIS Catalogue of Life partnership to produce updated consensus classifications AT LEAST down to the rank of Order for each kingdom. There is some ongoing wrangling in the team of contributors (including kingdom- and group-specific specialists and overarching workers), but most of the kingdom "management hierarchies" (as they are being called) are essentially complete, and the plan is to initially implement these in the structured ITIS database, then use that to implement the new hierarchy in CoL (though possibly with some super/sub/infra ranks dropped, depending on how CoL is able to manage it).

But the full hierarchy, including all the ranks will be in ITIS as these are finalized and implemented (we added support for several ranks to support this, with more to come). E.g., the new hierarchy for kingdom Plantae is already in ITIS (and it is radically improved over what we had before in Plantae), though there are a few additional adjustments we plan to make, based on new feedback from the hierarchy team. Animal hierarchy is next, once they finalize it and other priorities are closed out (it is likely to take a few months to fully integrate it with the existing ITIS data, then it will be loaded into ITIS after that).

But note that this will remain a consensus hierarchy for managing pertners' taxonomic data, and as such it will not always strictly aimed at phylogeny per se... Does that make sense? I guess it depends on where you draw the line between phylogenetics and taxonomy.

ANYWAY, since the subject came up, I wanted this group to at least know where things stand re the pending hierarchy work involving ITIS and CoL. The hierarchies will almost certainly be "published" in one way or another once things settle down, but for the moment, you can at least examine the hierarchy for Plantae as now shown in ITIS via the following page (choose Plantae, and other options):
http://www.itis.gov/hierarchy.html

Everyone who wants to can use that, or anything else from ITIS.

Cheers,
Dave

David Nicolson
Data Development Coordinator, Integrated Taxonomic Information System
Biologist, USGS Core Science Systems, Core Science Analytics & Synthesis Program
nico...@si.edu Office 202-633-2149 Fax 202-786-2934
http://www.itis.gov/
http://www.cbif.gc.ca/itis/
"Nihil sumas necesse est..."

--

[David Patterson]

With names, some quality issues are objective. These are addressed
through nomenclators

Some issues relate to difference of opinions (I think this species is
real vs I think it is the same as that one over there), or the
hierarchy in which they are placed.

We can easily list out the problems, work out how to code them, work
out how to automatically fix objective errors, and how to allow
alternative views to show differing points of view. The latter is
what we do when we offer Resolution Services against multiple
authoritative taxonomic sources.

But are we meandering off track here?

Paddy

[David Patterson]

In that particular case, Tony Rees in assembling IRMNG paid particular
attention to the genera, and because of that had to deal with homonyms
(and most homonyms are genera, not species). He has compiled a big
list, and we have disambiguating taxonomic context.

But it is a source that we need to effectively integrate, and it still
leaves the problem that if you get a species in a dataset that simply
says Aa atlantica, and no classification contains this species (Dima's
problem), how do you whether it is an Aa an orchid, or an Aa the
snail.

Fortunately, with time that problem will diminish, as we add
verifiable taxonomic context to more and more of the content. I
presume queries can be made clade-specific or raise warning signs when
homonymic genera (or species) are involved.

But recollect that cross-code homonyms are NOT illegal, so we will
have that matter to deal with for ever.


Paddy

[Cody Hinchliff]

But it is a source that we need to effectively integrate, and it still
leaves the problem that if you get a species in a dataset that simply
says Aa atlantica, and no classification contains this species (Dima's
problem), how do you whether it is an Aa an orchid, or an Aa the
snail.

Yes. We hope to avoid dealing with these problems directly by relying on a default taxonomy that contains all known species. This is admittedly a bit idealistic, so we will probably also have some kind of mechanism for users to add species manually. How that might work is pretty fuzzy at the moment.

 

But recollect that cross-code homonyms are NOT illegal, so we will
have that matter to deal with for ever.

Yes. Cross-code homonyms (I just call these valid homonyms) are not a problem; sections of the tree governed by different codes are treated independently.

[Arlin Stoltzfus]

Yeah, web services are not a cure-all.  What you are describing sounds like the way that ITIS works with partners (according to what ITIS folks told me).  They incorporate new taxonomic information (with expert curation) into their back-end data store, and then the partner integrates that datastore into the partner's custom in-house data management system.  The partner downloads the updated version of the data every month.  

Do you think that the working group should commit to an approach based on web services (understanding that this is just one [large] slice of the pie), or should also pursue a strategy for aggregating back-end name-banks?  If we can put them all in DWC-A, then translate that into RDF, they all can go into a triplestore.  But I suppose there will be complications.  

Arlin

--
 
 

[Arlin Stoltzfus]

I hope that some of this brainwork of identifying problems makes it into the proposal (right now its mostly hand-waving, but we need to show that we have thought about some things, as Matt noted earlier). 

Arlin

--
 
 

[Cody Hinchliff]

Do you think that the working group should commit to an approach based on web services (understanding that this is just one [large] slice of the pie), or should also pursue a strategy for aggregating back-end name-banks?  If we can put them all in DWC-A, then translate that into RDF, they all can go into a triplestore.  But I suppose there will be complications.  

Well, my general feeling is that data aggregation would be a key component of any approach that aims to integrate the various available resources (e.g. namebanks) and make this information accessible within a single framework.

I see taxonomic web services, such as TNRS (I am excluding other services like map of life and otol here) as one means to access the aggregate data, the other primary method being direct access to the aggregated data bank itself (this for the use of client services like mol, otol, etc.). Regarding TNRS specifically, this seems like a necessary side-effect of successful data aggregation. The integration of other services (map of life, phylotastic, otol, etc) on the basis of taxonomic name/concept would also become a trivial problem if all the clients used the same underlying data source, which would only be possible if this aggregated data source were available in an accessible (presumably standardized?) format that was flexible enough to suit the needs of diverse clients.

Obviously these are just my thoughts. I'm sure others with more experience can add expand on or contrast this with other ideas.

[David Patterson]

Experience leads me to favor option (b)

Paddy

[Robert Guralnick]

  I may not have the experience but I also opt for option B and think this gives us a very clear aim for a deliverable.  I have been thinking on deliverables and one worry is that a white paper after 1.5+ years plus of a meeting with all the needs, now now now, for solutions might be seen as yet more time spent without something tangible that helps us move forward.   That seems like a long time.Given Gaurav's semester long NESCent fellowship to work on what amounts to same problem, why not commit to more than a white paper, and try to at least represent some data in RDF format starting with DwC terms and determining gaps, problems, inconsistencies, and new issues that arise from there.   I understand that there is particularly a need for ontologies to capture more of the complexity here, but we need to use DwC terms as a starting point, yes?  Could we work both in tandem?

 I think circumscribing our role as dealing with the problems and challenges just in this domain of standardizing across datasets and working on the knowledge representations issues with taxon names makes a lot of sense.  It may be simultaneously possible to find some other funds (taxon resolution is key in Map of Life and we have resourced it in our renewal - pending) to build a prototpye TNRS workbench that leverages a backend triplestore and begin testing that against use cases etc.  

Just some thoughts.  I will try to dig into the proposal ASAP.  If we can decide to _limit_ our scope to just the assembly end, with some commitments to build on top, I think we can do some very cool things.

-r

--

[Matt Yoder]

I've pitched an alternate set of deliverables in the google doc. I
tend to agree with Rob that we should commit to something more
substantial. Essentially we create a sandbox, a set of rules for
playing in it, 1-2 demo tools for using it, and a white paper
describing the theory behind it's use. Don't get caught up on the
details of the sandbox. Think store documents and triples, that's it.
Everything else is annotations, which link/group data, and APIs to
pull annotated data out. We make the steps to populating and
extracting data as trivial as possible. We describe a modular
approach that let's anyone explore the development of APIs on top of
annotated data.

Comments welcome.

M

> --
>
>

[Shorthouse, David]

I like it! I see two things emerging from this:

1. Heavy use of GNRD to extract the "document" into a presentable form
(= might have to be careful about what kinds of documents are
initially ready for action)
2. A UI to enable, refine semantic mark-up, results of which get
dumped into the store. User need not be aware of the guts
3. APIs to pull-out as other value-added renditions

#2 might be possible with something similar to my cutesy jQuery
plug-in (note example 2 and the numerous callback functions that can
be used to capture user refinements):
http://biblio.globalnames.org/plugins/grabtag

- DPS

> --
>
>

[Matt Yoder]

Completely concur. We have many little pieces worked out here and
there including some brilliant interfaces like your annotator work.
We know we can also build software tools. We need legitimate help
from NESCent and each other to tease out the details of the store and
how to deal with the provenance of annotations (I'd suggest these
issues likely subsume the first 2 meetings). There are many other
groups focusing on other issues (the Euler group in particular would
love to play in this playground from a hard-core logic playground). I
have relatively little confidence in the reconcile in real time across
multiple databases approach- it didn't work for GBIF, it won't work
for names. I think we're also coming to the point where big datasets
of names are "congealing", and we need annotations on these as much as
we need to add the remaining 30%. Among other things we also know
that there will be many sets of names that will be created and not
further curated, a store like this let's them sit somewhere with the
hope that somebody/some tool dives in and looks at them.

M

[Matt Yoder]

Re 2- the opencontext folks demoed a really slick annotator for csv
data, we could get their small team on board as consultants. BTW-
they also have about as interesting demos of linked data that I've
seen, i.e. more than just determining that there are 8 endpoints all
presenting the same biological names and little else different.

M

On Tue, Nov 27, 2012 at 9:20 PM, Shorthouse, David
<david.sh...@umontreal.ca> wrote:

[Cynthia Parr]

I think the proposal is shaping up nicely -- needs smoothing to word
the deliverables a little more concretely. But I like the way it is
going. I've added some examples from Encyclopedia of Life.

It might need to be clarified in the proposal (if it isn't already)
that we recognize that the backends and resolution approaches taken by
individual projects are not the main topic for discussion. Rather, the
workshop's laser focus will be to leverage all the efforts towards
live use cases. User demands will feed back to individual projects
allowing each of them to improve.

> --
>
>