Why isn't this observation Research-grade?

[Mikael Behrens]

Howdy,

I was wondering why this observation isn't considered Research grade:

http://www.inaturalist.org/observations/245002

I sure do enjoy the site -- thanks so much for making a supporting it!

Mikael Behrens

[Scott Loarie]

iNat currently counts stho002's family level ID as a disagree from hkmoths & your ID species level ID.

Clearly not ideal, but its ambiguous whether stho002 is saying 'I know this is in Sphingidae but I have no input beyond that' or 'This is not Manduca quinquemaculatus, its some other Sphingidae'.  Difficult to think how to improve this without making IDing unduly complex.

Mikael Behrens

--
You received this message because you are subscribed to the Google Groups "iNaturalist" group.
To unsubscribe from this group and stop receiving emails from it, send an email to inaturalist...@googlegroups.com.
To post to this group, send email to inatu...@googlegroups.com.
Visit this group at http://groups.google.com/group/inaturalist?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
--------------------------------------------------
Scott R. Loarie
www.stanford.edu/~loarie
Email: loa...@stanford.edu
Phone: 415-278-1220

Post-Doc, Dept. Global Ecology
Carnegie Institution for Science
260 Panama Street
Stanford, CA 94305

Lecturer, Dept. Geography
561 McCone Hall
University of California
Berkeley, CA 94720

Co-director, iNaturalist.org
California Academy of Sciences
55 Music Concourse Dr
San Francisco, CA 94118
--------------------------------------------------

[Mikael Behrens]

That makes sense -- thanks!

Mikael

[Ry Beaver]

So in a strange way if you are contributing a lower order ID (ie Kingdom or Phylum) you may be preventing an observation becoming research grade?  Should we go back and remove the lower level ID if others have been able to provide genus and species with agreement?

 

I will often try and go through the many observations that come up as "Something" often generated by the mobile apps - especially if they are outside Australia I may not know more than Kingdom but hope to try and help them on their way to a full ID?

[Charlie Hohn]

I come up against that 'lower ID' thing too, sometimes I tag plants to family or even a higher level because it helps keep track of the records, and helps others find them.  But, if I forget to untag it it can be harmful to getting the observation to research grade.  I think one way to do it might be to allow a measure of confidence in ID suggestions (IE: if I am confident it should affect research grade but if not sure it would just tag without)... but that might make the system unduly complicated.  Often when I am not sure I just suggest ID in comments but that isn't ideal either.

[Scott Loarie]

Hi Ken,

Yes - everything on iNaturalist is accessible to researchers/scientists.

Research grade data means that:

1) if its licensed the data is harvested by external websites (so far GBIF, EOL & Calflora participating, see External Links on lower right e.g. http://www.inaturalist.org/observations/8734)

2) observations automatically update checklists, 

e.g. the Pepperwood amphibian checklist http://www.inaturalist.org/places/pepperwood-preserve#taxon=20978 has 13 species and Hellbender is not on it. But a research grade observation of Hellbender from within the park would update this checklist with a 14th species sourced to that observation.

I see three choices with this functionality, using the example where Ken and I say Taricha torosa but Ken-ichi says Taricha:

1) leave it as is (e.g. 1 agree and 1 disagree so not research grade)

2) make coarser IDs not count against consensus  (e.g. 1 agree and 0 disagrees so research grade - this would make research grade less strict)

3) build something more sophisticated - $$ or volunteer skills anyone?

-Scott

On Thu, May 16, 2013 at 3:23 PM, <kschn...@gmail.com> wrote:

Interesting.  This seems to me like it COULD be a serious problem, depending on how important the "research grade" designation is...  Can someone explain what happens to data that is not "research grade"?  Can it still be accessed by researchers/scientists looking to mine the data here?  If not, then it makes me not want to help with any ID's unless I know the actual species - otherwise, I may inhibit the ID process later on, right?

 

Ken 

[Franco Folini]

I agree with point #2: make coarser IDs not count against consensus

The system should need a minimum of 2 ID to agree in order to reach "research grade", the coarser ID counting 0 in both directions.

-- Franco
_________________________________________
Franco Folini 

San Francisco, CA 94114 

_________________________________________
Franco Folini
w: fol...@novedge.com | h: fol...@gmail.com

m: (415) 420 8748 | w: (415) 848 9018 x 708

[Ken-ichi]

Anyone can access data, regardless of quality grade. We use the
research grade status to determine what to share with other
repositories, like GBIF and Calflora.

Your IDs don't just help with data quality. They also help real people
learn about nature. So I personally wouldn't stop ID'ing just because
there was some impact on our weird data quality assessment. People are
more important than data.

All that being said, yes, this is a problem. The solution we've been
discussing is to have a separate concept of a "community
identification" that we use for establishing "research grade" instead
of using the owner's identification. So if I observe a cat but 10
people think it's a dog, my ID stays cat, and cat gets added to my
life list, but the community ID is dog, and the observation is
research grade as dog. This gives the community a bit more power of
the data (which is sort of communal in nature once people start adding
IDs), but it preserves the observer's ownership over the parts of the
identity that affect them. This would also be a good way around data
that people add and then ignore, even though lots of people have added
conflicting IDs.

Thoughts? Obviously that's a pretty involved change to how we do the
quality assessment, which is why it hasn't really happened yet.

On Thu, May 16, 2013 at 3:23 PM, <kschn...@gmail.com> wrote:
> Interesting. This seems to me like it COULD be a serious problem, depending
> on how important the "research grade" designation is... Can someone explain
> what happens to data that is not "research grade"? Can it still be accessed
> by researchers/scientists looking to mine the data here? If not, then it
> makes me not want to help with any ID's unless I know the actual species -
> otherwise, I may inhibit the ID process later on, right?
>
> Ken
>
>
>
> On Tuesday, May 14, 2013 8:18:14 PM UTC-7, Mikael Behrens wrote:
>>

[Lynn Watson]

There is one drawback to community identification,  in that common species are better known, such as Blue-eyed Grass and California Poppy, and other species are less well known, eg. Brodiaea terrestris kernensis or Eriodictyon traskiae smithii. Hence, the possibility of rarer species lagging behind in research grade and sometimes not even getting that grade even though the observer's id is correct and having that observation as research grade would be valuable. Frequently the species has been identified by a well-regarded and well-known botanist in the field when the observation was made, but the species, such as Eriodictyon traskiae smithii,  is unknown by any community member

[Ken-ichi]

Lynn, that's a valid and interesting point, but the only way I can
imagine that working is to build a reputation system, which is another
thing we've talked about doing but haven't had the time to experiment
with. Basically we could come up with some algorithm that scores you
on your ability to identify things based on your past identifications.
So if you identified 10 observations as Western blue-eyed grass and
they were all confirmed maybe we'd give you a score of 10. If you
added 10 identifications of Western blue-eyed grass but for each of
them the community ID was something else, maybe your score would be
-10. We could use these scores to say something like, "One ID from a
user with a score of 10 makes this research grade without the need for
confirmation." Scores would probably have to be taxonomically and
geographically specific. Again, haven't thought this through enough,
but I think that would be the direction we'd go if we were to do that.

[Charlie Hohn]

Just a few thoughts, Ken-Ichi, I like your idea about community consensus on ID.  I think the biggest case of it being an issue is with lapsed or fled users or those who don't review old observations.  In most cases if I disagreed with an ID and the other user didn't think my suggestion was right, there was some great discussion of identification.  However, if someone left the site after uploading an incorrectly-identified observation, it's hard to get rid of it.  For instance there are a few east coast tree species tagged to California by someone who probably used a tree guide from the entire US to try to make an ID.  It annoys me to see them on the species distribution maps, because I am fussy like that.  But as non-research grade it does display differently on some of the maps, which is great.

I would love to see a reputation system, for instance there are some excellent experts in Vermont using iNat, most experts "know when they don't know" so their IDs are valuable in many ways.  Though a system that favored 'professionals' over some of the very expert 'amateurs' would also be a disservice.  I think your suggestion would be a neat thing to try some time, if you get time and funding to set it up.  I also think it would be neat 'someday' to have a little value score on observations based on distance from other observations - IE a white pine observation in a new county is more valuable than one a half mile from another existing one (though both are still valuable)

C

[AfriBats]

Hi everyone - I hadn't realized the issue yet and agree that this is quite important to resolve.

What about factoring in the temporal dimension of the identification process: research grade won't be lost as long as the next user identifies it as the same or higher taxonomic level. For instance, if initial ID is insects, next ID is butterflies, and so on and so forth, research grade is achieved. However, if something was IDed at a certain level, let's say genus, and the subsequent ID is given at family level, research grade would be lost, because this is likely to indicate that the subsequent identifier disagrees with the previous one (not necessarily, but very likely).

Cheers, Jakob

[Lynn Watson]

Or I can take a video of the expert in the field identifying the plant, load it to Flickr and link to it in the obs description. :-)

It was one of the reasons I added links for the official Burton Mesa plant list for observations of lesser known plants, not that it helped - they are still without "agrees". Official plant lists can be helpful is the genus is recognized by another person, and there is only ONE species of that genus in that area - such as the potentilla silverleaf at Oso Flaco Lake. Which reminds me, I should add the official Oso Flaco plant list link to that obs.

L. 

[Mikael Behrens]

Thanks for all the ideas and discussion!

How about this observation?

http://www.inaturalist.org/observations/280471

Why *is* it Research Grade? I would expect that the single Genus Cleome ID would stop it from being Research Grade.

Mikael Behrens

[Charlie Hohn]

I think because the one ID that does not agree is counteracted by the many IDs that do agree.  I believe that always happens unless the one minority ID is that of the observer in which case it doesn't get research grade,