original author: 'our stimuli are not publicly available' --- what to do?

[Andy Woods]

Dear all,

we'd appreciate some advice. The first author of a paper we'd like to try and replicate states that their original stimuli are not available publicly (after ignoring my first email, and replying to a second email from a collaborator).  Sadly, their stimuli took many man hours to produce (497 men rated 1127 images of women on several attributes).  

For shear obstinacy (and frustration with this author) I'd go ahead and cloud source these stimuli and get them rated appropriately.  If we have to resort to this, I hope I can ask for everyone's help :)

Before resorting to this, can anyone offer any advice on this situation?

Thanks,
Andy.

PS Apologies for terseness... crying baby I must go and rescue...

Extract from first email from author "The stimulus set for that manuscript was developed and normed by a colleague of mine. They are not publically available at this time."

Extract frim reply "Could we contact your colleague who made the stimuli please. Can we have their email address?"

Extract from second email "As I mentioned, the stimuli are not available for release."

[Jeremy Gray]

Here's an idea: I would contact them again first, and say you understand their concerns and respect the immense effort that went into developing the stimulus set (including the ratings). Point out that a replication of their findings with exactly those same stimuli would actually add quite a lot of value to their stimulus set -- it would not detract from it in any way. Then make it clear that you are not seeking to obtain a copy for any other purpose except for replication. So they would not be making them "publicly available" -- they would be releasing them for a very narrow usage purpose to a specific individual. And you'll be happy to destroy your copy at the end of the project, and will not make copied available to any other persons. And be willing to formally state this in writing, and sign and date.

And perhaps also point out that APA guidelines specifically require published authors (in APA journals, but maybe also APA members etc) to release materials necessary to be able to conduct a replication. Not doing so can become public. Threats can backfire and make people dig in their heels more, so its a balancing act to get it right.

--
You received this message because you are subscribed to the Google Groups "Open Science Framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openscienceframe...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Ben Blohowiak]

Giving the researchers the benefit of the doubt, there may be (to them, anyway) very good reasons not to disclose the stimulus set. Maybe those reasons could get disclosed so folks aren't left speculating. 

Then again, perhaps discussing the reasons against stimulus set disclosure could open a dialogue that addresses whatever concerns there are such that the set may be shared for the purposes of replication. Otherwise, the replication may not happen and whatever findings or research techniques the researchers may feel protective of now could be rendered less robust in the esteem of the community; a scientist shines in as much as she is transparent.

The fact that there are multiple parties involved does add a layer of complexity; maybe a phone call or video chat could work better than email in terms of establishing contact and building rapport. It sounds like if you were able to more spontaneously and openly interact, you might be able to build the trust required to gain, if not access to the information you want, a clearer sense of where your interests converge and diverge. 

[Fred Hasselman]

It woud be nice to have a kind of consensus statement drafted by the open science movement one could use as a template in these matters (this would include data sharing requests)

I would certainly welcome an example I could use to politely point (by hyperlink) to formal rules by different societies: APA, APS, perhaps funding agencies and codes of conduct drafted by national academies of sciences. 

I imagine in the case of denial of the request, one could request the original authors to indicate which out of a number of (valid) reasons was the cause to deny access to stimuli or data. Such an answer could be reported on OSF or some other medium (f.i. Where the replication was preregistered), eventually in an article of course.

More people see the benefits of such a template?

Fred

Sent from my iPad

[Bob Ryan]

I agree. And even in a template for a request, not only should the request politely point to the formal rules, but also to the valid reasons, so that the request implied that if it was denied, then those reasons would have to be cited, rather than making a second request for the reasons.

--- Bob

==========================================
Robert S. Ryan, Ph.D.
Associate Professor of Psychology
385 Old Main
Box 730 - Psychology Department
Kutztown University
Kutztown, PA 19530

Voice: 484-646-4325
Fax: 610-683-4467
office email: rr...@kutztown.edu
URL: http://faculty.kutztown.edu/rryan

Home:
214 Taft Ave.
Reading, PA 19605-2154
610-698-9108
cogpro...@gmail.com

=== So often times it happens that we live our lives in chains ===
=== And we never even know we have the key ===
=== (from "I'm Already Gone" by The Eagles) ===

[Fred Hasselman]

but also to the valid reasons, so that the request implied that if it was denied, then those reasons would have to be cited, rather than making a second request for the reasons.

Yes, good point, that would be better.

Here' some policies summarised on data sharing, some interesting links

http://en.wikipedia.org/wiki/Data_sharing

APA's page on the subject:

http://www.apa.org/research/responsible/data/index.aspx

Interestingly, while searching on sharing materials this popped up, Nature has special rules for life sciences!

http://www.nature.com/authors/policies/availability.html

Reading the section on requirements for making available of biological / chemical materials, 

I am starting to wonder what the valid reasons in psychology might be to not share stimulus materials? (Aside from the occasioinal "whoops I pressed delete", or "but that was 20 years ago")

Sharing materials

A condition of publication in a Nature journal is that authors are required to make materials, data and associated protocols promptly available to others without undue qualifications.

It is acceptable to request reasonable payment to cover costs of distribution and reagents may be made available via commercial or non-commercial third party providers.

Authors reporting new chemical compounds integral to the conclusions of the paper must provide the chemical structure, synthesis and characterization of the compounds with sufficient experimental details to allow other researchers to reproduce the synthesis and characterization.

For biological materials such as mutant strains and cell lines, the Nature journals require authors to use established public repositories when one exists (for example, Jackson Laboratory, the European Mouse Mutant Archive (EMMA), the European Conditional Mouse Mutagenesis Program (EUCOMM), the Knockout Mouse Project (KOMP), Addgene, RIKEN Bioresource Centre, the Mutant Mouse Regional Resource Centers, American Type Culture Collection (Americas), American Type Culture Collection (Asia/Europe), UK Stem Cell Bank), and provide accession numbers in the manuscript.

Cell lines 

The distribution of human cell lines used in research should not be hindered by restrictions from donors. Researchers developing cell lines must investigate and disclose any restrictions associated with the human or other tissue they are using, particularly if someone else collected the samples, if the samples come from multiple clinical sources or if they come from several legal jurisdictions. If a scientist needs to create cell lines that might be used for as-yet-unforeseen purposes, only tissue with no restrictions should be used. Authors of papers that involve consent forms must, at time of submitting the manuscript, make editors aware of any limits that result from those forms. See theNature Editorial 'Common consent', which is listed at the foot of this page, for further explanation.

http://www.nature.com/authors/policies/availability.html

Sent from my iPad