Open Science/Reproducible Research manifesto (but short)?

[Tom Roche]

(Apologies if this is a FAQ, but I'm not seeing an answer @ https://osf.io/faq . I note

https://osf.io/4znzp/wiki/home/
> "Scientific Utopia" articles about opening scientific communication and improving scientific practices.

about which more below.)

Could someone pass a link to a short but convincing OS/RR "elevator pitch" suitable for colleagues who have no clue what "open science" or "reproducible research" are, much less why they would want to use something like OSF? What I mean, why I ask:

I saw your links about Dataverse and wanted to ask a guy I know @ Harvard if he or his group (which oversees a SOTA atmospheric model) know anything about it, or had used it. (It looks good, but I dunno if the Harvard Institute for Quantitative Social Science is ready for many TB of physical-science data, and I'd like to know if someone more like me has tried it.) However, my Harvard contact is not too different from most of my current colleagues (except that his group has a *much* better SDP), and most of my current colleagues (mostly atmospheric scientists, in government and university) have only the vaguest idea of what "open science" or "reprodible research" are, much less why they'd want to do *anything* other than pump out as many papers as they can in the shortest possible time. (And, yes, the folks in my federal dept are just as literally "publish-or-perish" as the folks in my academic dept.)

My current colleagues get "open access" (largely because OA publication is becoming mandatory) and "open source" (although they have not yet "drunk the kool-aid" the way most of my software-development colleagues had, when I was coding for a living), but

* (IMHO) OA and OSS are conceptually distinct from the sorts of concerns one would address by using a tool like OSF. (The latter are what *I* mean by OS/RR--YMMV.)

* many of my colleagues really don't (IMHO) get OS/RR. E.g., when I was discussing reproducibility with my first academic adviser, and the greater need I perceive for RR WRT computational science with policy implications (as in, we're asking polities to make multi-G$ investments to prevent harms forecast by our models), the guy literally said (and this is damn near a quote) "they just hafta trust us ... we're scientists." Note that this guy is young (if 30, not much more) and has a PhD from a brand-name/well-respected university--not some hack or dinosaur. (He is also my ex-academic adviser.)

Hence I'd like to be able to send colleagues email from a template like (feel free to improve the wording, or the approach more generally)

> I'm trying to make <link to my project/> be open, reproducible science. (Why? <link to manifesto/>) So <whatever I seek help with/>

Sooo ... where's the manifesto link? How do you pitch OS/RR to folks who would be daunted by, e.g., links to Scientific Utopia 1 & 2? I'm looking for something

1. shorter than http://en.wikipedia.org/wiki/Open_science and which better disaggregates OS/RR from OA

2. about the same length as https://en.wikipedia.org/wiki/Open_research but which better disaggregates OS/RR from OSS

3. "just better" than

https://en.wikipedia.org/wiki/Reproducible_research
https://en.wikipedia.org/wiki/Open_research_computation#Reproducibility_and_open_research_computing

4. "just evangelistic enough"

5. does not contain the term "utopia." YMMV, but when I see "utopia," I tend to think "bleeding edge," not "I can *do* this!"

An OS/RR "elevator pitch" also seems natural for http://centerforopenscience.org/about/ but YMMV.

TIA, Tom Roche <Tom_...@pobox.com>

[Ben Blohowiak]

@Tom: How about the first four sentences of the first paragraph in the "About" section of the Badges to Acknowledge Open Practices project?

"Openness is a core value of scientific practice. There is no central authority determining the validity of scientific claims. Accumulation of scientific knowledge proceeds via open communication with the community. Sharing evidence for scientific claims facilitates critique, extension, and application."

Best,

Ben

--
You received this message because you are subscribed to the Google Groups "Open Science Framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openscienceframe...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Tom Roche]

https://groups.google.com/d/msg/openscienceframework/Wn4zslAhslc/8zcQJ9tpF34J
>> [Tom Roche Tue, 21 Jan 2014 14:43:19 -0500]

>> How do you pitch OS/RR to folks who would be daunted by, e.g., links to Scientific Utopia 1 & 2? I'm looking for something

>> 1. shorter than http://en.wikipedia.org/wiki/Open_science and which better disaggregates OS/RR from OA

>> 2. about the same length as https://en.wikipedia.org/wiki/Open_research but which better disaggregates OS/RR from OSS

>> 3. "just better" than

>> https://en.wikipedia.org/wiki/Reproducible_research
>> https://en.wikipedia.org/wiki/Open_research_computation#Reproducibility_and_open_research_computing

>> 4. "just evangelistic enough"

>> 5. does not contain the term "utopia." YMMV, but when I see "utopia," I tend to think "bleeding edge," not "I can *do* this!"

>> An OS/RR "elevator pitch" also seems natural for http://centerforopenscience.org/about/ but YMMV.

https://groups.google.com/d/msg/openscienceframework/Wn4zslAhslc/ezDS7rY3oe8J
> [Ben Blohowiak Tue, 21 Jan 2014 15:01:07 -0500]

> How about the first four sentences of the first paragraph in the "About" section of the Badges to Acknowledge Open Practices

gotta get me somma them :-)

http://openscienceframework.org/project/TVyXZ/wiki/home
>>> [S1] Openness is a core value of scientific practice.

... and has been since ~1660 ( https://en.wikipedia.org/wiki/Royal_Society#Founding_and_early_years )

>>> [S2] There is no central authority determining the validity of scientific claims.

check! (might shock some worshipers at the Church of Peer Review :-)

>>> [S3] Accumulation of scientific knowledge proceeds via open communication with the community.

check, though I'd ditch the passive voice and make more of an optimality claim. Hell, just skip sentence 3 and go straight to sentence 4:

>>> [S4] Sharing evidence for scientific claims facilitates critique, extension, and application.

Pretty good! but

1. Nothing (per se) about reproducibility? IMHO reproducibility is a key (if unfortunately not currently core) practice: knowledge is justified belief, and reproduction of an empirical claim increases its justification. (Or "validation," Popperians be damned :-)

2. Should have links to "for more information," esp for the last sentence, which is itself both empirical and normative.

That being said, I'd definitely put sentences 1,2,4 on http://centerforopenscience.org/ (i.e., the "home page"), were I so empowered. For now, how 'bout putting that someplace outside the badging context?

YMMV, Tom Roche <Tom_...@pobox.com>

[Tom Roche]

http://openscienceframework.org/project/TVyXZ/wiki/home
>>> [S1] Openness is a core value of scientific practice.
>>> [S2] There is no central authority determining the validity of scientific claims.
>>> [S3] Accumulation of scientific knowledge proceeds via open communication with the community.
>>> [S4] Sharing evidence for scientific claims facilitates critique, extension, and application.

https://groups.google.com/d/msg/openscienceframework/Wn4zslAhslc/7nbN2fdoKoMJ
>> [Tom Roche Tue, 21 Jan 2014 16:05:14 -0500]
...

>> Pretty good! but

>> 1. Nothing (per se) about reproducibility? IMHO reproducibility is a key (if unfortunately not currently core) practice: knowledge is justified belief, and reproduction of an empirical claim increases its justification. (Or "validation," Popperians be damned :-)

>> 2. Should have links to "for more information," esp for the last sentence, which is itself both empirical and normative.

>> That being said, I'd definitely put sentences 1,2,4 on http://centerforopenscience.org/ (i.e., the "home page"), were I so empowered.

http://dx.doi.org/10.1177/1745691612462588 (via Brian Nosek Tue, 21 Jan 2014 16:33:17 -0500)
> Reproducibility--the extent to which consistent results are observed
> when scientific studies are repeated--is one of science's defining
> features (Bacon, 1267/1859; Jasny, Chin, Chong, & Vignieri, 2011;
> Kuhn, 1962; Popper, 1934/1992; Rosenthal, 1991),[2] and has even
> been described as the "demarcation criterion between science and
> nonscience" (Braude, 1979, p. 2).

+1! And the openness and frequency of reproduction (not merely reproducibility--the act, not the potential) was a major distinction between early modern science and, e.g., alchemy, probably leading to the ascent of the former. (But I digress :-)

> In principle, the entire body of scientific evidence could be
> reproduced independently by researchers following the original
> methods and drawing from insights gleaned by prior investigators.

and it's worth remembering that (IIRC :-) both Royal Societies (British and French) started not with publication, but with show'n'tell--publication came later, when groups and investigators communicated (e.g., "last night, Foo showed us his new bar").

> Other types of belief depend on the authority and motivations of the
> source; beliefs in science [should] not.

+1! FWIW, here's my short synthesis of the badging (author unknown to me) and Nosek texts:

Openness is a core value of scientific practice. No central authority validates scientific claims, and a claim that depends on its source's authority is not scientific. The extent to which consistent results are observed when well-defined procedures are repeated--reproducibility--demarcates science and nonscience [link to Braude 1979, or just cite if unavailable], thus replication should be a core scientific practice. Sharing detailed methods and results facilitates scientific confidence, education, and innovation.

>> For now, how 'bout putting [something like that] someplace outside the badging context?

YMMV, Tom Roche <Tom_...@pobox.com>

[Tom Roche]

See proposed manifesto implementation in pull request

https://github.com/CenterForOpenScience/centerforopenscience.org/pull/32

HTH, Tom Roche <Tom_...@pobox.com>

[Ben Blohowiak]

Interesting. I thought that you might be less focused on an empirical instantiation of your idea than on, say, assent to the merits of its general principles, but I must say that it seems like addressing the issue does make sense. I'll play with it a little and you can tell me what you think.

[Tom Roche]

https://groups.google.com/d/msg/openscienceframework/Wn4zslAhslc/8zcQJ9tpF34J
>>> [Tom Roche Tue, 21 Jan 2014 14:43:19 -0500]
...
>>> I'd like to be able to send colleagues email from a template like[:]

>>> > I'm trying to make <link to my project/> be open, reproducible science.
>>> > (Why? <link to manifesto/>) So <whatever I seek help with/>

>>> Sooo ... where's the manifesto link?

https://groups.google.com/d/msg/openscienceframework/Wn4zslAhslc/rOLc4dN6XicJ
>> [Tom Roche Thu, 23 Jan 2014 17:12:35 -0500]
>> https://github.com/CenterForOpenScience/centerforopenscience.org/pull/32

https://groups.google.com/d/msg/openscienceframework/Wn4zslAhslc/x1-szLgRsuQJ
> [Ben Blohowiak Thu, 23 Jan 2014 19:54:10 -0500]

> I thought that you might be less focused on an empirical instantiation
> of your idea than on, say, assent to the merits of its general principles,

1. See instantiation requirements above.

2. Regarding "the merits of [the] general principles": I drank that kool-aid while back, external validation not required :-)

> it seems like addressing the issue does make sense.

Given issue="Why Open Science?" and your group's name="Center for Open Science", it seems esp sense-making to address it here!

YMMV, Tom Roche <Tom_...@pobox.com>

[Tom Roche]

Updated pull request

https://github.com/CenterForOpenScience/centerforopenscience.org/pull/32

incorporating input from Ben Blohowiak

https://github.com/TomRoche/centerforopenscience.org/commit/26740a5610252436659316a81dbbdacbc086addc

Note: you can comment the pull request directly via its web UI (tab=Conversation--those GH guys think of everything !-) as well as indirectly via this list.

[Jeffrey Spies]

Let's discuss on GitHub--save the folks on this list the specific cross-talk that should happen there anyway. 

[Tom Roche]

https://groups.google.com/d/msg/openscienceframework/vO56clqDN-E/_4zMFgQlnpAJ
> [Philip Durbin Mon, 27 Jan 2014 11:37:40 -0500]
> curious if there's a working definition of "open science" someone can link me to.

You may find the first paragraph @ http://centerforopenscience.org/about/ useful, though it's worded more normatively (per its purpose--see Subject: above) than empirically:

http://centerforopenscience.org/about/ (see links in original)
>> Why Open Science

>> No central authority validates scientific claims; a claim that depends on its source's authority is not scientific. Rather, the observation of consistent results when well-defined procedures are repeated—reproducibility—demarcates science and nonscience [Braude 2002, p. 33]. Thus replication should be a core scientific practice, and scientists should routinely supply methods of sufficient detail to support replication with their observations. Scientists must also confront the "file-drawer problem" by routinely supplying all studies regarding a claim. "Open Science"—providing sufficiently complete methods and observations—improves public confidence in science, scientific education, and the community's ability to advance prior work. Thus openness should be a core scientific value.

HTH, Tom Roche <Tom_...@pobox.com>