Weird idea after reading Research Revolution 2.0 blogpost
fastch...@gmail.com
I had a thought, and subsequent wondering after reading this blogpost concerning things like linking, combining, collaboration, etc. http://morepops.wordpress.com/2013/05/27/research-revolution-2-0-the-current-crisis-how-technology-got-us-into-this-mess-and-how-technology-will-help-us-out/
That got me thinking. So, possible problems in science could be things like:
a) underpowered studies, undisclosed flexibility in data-analysis, etc.
b) less than optimal amount of replications
c) diffusion of information
d) file drawer problem
e) less than optimal theory testing
f) wasted resources (resulting from all of the above)
I wondered if
these things could be partly resolved by setting up a model/ an experiment in
conducting research, involving a few collaborating labs/researchers, all
working on the same general problem/ topic but perhaps doing things slightly
different than how they are doing things now.
Take post-traumatic stress disorder (PTSD) for instance as a general topic of interest, and zoom in on a more specific part of research related to this. You could set up a model/ experiment involving findings and researchers concerning this topic, and for instance have 5 teams of scientist collaborate in the following manner.
1) have them make, for instance, a top 5 of studies/findings they view as interesting/useful to replicate (possibly help solving b)
2) each of the 5 teams would replicate all 5 studies/findings (possibly help solving b, c, d)
3) they would do this in an “optimal manner”: high powered, pre-registered at 1 place, etc. (possibly help solving a, c, d)
4) they would investigate the results of all these findings, and then each team would come up with a follow up study (possibly help solving e)
5) repeat from step 2 onwards (possibly help solving f)
You could have a special issue of some journal publish the results of this model/ experiment after each "round" is complete. Other teams could join in later if they want to. Has this ever been done like this, and could such a model be useful, and/or interesting to investigate, and/or participate in ?
Brian Nosek
--
You received this message because you are subscribed to the Google Groups "Open Science Framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openscienceframe...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
fastch...@gmail.com
1) Does your prospective replication project then also relate to a single, general topic/ theory or will the findings/effects to be prospectively reproduced not necessarily be related to eachother ?
2) Could the format like described in the 1st post be interesting and/or useful to pursue by teams of scientists ? And if yes, would that be something they could try and set up with help of the COS, perhaps also with regard to (part) funding, because it could perhaps also be seen as some sort of "general format-project-idea" which could enhance openness, integrity, and reproducibility ? (and please stop me if I am being to forward or bold or talk rubbish :P)
Anyway, it was just a thought I had and I was curious whether this could be seen as interesting in some way or form. Thanks again for the reply! I am also still very curious what researchers involved in PTSD-related (or other psychological disorder-related) research, and/or editors from journals publishing that type of research, and/or funders think of such a format.
Regards.
fastchanges 77
-Given the fact that this would require some resources, I wondered if such a format (if deemed interesting in the first place) would especially be applicable to research concerning something "worthwhile" (if such a thing exists of course, but that explains the psychological disorder/ PTSD connection)
-I also wondered if it would be interesting to compare the results/findings after each "round" and if that would possibly provide interesting information. I reason that maybe such a format would result in slowly getting closer to having optimal/accurate/useful information. Combined with collaboration, and theory testing, perhaps the hypotheses of the follow-up studies in later "rounds" could turn out the be confirmed more and/or perhaps the results/findings of studies in later "rounds" could turn out to be more replicable, and/or less lab-resources would be wasted (no file-drawer based on possibly less than optimal research), etc. Maybe this could possibly be interesting additional information resulting from such a format, leaving aside the primary findings of the studies themselves.
I'll stop thinking now.