Pace of peer review process

[Ben Blohowiak]

I'm looking for four pieces of information.

1) What seems to be an "average" time between an initial submission and an action letter?

2) What might be an ideal (yet realistic) time frame for such an exchange?

3) What seems to be an "average" time between an acceptance and publication?

4) What might be an ideal (yet realistic) time frame for that lag? 

[Bryan Burnham]

This is something that could be examined fairly easy using a Google spreadsheet or another means for "crowd sourcing" to collect the data necessary to answer your questions. Specially, set up an online spreadsheet where authors could provide, for each of their published manuscripts, the following pieces of data:

1) date of initial submission in MM/DD/YYYY format

2) date of initial action letter in MM/DD/YYYY format

3) date of resubmissions in MM/DD/YYYY format

4) date of acceptance in MM/DD/YYYY format

Once these data are submitted, analyzing them using spss, R, or another software would be relative easy and would answer the questions you posed.

Authors could also include the number or revisions necessary before a paper was accepted for publication and could also provide their subjective/preferred number of days between an initial submission and an inital action and between initial submission and publication. Authors could also provide the names of the journals they submitted to, so you could examine differences in publication time and action time across sub-fields. 

The reason I suggest this approach is that my university's curriculum committee, which I am part of, recently started examining the lag time between the submission of a curriculum review proposal and final approval of the proposal. We decided to divy up the work and we have been posting our data to a Google spreadsheet, which has made the overall task quite easy. If you are interested, I'd be happy to begin settling something like this up.

Best,

Bryan

--
Bryan R. Burnham, PhD
Associate Professor of Psychology
University of Scranton
Scranton, PA 18510

office: 570-941-6687

Twitter: @DrShaggy

[Shauna Gordon-McKeon]

This is (I believe) for the "Journal Best Practices" OSF project that Ben and I are both working on.  It seems likely that we'll want input from a sizable sample of researchers for a number of different standards/issues, so we may hold off on forwarding around a set of questions until we've figured out everything we want to ask.

That said, your solution of a google doc is simple and nice.  If you'd like to set that up so that we can add to it as necessary, that would be great.

- Shauna

[Jeremy Gray]

some journals make some of this info available, at least in summary form. e.g., http://www.apa.org/pubs/journals/statistics.aspx

--Jeremy

[Ben Blohowiak]

Helpful feedback, thanks! :)

[sheila miguez]

If you set up a google doc, it's helpful to use a google form to have
people input the data. Let me know if I can help.

Here is an example that I've set up with Bryan's questions. I've made
the backing spreadsheet world readable, and gave form edit permission
to Ben. It's low tech since it does no input validation, hence people
are not constrained to actually enter dates. I've entered a lorem
ipsum as an example.

<https://docs.google.com/spreadsheet/viewform?formkey=dDdEYjZmMklSMDNVdFlwRm12RnN1Wmc6MQ#gid=0>

The resulting spreadsheet.
<https://docs.google.com/spreadsheet/ccc?key=0AsfwY_l0aoazdDdEYjZmMklSMDNVdFlwRm12RnN1Wmc>

On Wed, Jan 9, 2013 at 7:30 PM, Shauna Gordon-McKeon <shau...@gmail.com> wrote:
>
> That said, your solution of a google doc is simple and nice. If you'd like
> to set that up so that we can add to it as necessary, that would be great.

[...]



--
sheila

[sheila miguez]

I suggest adding some extra questions, such as adding a required field for a doi for the article.

I also suggest finding a survey site that does input validation for dates so that performing the date arithmetic to get the durations isn't impeded by user error.

But, I don't want the perfect to be the enemy of the good enough and google docs might be good enough.

[Ben Blohowiak]

@ Bryan, Jeremy, & Sheila; these contributions seem helpful in getting closer to establishing what is or has been, and I can envision continuing like this to clarify what's normative or actually happening on the ground. It may also make sense to categorize the entries by journal, and the journals by discipline so as to facilitate any future taxonomic differentiation. 

Any ideas on how to establish "ideal" timelines or how they might be judged? The best I've come up with thus far is merely the notion that a journal might volunteer its own ideal turnaround times as part of its editorial policies and then authors could report the extent to which actual practices corresponded to those declared policies.

[sheila miguez]

The doi might satisfy this? It is a compact way to collect the
article's information including authors and journal for the entry. You
wouldn't have to depend on the user inputting multiple fields
correctly (and in parsable form), only one.

On Thu, Jan 10, 2013 at 10:05 AM, Ben Blohowiak <ben.bl...@gmail.com> wrote:
[...]

> It may also make sense to categorize the entries by journal, and the
> journals by discipline so as to facilitate any future taxonomic
> differentiation.

[...]



--
sheila

[Bryan Burnham]

@ Sheila; I like the Google form idea, as this would make it much easier for authors to enter information. I also agree that using Google forms has the drawback of assuming respondents will enter the MM/DD/YYYY, or other, information accurately. But, in my experience and several colleagues' experience using Google forms to collect data, these instances are pretty rare.

As for other questions, maybe some subjective rating scales asking authors their "ideal" turn around time between submission and initial action. You could have an n-point "Likert-type scale" with each of the n-levels a difference number of weeks.

Another thought on information to collect: usually, when a manuscript is submitted and an email is generated the author is told approximately how much time it should take for them to hear back from the action editor. This might be interesting to compare to the actual turn around times.

@ Ben; I think your suggestion about establishing the "ideal" timeline is probably the best first step. Indeed, some journal websites  list their approximate turn around times so this should be easy to establish. Like you suggest, once these expected turn around times are identified you could compare them to the actual turn around times as reported by authors.

Another factor to consider is the journal to which you submit a manuscript and the length of the manuscript, as both of these could influence an author's "ideal" turn around time.

[Bobbie]

A big factor to consider is the length of paper and type of paper.  When I send a short paper to reviewers I am much more likely to get reviews back quickly than if I send them a long paper.  (And much more likely to get reviewers to agree.)  And, of course, once the reviews come back, on average it's easier to write an action letter for a short paper. Also, I suspect that empirical papers have shorter turn-arounds than theory/review papers -- but I have no solid data for that suspicion.

The time between acceptance and publication ("publication lag" in editorial speak) is something no one likes.  But here an important factor is how often the journal is published.  Paper journals published quarterly are different from those published monthly, of course.  But even with paper journals, some are starting to put up online versions of accepted papers before the whole issue is printed.

[Roger Giner-Sorolla]

What's emerged recently in an interesting PLOS ONE paper is the perverse incentive to have longer lags to (print) publication after online publication, because this inflates the citation stats.

http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053374

Clearly what this means is that impact metrics should start the clock from the point of online publication - another small step toward legitimizing online delivery ...