Improving referee reports

[Jonathan Newton]

Dear All,

We should probably consider how refereeing can be improved. 

Speaking for myself (but also for some of you with whom I've previously discussed this topic), the quality of referee reports has decreased massively since Bill's death. Naturally, given the breadth and depth of his knowledge, and how much he cared about every detail, he did a good job picking referees. Now, we don't have this at most of the 8 (by my count) journals at which he was an AE.

I still occasionally receive great (precise, knowledgeable about literature) reports, but these are now a rarity.

Instead, I get many more reports along the lines of:
"This is an important and interesting topic....[reference to some paper from the 1990s]....why doesn't the paper deal with [some well known tangential problem]....".
Clearly these reports are written by very smart people who I'd probably enjoy spending time with, but it seems insufficient for scientific advancement.

This is, of course, partially driven by my selfish interests. I want to hear smart people say interesting things about my work. I do my best to do the same. Christoph once told me that research should be "a conversation" and I strongly agree. It feels like the conversation isn't what it once was.

Does anyone have any solutions?

Always yours,
Jonathan

[Yuval Heller]

Hi Jonathan,

I also share the impression that there was some decrease in the level of referee reports and AE letters since Bill's death.

Sadly, I don't have any good idea of how to solve it.

Best wishes,

Yuval

---

Yuval Heller,

Professor, Department of Economics, Bar-Ilan University.

https://sites.google.com/site/yuval26/

--
You received this message because you are subscribed to the Google Groups "Learning Evolution and Games Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to learning-evolution-and-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/learning-evolution-and-games-forum/c953a948-ab3b-4113-985a-24dfcc3df673n%40googlegroups.com.

[Takako Greve]

Hi Jonathan,

I understand.  Perhaps there is a game (signal-jamming?) going on, and we lost one of the best organizers of the game.

https://www.aeaweb.org/articles?id=10.1257/jep.31.1.231

This LEG community is one way to do that conversation.  Although I may not be able to contribute much, I enjoy reading the exchanges.

(When I get time, maybe I can build a model of a referee process evolution!)

Best wishes,

Takako

2024年8月13日(火) 12:36 Jonathan Newton <jcsn...@gmail.com>:

[LUIS RODRIGO IZQUIERDO MILLAN]

Dear All,

Thank you, Jonathan, for bringing this up. I just wanted to share that we’ve had very similar (and unfortunate) experiences. In the last few years, we have received several referee reports that seem to come from scholars who are not expert on (or, in some cases, even familiar with) Evolutionary Game Theory. In our view, the quality of reviews has decreased substantially since “Bill Sandholm’s renaissance age in Evolutionary Game Theory".

Thanks a lot for sharing your experiences and concerns.

Luis

[YANNICK VIOSSAT]

Hi everybody, if finding a suitable referee is one of the issues, maybe making a list of experts in evolutionary game theory would help (ideally indicating subfields of expertise, website, e-mail, etc.). I would be happy to contribute to a collective effort in that direction.

All the best,

Yannick

Envoyé à partir de Outlook pour iOS

---

De : learning-evolutio...@googlegroups.com <learning-evolutio...@googlegroups.com> de la part de LUIS RODRIGO IZQUIERDO MILLAN <lrizq...@ubu.es>
Envoyé : Thursday, August 15, 2024 11:17:56 AM
À : Jonathan Newton <jcsn...@gmail.com>
Cc : Learning Evolution and Games Forum <learning-evolutio...@googlegroups.com>
Objet : Re: [LEG_Forum] Improving referee reports

 

To view this discussion on the web visit https://groups.google.com/d/msgid/learning-evolution-and-games-forum/9EFBAA2C-2F2B-4190-B5DE-DFDD65852DA9%40ubu.es.

[Jonathan Newton]

Dear Yannick (& all),

That sounds like a good step, and one that has no downside.

Given that it was my grumbling that started this thread, I'm happy to help. I'll be in contact next week when I get back from the mountains.

All the best,

Jonathan

PS. I just read & enjoyed your paper with Panayotis - "survival of dominated strategies under imitation dynamics". Very good stuff.

[Kuzmics, Christoph (christoph.kuzmics@uni-graz.at)]

Dear all,

Thanks, Jonathan, for raising this issue. I am sure it would be useful for editors to have a list of (evolutionary) game theory referees.

However, I think there is a more fundamental problem with publishing (evolutionary) game theory papers in, what we consider, good journals. Many of us, I believe, are in economics departments, and many of us, therefore, aim to publish in the top economic journals. But, correct me if I am wrong, many of us are not necessarily directly motivated by economic questions. I, for instance, am often motivated by micro-sociological questions, or sometimes more foundational questions. Explainably, but not necessarily completely naturally, the best game theory research is published in econ journals, so it makes sense to publish there even if we weren’t in economics departments.

But I can understand the argument of editors of economic journals that any paper that they publish should add to the understanding of some economic problem. I feel that I often don’t make a strong enough case that what I am doing is economically relevant. In fact, the best papers in economic journals are probably those that had an economic question or problem (and not a model) as its starting point. The economics research discipline has historically been quite forgiving, I feel, in this regard. For us, having Bill as editor in many journals has helped, of course. And some people would claim that almost anything is economics, but I feel that editors are becoming less lenient nowadays. [This may be due to the more recent need to make key findings of papers relevant to the general (science) public, for dissemination through social media. I may well be wrong about this, though.]

I am not quite sure what one could do about it. One option is to create new journals for us with the hope that the best of our work will create some impact beyond our discipline and will make the new journals great. Another would be to make our work more relevant to economic problems. Sometimes there may actually a deep economic problem that started a subfield of research that some of the researchers almost forgot about or, at least, forget to mention. One could try and go back and start the papers with this underlying economic problem and explain how the paper adds insights to that problem. Alternatively, one could try and publish in other disciplines. I should probably try to publish some of my work in sociological journals if I think that I am making a contribution to sociology. [Yet I seem to be too lazy to make an effort to make my work understandably relevant for sociologists and to make my papers fit into the related sociological literature, which exists and I somewhat arrogantly ignore.]

Sorry about the long email! Best wishes,

Christoph

Christoph Kuzmics
https://homepage.uni-graz.at/de/christoph.kuzmics/
Professor of Microeconomics
School of Business, Economics, and Social Sciences
and Complexity of Life (COLIBRI)
University of Graz
blog: https://gametheory.life/

---

Von: learning-evolutio...@googlegroups.com <learning-evolutio...@googlegroups.com> im Auftrag von Jonathan Newton <jcsn...@gmail.com>
Gesendet: Samstag, 17. August 2024 08:00
An: YANNICK VIOSSAT
Cc: LUIS RODRIGO IZQUIERDO MILLAN; Learning Evolution and Games Forum
Betreff: [Caution: Message contains Redirect URL] Re: [LEG_Forum] Improving referee reports

 

To view this discussion on the web visit https://groups.google.com/d/msgid/learning-evolution-and-games-forum/CAKavRkdZJeN57G_Uu7fRnnmDyOnB3YGtxA%3DMy3L_9-AXmOevFw%40mail.gmail.com.