Idea: badges/icons for CV's

[a.]

I like the badges project (https://osf.io/tvyxz/) so readers of papers can clearly see that good/open practices were followed. I like rewarding researchers who follow good/open practices. And I like uniformity as well.

I think it could be very useful if individual researchers could somehow indicate on their CV that they used open/good practices concerning their listed individual papers. I reason this might help signaling, promoting, and rewarding good/open practices when researchers apply for jobs, grants, concerning promotions, etc.  Perhaps having the option to indicate this on your CV could even be an incentive for engaging in these practices.

I wondered if it would be useful to come up with a project (or adapt an existing project) that:

1) Provides several badges/icons that researchers could use on their CV regarding individual papers which adhered to certain specific open/good practices.

2) Provides a link to the project that researchers could include in their CV that briefly explains these badges/icons, and why the practices could be seen as useful where readers of the CV could go to for more information.

I reason this could possibly further help in explaining, signaling, promoting, and rewarding good/open practices at a possibly very important level.

You could use badges/icons for practices that the existing badges represent (i.c. open materials, open data, pre-registration) and possibly add things like using the “21 word solution” (https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2160588) to indicate full methods disclosure, etc. 

An important criterion for these badges/icons used on CV’s could be that all the information that the badge/icon represents should be able to be checked by anyone who reads the paper. That way, possible misuse can be detected by anyone who reads the CV and papers (and basically boils down to the same thing as adding a publication to your CV for which you are not an author).

I am curious as to whether you would personally use this on your CV if something like this were to be made available, and whether you think this could be useful in the first place.

Thank you in advance for any possible replies!

Kind regards.

[a.]

There were no reactions to this post, which makes it hard for me to decide whether it makes any sense and/or whether it has been considered but not deemed useful. I don’t want to waste people’s time, but I also don’t want to throw away a possibly good idea without giving it another shot. So, here is my final attempt at trying to contribute something useful by trying to get people to possibly think about it a 2^nd time, and/or post a reply.

1) I have heard that the existing badges-project already encourages researchers to use badges on their CV, but if I understood things correctly *only* if a journal awarded them. As of this date, 15 journals (?) provide, or will provide, badges.I think you could effectively increase the potential use of badges on CV’s for publications that used open/good practices by a very, very large amount if you would allow researchers to use badges on their CV based on their used practices and *not* depending on the particular journal they happened to have published their article in.

In my reasoning, one of the strengths of things like posting data, posting materials, and pre-registration is that individual researchers can use them *regardless* of journal policy. Based on that reasoning, only allowing researchers to use badges on their CV for publications in journals that award them, to me is similar to only allowing researchers to post their data if a particular journal allows this.

Why wait for journals to adopt awarding badges? Why not provide the opportunity for researchers to indicate they used open/good practices for *any* publication that they have published using open/good practices *regardless* of the journal’s policy regarding awarding badges?

2) There has been a lot of talk about rewarding researchers based on other things than the no. of publications on their CV. Using badges on CV’s could possibly help tremendously in this regard by making clear that open/good practices were *actually* followed.

I am glad to read, that this possibly important issue of how to signal the use of open/good practices to employers, grant commissions, etc. is being recognized. I have just viewed a SIPS project that aims to provide recommendations for job candidates to emphasize their use of open/good practices (https://osf.io/rw8v9/). The text currently mentions providing links of pre-registered projects on the OSF, and adding footnotes or asterisks to publications on CV’s that used open/good practices.

Why not use an already existing project like the badges-project and extend it to  include the use of badges for CV’s, which accomplishes the same things but does so in a more uniform manner which could be beneficial regarding clarity, validity, and recognizability? Perhaps doing so could even help getting journals to adopt the badges, by making clear that these badges exist, and researchers (would like to) use them.

Kind regards.

[Etienne LeBel]

Great ideas, Alex!

>>>>but if I understood things correctly *only* if a journal awarded them. As of this date, 15 journals (?) provide, or will provide, badges.I think you could effectively increase the potential use of badges on CV’s for publications that used open/good practices by a very, very large amount if you would allow researchers to use badges on their CV based on their used practices and *not* depending on the particular journal they happened to have published their article in.

To my understanding from reading the official COS open practices badges specs, researchers "earn" badges based on meeting the specified criteria for the different badges. Hence, I don't believe there are formal reasons preventing researchers from using (earned) badges on their CV/website for publications at journals that aren't yet formally awarding badges.

--
You received this message because you are subscribed to the Google Groups "Open Science Framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openscienceframework+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alicia Hofelich Mohr]

Another option is for researchers to deposit data/materials in a repository that issues a permanent URL or DOI, and add the data citation to their CV, along with the article citation. That can also demonstrate their commitment to open science and doesn't depend on where they publish the article. 

Best, 

Alicia

To unsubscribe from this group and stop receiving emails from it, send an email to openscienceframework+unsubscrib...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Open Science Framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openscienceframework+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Alicia Hofelich Mohr, Ph.D.

Research Data Manager

Research Support Services | CLA LATIS

University of Minnesota

612-626-8456

[Gustav Nilsonne]

Hi,

Thanks for these thoughts. 

The badges for open practices can in principle be issued by anyone - journals, third parties, and even authors themselves. In interpreting the meaning of a badge, it helps to know who awarded it and what kind of process they used, e.g. self-report, verification by reviewers etc. Since badges are issued to papers rather than to people, it is usually only the journals that can establish a linkage such that the badge appears on the paper.

We've not seen any examples, as far as I know, of people awarding badges for open practices to their own papers and listing them on the CV. Please post to the list if you see anyone doing it, as it would be interesting to see how that turns out.

Best wishes,

Gustav

Chairperson for the Badges committee

Gustav Nilsonne, MD, PhD
Researcher

+46 (0) 736-798 743

Stockholm University 

Stress Research Institute

106 91 Stockholm

Karolinska Institutet 

Department of Clinical Neuroscience
Nobels väg 9
171 77 Stockholm

gustav....@ki.se

---

From: openscienc...@googlegroups.com [openscienc...@googlegroups.com] on behalf of Etienne LeBel [etienn...@gmail.com]
Sent: 24 January 2017 16:52
To: openscienc...@googlegroups.com; bad...@cos.io
Subject: Re: [OpenScienceFramework] Re: Idea: badges/icons for CV's

To unsubscribe from this group and stop receiving emails from it, send an email to openscienceframe...@googlegroups.com.

[David Mellor]

Hello Alex et al,

One of the greatest benefits of the open practice badges is that they signal adherence to best practices and increase adoption of them without onerous mandates from journals or publishers. One important trait for these signals to be effective is that they remain "honest" in the sense that there is no easy way to dilute the pools of earned badges with badges that do not signal true adherence to the data, materials, or preregistration standards (silly aside: that "honest signal" rationale is a linchpin of sexual selection theory- the bright red plumage has to be really harmful in order to signal "good genes" to potential mates!). Keeping that signal honest means ensuring that there is a third party issuer of a badge, so that the recipient (in this case, a paper) can point to that issuer as having certified compliance with standards. So while self-issuing of badges does risk diluting their value, other entities that in the business of "publishing" research outputs (e.g. IRs, conference committees) are welcome and encouraged to become issuers.

I think a first step in this is identifying possible entities who could become issuers. That is an ongoing task that we have been using for related efforts on the Transparency and Openness Promotion Guidelines and Registered Reports, and we are compiling a similar list of target, potential issuers now. 

Best,

David

David Mellor, PhD

Project Manager, Journal and Funder Initiatives

Center for Open Science

(434) 352-1066, @EvoMellor, Skype: evomellor

Are you ready to take the Prereg Challenge?

--

Alicia Hofelich Mohr, Ph.D.

Research Data Manager

Research Support Services | CLA LATIS

University of Minnesota

612-626-8456

--

[Andy DeSoto]

This is an interesting idea. I know that there are ways to display Altmetric badges in inline format (see this link: https://www.altmetric.com/products/altmetric-badges/). After entries on my CV, I include the inline code to show the Altmetric reach of each article (albeit piddly!) I bet there is a way to set something like that up for the badges. You could imagine a "Citation 2.0" system which includes options to turn off and on elements like badges, Altmetric, etc. customizable by the user. I say you go for it, "a."

[a.]

"Keeping that signal honest means ensuring that there is a third party issuer of a badge, so that the recipient (in this case, a paper) can point to that issuer as having certified compliance with standards. So while self-issuing of badges does risk diluting their value, other entities that in the business of "publishing" research outputs (e.g. IRs, conference committees) are welcome and encouraged to become issuers."

I don't understand this reasoning at all!?! In my view, the key thing about badges is that they signal (and possibly promote and reward) open/good practices which any reader can check !! The only standards should be/are that any reader can have a look at the open data, open materials, and pre-registration information. I don't understand why there needs to be a third party issuer, because (if done correctly) open practices, and thus the badges, can be checked by anyone who reads the paper. Sure, when a paper is published it would make sense for the journal to check whether all this information is available before they award a badge, but that should not be confused with them being a third party issuer with some special power which only makes them the ones who should/could hand out badges.

In my opinion, and reasoning, there is no third party issuer necessary for keeping the signal honest. All is needed is that any reader can check whether the signal of the badges is honest in any particular case, simply by following the links of open materials, open data, and pre-registration information (which should be) posted in the paper itself. If anything, trying to "keep the signal honest" by allowing only a "third party" to distribute badges allows for a potential dis-honest signal: what if a journal keeps the pre-registration information hidden from the reader but does award a pre-registration badge? Is that being transparent? Is that following good/open practices? Does that pave the way for misuse? Is that where you want the badges to go?

What is better for science, a world in which individual authors use "self -issued" badges on their CV which any reader can check if they are used correctly, or a world in which individual authors use "third-party issued" badges on their CV which readers can not check but are somehow given some sort of credit and authority because some fancy, "official" journal handed them out?

In my view you are putting the power, and responsibility, of the badges at exactly the wrong place: the reviewers, editor, and journal which, if i am not mistaken, have a history of making authors leave out critical information just so the paper looks cleaner, hide null-results and failed replications, etc.

Instead, you should put the power where it belongs: which in my reasoning is the author and the reader.

I am bringing this up again, because the SIPS 2017 is starting soon, which has a hack-a-thon which is possibly related to this. I sincerely hope, smarter people than me, will think about this some more, and discuss these issues. I still stand behind everything i wrote above, and i still think this could be a great help in signalling, promoting, and rewarding good/open practices. I also still think that it could help in journals adopting these badges.

[a.]

What is better for science, a world in which individual authors use "self -issued" badges on their CV which any reader can check if they are used correctly, or a world in which individual authors use "third-party issued" badges on their CV which readers can not check but are somehow given some sort of credit and authority because some fancy, "official" journal handed them out?

Perhaps better formulated:

What is better for signalling, promoting, and (possibly) rewarding open/good practices: a world in which individual authors use "self-issued" badges on their CV for any paper that uses open/good practices, or a world in which individual authors can only use "third party issued" badges (currently available for about 16 journals).

I hope the following is useful:

You may already have come up with a possible solution yourselves. I took a quick look at the badges (https://osf.io/tvyxz/wiki/1.%20View%20the%20Badges/) which are nicely depicted in color and black and white. Boom, problem solved !

Journals (or other "third party issuers") can use the colored version and individual researchers who have used open/good practices for papers in journals with a badges-policy can used the colored version on their CV and for papers in journals without a badges-policy they can use the black and white version on their CV.

Everyone wins! There is a clear distinction between "third party issued" badges, and "self issued badges" hereby "keeping the signal honest". You still have clarity and uniformity, which i reason could be of tremendous help with regard to recognition and possible adoption of the badges by journals (once they see researchers possibly use them on their CV's). And most importantly you are possibly helping out with rewarding (early career!!) researchers for their open/good practices by (finally!!) giving them a way to make clear on their CV that they have used open/good practices.

Done!

"Do not fear failure, but rather fear not trying"

[a.]

"Do not fear failure, but rather fear not trying"

I thought changing incentives were super important but now i am not so sure anymore. I have officially stopped caring about this idea today when i wondered why i was even trying.

Then it hit me: i possibly made a huge mistake in my reasoning! I was under the impression that researchers who got badges awarded via official "third party issuers" were encouraged to include this on their CV, but this can never be true of course. If researchers would use badges on their CV for papers for which a "third party issuer" has awarded them badges, they would still be a "self-issuer" concerning their CV-badges if i'm not mistaken. This is because there is no official "third party issuer" for badges on CV's to verify whether a person awarded the badge on their CV correctly, which could result in issues concerning "keeping the signal honest".

I guess it might sometimes be a good idea to just stop trying. Perhaps that could sometimes lead to some useful insights.

[a.]

On Tuesday, January 24, 2017 at 5:12:22 PM UTC+1, Gustav Nilsonne wrote:

Hi,

Thanks for these thoughts. 

The badges for open practices can in principle be issued by anyone - journals, third parties, and even authors themselves. In interpreting the meaning of a badge, it helps to know who awarded it and what kind of process they used, e.g. self-report, verification by reviewers etc. Since badges are issued to papers rather than to people, it is usually only the journals that can establish a linkage such that the badge appears on the paper.

We've not seen any examples, as far as I know, of people awarding badges for open practices to their own papers and listing them on the CV. Please post to the list if you see anyone doing it, as it would be interesting to see how that turns out.

 

If 1) authors can issue the badges themselves for their CV's, and 2) it can be made clear what criteria and process they used (e.g. by them providing a link to a paper about this), do you think publishing something like what i included in the attachment on psyarxiv or something like that would be useful and okay to do? Or am i missing something that i should be aware of?

I just want to provide authors with the opportunity to 1) (re-) present themselves and their work optimally, and 2) increase the signaling, promotion, and (hopefully) rewarding of open practices.

I think it will be much better if the badges committee would write and publish something about this, but it looks like you don't want to do that and/or are not providing all the information. I do not want to mess up anything here, but i am trying to find out why you don't seem to think this i a good idea, and have tried to provide arguments for my position but i got no replies, which makes it impossible for me to find out where i am possibly mistaken.

 

[David Mellor]

Hello A,

I've seen the badges show up in places like personal websites as simple placeholders or section heads for data, materials, and preregistrations. I don't think that this presents the same risk that I mentioned earlier of someone repeatedly self-issuing, and thereby diluting, badges through a system like Badgr. Rather, it is just stating that these are recognizable symbols of data/materials/preregistrations.

Thanks for advocating for open science principles!

Best,

David

David Mellor, PhD

Project Manager, Journal and Funder Initiatives

Center for Open Science

1-434-352-1066, @EvoMellor

Are you ready to take the Prereg Challenge?

--

[a.]

"I don't think that this presents the same risk that I mentioned earlier of someone repeatedly self-issuing, and thereby diluting, badges through a system like Badgr".

1) If i am not mistaken, Registered Reports are already being published without the reader having access to pre-registration information. If this is correct, people are already truly diluting a lot of open science.

2) On your site about open science badges you talk about how any organization can adopt badges. That opens the way for diluting when you, for instance, allow organizations to keep the pre-registered information a secret to the reader of the paper while still awarding a badge. I tried to make this clear.

3) You told me receivers of "official" badges were encouraged to put this on their CV. I tried to make clear how this could be seen as strange if a "third party" is so important to validate badges, because there is no "third party" to check whether someone used the badge on their CV correctly, i.c. badges on CV's are always "self-issued". I tried to make this clear.

I feel that you are simply repeating that there is a chance of "dilution", but i fail to see how this would happen, and i fail to read any replies concerning my arguments as to why there is a bigger risk of "dilution" if you give the power and responsibility to journals.

I'll repeat my questions for which i haven't got any answers yet:

1) Is it okay for people to use badges on their CV when they have not been issued by an "offical" party?
2) If that's okay, is it okay for me to post the attachment on something like psyarxiv?

Kind regards (and advocating real open science!),

[David Mellor]

Hello A,

For the first question, I still think it is fine to use the badge images on your CV in a way that denotes "these are data sets that I have shared" or "here are preregistrations that I have created" and then posting links to the relevant materials. Again, I think that is a simple way to use the images as a visual indicator in a way that is not self-awarding, which I know is an issue we disagree about.

Please do post your proposal to PsyArXiv. That venue was specifically created to post content about new ideas, findings, or work that you want disseminate. 

You mentioned SIPS in an earlier email, I'm not sure if you will be in attendance, but I will be there all three days and would be happy to look at any additional materials or discuss other ways to advance openness in science. 

Best,

David

David Mellor, PhD

Project Manager, Journal and Funder Initiatives

Center for Open Science

1-434-352-1066, @EvoMellor

Are you ready to take the Prereg Challenge?

[a.]

Dear David Mellor,

Thank you for the replies! That is (finally :) ) helpful !

(Un) fortunately, it is too late for me. I have already quit a few days ago, deleted my OSF account, and as a result will not be posting what i wrote in the attachment above on psyarxiv. I will not be attending SIPS, or be involved in anything psychological science-related anymore. I think it has become quite clear that others than me should be worrying about these matters, and possibly tackling them, as I don't think i have the right communication-skills and/or patience for all of this.

Perhaps the most important point of this whole thing was/is for you, and your fellow-SIPS attendees, and fellow-badges committee members, and fellow-open science people to seriously think about these matters and possibly take action. I think the time to do so is now, and i think you are in the perfect position to do so, not me (i am not even working in science, planning to work in science, or have a PhD or anything like that).

Maybe you can talk about this some more at SIPS with others, and/or you can publish an article about how researchers can indicate on their CV that they have used open practices for certain specific articles (not some general statement that people can put up there that holds little value i.m.o.). Perhaps people can combine badges with links on their CV's.  Perhaps what i wrote in the attachment above can be some sort of inspiration for publishing something like that.

I think i have done what (i hope is) best for open science with the cards i was dealt, which is the only thing that i can do and the only thing that really matters ;)  

Good luck with (real) open science, and signaling, promoting, and (hopefully) rewarding open practices!

Kind regards.

[a.]

What is better for science, a world in which individual authors use "self -issued" badges on their CV which any reader can check if they are used correctly, or a world in which individual authors use "third-party issued" badges on their CV which readers can not check but are somehow given some sort of credit and authority because some fancy, "official" journal handed them out?

In my view you are putting the power, and responsibility, of the badges at exactly the wrong place: the reviewers, editor, and journal which, if i am not mistaken, have a history of making authors leave out critical information just so the paper looks cleaner, hide null-results and failed replications, etc.

Instead, you should put the power where it belongs: which in my reasoning is the author and the reader.

Elitism: https://en.wikipedia.org/wiki/Elitism

"Elitist status can be based on personal achievement, such as degrees from top-rate universities or impressive internships and job offers (...)"

"Elitists also believe only a few "shakers and movers" truly change society rather than society being changed by the majority of people who only vote and elect the elites into power."

"To elitists, the public is abjectly powerless and can be manipulated only by the top group of elites"

[a.]

Also sent a mail to try and get my account back to try and post my pre-print.

"Dear Rebecca (or someone else),

After i deleted my account you wrote: "We’re sorry to lose a user who has actively worked to promote open science. We hope you will consider creating an account in the future."

I thought long and hard. Also see this: https://groups.google.com/forum/#!topic/openscienceframework/7oeul9pusKg

Open science is very important to me. I have tried to get answers to my questions of why my idea (see message in thread Jul. 21st) would not be useful for promoting and rewarding open science practices, but it seems to me that people involved are reluctant to give me answers.

So, what am i supposed to do if i think i have a good idea that could help improve science?

I don't know. This is my attempt to try and find a solution.

I got permission to post my pre-print, but that was after i deleted my account. I think this is still very important, and could be very useful, and i would like to post the pre-print to Psyarxiv, but i need an account for that.

I tried to sign up again to the OSF, but got an error message. Would it be possible for me to get my account back, so i could post my pre-print?

Kind regards,

Alexander A. Aarts

[reb...@cos.io]

Hi Alexander,

See my email via our support address.

Best,

Rebecca

[a.]

Hi Alexander,

See my email via our support address.

Best,

Rebecca

Yes, got it!

Thank you for giving me a 2nd chance, and allowing me to 1) re-activate my account, and 2) post the pre-print.

I've tried to 1) have the right intentions, and 2) use my brain. These may be the 2 most useful tools that are available to me to try and help improve (psychological) science. What the results will be nobody knows, but i assume that is the case for everyone involved in science writing and/or proposing things.

Thanks to everyone who contributed to the discussion in (and perhaps outside) this thread.

Here is the pre-print: https://psyarxiv.com/n5rdv/

Kind regards.