Conversation around onPhD assessment and badges

[Leigh Blackall]

I've attempted to document a flurry of discussion around the open and networked PhD concept today: http://en.wikiversity.org/wiki/Talk:Doctor_of_Philosophy#Communications_log

Hopefully it will converge into this email forum, where I think it might be just a little more manageable...

In short, there might be a small group keen on developing the Wikiversity page for PhD, into a criteria space for peer to peer assessment of thesis, and award of badges aligned to the Mozilla Open Badges initiative.

[Peter Rawsthorne]

Leigh,

This is an exciting development. I particularly like how it fits both as a way of supporting the supervisory duties in getting someone to completion of their traditional PhD and it could provide an approach to recognizing a OnPhD done outside the institutions...

I did a whole bunch of work during my time at Memorial University on supervisory skills, from both the graduate student perspective and the supervisor perspective... Also, there is some discussion happening around the use of badges for hitting milestones and then having a meta-badge for something larger (the OnPhD). I agree, if we get this right it could be of great assistance to students and supervisors. I wrote a post about a related badging issue... the point here is David Wiley had a student create his own criteria and then provide evidence. A model we could follow; http://criticaltechnology.blogspot.ca/2012/06/open-badges-evidence.html

I believe this criteria and evidence should become another post to this group... I will follow-up.

Peter

[Peter Rawsthorne]

I'm still in research mode... here are some excellent reads about peer review, assessment and open badges...

http://mediacommons.futureofthebook.org/mcpress/open-review/
http://carlacasilli.wordpress.com/2012/05/21/badge-system-design-what-we-talk-about-when-we-talk-about-validity/
http://remediatingassessment.blogspot.ca/2012/07/responding-to-michael-coles-question.html

Looking Forward...

[Leigh Blackall]

These are valuable links Peter! Thanks for sharing them, I passed them on through Google+, Twitter and FB.

I'll hopefully make time for a more considered response, and I think it will be along the lines of focusing on the intrinsic value of the peer to peer aspect of badges, rather than in relation to more formal, or institutionalised forms of assessment. I know you too will share this interest.

Wikipedia has had a peer to peer badging system running for a long time now, they just haven't called it badging. Based around the Original Barn Star. I think a study of the use, value and effectiveness-for-what would be good additions to the discussion contained in those links you've shared...

[Thomas Steele-Maley]

Very valuable links and clarity on what exists in Wikipedia.  I want to see a learning exchange rise from any motion on Badges....how we find those to support us outside of the academy who we can validate is of interest to me.  Not necessarily vitas and blah.... But rather a place for pedagogues co-researchers and praxis weaving to happen 

Thomas Steele-Maley

http://steelemaley.net

207.629.7881

[Leigh Blackall]

And there is where we practice what we preach, documenting to dates and methods we used in an attempt to build a network around #onphd, in the hope we will reach a point where finding peers is likely. We are lucky, in that we are early. It should be easy for us. Tomorrow, it will be more difficult.

[Joelle]

Thanks for sharing information on Badges....interesting. Was going to have a go at the criteria on wikiversity (and will do so later) but at this stage I would just like to point out that the PhD is not just about the end product. Criteria would need to include skills acquired in the process and that can be made evident e.g. by a paper published on line....

[Peter Rawsthorne]

Joelle,

I agree, a PhD is not a single body of work towards an end product. It has many events within a journey. This one of the reasons I have created my profile page on Wikiversity ( http://en.wikiversity.org/wiki/User:Peterrawsthorne/PhD ) and keep it reasonably up to date. I also blog extensively within my domain of study. I believe all this with a number of other published and scholarly works will coalesce into a PhD level of work.

I believe the criteria of an OnPhD would include milestones and completed works along the way. It would not end in one product. The great thing about open badges is they could be awarded for individual milestones, works and events, then a meta-badge could be awarded once all criteria has been completed and the peer review agrees a PhD level of work has been completed.

Open badges could support this very well. Each badge has a criteria and evidence and these can be whatever the badge issuer wants them to be. As a group we may decide the criteria for an OnPhD would be;
* > 10 published articles in a scholarly journal
* > 300 blog posts within subject domain
* > 4 open data contributions
* > 3 active networks with cumulative > 1000 followers
* contribution to two published books
* completed a peer reviewed scholarly work equivalent to PhD thesis
* completed > 25 reviews of others scholarly work
* > 3 conference presentations
Each OnPhD candidate would create the criteria of this themselves with guidance from this peer group and guidelines for the OnPhD criteria creation. I explore this idea of creating your own criteria in this blog post about the work done by David Wiley http://criticaltechnology.blogspot.ca/2012/06/open-badges-evidence.html

Once someone has published their criteria (which has completed a peer review) their OnPhD would continue. I believe a lot of what should be in the criteria guidelines should encourage openness, peer review and network engagement.

The badges we issue for the OnPhD could include a number of these above mentioned milestones. I also think this could work well in keeping people engaged...

This is a great conversation... looking forward to further developments...

Peter

--
Peter Rawsthorne
pe...@rawsthorne.org
(778) 855-2512

[Leigh Blackall]

I wonder if we could scale this idea more, and award separate badges for different research skills that are demonstrated in projects, and these badges accumulate to an onphd.

This would mean for example, creating a badge for different research methods like say, video ethnography, where the criteria would include ethics, recording, cataloging and analysis... All openly but apprpriately documented in process. Other badges would include, survey and statistics; lit review; action research (help me out here Joelle, there would be many!)

Therefore the onphd badge would be awarded when a number of these badges are earned, and so encourage and support mixed methods research. This would give people multiple access and exit points if they do not complete the full onphd badge. And with levels we could set these up to work for high school, undergraduate, honours and masters equivalents...

I realise that many people would shy away from this concept,  seeing it as just further encroachment by rationalised, competency based training.. this would indeed be a negative consequence but, flexibility and creativity is afforded to those with imagination.

On Jul 17, 2012 12:08 AM, "Peter Rawsthorne" <pe...@rawsthorne.org> wrote:

[Thomas Steele-Maley]

Love the idea of scaling Leigh with critical caution.

The idea of a set "criterion" say an amount of blog posts rubs rough on me.  That said, bodies of multi- media and in my case praxis, and the applied action-design based research that goes with need to be supported by those with varied experience around the world away from institutions.  

We must be ever careful to not bound the work of our human self determination in learning.  We seek to mutate fully that which exists, not give the world a new "free" or even "open" version of what exists.  How we enable individuals who seek depths of learning is vital across socio-economic ethnospheres.  

I am involved in a group working on open badging for "k-12" just past ideation.... I will keep this group in the loop with our progress and any resources past those given already on  ashes would be appreciated!

Onward,

Thomas

Thomas Steelen-Maley

http://steelemaley.net

207.629.7881

Thomas Steele-Maley

http://steelemaley.net

207.629.7881

[Yves]

Hello guys,

I have been reading all your posts and I think we have taken significant steps in getting this onphd work done. Can we make a list of all the guidelines we have set so far and come to a common agreement as a set of guidelines for an onphd. I think all of them are valuable. I think two or three research articles published in a scholarly journal can also be acceptable. We can try to work out the list presented by Peter to make each criterion more attainable by the candidates. Obviously these guidelines will be open and each candidate has the freedom to issue his own criteria as mentioned in the Wiley blog post referred by Peter. Once that done we can start issuing the badges and I think Peter is in charge of this. Following our email discussions we agreed that we can still pursue the assessment at Charles Darwin university and Leigh was ready earlier to approach the university in this regard. Can Leigh approach the university concerned faculty both in terms of assessment methods used and financial cost? I think we can have this information and decide later if this is something we want to pursue. Concerning the last response to this discussion by Thomas about offering "deep learning" to individuals I think each candidate can deepen his learning by setting his own criteria. This is also something we can work later after our work is completed. Obviously all of the candidates are pursuing this PhD only to satisfy their intellectual needs or to innovate in the academics field not to pursue a paid position as the current system is still holding to the traditional system of credentials by grades and diploma. I would like to know how  you see the future of badges as a recognized set of credentials. I think all who are involved in this should get together in order to make this form of assessment acceptable by all the institutions and the society in general. 

[Leigh Blackall]

Hi Yves. I will follow up with CDU in 24hrs.

There is the beginning of a basic criteria, in the badge section of the Wikiversity article for PhD. If conversation continues here on criteria, we need to try a capture it on the wiki I think. You've suggested a few, such as 3 articles published in a peer reviewed journal. I think this number is more common in hard science like medicine, engineering, or similar. I've not heard of a social science PhD by publication awarded on 3 publications however..

Again, Joelle might be able to offer some comment..

[Joelle.Vandermensbrugghe]

A list of guidelines would be a great idea and starting point for a discussion.

 

I think publishing articles are part of the PhD process. But PhDs are not awarded solely for the articles, but for the discussion situating the articles within the field and as contributions to knowledge. This has to be part of it and it could be in the form of a public forum discussion about the articles written.

 

Guidelines need to be open as people undertaking PhDs do it for so many different reasons, in so many different fields, using so many different research methods. But that also makes it so difficult.

 

Skills I see as important are

Ethics

 

Thinking logically

 

Understanding field (peer reviewed literature review)

 

Communication (evidenced in presentations and publication)

 

Contribution to knowledge and willingness to share knowledge

 

Then there are all issues of methodology. Understanding ontology and epistemology underpinning research and being able to justify choices made. Then also understanding of statistics, of data sampling…….till data analysis….it is important that this is somehow articulated. Papers do not necessarily do so.

 

I just had a look at Coursera - https://www.coursera.org/#about . It offers a few courses that could be really useful. Could completing some of these courses count? (E.g.  Introduction to statistics(Princeton University Course) https://www.coursera.org/#course/compdata; Introduction to Logic (Stanford University); https://www.coursera.org/#course/intrologic;  Think Again: How to Reason and Argue (Duke University). https://www.coursera.org/#course/thinkagain; Data Analysis (John Bloomberg School of Public Health). https://www.coursera.org/#course/dataanalysis; Critical Thinking in Global Challenges (University of Edinburgh) https://www.coursera.org/#course/criticalthinking;  Introduction to Philosophy (University of Edinburgh) https://www.coursera.org/#course/introphil

 

 

Joelle Vandermensbrugghe

[Leigh Blackall]

Awesome Joelle! I'm about to go offline for a week. CDU Research Office wasn't available to talk last week. Can someone transfer what we have here in this email thread, to the wiki? I'll get back on things next week.

[Leigh Blackall]

Well, not 24hours, but now.. 

Each time I tried talking with CDU Research Office, the key person was out.

I bailed her up in the corridor however. This is my 3rd conversation with people from the Research Office here at Charles Darwin University. Each time I glean a little more information.

Yes, as we know, CDU offers a PhD by pre publication. As do several other Australian universities... which ones though, wasn't made clear.

It costs ~AU$9000 per semester for internationals to enrol in such a program, and in her experience, it usually takes 12 months or 2 semesters to complete, due to a range of issues that I'll list later.

The suggestion that someone might arrive at CDU with all the evidence of supervision, peer review, etc, and expect simply an assessment for the award of PhD was predictably a challenge in our conversation. But pushing for an explanation why, lead us back to those range of issues that usually results in someone having to enrol for at least 2 semesters, and receive the supervision of a local academic. 

CDU want to make more explicit, its requirements for enrolling in and completing a PhD by pre publication. This includes:

1. Clarifying what an appropriate equivalence is for academic history. So where a candidate has not completed an Honours or Masters Degree, what is a satisfactory equivalent?
2. What quantity of work submitted as pre publication must be the original work, and primary authorship of the candidate?
3. What range of methods and processes need to be demonstrated to qualify as a PhD?

More issues where implied, and while they hope to define answers to them before September's cut off for International Scholarship applications, more likely it won't be before March 2012. At this I mentioned P2PU, and our discussions here in ONPhD around the concept of badging research methods and processes. She mentioned copyright, and I responded with a mention of the P2PU course on copyright. I suggested to her that if she engaged with P2PU, she might find volunteers to help formulate that criteria for the PhD by pre publication... I got the impression that she was not terribly inspired by the suggestion however.

So, do we think our little group could gather enough people to further our project around badging various research methods? I imagine it would be a case of finding existing open courses, bringing them together in a the one place (maybe only as links) and working out a peer assessment for each badge...

--
--
Leigh Blackall

+61(0)404561009

[Peter Rawsthorne]

this is interesting...

http://degreed.com/

Be Well...

On Friday, July 13, 2012 5:16:27 AM UTC-7, Leigh Blackall wrote:

[Leigh Blackall]

Looks like what we're trying to do. Nice and simple presentation. I couldn't work out how to try it out though!?