Course review (proposed process)

[Jane Park]

Hi all,

Thanks for those who have given feedback last week and in the past. Taking all of that into account, we propose the following lightweight process and potential future features.

First, some background: Currently, the School organizer must manually add a course to the School of Open for it to be featured or otherwise show up on the landing page at http://schoolofopen.org. Anything after "Featured" is also manually handled, eg. draft course and those in-development.

Second, the proposed process:

1) School of Open organizer can still manually add a course, but at least two organizers must review and approve the course before it is added. Since the School of Open is a joint initiative being coordinated by Creative Commons and P2PU, one organizer from each party must review a course before it is featured on the landing page. CC-side, that is currently me. P2PU-side, that can be staff or community volunteer that is designated as an organizer of the School. Additional parties, such as content or domain experts, may be pulled into review a course on a case by case basis. This process does not apply to draft or in-development courses which are clearly noted as such on the School page.

2) The landing page will make clear that School of Open courses are created by the community and not all endorsed or owned by any one organization. This will be aided by:
- clear explanation and descriptive text
- the ability for any School of Open course to be "branded" as created/vetted/or associated with a specific organization on the School of Open landing page. For example, any course CC creates will be visibly shown as being created by CC while any course Open.Michigan creates will be visibly shown as being created by Open.Michigan.
- the landing page will make clear that there are courses created by individuals that are not associated with any organization.

3) We will integrate any community review process that is integrated at P2PU as a whole, eg. community rating or badge award system for good courses, mentoring platform, etc. The key point is that we intend to explore this with what P2PU is already doing in this direction, and not doing something that is distinctive to School of Open itself. Community review and mentorship will not be required for a course to be featured on the SOO page, but will be essential for continued improvement of the course and community participation.

Some things to note:
* Of course, outside of SOO, anyone can still create and publish a course at P2PU at anytime. All P2PU terms and community norms still apply to anything on their site/platform.
* Right now, only 1) is immediately implementable. 2) and 3) will require future work, which School of Open will have a hand in shaping.
* All of the above, once agreed on by this community, will be documented as clear steps to take for course organizers who want to create a course as part of the School of Open. Clear steps will include reference to our guidelines: http://pad.p2pu.org/p/school-of-open-guidelines.

So what do you guys think? Please let's discuss and share any feedback on this thread by the end of the week!

Best,
Jane

--
Jane Park
Project Manager
Creative Commons

School of Open, a collaboration with P2PU: http://schoolofopen.org

[Philipp Schmidt]

On 24 October 2012 14:33, Jane Park <jane...@creativecommons.org> wrote:

Hi all,

Thanks for those who have given feedback last week and in the past. Taking all of that into account, we propose the following lightweight process and potential future features.

First, some background: Currently, the School organizer must manually add a course to the School of Open for it to be featured or otherwise show up on the landing page at http://schoolofopen.org. Anything after "Featured" is also manually handled, eg. draft course and those in-development.

Second, the proposed process:

1) School of Open organizer can still manually add a course, but at least two organizers must review and approve the course before it is added. Since the School of Open is a joint initiative being coordinated by Creative Commons and P2PU, one organizer from each party must review a course before it is featured on the landing page. CC-side, that is currently me. P2PU-side, that can be staff or community volunteer that is designated as an organizer of the School. Additional parties, such as content or domain experts, may be pulled into review a course on a case by case basis. This process does not apply to draft or in-development courses which are clearly noted as such on the School page.

I think the term "approve" might create wrong expectations. I look at this more as SPAM prevention and an opportunity to engage with course organizers and give feedback on improving their courses (where applicable). 

It would be great if we could find someone in the community who feels passionate about the content area. Ping me off list if this is something you might be interested in. 

I'll also forward this to the P2PU community list (to make sure they have a chance to chime in).

P

[Jane Park]

I think that's a logical point. And if CC owned the School of Open and were the ones deciding all courses that were a part of it, then we would be endorsing them in a sense. But I don't think that's the case here. The School of Open is a joint initiative, and technically P2PU owns the domain and the name and has an MOU with CC that outlines that sort of stuff. CC is playing a coordinating role. And I think we can separate the coordinating roles of the School organizers  from whatever the organizations themselves (legally) endorse. In a coordinating role I'll provide feedback for a course and say I think it will be ready to be featured on the School's page at this point. Then someone in the P2PU community will agree, and the course will be added to the School. But that doesn't mean that Creative Commons the organization endorses that particular course. It means that the course has gone through the School's review process. 

Does that make sense? I know it's a little too nitpicky - but I really think it's just a way to legally keep things clean. Maybe I will just steer clear of using the ter "endorse" ever again. :)

On the "approve" language I think that's a good point when explaining it on the website. How about something like: "Before your course becomes part of the School of Open, two School of Open organizers will provide feedback on your course, giving you the opportunity to improve your course. When your course is ready, it wil be featured.. etc."

On Wed, Oct 24, 2012 at 12:18 PM, Vanessa Gennarelli <van...@p2pu.org> wrote:

Hey Jane--agree with Philipp about the term "approve." 

Also, as a potential School of Open course organizer I might find it strange that my course will be vetted/reviewed by Creative Commons and P2PU, but not endorsed by them. I understand your intent, but I think vetting/endorsement sort of fit together logically.

Maybe I don't understand, tho--set me straight if that's the case! :)

VMG

On Wed, Oct 24, 2012 at 3:03 PM, Philipp Schmidt <phi...@p2pu.org> wrote:

Forwarding this message from Jane, which raises a few interesting questions about review of courses. 

I've already chimed in on the school of open list, but my points regarding section (2) are:

* I think the term "approve" might create wrong expectations. I look at this more as SPAM prevention and an opportunity to engage with course organizers and give feedback on improving their courses (where applicable). 

* It would be great if we could find someone in the community who feels passionate about the content area. Ping me off list if this is something you might be interested in. 

--
http://schoolofopen.org
---
You received this message because you are subscribed to the Google Groups "School of Open" group.
To post to this group, send email to school-...@googlegroups.com.
To unsubscribe from this group, send email to school-of-ope...@googlegroups.com.
Visit this group at http://groups.google.com/group/school-of-open?hl=en.
 
 

--
You received this message because you are subscribed to the "P2PU Community" group. Please stick to the ground rules:
http://groups.google.com/group/p2pu-community/web/ground-rules
 
Specific topics such as research, web development and course design are discussed in separate working groups:
http://wiki.p2pu.org/mailing-lists

--
Vanessa Gennarelli

Learning Lead, Peer 2 Peer University

van...@p2pu.org

@mozzadrella

--
You received this message because you are subscribed to the "P2PU Community" group. Please stick to the ground rules:
http://groups.google.com/group/p2pu-community/web/ground-rules
 
Specific topics such as research, web development and course design are discussed in separate working groups:
http://wiki.p2pu.org/mailing-lists

--

School of Open

http://schoolofopen.org

Latest news: http://info.p2pu.org/tag/school-of-open

[Philipp Schmidt]

I endorse your opinion Jane. 

I wouldn't make the process too technical. Just put a link on school of open home page that says, "Want your course to be part of the school of open? Send an email to the school of open mailing list and tell us about it." Then it becomes less about two people endorsing it, but rather the community can chime in if they like the course or if they don't. Makes the process open and transparent. 

P

[Pete Forsyth]

Jane, I like your proposal.

If I'm not mistaken, I see echoes of my suggestion on the last call that Wikipedia's Featured Article process might be a good model. I think that has some bearing on the points under discussion, and maybe a little more exploration of how that works will be useful.

In practice, here is how the FA process on Wikipedia works today [1]:

* There is a published set of standards that must be met before an article is featured [2]

* There is a "Featured Article Editor" -- I think this has always been the same guy -- who has the final authority on whether or not an article is featured.

* The Editor has three appointed delegates who, in practice, make all those determinations. The determination is essentially an evaluation of the consensus of:

* The real work is done ad hoc by anyone who wants to participate, by discussing any point they feel is important, but ideally in reference to the FA standards. Sample discussion: [3]

So, I guess the one thing I would add -- and I'm not sure if this is a change to the substance of what is proposed, or just emphasis -- is that the approval of the two organizers should ideally be done in an open and transparent way, where interested community members have ample opportunity to voice their praise, criticisms, suggestions, questions, prior to a decision. I think if we imagine SoU getting very big and successful, we will find that a variety of experts is needed, and will emerge, to evaluate courses that may be beyond the expertise of the organizers.

At any rate, I think this core is solid, and would love to see it move forward. Thanks for posting this, Jane!

-Pete

[1] https://en.wikipedia.org/wiki/Wikipedia:Featured_article_candidates

[2] https://en.wikipedia.org/wiki/Wikipedia:Featured_article_criteria

[3] https://en.wikipedia.org/wiki/Wikipedia:Featured_article_candidates/Columbia_River/archive1

Pete Forsyth

petef...@gmail.com

503-383-9454 mobile

[Jane Park]

On Wed, Oct 24, 2012 at 4:29 PM, Pete Forsyth <petef...@gmail.com> wrote:

Jane, I like your proposal.

If I'm not mistaken, I see echoes of my suggestion on the last call that Wikipedia's Featured Article process might be a good model. I think that has some bearing on the points under discussion, and maybe a little more exploration of how that works will be useful.

In practice, here is how the FA process on Wikipedia works today [1]:

* There is a published set of standards that must be met before an article is featured [2]

* There is a "Featured Article Editor" -- I think this has always been the same guy -- who has the final authority on whether or not an article is featured.

* The Editor has three appointed delegates who, in practice, make all those determinations. The determination is essentially an evaluation of the consensus of:

* The real work is done ad hoc by anyone who wants to participate, by discussing any point they feel is important, but ideally in reference to the FA standards. Sample discussion: [3]

So, I guess the one thing I would add -- and I'm not sure if this is a change to the substance of what is proposed, or just emphasis -- is that the approval of the two organizers should ideally be done in an open and transparent way, where interested community members have ample opportunity to voice their praise, criticisms, suggestions, questions, prior to a decision. I think if we imagine SoU getting very big and successful, we will find that a variety of experts is needed, and will emerge, to evaluate courses that may be beyond the expertise of the organizers.

Very good point Pete, and I full-heartedly agree. I think any feedback on part of organizers or community should be done on this discussion list or on an etherpad (which is linked from here). Also open to other (uncomplicated) suggestions for communicating open feedback.

Regarding your example above regarding published standards, I think there was some pushback from this group regarding anything so formal (yet anyway), but we do have some working guidelines here (http://pad.p2pu.org/p/school-of-open-guidelines) that people can use to review or improve their course. School organizers could provide their feedback along these guidelines, which are open for change and improvement of course.

Over time the community may decide they want a set of published standards, but I'm happy to let that happen organically as the need arises.. (has the need arisen?)

[Rebecca Kahn]

I totally agree with everything said up to this point. 

I also remember that there were often issues with the School of Webcraft when certain courses, which on would logically have nested under the umbrella of SoW didn't fit because of the inclusion or non-inclusion of open source software. The more boxes you ask people to tick to be included, the less inclined they will be, I suspect, to join.

Also, from a purely logistical perpective, it's a lot easier to respond to an email from a real person asking a list for endorsement than to one of the 20-million notifications one might get from the site (or not get, if you've switched them off).

B

[Pieter Kleymeer]

Thanks for proposing a robust process, Jane. I really like Philipp and Pete's suggestions to let the community comment on new courses, keeping that "approval" lightweight and transparent. 

One thing I'd like to add is a use case that Open.Michigan has talked about for a while now. It goes like this:

Someone in the community creates a course or challenge on P2PU and adds it to the School of Open. Open.Michigan representatives acting on their own take the course and really like it. They create an Open.Michigan endorsed badge and ask the course creator to add it to the course. Anyone who completes the course then gets the Open.Michigan badge. In effect, Open.Michigan has both approved and endorsed the course simply by offering a badge to those who take the course. It's a simple mechanism that can solve some of the issues we're discussing. Other orgs like CC can also offer a badge for that course, effectively endorsing it. Maybe CC decides they only want to issue badges for courses they create - that's cool too. Maybe CC chooses to endorse a badge offered by Open.Michigan (this was supposed to be possible with Open Badges at some point). 

I guess the idea is that there are a number of ways courses/challenges can be vetted and endorsed that won't hold up the process of creating and launching them. 

-Piet

[Jane Park]

Hi everyone,

Thanks for all of your great feedback on this. I have drafted an overview of the course review process on an etherpad at http://pad.p2pu.org/p/school-of-open-course-review, keeping in tradition with our other living documents (guidelines, philosophy) so that it is open to change over time and according to our needs.

I tried to incorporate everything expressed here. Of course, the actual site (schoolofopen.org) will merely say state something simple like a three step process, eg. 1. send your draft to the list and get feedback 2. incorporate feedback and iterate 3. get school organizers' approval and featured as part of SOO landing page... you get the point. But I thought documenting all our thinking here on a pad was worthwhile for transparency and open governance sake.

I plan to link to this pad at http://wiki.creativecommons.org/School_of_Open#Running_the_School_of_Open_openly and http://info.p2pu.org/schools/open/ under the governance sections, as we similarly already have links to the guidelines and philosophy.

Please add your edits, comments, suggestions to the pad!

Cheers,
Jane

--
Creative Commons is turning 10! Give a birthday gift. https://creativecommons.net/donate/

[Jane Park]

Now linked from http://wiki.creativecommons.org/School_of_Open#Running_the_School_of_Open_openly and http://info.p2pu.org/schools/open/.