OpenRefine Documentation

[Owen Stephens]

Hi all,

I've been wondering for a while about what could be done to improve OpenRefine documentation https://github.com/OpenRefine/OpenRefine/wiki. I think the OpenRefine wiki is full of useful information - which I use and contribute to. But I'd also say it can be hard to find the right information and sometimes information is either missing or not up-to-date. I'd like to carry out an overall review and restructure of the documentation.

Since this would make quite big changes to the existing documentation (not necessarily in content so much, but definitely in terms of structure) I wanted to see how others felt about it before I got more serious about it. So I've outlined my current thinking here and would like to know what people think. Also if anyone wants to volunteer to help out in anyway (whether in planning, or in terms of reviewing existing documentation or writing new documentation) then please let me know! If you have examples of documentation you really like or think shows best practice for documenting applications please also share.

My thinking on this is currently:

OpenRefine Documentation restructure proposal

Propose to restructure the wiki at https://github.com/OpenRefine/OpenRefine/wiki to:

• Simplify

• Improve navigation within the documentation

• Ensure users and developers can easily find relevant documentation

• Remove or update outdated documentation

• Move from creole to MarkDown as the markup standard for the wiki

• Assume we will stick to using Github Wiki for documentation at the moment

Guidance on writing good quality documentation

• https://jacobian.org/writing/what-to-write/

Good practice examples of Github wiki documentation

• https://github.com/snowplow/snowplow/wiki

• https://github.com/mcMMO-Dev/mcMMO/wiki

• https://github.com/d3/d3/wiki (especially the right-hand sidebar for navigation)

Proposed structure

• Home - what is OpenRefine + Navigation

• User Documentation

  • Installation and Configuration

  • Using OpenRefine (basic functionality etc.)

  • Recipes

  • Getting help/support

  • OpenRefine Reference (GREL / Variables)

    • Develop a page per function linking from the GREL reference pages to more detailed information and examples of using each function

  • Other resources

  • Contributing

• Developer Documentation

  • Architecture

  • APIs

  • Contributing

• Releases

  • Link to downloads

  • Release notes

• News/What's new

  
  

[Ettore Rizza]

Very good idea, Owen. If I may, the column "recipes" should perhaps be after GREL, or in a separate section "how to ..." We could put in together some of the most recurring questions on the Google group and StackOverflow.

Regarding the conversion creole> md, I just made a test by converting the documentation first into HTML, then into markdown. We now have to examine each files to track down the bugs. Here is the repo : https://github.com/ettorerizza/openrefine.wiki.html

[Thad Guidry]

@Owen - yeah that's a good start. Thanks !

@Ettore - Looks like Links are not in the right MD format?  This site was built using [GitHub Pages](https://pages.github.com/) 

Creole:               OAuth functionality is provided by the [[Signpost|http://code.google.com/p/oauth-signpost/]] project.

MD Converted:  OAuth functionality is provided by the [](Signpost)<http://code.google.com/p/oauth-signpost/%5D%5D> project.

There were other Creole to MD conversion tools out there on the web.  Perhaps some are better than others?

-Thad

+ThadGuidry

[Owen Stephens]

Thanks Ettore,

I'll let this sit in the group for a little bit to see if there is other feedback, but assuming there are no big objections from within the community, my next step will be to set up some docs which describe a workflow for the re-organisation - this will include a review of each page to check we still need it and if it needs updating, which would also be the ideal time to convert from creole markup to Markdown instead - at which point we could use these conversions you've done as the basis for the revised page.

Thanks again

Owen

[Ettore RIZZA]

@Thad : yep, according to the Creole tool documentation :

• The alternate link syntax [[ -> ]] is not implemented.

We have also a lot of %5D%5D at the end of the links, I don't know why. I can try to solve these problems with some regex.

--
You received this message because you are subscribed to the Google Groups "OpenRefine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ettore RIZZA]

This Ruby tool seems better. I'll give another try with it : https://github.com/weakish/creole2md

[Thad Guidry]

There's also this Python one.  https://gist.github.com/snopoke/39a062b334473b18ddb0

To unsubscribe from this group and stop receiving emails from it, send an email to openrefine+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "OpenRefine" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openrefine+...@googlegroups.com.

[Thad Guidry]

https://pythonhosted.org/Markdown/

[Ettore Rizza]

I moved the markdown files into a new repo. There are still a lot of problems with the links. Very often, the link and its url are reversed. Example:

[http:_www.somelink.com_][link+description]

We have to re-invert them, remove some undescores and escape characters... A little script is needed for that.

[Thad Guidry]

Ettore,

Sure if you can do that script in Python and save it in your Github would be nice.

[Owen Stephens]

I've setup some documentation and a workflow to start on the first phase of improving the OpenRefine documentation on the github wiki. If you'd like to contribute to the project - whether that's just commenting on the proposal, or doing some of the work, please have a look at:

https://github.com/OpenRefine/OpenRefine/wiki/OpenRefine-Documentation-Improvements-background-and-resources

This describes the aims for this phase of the project (improve navigation, make sure documentation we already have is relevant and current, move to using markdown for pages) and links to:

• A navigation and structure document for the new documentation (this proposes a navigation/structure for the documentation an is what I'll implement unless anyone chips in with comments/alternatives/suggestions)
  
• A workflow document for phase 1

In addition I've setup a tracker spreadsheet so if multiple people start contributing to the effort we can track what is happening - this is linked from the Workflow instructions

The tracker spreadsheet is a Google Sheet for structure

The navigation and structure document is a Google doc to enable commenting

All other documentation is on the Github wiki (natch!)

Best wishes

Owen

[Owen Stephens]

I've now moved all the documentation on the wiki aimed at developers into the new structure. You can see how this works by visiting

https://github.com/OpenRefine/OpenRefine/wiki/Documentation-For-Developers

and looking at the navigation bar on the righthand side which replaces the usual 'list of all the pages in the wiki'. Even if you aren't a developer you might want to have a look and comment (here) on if you feel the structure and navigation works, as my next step will be to do the same for the user facing documentation.

In addition to implementing a new structure & navigation, I've updated documentation to varying degrees as I've worked through it. This includes removing all mentions/examples related to Freebase, removing anything I know to be incorrect, adding detail where I'm easily able to do so and changing to using Markdown as the page syntax.

Comments welcome.

Owen

[Ettore Rizza]

It's much cleaner! Thanks Owen. I will try to take a few hours this week to fix the problem of Creole -> Markdown conversion.

[Thad Guidry]

Good Job Owen so far !  Keep it going ! :)