[TW5] Documentation Contributors Edition

[Jeremy Ruston]

Following up some recent discussions about making it easier for people to contribute improvements to the documentation on tiddlywiki.com, I've been experimenting with a new edition for documentation contributors:

http://tiddlywiki.com/prerelease/editions/tw5.com-docs/

It's based on a new plugin $:/plugins/tiddlywiki/tw5.com-docs that contains all the documentation tiddlers from tiddlywiki.com. The advantage is that it's easier to tell which tiddlers you've modified, and to revert to the original.

The idea is that a would-be contributor would clone this edition to TiddlySpot (step-by-step instructions forthcoming), and then make their changes, saving them back to TiddlySpot. I, or anyone else, can fairly easily then pick up the wiki and extract the changed tiddlers into a pull request.

One shortcoming at the moment is that TiddlyWiki gives a warning each time you modify a shadow tiddler.

A desirable feature is the ability to see line-by-line diffs for the changes, and perhaps to be able to easily export a JSON file of the changed tiddlers.

Anyhow, feedback and questions are welcome,

Best wishes

Jeremy.

--

Jeremy Ruston
mailto:jeremy...@gmail.com

[Mat]

Interesting idea! :-)

The idea is that a would-be contributor would clone this edition to TiddlySpot (step-by-step instructions forthcoming), and then make their changes, saving them back to TiddlySpot. I, or anyone else, can fairly easily then pick up the wiki and extract the changed tiddlers into a pull request.

Unless it will be included in the coming instructions; are you thinking one could/should have a more permanent "matsproposals.tiddlyspot.com" to modify - or does the TS become obsolete after you, or anyone else, has extracted the changes into a pull?

Is there any way to "mark" such a TS as "done"? Maybe the password should be shared so it? We're not talking secret stuff here.

Particularly, if the pull request is denied, is there any way to find out?

Maybe the edition could contain a link to the folder on GitHub where the issue will be treated?

Are there restrictions to what kinds of modifications one should make in this?  You refer only to documentation (which I, btw, interpret as the descriptive texts but also eventual code in the tiddlers these texts are in... macrocalls etc). 

Also, I typically use my SideEditor. What are your thoughts on the state of the uploaded TWs? Ideally it would suit both you and the contributors.

<:-)

[Hegart Dmishiv]

@Mat, it would seem to me that, once the merge is completed, your own TiddlySpot wiki would need to be purged and replaced with a fresh one, downloaded again from TiddlyWiki and uploaded to your own TiddlySpot, containing all the merges from yours and other documentation editor's work, before you continue editing the TW documentation again. I'm only very new to all of this, but that's the way I understand it. If you're downloading from http://tiddlywiki.com/prerelease/editions/tw5.com-docs/, then you're getting the freshest stuff available, even before @Jeremy rolls out a new version. I like this idea, as much of it as I understand.

[Tobias Beer]

First of all, this is a very helpful idea and shall simplify both the process of documentation as well as distributing the results. 

I believe, after having made edits, one would submit the contributions to either a GitHub issue or a groups post, whereas the former is more easy to track for Jeremy while the latter might take longer to get picked up and processed.

At that point, nothing should be done with that wiki either until the changes are accepted AND merged, or further modifications are asked for. Come to think of it, for tracability and to ensure that a wiki is in a frozen state, it could be a good idea to actually attach that file to the group post or github issue, rather than point to a TiddlySpot or any other location on the web. This avoids any troubles over flagging doc-wikis as "done". What gets set to done, merged, closed or completed would be the group post or github issue, which is also the place for notifications if edits should ever be denied.

Apart from the documentation, the easiest would be to not have any other stuff in the wiki. This way it would be easy for Jeremy to track which Doc-Plugin-Shadows were edited and which other tiddlers were created. A desire to delete a tiddler could be done simply by replacing the content with DELETE but should possibly be discussed anyway.

While you might reuse a location for future edits, you must not reuse the wiki, imho.

I'm pretty sure that Jeremy will provide some "doc-editor's edition" to download from which you can start making your changes. This will contain a documentation plugin having a readme that will tell you about how to go about editing and submitting. The scope of your edits is basically the content of that plugin or any new tiddlers you create.

If you wished to use a special edition to make your edits, e.g. by importing stuff, you would need...

• to indicate that
• a simple list with links to any new tiddlers, otherwise it would be difficult to separate additions from editing tools

What you should be mindful about is to not use any of those features in the documentation, otherwise you need to actually submit them as contributions.

This could also cater for a kind of personal documentation wiki where a user can collect personal notes on documentation tiddlers, while showing the plugin-shadow-contents underneath. One way would be to actually replace the shadow contents with your notes and have some view-template magic show the original shadow contents nonetheless.

Last but not least, I don't see the alert that pops up for editing shadows as problematic at all.

— tb

[Hegart Dmishiv]

Oh boo, I've just found out that uploading a TW wiki to TiddlySpot, at least the first time, requires the Google Chrome browser. Everything "Googlish" that I use at present, starting with my Gmail email account, Groups, Google+ profile, Drive, and even their link shortner, I've only started using since I found TiddlyWiki. Normally I avoid anything to do with Google altogether. I definitely won't be installing Google Chrome browser on my home computer. Ever. I wouldn't even want to install it temporarily, just to be able to upload this TW documentation wiki to TiddlySpot the first time, then uninstall Chrome. As I mentioned in my previous post, my understanding of what Jeremy has proposed is that TW documentation editors will need to purge, re-download from TW and re-upload to TS after each merge. I may be wrong on that, but if it is the case, then I'd need to keep using Chrome for the first upload to TS each time.

[Tobias Beer]

I definitely won't be installing Google Chrome browser on my home computer.

Consider either of these:

• chromium
  
• srware iron 

— tb

[Mat]

On Wednesday, September 16, 2015 at 6:52:27 AM UTC+2, Tobias Beer wrote:

First of all, this is a very helpful idea and shall simplify both the process of documentation as well as distributing the results. 

I believe, after having made edits, one would submit the contributions to...

Let's not forget possibilities with a tw-aggregator :-)

The plugin could be such that it per default is reported to an aggregator and in there, the tiddlers can be sorted by modification time. 

Come to think of it, this could probably also filter out any undersired stuff like user-added plugins 

We'll see what Big J says ;-)

 

<:-)

[Mat]

On Wednesday, September 16, 2015 at 7:07:37 AM UTC+2, Hegart Dmishiv wrote:

Oh boo, I've just found out that uploading a TW wiki to TiddlySpot, at least the first time, requires the Google Chrome browser.

Yeah. Unfortunate, but please don't forget that whatever IS working is thanks to individuals just like you and me here in the community - with the addition that they've really put in their heart to build something for us.

 

Everything "Googlish" that I use at present, starting with my Gmail email account, Groups, Google+ profile, Drive, and even their link shortner, I've only started using since I found TiddlyWiki.

Yeah... same here. But actually, in my case it is mainly because FF can't handle my load of tabs without crashing.

Regarding setting up TiddlySpots, I've optimized a routine to at least make it as quick as possible. I believe I described it the other day so it's in the recent discussions.

But it should be emphasized, it is only for the first upload to TS that you can't use FF.

<:-)

[Hegart Dmishiv]

On Wednesday, September 16, 2015 at 5:50:28 PM UTC+12, Mat wrote:

Yeah. Unfortunate, but please don't forget that whatever IS working is thanks to individuals just like you and me here in the community - with the addition that they've really put in their heart to build something for us.

Sorry, no disrespect intended, Mat. My complaint was more about Google allowing something willy-nilly, which Firefox treats with suspicion. It wasn't meant to be denigrating how TiddlySpot itself works.

Regarding setting up TiddlySpots, I've optimized a routine to at least make it as quick as possible. I believe I described it the other day so it's in the recent discussions.

Yep, thanks, I saw that and gave it a go. I've registered an account on TiddlySpot, but currently it only has the default TW wiki on it (whichever version that is), as Firefox won't allow me to upload the latest version. I've put all the necessary system tiddlers in and saved and refreshed, it's just that NS_ERROR_DOM_BAD_URI error preventing me from goin any further with it.
 

But it should be emphasized, it is only for the first upload to TS that you can't use FF.

I might go down to our local public library at some time and use their computers for this. They'll no doubt have Chrome installed. I just chuck my KeePass password database onto a geek stick and can then access sites from other people's computers. I'm careful to not allow their browser to save my passwords.

#cybersecurity

 

[Tobias Beer]

We'll see what Big J says ;-)

I think we're going to take one step at a time. In other words, I don't see an automated aggregate-contributions-and-provide-a-workflow-to-figure-out-what-to-merge-and-how-and-why-thingy coming up soon. These things never turn out as "simple" as one would initially imagine. ^_^ 

— tb

[Birthe C]

Hi Tobias

You are right. I think it has to be made as simple as possible, not to add unnecessary workload on the person (I suspect Jeremy], who would have to collect the documentation suggestions.

That would explain:

A desirable feature is the ability to see line-by-line diffs for the changes, and perhaps to be able to easily export a JSON file of the changed tiddlers.

Birthe

[Tobias Beer]

Hi Birthe,

 

A desirable feature is the ability to see line-by-line diffs for the changes, and perhaps to be able to easily export a JSON file of the changed tiddlers.

I suspect something like this already exists outside of TiddlyWiki in how Jeremy handles translations, although I wouldn't know the exact method / workflow / effort.

Best wishes,

— tb

[Erwan]

Hi Jeremy,

I don't have a clear understanding of all the parts in this process, but if the/an aggregator can help at least for some of the parts I'd be happy to give it a try.

I started recently using git with the CommunitySearch aggregator not only to publish the standalone html file but also to track changes in the content (one of the ideas is to be able to output statistics over time, but nothing specific is implemented yet). As far as I can tell, it seems to work: if you go to https://github.com/erwanm/tw-community-search/tree/automatic/tw-community-search you can see the diff corresponding the latest automatic update (the diff is actually very big because the tag tiddlers are re-generated every time, it's something I'll try to solve in the future).

So basically I think it's possible with an aggregator-like system to:

1) download a wiki "mydoc" from tiddlyspot
2) convert it to node.js
3) pull the official TW github repo (or maybe only the documentation part, if possible)
4) replace the official TW tiddlers with the "mydocs" tiddlers (including the ones left unchanged)
5) commit changes to the local git repo

I think that leaves us at least with the line-by-line diff, and maybe it can be taken further by pushing these changes... but I don't know where/how exactly, or how to deal with conflicts?

Issues that I can think of:

* how to transmit the name/address of the wiki to the aggregator?

* if the TW repo has changed since the revision uploaded to tiddlyspot, the commit would revert these modifications. as far as I know, the only way to avoid that is to know exactly which revision was used on the tiddlyspot wiki, but I don't see how this is possible.

* I'm not sure what to do at the end: is it possible to send a github pull request programmatically? or maybe the commit could be pushed to a special repo or branch?

* modifications from several wikis can cause conflicts as well

(disclaimer: I'm familiar with git but no expert)


Regards
Erwan

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/CAPKKYJbx%3D28BjWYwra3%3DWjmS0fd8FjKmT0JQ4uK%3DXwK-1f%2B9RA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Jeremy Ruston]

Hi Erwan

So basically I think it's possible with an aggregator-like system to:

Have you seen the process I use with the Translators edition; it's very similar:

http://tiddlywiki.com/prerelease/editions/translators/#Extracting%20Translations

 

* how to transmit the name/address of the wiki to the aggregator?

RSS solved this problem by having dedicated "ping" servers. In our case, wikis that want to be aggregated could perform an HTTP POST to https://ping.tiddlywiki.com, with the payload being the URL of the wiki.

That would only work if you were in a position to have a server that was up all the time. An alternative might be to re-use an existing infrastructure; for example, having wiki authors tweet their updates with a hashtag.

 

* if the TW repo has changed since the revision uploaded to tiddlyspot, the commit would revert these modifications. as far as I know, the only way to avoid that is to know exactly which revision was used on the tiddlyspot wiki, but I don't see how this is possible.

The core documentation plugin can be given a finer grained version number that maybe even includes includes the commit details.

 

* I'm not sure what to do at the end: is it possible to send a github pull request programmatically? or maybe the commit could be pushed to a special repo or branch?

Yes, there's an extensive API for GitHub, and it's quite easy to create pull requests automatically.

 

* modifications from several wikis can cause conflicts as well

For tiddlywiki.com itself, I think it's reasonable that the docs should have a manual editorial process. Some details of a contribution might be inaccurate; that's OK for a community wiki, but I don't think it's OK for the canonical reference documentation.to

 

Many thanks for your continued work on this stuff, it's developing nicely,

Best wishes

Jeremy.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/55F9C37B.6050608%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Erwan]

Hi Jeremy,

Thank you for the details.

I should be able to provide a script which handles the git stuff, but I would need some help with the rest, especially the question of receiving the request from the user.
In case anybody wants to help, including if somebody has a server and they would be ok to use it for this service, I opened an issue on my repo here: https://github.com/erwanm/tw-aggregator/issues/81

Regards,
Erwan

[Jeremy Ruston]

Hi Erwan

I should be able to provide a script which handles the git stuff, but I would need some help with the rest, especially the question of receiving the request from the user.
In case anybody wants to help, including if somebody has a server and they would be ok to use it for this service, I opened an issue on my repo here: https://github.com/erwanm/tw-aggregator/issues/81

I'm not sure that the request handling needs to be automated much. As I mentioned above, any contribution is going to have to be reviewed for accuracy and quality, so it's always going to be a semi-manual process.

Overall, I am beginning to worry that this is a bit of a rabbit hole. We're putting effort into engineering a whole new pipeline for contribution requests, somewhat duplicating GitHub. But we don't really know that the contribution requests we'll get will warrant the effort. And even though we're lowering the barriers to contribution, they are still quite high: somebody spotting a typo needs to find out how to clone the wiki, save their changes, and submit the change for review. That's conceptually quite complex. It still seems that the simplest flow for people to fix simple errors like typos is:

* Visit http://tiddlywiki.com

* Put tiddler into edit mode

* Click on the link to edit the tiddler on GitHub

* Edit the text

* Click a button

Clearly, most people would need to be taught how to perform those steps; but so they would if they went the TiddlySpot route.

Meanwhile, there are so many other things that we could put effort into:

* Proper multi-user support for the Node.js configuration

* Establishing a community documentation space

* Getting third-party plugins into the plugin library

* Building a better library of editions

* More cool features built on top of the community aggregator

* Federation

The minimum work to get this new idea of contributing docs changes via TiddlySpot is figuring out a process for generating a pull request. As stated above, I think a prerequisite is that the docs plugin includes the commit ID from which it was generated.

Best wishes

Jeremy.

[Mark S.]

I didn't find the Github process too arduous. What was problematic was waiting many weeks before simple, innocent documentation changes showed up or even were pulled. Apparently, documentation is treated as code (maybe because it's mixed in with the code). It doesn't seem like the proposed system will fix that.. 

Everyone in the TW github documentation system has to go through a signing/initiation process. They are essentially known quantities. So there shouldn't be serious trust issues.

It's odd that you can post to the world's largest online encyclopedia and see it up in 20 minutes, but you can't add a simple TW document page without waiting for weeks.

On a technical note, with the new process won't tiddlyspot become cluttered with dozens or hundreds of "dead" TW's after a year or two?

Mark
 

[Jeremy Ruston]

I think I've said before that the problem with recent content updates is that a mistake that I made at the start of 5.1.10 means that I can't update tiddlywiki.com content until the next release. During the next cycle we shall return to the usual situation where I can push content changes as they happen.

Best wishes

Jeremy

—
jeremy...@gmail.com

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/b8f43eb5-a6a8-40a6-b78b-038ba547878c%40googlegroups.com.

[Hegart Dmishiv]

Thanks @Jeremy, I was composing a long reply to @Mark S, linking to those previous discussions, but you replied before I had finished, making my reply moot. Here's the link instead to Jeremy's previous post on this issue.

It was interesting that Mark's argument was basically the same as mine, clearly without having read the previous discussions first.

[Hegart Dmishiv]

Hi @Mark S.,

On Wednesday, September 23, 2015 at 3:26:51 AM UTC+12, Mark S. wrote:

On a technical note, with the new process won't tiddlyspot become cluttered with dozens or hundreds of "dead" TW's after a year or two?

As I mentioned earlier in this thread, I'm pretty sure the idea is that, once you've submitted your change and it has been accepted, you'll need to download a fresh copy of Jeremy's tw5.com-docs instance, to get all the latest updates to the documentation, including those of other documentation contributor's. Then you just upload the latest one to your TiddlySpot before you continue contributing to the TW5 documentation. At least, that's how I understand it, I may be entirely wrong on this, but nobody corrected me when I mentioned it earlier.

[Tobias Beer]

Hi Jeremy,

 

During the next cycle we shall return to the usual situation where I can push content changes as they happen.

That is good to know!

Best wishes,

— tb 

[Matabele]

Hi

The process of exporting a tiddler to a .tid file, then attaching this file to a post on this forum is fairly straightforward. 

A possible interim solution:

1. Publish a periodically updated TW5-docs version from where contributors can obtain a fresh copy (as you have already done here.)

2. Set up a TiddlyWiki-Docs forum here on Google groups.

3. Place a few pinned threads at the top of the forum -- one for each documentation moderator (perhaps, one per language.)

4. A contributor then:

  (a) downloads a copy of the latest TW5-docs

  (b) modifies (or creates a new) tiddler

  (c) exports this tiddler in .tid format

  (d) attaches this .tid file to a post explaining the reason for the change in the appropriate thread of the TiddlyWik-Docs forum

5. The moderator then:

  (a) downloads the .tid file

  (b) drags it onto his copy of the docs TW

  (c) checks the content and if OK

  (d) updates the TW5-docs plugin

  (e) periodically posts the updated plugin to the TW5-docs edition

Looks to me that the TW5-docs plugin should, therefore, be broken into several sections -- one for each TW5-docs moderator.

regards

[Jeremy Ruston]

Hi Matabele

The process of exporting a tiddler to a .tid file, then attaching this file to a post on this forum is fairly straightforward. 

Another possibility is to use the "export all" button in the sidebar tools tab to export a JSON file of just the changed tiddlers.

Looks to me that the TW5-docs plugin should, therefore, be broken into several sections -- one for each TW5-docs moderator.

Things get a bit easier if we can assume that moderators are working in GitHub, even if the contributors aren't. 

Best wishes

Jeremy.

 

regards

On Tuesday, 15 September 2015 18:18:42 UTC+2, Jeremy Ruston wrote:

Following up some recent discussions about making it easier for people to contribute improvements to the documentation on tiddlywiki.com, I've been experimenting with a new edition for documentation contributors:

http://tiddlywiki.com/prerelease/editions/tw5.com-docs/

It's based on a new plugin $:/plugins/tiddlywiki/tw5.com-docs that contains all the documentation tiddlers from tiddlywiki.com. The advantage is that it's easier to tell which tiddlers you've modified, and to revert to the original.

The idea is that a would-be contributor would clone this edition to TiddlySpot (step-by-step instructions forthcoming), and then make their changes, saving them back to TiddlySpot. I, or anyone else, can fairly easily then pick up the wiki and extract the changed tiddlers into a pull request.

One shortcoming at the moment is that TiddlyWiki gives a warning each time you modify a shadow tiddler.

A desirable feature is the ability to see line-by-line diffs for the changes, and perhaps to be able to easily export a JSON file of the changed tiddlers.

Anyhow, feedback and questions are welcome,

Best wishes

Jeremy.

--

Jeremy Ruston
mailto:jeremy...@gmail.com

[Matabele]

Hi Jeremy

On Thursday, 24 September 2015 13:44:04 UTC+2, Jeremy Ruston wrote:

Hi Matabele

The process of exporting a tiddler to a .tid file, then attaching this file to a post on this forum is fairly straightforward. 

Another possibility is to use the "export all" button in the sidebar tools tab to export a JSON file of just the changed tiddlers.

That makes things easier -- it had escaped my notice that once the docs had been placed into a plugin, an 'export all' would export only modified documentation.

Looks to me that the TW5-docs plugin should, therefore, be broken into several sections -- one for each TW5-docs moderator.

Things get a bit easier if we can assume that moderators are working in GitHub, even if the contributors aren't. 

Again -- this streamlines the merge  process.

With these two modifications, the process should be fairly simple for contributors:

A contributor:

  (a) downloads and saves a copy of the latest TW5-docs

  (b) modifies (or creates) the relevant tiddlers 

  (c) exports the modified tiddlers in JSON format (with 'Export all tiddlers' button)

  (d) attaches this JSON file to a post (explaining the reason for the changes) in the appropriate thread of the TiddlyWik-Docs forum.

And the moderators picks up the process from there.

It seems that downloading JSON files from a Google group is easier for a moderator than visiting a list of blogspot wikis and, attaching a JSON file to a post is easier for a contributor than creating and updating a blogspot wiki. 

This also provides a forum for contributors to discuss their documentation modifications (with moderators and other contributors) before they post to the pinned threads. 

regards

[Erwan]

Hi Jeremy,

> Overall, I am beginning to worry that this is a bit of a rabbit hole.
> We're putting effort into engineering a whole new pipeline for
> contribution requests, somewhat duplicating GitHub. But we don't
> really know that the contribution requests we'll get will warrant the
> effort. And even though we're lowering the barriers to contribution,
> they are still quite high: somebody spotting a typo needs to find out
> how to clone the wiki, save their changes, and submit the change for
> review. That's conceptually quite complex.

You're right, and I must admit that it would be quite some work to make
the aggregator do only the partial process I described earlier, with
many possible problems which might scare users away.

I think that currently if someone who can't use github (or doesn't want
to) wants to modify the documentation, the best option is simply to send
their modified/new content to this discussion group, or even better to
tiddlywikidocs, where someone will pick it up and create a pull request.

when I was writing this I realized that I could propose a documentation
change myself with this idea, so I did:
https://github.com/Jermolene/TiddlyWiki5/pull/1969

Erwan

[Tobias Beer]

Hi Metabele,

 

It seems that downloading JSON files from a Google group is easier for a moderator than visiting a list of blogspot wikis and, attaching a JSON file to a post is easier for a contributor than creating and updating a blogspot wiki.

I would argue it complicates things, as you can't just click-open-see things but have to import them first... whereas in the GitHub workflow you get that diff the minute you look at the PR.

I think we should encourage using the GitHub Web-UI as much as possible ...and for anyone willing and able to do it properly.

Otherwise, there isn't even a need to do all of the above but just post the source of the desired changes right into a google groups post for anyone to see, marking any changes in bold. This makes everything much more visible than any attached files. For extensive changes, just don't use this process and go about it the Github way, otherwise it's just too much to deal with in terms of merging, reviewing, discussing.

Best wishes,

– tb