[TW5] TWederation edition progress

[Jed Carty]

At the moment everything is ready but automation and documentation. If you want to poke it and see what happens I have the most recent version here.

To make up for the near complete lack of documentation, here is a quick run down of how it works:

Make a copy of the wiki wherever you want your TWederation wiki to be (or just put a wiki there and import the plugins, all 4 of the plugins that start with $:/plugins/inmysocks/TWederation-)

Fill out the form that appears as the default tiddler ($:/plugins/inmysocks/TWederation/Initial Configuration/Create twCards) and click on 'Create twCard' at the bottom. Then go to to the Communication (in the table of contents tab on the right)

In that tiddler, go to the Fetch Communication tab, next to What to Fetch select All Wiki twCards and next to Select Wiki select TWederation, ignore the other things and click on 'Request Bundle'

In a moment there should be a link under the Request Bundle button, open that tiddler. In that tiddler click on Unpack All

Then in the Communications tiddler change What to Fetch to All Identity twCards and click on Request Bundle again, a new link will appear in a moment (so now there are two of them), click on the new one and click on Unpack All in the tiddler that opens.

You now have what you need to leave or fetch messages from whoever else has been listed. It may just be me.

Then in the fetch communication tab you can switch to blog posts, messages or whatever you want to try fetching. Under the other tabs you can view messages or posts or make posts and leave messages for other people.

Note I said 'leave' not 'send', this is a distributed system, not a client-server architecture. So you need to fetch messages or posts from people and they need to fetch them from you. In the future that will be automated so there will just be a 'check message' button that will get them for you.

You can fetch the twCards from TWederation again later to check if other people have been added. Right now without the automation it is all a bit tedious, but it works.

Some other notes:

When you make the twCard at first it also sends an xmlhttprequests that submits some information in a google sheet so that I can then retrieve that data and make the twCards from it. So I have made a widget to submit google forms and one to retrieve data from a google sheet. Unfortunately they aren't very flexible at the moment so if you want to do something else with them you will have to play with the code a bit.

[Hegart Dmishiv]

Hi Jed,

I've finally gotten around to having a go with the Tiddlyverse. My test wiki is called Twederation Test Hegart Dmishiv. I've filled in the form, but clicking on the "Create twCard" button doesn't seem to do anything, so I can't go any further.

Hegart.

[Jed Carty]

It did everything it was supposed to do, it just doesn't do it visibly. I will have to add some visible indication. You are listed in the google sheet and now and on the twederation.tiddlyspot.com
Although it looks like there was something wrong with the name because you are named 'undefined'. I will look into it tomorrow.

[Hegart Dmishiv]

Okay, thanks for that Jed. I still don't seem to have a $:/twCard tiddler yet in my wiki, and following the next steps to get the bundle of  All Wiki twCards  doesn't seem to work yet either.

Hegart.

[Jed Carty]

I am not sure why it isn't fetching bundles correctly. My best guess is that there is some problem with the https vs http interaction. Which is going to be annoying.

The tiddler listed there has the wrong name, there are two of them, $:/SitetwCard and $:/SiteAuthor, although the second one did get created on yours.

Of course as soon as someone else sees it there are a bunch of problems to fix. I will see what I can do.

[Hegart Dmishiv]

No problem, please do let me know of your progress. I'm keen to see this thing working. I'm happy to either apply any fixes you suggest, or just trash my current one and start again. Whatever will get it going properly.

Hegart.

[Jed Carty]

It can't open the iframe when hosted in dropbox, this is because dropbox uses https and my sites and tiddlyspot do not. Unfortunately if this is the case than the two biggest places where people will be hosting wikis won't be able to talk to each other since it looks like dropbox and tiddlyspot are by far the easiest places to host a wiki. I haven't tried communication between two dropbox hosted wikis, but I can connect to your wiki on dropbox from a non-https site, so the problem is only from an https wiki pulling things from a non-https wiki.

I heard somewhere that github may have (or may have in the future, I don't remember) free hosting services we could use with https so this doesn’t mean that it won't work but it is going to be a pain to work around.

Other than that things should work.

I am planning on working on this for today. I don't think I will make any progress on the https-vs-http problem but I will write up documentation and other tools so it should be usable.

So, if you are going to use it follow the instructions please, people keep trying to use dummy urls for their wikis. Don't do that, it won't work. For now put the wiki on tiddlyspot and copy the url from the browser bar. I will look into what can be done about the http vs https problem, but it may have to be something like bridge wikis on https servers that manually get content from other peoples http hosted wikis or something equally annoying.

If you want to contact me I should be on gchat or in the tiddlywiki irc channel.

[Jeremy Ruston]

Hi Jed

It can't open the iframe when hosted in dropbox, this is because dropbox uses https and my sites and tiddlyspot do not. Unfortunately if this is the case than the two biggest places where people will be hosting wikis won't be able to talk to each other since it looks like dropbox and tiddlyspot are by far the easiest places to host a wiki. I haven't tried communication between two dropbox hosted wikis, but I can connect to your wiki on dropbox from a non-https site, so the problem is only from an https wiki pulling things from a non-https wiki.

Ouch. One possibility is to use a service like https://www.cloudflare.com as an HTTPs proxy. I use it to give me https://federatial.com pointing to GitHub Pages http://federatial.github.io.

Best wishes

Jeremy.

I heard somewhere that github may have (or may have in the future, I don't remember) free hosting services we could use with https so this doesn’t mean that it won't work but it is going to be a pain to work around.

Other than that things should work.

I am planning on working on this for today. I don't think I will make any progress on the https-vs-http problem but I will write up documentation and other tools so it should be usable.

So, if you are going to use it follow the instructions please, people keep trying to use dummy urls for their wikis. Don't do that, it won't work. For now put the wiki on tiddlyspot and copy the url from the browser bar. I will look into what can be done about the http vs https problem, but it may have to be something like bridge wikis on https servers that manually get content from other peoples http hosted wikis or something equally annoying.

If you want to contact me I should be on gchat or in the tiddlywiki irc channel.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/d335e294-8b53-4ef3-8aca-d3de6e18d01e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mat]

Jed Carty wrote:

It can't open the iframe when hosted in dropbox, this is because dropbox uses https and my sites and tiddlyspot do not. Unfortunately if this is the case than the two biggest places where people will be hosting wikis won't be able to talk to each other since it looks like dropbox and tiddlyspot are by far the easiest places to host a wiki.

Hmm... that's a biggie. What does it take to introduce https on a server like tiddlyspot? And if any of the cousins Baird are reading this, is this anything that could be imagined? The context is to get TWederation working, i.e communication between tiddlywikis (we're on the verge, or even beyond the verge  - and it will revolutionize TW :-)

<:-)

[Mark S.]

Usually it's no biggie to turn on https -- the problem comes if you want a certified SSL. Without certification, each user has to tell his browser (which, if FF, will complain loudly) that he/she will accept the uncertified communication.

The certificates used to run around $100 a year -- for the cheap ones.

Mark

[Mark S.]

Since it's only fetching, Is there a way to make it fetch from the desktop? Then you could fetch tiddlers, edit, and repost to dropbox or wherever.

If everyone ends up at tiddlyspot, on the same server, it's not really all that distributed, is it?

Thanks,
Mark

[Jed Carty]

If by fetching from the desktop you mean local file:// urls then yes, it works with them. 

[Thomas Elmiger]

Hi. There are free certificates now from https://letsencrypt.org/getting-started/ – From their website: Let’s Encrypt is a free, automated, and open certificate authority brought to you by the non-profit Internet Security Research Group (ISRG).

In Switzerland we have providers offering such certificates for free to their customers, I believe. (I never used one myself.)
Thomas

[Mark S.]

But can your federated desktop TW fetch from the external http:// TW's ?

Thanks,
Mark

[Jed Carty]

If you are online yes it can. The only problem is that a wiki hosted on an https server can't pull from non-https sources without something else. 

[Mark S.]

Well, I've followed the steps, but can't seem to find a single post.

Under posts, only inmysocks is showing up, but no associated posts.

In Fetch Communications I can see Mat and others. I also see Twederation posted multiple times, so not sure what that's about. When I try to select all bundles (after the initial communication) I get the red screen of embarrassment  or a message which seems to suggest that it can't see *my* bundle.

Trying to grab individual bundles doesn't result in any messages showing up. Overall it's very random, and I don't seem to have a single posting or blog so far.

Good luck!
Mark

[Mark S.]

Posting.

In messages/short messages the drop down to and from show

-
<blank>
Inmysocks

I don't see my own card (?) to use for the from, and there are no other cards/labels to "send" to.

Looking through the tiddlers, I see only inmysocks as an identity tiddler, though there are cards  for other participants. Is this because of something they (including me) failed to fill out?

Speaking of filling out, on the opening page there was a button like

1b. Optional  - nice if you include!
{Button "Open" <UNREADABLE CHARACTER>}

The unreadable character was somewhat off-putting. It gives the page a broken feel. Whatever it was meant to be, maybe it's better to leave off for now?

Good luck!
Mark

[Mark S.]

I see that "twederation" shows up 3 times on the communcation list, probably because the url is in 3 forms

http://twederation.tiddlyspot.com/
http://twederation.tiddlyspot.com
twederation.tiddlyspot.com

[Mark S.]

Suggestion: Until the process is automated, add a contact email address next to the "list admin" so that it's clearer that something is required of the user.

[Mat]

Hi Mark. 

The TWederation pieces are currently under reworking/restructuring so there is much that has to be improved and corrected. It is not functional yet as you note but it is *really* great to see that you're interested in testing out stuff! 

<:-)

[Jed Carty]

Mark,

In addition to what Mat said about us being in the middle of updating things, it looks like you have an old version. The blank line is supposed to be your card but because of some bugs it isn't put in the correct place for your version. You can try getting what is on http://ooktech.com/jed/ExampleWikis/TWederation/ but I would advise against doing too much with it for a little while. Hopefully in the next few hours, or by tomorrow evening depending on how much life tries to get in the way, the basic user parts should be functional.

The reason everything is a mess at the moment is because me and Mat decided yesterday to restructure some of the basic things about twederation and I haven't gotten everything updated yet. Hopefully the new changes will make it easier for other people to create their own things on top of what I have made so people don't have to wait for me to do anything.

Since there are now at least three people looking at it I have a bit more motivation to make it work in a timely fashion.

[Mark S.]

But I got the TW from your first post in this thread!

Do you have a process for updating one's twederation? Or do we start from scratch?

Hopefully you got my request to be on the list -- I had it bounce out twice this AM.

Thanks,
Mark

[Mark S.]

More feedback:

1. Required - accept the pre-filled information or adjust it

My form was not pre-filled.
Thanks,
Mark

[Mark S.]

Comment about bundles.

After you have more than a half dozen bundles, it starts becoming difficult to tell which ones are new -- unless you're into memorizing 12 digit numbers on the fly. The date format seems weird -- a reverse of the time stamp which means that you can't depend on the newest one to be at the top or bottom of the list.

Thanks,
Mark

[Jed Carty]

I have made a bit of progress with this, now it should be much easier to figure out how to set things up. The problem with http vs https is still there, so hosting on dropbox isn't going to work out well unless you have some way to get around this problem.

I am sure that it will take about 30 seconds until the first person who tries it finds some error that keeps everything from working, but if you want to play with it you should copy the wiki here and put a version up on tiddlyspot or your own server. Nothing interesting is going to happen if you give it a fake url, or if you don't have it online.

At the moment it is set up with a simple thing that lets you make blog-ish posts and share them, and people can comment on them. The process for fetching comments other people make on your posts is a bit unreliable at the moment since it tries to fetch from multiple wikis at the same time. I will figure out how to improve it at some point.

[Mark S.]

The interface is nicer, but I still don't seem to be able to pick up any blogs or messages. There are occasional red embarrassment messages, though not sure about a consistent pattern yet. 

I was able to bring in a bundle from Mat's Place using 'messages'. The bundle was called "Default" without the date stamp, and it clearly contained a lot of text, but when I unpacked it, I couldn't find that the message(s) were made available anywhere.

If you tell it to bring in bundles or messages from "all", it gives you an error message when it gets to your own Tweder. Maybe it should skip your own tweder, since that's redundant.

Not sure how to proceed with testing, or what I should expect to work.

Good luck,
Mark

[RichardWilliamSmith]

Hi Jed,

I'm going to play with this when I get time. 

Have you looked at Neocities as a hosting option? They have https by default and a generous free tier. The guy who runs it is pretty forthright in his support of a free and open web - he is even integrating his site with ipfs. If it works as a host, it would be very quick for other people to set up an account and they would be able to host multiple wikis in the same folder, alongside any other content hey have.

Regards,

Richard

[Mat]

On Monday, June 13, 2016 at 1:36:23 AM UTC+2, Mark S. wrote:

The interface is nicer, but I still don't seem to be able to pick up any blogs or messages.

I'm not sure there are any good places to pick up from (or what do you say @Jed?). So far the other TWs are mostly stuff for testing, and most of it is a bit old, so you can't rely on them. 

@Jed, please remove all the twCards for, at least, my test sites from the public stuff. There should probably be one or several official test sites to fetch from.

<:-)

[Birthe C]

Hi Mat,

I tried to fetch all twCards, and got 3 x twederation and 2 undefined. Nothing really to fetch. So I really do not know how to continue. I did not see any twcCards or material from you, Mat. I did not experience any errors, as Mark described. Following one of the undefined - but link on twCards...I think it was Marks site I saw. His version is definitely not the one Jed linked to yesterday.

Birthe

[Jed Carty]

To be clear because people are using old versions, you have to use the version from http://ooktech.com/jed/ExampleWikis/TWederation/ apparently some people are trying to use other versions which won't work.

Mark,

You are the third or 4th person, including me and Mat, to try this out so there isn't really anyone posting yet aside from the test things I have put up. At least one person has posted and commented successfully so hopefully this will continue. What are you doing when you get the javascript errors (the red boxes). While I have found plenty of other problems I haven't been able to find any javascript errors in my tests.

[Jed Carty]

Birthe,

Thank you! I saw that you made a post and commented on one. It worked with a bit of poking and I think I know where the problems are now. From a helping out point of view you did exactly what you should do. I responded to your comment so if you want to see if you can get that you could try, but don't expect it to work correctly. I need to change how the wiki ask for comments, at the moment it asks all wikis if they have comments all at once and only manages to hear a few of the replies. I have some ideas on how to fix this.

So once again, thank you! You have been a huge help.

[Birthe C]

Hi Jed,

You were right, I could not fetch your comment to my blogpost.
That is until a few minutes ago! It feels so lovely to see it work. Thank you!
For the fun of it I commented your comment, now we will have to see about that ;-)

Birthe

[Jed Carty]

Unfortunately it isn't wizardry on my side, it was just random chance that the comment from me was one of the bundles that made it thourgh without colliding. I am working on the problem but haven't fixed it yet. I am very excited about all of this.

[Ste Wilson]

As a non tech tinkerer I followed the link and had a go. I think I ended up setting up my tiddlywiki from inside jeds. Not sure what polygons or tiddlers I need to drag across to my tiddlywiki.

[Ste Wilson]

Plugins not polygons. Damm you auto correct.

[Jeremy Ruston]

Hi Richard

On 13 Jun 2016, at 04:17, RichardWilliamSmith <RichardWi...@gmail.com> wrote:

Have you looked at Neocities as a hosting option? They have https by default and a generous free tier. The guy who runs it is pretty forthright in his support of a free and open web - he is even integrating his site with ipfs. If it works as a host, it would be very quick for other people to set up an account and they would be able to host multiple wikis in the same folder, alongside any other content hey have.

I have indeed been investigating Neocities (https://neocities.org). I got as far as uploading a TW5 document and finding that it didn’t work, but intend to return to it once 5.1.12 is out.

http://tiddlywiki.neocities.org

Interestingly, Neocities permits either HTTP or HTTPS to be used, so a wiki hosted there would be accessible to both twederation universes.

Here’s more about their interest in ipfs: https://neocities.org/permanent-web

And here’s more about ipfs:

https://ipfs.io

https://ipfs.io/ipfs/QmNhFJjGcMPqpuYfxL62VVB9528NXqDNMFXiqN5bgFYiZ1/its-time-for-the-permanent-web.html

Best wishes

Jeremy

[Mark S.]

Hi Jed,

Please remove my card for twederation.html -- installed next gen as twederation2.html.

As I mentioned, I could see a bundle coming in from Mat that obviously had lots of text in it.  But have no idea where to look. It may have been a much older message, since I could see creation dates in January.

If each person made 2 messages and/or blogs, there should be at least 4 messages at this point. Not sure about difference between Blogs, Messages, and Short Messages.

Just got a red screen with message: TypeError: iframeInfo.domNode.contentWindow is null
Was trying to messages from all Wikis.

I've just retrieved the "short demo" bundle. But it doesn't show up under Posts no matter how many drop-downs I look at.

Mark

[RichardWilliamSmith]

Hi Jeremy,

Worked for me without complaint - http://elsewhere.neocities.org/tiddlywikicom.html and I find that I am able to access it at both http and https immediately - https://elsewhere.neocities.org/tiddlywikicom.html

Initially it seems that the daily ipfs cache of neocities sites will store each one as an archive blob (I'm not sure of this) but if, eventually, they archive individual files, then it would be very easy to get the power of ipfs working for us. I am excited about the possibility of having a 'tiddler manifest' document which then pulls all of the tiddlers from IPFS individually. To me the idea of distributed/permanent/versioned content (including code) and Tiddlywiki as a trusted personal tool for authoring and consuming that content is very interesting; ipfs is the ultimate way to 'set the tiddlers free' and tiddlers may even prove to be a useful paradigm in themselves for thinking about truly distributed content.

Regards,

Richard

[Birthe C]

Hi Mark,

I am surprised, that you seem to see things, I do not after installing twederation. When I ask for all twCards I do not get any from Mat, I do get the ones listed http://twederation.tiddlyspot.com/#Wiki%20Listing.
Admittedly I have only been able to fetch from Jed.

If each person made 2 messages and/or blogs, there should be at least 4 messages at this point. Not sure about difference between Blogs, Messages, and Short Messages.

You are so right. I have not been able to fetch anything from you though.


Birthe

[Mat]

Birthe and Mark,

If I understand, there's currently an issue in that fetching stuff "simultaneously" makes them collide so that not everything comes through properly. I think it can be compared to several people uploading at once to a site - only the latest is what ends up there. The Jedi master is on the case though.

Please note that posts and stuff from a few months ago are not reliable anymore.

<:-)

[Mark S.]

Trying to fetch all messages or all blogs will pretty unfailingly give me the red screen with something about iframe.

I think it is hiccuping because it's reading my own card, but not sure.

Is the "-" on the various view messages supposed to allow you to see all?  There doesn't seem to be any practicable way to see all messages at once. Trying to review every to/from combo would quickly become unsupportable.

In the posts section, I'm finally able to see activity from Jed's test 1 and Birthie. I've posted comments. I've also posted two blog entries, but haven't seen any comments. Then again, maybe I don't know how to fetch them.

In the post view section, having to select every author in order to check if there are new posts will become tiring when there are more than a handful of authors.

[Birthe C]

Hi Mark,

Yabadoo...I got two blog posts from you. When I tried to fetch blogpost from all, I got everything from Jed and nothing from you. But fetching from your card only....success. I have commented, so maybe you will be able to get it. I have only had one success doing that...by pure chance.

I also get the error message not able to read from myself. I just have to click close to that. No red screens.


Birthe

[Jeremy Ruston]

Hi Richard

Hi Jeremy,

Worked for me without complaint - http://elsewhere.neocities.org/tiddlywikicom.html and I find that I am able to access it at both http and https immediately - https://elsewhere.neocities.org/tiddlywikicom.html

Excellent news, thank you.

Initially it seems that the daily ipfs cache of neocities sites will store each one as an archive blob (I'm not sure of this) but if, eventually, they archive individual files, then it would be very easy to get the power of ipfs working for us. I am excited about the possibility of having a 'tiddler manifest' document which then pulls all of the tiddlers from IPFS individually. To me the idea of distributed/permanent/versioned content (including code) and Tiddlywiki as a trusted personal tool for authoring and consuming that content is very interesting; ipfs is the ultimate way to 'set the tiddlers free' and tiddlers may even prove to be a useful paradigm in themselves for thinking about truly distributed content.

Well put; there's a great appeal to assembling the wiki from individual tiddler files. I did some experimentation with Dropbox's JavaScript API a couple of years ago and concluded that performance was a bit of an obstacle: the trouble is that HTTP is an expensive protocol, hence all the various hacks (like image sprites) that are designed to help pack multiple resources into a single network request. There is also an issue with atomicity of writing to multiple files; it's hard to cope elegantly with the browser tab being closed (or network connectivity being lost) in the middle of a save/sync operation.

Meanwhile, and somewhat orthogonally, one of the characteristics of TiddlyWiki that makes it particularly interesting for experimenting with distributed webby stuff is actually that it (can be) a completely self contained single file. For systems like Tahoe-LAFS and (possibly IPFS) the problem of serving a single file is a bit easier than serving multiple files while maintaining the same relative addressing.

Best wishes

Jeremy.

Regards,

Richard

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/7f5c7679-a029-4e70-80a9-6f9781b3de78%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jeremy Ruston]

By the way, here's an excellent technical blog post about the difficulties of making a distributed HTTP:

https://www.mnot.net/blog/2015/08/18/distributed_http

Best wishes

Jeremy

--

Jeremy Ruston

jer...@jermolene.com

http://jermolene.com