[TW5] WebDav Saver Observations.

[Lost Admin]

A while back, Jeremy mentioned in another post that TiddlyWiki supports saving via WebDav. I didn't know this existed until he mentioned it. I do like the idea. Most web server software has support for WebDav, or there are webdav add-ons. It means I don't need to run PHP (or node.js) on the server, which significantly reduces the resources needed on the server. So, I've been experimenting with WebDav. 

I've found that it works very well once the web server is set-up properly.

Using Lighttpd, it works without issues. Http Basic Auth (or digest auth) works without issues. I haven't tried, but I don't think there would be issues using client certificate authentication if one wanted that level of security on the network link.

Using Apache, it also works with basic or digest auth. One interesting difference, Apache seams to track the last time you loaded the file (get request) and compares that to the last file modification time-stamp. If the last modification is newer than when you loaded the tiddlywiki file, you get an error and the change doesn't get saved. 

You can (easily) get around this by reloading the file (browser reload/reread button) after you save the tiddlywiki file. It might be a bit annoying but it does solve the issue of conflicting edits if two people are working on it at the same time. On the down side, the 2nd person will have to re-load the file and re-do all their work (of course browser tabs help with that).

I don't use nginx, so I haven't tried that one.

[UPDATE]

PMario created some good videos on how to set up a webdav server on your Windows 10 desktop Using MS IIS for local WebDav saving. https://groups.google.com/forum/#!topic/tiddlywiki/_YwmiKqMyrI

[Arlen Beiler]

Does it work in both Chrome and Firefox?

On Aug 24, 2017 09:53, "Lost Admin" <thelos...@gmail.com> wrote:

A while back, Jeremy mentioned in another post that TiddlyWiki supports saving via WebDav. I didn't know this existed until he mentioned it. I do like the idea. Most web server software has support for WebDav, or there are webdav add-ons. It means I don't need to run PHP (or node.js) on the server, which significantly reduces the resources needed on the server. So, I've been experimenting with WebDav. 

I've found that it works very well once the web server is set-up properly.

Using Lighttpd, it works without issues. Http Basic Auth (or digest auth) works without issues. I haven't tried, but I don't think there would be issues using client certificate authentication if one wanted that level of security on the network link.

Using Apache, it also works with basic or digest auth. One interesting difference, Apache seams to track the last time you loaded the file (get request) and compares that to the last file modification time-stamp. If the last modification is newer than when you loaded the tiddlywiki file, you get an error and the change doesn't get saved. 

You can (easily) get around this by reloading the file (browser reload/reread button) after you save the tiddlywiki file. It might be a bit annoying but it does solve the issue of conflicting edits if two people are working on it at the same time. On the down side, the 2nd person will have to re-load the file and re-do all their work (of course browser tabs help with that).

I don't use nginx, so I haven't tried that one.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscribe@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/603ae837-720a-4f5c-a4e4-9de40d772783%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Lost Admin]

Yes. I've tested from Windows with Firefox, MS Edge, MSIE11, Chrome. All worked. 

I've also tested using Android Firefox and Android Chrome as well as Apple iOS Safari.

On Thursday, August 24, 2017 at 11:40:07 AM UTC-4, Arlen Beiler wrote:

Does it work in both Chrome and Firefox?

On Aug 24, 2017 09:53, "Lost Admin" <thelos...@gmail.com> wrote:

A while back, Jeremy mentioned in another post that TiddlyWiki supports saving via WebDav. I didn't know this existed until he mentioned it. I do like the idea. Most web server software has support for WebDav, or there are webdav add-ons. It means I don't need to run PHP (or node.js) on the server, which significantly reduces the resources needed on the server. So, I've been experimenting with WebDav. 

I've found that it works very well once the web server is set-up properly.

Using Lighttpd, it works without issues. Http Basic Auth (or digest auth) works without issues. I haven't tried, but I don't think there would be issues using client certificate authentication if one wanted that level of security on the network link.

Using Apache, it also works with basic or digest auth. One interesting difference, Apache seams to track the last time you loaded the file (get request) and compares that to the last file modification time-stamp. If the last modification is newer than when you loaded the tiddlywiki file, you get an error and the change doesn't get saved. 

You can (easily) get around this by reloading the file (browser reload/reread button) after you save the tiddlywiki file. It might be a bit annoying but it does solve the issue of conflicting edits if two people are working on it at the same time. On the down side, the 2nd person will have to re-load the file and re-do all their work (of course browser tabs help with that).

I don't use nginx, so I haven't tried that one.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

[PMario]

Hi LostAdmin,

That's a perfect reminder, to have a closer look at MS built-in stuff. WebDav is part of Internet Information Services (IIS), which seems to be available for up to 20 users for Windows Vista, 7, 8.x and 10. ...

I'm basically interested in a replacement for TiddlyFox and not in a "internet facing" setup. ...

from win10 user terms.

2.      Installation and Use Rights.

<snip>

d.      Multi use scenarios.

<snip>

(iii)    Device connections. You may allow up to 20 other devices to access the software installed on the licensed device for the purpose of using the following software features: file services, print services, Internet information services, and Internet connection sharing and telephony services on the licensed device. You may allow any number of devices to access the software on the licensed device to synchronize data between devices. This section does not mean, however, that you have the right to install the software, or use the primary function of the software (other than the features listed in this section), on any of these other devices.

So with the right settings, it may be possible to activate it for most windows machines. ... The important phrase is "may be", since nobody really knows, how  crippeld "Home xxx" and OEM Licenses really are.

For those who are courious, IIS is the equivalent of the Apache and Lighttdp servers, that Lost Admin mentioned. ... So it should be possible to use the stuff for local installations.

... I'm not at home at the moment. So I can't run tests with my machine. But anyone who is interested, should expolore the docs, run some tests and report back here.

Especially interesting: OS version used! So we can see, if it works with "Home xxxx" licenses. I do have a Win10 Pro, wich I can test on :)

have fun!
mario

[Arlen Beiler]

Can anyone test whether the WebDAV saver works correctly if the file names have spaces in them, and which server/browser combinations they work on?

Thanks,

-Arlen

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscribe@googlegroups.com.

To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/ce0059f5-2807-435f-b9c3-7ee49f45f0f6%40googlegroups.com.

[Lost Admin]

When I created a wiki with a space in the filename and used the webdav saver, it replaced the space with the URL encoding (%20) in the filename on save.

This was tested using my Apache setup from Safari on iOS. Haven't tested other browsers but I doubt they will be any different.

[Arlen Beiler]

And can you reload the renamed file and continue saving to it? Or not?

On Tue, Aug 29, 2017 at 7:59 PM, Lost Admin <thelos...@gmail.com> wrote:

When I created a wiki with a space in the filename and used the webdav saver, it replaced the space with the URL encoding (%20) in the filename on save.

This was tested using my Apache setup from Safari on iOS. Haven't tested other browsers but I doubt they will be any different.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscribe@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/7267242b-b825-453b-b575-5041f655543a%40googlegroups.com.

[PMario]

Hi foks,

I just did a proof of concept using IIS with WebDAV on windows 10 pro. .. It seems to work out of the box, with IE-11, Edge, FF55 and FF57-nightly.

I will record a short video, so everyone interested, should be able to get it going. ... There is a little issue, with the TW default saver, but it should be streight forward to  fix it.

have fun!
mario

[Mark S.]

What would the performance and safety concerns be for running IIS/WebDav on a semi-permanent basis? If you forward the ports (and know your IP) can you access your files outside of the home?

Mark

[Lost Admin]

Safety/security concerns with WebDav:

WebDav is conceptually a share network folder (so much so you can mount them with a drive letter on Windows) that is provided over HTTP. This means anyone who can access the webdav url can read, write, and delete to all the files available there. This includes making new directories.

To protect it, one typically uses HTTP Basic (or Digest) Authentication (part of the web server set-up). With basic authentication that means the password (and user name) are going across the network (including Internet when doing so remotely) in clear text and anyone who sees the traffic can read the login credentials. Using digest authentication reduces this risk as the password is no longer sent across the network. Usually it is recommended that you use HTTPS for webdav and not allow HTTP (unencrypted) connections. However, SSL/TLS has a lot of insecure configurations, so you need to know what you are doing (and what encryption protocols to allow).

Also, all the files that are stored on your hard drive for use with webdav need to be owned by the system user that the web server is running as. This makes it easy for a hacker who manages to breach your web server to mess with those files (website defacement happens because of this sort of thing).

Of course, several cloud providers do feel that they can sufficiently secure webdav and offer it as part of their service. box.com offers webdav access. Under the covers MS SharePoint (and therefore OneDrive) use webdav.

My home system has webdav exposed to the Internet (https only, basic authentication). my Wiki.suntrap.ca site also has webdav enabled.

If you are setting this up on your home PC for access from the Internet (with port forwarding), I strongly suggest setting it up with SSL/TLS and using digest authentication. Also, set-up a system account (not your user account and not the windows "system" account) specifically to run the webdav server. Also, keep an eye on the space that is being used on your hard drive by that account (a hacker who manages to get access may try to adjust it to set-up a "warez server").

The above was all the scary stuff, here are some of the advantages:

If you are running Windows or Apple OS X, you can mount a webdav like a network drive and save any files you want (not just tiddlywiki).

If you know how to configure your web server, you can set up public directories that don't require authentication to read but do require authentication to write files. You can also set-up private directories that require authentication for all access to files (in theory you could set up a blind drop where people can send you files without authenticating but not read them, although this is probably a bad idea).

You can pretty easily create multiple account so people can share files. It's a bit more complicated to give per-person private directories.

If you are extremely paranoid, you can set-up SSL/TLS client authentication which would require the browser to have a specific certificate (similar to the way the server needs one for HTTPS).

You could set-up your own carddav (address book) or caldav (calendar) server.

[Lost Admin]

"space test.html" got saved as "space%20test.html" which got saved as "space%2520test.html" which got re-loaded as "space%252520.test.html", so the %20 encoding doesn't really play nice.

It works but you have to constantly mess with the address when saving. I would say replacing spaces with underscores in your filenames would be less painful.

On Tuesday, August 29, 2017 at 10:58:04 PM UTC-4, Arlen Beiler wrote:

And can you reload the renamed file and continue saving to it? Or not?

On Tue, Aug 29, 2017 at 7:59 PM, Lost Admin <thelos...@gmail.com> wrote:

When I created a wiki with a space in the filename and used the webdav saver, it replaced the space with the URL encoding (%20) in the filename on save.

This was tested using my Apache setup from Safari on iOS. Haven't tested other browsers but I doubt they will be any different.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

[@TiddlyTweeter]

Double encoding issue? Is it an infinite regress?

[PMario]

On Thursday, August 31, 2017 at 3:12:10 PM UTC+2, Lost Admin wrote:

Safety/security concerns with WebDav:

Well written!

 

WebDav is conceptually a share network folder (so much so you can mount them with a drive letter on Windows) that is provided over HTTP. This means anyone who can access the webdav url can read, write, and delete to all the files available there. This includes making new directories.

Somewhere I read, that starting with IIS-7 write commands have to use basic-auth as minimal requirement.

 

To protect it, one typically uses HTTP Basic (or Digest) Authentication (part of the web server set-up). With basic authentication that means the password (and user name) are going across the network (including Internet when doing so remotely) in clear text and anyone who sees the traffic can read the login credentials. Using digest authentication reduces this risk as the password is no longer sent across the network. Usually it is recommended that you use HTTPS for webdav and not allow HTTP (unencrypted) connections. However, SSL/TLS has a lot of insecure configurations, so you need to know what you are doing (and what encryption protocols to allow).

Therefore IMO you have to use HTTPS as a minimal requrement if you face the internet.

... snip ...

 

If you are extremely paranoid, you can set-up SSL/TLS client authentication which would require the browser to have a specific certificate (similar to the way the server needs one for HTTPS).

IMO no need to be extreamly paranoid. .. It's just a very convenient and secure workflow, once you could manage the certificate deployment.

You could set-up your own carddav (address book) or caldav (calendar) server.

That's a nice plus

have fun!

mario

[PMario]

On Wednesday, August 30, 2017 at 8:52:07 PM UTC+2, Mark S. wrote:

What would the performance and safety concerns be for running IIS/WebDav on a semi-permanent basis? If you forward the ports (and know your IP) can you access your files outside of the home?

As Lost Admin points out. If you want to open up the internet, it gets a bit more tricky. The complexity raises quite a bit :)

There's a reason, why I wrote:

I'm basically interested in a replacement for TiddlyFox and not in a "internet facing" setup. ...

I knew, that those questions are coming. I want to show what's possible "out of the box", with built in windows components, that should be available for many users.

FF57 produces some headaches. ... The setting I'm imagine should work with every browser, without any addOns. ...

In my first post I also did point out, that M$ limits the possibilities quite a bit. see: 2.b-(iii) in win10 user terms. So for me it was clear, to use the stuff in a small intranet setting only. For real projects I'd go with an open source setting as introduced by Lost Admin. ...

... I didn't have a look, what IIS licenses would cost for real world projects. ... Probably no problem for companies, but way to heavy for personal use.

have fun!

mario

PS: Since I have an english Win10 VM now, I can start to record the stuff.

... to be continued ...

 

[Lost Admin]

Is there an easy way to get TiddlyWiki to re-read itself after saving?

Apache tracks the time that you read the file and issues an error if you try to save back to the server if the file on disk has a newer timestamp. Which means: after I save changes to a file, it give me an error on subsequent changes unless I reload the wiki immediately after saving.

[Arlen Beiler]

I remember running into that when writing TiddlyServer. The apache webdav module should return some kind of revision string. The put saver expects an Etag, but this may not necessarily be what Apache sends. However, I believe that is where the problem is.

On Fri, Sep 1, 2017 at 9:22 AM, Lost Admin <thelos...@gmail.com> wrote:

Is there an easy way to get TiddlyWiki to re-read itself after saving?

Apache tracks the time that you read the file and issues an error if you try to save back to the server if the file on disk has a newer timestamp. Which means: after I save changes to a file, it give me an error on subsequent changes unless I reload the wiki.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscribe@googlegroups.com.

To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/9084db99-3deb-446a-a6e3-d205ddc5269e%40googlegroups.com.

[Lost Admin]

Ah! you are ahead of me in figuring out what needs to be done. Unfortunately, my javascript skills are nowhere near good enough to figure out how to integrate the Apache way of doing things into TiddlyWiki.

On Friday, September 1, 2017 at 9:47:32 AM UTC-4, Arlen Beiler wrote:

I remember running into that when writing TiddlyServer. The apache webdav module should return some kind of revision string. The put saver expects an Etag, but this may not necessarily be what Apache sends. However, I believe that is where the problem is.

On Fri, Sep 1, 2017 at 9:22 AM, Lost Admin <thelos...@gmail.com> wrote:

Is there an easy way to get TiddlyWiki to re-read itself after saving?

Apache tracks the time that you read the file and issues an error if you try to save back to the server if the file on disk has a newer timestamp. Which means: after I save changes to a file, it give me an error on subsequent changes unless I reload the wiki.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

[Arlen Beiler]

Ok, well all I need is the request and response info for the put request. Actually, just the response headers. If you open your Developer Tools and go to the network tab, it will show you all the requests the page makes for everything. Find the put request it sends to save the file and click on it. It should show you the response headers. You can copy it here or email it to me directly if you like. 

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscribe@googlegroups.com.

To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/f2c207bb-f3de-44d6-9954-f5ad9bd0361c%40googlegroups.com.

[Lost Admin]

Here you go ... This only covers the save, including the authentication step (minus credentials)

Initial PUT http://domain.dom/index.html
Status code: 404

Response headers (296 B)
Date	Sat, 02 Sep 2017 17:19:45 GMT
Server	Apache/2.4.27 (FreeBSD) OpenSSL/1.0.1s-freebsd PHP/5.6.31
Keep-Alive	timeout=5, max=99
Connection	Keep-Alive
Content-Type	text/html
Request headers (452 B)
Host	domain.dom
User-Agent	Mozilla/5.0 (Windows NT 10.0; …) Gecko/20100101 Firefox/55.0
Accept	*/*
Accept-Language	en-US,en;q=0.5
Accept-Encoding	gzip, deflate
Referer	http://domain.dom/index.html
Content-Type	text/html;charset=UTF-8, appli…orm-urlencoded; charset=UTF-8
If-Match	"1eb890-5582095dd9986"
Content-Length	2013328
DNT	1
Connection	keep-alive

PUT http://domain.dom/index.html
Status Code: 204
Version HTTP/1.1

Request headers (499 B)
Host	ariel.suntrap.ca
User-Agent	Mozilla/5.0 (Windows NT 10.0; …) Gecko/20100101 Firefox/55.0
Accept	*/*
Accept-Language	en-US,en;q=0.5
Accept-Encoding	gzip, deflate
Referer	http://domain.dom/index.html
Content-Type	text/html;charset=UTF-8, appli…orm-urlencoded; charset=UTF-8
If-Match	"1eb890-5582095dd9986"
Content-Length	2013328
DNT	1
Connection	keep-alive
Authorization	Basic <redacted>

Now save again without reloading
PUT http://domain.dom/index.html
Status Code: 412 Precondition Failed

Response headers (262 B)
Date	Sat, 02 Sep 2017 17:27:15 GMT
Server	Apache/2.4.27 (FreeBSD) OpenSSL/1.0.1s-freebsd PHP/5.6.31
Content-Length	249
Keep-Alive	timeout=5, max=100
Connection	Keep-Alive
Content-Type	text/html; charset=iso-8859-1
Request headers (499 B)
Host	domain.dom
User-Agent	Mozilla/5.0 (Windows NT 10.0; …) Gecko/20100101 Firefox/55.0
Accept	*/*
Accept-Language	en-US,en;q=0.5
Accept-Encoding	gzip, deflate
Referer	http://domain.dom/index.html
Content-Type	text/html;charset=UTF-8, appli…orm-urlencoded; charset=UTF-8
If-Match	"1eb890-5582095dd9986"
Content-Length	2013314
DNT	1
Authorization	Basic <redacted>
Connection	keep-alive

Then reload the page (skipping the get details) and saving the failed change after re-editing:

PUT http://domain.dom/index.html

Status code: 204 No content

Response headers (212 B)
Date	Sat, 02 Sep 2017 17:34:52 GMT
Server	Apache/2.4.27 (FreeBSD) OpenSSL/1.0.1s-freebsd PHP/5.6.31
Keep-Alive	timeout=5, max=100
Connection	Keep-Alive
Content-Type	text/html
Request headers (499 B)
Host	domain.dom
User-Agent	Mozilla/5.0 (Windows NT 10.0; …) Gecko/20100101 Firefox/55.0
Accept	*/*
Accept-Language	en-US,en;q=0.5
Accept-Encoding	gzip, deflate
Referer	http://domain.dom/index.html
Content-Type	text/html;charset=UTF-8, appli…orm-urlencoded; charset=UTF-8
If-Match	"1eb892-55838491dd106"
Content-Length	2013314
DNT	1
Authorization	Basic <redacted>
Connection	keep-alive

It looks to me like Apache is using the If-Match header.

On Friday, September 1, 2017 at 10:26:15 AM UTC-4, Arlen Beiler wrote:

Ok, well all I need is the request and response info for the put request. Actually, just the response headers. If you open your Developer Tools and go to the network tab, it will show you all the requests the page makes for everything. Find the put request it sends to save the file and click on it. It should show you the response headers. You can copy it here or email it to me directly if you like. 

On Fri, Sep 1, 2017 at 10:11 AM, Lost Admin <thelos...@gmail.com> wrote:

Ah! you are ahead of me in figuring out what needs to be done. Unfortunately, my javascript skills are nowhere near good enough to figure out how to integrate the Apache way of doing things into TiddlyWiki.

On Friday, September 1, 2017 at 9:47:32 AM UTC-4, Arlen Beiler wrote:

I remember running into that when writing TiddlyServer. The apache webdav module should return some kind of revision string. The put saver expects an Etag, but this may not necessarily be what Apache sends. However, I believe that is where the problem is.

On Fri, Sep 1, 2017 at 9:22 AM, Lost Admin <thelos...@gmail.com> wrote:

Is there an easy way to get TiddlyWiki to re-read itself after saving?

Apache tracks the time that you read the file and issues an error if you try to save back to the server if the file on disk has a newer timestamp. Which means: after I save changes to a file, it give me an error on subsequent changes unless I reload the wiki.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/9084db99-3deb-446a-a6e3-d205ddc5269e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

[Arlen Beiler]

Good afternoon,

What is the response headers for the request with the status code 204? And why does the initial put request have a status of 404?

Thanks

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscribe@googlegroups.com.

To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/ae7f53cc-1797-4a21-ae58-4c0c34437e79%40googlegroups.com.

[Lost Admin]

I think the 404 was a mis-type on my part and should have been 401 (unauthorized). As this was the first attempt to "save" I had not yet authenticated.

Re-doing my testing, the 204 is as follows (this is the successful save of tiddlywiki):

Thanks

[Arlen Beiler]

Interesting. Could you also post the request and response for the initial get request for the file? This format works perfect.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscribe@googlegroups.com.

To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/44825365-1645-45d8-beb1-47b7882428e5%40googlegroups.com.

[Lost Admin]

I put the whole sequence of steps from intial GET request through the succeeding and failing PUT requests here https://wiki.suntrap.ca/davsaver.html I figure this would be easier to refer to while analyzing.

[Arlen Beiler]

It is becoming pretty clear that for some reason the Etag is not being set in the response header, nor anything else equivalent to it. Per our discussion privately, it does seem that this is an Apache issue, however I have not been able to look into it further. 

A couple of articles which touch on this: 

• https://fullstackhack.wordpress.com/2014/12/10/the-pain-of-etags-mod_deflate-apache-2-4-and-tomcat-7/ 
• https://httpd.apache.org/docs/2.4/caching.html
  

At some point I will test it on one of my servers and see if I can get it working. However, it is obvious that this is the problem. One option would be to make a second head request if the 204 response does not contain an Etag, but I guess that wouldn't be atomic either.

[Lost Admin]

I was coming to the same conclusion, except it appears that IIS has the same problem (judging by what was going on in the IIS WebDAV setup video thread).

Also, if I read the technical docs on Etag, the PUT shouldn't send a new Etag as there is no content in the response (the Etag is tied to the reply content).

It looks like the HEAD call does include an Etag, so conceivably issuing a HEAD after confirming the PUT succeeded would get the new Etag. Unfortunately, that would introduce a race condition if there are multiple people editing the file.

Does the TiddlyWiki WebDAV saver support anything other than Etag? While it appears Etag is the default, I ran across some cryptic comments about disabling Etag and adding a last-changed timestamp. Possibly the PUT would include the timestamp. Unfortunately, this is not the default for Apache and would require clear documentation  on our part to explain to people how to set-up their WebDAV servers (and significantly reduces the likelihood that WebDAV supporting cloud providers will work properly).

I've been looking but haven't (yet) figured out how to tell Apache to include the Etag header in the PUT response header.

NOTE: to those who have no idea what I was talking about above, Arlen and I are digging into the finer points of the HTTP protocol and WebDAV protocol to see if we can fix an issue that's been bugging us (well, at least me).

[PMario]

On Tuesday, September 5, 2017 at 3:46:50 PM UTC+2, Lost Admin wrote:

I was coming to the same conclusion, except it appears that IIS has the same problem (judging by what was going on in the IIS WebDAV setup video thread).

Is there a minimal test-case? So I can reproduce the issue?

 

It looks like the HEAD call does include an Etag, so conceivably issuing a HEAD after confirming the PUT succeeded would get the new Etag.

Unfortunately, that would introduce a race condition if there are multiple people editing the file.

WebDAV supports a locking mechanism. ... It could be used to prevent 2 users editing a TW at the same time.

 

-m

[@TiddlyTweeter]

I am absolutely loving this thread. Its so technical it could be an opera in Latvian.

Carry on. I'm just making notes.

Best wishes
Josiah

[Lost Admin]

Minimal, and reliable test case:

1) Set-up a WebDAV server on Apache (follow the Apache instructions, nothing special).

2) Put a TiddlyWiki on it. Also nothing special, just the standard empty.html from TiddlyWiki.com.

3) Edit a Tiddler and save it (make sure it saves to the WebDAV server.

4) Wait for the save to finish.

5) Make another edit (of the same tiddler if you want)

6) Save again.

7) The error pops up.

In Firefox the developer tools can be used to see what is passing on the network. Pay close attention to the ETAG header and it's value (and when it doesn't show up). In theory you could do this with Chrome too, but when I try the Chrome developer tools keep hanging so I can't see the headers on each step.

Judging by what is going on in the IIS thread, I think IIS has the same issue. I will be investigating this more later today.

On my Lighttpd server, I don't have this issue.

Yes, that is sort of the problem. the mechanism to prevent the editing conflicts is handled by the ETAG header. But, when you save changes (HTTP PUT), you don't get a new ETAG with the response from Apache. However, the ETAG changes whenever the file changes. At least, I think that is what is going on.

[Arlen Beiler]

I just thought I would mention that TiddlyServer does not have this problem either. I specifically designed it to support the current webdav (put) saver in TiddlyWiki.

Judging by what is going on in the IIS thread, I think IIS has the same issue.

[Lost Admin]

On Tuesday, September 5, 2017 at 1:21:04 PM UTC-4, Lost Admin wrote:

...

On my Lighttpd server, I don't have this issue.

...

I've tested using Firefox and put screenshots up at https://wiki.suntrap.ca/webdav/davsaver.html (for now). Lighttpd does not exhibit this issue but it also doesn't send the If-Match: header on then 2nd PUT request (the one that fails with Apache and maybe IIS).

[Lost Admin]

That is useful when running it on your local computer, but how robust is it if you drop it on the Internet?

There are so many configuration settings in Apache, there must be a way to tell it to include an ETag header in PUT requests.

[TonyM]

Folks,

I just want to report I am using IIS WebDav Implementation to access a TW5 and whilst it seems to work most of the time the tab it was in has mysteriously disappeared from firefox and I cant return to the wiki (even in another browser) and avoid the 412? error  until I reboot my computer.

When I next have time to reproduce the problem which sounds related to the Apache errors discussed below I can post in more detail.

Regards
Tony


On Monday, August 28, 2017 at 11:10:36 PM UTC+10, PMario wrote:

Hi LostAdmin,

That's a perfect reminder, to have a closer look at MS built-in stuff. WebDav is part of Internet Information Services (IIS), which seems to be available for up to 20 users for Windows Vista, 7, 8.x and 10. ...

I'm basically interested in a replacement for TiddlyFox and not in a "internet facing" setup. ...

from win10 user terms.

2.      Installation and Use Rights.

<snip>

d.      Multi use scenarios.

<snip>

(iii)    Device connections. You may allow up to 20 other devices to access the software installed on the licensed device for the purpose of using the following software features: file services, print services, Internet information services, and Internet connection sharing and telephony services on the licensed device. You may allow any number of devices to access the software on the licensed device to synchronize data between devices. This section does not mean, however, that you have the right to install the software, or use the primary function of the software (other than the features listed in this section), on any of these other devices.

So with the right settings, it may be possible to activate it for most windows machines. ... The important phrase is "may be", since nobody really knows, how  crippeld "Home xxx" and OEM Licenses really are.

For those who are courious, IIS is the equivalent of the Apache and Lighttdp servers, that Lost Admin mentioned. ... So it should be possible to use the stuff for local installations.

... I'm not at home at the moment. So I can't run tests with my machine. But anyone who is interested, should expolore the docs, run some tests and report back here.

Especially interesting: OS version used! So we can see, if it works with "Home xxxx" licenses. I do have a Win10 Pro, wich I can test on :)

have fun!
mario

[Arlen Beiler]

It has basic auth, but that's it. So it really isn't. However, I would welcome your input on all this as it is probably where I should focus next. This is a good point. 

On Tue, Sep 5, 2017 at 5:50 PM, Lost Admin <thelos...@gmail.com> wrote:

That is useful when running it on your local computer, but how robust is it if you drop it on the Internet?

[PMario]

On Tuesday, September 5, 2017 at 7:21:04 PM UTC+2, Lost Admin wrote:

Minimal, and reliable test case:

1) Set-up a WebDAV server on Apache (follow the Apache instructions, nothing special).

:) .. I was thinking about a IIS testcase. .. Will run some tests

-m

[Lost Admin]

There is a video about setting up IIS. I agree, it would be good to test the popular web servers that support WebDAV and eventually collect sets of instructions on setting them up. 

I was planning on reviewing the video (from another thread) and setting up IIS on my home computer (Windows 10 Home), if it is possible, as a test environment for IIS. I haven't gotten around to it. Apache is my preferred web server. I messed with lighttpd because it has a smaller memory footprint. Someone should probably test things with Nginx as well (but it won't be me).

 

[PMario]

On Wednesday, September 6, 2017 at 4:04:12 PM UTC+2, Lost Admin wrote:

... Someone should probably test things with Nginx as well (but it won't be me).

IMO no need to go with Nginx atm. It only implements basic support http://nginx.org/en/docs/http/ngx_http_dav_module.html ... which will cause trouble.

-m

[Lost Admin]

I've been trying to dig into the proper specs on the use of ETag and it looks like it is only supposed to be sent from the server along with the data. Thus the PUT request is not supposed to include a new ETag. I think Apache is actually doing it right.

Also, I did the same series of screenshots on my test Lighttpd server (which doesn't experience the same 412 error) and for some reason, the If-Match header gets dropped from the subsequent PUT requests headers. I don't know why it would be different as I think that header is coming from the client side.

[Arlen Beiler]

Ok, so how does web dav take care of making sure someone is editing the latest version? Or does it use the entirely file-system concept of locking files for editing?

Are we barking up the wrong tree with the idea of using web DAV? It is entirely file system centered. The fact that it can handle web requests seems almost incidental. Or maybe it is just the simple fact that the PUT saver nowhere near implements the entire DAV protocol. 

What protocol talks about Etags in 204 responses? The one I found only mentions it once in relation to a PUT request by saying that there is no specific definition of whether it should guarantee the file content is exactly byte-for-byte identical to the PUT request.  http://www.ietf.org/rfc/rfc4918.txt

The HTTP/1.1 spec is at https://www.ietf.org/rfc/rfc2616.txt

I can't find anything in either of those just by searching for "etag".

Just some thoughts.

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscribe@googlegroups.com.

To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/a827b52a-100c-453d-b146-a48d229be428%40googlegroups.com.

[Lost Admin]

I was looking through the Apache (which is a bit unfair) and www.webdav.org sites. I inferred a lot of what I think I understand.

From what I can tell, there is no solid solution for data locking with WebDAV itself but there are some options available:

WebDAV does have a built-in locking mechanism. Apache does support it but you have to turn it on. I haven't researched this one very far beyond reading about it's existence. The client would need to send a lock request (I haven't figured out how that is done yet) before (or during?) the GET request. Then use some reference to that with the PUT request. After that, as I understand it, the calling application would need to request a new lock after the PUT to continue editing (although that might actually be a misunderstanding on my part). Sorry, I can't find the reference to this now.

For hard locks WebDAV Locks, the most coherent explantion I found is http://www.webdav.org/mod_dav/install.html#apache (part way down, beginning with Enabling DAV).

Apache support two methods of "lazy updates". A lazy update is where the application doesn't actually request a lock. Instead the requesting application sends a "update based on some earlier version based on some value" and if the file hasn't been changed since that value was current, then the update happens, otherwise you get the 412. The two choices (for Apache) that I've found are:

ETag and If-Match, which we currently only sort-of use right (based on how Apache behaves). I don't have a good clear single source of information. I read a bunch of blogs from programmers complaining about it (not specifically related to WebDAV) and a variety of Apache docs on cache control.

If-Modified-Since header, which is based on the timestamp of the GET request. I stumbled across this one in a blog post but now I can't find it.

Both of the ETag/If-Match and If-Modified-Since are actually cache control mechanisms.

Sorry for my crappy references.

In my personal opinion, I do think that ETag (or If-Modified-Since) is the way to go but we need to figure out a way to either get one back as a response to a PUT request (I've been trying to figure this out but so far have failed) or to figure out a way to do a follow-up GET request to reload the file from the WebDAV server after saving.

If you look at the series of screenshots I created (https://wiki.suntrap.ca/davsaver.html), you will note that although Lighttpd doesn't send a 412 with the second PUT request, the ETag/If-Match is dropped in the 2nd PUT request. I *think* something has happened to remove all file locking protections after the first PUT. I haven't had time to set-up a working IIS test environment or I would have that sequence set-up too.

On Thursday, September 7, 2017 at 4:59:40 PM UTC-4, Arlen Beiler wrote:

Ok, so how does web dav take care of making sure someone is editing the latest version? Or does it use the entirely file-system concept of locking files for editing?

Are we barking up the wrong tree with the idea of using web DAV? It is entirely file system centered. The fact that it can handle web requests seems almost incidental. Or maybe it is just the simple fact that the PUT saver nowhere near implements the entire DAV protocol. 

What protocol talks about Etags in 204 responses? The one I found only mentions it once in relation to a PUT request by saying that there is no specific definition of whether it should guarantee the file content is exactly byte-for-byte identical to the PUT request.  http://www.ietf.org/rfc/rfc4918.txt

The HTTP/1.1 spec is at https://www.ietf.org/rfc/rfc2616.txt

I can't find anything in either of those just by searching for "etag".

Just some thoughts.

On Thu, Sep 7, 2017 at 10:48 AM, Lost Admin <thelos...@gmail.com> wrote:

I've been trying to dig into the proper specs on the use of ETag and it looks like it is only supposed to be sent from the server along with the data. Thus the PUT request is not supposed to include a new ETag. I think Apache is actually doing it right.

Also, I did the same series of screenshots on my test Lighttpd server (which doesn't experience the same 412 error) and for some reason, the If-Match header gets dropped from the subsequent PUT requests headers. I don't know why it would be different as I think that header is coming from the client side.

On Tuesday, September 5, 2017 at 9:18:07 AM UTC-4, Arlen Beiler wrote:

It is becoming pretty clear that for some reason the Etag is not being set in the response header, nor anything else equivalent to it. Per our discussion privately, it does seem that this is an Apache issue, however I have not been able to look into it further. 

A couple of articles which touch on this: 

• https://fullstackhack.wordpress.com/2014/12/10/the-pain-of-etags-mod_deflate-apache-2-4-and-tomcat-7/ 
• https://httpd.apache.org/docs/2.4/caching.html
  

At some point I will test it on one of my servers and see if I can get it working. However, it is obvious that this is the problem. One option would be to make a second head request if the 204 response does not contain an Etag, but I guess that wouldn't be atomic either.

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

[Lost Admin]

We should just completely ignore Lighttpd as it isn't doing any locking at all. I just tried to force an error with two different browsers that should have resulted the 2nd one getting an error. Instead, the second one overwrote the first. Lighttpd might be sending an ETag, but it is ignoring the If-Match.

[Lost Admin]

I finally found something that properly talks about Etags. You can infer why we don't get an ETag header in the response to a PUT (for both Apache and IIS). https://stackoverflow.com/questions/42246577/why-responses-to-put-requests-must-not-provide-an-etag

You can skip going to the link if you want to. It is asking for an explanation of when to send an ETag or not and I think we can use what it quotes to get the understanding.

An origin server MUST NOT send a validator header field (Section 7.2), such as an ETag or Last-Modified field, in a successful response to PUT unless the request's representation data was saved without any transformation applied to the body (i.e., the resource's new representation data is identical to the representation data received in the PUT request) and the validator field value reflects the new representation. This requirement allows a user agent to know when the representation body it has in memory remains current as a result of the PUT, thus not in need of being retrieved again from the origin server, and that the new validator(s) received in the response can be used for future conditional requests in order to prevent accidental overwrites (Section 5.2).

Apache (and IIS) allow for the server to be configured such that you can add a custom processor to specific HTTP actions (like PUT) that are completely independent of WebDAV (I forget how but I did one for a post call once because we didn't have source and needed to re-mangle some of the user input).

This means that the Apache web server doesn't know for sure that what you send in the PUT through WebDAV will be what it picks up on the next GET. So, can't send and updated ETag.

I imagine the same goes for IIS.

Lighttpd doesn't appear to honor etag and If-Match headers so Lighttpd is broken. Nginx (I've been told) has a very primitive webdav module so is also not suitable for this discussion.

On Thursday, September 7, 2017 at 4:59:40 PM UTC-4, Arlen Beiler wrote:

Ok, so how does web dav take care of making sure someone is editing the latest version? Or does it use the entirely file-system concept of locking files for editing?

Are we barking up the wrong tree with the idea of using web DAV? It is entirely file system centered. The fact that it can handle web requests seems almost incidental. Or maybe it is just the simple fact that the PUT saver nowhere near implements the entire DAV protocol. 

What protocol talks about Etags in 204 responses? The one I found only mentions it once in relation to a PUT request by saying that there is no specific definition of whether it should guarantee the file content is exactly byte-for-byte identical to the PUT request.  http://www.ietf.org/rfc/rfc4918.txt

The HTTP/1.1 spec is at https://www.ietf.org/rfc/rfc2616.txt

I can't find anything in either of those just by searching for "etag".

Just some thoughts.

On Thu, Sep 7, 2017 at 10:48 AM, Lost Admin <thelos...@gmail.com> wrote:

I've been trying to dig into the proper specs on the use of ETag and it looks like it is only supposed to be sent from the server along with the data. Thus the PUT request is not supposed to include a new ETag. I think Apache is actually doing it right.

Also, I did the same series of screenshots on my test Lighttpd server (which doesn't experience the same 412 error) and for some reason, the If-Match header gets dropped from the subsequent PUT requests headers. I don't know why it would be different as I think that header is coming from the client side.

On Tuesday, September 5, 2017 at 9:18:07 AM UTC-4, Arlen Beiler wrote:

It is becoming pretty clear that for some reason the Etag is not being set in the response header, nor anything else equivalent to it. Per our discussion privately, it does seem that this is an Apache issue, however I have not been able to look into it further. 

A couple of articles which touch on this: 

• https://fullstackhack.wordpress.com/2014/12/10/the-pain-of-etags-mod_deflate-apache-2-4-and-tomcat-7/ 
• https://httpd.apache.org/docs/2.4/caching.html
  

At some point I will test it on one of my servers and see if I can get it working. However, it is obvious that this is the problem. One option would be to make a second head request if the 204 response does not contain an Etag, but I guess that wouldn't be atomic either.

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

[Lost Admin]

Further on the complexity of ETag and why Apache (and generic web servers) should not send an updated ETag in response to a PUT:

https://tools.ietf.org/id/draft-reschke-http-etag-on-write-09.html

[@TiddlyTweeter]

Where are we with this?

Could a newbie cope?

Is the install outlined above consistent & reliable?

Josiah

[Lost Admin]

If you are familiar with setting up Apache, you could definitely set-up an Apache WebDav server. I just followed some generic how-to guides that explains the settings needed to get WebDav, TLS (and letsencrypt), and authentication working on Apache.

The issue is with using TiddlyWiki. When you save your wiki, it sends an HTTP PUT method request. You either get a normal save (like we are used to with TiddlyWiki) or a somewhat cryptic 412 error.

The 412 error indicates the save didn't happen because the version on the server is "newer" than the version your browser has. It works nicely when multiple people try to edit the same TiddlyWiki file (I tested this myself).

Unfortunately, in order to get the magic ETag value after you save a change, you need to reload TiddlyWiki because you don't get an update to it after the PUT request.

Note: ETag is just an HTTP header that the server sends with a GET request. It changes when the file changes. When TiddlyWiki saves (does the PUT) it sends the ETag back to the server effectively saying, "save the file if the ETag is this otherwise give me an error".

I am planning on writing up a full setup guide to create your own WebDav server using Apache, TLS, Letsencrypt, and HTTP Basic Auth (in digest mode) but I'm not going to until it works properly. Which means without needing to reload. Sadly, my programming skills are seriously lacking. I think I know what should be done in TiddlyWiki's webdav saver, but I don't have the skills to do it.

Note 2: I replaced my old store.php saver with the webdav on Apache at home quite a while ago. The only pain point is remembering to reload after every save.

[Lost Admin]

Could someone help me a bit with something? My coding skills are not up to even this simple task sadly.

The WebDAV saver for TiddlyWiki saves using the HTTP PUT method. Unfortunately the Apache and IIS implementations of WebDAV do not respond with the updated ETag header. However, according to the documentation, the HTTP HEAD method should respond with the new ETAG. So, I would like to modify the WebDAV saver to do the following:

1) call the HTTP LOCK method to lock the file

2) call the HTTP PUT method to save the file (as it does today with the ETAG and everything)

3) If save fails: return the error message (like we do today)

4) if save succeeds: call the HTTP HEAD method to get the updated ETag

5) call the HTTP UNLOCK method to release the lock on the file

NOTES:

If I read the documentation correctly, HTTP locks have a timeout. So if an issue occurs during the locked phase, the file should be released in a few minutes.

I'm not bothering to actually check if we got a lock after step 1 because we will still get an error from the PUT call due to either another file lock or the ETag miss-match.

The UNLOCK response doesn't need to be communicated to the end-user because it should only fail if the initial lock failed (or there is some miss-match in the lock token). This could even be done asynchronously.

We could conceivably use the HTTP LOCK method to lock a tiddlywiki when it is being edited but the LOCK method has a timeout, so we would need to periodically re-request the lock. Right now I prefer the  opportunistic locking method we are using with ETag. The only reason to use the lock around the PUT and HEAD calls is to ensure that the ETag we get is the one that matches the data we just saved and not the data submitted by another person at almost the same time.

In Apache, the lock method is optional and requires some additional set-up. By ignoring the success/failure of the lock, we nicely fall-back to the current ETag method.

Thoughts? Suggestions? Pointers to where I find the parts of the WebDAV saver code in tiddlywiki to attempt to make these changes myself?

On Thursday, September 7, 2017 at 4:59:40 PM UTC-4, Arlen Beiler wrote:

Ok, so how does web dav take care of making sure someone is editing the latest version? Or does it use the entirely file-system concept of locking files for editing?

Are we barking up the wrong tree with the idea of using web DAV? It is entirely file system centered. The fact that it can handle web requests seems almost incidental. Or maybe it is just the simple fact that the PUT saver nowhere near implements the entire DAV protocol. 

What protocol talks about Etags in 204 responses? The one I found only mentions it once in relation to a PUT request by saying that there is no specific definition of whether it should guarantee the file content is exactly byte-for-byte identical to the PUT request.  http://www.ietf.org/rfc/rfc4918.txt

The HTTP/1.1 spec is at https://www.ietf.org/rfc/rfc2616.txt

I can't find anything in either of those just by searching for "etag".

Just some thoughts.

On Thu, Sep 7, 2017 at 10:48 AM, Lost Admin <thelos...@gmail.com> wrote:

I've been trying to dig into the proper specs on the use of ETag and it looks like it is only supposed to be sent from the server along with the data. Thus the PUT request is not supposed to include a new ETag. I think Apache is actually doing it right.

Also, I did the same series of screenshots on my test Lighttpd server (which doesn't experience the same 412 error) and for some reason, the If-Match header gets dropped from the subsequent PUT requests headers. I don't know why it would be different as I think that header is coming from the client side.

On Tuesday, September 5, 2017 at 9:18:07 AM UTC-4, Arlen Beiler wrote:

It is becoming pretty clear that for some reason the Etag is not being set in the response header, nor anything else equivalent to it. Per our discussion privately, it does seem that this is an Apache issue, however I have not been able to look into it further. 

A couple of articles which touch on this: 

• https://fullstackhack.wordpress.com/2014/12/10/the-pain-of-etags-mod_deflate-apache-2-4-and-tomcat-7/ 
• https://httpd.apache.org/docs/2.4/caching.html
  

At some point I will test it on one of my servers and see if I can get it working. However, it is obvious that this is the problem. One option would be to make a second head request if the 204 response does not contain an Etag, but I guess that wouldn't be atomic either.

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

[Arlen Beiler]

The webdav saver code in TiddlyWiki should be in $:/core/modules/savers/put.js or something like that. I know it ends with put.js

[PMario]

On Wednesday, October 18, 2017 at 3:12:00 PM UTC+2, Lost Admin wrote:

Could someone help me a bit with something? My coding skills are not up to even this simple task sadly.

The WebDAV saver for TiddlyWiki saves using the HTTP PUT method. Unfortunately the Apache and IIS implementations of WebDAV do not respond with the updated ETag header. However, according to the documentation, ....

Hi, Can you add some links to your docs? Especially the ETag handling stuff, that you refer to?

 

-m

[Lost Admin]

I can't currently find what I was reading that showed me that HEAD gets an e-tag, but this blog indicates it does:

http://joshua.schachter.org/2006/11/apache-etags

But I did use this one in my research. It explains why the Etag doesn't usually show up with a PUT.

https://stackoverflow.com/questions/42246577/why-responses-to-put-requests-must-not-provide-an-etag

The apache docs location should be obvious. I had to read the cache control and mod_dav, mod_dav_fs, and mod_headers section to sort out that Apache is not able to confirm that what as sent in the PUT will be exactly what is returned for the next GET request. So, no etag header.

I can show (because I tested) that the HEAD method does appear to return an etag.

[MagoArcade]

Hi all - did this result in a dead end? I'm trying to serve tw via IIS and Webdav to enable editing over the internet. Have it connecting fine in terms of reading, but consistently get "Error whilst saving: File changed on server" no matter how much I tinker with WebDav/Auth/SSL settings. 

I've tried other methods too (node.js server etc) none of which are working for this scenario. 

TiddlyWiki looks fab, but this saving malarky appears to be its Achilles heel?

[TonyM]

Mago

This should all be resolved with new and better options available. Rather than dig up this old thread, try a new one for support

Tony

[PMario]

Hi,

Did you turn on server side compression? If yes the default saver doesn't work. It uses etags to detect changes, which seems to be implemented very different on the different platforms.

I did create a put-saver that uses last-modified timestamp. It should be more forgifing.

see: https://wikilabs.github.io/editions/webdav-lm/

-mario

[MagoArcade]

@PMario - you, sir, are a star. That just nudged me enough to get it all working! It also came with additional rewards. Not only can I edit my wiki securely (due to the password protection through webdav) but I can also edit it on my LAN with TiddlyDesktop (putting it in a shared network folder) and additionally I can create another Virtual Directory which allows the public to see (but not save) it. 3 for 1! 

Well, it took me 2 solid days to get this working, but now I have, I thought I'd make a step-by-step video and blog guide to help others in the future. Why don't you take a look?

Blog: http://magoarcade.org/wp/serving-tiddlywiki-via-windows-home-server-iis-and-webdav/

Video: https://youtu.be/VMQ3Lfko8uQ

When I found something good, I like to contribute. Is there anywhere else I could post these to help out other struggling Tiddliers?

Thanks again.

[PMario]

Hi MagoArcade,

It's nice to see, that the plugin also works in a "real" windows server setting. ... I couldn't test it, since I don't own an IIS server.

Using Windows Auth with an ActiveDirectory setting should be more secure, than the "basic auth", that I did in my videos. But with a windows-client only setting I don't have ActiveDirectory set up. .. So basic-auth is the only possibility left.

I should have pointed you to the WebDav-saver intro page. Because it shows, that it should be possible to use server side compression with this plugin. .. Which imo is important, if your users are on the web.

Switching it off initially, as you did in your video is a good thing for testing. In an optimization step 2 it can be switched on again.

Saving and reading the file from a local network, doesn't make a difference, since compression also needs some time, which eats up the time savings for network transfer. But it the internet gets involved, server side compression should make the user experience better.

empty.html is about 1.8 MByte, because all the js code is contained in plain text, which makes "user hacking" much simpler. ... empty - compressed is about 350kByte. Which makes a difference for web-based users.

----------- a bit of background

The TW default WebDav saver uses etags, to create a "modified" warning. Since the plugin was developed with an Apache WebDav setup, which seems to use no compression with the initial setting. The saver worked. ... In the group here, there should be a thread, where users found out, that other environments didn't work: IIS and nginx-webdav

I did a lot of experimenting and reading about etags. ... It seems the different server vendors do implement etag handling in different ways. ... So it is close to impossible to create a generic saver, that works with different server brands, server side compression and etag cache handling. ..

That was the reason, why I did create the plugin. last-modified and if-unmodified-since header info should be more forgiving, since they basically use the file timestamps to detect a "modified" status.





In your video you did enable "file locking", which imo shouldn't make a difference for the client, since TW doesn't detect it.





Nice video! .. Now I'll have to have a look at the blog post.


have fun!

mario


[PMario]

Hi,

I did link to your video, from my 1st and 3rd videos. .. So it would be nice if you could link to mine too.

-m

see: https://www.youtube.com/watch?v=tpkQhKyqPzc&list=PLuiC_HFhI4OwoVDb-B-VK0ydj-mBPNn-1

[MagoArcade]

Thanks for the feedback - have updated things accordingly. I can confirm that re-enabling compression works fine. Also - your vids linked from mine and the blog. Thanks for your input into this. 

[PMario]

Hi,

It would be nice, if you could do 1 more test. In your video at: https://youtu.be/VMQ3Lfko8uQ?t=1138 you did stop interaction for a "1 shot possibility". ... I think that shouldn't be true, if server side compression is switched off already.

I did test the default WebDav saver, which uses etags, on a Windows 10 Pro Client machine. .. IIS is enabled using webdav and basic auth. ... So that's the main difference between your setup and mine. ..

I could save several times, as long as I didn't change the TW with TDesktop or by hand ...

If you really can't save 2 times at this stage, then ISS stand alone server and ISS client server implement etags in different ways. Which I don't think is true. My IIS Manger says: IIS Version 10.0.18362.1

It would be nice to get feedback here.

----------------

There is a second possibility to include the plugin. .. Just use TDesktop first and import the plugin there. ... If you load it with WebDav, then it would be there already.

have fun!

mario

[MagoArcade]

Hmm. Just thinking how to do this test without doing a full re-install. If I were to disable your plugin and then change the auth method to Basic - would that test your scenario. 

I'm very confident about the 'one shot' save when using Windows Auth - it was a clear pattern - restart server - first save goes well, any subsequent doesn't. 

Let me know how to proceed with any testing - happy to help. 

[PMario]

On Thursday, July 18, 2019 at 1:07:52 PM UTC+2, MagoArcade wrote:

Hmm. Just thinking how to do this test without doing a full re-install. If I were to disable your plugin and then change the auth method to Basic - would that test your scenario. 

I'm interested with "windows auth". If the default WebDav save has no chance to run with ISS server at all, imo it would be reason to make the webdav-lm the default and the etag version a plugin.

It's relatively easy to disable a plugin, without removing it. Just click the "disable" button -> Save -> Reload.

 

I'm very confident about the 'one shot' save when using Windows Auth - it was a clear pattern - restart server - first save goes well, any subsequent doesn't. 

Even if compression switched off?

I do know, that my plugin works well with many different configuration options, with IIS, apache, nginx, lighttpd ... The default on the other hand is very "brittle".

have fun!

mario