Tiddlywiki, Seafile and the iPhone

[Willy Tanner]

First post here.

For a long time I have been looking for a note-taking solution that works equally on my Mac and iPhone and which does not force me to put any data on servers I do not have control over.

I realise that this may be overkill just in order to be able to use Tiddlywiki but Seafile (see https://www.seafile.com/en/home/) is amongst other things a flexible and very powerful data synchronisation tool. The community edition is completely free and has served me well for several years already.

Although not activated by default, Webdav works beautifully well. The desktop side of things works well and the iPhone can access and more importantly save back onto the Webdav server without problems. Note that saving does not work when accessing the file via the iPhone client and sending it to be opened in the browser.

I only mention this because https://tiddlywiki.com/static/Saving%2520via%2520WebDAV.html mentions that none of the free hosting possibilities work for TW. This self-hosted possibility seems to work if you are willing to put some work in.

Myself I am hoping to get Noteself up and running but I'd rather make my first attempt when I have a lot of time...

[AltugOz]

Hi Willy,

Thanks for sharing. I just wanted to confirm that TW and Seafile generally play nicely together on Android as well. I previously used the android app Andtidwiki with the Seafile client. When you mark a tiddlywiki file as favorite in the Seafile client it will save back the changes made to it by Andtidwiki. This setup has worked quite well for me, but I also recently started exploring the webdav saver with Seafile. As you say it works great. The only problem I have is I can't save the file more than once unless I reload it in browser. The second time I try to save, it will give me a "file changed on server" error. I must accept it is a minor annoyance. I don't know whether you also have this issue.

Best wishes,
Altug

[Willy Tanner]

Hi Altug

good to know it works on Android as well.

On Tuesday, August 28, 2018 at 7:31:27 AM UTC+2, AltugOz wrote:

Hi Willy,

Thanks for sharing. I just wanted to confirm that TW and Seafile generally play nicely together on Android as well. I previously used the android app Andtidwiki with the Seafile client. When you mark a tiddlywiki file as favorite in the Seafile client it will save back the changes made to it by Andtidwiki.

Is marking the file as a favourite a convenience to find it more easily or would saving only work if marked as such? How would that happen?

 

This setup has worked quite well for me, but I also recently started exploring the webdav saver with Seafile. As you say it works great. The only problem I have is I can't save the file more than once unless I reload it in browser. The second time I try to save, it will give me a "file changed on server" error. I must accept it is a minor annoyance. I don't know whether you also have this issue.

I do not have the same issue, actually. On the Mac as well on the iPhone, saving the tiddler would also save the entire wiki, after a second or so the yellow alert pops up telling me as much, done. 

If I keep editing and save again, it works, too, although thanks to the autosave I rarely hit the save wiki icon but it works. 

I hope this helps

Willy

[Jeremy Ruston]

Hi Willy

It’s good to hear of an online WebDAV service that works well with TiddlyWiki. You mentioned that there are some setup steps. Would you perhaps be able to contribute a docs tiddler I can link from the existing “Saving via WebDAV”  tiddler? You can either contribute by posting a .tid file here, or make a PR via GitHub if you’re familiar with the process.

Many thanks

Jeremy

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/a3c0cbe3-294e-4d87-8412-7661def16407%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Willy Tanner]

Hi Jeremy

it not an online service but a self-hostable solution. There used to be ready-made hosted Seafile solutions but I know nothing about them nor if the optional Webdav component is active or not. 

Is a self-hosted solution interesting nonetheless?

Willy

[Jeremy Ruston]

Hi Willy

On 28 Aug 2018, at 16:20, Willy Tanner <pri...@gmail.com> wrote:

it not an online service but a self-hostable solution. There used to be ready-made hosted Seafile solutions but I know nothing about them nor if the optional Webdav component is active or not. 

Is a self-hosted solution interesting nonetheless?

Apologies, I misread your post. But yes indeed, I think a self-installable WebDAV server is pretty useful, please do go ahead,

Best wishes

Jeremy

[AltugOz]

Hi Willy,

Is marking the file as a favourite a convenience to find it more easily or would saving only work if marked as such? How would that happen?

I see where you are heading with this. You are absolutely correct. Marking the file turns out to be just a convenience so that you can locate it easily with the Seafile app. For some reason, when I started using Seafile two years ago, I had the impression that marking a file was the only way to hold a local copy that would sync back to the server. Apparently it is not.

I just checked and any file downloaded via the Seafile app will show up in a folder called `Seafile` in device internal storage and will also sync back to the server when modified (You can actually download whole folders! I uploaded a screenshot below showing the download option in the Seafile app for a folder in case others are curious about how it looks like). It is important to note that this is one way sync in the sense that changes done on the server will not be automatically downloaded, only changes done on the device be uploaded back. However I still find this pretty neat. Anyway, thanks for asking. Your question helped me find out something useful today.

I do not have the same issue, actually. On the Mac as well on the iPhone, saving the tiddler would also save the entire wiki, after a second or so the yellow alert pops up telling me as much, done. 

If I keep editing and save again, it works, too, although thanks to the autosave I rarely hit the save wiki icon but it works. 

Thanks for letting me know. This confirms the problem is in my webdav setup or mobile browser. I will check both when I get a chance.

Best,
Altug

[Willy Tanner]

Hi Altug

ah, I see, we got our wires crossed here. As I mentioned in my first post, Seafile is many things in one, first and foremost it is a file synchronisation tool, which can be used to keep files and folders in sync, with version history, optional web interface, sharing links etc etc. The behaviour you describe is normal for the Seafile companion app (except a little detail about which later), the file is downloaded to your phone and once saved also synced back to the server and any other machines that are connected to that repository.

The point of this thread, however, is that Seafile can if you so wish be configured as a Webdav server. In the process you designate a folder on the server which then can be accessed from the outside (via your phone or another computer) without the need to download the file first. This obviously only works when online but if so, one can edit the file and the save is done directly on the server. Importantly, the file will be accessed via a special URL directly in the browser, not through the Seafile app on the phone. 

Note that the Seafile server is by default off but once on, the edits on the phone or some other computer are autosaved back and instantly available anywhere. 

Question to you, Altug, you were reporting the behaviour of a Tiddlywiki file after calling that file from the Seafile app on your phone, correct? 

Can you please check if any saves on your phone were indeed saved back to the server by accessing that file from a different computer (not your phone, in case the edited file still sits in the cache)? On the iPhone at first glance it looks like saving an edit on a local file via the Seafile app back to the Seafile repo on the server works but it is purely in the phone's cache. No other computer sees that edit nor the phone once you force it to reload the content from the server.

If this were different indeed on Android I'd be surprised.

Cheers

Willy

[PMario]

Hi Altug,

On Tuesday, August 28, 2018 at 7:31:27 AM UTC+2, Altug Ozcelikkale wrote:

... The only problem I have is I can't save the file more than once unless I reload it in browser. The second time I try to save, it will give me a "file changed on server" error. I must accept it is a minor annoyance. I don't know whether you also have this issue.

Which TW version do you use? .. We changed the WebDav PUT saver with the last version quite a bit. Especially the ETag handling was improved, to make it more robust. ... BUT ...

If your server transfers the TW in "compressed" mode, it seems the ETags are different to the file that is saved from the client to the server. ... So the second "save" fails.

You could try to disable the "compression" on the server. ... This should fix the problem. .. But makes the download slower.

I did create an experimental plugin, that doesn't use the ETag, but the "last-modified-" info, which also works with if the browser sends the TW file in compressed mode.

Just some thoughts!

-m

[PMario]

Hi Willy,

Thanks for sharing your experience with Seafile. ...

There has been a thread some time ago, when the project was still in it's early days. ...It seems the WebDav feature has been introduced with V2.1, which I demoed in the hangout #33. ... BUT I completely missed this feature :(

So it's definitely time to "re-visit" the project and run some more tests.

As Jeremy wrote, it would be nice if you can share your settings.

have fun!

mario

[Altug Ozcelikkale]

Hi Willy,

To answer your question, the android client does sync back cached (downloaded) files that are modified on the phone. In particular, the tiddlywiki file modified on the phone will be uploaded to the server and become available on other devices. Now, I have been using this feature to go back and forth between TW on my desktop and phone for the last two years, so the feature has been there for a while. I also just confirmed this by a fresh empty.html from tiddlywiki.com and it works. I have the impression that the iphone client somehow doesn't do this file-watching and automatic upload and the difference between the mobile clients is indeed quite intriguing. 

Nevertheless I guess with webdav working well, having that feature on your iphone is not that critical for you.

I know the thread is mainly for discussion of the Seafile's webdav feature and I don't want to go too much off topic here but for the sake of completeness, here are the steps of the workflow to achieve what I described above.

Works on Android with a Seafile server without webdav enabled.

1. Download the TW file to the phone using the Seafile app. The downloaded file will have a orange checkmark at the corner when viewed in the Seafile app and will be located in `Seafile\<your account name>\path\to\file` under device storage partition.
   
2. Modify the file: Option 1 (recommened): Open with AndTidWiki. Upon editing, auto-saves in place and directly modifies the file. Option 2: Open with your favorite browser, make edits and click `save` in TW. A new file will be downloaded. Copy and replace the original file with the downloaded file using your device file explorer (this option is not fun to do and not very useful in practice).
3. Go back and check the status of the file in Seafile app. The last modified timestamp will change indicating that Seafile app recognizes the local modification of the file and it will silently upload the file back to server.
4. Allow ~1 min for changes to propagate. Access the file on another device. You will see the updated file with the recent changes made on the phone. 

What this doesn't do: automatically download the files to the phone that were changed elsewhere. you need to download again to get the up-to-date version. Luckily Seafile app keeps track of that and if you try to open the file in the app, it will download the file only if the file on the server is different than the local copy.

As I said, when webdav server in Seafile works properly as you described, it pretty much eradicates the need for the file sync mechanism that the mobile client provides. I fixed the save issues on my webdav setup thanks to PMario's comment below and will probably use it almost exclusively for TW sync from now on.

Best wishes,
Altug

[Altug Ozcelikkale]

Hi PMario,

Thank you so much! Disabling compression on the server fixes it :) 

I am using TW version 5.1.17 and it indeed behaves differently than 5.1.16 (where I can't save at all due to an error message with a request error code 412)

As for the solution for those with the same issue, I am running Seafile with reverse proxy behind Nginx. I added:

gzip off;

in the `nginx.conf` under `http` block. Restarted Nginx and Seafile/Seahub and voila! I can save files multiple times without any issues so far. I hope I won't get too much of a performance hit due to disabled compression. I think it might be possible to optimize the compression settings by excluding html files in `gzip_types` rather than disabling compression as a whole but I will leave exploring it to later. 

Also please note that this only fixes the `file changed on server` issue with TW 5.1.17. It won't fix the request error I am getting with the other version.

Best wishes,

Altug

[PMario]

On Thursday, August 30, 2018 at 8:44:39 AM UTC+2, Altug Ozcelikkale wrote:

Hi PMario,

Thank you so much! Disabling compression on the server fixes it :) 

Cool :)

 

I am using TW version 5.1.17 and it indeed behaves differently than 5.1.16 (where I can't save at all due to an error message with a request error code 412)

Yes. ... There was a problem, which should be fixed for nginx, apache and IIS (MS internet information service) - WebDav settings now.

All of them behave a little bit different, which makes it hard to do it right :)

 

As for the solution for those with the same issue, I am running Seafile with reverse proxy behind Nginx. I added:

gzip off;

in the `nginx.conf` under `http` block. Restarted Nginx and Seafile/Seahub and voila! I can save files multiple times without any issues so far.

I hope I won't get too much of a performance hit due to disabled compression. I think it might be possible to optimize the compression settings by excluding html files in `gzip_types` rather than disabling compression as a whole but I will leave exploring it to later. 

I did some tests in a "local" setting, where download performance isn't a problem. ... With compression ON a 2MByte empty TW will be about 350kByte in transit! So if you have a slow connection it will make a difference.

 

Also please note that this only fixes the `file changed on server` issue with TW 5.1.17.

OK

It won't fix the request error I am getting with the other version.

I may have missed this info. Can you give more details?

-m

[Altug Ozcelikkale]

On Thursday, August 30, 2018 at 2:43:19 PM UTC-4, PMario wrote:

I did some tests in a "local" setting, where download performance isn't a problem. ... With compression ON a 2MByte empty TW will be about 350kByte in transit! So if you have a slow connection it will make a difference. 

I see. I am not much concerned for my current use case (my biggest TW is 10 mb) but that can lead to quite a bit of performance hit for larger files.

I may have missed this info. Can you give more details?

Actually I was having the exact problem previously discussed among you and others in this thread. when saving a fresh empty @version 5.1.16 over webdav, first save succeeds, second save fails with error: "Error while saving: XMLHttpRequest error code: 412". It is all explained there, I must have missed it.   

Best,

Altug 

[PMario]

On Friday, August 31, 2018 at 5:15:25 AM UTC+2, Altug Ozcelikkale wrote:

...

I may have missed this info. Can you give more details?

Actually I was having the exact problem previously discussed among you and others in this thread. when saving a fresh empty @version 5.1.16 over webdav, first save succeeds, second save fails with error: "Error while saving: XMLHttpRequest error code: 412". It is all explained there, I must have missed it.  

I think the problem was introduced 5 month ago, as a different "code-typo" was fixed. So 5.1.16 is the only version with the problem, you described.

Every earlier version has a different WebDav problem, which is less obvious: Etags aren't used at all, because of the code-typo ...

So V5.1.17+ should be used with WebDav settings.

-m