Server setup tutorials; anyone interested?

[Lost Admin]

A while back PMario made a video on how to set-up a webdav server on IIS. I also made some observations on tiddlywiki's WebDAV saver using Apache HTTP. Since then, the issue I had in my observations has been corrected (awesome work by the way to Jeremy and anyone who helped). As far as I can tell the WebDAV Saver works very well on IIS webdav and Apache webdav (I run both).

I imagine many people on this list are technically savvy and capable of setting up their own server for TiddlyWiki. But I do wonder if there are people who want to do so but would like a tutorial to follow.

When I first found Tiddlywiki, I initially used the download saver. Shortly after I learned about TiddlySpot and started using it (I still do). That lead me to finding out how it works and learning about "store.php" and the various GitHub repositories for variations of it.

When the WebDAV saver became available, I switched my home server (just a little always-on Intel NUC) to use WebDAV and removed store.php (and PHP). Lately I've been playing with Note Self, which uses the Apache CouchDB as a back end database, and set-up my own small infrastructure on vultr.com (cloud vm hosting).

None of what I've done is particularly exciting but it does require a certain amount of knowledge. For the most part, I found tutorials on how to set-up the various components, followed them, and then read documentation and fiddled with settings until I was satisfied everything worked smoothly.

What I didn't do, was create good documentation on how I actually did what I did. Before I actually set-out on doing so, would people actually use it?

or

How many of you want to set-up your own dedicated server for TiddlyWiki but want a tutorial to follow?

I'll be creating one for a basic self-hosted Note Self CouchDB back-end and posting it on the Note Self forums regardless.

[Ste Wilson]

I'd be up for those. Currently trying to set up a raspberry pi headless server for some light use/ proof of concept. As you say.. Its all out there but sometimes it's more tricky than it could be to string it all together.
It's the simple things that trip me up. Spent far too long last night trying to get an ftp server running from the pi and be able to successfully connect to it...
But... You know.. Linux.. It's ready to take over windows... Honest guv! ;)

[@TiddlyTweeter]

A Noteself backend tutorial I absolutely know would be used. It is where tech knowledge is less DIY enthusiast and more "I'll give it a shot if I know what buttons to press to get this great thing working."

I did notice unclarity about WebDAV IIS. That's maybe slightly more DIY enthusiast, but sure a step by step tutorial could be really good too.

You should maybe ask us to agree to actually do it to check the instructions work and that we are not just giving you work :-)

Best wishes
Josiah

[Lost Admin]

On Friday, May 18, 2018 at 10:57:50 AM UTC-4, @TiddlyTweeter wrote:

...

You should maybe ask us to agree to actually do it to check the instructions work and that we are not just giving you work :-)

Best wishes
Josiah

I'll take that as you agreeing to test it out once I publish it. :-) 

[@TiddlyTweeter]

Yep. So long as you grasp I am an idiot :-).

[Tristan Kohl]

In March I wrote a post about how to setup a almost plug and play WebDAV server for Tiddlywiki using my favorite webserver Caddy. Maybe that helps some people who do not want to fiddle around with complicated Apache configurations and there likes.

https://groups.google.com/forum/#!msg/tiddlywiki/EgJrutChcNc/AYXQYfQ0BwAJ

Cheers

[PMario]

On Friday, May 18, 2018 at 4:30:32 PM UTC+2, Lost Admin wrote:

A while back PMario made a video on how to set-up a webdav server on IIS. I also made some observations on tiddlywiki's WebDAV saver using Apache HTTP. Since then, the issue I had in my observations has been corrected (awesome work by the way to Jeremy and anyone who helped). As far as I can tell the WebDAV Saver works very well on IIS webdav and Apache webdav (I run both).

I did some tests with a nginx webdav setup. It works there too, with the right settings :)

I also made some experimental modifictaions to the existing WebDAV saver. The etags generation seems to be very different, depending on the server. .. That's why I did change the setting to use "if-unmodified-since" header. Which also allows the server to use compression. I personally think, this setting is less "damageable" as the etag handling.

I'll create a plugin for this one, which may replace the default WebDav saver if installed. .. They can't be active at the same time. ...

Or we make it more configurable. ... But I'm not sure ... yet

-m

[Mark S.]

It's been a long time since I've had to do web-work without some interface like cpanel. It would be interesting to see how you set up webdav on vultr. Is it considered secure? (at least secure from hackers, that are always looking for a place to put their spam-links) ?

The next question is, for home/office use, what is the advantage of WebDav (which seems to have a very convoluted set up) over tiddlyserver or Bob ?

Thanks!
-- Mark

[Tristan Kohl]

The main reason for me to take the WebDAV route was the limited hardware I run on. I have a Raspberry 3 as my home server which serves me quite nice for all my needs (contacts, calendar, file sync,...) and I did not want to upgrade just to have more than a few wikis. The problem with Arlen's TiddlyServer was that it starts a new TW instance for everytime a new wiki gets browsed for the first time. Since my Pi runs on 1GB of RAM it was only a question of time when that memory runs full - and I hit that wall, it was not nice.

As far as I can tell from reading Jed's post Bob is for multiuser environments which I have limited use for. My main use for TW is to record and keep track of various projects and things around me, so most of the time it is just me adding stuff to the files. That is also why it is ok for me that the WebDAV saver uploads the full wiki on every save as there are only occasionally updates to the files I have.

Cheers

[@TiddlyTweeter]

As far as I can tell from reading Jed's post Bob is for multiuser environments

Jed's "multi-user" has had terrible issues with being misunderstood. Yes, it enables simultaneous access over a network IF you want--but it can also be used for immediate local needs just as well.

BTW Jed partly developed "Bob", I believe, (what used to be called "Multi-User") in order to be able to control robots through Raspberry :-)

Your own WebDAV route sounds good!!

@TiddlyTweeter

[TonyM]

Lost Admin,

I am keen to learn how to set up my own CouchDB (I am on the NoteSelf Forums) and I try every saver and hosting option to try and establish its use cases.

My biggest gap is when It comes to a Cpanel host, I am a re-seller of hosting and would like to host  savable TiddlyWikis there so I can make them read writable for user and client interactions but have never got it working. Do you have any idea on this.

I will also test your published documents and help in any other way I can.

Thanks for such helpful contributions

Tony

[Jed Carty]

One reason Bob works the way it does us because I run it on a raspberry pi and tiddlyserver kept using up all the resources, so Bob uses a single node process that serves all the wikis.

the multiuser part isn't the big selling point for Bob, the immediate two way syncing of individual tiddlers between the browser(s) and the server so you don't accidentally open a new tab and lose changes and all the browser tabs are in sync is the biggest reason I use it.

simple solutions are good, if webdav is doing what you want than it is a good solution. I just don't want there to be too many misconceptions about what Bob is.

[Lost Admin]

My WebDAV setup is TLS only and currently uses Apache Basic Auth (so you need a username and password). I thought about setting up Client Certificate auth but haven't done so yet.

It's been running for many months so far with no weird files showing up unexpectedly (so either it's secure or the hackers are better at hiding files than I am at finding them).

[Lost Admin]

I only use Cpanel when I am forced to by a service provider. Given the choice, I'm old school and edit config files by hand. If you know how to turn that into something that can be managed through Cpanel, I wouldn't mind you explaining it to me after I write up the tutorials.

[Lost Admin]

As there does seam to be interest, I've started. Note: I haven't even finished the first tutorial as I'm writing this. But, you can have a look anyway. I'm putting them on Tiddlyspot. It seams like a good place.

http://thelostadmin.tiddlyspot.com/#Tutorials lists what I've got planned.

The first tutorial is a minimal setup of CouchDB so that people can test running their own server for Note Self. You can see the progress on that one at http://thelostadmin.tiddlyspot.com/#How%20to%20Setup%20CouchDB%20for%20Note%20Self

[TonyM]

Lost Admin,

I have spent years at the Windows DOS Prompt and can do that very well, I am not so quick at the bash prompt, in part because they are "similar but different". The Key advantage to cpanel is I can browse files with a file manager, edit files in place upload/download, set DNS all through a dialogue. Of course the key features of the cpanel environment is PHP MySQL and apache etc... on on a linux variant.

If I had instructions at a logical level eg; edit the .htaccess in the Public_html folder to include... I would find my own assisted way to do everything necessary.

I have struggled trying to make store.php work to host editable tiddlywikis, and are keen to find any practical way to host Tiddlywiki on the internet (other than read only or TiddlySpot) , I believe by definition I actually have a lot of different opportunities in my hosting platform, however I do not have the information to proceed with. Of course NoteSelf with a PuochDB/CouchDB is clear (Since you only serve the file) but I wonder if I could install nodeJS?

Thanks for your effort.

Tony

[TonyM]

PS The hosting service seems to run on CLOUDLINUX 6.9 vmware

Regards

Tony

[Mat]

Lost Admin wrote:

How many of you want to set-up your own dedicated server for TiddlyWiki but want a tutorial to follow?

Half the reason I'm working on the Backend project is to help others. The other half is to help myself; I'm totally lost in this jungle. So yes, I would very much appreciate tutorials on this. I will likely have Backend link to the all tutorials and guides available.

Thank you for generously sharing your knowledge.

<:-)

[Tristan Kohl]

Hi Jed,

well it looks like I fell for the multiuser keyword. I will give Bob a try for a few wikis I share with my family so there might be race conditions - altough there never were so far.

Cheers

[@TiddlyTweeter]

Got to the beginning of Step 2, no problem on LeCouch. Behaved word-for-word as described on Windows 7.

Looking forward to the next instalment to fill in the TBD!

Best wishes
Josiah

[@TiddlyTweeter]

Lost Admin wrote:

The first tutorial is a minimal setup of CouchDB so that people can test running their own server for Note Self. You can see the progress on that one at http://thelostadmin.tiddlyspot.com/#How%20to%20Setup%20CouchDB%20for%20Note%20Self

Great stuff as mentioned in previous post.

ONE THING I'd like to grasp better is what the serving executable is called--so I can abort it in Task Manager until the TBD gets live :-)

Best wishes
Josiah

[@TiddlyTweeter]

Further to last, in Windows, CouchDB sets itself as a Service. Its easy to stop, BUT you need Administrator privileges.

Am I right?

Just wondering.

Josiah

[Lost Admin]

I know it's been a while but I've mostly finished the Note Self CouchDB setup tutorial. It should get you to the point where you actually have a working Note Self & CouchDB test environment. Anyone willing to test, please do so and let me know where things aren't as clear as they could be.

I need to reset my Windows setup and re-built it to get a few more screen shots (and validate I didn't miss anything).

Once that is done, I'm going to make a second tutorial that adds HTTPS support and makes it reasonable to consider putting it on the Internet. You can get a VM for as little as US$2.50 that is sufficient for a single user, so I figured HTTPS would be worth explaining. I'm currently doing this with Vultr.com (https://www.vultr.com/?ref=7210118)[1] and using Note Self as a privately hosted One Note alternative. 

Once I'm done with the tutorials I will configure the CouchDB on my home computer to sync with the one on Vultr so that I even have off-site backup. I'll probably make a tutorial on this as well. It is dead easy.

[1] If you sign up with Vultr and pay for hosting, I get a token amount of credit for directing you there. If this is considered inappropriate for this forum, let me know and I'll remove the link and mention of Vultr.

[Ste Wilson]

Am attempting to follow on my pi.
Couchdb is not a package automatically in the repositry.
Just to be picky, if it were, it would be sudo apt-get install couchdb

I'm currently so far unsucsesfully following the instuctions on Apache couchdb to install it.

[Mark S.]

I'm using xubuntu 18.04. Couchdb is not in the repository. Wonder why? And, attempting to add it with the specified PPA doesn't work either -- says there is no Release file.

It appears the couchdb files for ubuntu were released just 4 days ago. It kind of looks like there is a release file.

-- Mark

[Mark S.]

Ok. I was able to install Couchdb on a live session of xubuntu. Imagine that internal arrangements will be similar with Debian and Ubuntu. I used the existing Couchdb instructions but substituted "bionic" for "distribution", since the existing documentation is lagging behind.

The option configuration file appears to be at :

/opt/couchdb/etc/local.ini

The Couch setup menu seems to have been updated. The "Security" menu isn't there, but instead there is "Permissions". You don't appear to need to specify admin and members with ["name"] syntax any more. There is no "Update" button.

I was not able to get noteself to connect to the new database. Well, I have no evidence that i connected.
At no point did it pop up and ask me for name/password. Attempting to create new tiddlers did not appear to update the database I had set up. There were cryptic error messages in NS. The couch  screen reported 0 # of docs. I assume that NS will create docs? But it also said 225KB of data. But I could not find any way to display that data or even a field listing, and the number didn't budge when I copied in an image file. The only data I could display was a short pieces of JSON, which could not account for the 225k.

Even if it all worked, how would someone set it up offline, if they weren't able to connect to https://noteself.github.io/online/ ??

I attempted to delete the database data locally (through the pouch plugin utility) and then re-connect to the database. But no tiddlers were restored from the local couch server. As far as I can tell, there was no communication between the server and the NS instance.

The main configuration screen doesn't have a place for a user name, but the pouchdb in the settings does. Shouldn't they both have a field for user name?

As I've mentioned before, the code for deleting an instance appears to leave significant bits behind. So when you try to reconnect there is a conflict between the pieces that it does and doesn't remember.


-- Mark