Taking Node Server to the next level

[Arlen Beiler]

I have a radical proposal which would take data folders to the next level. What if instead of the file system adapter we would write a new adapter to use a database. We could use PouchDB, but I would vote for something much more widespread like SQLite. We could also write it in a generic way that makes it easy to use with the regular SQL databases. It would be easy to build but I would want to make it robust enough to use in a wide array of platforms. 

I have already worked with the tiddler loading code enough to be certain it is self contained and can easily be made asynchronous to accommodate this feature. Or the preload tiddlers feature can be used, but I think it’s better to separate out the loaders as Jeremy has mentioned. 

Any thoughts on this or things I should keep in mind as I brainstorm? 

Arlen

[TonyM]

Arlen,

I would think this a great idea. SQLite, MariaDB etc... could be used. I use a cpanel account on a hosting service and often jump in behind the scenes to manipulate databases. It would be quite easy to define additional databases with an sql script designed for Tiddlers or TiddlyWikis.

In the spirit of  brainstorming.The vast majority of my ideas may be implemented in tiddlywiki itself with SQL only the underlying database.

The ease of securing online hosting for SQL databases would be a big plus as it is less onerous than the lesser available databases like CouchDB.

Of course in time it would be good if we can ensure an editor at a time, or multi-user access to such a tiddlywiki database but simple single edit (Multiple read/Public) user implementations would be valuable. I believe a lot of control can be designed into this at the tiddlywiki level, just using the database as storage, tiddler at a time. Backend database tools could also clone wikis etc... and if we could get such an implementation script into installatron, softaliciouse etc... more people would discover tiddlywiki.

The ability to store tiddlers in a sql database could be handled more than one way.

• Allow an online readlony hosted wiki to connect to a database and with the right credentials save changes to tiddlers in an sql database and reload them as needed somewhat like Browser local storage does.
• Store all of the wiki in a database. Or perhaps with the javascript or core in a file to help launch the wiki.
• I believe there is a possibility for users to easily install SQL DB's they can connect to locally (Device or LAN) and overcome the limitations of browser local storage with trusted persistent storage (We can possible find multiple OS installs for this).
• One minimal tiddlywiki could be configured to connect to more than one database, but one at a time, so the user or designer selects which database they are using (Like CouchDB), thus each user could perhaps have their own version of a tiddlywiki.

I think however we need to ensure that a full wiki single file wiki can still be exported and ideally one imported into a database. Perhaps we could generate an SQL script from a living tiddlywiki and run that in PHPAdmin to build the database, then such scripts could be generated for various editions.

One big advantage with SQL is there will be new opportunities for integration with other systems. Other systems could submit tiddlers to the database or query the database to interact with tiddlywiki. This integration includes between tiddlywikis, eg same database different tables/wikis and some primitives to use the other wiks tiddlers.

Since I have investigated the development of a community resource website I have fancied building a WordPress site and having tiddlers in one or more wiki tables but with the ability to access content from the wikis in WordPress and Visa Versa, such as plugin and macro code and more. This would allow us to build a meta community site that can handle all the different TiddlyWiki objects, plugins, macros, editions, documentation submissions and discussions and more. I have toyed with new post types that allow people to submit content such as a plugin via a custom wordpress post that can then be dropped on a wiki. Buy doing this we can categorize sort, file, discuss .... Making us of WordPresses multi-user and OpenID and Auth systems over multiple tiddlywiki resources.

I would be quite prepared to test, participate and feed back on any such project. To me this could be the mainstreaming of tiddlywiki.

Regards

Tony

[ILYA]

Nice idea. Another option is git instead of a database.

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[Jed Carty]

A while ago I made a syncer that used sqlite and from that I wouldn't suggest a relational database, SQL seems like it would be almost perfect for storing tiddlers but I immediately ran into problems with the database schema because aside from text, title, tags, created, type and modified you have on way of knowing what fields are going to be used so everything else just gets thrown into an otherfields column and you lose the desirable features of a relational database.

I found pouchdb to work much better because you can have arbitrary structures in the documents and implement the tiddlywiki filters pretty exactly in calls to allDocs and as a document store it can store single file wikis without any trouble at all.

I have been considering skipping pouchdb and just using leveldb directly on the back-end but I don't know if the performance improvements are worth the extra hassle.

Also in pouchdb you just make a new database for each wiki and never run into problems of scale except in the most extreme cases. I have had pouchdb databases that were close to a gig without any real performance hit, I think that wikis anywhere near that size are going to be rare.

I have looked into other nosql databases and none of them look like they would be any better than pouchdb or using leveldb with some basic namespaced storage setup.

also in my experience database maintenance is much easier in pouchdb and leveldb than in MySQL/MariaDB. You don't have the same problems of having database files taking up 100gigs when you only have 4 gigs of data that changes often. I am sure that problem can be solved by having an experienced database maintenance person, but I think that with one of the nosql  databases you could skip that completely.

[PMario]

Hi,

If you would want to have a cool new database ArangoDB may be an option.

ArangoDB is the open-source native multi-model database for graph, document, key/value and search needs.

So we could store tiddlers as "documents" and add some graph info to create a "map" like structure.

It also contains a JS framework named Foxx:

Foxx is a JavaScript framework for writing data-centric HTTP microservices that run directly inside of ArangoDB.

So the nodejs stuff could directly run inside the database. The only thing we would need to define is a consistent API.

Just some thoughts.

-m

[PMario]

Some more thoughts.

The problem with databases I see are:

 - Our users like files and will go a long way to keep it that way.

 - DBs need maintenence

 - DBs need to be secured

 - ...

So in the end we need a service provider, that abstracts the complexity away.

I personally would like to use a dat-project based approach, which is similar to a "file sync" mechanism. ... The beaker-browser uses dat as its foundation.

PROBLEM here is:

 - It's a young project with contains some fast moving targets.

If you have a closer look at the dev-diary videos, I think you can see, how nicely the "new" approaches would fit to tiddler-based TWs.

 - Concepts show in the videos can only be tested, if you compile the developers branch on your own.

have fun!

mario

[PMario]

One more thing:

 - The cool thing about DAT is, that it will nicely run on a raspberry pi 3, with a big harddrive attached.

-m

[Jed Carty]

ArangoDB looks very interesting as an option. I haven't run into any multi-modal databases like that before.

It does look like the data model is just about ideal for tiddlywiki in terms of storing tiddlers and full html files, storing html files as a string shouldn't be a problem. A quick glance didn't show any binary data storage for media attachments, but base64 encoding should work there.

My only concern with ArangoDB is that it looks like it is a system wide install instead of specific to the process running it, leveldb (and by extension pouchdb) don't require additional installation to function.

Running a ArangoDB for hosting peoples tiddlywiki like tiddlyspot would probably work well, I am less certain about local installations.

[PMario]

On Friday, November 29, 2019 at 12:45:17 PM UTC+1, Jed Carty wrote:

...

Running a ArangoDB for hosting peoples tiddlywiki like tiddlyspot would probably work well, I am less certain about local installations.

You are on the point. I do have similar concerns about every database ... They store the info in "hidden" places. I think our users want files, somewhere in the workflow.

That's why I'm so interested in the peer-to-peer approach of the dat-project. I think the "file-system" they use has a lot of functions built in, that we want to have for TW. _and_ at the end the binary data-store can be mapped to real files.

Watch the videos. All of them and you will see, what I mean.

-m

[Mark S.]

The problem with solutions like PouchDb is that there are no turn-key installations. You have to pretty much do everything yourself down to the bare metal.

SQL, on the other hand, is supported almost everywhere.

The workaround re fields is to specify the fields you want in advance. Rebuild the database when you need to add fields.

Or to have a dozen fields with generic names in the database, but then map them with a configuration file.

Mostly, I'm saying that a solution that is for everyone needs to based on technology that's commonly available.

Of course, there's nothing to prevent someone from creating cool stuff, just because.

[Jed Carty]

Mark,

The benefit of PouchDB is that it can be installed using NPM, or packaged into something like BobEXE without having to do a system-wide installation. For a local installation PouchDB is much easier to set up than MySQL/MariaDb. If you didn't go the packaged route like BobEXE it would at worst be cloning a repo and typing `npm install` to install the node dependencies. Then you don't have to worry about setting up users or anything on the databases, it is all self-contained.

It is as easy to set up as sqlite, and if there are any MySQL/MariaDB servers that are that easy to set up I would very much like to know because every time I set up a database I forget how to do it and I have to look up how to set up new users and permissions all over again.

On a remote server it would be at most one extra command when installing node and tiddlywiki, and that is only if we don't have an install script set up.

If you want something larger on a remote server, CouchDB has the same interface as pouchdb and is as easy to set up as MySQL/MariaDB in my experience.

I don't know if many hosting providers give out of the box CouchDB installations, so that may be a problem, but I think that anyone in a situation to need something that heavy duty would be experienced enough to not have trouble with it.

[Arlen Beiler]

I'm kind of thinking that if the user is setting up MySQL themselves, then they have control of the server and might as well be setting up TiddlyServer or Bob anyway, unless MySQL is up their alley, in which case it's no big deal. On the flip side, using PouchDB or SQLite instead of the file system would allow some more advanced solutions such as multi-user editing and version control to be built in much easier. 

I'm hoping eventually to find a way to host wikis in TiddlyServer without using the Node server instance, since it does consume a certain amount of memory per wiki, and this is one step in that direction. Once that is accomplished we could transparently write a PHP version of TiddlyServer that could be used on any shared hosting. Once that happens TiddlyWiki could become a serious contender as a self-hosted multi-user wiki. 

But since my paradigm is to operate within the constrains of current TiddlyWiki code, I have to take it one step at a time. 

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/1fb383cf-b1d0-4cd0-9768-a6f315076b53%40googlegroups.com.

[Mark S.]

I was thinking of remote hosting and CPanel. I found there was pretty much no support for that or any easy equivalent.

But if it can be installed locally with NPM, then that could be interesting.

[TonyM]

Npm node etc... Is really easy to use locally and to serve to local area networks inside a firewall. However as pointed out by Jed this can be a liability on mobile computers joining other networks. This can be managed. My wholesale host let me install node internet facing but its not a documented service an

[TonyM]

Cont...

Sorry. Mobile gg can be a pain.

I understand Jed has done a lot of on securing bob when internet facing.

So what I am asking is if node where more common on hosting and safe to open to the internet could we not focus on a smaller subset of technologies with similar outcomes?

Of course we value a diversity of solutions.

[Arlen Beiler]

Most Node solutions that I am aware of are either request-based (AWS Lambda and Cloudfront Workers), or VM based. There's also Heroku, but it doesn't allow a persistent file system. However, the data storage structure is a more serious bottleneck for innovation, I believe, and once we have a working system, it's easier to move to the next step. The biggest thing I see is the unpredictable nature of the file system as a predictable per-tiddler storage system. While it is possible to load tiddlers from the file system, I feel like some of the inherent drawbacks in this system are preventing us from taking it to the next level. 

Once we store tiddlers in a database, we can write an alternate server that just stores client tiddlers straight to the database without going through a $tw instance on the server. We'll still have use for the server to bootstrap this system, but once setup, it can be fairly self-contained. In that sense, perhaps taking the node server to the next level means becoming smarter about the common use cases and writing software that can handle them more efficiently. Not much has been done to make use of the server itself -- most of the Node-side innovation is in other commands. The server was recently rewritten to make it more flexible for server-based solutions, which is useful. But I'm seeing a lot of client-side solutions that don't need the server at all, except to store tiddlers and files. 

In one sense it feels like a bit of a departure from the major possibilities that the TiddlyWiki server presents. And for me that's a little sad because I read source code and the TiddlyWiki server has enormous potential to work along-side the TiddlyWiki client in the browser for various solutions. But the general use-cases do not need this type of functionality, and indeed are encumbered by it because I there is no good way to tell whether a particular data folder needs the server-side components or not. 

Right now I could write a server using the filesystem adapter as my spec that would serve data folders using Apache and PHP, but there is no good way to tell whether any particular data folder might require the server-side instance, so it has to be explicitly stated in some way, and for that we need to come up with a new data folder format. 

The data folder could be marked by the specific presence or absence of certain plugins or other tiddlywiki.info properties, in order to properly trigger an alternate format, but I think the best way would probably be simply to add a property to tiddlywiki.info like "clientOnly": true. This would allow existing data folders to be converted easily if the owner knows it is safe to do so. It's not hard to implement this system with the current data folders either, it just requires a similar amount of effort, which may be necessary anyway.

So I guess that breaks this up into several parts. 

• First we need to "invent" client-only data folders and implement them. 
• Second, we need to attach a more robust storage system that isn't encumbered by the various file systems. This would allow us to implement multi-user and version control quite a bit easier. 
• Third, we need to implement this in PHP as an identical implementation. 

I know it's a lot, but that's my summary of the what I'm seeing with what we've discussed so far. 

Arlen

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/e7fd7b93-d44d-4de2-bbe5-63492ce18501%40googlegroups.com.

[Jeremy Ruston]

Hi Arlen,

Thanks for starting an interesting thread. While the syncadaptor architecture of TW5 is very basic, there’s still a lot of potential for connecting to other things. One of the pleasures of open source is that dozens of people can be working on different syncadaptors and we all benefit from the rich ecosystem that results.

> Any thoughts on this or things I should keep in mind as I brainstorm? 

Over on GitHub I’ve been working on a PR that fixes a number of problems with the current syncer architecture. It’s nearly done, but in the meantime you may want to base your efforts off the “fix-syncer” branch:

https://github.com/Jermolene/TiddlyWiki5/pull/4373

The scope of the changes is mostly the syncer. The syncadaptor interface is the same, but the internals of the syncer are thoroughly overhauled.

* Proper recovery from network errors in the browser

* Syncing changes to system tiddlers from the server to the browser

* Syncing deletions of tiddlers from the server to the browser

* Adding optional parameters to GET recipes/default/tiddlers.json for filtering which tiddlers and fields are returned

There’s also a new extensible dropdown menu for the synchronisation status (see below).

Best wishes

Jeremy.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/CAJ1vdSS3Z4J5asgfhG82aHK6%2BwRxVwR-kpH%3DvBuea%3DnBD2uBdg%40mail.gmail.com.

[Dave]

I'm just a TW user (not in the computing profession), but if y'all are open to requests at this early stage here are my 2 cents:

Yay for increased stability (I assume that's what this is about)

1 vote for making this thing a snap or app image or flat pak that an amateur like me could install in an old computer or pi on the home network and see from another network safely.

1 vote for at least being able to act as if tiddlers were still text files you can modify outside the browser (e.g. an API that allows you to modify or create tiddlers from the command line)

P.s. I love the "save snapshot for offline use later" idea. I've more than once lost work because I didn't notice the big red banner that I was not connected to the server, ha ha

[Jed Carty]

After finding out that I have very little idea how syncadaptors work I started playing with making a new one and ended up with a working pouchdb syncadaptor.

It is very rough but it works, it will create a database and save tiddlers to it, delete them when appropriate and then load the tiddlers when you open it up again.

At the moment the implementation is just for one wiki, but extending it to have multiple wikis on the same node process would be very straight forward.

It has potential for some things people have asked for like being able to search all tiddlers in all available wikis.

As a note, this is not noteself, while it could be configured to connect to a remote database, either pouchdb or CouchDB, it doesn't have any of the nice features of noteself other than saving to a database.

Considering how straight froward this was making similar adaptors that work with the various flavours of sql databases wouldn't be too hard aside from figuring out the appropriate database schema. That is something I have absolutely no interest in doing but this may work as a reasonable reference design for anyone who wants to make a sql database adaptor.

There are instructions on how to use the adaptor in the readme, it is not at all polished and requires node.

https://github.com/OokTech/TW5-PouchDBAdaptor

[TonyM]

Jed,

Great work, what a Wiz. I will have a closer look. With the pouchDB have any more persistence that local storage I wonder?

Such node solutions are great but we must find an easy way to host such things online when needed.

Regards

Tony

[Jed Carty]

It is stored locally on the hard drive like other node wikis, the difference is that this is stored in some database files instead of individual tiddler files. It is not localStorage or a browser based solution.

If you use the browser based version of pouchdb it is generally built on top of indexeddb or websql which are just localStorage. It doesn't have any magical fixes and isn't any more persistent than the localstorage plugin. In browser it is just localStorage with a slightly different interface and data model on top of it.

To put this online you would end up just duplicating noteself I think.

[Jeremy Ruston]

There have been quite a few experimental syncadaptors over the years. Some of them come back from a simple GitHub search:

https://github.com/search?q=syncadaptor

I went through the first couple of pages and found:

* an old syncadaptor for CouchDB: https://github.com/cjxgm/tearly/

* a syncadaptor for GitHub: https://ustuehler.github.io/tw5-github/ 

* another old syncadaptor for GitHub: https://github.com/kpe/tw5-github-syncadaptor/

* a syncadaptor for SoLID linked data stores (see https://solid.mit.edu/): https://github.com/linonetwo/solid-tiddlywiki-syncadaptor

Another syncadaptor that I'd love to see is one that stores tiddlers in an online Google Sheet. Done right, one would be able to seamlessly switch between editing the same data within TiddlyWiki and via the Google Sheet user interface.

Best wishes

Jeremy

--

Jeremy Ruston

jer...@jermolene.com

https://jermolene.com

On 2 Dec 2019, at 07:29, Jed Carty <inmy...@gmail.com> wrote:



It is stored locally on the hard drive like other node wikis, the difference is that this is stored in some database files instead of individual tiddler files. It is not localStorage or a browser based solution.

If you use the browser based version of pouchdb it is generally built on top of indexeddb or websql which are just localStorage. It doesn't have any magical fixes and isn't any more persistent than the localstorage plugin. In browser it is just localStorage with a slightly different interface and data model on top of it.

To put this online you would end up just duplicating noteself I think.

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/d406b178-989e-4c43-a9e4-a2efb476bdc2%40googlegroups.com.

[mauloop]

Why not to reuse Tiddlyweb/Tiddlyspace work?

• It has MySQL backend
• It is multiuser and allows concurrent editing
• It has public and private spaces
• It uses the standard Node.js sync-adaptor
• It works with TWC as well

IMO it is a great piece of software. Not easy to set up. I did it on a local VM just to give it a try and Chris kindly helped me to solve some problems I fell into. I always feel bad when I see that good software falls into oblivion. Maybe it is not to be taken as is, but it could be a good starting point. Just to not reinvent the wheel.

Another thought is... this is an old paradigm. PMario's suggestion (DAT protocol) goes one step beyond, moving towards the p2p paradigm. I use Syncthing to keep my devices synced. Both (DAT and Syncthing, which, I think, uses QUIC behind the scenes) provides versioning, encryption and, last but not least, ... privacy!

TWedereation goes this direction too, isn't it?

I have no much developing skills to contribute, but I would be happy to help testing if needed.

)+(

[TonyM]

Jeremy

I do have a possibly faulty distant memory of a tiddlywiki writing to a Google sheet. I found it curious but it had no use case for me at the time. I will see if I can find anything.

We are spoilt for choice in some ways yet I feel we need to remove a few speed humps.

Regards
Tony

[TiddlyTweeter]

TonyM wrote:

I do have a possibly faulty distant memory of a tiddlywiki writing to a Google sheet.

I *think* Mat did that ... hang-on ... yes: https://groups.google.com/d/msg/tiddlywiki/QcTOJCVXkDw/kS5VBW0ACQAJ

It uses Google Forms & Spreadsheets.

Best

TT

[Arlen Beiler]

PouchDB is a very good first step because of the way it is setup. SQLite would also be a great first step for targeting SQL databases. 

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/952167f3-cc7f-44a7-8093-629b2b3ef1a3%40googlegroups.com.

[Arlen Beiler]

I think in this case I'm rather aiming to build something new on top of TiddlyWiki5. I may revisit the rest of the TiddlyWeb protocol at a later time and take some ideas from it. However, I think that to build something new it would need to be built using TiddlyWiki5 from the ground up, whereas tiddlyspot was built for TWC. That's just how I work best. For someone else to take a different approach would not bother me at all. 

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/cb3cdfd0-e05a-417a-a329-1281e2064ecb%40googlegroups.com.

[Arlen Beiler]

Well, Jed's adapter and everyone's thoughts so far inspired me to do a little exploring and I managed to put this little proof of concept together. It turned out I did not need to touch the client at all and only modify four routes on the server. What's nice is that the index is still generated as usual so you can turn off the plugin and make changes to the initial HTML document if you want, then turn it back on and all your database tiddlers come back again. Almost like a template. The plugin is inside the data folder for now, but the npm install needs to be done in the tiddlywiki root folder (the folder containing tiddlywiki.js). The readme has rather simple instructions and the npm install command that needs to be run. 

[Arlen Beiler]

And here's the link for it:

https://github.com/Arlen22/TW5-storage-plugin

[bimlas]

Arlen, Jed,

Thank you for supporting the community with your knowledge! Big respect!

Database backends look really useful. If I understand it, maybe even PouchDB can be solved online by Glitch: https://glitch.com/~pouchdb-server (click on show to fetch some data).

Another option is Cozy, which, if I understand it, is a free online cloud. This does not require the PouchDB saver, but a unique one, but may be used. If I have more time, I'll dig into the saver world more.

I don't know how Bob is solved, but here is a tutorial that can easily wrap both Tiddly and saver, so you can run it without installing Node.

[bimlas]

Cozy link: https://docs.cozy.io/en/tutorials/konnector/scrape-data/

[bimlas]

There are two things that I think can be used for online storage:

On the one hand, the GitHub saver could be upgraded to save the tiddlers one at a time using the datafolder format instead of a large HTML (so you wouldn't have to wait minutes to save, just wait for one tiddler to save, like the Node server) work). This way, the wiki would be completely online, because when we save the changes, there would always be a commit that triggers the HTML wiki generation, so we could open it and modify it on GitHub without ever having to store it on our own machine.

My other idea is that Disqus, or a similar messaging system, can be used to store the tiddlers that we could access through the API.

For the time being, these are just ideas, but I will write them down to remember them and to give others ideas to complement.

[Jeremy Ruston]

Another interesting sync adaptor would be one that retrieved and stored tiddlers via the Wordpress API. The attraction is that WordPress hosting is highly commoditised, being readily available and cheap. It should be possible to store tiddlers as “Pages”, and to inherit WordPress’s very smooth handling for media and attachments.

https://developer.wordpress.org/rest-api/

Best wishes

Jeremy.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/5bb9370d-0f8c-4623-9033-94c8f7de42c5%40googlegroups.com.

[TonyM]

Jeremy,

we could make use of the sql database that is build to support word press. However as you say using the api would be more sophisticated.

I have played with building custom post types and would be interested in creating a tiddler post type. We could also support plugins as JSON packages.

Tiddlers as a custom post type would make them full word press citizens and enable a meta site. We could store macros doco and more. Perhaps even click and create wiki editions.

See my initial investigation and ideas in the choice of menus. http://www.colabteam.net/tiddlywiki/ I own the domain and hosting. Bit I have not put cloudflare can on yet.

I am all for drinking our own Champaign but until we get multiuser sites worked out I see value in an open id community site. I can install almost anything on my host including node but the security on internet facing node is unknowen to me.

Regards
Tony

Tony

[TonyM]

Bimlas

I would love to enable contributions to tiddler content via got hub a bit like when you edit tiddlywiki.com but have the changes appear online. We could open our published editions to issues and changes. But your idea sounds even better.

Regards
Tony

[PMario]

Hi,

There has been an idea to use e-mail accounts to store tiddlers using the IMAP format. e-mail hosting providers are reliable and cheep. You may even have an account combined with your smart phone contract, but probably not used.

just a thought

mario

[bimlas]

Jeremy, TonyM,

Another interesting sync adaptor would be one that retrieved and stored tiddlers via the Wordpress API.

I think the main problem with "database backends" (Google Sheets, Wordpress, PouchDB) is that by default they are not accessible as a web page, but everyone has to set up credentials to read and edit the wiki. Sorry if I misunderstood it but haven't used Tiddly in this way. If I understand it, something like a database backend would look like this:

* You open an empty Tiddly

* Configure credentials for the saver

* Saver synchronizes (downloads) tiddlers

* When you make changes, the changes are saved to the database

If it doesn't work that way, I apologize and please someone write down the workflow.

The problem is you don't always want to give credentials - for example, you want to make tw.com visible without logging in. The easiest way to make your wiki public is to recreate the HTML (which anyone can read) after each change, and if you have credentials, you can also edit the tiddlers via this generated HTML. I find GitHub / GitLab best suited for this workflow, thanks to continous deployment.

I would love to enable contributions to tiddler content via got hub a bit like when you edit tiddlywiki.com but have the changes appear online.

A workaround is to add new Personal Access Tokens to the repository, one for each user, so they will have their own "TiddlyWiki password". 

[TiddlyTweeter]

 bimlas wrote:

I think the main problem with "database backends" (Google Sheets, Wordpress, PouchDB) is that by default they are not accessible as a web page, but everyone has to set up credentials to read and edit the wiki. 

 

A workaround is to add new Personal Access Tokens to the repository, one for each user, so they will have their own "TiddlyWiki password". 

I think the use of "Tokens" is a good way.

The issue I'm looking at as a non-tech user is simply: what would my potential users want?

Personally when I hit a new site that has, say, a dedicated (i.e. its own, not a disqus etc.) commenting system, the barrier to entry is "yet another login credentials set" you have to maintain.

I think for wider apps the widely used system to login using existing credentials (Google, Twitter, LinkedIn, Facebook etc) for authorisation is easiest.

Not entirely irrelevant.

Best wishes

TT  

[bimlas]

TiddlyTweeter,

I think for wider apps the widely used system to login using existing credentials (Google, Twitter, LinkedIn, Facebook etc) for authorisation is easiest.

I don't see the whole process yet, but it might be feasible that when logging in with GitHub, the saver would create a Personal Access Token (valid for a day, for example), so the user would only see is that they need to login as GitHub user.

https://stackoverflow.com/questions/41879286/get-a-token-by-github-api

GitLab offers login by Google, GitHub, Twittwer accounts, and continous integration is much easier to use (you don't need Travis-CI because there is a builtin GitLab-CI), so that platform might be more ideal . The user only registers once with one of his accounts (so no need to memorize a new password) and the saver takes care of the rest.

I still have to think about it, but I think it's a viable way.

https://gitlab.com/users/sign_in

[Arlen Beiler]

WordPress (that's an interesting but very feasible suggestion), Google, and Github all support multi-user editing natively, so each user would have their own login credentials. 

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/3f3dd877-878f-4ad3-9c4c-4b8bc752567c%40googlegroups.com.

[bimlas]

Arlen,

WordPress (that's an interesting but very feasible suggestion), Google, and Github all support multi-user editing natively, so each user would have their own login credentials. 

True, but we want to access the API with the saver, so we need a Personal Access Token. Or is it possible to use the API with native user credentials? 

[Arlen Beiler]

All three of those sites have an OAuth flow setup for that. Basically you get redirected to the login page and then the login page returns a code back to the client page. Wordpress might just involve using the browser session, though, I’m not sure. 

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/9d41a1f9-5d7f-4384-8385-2689c29d0cfc%40googlegroups.com.

[TonyM]

All,

WordPress permits OAuth and the standard login process logs the browser in so I believe login once and at least that browser will have access until logout, or time out.

Open ID allows you to login with a WordPress, WordPress.com account, Google, Facebook etc... It is WordPRess that has the database credentials and the API's access it on your behalf, although you could write methods to access the database directly but the database credentials will have to be out there.

WordPress has a sophisticated authentication model including different access levels and more but most importantly self serve accounts, reset password features, email invitations  possibly even 2FA options. There are Rest API's but I do not understand them. 

The advantage of leveraging the WordPress model is its pervasive, mostly free, hosted all over, extensible, responding to threats, and much much more.

I think in time tiddlywiki plugins may be built to provide what services wordpress can already provide with mature technology. 

Custom posts (=pages) types is the way to go because ease of integration with tiddlywiki becomes possible, like editing a tiddler in WordPress as well and from a tiddlywiki. A whole range of integrations with wordpress then become possible to enable on TiddlyWiki including email sending, social sharing, linking to data feeds, rss, and much more.

Regards

Tony

On Saturday, December 7, 2019 at 12:58:58 AM UTC+11, Arlen Beiler wrote:

All three of those sites have an OAuth flow setup for that. Basically you get redirected to the login page and then the login page returns a code back to the client page. Wordpress might just involve using the browser session, though, I’m not sure. 

On Fri, Dec 6, 2019 at 07:58 bimlas <bimba...@gmail.com> wrote:

Arlen,

WordPress (that's an interesting but very feasible suggestion), Google, and Github all support multi-user editing natively, so each user would have their own login credentials. 

True, but we want to access the API with the saver, so we need a Personal Access Token. Or is it possible to use the API with native user credentials? 

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddl...@googlegroups.com.

[Mohamed Amin]

Hello All,

As I'm currently in the research phase in the same direction (which is very interested to me), please check my two cents https://gohugo.io/ and its Github Repo,

Best Regards

Mohamed Amin

On Friday, November 29, 2019 at 1:59:43 AM UTC+2, Arlen Beiler wrote:

I have a radical proposal which would take data folders to the next level. What if instead of the file system adapter we would write a new adapter to use a database. We could use PouchDB, but I would vote for something much more widespread like SQLite. We could also write it in a generic way that makes it easy to use with the regular SQL databases. It would be easy to build but I would want to make it robust enough to use in a wide array of platforms. 

I have already worked with the tiddler loading code enough to be certain it is self contained and can easily be made asynchronous to accommodate this feature. Or the preload tiddlers feature can be used, but I think it’s better to separate out the loaders as Jeremy has mentioned. 

Any thoughts on this or things I should keep in mind as I brainstorm? 

Arlen