Bob and TiddlyServer

[Diego Mesa]

Hello all,

Its that time of the year again, where a friend of mine wants to know if the community could help clarify the differences between two major players: Bob and TiddlyServer, as well as Arlens newer creation/tests?

Ill be sure to pass on your comments/observations to my friend!

Best,

Diego

[Arlen Beiler]

I’m not sure. Jed would probably know a lot better. 

The main feature of TiddlyServer is organization. It lets you reference all your various TiddlyWiki folders and universes across your drives and lets you work with all of them from the browser without messing with different ports and savers and all kinds of other hassles. It just works. The setup is still rough but I’m working on improving that.

But other than adding a web socket layer, it tries to be completely transparent, using existing TiddlyWiki technology under the hood. In fact, most of the inspiration for it came from Jeremy because of the way he designed the boot.js file and other server related components. The paradigm he used is what made it possible. In a way, it felt like I built a section of TiddlyWiki that had been planned for but never built. Maybe not, but Jeremy definitely laid a solid foundation.

I tried to stay within the parameters of the TiddlyWiki core, and I am confident that I succeeded. Other than adding a couple features to the core, including the server post start hook and the boot completion callback, I pretty much didn’t make any changes to TiddlyWiki, and I definitely didn’t need to fork it. 

A few features that are hopefully coming soon include a browser extension which will be able to easily manage and setup the server (plus a few bonuses like saving for file urls), and a plugin that automatically serves the core from a CDN instead of your server to speed up page load dramatically for online applications (it will include a fallback to TiddlyServer in case the internet is offline, and downloads inside the wiki should still be unaffected). 

Bob has a lot more features than that, I believe, but I’ll let others explain that part because I’m not familiar with it at the moment. 

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/7ac3f0a3-da45-4ba5-820f-f09cc3154d29%40googlegroups.com.

[Jeremy Ruston]

> In fact, most of the inspiration for it came from Jeremy because of the way he designed the boot.js file and other server related components. The paradigm he used is what made it possible. In a way, it felt like I built a section of TiddlyWiki that had been planned for but never built. Maybe not, but Jeremy definitely laid a solid foundation.

Yes indeed, it's really fun when it happens like that.

As some of you may have noticed, I might have a bias towards writing code, rather than writing docs or blog posts. It's fair to say that there's a part of me that thinks that writing code is a definitively superior means of expression because it's unambiguous and concrete. Seeing code like yours that I could have written myself is the most satisfying possible confirmation of that shaky theorising. It's also very good for the health of the community to have as many developers as possible with that level of understanding of the core code.

Best wishes

Jeremy

[Jed Carty]

I made Bob because I can't spell, can't remember command line arguments and can never remember where I put my wikis.

And I kept opening wikis in different tabs and forgetting about it so I would overwrite things I had saved, so it fixes that problem.

So I set up Bob so you can click on the executable and it starts, it doesn't require any configuration and you have access to almost all of its features, the features that do require configuration are more advanced and I am slowly making them work out of the box as well.

I have been working to make it use as little memory as I can, so it has a single tiddlywiki process and only loads tiddlers from each wiki instead of creating a new $tw object or making a new process for it.

I have my home server running on a raspberry pi along with gitea and some file servers and before I had a script to start up a new node process for each wiki which was making the pi run out of ram pretty quickly, now I have around 70 wikis and it has no trouble.

I have also tried to be very conscious of security problems because it is meant to be very simple to use, so people using it may not have the background to know what to look for when it comes to security.

I try to have notes anywhere there is a setting that could cause a security problem and explain some of the trade-offs between security and convenience, this is for things like having the wikis available on the local network and enabling the file server.

After that I have just been putting things into it when I think of them and need a break from other projects.

You can do all of this from inside the wiki with a UI, it can use lot of work but it is easier for me than having to editing json files and remember how everything goes together.

After that it just got away from me and I keep adding things to it.

- Create wikis

  - From editions

  - From duplicating other wikis

  - From single file wikis

  - From a filtered list of tiddlers from one or more other wikis

- Delete wikis

- Move/Rename wikis

- Export single file wikis with all of the server components removed

   - You can optionally give a filter to limit what tiddlers are included in the exported html file

- You can set up Bob as a plugin server

- You can change the plugins that a wiki uses with a checklist inside the wiki itself, this updates the tiddlywiki.info file as well

  - you can do the same with themes

- It has an experimental feature that fetches plugins directly from GitHub or gitlab and saves them so they can be included in any of your wikis

- messages from the server appear in the wiki as tiddlywikis alerts

   - This is configurable

- The file server component can serve from multiple locations

   - For security only files is the listed locations are available, and you can't use something like '../../../../.ssh/rsa_id'

- When importing media files Bob can save media files in a files folder and create _canonical_uri tiddlers instead of importing the media directly into a wiki

   - I have a wiki that has thousands of pictures of the construction on my house in it and there isn't any trouble

   - You can scan folders for media and Bob will automatically create the _canonical_uri tiddlers for you, and add the folder to the list of places the file server can reach, so I didn't have to manually make one tiddler for each image

- You can shutdown the wiki server from inside a wiki

- It also includes a saver server that, along with the BobSaver plugin, lets you just open a tiddlywiki file and save it like you could before we had the problems with browser restrictions.

- There is a local chat that lets people using the same Bob server to have a real-time chat setup inside the wiki. I am not sure how useful this is, but it is there.

There is also some federation stuff, like syncing wikis between different Bob servers, but that is in the early stages and I doubt anyone but me would use it in its current state.

[Dave]

You can shutdown the wiki server from inside a wiki

 

I never really thought about this.  I'm using the BobExe now, so is there a better way of shutting down the server using a bash script other than "pkill Bob"?  I know that killing a process like that is sometimes less than graceful.

[Mohammad]

Hi Jed

 Many thanks for this itemize but complete description.

 I have not used Bob within last year, but your list of feature encourage me to try the new version.

--Mohammad

[Mark S.]

They're both interesting, and do similar things, but in slightly different ways.

Don't quote me to your friend on any of this, because I might have it wrong.

One of the great, though recent, aspects of Bob is that for saving single files all you have to do is

1) Install the Bob Saver in the Wiki
2) Download BobWin
3) Run Bob

Now just double click or launch your wiki, and it works and saves AS LONG AS Bob is running. If all you want to do is work with single files, then Bob makes it pretty easy.

TS has a more complicated set up. You have to install node.js. You have to copy an example settings file and then edit it in a text editor.

TS also saves single file wikis, but you need to set up a simple settings file first. Then TS serves the single file as if it were a file on the web. Bob is, I think, slightly faster when saving single file wikis. Probably because the file doesn't have to travel back over the network.

A feature of TS is that you can set a backup directory, and each time it saves a backup to that directory.

For  data folder wikis, both Bob and TS can serve up images, which is important if you have external files. Bob limits how many file directories you can have (one per Wiki, I think).  So if your legacy wiki has files under "files" and under "images" then you may have to reorganize your file structure and change your image links. TS allows you to serve up as many paths as you want, relative to the data folder *.info file.

If you don't have legacy files to support, then I suppose they're about equal in terms of static file serving.

TS, or at least an older version of TS, can run on Termux on Android 5+.  I haven't tried running Bob on Termux, so maybe someone can comment on their experiences in that regard.  

Bob has it's own editor for configuration files, and (in theory -- I haven't tried it) will let you run batch scripts that can be launched from your wiki. The editor is a little bit fragile, I'd say. For instance, while writing this I attempted to add a single data folder to the structure. Not only did it fail, it wiped out my existing wikis setting. The same thing happened again when I enabled the file server. If you had a lot of intricate settings, you might find this frustrating.  

In TS you have to use an editor for configuration, but you can set up your own virtual tree that will help navigate if you have a lot of wikis spread out over an area. The navigation tree can lead you to both single file and data folder wikis. Bob's configuration sort of expects that all your wikis live mostly in one directory, though you can add individual paths to specific data folders. You can't make a path to single-file wikis, because they are not being served by Bob Server.

So there you have it. They're very close in functionality. TS is a little better if you want one dashboard for both single file and data folders. Bob is a little better if you want to use single-file wikis exactly as files on file system, and have them save fast. But you have to remember to install the Bob Saver plugin in your single-file Wiki first. Bob simplifies setup and configuration for you with a single-file installation and a configuration interface that eliminates the need for an editor. But the interface resets your configuration if anything goes wrong. TS is more flexible in terms of static file serving and offers file backup.

Which one do I prefer? It's usually whichever I'm using at the moment ;-)

[TiddlyTweeter]

Ciao Mark S.

Footnote on ...

Bob has it's own editor for configuration files, and (in theory -- I haven't tried it) will let you run batch scripts that can be launched from your wiki. 

It works well. There are some issues on script process. But these are largely to do with the O/S, not Bob per se. 

For instance, you might want to launch a program in the specific wiki's directory but the Windows program won't play ball without fiddling about.

But overall the scripting access to the O/S is superb in Bob, and is invaluable. 

Jed also took care that the way you invoke the scripts isn't too dangerous.

My 2 cents

TT

Mark S. wrote:

They're both interesting, and do similar things, but in slightly different ways.

<snip> 

[Mark S.]

It does sound like a neat feature. Do you have an example of how you are using it?

[TiddlyTweeter]

Ciao Mark S.

I can't explain all the details because it involves other programs you likely would not know.

The point is that in passing to O/S you can use one's "other" skills that then feed their result back ....

Here is an example that Bob does for me reliably ...

1 - Bob SCRIPT launches a Windows batch file, passing it parameters that invokes a program to "whack a website"; 

2 - on completion passes the whacked file to a regex engine that strips the file and converts it into TW (file) fragments (node) can read;

3 - copies the output of 2 into the wiki's directory;

4 - alerts you when it succeeded so you can refresh the wiki;

5 - your wiki now has the processed content.

For some things I like to do Bob most. It is very efficient in combining TW intelligence with O/S mediated grunt-work.

My 2 cents

TT

[Jed Carty]

Bob allows as many folders as you want for serving files, it is just that each wiki has its own dedicated folder as well. Media that you import into the wiki gets saved in a wiki specific folder.

And please make bug reports when you have trouble, Bob shouldn't ever reset its settings, it has been very stable for me for a while so I need bug reports or I can't fix things.

[Mark S.]

As many folders as you want RELATIVE to a particular wiki (to support legacy TW where you might have wiki/files, wiki/images, wiki/audio, etc.) ?

I imagine you haven't tried entering incorrect field/object setups into the editor, because you know what the editor wants. I wasn't planning on crashing it, so I didn't take notes for a bug report. But when it crashed, it reset completely.

[Diego Mesa]

Not to make a strong suggestion on others time, but would the community find it useful/instructive if both Jed and Arlen tried to install/use/play around with the others tool and report back here? If both are willing and able, of course.

[Jed Carty]

That is a pretty strong assumption to make.

What exactly do you mean by 'reset'? Where were you editing settings?

The file server mimics the behaviour of the core node server and then adds the option to add more folders. Setting up a file server to mimic the behaviour of an html file on a file:// uri with many different paths puts limits on wiki names that I didn't want to deal with.

[TonyM]

Diego,

I use both TiddlyServer and Bob. Many have documented the key differences below. I would add the following;

TiddlyServer

• Once configured TiddlyServer provides a way to map a folder then just include your wikis under that folder be they single file or node installs.
• This allows me to have wikis that create wikis and I save them in a folder and they are then available by simply browsing
• See the image

Bob (I use the EXE form)

• To me the key value of Bob is the multi-access/multi-user function. Bob is the best place for me to have critical wikis that opening in a different tab or browser will not result in accidental overwrites. 
• The recent Bob File Saver plugin that allows single file wikis to be saved after installing bob.exe is potentially the the most effective way to solve the tiddlywiki save issues for new users.

Using both

• I have added additional IP Addresses to my computer on the HOME LAN, I set both TiddlyServer and Bob to use these rather than localhost or the default IP Address this stops competition for addresses or port numbers.
• I make both available on my Home network, when elsewhere the custom IP Addresses are invalid.
  
• At any point I can move the dedicated IP address to another device on my lan to be a host.
• The exception is the Bob File Saver Plugin which is on local host and a special port.

Regards

Tony

 

[Mark S.]

It's an assumption I make, because I broke it twice in 10 minutes without trying. So I assume it's been only tested by someone who knows exactly what to type to get the results they want. There's no reason to get upset about it. I'm just reporting what happened.

By reset, I mean all the settings I had made, like wikiPath, were gone and I had to start over. I was editing in the JSON look-alike editor you have. Server -> Manual settings. That's the only way to set the wikipath, right? And add additional wiki locations? If there's some better way to do it, that would be interesting.

Thanks!

[Jed Carty]

That is useful to hear.

The settings weren't resetting themselves, they weren't saved. you have to click the save settings button or they don't get saved back to the server when you are using the manual settings. There isn't much spelled out there because there isn't much reason to use it and I was hoping that reading the documentation would point people in the right direction.

To add wikis you use the first tab under the bob settings in the control panel called 'Manage Wikis', then there is a tab called 'create wikis', then select 'a node wiki' and there is an interface to add them.

So, what documentation could I add that would have brought you to the 'manage wikis' tab instead of the manual settings tab to add a wiki?

[Mark S.]

I'm pretty sure the the settings were saved. I had already set up a wiki dir and generated a list, and  probably closed down/re-opened Bob more than once, because I was making comparisons with TS. I can only say, try using the manual settings and insert something the system wouldn't know how to process. I think what I did the first time is to try to emulate the __path structure of the previous wikis.

Anyway, the "create wikis" tabs is frightening to anyone adding a wiki -- the LAST thing I want to do is to accidentally OVERWRITE an existing wiki.

Thanks!

[Mark S.]

Well, I wasn't able to recreate those crashes, but I was able to reliably crash using the "a node wiki" feature.

I used

bible2

C:\temp\Bible2

in the respective fields and then "Add wiki".

Then "update list". It crashed

I believe this was the error

TypeError: Cannot read property 'useFileLogging' of undefined
    at Object.$tw.Bob.logger.error ($:/plugins/OokTech/Bob/BobLogger.js:152:29)
    at WebSocket.$tw.Bob.handleMessage ($:/plugins/OokTech/Bob/NodeWebSocketsSet
up.js:152:22)
    at WebSocket.emit (events.js:182:13)
    at Receiver.receiverOnMessage ($:/plugins/OokTech/Bob/External/WS/lib/websoc
ket.js:720:20)
    at Receiver.emit (events.js:182:13)
    at Receiver.dataMessage ($:/plugins/OokTech/Bob/External/WS/lib/receiver.js:
415:14)
    at Receiver.getData ($:/plugins/OokTech/Bob/External/WS/lib/receiver.js:347:
17)
    at Receiver.startLoop ($:/plugins/OokTech/Bob/External/WS/lib/receiver.js:13
4:22)
    at Receiver._write ($:/plugins/OokTech/Bob/External/WS/lib/receiver.js:70:10
)
    at doWrite (_stream_writable.js:410:12)

After this happened twice, I was able to add the Wiki successfully. BUT, now my original set of wikis was gone.

From my perspective, I'm getting errors regularly  AND, it's reset my prior list of wikis 3 times. Maybe it's a Windows thing? Like window pathing?

After the last crash, I added back in my wikispath via the manual setting. Now I've got the list back including the bible2 entry.

{{adding tw-commander while thinking ...}}

Ok, here's a user error I might be making. If I hit "update list" before it's finished doing the new route, could that be crashing? In that case, it might be helpful to have a message warning the user that a process is happening. Currently the user has no feedback -- it looks like they did something, but nothing happened.

Thanks!

On Wednesday, December 11, 2019 at 3:57:18 PM UTC-8, Jed Carty wrote:

[Jed Carty]

I have just spent the last 20 minutes putting random things into different places and you have found a way to crash bob that I was unaware of, but the way to get that error is not to put something into the adding a node wiki part, you have to edit the manual settings and remove the 'logging' entry, save the settings and then do something that would make the logger send a message.

Bob crashes there, but when you start Bob if an entry like that is missing that entry it gets replaced with the defaults, so the problem is fixed when you restart Bob.

By default if you put in a wiki location that doesn't exist that will be removed the next time the list of wikis is updated. I don't know if that is what you are describing because I am still not certain of the details of what you are describing as 'resetting the settings'.

As far as using the interface goes, if you are willing to make changes to the manual settings without knowing the details of it, but you are not willing to use an interface that is documented and that ensures things are in the correct format, I don't think I can provide support.

I will add a note somewhere saying that you should not expect support when making changes that do not follow the documentation. If I am missing pieces of required documentation please let me know.

[Mark S.]

On Thursday, December 12, 2019 at 1:11:13 AM UTC-8, Jed Carty wrote:

By default if you put in a wiki location that doesn't exist that will be removed the next time the list of wikis is updated. I don't know if that is what you are describing because I am still not certain of the details of what you are describing as 'resetting the settings'.

No. I have not TRIED to break Bob. I did not give it the address of a non-existent Wiki.

By reset, as I explained, I mean it went back to the same set up as when you first install Bob. The wikipath setting is gone. The list of Wikis is gone. You have to start completely over. Can I be any clearer?

That's why it's so troubling. If it just reset to the last stable configuration, that would be understandable.

 

As far as using the interface goes, if you are willing to make changes to the manual settings without knowing the details of it, but you are not willing to use an interface that is documented and that ensures things are in the correct format, I don't think I can provide support.

Your hostility doesn't really help. "Not willing" ?  Where do I change the wikipath without the manual interface?

As I mentioned, it crashed when I used the nearly HIDDEN feature to add a wiki. One thing is that Bob seems to  sometimes be doing things in the background, but there's no indication to the user that they need to wait. Maybe some type of hour glass thing would help.