TiddlyWeb Tutorial

[rich shumaker]

Are there any videos I can watch or introductory examples of TiddlyWeb usage?

I am trying to understand how I can create things to interact with the world.

Thanks for your help.

Rich Shumaker

[Tobias Beer]

Perhaps start here...

http://tiddlyweb.tiddlyspace.com

Best wishes, Tobias.

[rich shumaker]

Thanks Tobias.

I will check that out and see if I have any questions.

I am trying to wrap my brain around it and I know it is fairly straight forward thing.

Just not getting it quite as fast as I thought I would.

Rich Shumaker 

[chris...@gmail.com]

On Fri, 23 Jan 2015, rich shumaker wrote:

> Are there any videos I can watch or introductory examples of TiddlyWeb
> usage?

Pmario has done some videos related to installing tiddlyweb in
particular situations, but they may be a bit out of date:

https://www.youtube.com/results?search_query=tiddlyweb

There's a video here of me talking about something I created with
TiddlyWeb:

http://cdent.tumblr.com/post/591977969/manifestopheles

Throughout the rest of that tumblr are lots and lots of stories about
what's been created with TiddlyWeb and some of the how and why.

Similarly my now idle tiddlyspace space is full of random thoughts on
making TiddlyWeb do stuff:

http://cdent.tiddlyspace.com/

(follow recent, all tiddlers, or do a search to find more stuff).

All of that stuff will require some digging and searching but may be
useful for getting a sense of things.

However I think it is probably important to say that the struggle
you're having with trying to answer the question "what can I do with
it" is both common in the TiddlyWeb universe and rather hard to
answer. I think a lot of this is because TiddlyWeb is a lot like the
entire web. You can do pretty much anything with it, with sufficient
effort.

What TiddlyWeb layers on top of the web itself is a model or grammar
for interacting with web entities. That model is hugely flavored by
the tiddler concept that was inherited from TiddlyWiki, seasoned
with the concepts of bags, recipes and filters and passed through my
brain which has a particular orientation for how things are supposed
to work (which the tumblr and tiddlyspace above ought to shine some
light on).

--
Chris Dent http://burningchrome.com/
[...]

[rich shumaker]

Thanks Chris I will be checking this all out.

Mario was doing what I am trying now a few years ago so I need to talk to him again.

Rich Shumaker

[rich shumaker]

So now I am starting to understand better.

Please correct where wrong and add to this 'history'.

TiddlyWiki Classic came out.

TiddlyWeb was created to help TiddlyWiki Classic work collaboratively.

TiddlySpace was created with TiddlyWeb to allow people to use TiddlyWiki(deploy faster and easier)?

TANK was created as the next step beyond TiddlyWeb, where Tiddlers become the primary unit of measure.

How hard is TANK to implement VS TiddlyWeb for TiddlyWiki 5?

My goal is an 'easy' way to take a RasPi and put TiddlyWiki's(5 not C) on it to allow Kids(and Adult Kids) the ability to use TiddlyWiki 5.

Next step would be to create collaborative TiddlyWiki's to allow Kids(and again Adult Kids) to work together.

Mario has done a bunch of work on this a few years ago for TWC.

I want this to be TiddlyWiki 5 and I know that adds a level of 'difficulty'(complexity) to the mix.

I can see everyone needing TiddlyWiki for 'stuff'.

The second goal to the RasPi version would be to use TiddlyWiki 5 as a front end to GPIO and Camera and the Pi stuff.

Thanks as always everyone for the help.

Rich Shumaker

[chris...@gmail.com]

On Wed, 28 Jan 2015, rich shumaker wrote:

> So now I am starting to understand better.
> Please correct where wrong and add to this 'history'.

This is not incorrect. That is the things you say are true, but
there are lots of things too.

> TiddlyWiki Classic came out.
> TiddlyWeb was created to help TiddlyWiki Classic work collaboratively.

Yes, that was certainly the earliest motivator but as it was
developed and grew it took on the idea of "make tiddlers be useful
in lots of contexts".

> TiddlySpace was created with TiddlyWeb to allow people to use
> TiddlyWiki(deploy faster and easier)?

It was decided that TiddlyWeb alone was insufficient to make it easy
for people to use TiddlyWiki collaboratively. TiddlySpace was
created to provide a) a hosted solution (so people didn't have to do
any installing themselves, b) a UI with tools to enable
collaborative "stuff" (whatever that might be, the indecision on
what that stuff was sometimes harmed the progress of TiddlySpace).

> TANK was created as the next step beyond TiddlyWeb, where Tiddlers become
> the primary unit of measure.

Sort of. To be frank, the years I have spent working on TiddlySpace were
not entirely satisfying for me (its design and orientation were not what
I would have done) and then to be cut loose from it as my primary source
of income while at the same time development of it apparently ceased
left me a bit grumpy. So I created tank as an effort to sort of
purge the demons and be more focused on my own goals. Unfortunately tank
is as unfinished as TiddlySpace. I think it is more interesting but
getting at that power requires either a lot of work to figure things
out or already having things figured (ie being me).

I had hoped to supplement my income with subscriptions to tank but I
ran out of time before reaching what I would consider a viable
project so went back to regular employment and now that's consuming
enough time.

This, I'm sure, is a common tune in the semi-open-source world.

> How hard is TANK to implement VS TiddlyWeb for TiddlyWiki 5?

For someone with experience with unix servers and python development
tank is fair to middlin' tricky to make go. There's a thread[1] in
this group between Bill Seitz and me about making it happen but I
believe he eventually gave up in favor of experimenting with
something a little less trying.

In other words I think it is safe to say "hard".

I have done some work on making a dockerfile for tank[2] but that
too is incomplete. I suspect with a bit more work it would be
possible to "just"[3] build a container and be on your way.

Of course once you get tank going there's still the matter of
getting tw5 working as excellent as possible with tank. There's
still some distance to go on that, on the tw5 side. Some of those
problems we've already discussed on the threads about making
tiddlyweb work with a file-based tw5.

/me needs a clone

> My goal is an 'easy' way to take a RasPi and put TiddlyWiki's(5 not C) on
> it to allow Kids(and Adult Kids) the ability to use TiddlyWiki 5.
> Next step would be to create collaborative TiddlyWiki's to allow Kids(and
> again Adult Kids) to work together.

One of things I've not understood in your various statements about
your goals is where you plan for the pi itself to be and where you
plan for the users of the pi to be.

For example are you hoping to take a pi into a classroom (or similar
shared environment) with a TiddlyWiki/Web on it and have the
participants use it from a keyboard and mouse attached to the pi or
from a different computer that attaches to the pi over the network?

The answer to that question changes the options quite a bit.

There are a lot of people in the TiddlyWiki google group who keep
talking around this general goal (I know you've been interacting,
for example, with Tobias). That group is probably the best place for
discussion on the general architecture you're pursuing whereas here
is probably best for any specific questions you have on the
technology and techniques of tiddlyweb and tank.

Good luck.

[1] https://groups.google.com/d/topic/tiddlyweb/PERYbEtTO3g/discussion
[2] https://github.com/cdent/tankdock
[3] Never trust anyone who says "just" or "should".

[rich shumaker]

Thanks again Chris for all your help.

For example are you hoping to take a pi into a classroom (or similar 
shared environment) with a TiddlyWiki/Web on it and have the 
participants use it from a keyboard and mouse attached to the pi or 
from a different computer that attaches to the pi over the network? 

The answer to that question changes the options quite a bit. 

The goal is many folded unfortunately with the RasPi.

Goal One get TiddlyWiki running in a server type environment on RasPi.

This would allow for rapid deployment for groups in any environment(school, office, hobbyshop, hackerspace)

Current Status of this Goal

This is completed via BidiX store.php, node.js, and for TiddlyWeb I am 90+%.

I am trying to leverage all the extra tools TiddlyWeb has but I am a NOOB so that is difficult for me.

I am also a Node.js noob so that has been slow to get added features there as well.

Since TiddlyWeb is based on Python I see the RasPi community being able to hook into TiddlyWeb much faster than Node.js.

Goal Two

Use TiddlyWiki on a RasPi from a server on a different RasPi.

This allows teaching server side and client side usage.

Also it is a proof of concept to show that TW5 can be used on a RasPi.

Current Status of this Goal

This is in progress and I have crashed the newest official RasPi browser many times.

I need to load Firefox and test that on the RasPi.

Store.php seems slower in response than Node.js does.

I have not gotten TiddlyWeb fully working and had a chance to test yet.

Goal Three

Using the Server Side abilities to allow TiddlyWiki to control the RasPi.

Making TiddlyWiki a front end for RasPi stuff, i.e. Camera, GPIO, data sets.

This can not be done directly from TW5 HTML based on security protocols in the browser from the research and questions I have asked and had answered.

This is where TiddlyWeb and Node.js come into play.

As stated TiddlyWeb being based on Python has the best chance of being able to do this as Python is widely used and has lots of code built already.

Current Status of this Goal

Ground 0 I have gotten no where with this yet.

I apologize that my attempts at different things have been so scatter shot.

Day to day I work on a PC with HTML TiddlyWiki 5 and I have had a lot of success with RasPi's as servers so far for my uses.

I would highly recommend a RasPi BidiX store.php BerryBoot Server for anyone wanting to use many computers and devices around a location to access HTML TW5.

This solution took less than 20 minutes to fully implement once I figured it all out.

It also requires very small SD cards and has fast boots.

My most recent discussion with people is how to create a 'testbed' or 'all in one' TW Server side solution.

So it would be a single RasPi image that has Bidix store.php(and stuff to work with it), Node.js, TiddlyWeb and I was thinking TANK but I understand the difficulty with that and it is probably beyond reach.

I want anyone that wants to experiment with TW5 on a RasPi to be able to load and go easily and quickly.

The advantage the RasPi has in it's ability as a server as the browser is a bit under powered to fully take advantage of TW5.

Thank you very much again for all the help and for helping me to understand what is where and whatnot.

I unfortunately also understand what it is like to be working on something for someone for years and then shown the door.

Rich Shumaker

[Tobias Beer]

Hi Rich,

 

Store.php seems slower in response than Node.js does.

That should really depend on how you use both.

If you use node.js to create the desired wiki on the fly then loadong via node.js should be slower than loading from a folder to which you save TiddlyWiki using store.php, since store.php is entirely uninvolved in the loading process.

When it comes to saving things, store.php must surely be slower as we're not writing changes to individual tiddler files but the entire wiki-html, every time. However, store.php shouldn't be slower than saving an entire TiddlyWiki to whichever remote location. It mostly depends on the network speeds, I'd think.

 

Using the Server Side abilities to allow TiddlyWiki to control the RasPi. 

This is where TiddlyWeb and Node.js come into play.

Or whichever interfaces / API the RasPi exposes and how, e.g. via server or not.

Perhaps a TiddlyWeb / Node.js server only needs to reroute a given request rather than implement any serverside scripting.

I have no idea.

Not sure if the RasPi can, would, could or should run two servers,

one for its control interfaces and one for whichever TiddlyWiki serverside.

A first step could simlpy be to iframe this stuff from within TiddlyWiki...

http://www.instructables.com/id/Web-Control-of-Raspberry-Pi-GPIO

This solution took less than 20 minutes to fully implement once I figured it all out.

Do you have a concise write-up of that process on your site?

1. step a
2. step b
3. ...

The advantage the RasPi has in it's ability as a server as the browser is a bit under powered to fully take advantage of TW5.

Quite possibly, but servers may at some point also be asked to do quite some heavy lifting.

Sure, rendering an x-MB sized Wiki is demanding for an itsy wee Pi.

Best wishes, Tobias.

[rich shumaker]

Hey Tobias,

Answers Below.  You Rock!  Thanks for reminding me about the instructions for RasPi as I had them locally but had not updated my server for a bit.

 

 

Store.php seems slower in response than Node.js does.

That should really depend on how you use both.

If you use node.js to create the desired wiki on the fly then loadong via node.js should be slower than loading from a folder to which you save TiddlyWiki using store.php, since store.php is entirely uninvolved in the loading process.

When it comes to saving things, store.php must surely be slower as we're not writing changes to individual tiddler files but the entire wiki-html, every time. However, store.php shouldn't be slower than saving an entire TiddlyWiki to whichever remote location. It mostly depends on the network speeds, I'd think.

This comment about Store.php and Node.js "speeds" was in regards to using the official Raspberry Pi Browser with TW5.

I have done no load testing of the RasPi server for any of the versions I have loaded.

I have a RasPi with Berryboot server and store.php, a RasPi with Node.js, and a RasPi with TiddlyWeb(still need to work on that a lot).

The last RasPi has the standard Rasbian install from 12/24/2014 which is the newest version I could get when I started this.

The Viewer Pi with the new OS layout and Newest HTML 5 browser works fairly well with TW5 but it does crash.

I was able to load the TW Hangouts from the web and watch a video for a bit but then it crashed.

I was surprised it got that far honestly as previous versions of the Pi Browser could not make it that far.

I have not loaded Firefox yet to test it.

When you work on a TiddlyWiki that is on the node.js server it takes longer to load, a lot longer.

But once it is loaded it responds quicker to keystrokes.

The store.php version loaded fairly fast but it was sloooow when I would type in, very old skool laggy.

I felt like I was using a Commodore 64.

It would be interesting to see if the TW core could be converted to GPU code as that would make it Zip Zip around by comparison.

 

Using the Server Side abilities to allow TiddlyWiki to control the RasPi. 

This is where TiddlyWeb and Node.js come into play.

Or whichever interfaces / API the RasPi exposes and how, e.g. via server or not.

Perhaps a TiddlyWeb / Node.js server only needs to reroute a given request rather than implement any serverside scripting.

I have no idea.

Not sure if the RasPi can, would, could or should run two servers,

one for its control interfaces and one for whichever TiddlyWiki serverside.

A first step could simlpy be to iframe this stuff from within TiddlyWiki...

http://www.instructables.com/id/Web-Control-of-Raspberry-Pi-GPIO

I will have to explore using a node.js tiddler in an iframe as that would be very cool indeed.

TiddlyWeb is built on Python and the Raspberry Pi is all about Python so there is a serious synergy there.

My hope was to put "hooks" into existing Python code using TiddlyWeb stuff(don't know the specific term if it is a recipe bag or other)

Still need to do a bunch of testing and what not, mostly what not.

 

This solution took less than 20 minutes to fully implement once I figured it all out.

Do you have a concise write-up of that process on your site?

1. step a
2. step b
3. ...

I had this locally but had not updated the server recently so it was not there till you brought this up.

So it is there now on my RasPi Knowledge Site, please grab and put it in your repository if you want too as that is how I find most of the stuff I have done is on your site, #heheh.

Also I have started to update that site with more data on what I have done with RasPi's which is not as much as I was hoping honestly.

Still LOVE the Looper.  The RasPi makes a great video looper with and without audio.

 

The advantage the RasPi has in it's ability as a server as the browser is a bit under powered to fully take advantage of TW5.

Quite possibly, but servers may at some point also be asked to do quite some heavy lifting.

Sure, rendering an x-MB sized Wiki is demanding for an itsy wee Pi.

I agree and would like to see/do some load testing for the different flavors of server side implementation.

I think store.php would be the best if you stagger the loads and the saves.

Not sure though I might find out that node.js or TiddlyWeb has some special caching ability or something that makes it faster over time.

Don't really know but that is why I set all this up was to test and try, poke prod and learn.

Of course I have a bunch of projects so I need to devote time to each and I wanted to get some other people involved with the RasPi TW5 experiments.

Thanks as always for all your help Tobias as you have been instrumental in me getting from point A to point B in TW5 land as quickly as I have.

Oh and I was trying to get the UK born TW on the UK born RasPi, as a guy from the US I thought that made a lot of sense, #heheh.

Rich Shumaker

[PMario]

On Saturday, January 31, 2015 at 1:49:20 AM UTC+1, rich shumaker wrote:

This comment about Store.php and Node.js "speeds" was in regards to using the official Raspberry Pi Browser with TW5.

You could use the midory browser. I did a fast test with an old 256MB RasPi. The TW UI is relatively slow according to response time but it didn't crash.
There are some problems with some CSS rulse around text input boxes. But imo this can be fixed.
 

I have done no load testing of the RasPi server for any of the versions I have loaded.

I have a RasPi with Berryboot server and store.php, a RasPi with Node.js, and a RasPi with TiddlyWeb(still need to work on that a lot).

I don't exactly know, why you need Berryboot. It's a software to start different images on the pi. I'm using Raspbian since the beginning and didn't have any problems with it. Raspbian is a debian derivative, so any help in the web that is debian, ubuntu and the like will work well. So mixing Operating systems IMO will just slow you down, since every OS is a bit different. eg: startup procedure. ....

 

The last RasPi has the standard Rasbian install from 12/24/2014 which is the newest version I could get when I started this.

sudo apt-get update
sudo apt-get upgrade

will give you the latest and greatest stable versions of everything.

or you can use raspi-config in the command line. It's a simple UI to do system low level settings.

eg: If you only need the pi as a server and you don't start the graphical UI you can assign more memory to the "backends". 
Not automatically starting the graphical UI may speed up nodejs and tiddlyweb servers.

If you need the UI you can log in and use startx or if you log in wit putty you can start the lxpanel with X-forwarding to your windows machine.
 

The Viewer Pi with the new OS layout and Newest HTML 5 browser works fairly well with TW5 but it does crash.

I was able to load the TW Hangouts from the web and watch a video for a bit but then it crashed.

 

try the midori browser, which imo is pre-installed at the pi.
 

When you work on a TiddlyWiki that is on the node.js server it takes longer to load, a lot longer.

I'm not sure about this. I'll make some tests.
 

But once it is loaded it responds quicker to keystrokes.

I don't believe this, since there is no difference, if you load index.html as a file or if nodejs serves it.
Saving does make a difference, since php-save saves the whole file and nodes-save saves a single tiddler.

But the UI can't be slower or faster, since it's the same code, that needs to be executed by the browser javascript engine.
 

The store.php version loaded fairly fast but it was sloooow when I would type in, very old skool laggy.

hmmm. As I wrote above. That's strange. May be some different plugins? .. try midori
 

It would be interesting to see if the TW core could be converted to GPU code as that would make it Zip Zip around by comparison.

Why do you think, this would make TW faster?

The TW core creates html that is rendered by the browser. So if the browser uses the GPU to render HTML it is faster, as if the CPU needs to do the rendering.

I think there is a misunderstanding of the term "rendering".

 - TW rendering means: The TW core converts wikitext to html, that can be "drawn by the browser".
 - Browser rendering means: Take the hmtl code and draw it pixel by pixel to the screen.

TW rendering speed is only affected by the power of the CPU and the javascript engine implemented by the browser.

Perhaps a TiddlyWeb / Node.js server only needs to reroute a given request rather than implement any serverside scripting.

I have no idea.

To control the raspi with TiddlyWiki, we would need a REST API. The mechanism is the same as the TiddlyWeb or nodejs API. ... It needs someone that does the code. .. that's it.
 

I will have to explore using a node.js tiddler in an iframe as that would be very cool indeed.

Adding an iframe into a TW, is like haveing 2 or even more heavy pages open at the same time.
Since TW is already slow, imo that isn't the best idea. But it will be fast, since no TW API is needed. So worth exploring.
 

TiddlyWeb is built on Python and the Raspberry Pi is all about Python so there is a serious synergy there.

My hope was to put "hooks" into existing Python code using TiddlyWeb stuff(don't know the specific term if it is a recipe bag or other)

Still need to do a bunch of testing and what not, mostly what not.

As I wrote above. Someone would need to add some RaspberrPi REST API to the TiddlyWeb API. IMO as a TiddlyWeb plugin.
 

I agree and would like to see/do some load testing for the different flavors of server side implementation.

I think store.php would be the best if you stagger the loads and the saves.

Should be, since there is no load except if you load / save the file.
 

Not sure though I might find out that node.js or TiddlyWeb has some special caching ability or something that makes it faster over time.

 

There is no caching involved in the default setup but  TiddlyWeb shouldn't be a big problem even for 10+ concurrent users and the simple text store.
If I load a TWclassic from a TiddlyWeb server CPU usage goes up to about 22% and then returns to idle state. eg: htop (the program to show these values) uses 4% max, the rest is idle.

Starting the X environment needs about 50MByte more memory.
I did assign 16MByte for the GPU, which is the minimal setting (raspi-config) since I don't use the UI at the pi. So no GPU used.

Without X I have about 80 MByte ram free
With X I have 30 MByte ram free.

Note: I have the old 256MByte RasPi. So RAM is valuable. ... No slow SWAP memory used atm :)
 

Don't really know but that is why I set all this up was to test and try, poke prod and learn.

Of course I have a bunch of projects so I need to devote time to each and I wanted to get some other people involved with the RasPi TW5 experiments.

I did a lot of tinkering with the RaspberryPi and TiddlyWeb about a year ago. see: http://tweb-at-home.tiddlyspace.com/ and https://github.com/pmario/tweb-config (for TWc atm)

There is a good blog post about tweb-config: http://www.rs-online.com/designspark/electronics/blog/building-a-personal-microcontent-server-with-raspberry-pi

I'm rewriting tweb-config at the moment to use TW5. ... But the system / concept will be completely different, with one more year of experience and the development going on in the DevOps space.

Hope that helps.
have fun!
mario

[PMario]

On Saturday, January 31, 2015 at 2:36:17 PM UTC+1, PMario wrote:

I will have to explore using a node.js tiddler in an iframe as that would be very cool indeed.

Adding an iframe into a TW, is like haveing 2 or even more heavy pages open at the same time.
Since TW is already slow, imo that isn't the best idea. But it will be fast, since no TW API is needed. So worth exploring.

uups bad wording:  "But it will be fast" ... fast in terms or time needed to test it, not execution or UI speed.
-m

[rich shumaker]

This comment about Store.php and Node.js "speeds" was in regards to using the official Raspberry Pi Browser with TW5.

You could use the midory browser. I did a fast test with an old 256MB RasPi. The TW UI is relatively slow according to response time but it didn't crash.
There are some problems with some CSS rulse around text input boxes. But imo this can be fixed.

I pushed TW5 in RasPi as I was viewing videos and things like that and I too am on a 256MB Pi.

When I pushed it the browser broke every time.

Since I do a lot of Video work I needed to see if the Pi could handle it and it could for a little while, which is better than previous browsers on the PI.

 

 

I have done no load testing of the RasPi server for any of the versions I have loaded.

I have a RasPi with Berryboot server and store.php, a RasPi with Node.js, and a RasPi with TiddlyWeb(still need to work on that a lot).

I don't exactly know, why you need Berryboot. It's a software to start different images on the pi. I'm using Raspbian since the beginning and didn't have any problems with it. Raspbian is a debian derivative, so any help in the web that is debian, ubuntu and the like will work well. So mixing Operating systems IMO will just slow you down, since every OS is a bit different. eg: startup procedure. ....

Why I use BerryBoot?

It started as I wanted a multiOS boot SD card and BerryBoot did this I think a year before NOOBs came out.

I like BerryBoot as it is rock solid easy to set up, I think even easier and more friendly than NOOBs, which is ridiculously easy to set up, #heheh.

BerryBoot server has all I need, probably not as secure as it could be, is only I think 20mb, is set up in less than 20 minutes and works.

Raspbian takes hours to set up and I don't need all that stuff, just an easy to use headless server that I can store TW.html files using store.php

So I guess my answer is "Time".  I spend less with BerryBoot and get great results.

The hardest part is you can't go headless until you change the password, then you can go headless from there.

 

 

The last RasPi has the standard Rasbian install from 12/24/2014 which is the newest version I could get when I started this.

sudo apt-get update
sudo apt-get upgrade

will give you the latest and greatest stable versions of everything.

Yes that is what I do after I load the latest image I just didn't say that, Thank you for letting people know as this is an important step.

Also for everyone reading PLEASE update your Pi as many bugs get worked out and some of my earliest work on the Pi was difficult and things were not completely stable.

That was of course years ago and the Pi is ridiculously stable now.

Also on the Model A and B the power supply needs to be VERY STABLE and you CAN NOT plug and unplug USB stuff on the fly.

On the A+ and B+ you can use horrible power and plug and unplug usb all day.

Oh and the A+ is super cute.  Don't own one of those but they are very "kawaii".

Add a Unicorn hat and you have a party, sorry to digress.

or you can use raspi-config in the command line. It's a simple UI to do system low level settings.

eg: If you only need the pi as a server and you don't start the graphical UI you can assign more memory to the "backends". 
Not automatically starting the graphical UI may speed up nodejs and tiddlyweb servers.

If you need the UI you can log in and use startx or if you log in wit putty you can start the lxpanel with X-forwarding to your windows machine.
 

The Viewer Pi with the new OS layout and Newest HTML 5 browser works fairly well with TW5 but it does crash.

I was able to load the TW Hangouts from the web and watch a video for a bit but then it crashed.

 

try the midori browser, which imo is pre-installed at the pi.

The NEW Pi browser is GPU enabled and I understand what you are saying that you gain nothing by putting the Pi code on the GPU.

I was thinking that if Node.js was on the GPU it would allow for faster interactions as the GPU is about 10X the CPU.

Also it would free the CPU for other tasks.  Your point about putting TW on GPU makes perfect sense though as that won't do crap for the rendering to browser stuff.

 

 

When you work on a TiddlyWiki that is on the node.js server it takes longer to load, a lot longer.

I'm not sure about this. I'll make some tests. 

 

Remember this is using the Pi not another browser on another PC or Mac or linux box.

Pi to Pi is what this referred too.

 

But once it is loaded it responds quicker to keystrokes.

I don't believe this, since there is no difference, if you load index.html as a file or if nodejs serves it.
Saving does make a difference, since php-save saves the whole file and nodes-save saves a single tiddler.

But the UI can't be slower or faster, since it's the same code, that needs to be executed by the browser javascript engine.
 

The store.php version loaded fairly fast but it was sloooow when I would type in, very old skool laggy.

hmmm. As I wrote above. That's strange. May be some different plugins? .. try midori
 

Again this is LOADING ON A PI not loading on another PC or Mac or Linux box.

I was talking about loading on a Pi from the Node.js RasPi server and loading from a Pi with store.php

Store if I remember took 25 seconds to load but using it was Laaaaggggyyyy when you type.

Node.js took about 2 minutes to load if not longer but was t y p e perfect when I was typing.

Also the Node.js saved in real time and the save on the HTML TW took a little bit, again about I think 10 15 seconds.

 

It would be interesting to see if the TW core could be converted to GPU code as that would make it Zip Zip around by comparison.

Why do you think, this would make TW faster?

The TW core creates html that is rendered by the browser. So if the browser uses the GPU to render HTML it is faster, as if the CPU needs to do the rendering.

I think there is a misunderstanding of the term "rendering".

 - TW rendering means: The TW core converts wikitext to html, that can be "drawn by the browser".
 - Browser rendering means: Take the hmtl code and draw it pixel by pixel to the screen.

TW rendering speed is only affected by the power of the CPU and the javascript engine implemented by the browser.

You are correct I am wrong the browser that is now GPU enabled is what you need to put on the GPU.

BUT

I was also speaking of headless servers using Pi if you could move Node.js to the GPU then you would get exponential increases.

To serve content that is.

 

Perhaps a TiddlyWeb / Node.js server only needs to reroute a given request rather than implement any serverside scripting.

I have no idea.

To control the raspi with TiddlyWiki, we would need a REST API. The mechanism is the same as the TiddlyWeb or nodejs API. ... It needs someone that does the code. .. that's it.
 

I will have to explore using a node.js tiddler in an iframe as that would be very cool indeed.

Thanks Mario for explaining this as I knew something needed to be built on the server side but had no idea how to build.

Let me know your results for iframe as I won't have time for a few days to test.

Once I test I will report back

Adding an iframe into a TW, is like haveing 2 or even more heavy pages open at the same time.
Since TW is already slow, imo that isn't the best idea. But it will be fast, since no TW API is needed. So worth exploring.
 

TiddlyWeb is built on Python and the Raspberry Pi is all about Python so there is a serious synergy there.

My hope was to put "hooks" into existing Python code using TiddlyWeb stuff(don't know the specific term if it is a recipe bag or other)

Still need to do a bunch of testing and what not, mostly what not.

As I wrote above. Someone would need to add some RaspberrPi REST API to the TiddlyWeb API. IMO as a TiddlyWeb plugin.

Thanks again for explaining what is needed as I really don't know beyond, server side "stuff".

 

 

I agree and would like to see/do some load testing for the different flavors of server side implementation.

I think store.php would be the best if you stagger the loads and the saves.

Should be, since there is no load except if you load / save the file.
 

Not sure though I might find out that node.js or TiddlyWeb has some special caching ability or something that makes it faster over time.

 

There is no caching involved in the default setup but  TiddlyWeb shouldn't be a big problem even for 10+ concurrent users and the simple text store.
If I load a TWclassic from a TiddlyWeb server CPU usage goes up to about 22% and then returns to idle state. eg: htop (the program to show these values) uses 4% max, the rest is idle.

This is good to know as that is not bad for a low cost low power server to be able to handle 10+ concurrent users.

Starting the X environment needs about 50MByte more memory.
I did assign 16MByte for the GPU, which is the minimal setting (raspi-config) since I don't use the UI at the pi. So no GPU used.

Side note you can not use the new browser with less than I think 64mb of ram for GPU and 128 is recommend if I remember correctly.

Might need 128mb as they offload some of the work to the GPU.

 

Without X I have about 80 MByte ram free
With X I have 30 MByte ram free.

Note: I have the old 256MByte RasPi. So RAM is valuable. ... No slow SWAP memory used atm :)
 

All of my tests were on 256mb RasPi's so it would be interesting to try a B+ as the browser.

I will try to set that up and see if I notice a difference with llllaaagggg.

 

Don't really know but that is why I set all this up was to test and try, poke prod and learn.

Of course I have a bunch of projects so I need to devote time to each and I wanted to get some other people involved with the RasPi TW5 experiments.

I did a lot of tinkering with the RaspberryPi and TiddlyWeb about a year ago. see: http://tweb-at-home.tiddlyspace.com/ and https://github.com/pmario/tweb-config (for TWc atm)

There is a good blog post about tweb-config: http://www.rs-online.com/designspark/electronics/blog/building-a-personal-microcontent-server-with-raspberry-pi

Again, I probably would not have started this if I had known you did all this work, or I just would have grabbed your work and run with it.

Thank you very much for all the work you have done as I greatly appreciate it as it made what I did easier once I found what you did.

Still working out bugs and kinks and my idea is to build something for others that is easier than what I did.

 

I'm rewriting tweb-config at the moment to use TW5. ... But the system / concept will be completely different, with one more year of experience and the development going on in the DevOps space.

Let me know actually I will make sure I am watching Github and Github will let me know.

Thanks Mario for all the help it is greatly appreciated.

Rich Shumaker 

[PMario]

I was thinking that if Node.js was on the GPU it would allow for faster interactions as the GPU is about 10X the CPU.

GPUs are highly specialised processors for graphical calculations. So you can speed up graphics a lot, if you can write a graphic driver, that is able to use the GPU.

Nodejs is kind of a virtual machine (the javascript enginge) that can run javascript code and interact with the outside world. eg: file I/O, network, ...
The GPU doesn't have access to the file system, the network and so on. ...

Even if there are calculations that could be offloaded to the GPU, I doubt, that someone will create _and support_ a RasPi optimized nodejs version. Simply because it would use too much resources. ...

The nodejs project was forked some weeks ago. see: https://iojs.org/ So imo they are on pressure with there own problems.

have fun!
mario

[rich shumaker]

My concern about moving node.js to the GPU I think is a moot point after reading today's RasPi Blog.

I have a feeling that I will no longer have issues with using TW5 on RasPi either with the new hardware.

I also think this should increase speed of the server as well, won't know until the 'future' when I can get one as I have plenty of the old ones, #heheh.

Not sure if Node.js is multi-threaded but I am pretty sure it will be if it isn't right now.

So using a New RasPi B Ver 2 should work great with TW5 HTML and using it as a server will probably increase the total concurrent users as well to a big class of 30+(hopefully).

Now it really is a good pairing, TW5 on RasPi2 B version.

Rich Shumaker

[PMario]

On Tuesday, February 3, 2015 at 3:14:48 AM UTC+1, rich shumaker wrote:

My concern about moving node.js to the GPU I think is a moot point after reading today's RasPi Blog.

I did order one yesterday evening. I got an invoice, but no tracking number yet :)
So I hope I can run some more tests soon.

have fun!
mario

[rich shumaker]

You are ahead of me as you seem to be all the time, #heheh.

I saw some usage stuff yesterday and it was 5% CPU while running the browser.

There will be huge gains if they can reuse the GPU specialized code under the new OS over the next few months.

They accelerated a very slow car and then dropped in a much faster engine so I feel that TW5 on RasPi 2 will be great for students and teachers.

Keep me up to date on what you find out when it arrives.

Thank you very much for all your help.

Rich Shumaker

[rich shumaker]

I have done no load testing of the RasPi server for any of the versions I have loaded.

I have a RasPi with Berryboot server and store.php, a RasPi with Node.js, and a RasPi with TiddlyWeb(still need to work on that a lot).

I don't exactly know, why you need Berryboot. It's a software to start different images on the pi. I'm using Raspbian since the beginning and didn't have any problems with it. Raspbian is a debian derivative, so any help in the web that is debian, ubuntu and the like will work well. So mixing Operating systems IMO will just slow you down, since every OS is a bit different. eg: startup procedure. ....

Why I use BerryBoot?

It started as I wanted a multiOS boot SD card and BerryBoot did this I think a year before NOOBs came out.

I like BerryBoot as it is rock solid easy to set up, I think even easier and more friendly than NOOBs, which is ridiculously easy to set up, #heheh.

BerryBoot server has all I need, probably not as secure as it could be, is only I think 20mb, is set up in less than 20 minutes and works.

Raspbian takes hours to set up and I don't need all that stuff, just an easy to use headless server that I can store TW.html files using store.php

So I guess my answer is "Time".  I spend less with BerryBoot and get great results.

The hardest part is you can't go headless until you change the password, then you can go headless from there.

Mario,

I should have also said I wanted to use very low cost MicroSD memory cards and boot from a server to allow me to run several Pi's and keep the costs down. 

The more Pi's the more you can save using a 128mb BerryBoot MicroSD Card that only cost a dollar.

I might combine my two ideas and see if I can boot from Server Images for the TW5 different servers methods.

I do run into issues from time to time with BerryBoot and then I switch to dedicated Raspbian on an 8 GB or larger card.

Rich Shumaker

[PMario]

On Thursday, February 5, 2015 at 7:22:56 AM UTC+1, rich shumaker wrote:

I should have also said I wanted to use very low cost MicroSD memory cards and boot from a server to allow me to run several Pi's and keep the costs down. 

The more Pi's the more you can save using a 128mb BerryBoot MicroSD Card that only cost a dollar.

I might combine my two ideas and see if I can boot from Server Images for the TW5 different servers methods.

I do run into issues from time to time with BerryBoot and then I switch to dedicated Raspbian on an 8 GB or larger card.

see this info form the raspi webpage:

The Raspberry Pi should work with any SD-compatible cards, although there are some guidelines that should be followed:

- SD card size. For installation of NOOBS, the minimum recommended card size is 8GB. For image installations we recommend a minimum of 4GB; some distributions can run on much smaller cards, specifically OpenElec and Arch.

- SD card class. The card class determines the sustained write speed for the card; a class 4 card will be able to write at 4MB/s, whereas a class 10 should be able to attain 10 MB/s. However it should be noted that this does not mean a class 10 card will outperform a class 4 card for general usage, because often this write speed is achieved at the cost of read speed and increased seek times.

So the best option here is to get some info, or benchmarks for the overall performance of an micro sd card. ... Since the market builds its price by the "class X" it should be easy to find good cards at a cheap price since class 4 or class 6 may outperform class 10 for general use.

With my B+ Pi I use and 8gb card, with all the sw installed, there is about 3gb free for content, which imo will be enough for your usecase. ... except you intend to store a lot of images. ... and even there you can store several 1000s if you scale them down to 720p or 1080p, since most monitors can't show a 3000x4000 pixel image anyway.

You'll have to search for some benchmarks, and see what fits best.
If I find something useful, I'll post it :)

have fun!
mario

[Danielo Rodríguez]

Hello Rich,

How is this project? I'm also interested on running some backend server on raspberry pi, TANK or tiddlyweb.

[rich shumaker]

Currently Dead in the water. While it is possible I was unable to get over the technical hurdles.
I got 60% maybe more on my TiddlyWeb path with RasPi.
The method I use for TW5 on RasPi is my least viewed instructables, http://www.instructables.com/id/TiddlyWiki-5-on-Raspberry-Pi-Server-in-15-minutes/
I also got Node.js on RasPi working with TW5.
As a server RasPi's work well. For usage I found the RasPi slow using TW5, I don't have a RasPi 2 yet but they should work like PB&J together.
Mario said Firefox worked pretty well on the RasPi with TW5 but I didn't get a chance to try it.

Let me know if you have any luck.
Now that I know how to make a Raspberry Pi image if I do get this to work I will make an image and share it.

One thing that I remeber from all my research is that some code(one specific line) in TW5 could be improved.
That one line of code in TW5 that Chris has pointed out needs modification, I don't know how to modify that and I have asked anyone that can modify it to do a pull request from GitHub to do it.

I have a feeling that one line of code leads to a fundamental structural change and is a can of worms once you pull the loose string.

Rich Shumaker