IoT python library set and I had a question

[rich shumaker]

How would you implement a library set like this that uses Python for the IoT(Internet of Things) into TiddlyWeb to allow TW to use IoT stuff?

Thanks

Rich Shumaker

[PMario]

Hi Rich,

It highly depends on what you want to achieve.

 - MQTT is a machine-to-machine (M2M)/"Internet of Things" connectivity protocol [1].
 - It is based on a concept that is known as: "publish/subscribe messaging" [2] aka: PubSub

To make this work you need a client-side software and server-side software.

In your post you mentioned a client library. .. but implementing a client library is useless, if you don't say how your data flow looks like. ...
In MQTT terms the server-side software is called an information broker.  This broker is needed for publishing and subscribing to the information stream.

One possible data flow can look like this

sensor -> MQTT broker +-> MQTT client -+-http-> a) TiddlyWeb server <--http--> TW5
                ^                      |
                |                      +-http-> b) TiddlyWeb server <--http--> TW5 (no live update out of the box)
                |                                   ^
                |                                   |  
                |                                   v   
                +-----------<--MQTT protocol--> c) TW5 with TWeb and MQTT-client (live update with raw data both directions)


So it depends, on what you already have and what you need

 - If you have an existing MQTT broker,
    - add a) you can create an independent MQTT client software that connects to the broker and writes tiddlers to the TiddlyWeb server.
    - add a) there is no need that TW user is online to create new tiddlers

    - add b) TiddlyWiki doesn't need to know anything from MQTT, since it gets all the info from TWeb server
    - add b) If sensor data changes, TWeb creates a new tiddler, but since TW client doesn't know about MQTT it doesn't get live update
    - add b) live update is possible, but it needs plugins and imo increases complexity quite a bit.

 - add c) If you want TW to be part of IoT you may want to directly send messages to the broker, so other TWs can see them.
 - add c) TiddlyWeb connection can be used out of the box
 - add c) MQTT connection would need a new plugin with js libraries (there are several possibilities)

 - As an MQTT broker the mosquitto [3] project could be used.
 - For testing / low traffic usage there public test server [4] could be used.

have fun!
mario

[1] http://mqtt.org/
[2] http://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern
[3] http://mosquitto.org/
[4] http://test.mosquitto.org/

[rich shumaker]

Thanks Mario and I have read your post twice and I need to read it 2 or 3 more times as it all makes sense except I need to digest it.

I was seeing TiddlyWeb as the "collector" and "creator" of Tiddlers and when you load your TW it would load the Tiddlers that TiddlyWeb created.

So the idea in my head was that TiddlyWeb is always on and collects data in real time as it happens.

The data is stored as Tiddlers in TiddlyWeb and when you load your TiddlyWiki it looks for new content and loads all the data at that time.

So the TiddlyWiki load would load 'static' content from the dynamic TiddlyWeb content that is created.

Hopefully my explanation will explain 'how I thought' it worked, although it doesn't work that way.

As always I am greatly appreciative of all the help everyone provides.

Rich Shumaker

[PMario]

On Wednesday, February 18, 2015 at 9:33:00 PM UTC+1, rich shumaker wrote:

Thanks Mario and I have read your post twice and I need to read it 2 or 3 more times as it all makes sense except I need to digest it.

Sorry for that :)
 

I was seeing TiddlyWeb as the "collector" and "creator" of Tiddlers and when you load your TW it would load the Tiddlers that TiddlyWeb created.

TiddlyWeb is not an active "collector" or "creator". If you start the server, it basically does nothing. ... except listening to port 80 or 8080 if there is some communication. ...

eg:
If a TiddlyWiki wants to read (GET), write (PUT) or delete (DELETE) something.
So the TiddlyWiki user is the "creator" and the TW core knows how to write to TiddlyWeb.
TiddlyWeb is great data store :)

So the idea in my head was that TiddlyWeb is always on and collects data in real time as it happens.

As written above, that's not the case. TiddlyWeb provides an API to manipulate recipes, bags, tiddlers (aka HTTP API).

So the "always on" program can be an MQTT client program, which uses the library you mentioned.
This program needs to know how to talk to the MQTT broker and it needs to knows how to talk to TiddlyWeb. .. So this app can actively listen, if something happens and actively create new tiddlers.

The advantage here is: Separation of concerns [1]. So every application should do one thing well.

 - TiddlyWeb provides an interface to manipulate tiddlers, bags and recipes
 - MQTT client app listens to the broker channels and if something happens creates tiddlers.
 - MQTT broker is similar to TiddlyWeb server. It provides an interface to publish and subscribe to a live data stream.

This architecture increases testability and in normal cases creates simpler and smaller apps, which are easier to maintain.
 

The data is stored as Tiddlers in TiddlyWeb and when you load your TiddlyWiki it looks for new content and loads all the data at that time.

That's right. I did label this as a)
 

So the TiddlyWiki load would load 'static' content from the dynamic TiddlyWeb content that is created.

That's right. see a)
 

Hopefully my explanation will explain 'how I thought' it worked, although it doesn't work that way.

It does explain your thoughts. They are right. ... but as always, the devil is in the detail :)

I did some experiments with nodejs and mosquitto broker on the rpi. quick and dirty, but it seems to work well.

I'll post more details here.

have fun!
mario

[1] http://en.wikipedia.org/wiki/Separation_of_concerns

[rich shumaker]

Boat loads of awesome and thanks again Mario for the translation of what I thought to the real world.

I need to redo my TiddlyWeb on RasPi install from scratch to make sure I know how to build that and finalize it and work all the bugs and kinks out.

I am excited to see what you create with the quick and dirty solution.

Rich Shumaker