Announcing the "innerwiki" plugin

[Jeremy Ruston]

Software documentation is better with screenshots. However, the problem is that screenshots go out of date the moment a typo is fixed or a drop shadow is deepened. So I’ve been interested for some time in the idea of a tool that could automate the generation of TiddlyWiki screenshots from a set of tiddlers.

This could be done by creating a separate wiki for each screenshot, loading it up with the tiddlers required for the screenshot. The cool thing about the way that TiddlyWiki maintains the entire user interface state in the tiddler store is that we can restore any specific state of the user interface just by arranging the right tiddlers. So, we could create a wiki that shows a tiddler in the process of being edited, or one that shows the dropdown in the “filter” tab of the advanced search tiddler.

Then we’d need a mechanism for creating a PNG from the wiki under Node.js so that we can automate it. That part turns out to be quite straightforward: Google’s Puppeteer project allows a headless installation of Chrome to be controlled from a Node.js application. Thus the Node.js application can spin up a virtual page in Puppeteer, load it with HTML and then there’s an API call to take a screenshot. All of this is documented in numerous blog posts and official docs. For example:

https://buddy.works/guides/how-take-screenshots-with-puppeteer-headless-chrome

But, I wanted to do a bit better: setting up a wiki is time consuming and having one for each screemshot would mean a lot of jumping around between wikis. And I didn’t like the idea of having to set up an entire wiki with just a couple of tiddlers in it.

The solution I’ve come up with is a new plugin called “innerwiki” that enables wikis to be constructed on the fly and either embedded in a tiddler in the browser, or rendered as a PNG under Node.js. One can type a complete description of a wiki in just a few lines, editing a single tiddler. Here’s a simple example:

<$innerwiki width="1200" height="400" style="width:100%;" filename="screenshot-1">

<$data title="HelloThere" text="This tiddler is inside a wiki"/>

<$data title="$:/DefaultTiddlers" text="HelloThere"/>

</$innerwiki>

The innerwiki widget sets up a blank wiki of a specified virtual width and height, and then injects the tiddlers described by the data widgets embedded within it. In this example, we inject a “HelloThere” tiddler and then we set up $:/DefaultTiddlers so that the new tiddler is displayed at startup. Here’s how it renders:

Because it’s an iframe, the innerwiki is completely interactive (however everytime it is refreshed it will revert to its original content).

One quite satisfying-but-useless thing is that it is entirely possible to embed an innerwiki within an innerwiki:

Under Node.js these innerwikis can be turned into PNG screenshots with the screenshot command:

tiddlywiki mywiki --screenshot [tag[screenshot]]

You can try out the innerwiki plugin in the prerelease:

https://tiddlywiki.com/prerelease/plugins/tiddlywiki/innerwiki/

Any comments or questions are welcome.

Best wishes

Jeremy

[Brian Theado]

Jeremy,

Wow, this is great! Nice work!

When I wrote the demo at https://btheado.github.io/tw-ftlist/#ftlist%20demo, I was able to show step-by-step results of example user interactions. I did this by cloning a new example tiddler from the previous one and building on to it. There was one case where the global state of something which I wanted to demo in a later step "bled" onto an earlier step.

With the innerwiki plugin I could bypass that issue. Each step could be in its own iframe with each example having their own state. I will consider changing my examples to use this plugin and see if it works out better.

The 3 ways you provide of injecting tiddlers into the innerwiki looks very well thought out. Nicely done!

--
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/06943C70-B665-4B40-99C7-C4D6A74F0573%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Diego Mesa]

Wow!

Conceptually, I could see this as a "sandbox" mechanism where I could inject all of my tiddlers into the innerwiki and then try risky plugins/modifications/etc. How can we inject the outer wiki user contents into the inner wiki?

Thanks!

Diego

[Brian Theado]

Diego,

On Sun, Jan 27, 2019 at 6:23 PM Diego Mesa <dieg...@gmail.com> wrote:

Conceptually, I could see this as a "sandbox" mechanism where I could inject all of my tiddlers into the innerwiki and then try risky plugins/modifications/etc. How can we inject the outer wiki user contents into the inner wiki?

You can use the $tiddler and $filter attributes in the $data widget to grab whatever content you want from the outer wiki. See https://tiddlywiki.com/prerelease/plugins/tiddlywiki/innerwiki/#%24%3A%2Fplugins%2Ftiddlywiki%2Finnerwiki%2Fdocs. 

Brian

[Diego Mesa]

Hey Brian,

Sorry, Im probably missing something simple. I was just wanting something like this to work:

<$innerwiki width="1200" height="400" style="width:100%;" filename="screenshot-1">

<$data filter="[all[tiddlers]]"/>

</$innerwiki>

So on my wiki, I would inject all of my tiddlers into the innerwiki

[Brian Theado]

Diego,

Ok, I think I misunderstood. So you want to duplicate your wiki into the sandbox and manually make your dangerous changes there? You would have to be very careful to prevent the tiddler containing the innerwiki from being refreshed...otherwise you risk losing your changes. Though the saving mechanism inside the innerwiki works just fine to download a backup copy of that innerwiki. For extra peace of mind, you could even enable the savetrails plugin inside the innerwiki to save them as you change them.

I was thinking of a different approach for the dangerous modifications. Modify your tiddler in the outerwiki but in a way such that it won't be active with whatever you fear messing up. For example, use a different title for the tiddler or even leave it as draft. And in your innerwiki you inject it so that it is used (rename it or turn it from being a draft). You might be safer this way with whatever saving mechanism you have setup for your outerwiki.

Since the default innerwiki template probably loads all the system tiddlers, maybe this filter is what you need to get all the user tiddlers: 

<$data filter="[!is[system]]"/>

But I just tested it on the demo site after creating some new tiddlers and it doesn't seem to work. I'm not seeing any of the new tiddlers in the innerwiki.

Brian

--

You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/b73f0fc1-2584-4160-81a0-cf5a2c6c65bc%40googlegroups.com.

[Brian Theado]

Oops, I just forgot the dollar sign. Looks like you also forgot the dollar sign in your email. It should be:

<$data $filter="[!is[system]]"/>

That seems to work for me.

On Sun, Jan 27, 2019 at 7:38 PM Brian Theado <brian....@gmail.com> wrote:

[...] 

<$data filter="[!is[system]]"/>

But I just tested it on the demo site after creating some new tiddlers and it doesn't seem to work. I'm not seeing any of the new tiddlers in the innerwiki.

[...] 

[TonyM]

Jeremy,

While you are focused on this idea.

I can see using an innerwiki to generate or import test data that could be tested against the outer wiki with no permanent impact. No need to make wiki copies, and quick retesting after outer wiki is modified.

On a related matter to innerwikis

Would it be possible for us to inject tiddlywiki data into an iframe around a html javascript page and get some results back?

Such html javascript could be designed to do things we cant do in tiddlywiki like tw5 dom insensitive javascript tools and functions, or even php auth, database queries and more. If these could appear to be in tiddlywiki but are not we could speed the inclusion of a lot of other open source solutions - perhaps discord, image conversion tools, svg design tools and more.

I am possibly taking this too far, but what if these "seperate" pages are actualy served from tiddly server, tiddlers from the outer wiki, like the single tiddlers view can be served? Like innerwikis, these would be "inner pages". This would help us "script kiddies" and provide rapid intergrations perhaps even openid auth.

Regards
Tony

[Jeremy Ruston]

Hi Tony

> I can see using an innerwiki to generate or import test data that could be tested against the outer wiki with no permanent impact. No need to make wiki copies, and quick retesting after outer wiki is modified.

Yup, that’s an interesting application. I think there’s everything needed for experimentation.

> Would it be possible for us to inject tiddlywiki data into an iframe around a html javascript page and get some results back?
>
> Such html javascript could be designed to do things we cant do in tiddlywiki like tw5 dom insensitive javascript tools and functions, or even php auth, database queries and more. If these could appear to be in tiddlywiki but are not we could speed the inclusion of a lot of other open source solutions - perhaps discord, image conversion tools, svg design tools and more.

Well, there’s certainly mileage in embedding external services via iframes, but I don’t think it’s really related to the innerwiki idea because one is no longer injecting TiddlyWiki’s own source code into the generated iframe. Using JS to talk to an embedded iframe is a common technique, and we already use where it brings a benefit (ie the plugin library). I don’t think your other examples like database queries would make much sense offloaded to an iframe. I suppose one could treat iframes as primitive service workers.

> I am possibly taking this too far, but what if these "seperate" pages are actualy served from tiddly server, tiddlers from the outer wiki, like the single tiddlers view can be served? Like innerwikis, these would be "inner pages". This would help us "script kiddies" and provide rapid intergrations perhaps even openid auth.

You’re asking if the wiki in the iframe could be served from somewhere else, but still have tiddlers injected into it: that could be done via the window.postMessage() I mentioned on another thread.

Best wishes

Jeremy


>
> Regards
> Tony

>
> --
> You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
> To post to this group, send email to tiddly...@googlegroups.com.
> Visit this group at https://groups.google.com/group/tiddlywikidev.

> To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/4693096d-8bb7-4fcc-9bad-b235d81bc25e%40googlegroups.com.

[Jeremy Ruston]

I’ve enhanced the innerwiki plugin to add the capability to overlay SVG primitives over the iframe/screenshot. For example:

<$innerwiki width="1200" height="400" style="width:100%;" filename="screenshot-2.png">
	<$data title="HelloThere" text="This tiddler is inside a wiki"/>
	<$data title="$:/DefaultTiddlers" text="HelloThere"/>
	<circle cx="600" cy="50" r="40" stroke="black" stroke-width="2" fill="green" />
</$innerwiki>

One can also use macros to encapsulate fragments of SVG:

\define big-arrow(x,y,colour:"#ff0000",border:"#000000")
	<g transform="translate($x$,$y$)">
		<path d="m-81.43106,34.99315l40.25737,-49.78116l40.25737,49.78116l-20.12869,0l0,50.02069l-40.25737,0l0,-50.02069l-20.12869,0l0,0z" fill="$colour$" stroke="$border$" stroke-dasharray="null" stroke-linecap="null" stroke-linejoin="null" stroke-width="5" transform="rotate(49.3775 -41.1737 35.1129)"/>
	</g>
\end

And then:

<$innerwiki width="1200" height="400" style="width:100%;" filename="screenshot-2.png">
	<$data title="HelloThere" text="This tiddler is inside a wiki"/>
	<$data title="$:/DefaultTiddlers" text="HelloThere"/>
	<circle cx="600" cy="50" r="40" stroke="black" stroke-width="2" fill="green" />
	<<big-arrow 600 50>>
</$innerwiki>

Best wishes

Jeremy.

[TonyM]

Jeremy,

Thanks for responding. I mean something quite different here; 

> I am possibly taking this too far, but what if these "seperate" pages are actualy served from tiddly server, tiddlers from the outer wiki, like the single tiddlers view can be served? Like innerwikis, these would be "inner pages". This would help us "script kiddies" and provide rapid intergrations perhaps even openid auth.

You’re asking if the wiki in the iframe could be served from somewhere else, but still have tiddlers injected into it: that could be done via the window.postMessage() I mentioned on another thread.

 

I am suggesting what If I have a plain text tiddler, that contains a HTML page including head and body, javascript, including libraries etc... The kind of thing I may find available in a javascript library for special features or a nice landing page layout (It is not used for anything in the parent). It is merely stored in the parent wiki. Lets us call it the HTML-Tool ,Then we could do one of two things;

1. Present the HTML-Tool at an address such as https:/hostname.com/tiddlername similar to the single Tiddler view, but simply deliver it, as it is, to the browser, as if it were a standalone html+ file.
   
2. Generate an Inner (perhaps not innerwiki but innerhtml or innerpage) that is an iframe to the aforementioned HTML-Tool without any Tiddlywiki just rendered as in the first case.

In both circumstances it would be nice if some rudimentary string could be passed to and from such HTML-Tool, perhaps that looks like a json tiddler in the parent wiki, and json file in the HTML-Tool?

It is in someways taking the single file concept further allowing tiddlywiki to host other content in its server, a bit like the 5.1.18 feature Adds support for serving static file attachments over HTTP)

I could see myself hosting in my TiddlyWiki a html page that lets people register as a member in my wordpress member site. No need to manage a separate file, it lives in my wiki, but is served by the tiddlywiki server.

Regards

Tony

[Jeremy Ruston]

Hi Tony

I don't think these ideas are related to the innerwiki plugin so much as the underlying iframe primitive. Generally, there has been extensive exploration of using iframes as a means of integration in the browser; and iframes are widely used for advertisements and for embedded widgets, and hence the techniques you're interested in exploring are pretty widely documented/understood.

Generally speaking, though, integration via iframes is inferior to integration via the server. If you've got a server available, then things like handing off to a PHP login script in a separate window are pretty trivial without needing iframes. Iframes really shine when one doesn't have the option of a server.

Anyhow, via the experimental single tiddler view, the server already has the ability to host self-contained HTML apps as single tiddlers. And ordinary wikitext can freely create iframes and inject content into them, so I think you can experiment along the lines you describe.

Best wishes

Jeremy.

I am suggesting what If I have a plain text tiddler, that contains a HTML page including head and body, javascript, including libraries etc... The kind of thing I may find available in a javascript library for special features or a nice landing page layout (It is not used for anything in the parent). It is merely stored in the parent wiki. Lets us call it the HTML-Tool ,Then we could do one of two things;

1. Present the HTML-Tool at an address such as https:/hostname.com/tiddlername similar to the single Tiddler view, but simply deliver it, as it is, to the browser, as if it were a standalone html+ file.
   
2. Generate an Inner (perhaps not innerwiki but innerhtml or innerpage) that is an iframe to the aforementioned HTML-Tool without any Tiddlywiki just rendered as in the first case.

In both circumstances it would be nice if some rudimentary string could be passed to and from such HTML-Tool, perhaps that looks like a json tiddler in the parent wiki, and json file in the HTML-Tool?

It is in someways taking the single file concept further allowing tiddlywiki to host other content in its server, a bit like the 5.1.18 feature Adds support for serving static file attachments over HTTP)

I could see myself hosting in my TiddlyWiki a html page that lets people register as a member in my wordpress member site. No need to manage a separate file, it lives in my wiki, but is served by the tiddlywiki server.

Regards

Tony

--

You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/723001cf-e95e-4ecb-a853-04fa3f96db45%40googlegroups.com.

[TonyM]

Thanks Jeremy,

That is exciting, but could you point me to how I can do this, or even sufficent info for self study? I would like to use and document this since it is not suggested in relation to tiddlywiki and could provide some solutions to some of our lasting problems and expand tiddlywiki use and audience.

Thanks
Tony

[Jeremy Ruston]

Hi Tony

> That is exciting, but could you point me to how I can do this, or even sufficent info for self study? I would like to use and document this since it is not suggested in relation to tiddlywiki and could provide some solutions to some of our lasting problems and expand tiddlywiki use and audience.

Some of the topics you’ve raised will be hard to explore without getting into JavaScript.

To get you started, here’s how to make an iframe from an existing tiddler:

<iframe src={{{ [[test.html]get[text]encodeuricomponent[]addprefix[data:text/html;charset=utf-8,]] }}} width=100% height=400/>

Where test.html contains:

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>An HTML page</title>
</head>
<body>
<h1>An HTML page</h1>
<p>This is an HTML page.</p>
</body>
</html>

Best wishes

Jeremy


>
> Thanks

> Tony
>
> --
> You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
> To post to this group, send email to tiddly...@googlegroups.com.
> Visit this group at https://groups.google.com/group/tiddlywikidev.

> To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/38040930-21f1-48bc-96c3-f141bd182ab5%40googlegroups.com.

[TonyM]

Jeremy,

That is so easy. Thanks.

I will see if I can achieve marvellous things and share back.

Regards

Tony

[TonyM]

Jeremy,

I am working through the possibilities using the InnerWiki plugin, and as I suspected, I am discovering there is going to be a lot of "off script" uses. 

Unbelievably once again extending the possibilities in TiddlyWiki by orders of magnitude.

I will share these back in time with anyone who is interested, and can raise them here or in a new thread.

A Suggestion?

I do however note there are two widgets for this "solution" 

<$innerwiki> widget

<$data> widget

• Could I be so bold as to point out since the data widget is quite specific to the workings of Innerwiki it is quite a generic name.
  
• Not only may it cause confusion with processing data tiddlers but it is using a widget name that may very well have a future application.
  
• Would it be too late to rename this to the wikidata widget or some other name before this goes into production?
  

Speculation

One use I see at some point for the data widget would be the construction of a data widget designed to act on items in a datatiddler such as iterate, add, change and delete and lookup.

Such speculation is only that, however the idea of a data widget would simplify handling datasets, and imported database information.

This would open up datatiddler use to more than the enthusiast.

Best wishes

Tony

[Jeremy Ruston]

Hi Tony

It’s great that the innerwiki plugin is of interest; it was very satisfying to make.

• Could I be so bold as to point out since the data widget is quite specific to the workings of Innerwiki it is quite a generic name.
  

The data widget is in fact completely generic, and could be used with other widgets if we had a use case. It gives us a way to declare a tiddler-like blob of data that could be used anywhere we need a flexible way to generate data.I’m thinking of moving it into the core.

One obvious application for the core is to be able to use it to inject additional tiddlers into a saved wiki, just like the <$innerwiki> widget allows additional tiddlers to be injected into the inner wiki. That way, thinking of your earlier thread, one could furnish users with different download buttons that added different configuration tiddlers to the file that is output.

Interestingly, the <$anchor> widget that is also used by the <$innerwiki> widget is actually an alias for the <$data> widget.

• Not only may it cause confusion with processing data tiddlers but it is using a widget name that may very well have a future application.
  
• Would it be too late to rename this to the wikidata widget or some other name before this goes into production?
  

Just to be clear, we can of course change anything in the prerelease until the moment it is released.

Speculation

One use I see at some point for the data widget would be the construction of a data widget designed to act on items in a datatiddler such as iterate, add, change and delete and lookup.

I think we can do most of that already. The <$data> widget is declarative and so it only helps with operations like that as a way to package their parameters.

Best wishes

Jeremy

-- 
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/569572ed-32f3-481d-b0ca-2798c8df9f21%40googlegroups.com.

[@TiddlyTweeter]

Could this be used to test mobile phone layouts reliably?

[TonyM]

Jeremy,

Could you expand on the following please.

I think we can do most of that already. The <$data> widget is declarative and so it only helps with operations like that as a way to package their parameters.

Since your data widget defines blobs perhaps it could be a blob widget? I am still wrapping my head around this.

Regards
Tony

[TonyM]

FYI:

The developer update in FireFox 65 for developers may impact the proposed local storage and the Innerwiki plugins for good or bad

https://developer.mozilla.org/en-US/docs/Web/API/Storage_Access_API

Regards

Tony