[TW5] ThirdFlow plugin for developing plugins inside the browser

[TheDiveO]

Hi,

I've now published release v1.0.0 of my Third Flow plugin, see https://github.com/TheDiveO/ThirdFlow/releases/tag/v1.0.0

This is the first public release of the Third Flow plugin. It allows you to develop your plugins directly inside your browser, yet our source tiddlers are still well laid out hierarchically inside your source code repository. There is no need for external editing and maintaining separate plugin source folders. The plugin also helps you packing your plugins so that releasing plugins and accompanying demonstration wikis should be really easy. Of course, this plugin has been developed with itself.

On the release page you'll find both a demonstration wiki with more information and the plugin included as well as the plugin itself as a single .tid for import into your own TW5s.

Best regards,
TheDiveO

[PMario]

Hi,
Without having a closer look. The icon is great :))
.. So now I'll have a closer look.
-mario

[PMario]

Great stuff!

When I call "develop" script / cmd, it assumes, that "tiddlywiki" is a command.
Which actually isn't true, If I didn't install it with npm first.

So I think your github readme should contain an Install section.

git clone https://github.com/TheDiveO/ThirdFlow.git
cd ThirdFlow
npm install tiddlywiki

Then make sure, that your PATH variable is set to find the tiddlywiki binary at: ./node_modules/.bin

For windows see: eg: http://www.computerhope.com/issues/ch000549.htm

Or you can install TW globally with:  npm install -g tiddlywiki 

have fun!
mario

[Jeremy Ruston]

Hi TheDiveO

This is a superb piece of work, well done, you must have been very busy over the last few days.

It's a beautiful example of presenting a plugin, I'm very pleased to see details like the icon, plugin information tabs, and the excellent and clear documentation. It feels like you've really embraced TW5s potential and taken it forward.

Of course, reading the finished thing it's now totally clear what your overall vision was - which I'm afraid I'd missed when presented with the individual fragments (hfsa and repo structure). At first I was misdirected by the "developing in the browser" notion, which I'd come to associate with the workflow that avoids Node.js, Git etc (there's a surprising demand for that workflow).

I'd like to link to it in the community resources tiddler on tiddlywiki.com. Is there an online copy of the index.html?

Best wishes

Jeremy.

--
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywikidev.
For more options, visit https://groups.google.com/d/optout.

--
Jeremy Ruston
mailto:jeremy...@gmail.com

[TheDiveO]

To all: thank you very much for the encouraging positive feedback. I'm glad that my idea has not taken (mostly) clear shape and is better to understand.

@PMario: I'll yet need to, erm, dive (I almost hate to use that word, but then ... oh well) into the Node.js infrastructure in order to cut off the rough edges. At least you now made me understand why Jeremy's shell scripts use tiddlywiki.js directly instead of relying that the command is present.

@Jeremy: There is no online copy yet. Unfortunately, I found a problem in the way my "--build release" configuration works and where I need some help.

There is another idea hidden inside my plugin: during release I'm disabling the hierarchicalfilesystem sync adapter and then I pack the plugin. After that, I dump the demo TIddlyWiki. Now, the demo TiddlyWiki is basically the development TW but without any plugin source tiddlers. How do I exclude the source tiddlers from being rendered?

I'm basically throwing out all those tiddlers which also have a shadow tiddler which is of same creation and modification data and type. I'm also throwing out my own ThirdFlow plugin itself -- unless it's source code is present. The rationale is that plugin developers usually don't want the ThirdFlow plugin to be released as part of their demo TWs. Yet, I need to release it as part of my demo TW.

The problem now is that I'm currently relying on wiki.getShadowSource(). Trouble is, I'm packing the plugin (tiddler) after TW has booted and all (enabled) plugins have been enabled. So TW doesn't notice that there is change in the plugin. My [is[shadowinsync]] is universal in that it should work both when I'm releasing the ThirdFlow demo TW as well as plugin users releasing their plugin demo TWs. In order to work correctly and without further configuration of the template tiddlers shadowinsync and thus TW would need accurately updated shadow information.

At first, I simply called $tw.wikireadPluginInfo() (v1.0.0 release). In this situation the release of my own demo TW works as expected. Unfortunately, I only noticed late after release that this does not properly throws out the ThirdFlow plugin when a plugin developer releases his demo TW.

So I added a call to $tw.wiki.unpackPluginTiddlers() ... which is most surely not the, erm, best thing to do. Kind of brute force ... or taking the door into the house as we like to say over here. Now, I can't correctly release my own demo TW, but users now can.

A long story cut short ... as one idea: what about an additional boot(?) method that allows us to update a specific package at least with respect to correctly current shadow and plugin information? I've seen that the boot code hides and protects the corresponding shadowTiddlers and pluginInfo objects.

Or is there another way I missed in both correctly dumping a recently packed plugin but without the source tiddlers when there are also the exactly same source tiddlers available while dumping all source tiddlers that are differing from their plugin shadows? So far, I couldn't come up with any filter expression that achieves this without the need to explicitly code in specific plugin titles...

Best regards,
-- TheDiveO

[PMario]

On Thursday, September 4, 2014 9:28:39 AM UTC+2, TheDiveO wrote:

I've now published release v1.0.0 of my Third Flow plugin,

I just saw, that you directly started with version v1.0.0 .. which is ok, if you think your plugins are stable.

TW [2] uses the Semantic Versioning [1] concept. So I think plugins should too.
I'm not sure, if you know this mechanism, so here are the links, just to be sure ;)

-mario

[1] http://semver.org/
[2] http://tiddlywiki.com/#TiddlyWiki5%20Versioning

[Jeremy Ruston]

Not strictly relevant, but this is quite funny in relation to semantic versioning:

http://sentimentalversioning.org

:)

Best wishes

Jeremy.

--

You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywikidev.
For more options, visit https://groups.google.com/d/optout.

[PMario]

On Thursday, September 4, 2014 5:54:49 PM UTC+2, TheDiveO wrote:

@PMario: I'll yet need to, erm, dive (I almost hate to use that word, but then ... oh well)....

No problem, I'm a skuba diver too, so you can dive as much as you like :))
-m

[TheDiveO]

And I thought that I tested it well ... but mistakes do happen. Thanks for the sentimental versioning. I didn't knew it before and it is a nice read. So please see my 1.0.0 in this light. Shall I add a "-no-not-really" tag next time?

But let's focus on my question please in order to fix the bug I found only late. Pondering about this problem after a good evening walk I think that I made the mistake of not fully understanding [is[shadow]]. The fix is probably simple: [is[shadow]!is[tiddler]] is what I need.

I also reread the relevant part of boot.js and I'm under the impression that it should be just fine when I call unpackPluginTiddlers() as I'm doing now after I packed the plugin in order to refresh shadowTiddlers and thus shadowInfo. I need to rely on this later in my [is[shadowinsync]] filter.

[PMario]

On Thursday, September 4, 2014 10:22:38 PM UTC+2, TheDiveO wrote:

And I thought that I tested it well ... but mistakes do happen. Thanks for the sentimental versioning. I didn't knew it before and it is a nice read. So please see my 1.0.0 in this

light. Shall I add a "-no-not-really" tag next time?

No, just start counting
eg:
 - 1.0.1 for small patches / fixes - releases
 - 1.1.0 if there is a major improvement, new functions ... - releases
 - 2.0.0 If it's all new, or it breaks 1.x

-m

[PMario]

On Thursday, September 4, 2014 7:17:27 PM UTC+2, Jeremy Ruston wrote:

Not strictly relevant, but this is quite funny in relation to semantic versioning:

http://sentimentalversioning.org

nice
Just wait when TiddlyWiki is listed there too :)
-m

[Jeremy Ruston]

Hi TheDiveO

But let's focus on my question please in order to fix the bug I found only late. Pondering about this problem after a good evening walk I think that I made the mistake of not fully understanding [is[shadow]]. The fix is probably simple: [is[shadow]!is[tiddler]] is what I need.

I also reread the relevant part of boot.js and I'm under the impression that it should be just fine when I call unpackPluginTiddlers() as I'm doing now after I packed the plugin in order to refresh shadowTiddlers and thus shadowInfo. I need to rely on this later in my [is[shadowinsync]] filter.

Calling unpackPluginTiddlers() will often be necessary when fiddling with plugins - see $:/core/modules/pluginswitcher.js for example.

Best wishes

Jeremy

 

Am Donnerstag, 4. September 2014 09:28:39 UTC+2 schrieb TheDiveO:

Hi,

I've now published release v1.0.0 of my Third Flow plugin, see https://github.com/TheDiveO/ThirdFlow/releases/tag/v1.0.0

This is the first public release of the Third Flow plugin. It allows you to develop your plugins directly inside your browser, yet our source tiddlers are still well laid out hierarchically inside your source code repository. There is no need for external editing and maintaining separate plugin source folders. The plugin also helps you packing your plugins so that releasing plugins and accompanying demonstration wikis should be really easy. Of course, this plugin has been developed with itself.

On the release page you'll find both a demonstration wiki with more information and the plugin included as well as the plugin itself as a single .tid for import into your own TW5s.

Best regards,
TheDiveO

--
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywikidev.
For more options, visit https://groups.google.com/d/optout.

[TheDiveO]

I'm preparing the static project github page for the ThirdFlow plugin. Whenever I directly load my demo TW5 from http://thediveo.github.io/ThirdFlow/output/thirdflow.html I get a yellow TW5 warning popup saying "syncer-browser - XMLHttpRequest error code: 404".

I've just included
    "plugins": [
        "tiddlywiki/codemirror",
        "tiddlywiki/highlight"
    ],

Any idea how I can avoid this warning?

Best regards,
TheDiveO

[TheDiveO]

strange ... the problem seems to be gone ... and I don't know why.

[TheDiveO]

Jeremy, you may want to use this link: http://thediveo.github.io/TW5FontAwesome/output/fontawesome.html

[TheDiveO]

I've added two illustrations to the project web page (http://thediveo.github.io/ThirdFlow/): one for the development phase and one for the release phase. These illustrations show how user, web browser with TW5 and plugin, Node.js with TW5 and plugin and the tiddlers all work in concert.

[Danielo Rodríguez]

Hello TheDive0

I'm trying to use your flow to update one of my old plugins and I'm running into problems. I followed the instructions you described. I created the folder structure, I created the plugin info tiddler, but when I try to open the plugins tab I get this error:

Uncaught TypeError: Cannot read property '$:/plugins/danielo/tagSearch/tagSearch/icon' of undefined

Hope you can help me.

Thanks in advance.

[Danielo Rodríguez]

I'm using TW 5.1.1

I noticed that your example does not work neither so maybe a versioning problem?

The release script is generating an empty HTML.

One question:What if I want to include a tiddler that does not follow the naming convention? In mu plugin I can define the filter, but I can't figure how to do the same in your plugin.

Thanks in advance.

[TheDiveO]

Internally I've run the ThirdFlow with 5.1.0 but noticed no issues so far. But let's try to trace down what the cause may be. But without seeing your TW it is difficult.

My plugin should be independent of any naming conventions: whether you use the path plugin and one or two more levels shouldn't matter. The only place where the plugin needs the concrete plugin title is in the release build config for the packageplugin command.

It is important that you do not(!) modify the saving template. If you do you'll remove your plugin! The saving template does two things: 1. It skips all source tiddlers that are identically in a plugin. 2. It removes my plugin when no longer needed ... that is, when releasing a plugin (second generation).

Thus, never change the save template to reference your own plugin!

[TheDiveO]

One thing I notice now is the deprecated regexp, I'll update this.

[TheDiveO]

Danielo,

I've tested my ThirdFlow and TW5FontAwesome plugins on TW5.1.1 and fixed a deprecated filter expression in the packageplugin command. Other than that I noticed no issues. Releasing a second generation plugin, such as TW5FontAwesome, works at least on my dev system. I'm using the recent TW5 github version. I've updated all my repositories to show the most recent versions for ThirdFlow and TW5FontAwesome. Downloads are updated too.

Could you please be so kind as to give the updated ThirdFlow a new test drive?

Enjoy,
-- TheDiveO

[Danielo Rodríguez]

Hello TheDiveO

Thank you for your help. I'm testing your plugin version 1.0.0 with TW5.1.2 version and I'm getting the same error.

I'm just executing your example as it is.

Regards

[Danielo Rodríguez]

Hello again

I just noticed the last version is not 1.0. Maybe you want to update the first link.

Anyway, the error still the same.

To let you know what I do exactly: I unzip your plugin example. I run develop.cmd I navigate to localhost:8080, I open the plugins tab: Got the error.

Nothing changed.

[TheDiveO]

Which link (on which web page) are you exactly referring to?

The ZIP link on my project page http://thediveo.github.io/ThirdFlow/ refers to the most recent plugin release 1.0.3. I've re-checked the ZIP repository file and it contains the recent 1.0.3 release. I unpacked the ZIP file, started develop.cmd and navigated to localhost:8080. I don't get any error when loading the wiki, everything is clean and as it should be.

Next, I'm running a release. The output created is correct. The demo TW5 contains only the plugin and associated documentation. The plugin is correctly packaged. Nowhere can I trigger any error with my plugin ZIP distribution, so it is difficult to understand what really is causing the error you see.

I have a recent TW5 installation (5.1.2-prerelease) from the github repository side-by-side with my plugin. What TW5 installation do you have installed which is used by Node.js? Do you have global and local TW5 installations? Which one actually gets used? Which version do you use?

The error you mentioned above is from your own installation (at least this is what I understand from your post), but you are saying that you get the same error from my zip distribution ... I have no idea why this happens, as my clean installation of TW5 and my plugin work as advertised.

Regards,
-- TheDiveO

[Danielo Rodríguez]

Hello,

The link I'm referring to is the one posted on your first comment. Now I realized there is no way to change this.

About my tiddlywiki installation, I tried two: On my mac and on my windows machine. On every one I get the same error when OPENING THE PLUGINS TAB of the control panel. The Mac one is installed globally with NPM while the windows one is installed globally but manually. In both cases I'm running 5.1.2-prerelease. 

The build  scripts works correctly as you said, but the develop release gives the reported error. 

Did you tried opening the plugins tab at the control panel? This error does not appear on the offline version.

Regards.

[TheDiveO]

Now this precise description helped me to reproduce the bug and to track it down. It is actually caused by a typo in the plugin tiddler $:/plugins/TheDiveO/ThirdFlow itself: in its JSON content I've "mispselled" "tiddlers" as "tiddler" instead. I've fixed and checked this and will later release a new version.

Regards,
-- TheDiveO

[TheDiveO]

Bugfix release with more documentation on plugins, modules, and the crazy magic behind the Third Flow plugin is out: https://github.com/TheDiveO/ThirdFlow/releases/tag/v1.0.4

[Danielo Rodríguez]

Hello TheDive0.

Glad you fixed the problem. I think my description was precise at a first instance.

El miércoles, 24 de septiembre de 2014 23:02:46 UTC+2, Danielo Rodríguez escribió:

Hello TheDive0

I'm trying to use your flow to update one of my old plugins and I'm running into problems. I followed the instructions you described. I created the folder structure, I created the plugin info tiddler, but when I try to open the plugins tab I get this error:

Uncaught TypeError: Cannot read property '$:/plugins/danielo/tagSearch/tagSearch/icon' of undefined

As you can see I wrote "when I try to open the plugins tab". I find it precise enough. Maybe I should explained that the plugins tab is on the control panel. 

Anyway, thank you for providing us an easy way to setup development environments for TW plugins. Can you consider to create a repository with an empty structure with your plugin already installed? Something like a skeleton.  

[TheDiveO]

Yes, that's on my agenda. What I'm unclear about: should I package through a separate github repo or as a subfolder of the current repo that people then copy over to a new spot?

Yeah, the plugin tab got me derailed as I did not associate it with the control panel but with the advanced tab of the plugin tiddler itself. And the context of your link wasn't apparent to me together with the (stale) v1.0.0 that later came into the discovery. Alas, glad that we could finally sort it out. To better help I need information, probably a lot of it. :)

[Andreas Hahn]

Hi TheDiveO,

I think it makes sense to include the skeleton in the current github
repo as a subfolder and I would also like to see a 'testing' edition
added where people can put test code for their plugins.

/Andreas

[Danielo Rodríguez]

Defínetively a matter of taste. I would prefer a separate repository that I can clone to start a new project. But there is nothing that keeps you away of doing both things. You can even use the same sub folder for both repositories.

Regards.

[TheDiveO]

New release v1.0.8 is out: https://github.com/TheDiveO/ThirdFlow/releases/latestFixes creating new .js tiddlers. Throws in a new plugin tiddler view template that displays useful plugin meta information in a useful manner.

Enjoy!
-- TheDiveO

[Fidaak]

Hi TheDiveO,

Your ThirdFlow looks amazing and I would very much like to use it for the development of some of my ideas into TW5.

However, I am not able to run the skeleton ThirdFlow properly - when I create new tiddler, it will correctly appear in the filesystem, but as an empty file. After I restart the node.js, the file stays empty. The standard edition of TW5 is running properly.

Maybe I am not doing the skeleton project preparation right - I am still beginner with node.js and javascript. Here is a short description, how I setup the development of my plugin:

1. download the latest release of ThirdFlow

2. copy the skeleton folder from within the ThirdFlow folder to C:\

3. rename the skeleton folder to TestPlugin

4. running the develop.cmd script in node.js shell from the C:\TestPlugin\ folder

The ThirdFlow then behaves correctly, but it is now saving any data to disk, only creating empty files. After the restart of node.js, the empty files stays, but the content is gone. I am using Windows8.1 64bit edition and I start the node.js (x64) with administrative rights.

Could you please advise, what could be the problem?

Thank you!

Fidaak

Dne pátek, 28. listopadu 2014 22:14:56 UTC+1 TheDiveO napsal(a):

[TheDiveO]

Can you please give me some more information so I can try to reproduce your problem?

• When you start the ThirdFlow TW itself under Node.js (using the develop shell script) and then open it from your web browser, what version string exactly do you see? The version string is printed in bold and some color on the welcome page. Run the develop.cmd script from the command line, not from the node.js shell.
  
• You don't need to start nodejs with administrative rights; my understanding is that this actually is an extremely bad idea from the viewpoint of security: it gives some untrusted and unreliable code adminstrator priviledges to whoever manages to attach to your nodejs server process. From the perspective of file creation it may also be a possible source, but I'm not sure. Please try to run nodejs without administrator rights. I solely develop with normal user account priviledges, there is no need to do otherwise.
• Can you please run the ThirdFlow TW itself and create a new tiddler with some content? Does this work as expected or does it fail too and leaves only empty .tid files?
• What kind of tiddler did you try to create, ie. what title and what type? I'm asking as there are differences in tiddler creation depending on what the title is and the type of a tiddler. The rationale is that javascript tiddlers need to be handled differently from genuine wiki markup tiddlers.
  

[TheDiveO]

Okay, I've found the bug and I'm working on a fix.

[Fidaak]

Excellent, looking forward to next version! :-)

Dne čtvrtek, 18. prosince 2014 19:32:39 UTC+1 TheDiveO napsal(a):

[senechaux]

Seeing the same problem (0-byte files).  The rename of the Skeleton tiddlers (.json?) result in empty files.

As this is nearly a year after previous reply, perhaps TW 5.1.9 (node) broken with ThirdFlow?

Given the startup message below, I tinkered with the "filesystem" plugin dependency but no joy.

$ ./develop

HierarchicalFileSystem: folder policy config: default: hierarchical subfolders are enabled

HierarchicalFileSystem: folder policy config: draft: draft subfolder is: drafts

HierarchicalFileSystem: folder policy config: system: system subfolder is: system

HierarchicalFileSystem: 4 folder policies active

Boot log:

  Startup task: load-modules

  Startup task: info after: load-modules before: startup

  Startup task: startup after: load-modules

  Startup task: story after: startup

  Startup task: commands platforms: node after: story

Executing command: server 8080 $:/core/save/all text/plain text/html

Serving on 127.0.0.1:8080

(press ctrl-C to exit)

Warning: Plugins required for client-server operation ("tiddlywiki/filesystem" and "tiddlywiki/tiddlyweb") are missing from tiddlywiki.info file

syncer-server: Dispatching 'save' task: $:/StoryList

HierarchicalFileSystem: Saved file C:\PluginRepo\src\tiddlers\system\StoryList.tid

Hoping for any viable ideas/solutions and ready to answer questions if more details needed.

[emicius]

Since last post was nearly a year ago,  I assume the thread is abandoned. Anyway, just wanted to point a quick fix to senechaux's problem - the hierarchical file system adaptor seems to be missing one function. I made a pull request here: https://github.com/TheDiveO/ThirdFlow/pull/9/files you might just copy those lines to your files or clone my fork.

EM

[TheDiveO]

"I'm not quite dead yet" ... so I've finally managed to update the ThirdFlow plugin to work on recent TW5 5.1.13. Many thanks to those who send me pull requests; I've incorporated these fixes. There are minor updates to the docu, as well as the skeleton files.