Announcing TiddlyMarker v0.1.0-alpha for Firefox

[Yestin Harrison]

I've been working on a browser extension to suit a particular manner of bookmarking with TiddlyWiki, unimaginatively titled TiddlyMarker. The basic idea is a simple button that produces a bookmark tiddler from the current tab, with various ways of getting that tiddler into a wiki, and flexibility as to its structure. Currently, the two modes of saving work via the TW webserver API, and simple browser downloads, respectively.

Anyone who wants to give it a try can clone the repository at https://git.ylh.io/tiddlymarker and follow the README to get up and running.

Of interest to developers: the code is a bit of a mess (JS isn't really my forte), and it's missing the planned saving mode for adding to an already open TiddlyWiki tab; getting the extension to play nice with $tw in the page context is proving to be a bit of a faff, especially in a way that can easily be ported to Chrome (that's planned too). Commented sections show the beginnings of such a mode, and patches are welcome; email them to yes...@ylh.io.

It's in a reasonable working state, and I'm now soliciting feedback before I port it to Chrome and package it for Mozilla and Google.

Enjoy!

[PMario]

Hi,

Your manifest contains a content_security_policy": 'unsafe-eval' ... Why? IMO there is no need to execute unsafe code.

Can you please explain!

The variables in the code itself seem to be shortened, which makes reading the code very hard. Are you sure, it will pass the AMO code guidelines.

Your background.js code contains this header:

/* this is some particularly spicy garbage. tl;dr we can't await anything in the

user input handler if we hope to call openPopup(), which is rather crucial

to the whole “conditional popup” “quick mode” *thing*. so, instead of

accessing the appropriate storage in the input handler, we close over these

globals in the storage update handlers for local.state and sync.quickmode.

we fire off some fake updates to the values to make sure those closures run,

then in the input handler we use these globals and pretend we've accessed

storage.

with any luck once this addon is stable i will never write another line of

this hideous language as long as i live */

Why do you think, this makes users feel safe using your add on.

regards

-mario

[PMario]

On Wednesday, June 3, 2020 at 8:40:39 AM UTC+2, Yestin Harrison wrote:

Anyone who wants to give it a try can clone the repository at https://git. ylh.io/tiddlymarker and follow the README to get up and running.

Why didn't you point us to the gitHub repo at: https://github.com/ylh/tiddlymarker

-m

[Yestin Harrison]

> Can you please explain!
There are exactly two calls to eval in the addon, which correspond to
the fields in the settings for tiddler format. As best as I can tell,
there's no way around that if I'm to give the user control over how
the tiddlers should be structured, which is a non-negotiable feature
in my book. If there's a more controlled way to do that than eval(),
I'd love to know about it – again, I'm really not experienced in JS.
Part of why I like TiddlyWiki is that I don't usually have to write
any.

> Your background.js code contains this header:

That's not a header, it's a comment on the declarations directly
beneath it, to explain a rather ugly implementation detail about
glossing over asynchronous APIs in a strictly synchronous context
(handling the browser action click event).

> Why didn't you point us to the gitHub repo at: https://github.com/ylh/tiddlymarker

That works too; I'm mirroring to GitHub in case anything should ever
happen to my server.

[Yestin Harrison]

Well then, just today I found a footnote on the manifest docs
<https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/content_security_policy>.
It reads:
Note: Valid examples display the correct use of keys in CSP. However,
extensions with 'unsafe-eval', 'unsafe-inline', remote script, blob,
or remote sources in their CSP are not allowed for extensions listed
on addons.mozilla.org due to major security issues.

Well, as it turns out, it *is* possible to allow for user formatting
without permitting eval in the CSP; the mechanism is just a rather
strange one – it involves the use of content scripts via
tabs.executeScript(). I suppose the restricted context of content
scripts makes that safe enough for Mozilla not to dismiss out of hand.
manifest.json and background.js should reflect this in the latest
commit.

[Joshua Fontany]

Commenting to check back and try this. Getting my bookmarks out of browser storage and into tiddlywiki is going ok atm, but having a solution to use after the bulk import for day to day bookmarking will be excellent!

Mahalo (thanks),
Joshua Fontany

[TonyM]

Yestin,

As a superuser, not well versed with git hub, and I am sure there are others, I cant review until its a tiddler, json, plugin or demo wiki.

Do share one of these if you want more feedback.

Regards

Tony

[TonyM]

Or I just realised a plugin for Firefox.

Without more instructions.

Thanks for your contribution

Tony

[Yestin Harrison]

I'm happy to report that Mozilla have approved the latest version of TiddlyMarker for display on AMO. It should be much easier to install and try out now at <https://addons.mozilla.org/en-US/firefox/addon/tiddlymarker/>. I can confidently say I'm having a good time using it for my own purposes, and I'm excited to see what others think.

[Mark S.]

It's interesting, but seems to want only a traditional node connection, so Bob and Tiddlyserver won't work.

After trying several times, it seems that the web server address needs to be in form http://127.0.0.1:8080 . It would be helpful if the form of the address was explained.

After saving on node, the new item doesn't show up in the recent list until you refresh the browser.

For the download version, it popped up a "save" message box rather than observing the "Save to download" browser setting. This is too annoying for me.

Getting rid of the "save dialog" would be a good enhancement.

Thanks!

On Tuesday, June 2, 2020 at 11:40:39 PM UTC-7, Yestin Harrison wrote:

I've been working on a browser extension to suit a particular manner of bookmarking with TiddlyWiki, unimaginatively titled TiddlyMarker. The basic idea is a simple button that produces a bookmark tiddler from the current tab, with various ways of getting that tiddler into a wiki, and flexibility as to its structure. Currently, the two modes of saving work via the TW webserver API, and simple browser downloads, respectively.

Anyone who wants to give it a try can clone the repository at https://git.ylh.io/tiddlymarker and follow the README to get up and running.

Of interest to developers: the code is a bit of a mess (JS isn't really my forte), and it's missing the planned saving mode for adding to an already open TiddlyWiki tab; getting the extension to play nice with $tw in the page context is proving to be a bit of a faff, especially in a way that can easily be ported to Chrome (that's planned too). Commented sections show the beginnings of such a mode, and patches are welcome; email them to

[TonyM]

Yestin,

I have installed this and started playing with it. I think the documentation could be extended but it already seems quite powerful.

For others to read,

I went to facebook and clicked create bookmark, and it prompted to save Facebook.json, I dragged and dropped it on tiddlywiki.com and it imported the two tiddlers, one the icon the other the bookmark tiddler using the icon, the link field and a raw-title field. This works on online wikis as well.

Notes;

Its fantastic how it brings the icon with it, allowing us to harvest appropriate icons from the internet favicon.

One can update various things while doing this bookmark capture title text etc...

Questions;

Could we also be allowed to provide a name for the icon?, basing it on the domain name at a first step?

Could there be a R-click tiddlymarker option to harvest from a link on the page?

Note: I use the oneTab extension and often collapse open tabs into a list of links, a r-click would allow me to harvest these.

I am yet to work through its full functionality but it looks great.

Thanks

Tony

[Yestin Harrison]

Mark,

> It's interesting, but seems to want only a traditional node connection, so Bob and Tiddlyserver won't work.

I'm open to supporting these – As far as Bob is concerned, I've found
documentation and source code pertaining to API routes, which means I
could feasibly implement support. I've had no such luck poking through
TiddlyServer, but if someone were to point me to some docs, I could
make an attempt.

> It would be helpful if the form of the address was explained.

Good catch; that's an easy enough addition.

> After saving on node, the new item doesn't show up in the recent list until you refresh the browser.

I'm afraid this appears to be a limitation of the API, as there
doesn't seem to be a call to push changes to clients. I find that in
5.1.22, the new “cloud button” -> “Get latest changes from the server”
is much smoother than hitting refresh.

> Getting rid of the "save dialog" would be a good enhancement.

I'll likely add a checkbox to this effect; luckily, this is quite an easy fix.

Tony,

> Could we also be allowed to provide a name for the icon?, basing it on the domain name at a first step?

It would certainly be possible to add more parameters to favicon_fmt.
The rationale behind the current default is that a SHA-1 hash of the
icon data itself makes for an easy enough way of avoiding duplicates,
whereas depending on the site, different pages on the same domain can
easily feature different favicons. Are you envisioning an additional
field in the popup as well?

> Could there be a R-click tiddlymarker option to harvest from a link on the page?

The current mechanism relies heavily on reading from a given tab using
the corresponding API
<https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/tabs/Tab>,
in particular as a foolproof way of obtaining the favicon, altogether
avoiding reading the page DOM. If I were to implement something like
this, it would likely open the link as a tab in the background and try
to read from it once loaded. It would be a bit janky, but I do like
the sound of that workflow.

Thanks for the feedback, both of you. The code is messy and overdue
for a refactor, especially with the most recent additions, but I'll
keep what's been said here in mind as I push for 0.3.0.

[TonyM]

Yestin,

perhaps a field that contains the randomly named icon that can be overridden, perhaps click to name it domain/path.icon if I harvested the Facebook.com icon twice I would be happy to overwrite it rather than have two identical.

It may be a lot of work but I would love the same thing on a right-click on links.

Perhaps one day we could add a preview of the link to the tiddler.

Nice work
Tony

[Yestin Harrison]

Well, 0.3.0 is live on AMO. Besides fixing a rather nasty bug that
rendered the popup inoperable under certain circumstances, I took it
as an opportunity to make the download dialog opt-in (default off),
and add an explanatory label to the server URL.

As far as large features are concerned, there's one primary issue –
there are rather strict requirements on what execution context can
trigger the popup. This is ostensibly to protect users from surprising
behaviour, but the long and of it is that as soon as I await something
(such as making sure a tab is opened and then loaded, as would be the
case for a context menu item to bookmark a link), I no longer have the
ability to bring up the popup. That is to say, anything that depends
on a resource that is not the current tab would have to either
unconditionally run in “quick mode” with no ability for the user to
edit anything, or instantiate some different mechanism within the page
(i.e. port the popup to instantiate in a content script).

The latter would work with some effort, but it has the unfortunate
property that it would all be for *just* links. The lovely thing about
the context menu API is that among its supported contexts are native
browser bookmarks, which would be *incredibly* convenient for porting
over existing bookmarks. Frustratingly enough, though, doing anything
meaningful with those is just out of reach. With no corresponding page
context to run in, and no way to signal anything important before
bringing up the popup in a manner programmatically indistinguishable
from an ordinary click on the extension button (!), it would have no
way of presenting any user interface.

I've already had to hack around this strict behaviour to get the popup
to be whatsoever conditional, but I believe that's the most I can do
there. If there are any JS gurus reading who know some reason what I
wrote might be wrong, please do let me know – I'd love to implement
something like this if the extension APIs allowed me to.

With the disappointing technical bits out of the way, some good news:
page previews are actually incredibly easy to get from the tabs API,
so some form of support for that would be doable. I'm mulling over
ways to make favicon formatting more customisable. One potential
solution is to just expose a snapshot of the whole tab data structure
to both formatting functions and throw the user a link to MDN if they
want to mess around. Another is to split the favicon title into its
own formatting function, and evaluate it at slightly different points
than where the current two are, so it could be intercepted when the
popup is brought up.

Either way, the addon is in quite a healthy state at the moment, and
should get even handier and more flexible very soon.

[amreus]

I made and quickly deleted a post asking about using a prefix ore template for tiddler naming. 

I found the answer in the tiddlymarks options page. You can construct the fields as you wish in the Formatting section. Here is an example of using a system tiddler for the bookmark tiddler:

[amreus]

Oh, and thank you for creating this plugin!

[Mohammad]

@Yestin,

The addon works like a charm!

Two minor comments

1. is it possible to bookmark all open tabs? This will help preventing click TiddlyMarker on each tab you like to bookmark!

2. is there any chance to release the Chrome version

Best wishes

Mohammad

[Yestin Harrison]

I'm afraid I've missed the bus as far as porting to Chrome/Edge/etc, for now.
Previously, I hesitated because Chrome's API lacked modern promise/async/await interfaces to extension APIs, meaning I'd have had to either rewrite much of the extension logic as a 2010-style pyramid of doom, or vendor in Mozilla's MPL'd WebExtensions shim.
Now, though, Google are going full steam ahead with Manifest V3, which brings these to the APIs, but as Google giveth, Google taketh away: there seems to be literally zero support for arbitrary code execution, even arbitrary code specified by the user. This would completely eliminate arbitrary tiddler formatting, which I consider one of the core features of the addon: as recently demonstrated [1] by amreus, it lets users write in features after the fact that I didn't even anticipate.

There seems to be some vague interest in giving developers some of that power back in MV3, but for the time being, other addons intended to run user code, such as Tampermonkey, remain on MV2 [2].

I say I've missed the bus, because as of the 17th of this month, Google have stopped accepting new MV2 extensions on the web store, and I neglected reserve a cheeky spot by publishing a dummy addon or something before making a proper Chrome port.
Mozilla will be rolling out MV3 much more gently and slowly, preserving older APIs for now, so the Firefox version should work fine as is for the foreseeable future.
If I catch any news about some kind of sandboxed eval landing in MV3, I'll be sure to give an update on the feasibility of a Chrome port.

For the time being though, it looks like TiddlyMarker will remain Firefox-exclusive.

Many thanks go out to the current users of the addon; the appreciation and 5-star reviews are great to see. This was a personal project to achieve a workflow *I* wanted, and even 24 users listed on AMO is 23 more than I ever expected.

[1] https://github.com/ylh/tiddlymarker/issues/3
[2] https://github.com/Tampermonkey/tampermonkey/blob/master/build_sys/manifest.json.google.com

[Arlen Beiler]

To use this with TiddlyServer you just need to make sure you are making all requests relative to the tiddlywiki you are targeting. TiddlyServer loads NodeJS instances dynamically as they are requested. So most likely you are making the call to the root path ("http://localhost/") and usually that is not where the tiddlywiki is mounted. Instead you need to change those calls to the path "http://localhost/personal/notes/" or whatever the path to your wiki is. That's just my guess without really taking a look at it or trying anything out. I just saw you said you were using the TW webserver api.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/6c601e92-9508-4b4a-9b75-2b6d2f21d68f%40googlegroups.com.