Sanitizing JS

[AdamS]

Hi Folks,

I know it has been discussed a few times, but I keep coming back to the idea of inline javascript. Or at least something javascript-ish (javascript-esque?). I know the reason this capability isn't standard is because of security issues. I don't have much experience with this sort of thing, but I'm wondering how significant are the barriers to sanitizing inline javascript. What would need to be stripped out of a script tag to ensure that it would be safe? I'm guessing any DOM manipulation would be right out, as well as access to the window object. But even if we could just get a safe inline javascript for control flow, array, string, and number manipulation, that could be pretty cool. Could this be securely done?

Best wishes,

Adam

[TonyM]

Adam,

This question may be better placed in the tiddlywiki dev forum. There are reasons beyond secutity that javascript is not freely added, in part because it is through widgets such functionality is published. However you can include external javacript in a tiddler with raw system tag. Many plugins include javascript and libraries it is just tgat they need to be added within the framework tiddlywiki presents.

Regards
Tony

[Mark S.]

Is it a safety concern? Unless you're using a web-facing special deployment, who can change your TW code?

I kind of thought it was more of a desire to reduce maintenance stemming from changes made via javascript that violate internal TW mechanisms. Especially since people might be tempted to cut and paste code from the web based on the standard DOM model. That makes sense. The problem is that the substitute toolkit we're given has some perplexing omissions.

-- Mark

[AdamS]

I was under the impression that inline JS has been passed on because it would undermine the ability to confidently import plugins from other TWs. So all the javascript is contained within tiddlers that have been marked as modules as opposed to running inline in any old tiddler. That makes it easier to know what javascript you are going to get before you import. Otherwise somebody might be able to get hold of you tiddlywiki data because you've imported something malicious. I thought the tiddlywiki framework was made the way it was to avoid resorting to javascript for these reasons. I'm not sure where I got that impression from, but I surely did.

[Jed Carty]

Yes, importing arbitrary javascript into a wiki is generally a very bad idea. Even with browser security it could do a lot of damage.

[Mark S.]

That's probably right. But it all still comes down to whether you trust the source. If you trust the source, it doesn't matter whether the javascript is wrapped or not. If the source is untrustworthy, then it could still be doing something behind the scenes. And, except for Bob (I think), I don't think you have any access to write anything to the hard drive without going through the download dialog. So even malicious code could only get so far.

Thanks!

-- Mark

[@TiddlyTweeter]

My two cents as a person who understands quite a lot of what JS does but has little comprehension of how it does what it does. Nor much interest in it for its own sake.

All I'd say is its probably a good idea to know what you are doing before enabling it without restriction (which you can do).

As far as I grasp it, TW JS has a specific model of the "DOM" that is different from a lot of JS? So its coding needs follow a specific approach?

On the other hand, if you budding into JS and like that kind of thing, why not?

Just thoughts

Josiah

[AdamS]

Yeah, that's kind of my point. I understand why standard inline javascript is not something that can be part of the core, or anything so fundamental to the way tiddlywiki works. That's why I'm wondering if it would be possible to sanitize it, removing any DOM related stuff or anything that might pose a security problem, but allow things like string and array manipulation and basic logic. I've found the wikitext and widget syntax within TW5 to be a lot more difficult to learn than Javascript. If I'm not alone in that, being able to do some of the lifting with a skill that is more widely applicable might help with uptake with new users.

Best wishes,

Adam

[Eric Shulman]

On Tuesday, January 15, 2019 at 11:41:16 AM UTC-8, AdamS wrote:

Yeah, that's kind of my point. I understand why standard inline javascript is not something that can be part of the core, or anything so fundamental to the way tiddlywiki works. That's why I'm wondering if it would be possible to sanitize it, removing any DOM related stuff or anything that might pose a security problem, but allow things like string and array manipulation and basic logic. I've found the wikitext and widget syntax within TW5 to be a lot more difficult to learn than Javascript. If I'm not alone in that, being able to do some of the lifting with a skill that is more widely applicable might help with uptake with new users.

There are two separate issues here:

1) Being able to use custom javascript code to do calculations and other logic

2) Loading the code "on-the-fly"

As has been noted, (2) is problematic from a security standpoint... so let's just ignore that and focus on (1)...

Fortunately, the solution is *already provided* by the TWCore.  You can use

   https://tiddlywiki.com/dev/#JavaScript%20Macros

to define custom javascript functions that are loaded during startup processing (just like plugins).

Then, after startup, these javascript functions can be invoked from within tiddler content using the standard macro syntax (e.g., <<myMacro arg1:"foo" arg2:"bar">> or <$macrocall $name="myMacro" ... />.

In addition, just like the definition of regular in-line macros (created using "\define foo(...)"), a javascript macro's "job" is simply to generate and return content for further rendering by the TWCore engine.

A good example of a simple javascript macro can be found here:

   https://tiddlywiki.com/#%24%3A%2Fcore%2Fmodules%2Fmacros%2Fnow.js

This code shows the definition of the TWCore's <<now>> macro, which accepts one optional parameter -- a datetime format -- and returns the corresponding datetime text as it's result. 

In addition to the <<now>> macro, there are several other javascript macros of varying complexity that are part of the TWCore.  Use $:/AdvancedSearch to search the *shadow* tiddlers for "core/modules/macros" and you will see about a dozen results for you to study.

Hopefully, this is enough to get you started.  Let me know how it goes.

enjoy,

-e

Eric Shulman

TiddlyTools.com: "Small Tools for Big Ideas!" (tm)

InsideTiddlyWiki: The Missing Manuals

[Matthew Lauber]

Another issue with inline javascript specific to TW is that it interferes with the parse/render/update loop.  Let's say you include some inline javascript in your tiddler.  When is it supposed to execute?  When rendered?  How often?  Keep in mind that TW rerenders tiddlers if any of the state they depend on changes (like the React Virtual DOM).  These renders may not be visible to you the user.  And what happens when your javascript is executed twice?  Overall, it is a lot simpler to reason about, and less bug prone, to require all javascript calls to be part of the parse/render/update loop.  

[AdamS]

Thank you all. That's a lot of good information for me to look through and cogitate on!

Best wishes,

Adam

[Jeremy Ruston]

Hi Adam

As others have said, the primary goal of the current arrangement is to make it easier to detect/filter JS content by requiring it to be in a module tiddler. A secondary benefit of excluding inline JS is that it discourages blindly copying and pasting the kind of old-style DOM-based JS snippets that don’t work in TW5.

Funnily enough, I’ve just come across Secure EcmaScript which seems a good snapshot of the state of the art of safely executing untrusted JavaScript:

https://github.com/Agoric/SES

Here’s a demo:

https://github.com/jimpick/cabal-ses-bot

Best wishes

Jeremy

On 16 Jan 2019, at 23:25, AdamS <adam.the...@gmail.com> wrote:

Thank you all. That's a lot of good information for my to look through and cogitate on!

-- 
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/8ddce635-574b-4a79-8ccd-027ef0fd6c17%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[admls]

Hi Jeremy,

Thanks for the info. I guess I'm just wondering if there is a convenient way to filter out some features of JS and allow others to go through. I see lots of plugins that provide atomized functionality of simple capabilities of javascript. It just seems to me that if it were possible to build an effective filter, one might be able to have all that functionality in the core without bloating tw.

The Secure EcmaScript is really interesting. Though I wonder if there is a simpler way to do this without executing the code in a sandbox environment. Is it possible to, in the parsing stage, identify blacklisted keywords, and if those words appear, don't make a script tag out of the code? Or maybe (if a blacklist would be too large) just have a list of whitelisted words and symbols (including a set of allotted variable names). If the code is built only of these parts, then a script tag is made of it. Else, it doesn't run. Or it throws an error or something.

I'm fairly new to JS and programming generally; so this may all be hopelessly naive of me. I just wonder if the bloat of multiple (and sometimes overlapping) plugins to perform string, numerical, JSON, or array manipulation could be reduced.

Best wishes,

Adam

[Jeremy Ruston]

Hi Adam

Thanks for the info. I guess I'm just wondering if there is a convenient way to filter out some features of JS and allow others to go through. I see lots of plugins that provide atomized functionality of simple capabilities of javascript. It just seems to me that if it were possible to build an effective filter, one might be able to have all that functionality in the core without bloating tw.

The Secure EcmaScript is really interesting. Though I wonder if there is a simpler way to do this without executing the code in a sandbox environment. Is it possible to, in the parsing stage, identify blacklisted keywords, and if those words appear, don't make a script tag out of the code? Or maybe (if a blacklist would be too large) just have a list of whitelisted words and symbols (including a set of allotted variable names). If the code is built only of these parts, then a script tag is made of it. Else, it doesn't run. Or it throws an error or something.

I'm fairly new to JS and programming generally; so this may all be hopelessly naive of me. I just wonder if the bloat of multiple (and sometimes overlapping) plugins to perform string, numerical, JSON, or array manipulation could be reduced.

Your questions are exactly the right ones if you are interested in digging into this area.  My understanding is that it is only possible to use static analysis to sanitise a language like JS in very restricted situations. So I think most tools are actually transpilers that use a JavaScript parser to make an abstract syntax tree for the source code, and then to transliterate that back into JavaScript with the addition of guard functions that check the safety of operations dynamically.

Best wishes

Jeremy

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/1726d702-9a30-4682-92f1-fe0427743abb%40googlegroups.com.