[TW5] Could TW display a file that is actually a tiddler?

[Mat]

Comparable to, perhaps, having a TW display a local image - could we have a (not node.js) TW display a local file that is actually a tiddler? So that is appears as a tiddler.

Thank you

<:-)

[Tobias Beer]

Comparable to, perhaps, having a TW display a local image 

could we have a (not node.js) TW display a local file that is actually a tiddler? 

So that is appears as a tiddler.

You can load a tiddler via object embed and render as text/plain...

Embedding A Text File @ tb5

This doesn't work with tid files on GitHub.

GitHub's CORS policies won't allow you to do that.

Best wishes, Tobias.

[Mat]

Thank you Tobias - that is very cool and useful ! The result there is basically a tiddler in my TW merely display of the full code for that external tiddler.

However, I'm wondering if it is possible to actually make it display as a normal tiddler? (Pretty much making TW behave like node.js TW in this specific matter).

I'm guessing the answer is that we dont'.

I think it would be useful and I see a few possible scenarios for what this could actually be;

We could have a type of "display window tiddler", in the TW, where the fields are kind of empty place holders merely displaying "txt" as is shown under each respecitve field definition in the tiddler file. (A kind of transclusion, I guess?) This is kind of what the solution you suggested does but that solution doesn't split up the file into each respective field. I'm not quite sure how to handle the title - should it be titled in the TW or should it pull/transclude the title from the file? If the former, then what to do with the title in the file? Add a field "sourcetiddlertitle:..."?

Why useful, well not so much at that stage, but one could imagine a mechanism to really convert it to be actual code. Perhaps the tiddler file is displayed as explained above in edit mode and it gets converted into actual tiddler content on saving the tiddler.

IMO the most interesting application would be if the external tiddler could remain as the external source and that kind of just temporarily pulled in on demand - i.e processed/rendered temporarily! Again, a bit like external images but with possibility to execute the code and have text readin [[link]] to actually, but temporarily, be link. (What is it called when the text or code is "interpreted" in TW? Wikified?

This could perhaps allow meta data to be added in "shell" tiddler e.g tags and perhaps even text, if the pulled content was appended.

You could use the exporting mechanism of a tiddler in some external TW and save a tiddler to your local directory.

Thank you!

<:-)

[Tobias Beer]

Hi Mat,

 

However, I'm wondering if it is possible to actually make it display as a normal tiddler? 

(Pretty much making TW behave like node.js TW in this specific matter).

That requires a server for TiddlyWiki to talk to and make XmlHttpRequests delivering the tiddlers wrapped in the expected response. So, just accessing the filesystem won't do that for you. You need...

• some TiddlyWiki code to make a request for something specific
• and the necessary response headers that tell TiddlyWiki what to do with it
  
• some TiddlyWiki code to handle the response, e.g. create tiddlers

...pretty much something like TiddlyDrive, only local.

I'm guessing the answer is that we dont'.

I think it would be useful and I see a few possible scenarios for what this could actually be

So, yes, TiddlyWiki itself, in the browser cannot provide that functionality by itself.

 

IMO the most interesting application would be if the external tiddler could remain as the external source and that kind of just temporarily pulled in on demand - i.e processed/rendered temporarily! Again, a bit like external images but with possibility to execute the code and have text readin [[link]] to actually, but temporarily, be link. (What is it called when the text or code is "interpreted" in TW? Wikified?

The method form my link above tells the browser to handle and embed the file the way it is able to, if it is. All TiddlyWiki does at that point is tell the browser to do just that. It's got no means to access that content in the same way it's got no means to access iframes just like that.

Best wishes, Tobias.

[Mat]

Thanks for your always informative answers Tobias!

However, I'm wondering if it is possible to actually make it display as a normal tiddler? 

(Pretty much making TW behave like node.js TW in this specific matter).

That requires a server for TiddlyWiki to talk to and make XmlHttpRequests delivering the tiddlers wrapped in the expected response. So, just accessing the filesystem won't do that for you.

Is that really necessary even if the tiddler file is on my local computer? By "external tiddler" I simply meant it is outside of the TW file not on a different server.

<:-)

[Tobias Beer]

Is that really necessary even if the tiddler file is on my local computer? By "external tiddler" I simply meant it is outside of the TW file not on a different server.

Unfortunately it is. Browsers are rather strict in accessing the local filesystem these days and considering all those extensions constantly talking to the web, they probably should. Personally, I would prefer a model where I can decide what gets which access to my filesystem and where. Somehow, noone seems to have bothered adding a little more flexibility, after it was stripped down ...on the browser front.

Best wishes, Tobias.

[Mat]

Here is some stuff I think may be relevant;

Read Textfiles Using the JavaScript FileReader (2013)
How to read local files
Reading files in JavaScrip using the fiel APIs (2010)
 
Do you think any of it is applicable?

Thanks!

<:-)

[Tobias Beer]

Reading files by selecting with an input or drag and drop and then being able to get certain properties is not the same as handling them via embedding. But the important question is... Why are you asking any of this?

Best wishes, Tobias. 

[Mat]

Reading files by selecting with an input or drag and drop and then being able to get certain properties is not the same as handling them via embedding. But the important question is... Why are you asking any of this?

Ok, I'm asking because of (still) the Federation thoughts. I'm trying to identify possible routes for this to see what advantages and disadvantages there are with different methods... even if I so far haven't really found any method at all. "Dragn'drop" is, I guess, the most intuitively simpe since this is what we do for general import but, failing that, I'm thinking maybe visiting a tiddler via an iframe and using its own toolbar export command to download it to your local file is something we might be able to automate. I hope to find "the best" technical solution for something that will not turn out to be a dead end for some later desired Federation features. Then to bring this to Eric so he could create a first proof of concept that is exciting enough so others will see the potential and want help take it further.

Digression: I can't let go of the Federation thoughts because I'm sure this will revolutionize our little community. It will open up for "social" behaviour in a completely different way than now. I think special interest groups will form, micro-communites if you will, that will naturally lead to much more application oriented solutions in TW. For instance, "the authors", "the graphical artists", "the mathematicians" will have incentive to develop a great set of tools and themes for their respecitve niche group. This will, in turn, make it much more easy to attract people to TW. "Hey physics teachers there is a special wiki tool just for you!", "Setting up a business? Here's the web shop toolkit!" Currently TW demands a lot of imagination and investigation, even for very competent and technical people, because it is not obvious what it can be used for.

Forgive me if the things I bring up on this are a bit "all over the place" but I'm a bit in the dark.

Regarding

That requires a server for TiddlyWiki to talk to and make XmlHttpRequests delivering the tiddlers wrapped in the expected response. So, just accessing the filesystem won't do that for you.

I just found a thread on How do I load the contents of a text file into a javascript variable? which mentions some "fudgery" (sounds tasty!) I figure this means the full contents could be pulled into a script and then one could use the fieldnames as identifiers for various parts. That should work, right?

It also brings up JSONP. On wikipedia it says "Note that for JSONP to work, a server must know how to reply with JSONP-formatted results." - but this should not be a problem in TW because both ends are TWs, right? I figure there must be some advantage that we know that the communicatinon between server and the client is in TW.
Do programmers generally know about JASONP - (and specifically Jeremy, I wonder)?


<:-)

[Astrid Elocson]

Hi Mat,

I think Jeremy talked about JSONP in one of the three or four most recent hangouts. I recall him describing it as "really horrible" or words to that effect :)

– æ

[Tobias Beer]

I think Jeremy talked about JSONP in one of the three or four most recent hangouts. I recall him describing it as "really horrible" or words to that effect :)

http://tb5dev.tiddlyspot.com/#JSONP

Best wishes, Tobias.

[Tobias Beer]

Digression: I can't let go of the Federation thoughts because I'm sure this will revolutionize our little community. It will open up for "social" behaviour in a completely different way than now. I think special interest groups will form, micro-communites if you will, that will naturally lead to much more application oriented solutions in TW.

It's pretty simple (to me)

• we need a tiddlyweb like store
  
• where registered users can host tiddlers
  
• that can be queried for relatedness
  

No server, no nothing... unless you want to drag and drop exports from wiki to wiki every the time.

I just found a thread on How do I load the contents of a text file into a javascript variable? which mentions some "fudgery" (sounds tasty!) I figure this means the full contents could be pulled into a script and then one could use the fieldnames as identifiers for various parts. That should work, right? 

It does not. We are talking about the web here, not local filesystems. Afaik, there is no local XMLHttpRequest, unless you're running a local server at some local ip address.... via node.js.

It also brings up JSONP. On wikipedia it says "Note that for JSONP to work, a server must know how to reply with JSONP-formatted results." - but this should not be a problem in TW because both ends are TWs, right?

Not right. TiddlyWiki is an html file. It does not serve anything, it is being served. You cannot call an html file and ask it to return json. Instead you call a server and ask it to get a resource at a uri. This resource hopefully has a mime type attached, so the browser knows what to do with it, e.g. display this as html, because that's what it is.

Best wishes, Tobias.

[Mat]

Tobias Beer wrote:

http://tb5dev.tiddlyspot.com/#JSONP

WHAT!? Are you psychic or something? You AGAIN write something mere hours before I bring it up. You're clearly using your psychic powers to make me look foolish - but note that my signature is a tiddlywicked magician, so I warn thee!!! I shall tiddlify you!!!

...later.

• we need a tiddlyweb like store

But that's not really federated, is it?

No server, no nothing... unless you want to drag and drop exports from wiki to wiki every the time.

I just found a thread on How do I load the contents of a text file into a javascript variable? which mentions some "fudgery" (sounds tasty!) I figure this means the full contents could be pulled into a script and then one could use the fieldnames as identifiers for various parts. That should work, right? 

It does not. We are talking about the web here, not local filesystems. Afaik, there is no local XMLHttpRequest, unless you're running a local server at some local ip address.... via node.js.

But, I am talking about a local file here! At least, that too. Or are you saying this wouldn't help either? If I go to an external TW, use the toolbar command to export that tiddler into my own local folder, and then I want to pluck out that data into some kind of tiddler shell so it displays as a tiddler in my local TW.

II'm trying this: I have a local .tid file, and making it appear in a tiddler via a method you documentet on tb5:

<object data="file:///C:/Users/....../Tiddler.tid" type="text/plain" style="width:100%;height:400px;" scrolling:yes>
</object>

This shows the whole file content of Tiddler.tid, but could I extract only what is after, say, "text:" in that file? It all feels so defined that I figure it ought to be posible.

<:-)

[Tobias Beer]

But, I am talking about a local file here!

I know, and it's precisely the problem, because — by default — you have no local server running.

 

Or are you saying this wouldn't help either?

So, yes, that.

 

If I go to an external TW, use the toolbar command to export that tiddler into my own local folder, and then I want to pluck out that data into some kind of tiddler shell so it displays as a tiddler in my local TW.

What you can do is drag and drop or use some input element to select the file, and that's it.

II'm trying this: I have a local .tid file, and making it appear in a tiddler via a method you documentet on tb5:

<object data="file:///C:/Users/....../Tiddler.tid" type="text/plain" style="width:100%;height:400px;" scrolling:yes>
</object>

This works, because your browser is able to render the file as an object, all encapsulated. It decides what to do with it for you, with its own text-viewer thingy. But, as I tried to explain before, this comes with the very same access restrictions, as, say, an iframe. You just can't tell that thing to do something. All you can do is replace it with another thing. But you can't access its internal data ...e.g. some tiddler json... just like you can't tell an element in an iframe to change its color for you... unless you have a server running and both, the iframed file and the file iframing it are hosted at the same domain.

This shows the whole file content of Tiddler.tid, but could I extract only what is after, say, "text:" in that file? It all feels so defined that I figure it ought to be posible.

So, no, there is no "extracting" ...not without a server giving you a file with a proper response that you can handle in your javascript.

Best wishes, Tobias.

[Chris Dent]

On Wednesday, January 21, 2015 at 11:32:57 PM UTC, Mat wrote:

Tobias Beer wrote:

• we need a tiddlyweb like store

But that's not really federated, is it?

TiddlyWeb itself does not have federation built in and it is not a concept built into the default API, but here are some thoughts which may be of use:

* There's a plugin[1] that implements a concept known as remotebag[2] which probably fits in here somewhere.

* Federation is something happens as a result of an aggregation of servers and clients that are aware of each others existence. That awareness is the problem which needs to be solved/managed.

* A file:// based TW5 configured with awareness of multiple TiddlyWeb servers[3] (or one TiddlyWeb server that holds knowledge of multiple TiddlyWeb servers) could self-configure to talk to those multiple servers and dynamically use content from them. It doesn't strictly have to be file-based even.

I've experimented with this latter idea a little bit with only one server[4]. I think TW5 would need some adjustments to cleanly do this with multiple servers but I can't see any huge technical barriers to it happening. Content is relatively easy, plugin loading less so.

I agree that once it becomes straightforward to take an existing TW5 and make it talk to arbitrary places on the network some very interesting things will be possible.

[1] https://pypi.python.org/pypi/tiddlywebplugins.remotebag

[2] http://cdent.tiddlyspace.com/remotebag

[3] That are CORS-enabled

[4] https://groups.google.com/d/msg/tiddlyweb/UcYrnvGwNUQ/UJRgSn4VjfYJ

[Tobias Beer]

Hi Chris,

Some very interesting thoughts here!

 

there's a plugin[1] that implements a concept known as 

remotebag[2] which probably fits in here somewhere

might not even be needed if TiddlyWiki itself does the cross referencing

when talking to a number of TiddlyWeb servers

one question is, at what point would a TiddlyWeb

go talk to remote bags if not through user interaction?

there could be some polling going on to cyclically pull updates

sounds a bit like messaging, although optimizing this process

might need some thinking about replicating / caching stuff,

e.g. only fetch tids where modified > last request.

so, you would essentially mirror the remotebag contents, read-only

perhaps not caching things older than X

Federation is something happens as a result of 

an aggregation of servers and clients that are aware of each others existence. 

That awareness is the problem which needs to be solved/managed.

There are two ways to approach this...

1. have TiddlyWeb do the remotebag talky-talky
2. have TiddlyWiki talk to a number of TiddlyWeb instances
   and cross reference the tiddlers in some way,
   e.g. via fields that read like...
• reply-to: http://unique.com#origin
• reply-to: http://unique.com/tweb/bags/{bag_name}/tiddlers/{tiddler_title}

A file:// based TW5 configured with awareness of multiple TiddlyWeb servers[3] 

(or one TiddlyWeb server that holds knowledge of multiple TiddlyWeb servers) 

could self-configure to talk to those multiple servers and dynamically use content from them. 

It doesn't strictly have to be file-based even.

Yes, I take it that there are db adaptors for TiddlyWeb.

So, storing tiddlers in bags and defining recipes does not require a purely file-based storage.

TiddlyWeb decides on how to pull the tiddlers and from where,

so all you need is to properly address the api, the rest is under the hood.

I've experimented with this latter idea a little bit with only one server[4]. I think TW5 would need some adjustments to cleanly do this with multiple servers but I can't see any huge technical barriers to it happening. Content is relatively easy, plugin loading less so.

Right now, we're just talking content, imho.

But plugin loading should also work,

you just need to locally save that thing to your wiki and then reload.

Loading / updating plugins should be a conscious act.

However, talking to those remote servers might provide a mechanism

to indicate that there is an update for some plugin of yours... when in authoring mode.

I agree that once it becomes straightforward to take an existing TW5 

and make it talk to arbitrary places on the network 

some very interesting things will be possible.

I really think the syncer module(s) should allow multiple instances talking to distinctly configured stores... with appropriate conflict handling, i.e. store that content in some unique namespace where it doesn't conflict with the "locally owned" tiddlers, e.g.

$:/remote/origin-url/api-path

Locally owned meaning: some origin must be defined as the master for this TiddlyWiki, e.g. providing the tiddlers as-is, no system-namespacing.

The above would eventually hold a tiddler name at the very end. Some filter could extract all remote tiddlers that have the same name as a local one and appropriately show a conditional button in the toolbar which opens a popup containing a list of remote tiddlers ...when opened, a custom ViewTemplate shows this tiddler, but without all the $:/remote/foo... rather showing a clean Tiddler ui that hides the ugly title and clearly indicates that this tiddler came from someplace else, while pointing there.

Anyhow, ownership seems to be the key in what needs configuration within TiddlyWiki, i.e.

• "Am I the owner of the content from a given remote server configuration?"
• yes: ok, I can fully CRUD — so long as the server allows me to
• no: I can only read and duplicate into whatever is my own local store
• "Is this my master store?"
• ok, so these are the tiddlers loaded without any namespace-prefixing

Best wishes, Tobias.

[chris...@gmail.com]

On Thu, 22 Jan 2015, Tobias Beer wrote:

> might not even be needed if TiddlyWiki itself does the cross referencing
> when talking to a number of TiddlyWeb servers

Yes, exactly the point I was trending towards.

> one question is, at what point would a TiddlyWeb
> go talk to remote bags if not through user interaction?

In the current implementation it is only through a user agent
(generally prompted by some kind of click) leading to a request for
a tiddler in a remote bag. That could indeed be driving by polling,
events, or anything else, as you say.

But again, I think doing in TW5 would be more flexible and powerful.

> There are two ways to approach this...
>

> 1. have TiddlyWeb do the *remotebag* talky-talky

Fun for interesting tricks[1] but not ideal.

> 2. have TiddlyWiki talk to a number of TiddlyWeb instances

> and cross reference the tiddlers in some way,
> e.g. via fields that read like...

> - *reply-to*: http://unique.com#origin
> - *reply-to*:

> http://unique.com/tweb/bags/{bag_name}/tiddlers/{tiddler_title}

Yes, something like that.

>> It doesn't strictly have to be file-based even.

(What I'm meaning here is that you can do the fun with TW5 when the
wiki is loaded from file: or http:, not file-based as in data being
stored in files.)

> Yes, I take it that there are db adaptors for TiddlyWeb.
> So, storing tiddlers in bags and defining recipes does not require a purely
> file-based storage.
> TiddlyWeb decides on how to pull the tiddlers and from where,
> so all you need is to properly address the api, the rest is under the hood.

Yes.

> I really think the syncer module(s) should allow multiple instances talking
> to distinctly configured stores... with appropriate conflict handling, i.e.
> store that content in some unique namespace where it doesn't conflict with
> the "locally owned" tiddlers, e.g.

That makes sense.

> The above would eventually hold a tiddler name at the very end. Some filter
> could extract all remote tiddlers that have the same name as a local one
> and appropriately show a conditional button in the toolbar which opens a
> popup containing a list of remote tiddlers ...when opened, a custom
> ViewTemplate shows this tiddler, but without all the $:/remote/foo...
> rather showing a clean Tiddler ui that hides the ugly title and clearly
> indicates that this tiddler came from someplace else, while pointing there.

This too.

> Anyhow, ownership seems to be the key in what needs configuration within
> TiddlyWiki, i.e.
>

> - "Am I the owner of the content from a given remote server configuration
> ?"
> - *yes*: ok, I can fully CRUD — so long as the server allows me to
> - *no*: I can only read and duplicate into whatever is my own local
> store

If you have a look at the JSON representation for a tiddler produced
by TiddlyWeb it has a permissions attribute that says what the
current user can do to the current tiddler. See:

http://tiddlyweb.tiddlyspace.com/HomePage.json

for an example. For me it says:

"permissions": ["read", "write", "create", "delete"]

That's there pretty much for the reasons you're describing.

[1] "remotebag" is a subclass of a concept in TiddlyWeb called
"specialbag" which was created to allow pretty much anything to be
transformed into tiddler in a bag. It provides the interface that
allow some thing to be presented as a list of tiddlers, and some
(other) thing within that thing to be presented as a single tiddler.
Examples included atom feeds, tweets, text files on disk, etc.
--
Chris Dent http://burningchrome.com/
[...]

[Tobias Beer]

[1] "remotebag" is a subclass of a concept in TiddlyWeb called
"specialbag" which was created to allow pretty much anything to be
transformed into tiddler in a bag. It provides the interface that
allow some thing to be presented as a list of tiddlers, and some
(other) thing within that thing to be presented as a single tiddler.
Examples included atom feeds, tweets, text files on disk, etc.

I understand now... remotebag really

isn't designed for TiddlyWeb instances cross-talking,

but rather to "transform" otherwise non-tiddler stuff

into TiddlyWeb-accessible data by whichever outside-world server

being so kind as to produce and provide content

in ways consumable by TiddlyWeb ...on-demand.

Best wishes, Tobias.