Twoter: TiddlyWeb Based Web Clippings

[chris...@gmail.com]

As part of my continuing work on TiddlyWeb I've been exploring ways to
use and extend it. I've written about the latest experiment on my
blog:

http://cdent.tumblr.com/post/51774343/twoter-tiddlyweb-based-web-clippings

(That posting also includes links to other similar experiments).

Twoter is an experimental service running on Google App Engine that is
a sort of bookmark or web clippings system. In some ways it is like
TiddlySnip, but the clippings are saved to a centralized TiddlyWeb
server where you can access those clippings in a variety of ways,
including as a TiddlyWiki. You can go to it directly here:

http://tiddlyweb.appspot.com/twoter

If you choose to try this experiment, please read the blog posting
above for details on how it works and what it is doing and for some
warnings. The most important of these warnings are that it might blow
up in your face, that the information may not be secure, and that the
content may go away at any time[1].

I'm tossing this out to see what people think. Response in IRC was
pretty positive so this may be more useful than I initially thought.
If it is useful, I'll tune it up a bit more and announce it more
broadly.

[1] If you gather content that you think is valuable you can make a
persistent backup of the content by creating a TiddlyWiki of all your
content. You can do this by constructing an appropriate URL. If you're
google username is 'chris.dent' then the URL would be:

http://tiddlyweb.appspot.com/recipes/chris_dent-all/tiddlers.wiki?download=twoter.html

(note that . has been changed to _ in the username). This will
download a file called twoter.html to your system.

[Neil O]

That's really neat. I can definitely see myself using this, e.g. to
grab part of a wikipedia article, which I can then modify.

One thing: it doesn't seem to let you take more then one snippet from
the same URL. Could a new tiddler be added, but with a slightly
modified title? Webpagetitle (2), Webpagetitle (3) etc?

Cheers
Neil

On Sep 25, 6:42 pm, "cd...@peermore.com" <chris.d...@gmail.com> wrote:
> As part of my continuing work on TiddlyWeb I've been exploring ways to
> use and extend it. I've written about the latest experiment on my
> blog:
>

>  http://cdent.tumblr.com/post/51774343/twoter-tiddlyweb-based-web-clip...

>
> (That posting also includes links to other similar experiments).
>
> Twoter is an experimental service running on Google App Engine that is
> a sort of bookmark or web clippings system. In some ways it is like
> TiddlySnip, but the clippings are saved to a centralized TiddlyWeb
> server where you can access those clippings in a variety of ways,
> including as a TiddlyWiki. You can go to it directly here:
>
>  http://tiddlyweb.appspot.com/twoter
>
> If you choose to try this experiment, please read the blog posting
> above for details on how it works and what it is doing and for some
> warnings. The most important of these warnings are that it might blow
> up in your face, that the information may not be secure, and that the
> content may go away at any time[1].
>
> I'm tossing this out to see what people think. Response in IRC was
> pretty positive so this may be more useful than I initially thought.
> If it is useful, I'll tune it up a bit more and announce it more
> broadly.
>
> [1] If you gather content that you think is valuable you can make a
> persistent backup of the content by creating a TiddlyWiki of all your
> content. You can do this by constructing an appropriate URL. If you're
> google username is 'chris.dent' then the URL would be:
>

>  http://tiddlyweb.appspot.com/recipes/chris_dent-all/tiddlers.wiki?dow...

[chris...@gmail.com]

On Sep 26, 3:49 am, Neil O <NeilInCana...@gmail.com> wrote:
> One thing: it doesn't seem to let you take more then one snippet from
> the same URL. Could a new tiddler be added, but with a slightly
> modified title? Webpagetitle (2), Webpagetitle (3) etc?

That's a good idea. I should be able to put something like that in.
I'll try to do it today.

Thanks very much for the feedback.

[chris...@gmail.com]

This is done. If a Tiddler already exists with the same title it gets
-2 appended, and then -3 etc.

I also fixed a bug that Jeremy found when pages have | (pipe or
vertical bar) in their titles. This was messing up the server side
wikifier.

[Xavier Verges]

This is cool. Can stuff be deleted/edited?

I'm puzzled, though. The bookmarklet code does not work for me. Live
HTTP headers does not show any activity, and no complain goes to the
console.

However, if I edit the javascript in there and remove the
iframe.style.display='none';
thing work ok... Firefox 2.0.0.17

Anyone got a clue?

-Xavier

On Sep 26, 12:42 am, "cd...@peermore.com" <chris.d...@gmail.com>
wrote:

> As part of my continuing work on TiddlyWeb I've been exploring ways to
> use and extend it. I've written about the latest experiment on my
> blog:
>

> http://cdent.tumblr.com/post/51774343/twoter-tiddlyweb-based-web-clip...

>
> (That posting also includes links to other similar experiments).
>
> Twoter is an experimental service running on Google App Engine that is
> a sort of bookmark or web clippings system. In some ways it is like
> TiddlySnip, but the clippings are saved to a centralized TiddlyWeb
> server where you can access those clippings in a variety of ways,
> including as a TiddlyWiki. You can go to it directly here:
>
> http://tiddlyweb.appspot.com/twoter
>
> If you choose to try this experiment, please read the blog posting
> above for details on how it works and what it is doing and for some
> warnings. The most important of these warnings are that it might blow
> up in your face, that the information may not be secure, and that the
> content may go away at any time[1].
>
> I'm tossing this out to see what people think. Response in IRC was
> pretty positive so this may be more useful than I initially thought.
> If it is useful, I'll tune it up a bit more and announce it more
> broadly.
>
> [1] If you gather content that you think is valuable you can make a
> persistent backup of the content by creating a TiddlyWiki of all your
> content. You can do this by constructing an appropriate URL. If you're
> google username is 'chris.dent' then the URL would be:
>

> http://tiddlyweb.appspot.com/recipes/chris_dent-all/tiddlers.wiki?dow...

[FND]

Great job, Chris!
I love how easy TiddlyWeb makes this sort of thing...

Just one concern: The bag name exposes the respective e-mail address. A
simple .replace("@","") might be all we need there.

-- F.

[FND]

> Just one concern: The bag name exposes the respective e-mail address.

Never mind that - I still had yesterday's prototype URLs in the browser
history. Sorry for the confusion.

-- F.

[chris...@gmail.com]

On Sep 26, 4:18 pm, Xavier Verges <xver...@gmail.com> wrote:
> This is cool. Can stuff be deleted/edited?

The functionality is there, but the path to it is a bit less
straightforward than I would like (suggestions on how to improve most
welcome).

Basically, every twote you make is placed into a storage location
(based on your user name). You can view the list of twotes by the
links on the /twoter/{username} page and click on them to get an HTML
rendering. From that list page there is also a link that allows you to
load the twotes up into a TiddlyWiki. Inside that TiddlyWiki you can
edit the content in all the usual Tiddly ways and an included plugin
(TiddlyWebAutoSavePlugin and TiddlyWebAdaptorPlugin) will save your
changes to the server (if you're connected to the net).

There's no UI for deleting but TiddlyWeb will let you delete content
by sending a DELETE to the canonical URL of the tiddler. This is based
on the name of the bag and the title of the tiddler. For a tiddler
with name 'Logs' in my storage the url is:
http://tiddlyweb.appspot.com/bags/chris_dent-all/tiddlers/Logs

Ideally there would be support for delete in the TiddlyWiki that is
generated by the system. This hasn't happened yet, in part because
I've been focused on making TiddlyWeb and not on making TiddlyWiki
plugins. If people are interested, I'd love some help. Things that are
needed are:

* An easy way to delete tiddlers out of a selected bag.
* A nice interface for controlling access to content. Right now things
are setup so that only you have any access to your content. It doesn't
have to be like this, stuff can be shared (based on what are called
Policies on storage Bags).
* A more scrumptious style for the generated TiddlyWiki. Right now it
is the default, which is a bit tired and not particularly suited to
the twoter use case.
* The bookmarklet working for IE.

Longer term there are going to be some scaling issues which will need
to be addressed. Right now each person's twotes goes into just one
container, which could get huge over time.

> I'm puzzled, though. The bookmarklet code does not work for me. Live
> HTTP headers does not show any activity, and no complain goes to the
> console.
>
> However, if I edit the javascript in there and remove the
> iframe.style.display='none';
> thing work ok...  Firefox 2.0.0.17

That's really odd. I've only tested on Safari and Firefox (somewhere
in the version 3), but in my testing I did find that the silliest
little tweaks would make things blow up. It's fragile stuff.

[chris...@gmail.com]

On Sep 26, 4:18 pm, Xavier Verges <xver...@gmail.com> wrote:

> This is cool. Can stuff be deleted/edited?
>
> I'm puzzled, though. The bookmarklet code does not work for me. Live
> HTTP headers does not show any activity, and no complain goes to the
> console.

I'm getting intermittent failures of the bookmarklet and am unable to
discern any pattern. That is I can't tell if it is something to do
with the website being looked at, or some other factor. I guess I will
need to instrument the bookmarklet a bit to be more noisy about what
it is up to, and see where the failure is happening.

Is anyone getting 100% success when using the bookmarklet? Anyone
determined what might be wrong with it?

Thanks.

[chris...@gmail.com]

On Sep 29, 2:44 pm, "cd...@peermore.com" <chris.d...@gmail.com> wrote:
> I'm getting intermittent failures of the bookmarklet and am unable to
> discern any pattern. That is I can't tell if it is something to do
> with the website being looked at, or some other factor. I guess I will
> need to instrument the bookmarklet a bit to be more noisy about what
> it is up to, and see where the failure is happening.

I've added some detecting and handling of character encoding. It seems
to improve things somewhat.

http://trac.tiddlywiki.org/changeset/6996

[chris...@gmail.com]

And today I've changed around the twoter page a bit so it includes
links to more of the stuff you can do with the twotes that you gather.
See:

http://tiddlyweb.appspot.com/twoter

[chris...@gmail.com]

Just a heads up to say that twoter doesn't work in recent WebKit
nightlies because they do not allow javascript in frames to operate on
the containing document. Off the top of my head I don't know what the
solution to this is, so for now twoter doesn't work in that
environment. Any suggestions?

[FND]

> Just a heads up to say that twoter doesn't work in recent WebKit
> nightlies because they do not allow javascript in frames to operate on
> the containing document.

That might be a problem, as I assume TiddlyWiki makes use of this
technique as well (though I might be wrong).
I'll try to get some details on this.

-- F.

[Jeremy Ruston]

Not quite; the TW import mechanism uses javascript to operate on
documents contained in an iframe (kinda the other way around), and
I've just checked and it still works properly in the latest Safari
nightlies.

Cheers

Jerm

>
> -- F.
>

--
Jeremy Ruston
mailto:jer...@osmosoft.com
http://www.tiddlywiki.com