Client-side encryption of TiddlyWebWiki
Poojan Wagh
plugin: [http://remotely-helpful.com/TiddlyWiki/
TiddlerEncryptionPlugin.html#TiddlerEncryptionPlugin] but it does not
seem to encrypt. The data is displayed in plaintext in the browser,
and the TiddlyWeb store has the original plaintext data on disk.
My guess is that TiddlyWeb's save functionality gets triggered before
the encryption plugin can work. Or, the encryption plugin is too old
and is not compatible with TiddlyWebWiki's more recent version of
TiddlyWiki.
Any help in getting a client-side encryption plugin to work would be
much-appreciated.
FND
> displayed in plaintext in the browser, and the TiddlyWeb store has
> the original plaintext data on disk.
Indeed. That's because TEP operates on the document level, hooking into
TiddlyWiki's file-saving mechanism (saveChanges and externalizeTiddler)
while TiddlyWebWiki's ServerSideSavingPlugin operates on individual
tiddlers.
My initial investigation just now didn't lead anywhere. Perhaps Lyall
can chime in here? In theory, it shouldn't be hard to make this work,
but there might be subtle conceptual issues I'm not aware of.
-- F.
Poojan Wagh
constraint and just had server-side, I could just go in the
serializations/ directory and create my own serialization, right? (I'd
have to reference it from the serializer.py module, but I could
essentially copy text.py (from serializations/) to encrypted.py. Is it
as simple as changing as_recipe, recipe_as, as_tiddler, tiddler_as to
decrypt/encrypt the plaintext?
If so, the only remaining problem would be what to use as a key (which
I don't want saved on the server). However, in that case, I could use
the user's password (or a hash thereof). In that case, how would I
pass the key/password to the serializer?
FND
> server-side, I could just go in the serializations/ directory and
> create my own serialization, right?
Well, you'd also want to make the text store (tiddlyweb/stores/text.py)
use that serializer - you could do this by sub-classing it, as
demonstrated by the (untested) code in the attached sample plugin.
> Is it as simple as changing as_recipe, recipe_as, as_tiddler,
> tiddler_as to decrypt/encrypt the plaintext?
That should do it. However, I can't imagine recipes containing any
sensitive info?
> If so, the only remaining problem would be what to use as a key (which
> I don't want saved on the server). However, in that case, I could use
> the user's password (or a hash thereof). In that case, how would I
> pass the key/password to the serializer?
That could probably be done by some simple WSGI middleware inspecting
the HTTP headers and storing it in the environment. I reckon Chris will
chime in here - for the moment, just assume there's something like
environ['cypher'] available to the store methods (via self.environ).
While I think this is definitely a worthwhile endeavor, I kinda feel
like client-side en-/decryption would be the better solution. Perhaps
I'll find some time to look into this next month - can't promise this
though.
-- F.
Poojan Wagh
> < 1KViewDownload
Thanks, FND. I see your point about client-side encryption: If it's
worth encrypting, it's probably better done before it hits the server.
In my case, I'm more concerned with people stumbling across sensitive
data stored on disk, not so much in-memory. However, people who know
about encryption and security usually don't like plaintext in memory
on a server.
Question about client-side encryption: if this is implemented, does it
mean the search functions go away? It seems like the adaptor has
functions that run a search query on the server.
FND
> mean the search functions go away? It seems like the adaptor has
> functions that run a search query on the server.
Well, since the client would only send encrypted data*, the server would
not be able to perform a full-text search on those tiddlers.
However, while supported by the adaptor, that server-side search
capability is currently not being used by TiddlyWebWiki, relying on
TiddlyWiki's default (local, client-side) search instead.
-- F.
* presumably metadata like tags should remain unencrypted though
FND
Somehow this tickled my fancy, so (annoyingly) I felt compelled to
create a proof of concept:
http://groups.google.com/group/tiddlywikidev/browse_thread/thread/82ca070af96353e0
Hopefully that's gonna bring us closer to a satisfying solution.
-- F.
Poojan Wagh
> > Perhaps I'll find some time to look into this next month
>
> Somehow this tickled my fancy, so (annoyingly) I felt compelled to
> Hopefully that's gonna bring us closer to a satisfying solution.
>
> -- F.
to incorporate your plugin into the TiddlyWebWiki save/load functions.
That way, I can ensure that all my TiddlyWeb tiddlers are encrypted.
Thanks again,
--
Poojan
FND
> to incorporate your plugin into the TiddlyWebWiki save/load functions.
That should not be necessary, as the ServerSideSavingPlugin ensures that
the TiddlyWiki API calls trigger a sync with the server.
> That way, I can ensure that all my TiddlyWeb tiddlers are encrypted.
It might be worth watching the HTTP traffic (e.g. via Firebug, or the
server logs) to ensure there are no bugs.
-- F.
Poojan Wagh
My intent is to hack up the TiddlyWebAdaptor plugin to enable seamless
encryption. That is, the getTiddler and putTiddler methods
automatically decrypt/encrypt. This way, I can be assured that data--
by which I mean the text body of the tiddler, not tags and titles--
sent to the server is encrypted. In addition, there's nothing more for
the user to do (other than defining a key).
To test this out in small steps, I tried to modify the adaptor to
prepend 'ZZ' instead of encrypting, and to strip off the 'ZZ' instead
of decrypting. This way, I can make sure I'm modifying the correct
methods.
The prepending works. I can see the store on the server has the
prepended 'ZZ'. However, when I reload the tiddler, the 'ZZ' is not
stripped. I modified the adaptor in the following way:
getTiddlerListCallback: changed to
tiddler.assign(t.title, t.text.slice(2), t.modifier, t.modified,
t.tags, t.created, t.fields);
getTiddlerCallback: changed to:
context.tiddler.assign(context.tiddler.title, t.text.slice(2),
t.modifier,
Date.convertFromYYYYMMDDHHMM(t.modified), t.tags || [],
Date.convertFromYYYYMMDDHHMM(t.created), context.tiddler.fields,
t.creator); // XXX: merge extended fields!?
putTiddler: changed to:
var payload = {
title: tiddler.title,
text: "ZZ" + tiddler.text,
modifier: tiddler.modifier,
tags: tiddler.tags,
fields: $.extend({}, tiddler.fields)
};
Curiously, when I create a tiddler, I can see the 'ZZ' pre-pended on
the server. If I immediately re-edit the same tiddler, I don't see the
'ZZ' pre-pended (which is the desired outcome). If I reload the page,
the same tiddler does show a 'ZZ' pre-pended to the text body. Am I
missing some fundamental method that pulls tiddlers from the server?
Any help would be much-appreciated.
FND
> encryption. That is, the getTiddler and putTiddler methods
> automatically decrypt/encrypt.
I see - so you're making it part of the interface, which is an
interesting and worthwhile approach.
Of course this means non-TiddlyWeb scenarios are not taken into account
(e.g. downloading a stand-alone TiddlyWiki) - but I realize that's a
conscious choice.
> Curiously, when I create a tiddler, I can see the 'ZZ' pre-pended on
> the server. If I immediately re-edit the same tiddler, I don't see the
> 'ZZ' pre-pended (which is the desired outcome). If I reload the page,
> the same tiddler does show a 'ZZ' pre-pended to the text body. Am I
> missing some fundamental method that pulls tiddlers from the server?
This is because the server-side serialization is not aware of the prefix.
Currently, tiddlers are generally not loaded on demand in TiddlyWebWiki,
so the adaptor is not invoked on startup. That is, when you load the
TiddlyWiki serialization of a collection of tiddlers (e.g.
/recipes/default/tiddlers.wiki), the server assembles a TiddlyWiki
document which includes those tiddlers and sends it over the wire.
-- F.
Poojan Wagh
> This is because the server-side serialization is not aware of the prefix.
> Currently, tiddlers are generally not loaded on demand in TiddlyWebWiki,
> so the adaptor is not invoked on startup. That is, when you load the
> TiddlyWiki serialization of a collection of tiddlers (e.g.
> /recipes/default/tiddlers.wiki), the server assembles a TiddlyWiki
> document which includes those tiddlers and sends it over the wire.
would happen:
(Take, for example a tiddler with the text 'AAAA')
text: 'AAAA' (entered by user) --> putTiddler (prefixes 'ZZ') ==>
'ZZAAAA' (sent to server)
text: 'AAAA' (displayed to user) <== getTiddler (removes 'ZZ') <--
'ZZAAAA' (fetched from server)
From this construction, the server doesn't need to know about the
prefix/suffixing; that's all handle on the client in Javascript. From
my experiments, however, this intended behavior isn't happening. Is it
even possible?
Thanks again for your help,
--
Poojan
Poojan Wagh
isn't really an asynchronous store of the data. When I reload the
page, the entire contents of all tiddlers get shipped as part of the
TiddlyWiki html file. So, I can't perform the translation in the
process. I suppose I could hijack the edit event or something like
that, but it still seems like this isn't the the right approach.
FND
> Perhaps what I want isn't possible. This is what I thought would happen
> [user input --data-mangling-> PUT; GET --data-mangling-> user display]
>>
>> From this construction, the server doesn't need to know about the
>> prefix/suffixing; that's all handle on the client in Javascript.
This is basically correct - however, there's a bit of a special case on
startup (unfortunately, my previous explanation of this was too clumsy),
as the TiddlyWiki document is prepopulated with the tiddlers.
Your scenario pretty much requires a GET for each individual tiddler
(i.e. dynamic / on-demand loading) - which is entirely possible, since
we know which tiddlers need to be handled separately.
Let's assume your plugin adds an "encrypted" tag to the respective
tiddlers. You could exclude those tiddlers on startup with a filter
("select=tag:!encrypted" - either within your recipes or directly on the
URL). Then your plugin could dynamically load only those tiddlers
("select=tag:encrypted"; note the lack of negation) after startup.
Does this make sense? I could provide you with some sample code to
illustrate how to GET the list of tiddlers and subsequently retrieve
each one via the adaptor's getTiddler method.
-- F.