Re: [WikklyText] Backend for NON-flat file store (and TiddlyWeb-ish)

[Frank McIngvale]

On Sun, Sep 27, 2009 at 6:08 AM, Randy <Randy...@data-warp.com> wrote:

Frank

If a backend for non-flat file store was created
(i.e. store the page text files in a directory tree),
what are the issues that should be considered?

For example:

1) Should a new Python module (?term?) be created or
should the existing one be enhanced with an option?
I like the idea of it being an option.

I think making it a separate module would be better -- can keep the flat-files code as-is (don't break compatibility, and keeps things simple). A directory store is sufficiently different than flat files that I think a separate module make sense.
 

2) How should the directory and file names be mangled?
Can mangling be disabled (or made optional)?
Would the Wiki remain portable across platforms if
users limited themselves to portable naming conventions?

You could limit the charset and then throw an exception if the user tries to use a character that would require mangling. Or you could assume a Unix/Linux system and not worry about Windows name quirks, or ... pretty much anything you can imagine. This is (IMO) the best reason to make a separate module -- you can do whatever you want without breaking anything for anyone else not using the store type.
 

Should ".txt" suffixes be implicitly added to Wiki names
to derive the text filename?
Should this be optional?

I'm not sure if this is where you are headed but here is the generic URL scheme right now:

http://www.example.com/wiki/NAME  -- fetch rendered page for item NAME
http://www.example.com/wiki/NAME.html  -- ditto (the above is an alias for this)
http://www.example.com/wiki/api/...  -- plugins live in this namespace

Now, eventually I plan to do a TiddlyWeb-ish thing where:
http://www.example.com/wiki/NAME.txt -- fetch unrendered text for NAME
http://www.example.com/wiki/NAME.xml -- fetch rendered XML for NAME

(These would be optional and default to OFF, BTW, since it would allow a third party to easily crawl and extract your entire site's markup :-)
 

3) What is needed for security?
Should the page directory tree be limited to a root,
i.e. only text files in the .../wiki/pages/... can be served?
This would be similar to the .../wiki/files/... convention.

Should the root name be configurable?
I don't see much need for this, since .../wiki/pages/
could be a symbolic link to wherever.

I think the best/cleanest thing would be to have the store recognize a name like 'pages/' as the root so it would serve:
http://www.example.com/wiki/pages/NAME

That way you don't inadvertantly conflict with a file located in the wiki root. But it could just as well serve from '/wiki' like the other stores.
 

4) Should Wiki paths be relative to the root or
the page containing the link or both?

For example, paths that begin with "/" could use
.../wiki/ as the "root", while other paths are
relative to the page's directory, i.e. pages within
the same directory could refer to each other without
any path prefix, while foo_section/foo.txt walks deeper
into the tree, and ../bar.txt would walk out (but
not past the "root").

I think the whole ".." thing gets messy quickly -- remember that wherever you make the root, the scheme is STILL:

http://site.com/WIKI_ROOT/NAME

To the wiki, its ALWAYS 'NAME' -- if the store wants to interpret '/' chars as a path that's fine, but the wiki doesn't know/care. The store could in theory look at HTTP_REFERRER in order to interpret '..', but none of the stores currently does that. Also, I know the client browser will sometimes try and interpret ".." so the server may never see it. Anyways ... I think this gets messy quick for questionable benefit.

 

5) Could the Wiki name for the page default to the
text file name ?  and visa versa ?

So NAME would be an alias for NAME.txt instead of NAME.html? That's possible to make configurable -- it would of course break existing links to NAME but if its your wiki I'd assume that's OK with you :-) Likewise, NAME could be an alias for NAME.xml. I'll put that in the TODO. The default would of course remain .html
 

How would this interact with relative path names?
Perhaps all relative path names should require
at least a "./" path prefix to distinguish them from
simple Wiki page names.  Example:

"FooBar" (AKA "/FooBar") is a simple Wiki page name in the
".../wiki/" directory.

"/pages/FooBar" refers to the FooBar text file in the
directory for Wiki pages, i.e. .../wiki/pages/FooBar

"./FooBar" refers to the FooBar text file in the
same directory.

"../FooBar" refers to the FooBar text file in the
parent directory.

Like above, the ".." gets messy quickly and I think (??) the client browser is going to try and interpret that before the server ever sees it.

 

"sect/FooBar" refers to the FooBar text file in the
sect subdirectory.

6) Should the Wiki engine manage revisions?
Implicitly ?  Explicitly?

Eventually yes :-) I picture a mediawiki-type scheme where revs are made automatically every time you save a page. Implementation TBD :-)

frank

- RLS

[Randy]

So much to learn, so little time.
I really appreciate your reply, clear and detailed.
It will take some time to digest at my end.

On Sep 27, 5:16 pm, Frank McIngvale <fmcingv...@gmail.com> wrote:

> http://www.example.com/wiki/NAME -- fetch rendered page for item NAMEhttp://www.example.com/wiki/NAME.html -- ditto (the above is an alias for
> this)http://www.example.com/wiki/api/...  -- plugins live in this namespace
>
> Now, eventually I plan to do a TiddlyWeb-ish thing where:http://www.example.com/wiki/NAME.txt-- fetch unrendered text for NAMEhttp://www.example.com/wiki/NAME.xml-- fetch rendered XML for NAME