Slices with non-latin names: to plugin or to core?

[Yakov]

! Short intro:

in 2.6.5 names of slices are defined with the

[\.\w]+

part(s) of the TiddlyWiki.prototype.slicesRE [1] (for both |sliceName|
sliceValue| and sliceName: sliceValue syntax). This is set of Latin
letters and dots. However, for some reasons it can be desirable to
have extended names of slices. In my case, I'd like to use Eric's
GridPlugin [2], and much more rarely -- tiddler macros -- with slices
with Cyrillic names (containing spaces), for complex "databases".

For more details see previous discussion [0].

! The idea

Slices are defined as

slices = {};

[3] which is similar to tiddlers [4], and, as I can see from their
usage in functions, they are not interpreted as object properties, so
also can be treated as hashmap (although I'm not quite familiar with
the language mechanisms of this).

Because of this, the idea rised in [0] and tested by me was to change
the slicesRE definition. For fast hack, I tried just

TiddlyWiki.prototype.slicesRE = /(?:^([\'\/]{0,2})~?([\.

\wабвгдеёжзийклмнопрстуфхцчшщъыьэюяАБВГДЕЁЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯ ]
+)\:\1[\t\x20]*([^\n]*)[\t\x20]*$)|(?:^\|([\'\/]{0,2})~?([\.

\wабвгдеёжзийклмнопрстуфхцчшщъыьэюяАБВГДЕЁЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯ ]
+)\:?\4\|[\t\x20]*([^\|\n]*)[\t\x20]*\|$)/gm;

(Russian alphabet and space symbol added). The result is: if I change
this in the core, it works perfectly, if I make a plugin with such
line, it doesn't work directly, as I understand because slices are
built *before* the plugin is launched (this is supported by the fact
that if I edit tiddler with slices, than save it, they are rebuilt for
it, which is confirmed by tiddler macro showing them). This can be
overwhelmed by rebuilding all the slices, but it seems rather bulky.

So why not change the core definition of slicesRE?

! Analysis

Of'course, the


абвгдеёжзийклмнопрстуфхцчшщъыьэюяАБВГДЕЁЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯ

is ugly, language-specific, just not good enough. How exectly slicesRE
can be changed?

First, let's see its structure:

/
(?: // sliceName: sliceValue syntax
^ // beginning of line
([\'\/]{0,2}) // ' or / 2 or 0 times
~? // tilde, perhaps
([\.\w]+) // *slice name*
\:
\1 // closing '' or //
[\t\x20]* // optional leading whitespace
([^\n]*) // *slice content*
[\t\x20]*
$ // eol
)
|(?:
^
\| // "table" version of slices
([\'\/]{0,2})
~?
([\.\w]+) // *slice name*, change stuff here
\:? // optional :
\4 // closing '' or //
\|
[\t\x20]*
([^\|\n]*)
[\t\x20]*
\|
$
)
/gm;

No need to change the definition of slices for sliceName: sliceValue
syntax (or at least space symbols should be forbidden, because
otherwise whole paragraphs with colons will turn into slices). So
let's consider only the second part.

My idea is to substitute the [\.\w] part with another, which would
allow more symbols. To support different alphabets etc, I think the
[^ .... ] construction should be used (meaning any symbol except ....
ones).

Now, what to forbid? Of'course, the pipe "|", since defines the slice
boundary. What else? There are various special symbols with which we
should be careful:

:;*()
'/_^-@
"
%
!~&
[]
()
<>
{}
space symbols

What to forbid? In principle, the only thing which should bother us is
symbols which would shuffle the slices borders (that's why it's
desirable to get rid of "|") or hurt the whole RegExp behavior. So, \n
should also go away (no other space symbols hurt anything, right?).
Right, if sliceName contains ">>", one will have problems with using
the tiddler macro, but is that a reason to forbid ">"? If it is
forbidden, &gt; can be used to avoid that fact, but is this really a
reason to complicate the code.

The only thing that bothers me in this context is that backward
compability requires to keep the

([\'\/]{0,2})
~?
...
\4

part which means that //name// will be treated as "name" slice name,
but __name__ will be treated as "__name__" which is a bit messy.

So, probably

[^\|\n]

instead of second

[\.\w]

! The question

So, first, any comments on this? Did I missed something?
And, second, plugin or core? I'm absolutely for moving this to the
core (as one can notice, this makes it only one symbol bigger, unless
I missed something important).

[0] http://groups.google.com/group/tiddlywiki/browse_thread/thread/662bd56be68ff629
[1] https://github.com/TiddlyWiki/tiddlywiki/blob/master/js/TiddlyWiki.js#L173
[2] http://tiddlytools.com/#GridPlugin
[3] https://github.com/TiddlyWiki/tiddlywiki/blob/master/js/TiddlyWiki.js#L7
[4] https://github.com/TiddlyWiki/tiddlywiki/blob/master/js/TiddlyWiki.js#L14

[Tobias Beer]

+1 on core & a set of instructions on what characters best to avoid.

Great analysis!

Tobias.

> [0]http://groups.google.com/group/tiddlywiki/browse_thread/thread/662bd5...
> [1]https://github.com/TiddlyWiki/tiddlywiki/blob/master/js/TiddlyWiki.js...

> [2]http://tiddlytools.com/#GridPlugin
> [3]https://github.com/TiddlyWiki/tiddlywiki/blob/master/js/TiddlyWiki.js#L7

> [4]https://github.com/TiddlyWiki/tiddlywiki/blob/master/js/TiddlyWiki.js...

[Yakov]

Thanks!

So, let's wait the core devs' comments.