Houston, we have a problem with gnx's

[Edward K. Ream]

Terry congratulated me on fixing #76 erroneous clone markers in @auto trees. The celebrations are premature.

It's easy to see at least one flaw in the present gnx allocation code: the post-pass calculates ni.lastIndex after reading all external files **including @auto nodes**.

But suppose the @auto node contains a new node (a node created outside of Leo).  The @auto read code will *immediately* allocate a gnx for the new node.  This happens before the post-pass runs, so the newly allocated gnx could clash with a previously allocated gnx!

Yes, this can only happen if the present invocation of Leo clashes with a previous invocation of Leo.  (Two invocations of Leo **clash** if both are based on the same timestamp.) We all now agree that clashes *do* happen.  The post-pass was supposed to be a workaround. But we shall soon see that it isn't.

My first thought was to delay the allocation of new gnx's until after the postpass.  Like this:

1. ni.getNewIndex would allocate None as the gnx if it is called before the post-pass. In that case, getNewIndex would add the vnode with the dummy gnx to a global list.

2. After the post-pass, all nodes on this list would get a proper, safe, gnx based on the updated ni.lastIndex.

But this is all delusion!  Consider these itsy-bitsy teeny-weeny questions:

1. How many .leo files did the previous clashing invocation of Leo open?

2. When is *this* invocation going to open all the files opened in the *other* invocation of Leo?

Oops! Leo can only run the post-pass after we open *all* the .leo files opened by the "clashing" invocation of Leo.  But Leo has no idea what those .leo files were!  As a result, Leo can *never* be sure a .leo file that is opened *after* the post-pass won't contain gnx's that clash with newly-allocated gnx's in *this* invocation of Leo.

This problem has no solution: it is always too early to run the post-pass! Leo must create unique gnx's without using a post-pass.

Terry's Aussie brother Tony reminds us that *no* timestamp format, no matter how fine-grained, can guarantee that timestamps will *never* collide. I doubt that SegundoBob's timestamp-adjusting hack will suffice either in general.

It seems that the *only* solution is to use uuid's for all newly-allocated gnxs.  This would eliminate the harmful effects of clashing invocations. Indeed, the notion of clashing invocations disappears if Leo doesn't base gnx's on timestamps.

Your comments please.

Edward

[Terry Brown]

On Tue, 21 Oct 2014 17:29:08 -0700 (PDT)
"Edward K. Ream" <edre...@gmail.com> wrote:

> My first thought was to delay the allocation of new gnx's until after
> the postpass. Like this:
>
> 1. ni.getNewIndex would allocate None as the gnx if it is called
> before the post-pass. In that case, getNewIndex would add the vnode
> with the dummy gnx to a global list.
>
> 2. After the post-pass, all nodes on this list would get a proper,
> safe, gnx based on the updated ni.lastIndex.
>
> But this is all delusion! Consider these itsy-bitsy teeny-weeny
> questions:
>
> 1. How many .leo files did the previous clashing invocation of Leo
> open?
>
> 2. When is *this* invocation going to open all the files opened in
> the *other* invocation of Leo?
>
> Oops! Leo can only run the post-pass after we open *all* the .leo
> files opened by the "clashing" invocation of Leo. But Leo has no
> idea what those .leo files were! As a result, Leo can *never* be
> sure a .leo file that is opened *after* the post-pass won't contain
> gnx's that clash with newly-allocated gnx's in *this* invocation of
> Leo.

I don't understand the above - I don't see how there's more
than one invocation involved. I certainly thought of the possibility
of Leo needing to allocate gnxs before all the @auto nodes were loaded,
if I didn't mention it, and figured that something like the solution
you mention would work - so I'm not seeing why it doesn't?

Cheers -Terry

[Kent Tenney]

I must admit I'm not using new-auto, embarrassing after all my fussing.

- the @persistence tree got huge, cluttered versioning
- turned out my back end storage needed to implement it's own keys for
@auto nodes
- if I have @auto myfile.py in multiple Leo files, they need the same keys

The back end db uses gnx for native nodes, concats the headlines for ephemerals:
like <host.file.class.method>

Sorry, presumably others are benefiting from all the work that went
into @persistence.

Regarding gnx/uuid, I'm an outlier in never seeing sentinels, so I
should be quiet.

> --
> You received this message because you are subscribed to the Google Groups
> "leo-editor" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to leo-editor+...@googlegroups.com.
> To post to this group, send email to leo-e...@googlegroups.com.
> Visit this group at http://groups.google.com/group/leo-editor.
> For more options, visit https://groups.google.com/d/optout.

[Edward K. Ream]

On Tue, Oct 21, 2014 at 8:27 PM, 'Terry Brown' via leo-editor

> I don't understand the above - I don't see how there's more
> than one invocation involved. I certainly thought of the possibility
> of Leo needing to allocate gnxs before all the @auto nodes were loaded,
> if I didn't mention it, and figured that something like the solution
> you mention would work - so I'm not seeing why it doesn't?

Good question. I'll discuss it below. To avoid tl;dr (too long;
didn't read) I'll state my conclusions first...

===== Executive summary

1. The post-pass must go.

2. Aside from Bob's automated test cases, the probability of real
problems arising from clashing gnx's is negligible.

3. A new unique_timestamps plugin will repackage Bob's timestamp code.
Bob will have to enable this plugin when running his unit tests.

4. Your comments are essential. I won't do anything until I've heard from you.

===== The answer to Terry's question

Delayed allocation of gnx's would work as I described, but it's only a
partial fix.

The fundamental problem is that two different invocations of Leo can
clash, which can lead to two *different* vnodes (different purposes)
getting assigned the same gnx. Like this:

Invocation 1 opens two files, f1.leo and f2.leo and allocates v1 in
f1.leo and v2 in f2.leo.

Invocation 2 clashes with invocation 1: the timestamps of both
invocations are the same. Invocation 2 opens f1.leo only and
allocates v3 with a gnx that clashes with v2 (because the scan doesn't
include f2.leo).

The clash between v3 in f1.leo and v2 in f2.leo could cause problems
if a later invocation opens both f1.leo and f2.leo. Pasting v2 with
paste-retaining-clones from f2 to f1 will cause data loss. Leo will
consider that v2 and v3 are clones, so the contents of one of them
will be lost.

Yes, this will happen rarely, but it was supposed *never* to happen.

The post script discusses two solutions. But first, let's step back
and consider the big picture...

===== How serious are gnx clashes?

Remember that this whole issue arose because Bob *automatically*
invoked Leo twice in such a way that two distinct invocation of Leo
started with the same timestamp. This happens about once every four
invocations, in my tests.

Without the automatic invocation of Leo, problems can occur only in
the following *extremely rare* scenario:

- Two different people share the same user id. This will happen
fairly often, say with probability P1.

- These two people each create separately a node that just happens to
have the same timestamp. This will happen rarely, but it *can*
happen: probability P2.

- These two people share data by sharing a .leo file that contains one
of the nodes: probability P3.

- The recipient does a past-retaining-clones a part of the shared .leo
file that contains the clashing vnodes. This is extremely unlikely,
but it *could* happen: probability P4.

The probability of data loss is P1*P2*P3*P4. The product of small
probabilities is extremely small. Imo, we can ignore this scenario
for now.

===== Summary

Aside from Bob's automated collision process (TM), the probability of
clashing gnx's is negligible. There is no urgent need to use uuids
now.

We must solve Bob's problem, but the present "solution" is not good.
The post-pass is a ticking time bomb because it's implications are so
subtle. I'm confused, and every other Leo developer will be too.
Hacking on ni.getNewIndex would be a band-aid. Imo, the post pass
must go.

If comments are favorable, I'll write a unique_timestamps plugin. This
plugin will repackage Bob's original timestamp adjustment. See the
post script for details.

Yes, Bob will have to enable this plugin when running his automated unit tests.

Your comments please, Amigos. I won't do anything until I've heard from you.

Edward

P.S. This post script discusses two solutions to the present gnx difficulties.

===== Solution 1. Use uuid's for all new gnxs.

This may not be as odious as I first thought. At present, node
sentinels have the form::

#@+node:<gnx>:<headline>

Instead, they could have the form::

#@+node::<headline>:<gnx>

The second form is better because the headline is first, which is what
humans want to see. The opening :: distinguishes the second form from
the first. The trailing gnx will be scanned *backward*, starting from
the end of the line. This ensures that headlines may contain any text
whatever.

Lengthening sentinel comments is not pleasant, but using uuid's in
sentinels appears to be inevitable at some point.

===== Solution 2: Create a plugin that ensures unique timestamps.

This plugin would repackage Bob's timestamp hack. Unless I am
severely mistaken, ensuring that all invocations of Leo are based on
separate timestamps will solve all the present gnx problems.

I like this solution:

1. It requires no nerdy command-line option.
2. Only those who have Bob's particular use-case need activate the plugin.
3. It simplifies Leo's code and *much more importantly* simplifies the
underlying assumptions behind Leo's code.

Otoh, this solution resuscitates but #35

Happily, I still have the tests lying around. I spent enough time with
those tests that I should be able to tell whether they fail or not.
It's a statistical thing whether the two invocations actually share
the same timestamp: one must run the test until they do.

EKR

[Edward K. Ream]

On Wed, Oct 22, 2014 at 7:03 AM, Kent Tenney <kte...@gmail.com> wrote:
> I must admit I'm not using new-auto, embarrassing after all my fussing.

I forget: how did you disable it?

> - the @persistence tree got huge, cluttered versioning

Interesting.

> The back end db uses gnx for native nodes, concats the headlines for ephemerals

> Presumably others are benefiting from all the work that went
> into @persistence.

Hard to know for sure. I think yesterday's work makes the code more
solid. It's certainly simpler.

> Regarding gnx/uuid, I'm an outlier in never seeing sentinels, so I should be quiet.

As my previous reply to Terry indicates, it appears that uuid's are
not needed at present. But all my conclusions are conditional on
further comments from Terry, Bob et. al. I'm always eager to hear
your opinions :-)

Edward

[Kent Tenney]

in leoAtFile.py
new_auto = False

[Terry Brown]

TL;DR: re-instate Bob's timestamp fix in core, doesn't make sense to be
an off-by-default plug-in. UUID gnxs: wishlist priority, current time
vs. session start time timestamps would be nice though.

On Wed, 22 Oct 2014 08:46:32 -0500

"Edward K. Ream" <edre...@gmail.com> wrote:

[snip]

> The fundamental problem is that two different invocations of Leo can
> clash, which can lead to two *different* vnodes (different purposes)
> getting assigned the same gnx. Like this:

Ok, so your different invocations just refers to the fact that the
current gnx is not guaranteed to be universally unique, and in the
extremely rare case where it clashes *and* someone uses paste retaining
clones, stuff breaks. I think Leo should be around for a long time to
come, but I'm not sure it will be around for the length of time it will
take that to happen in the wild :-) Although P1, same user id, could
be higher than you think, if say one person's using Leo on machines
running in different time zones... unlikely, but so is P1*P2*P3*P4 :-)

[snip]

> We must solve Bob's problem, but the present "solution" is not good.
> The post-pass is a ticking time bomb because it's implications are so
> subtle. I'm confused, and every other Leo developer will be too.
> Hacking on ni.getNewIndex would be a band-aid. Imo, the post pass
> must go.

So the post-past adds complexity and fails in the P1*P2*P3*P4 (paste
retaining clones) case. Also, Bob's always increasing timestamp
solution is a sufficient remedy for Bob's twice in one second problem.
I'm trying to mentally weight the cost benefit of removing the
post-past. I guess it fails to solve a problem we'll probably never
see, and is not needed for the problem that has been encountered, if we
re-instate the increasing timestamp fix. I can't see a strong argument
for not removing it.

> If comments are favorable, I'll write a unique_timestamps plugin. This
> plugin will repackage Bob's original timestamp adjustment. See the
> post script for details.
>
> Yes, Bob will have to enable this plugin when running his automated
> unit tests.

How is having to enable a plugin better than having to use a
'--no-duplicate-gnx' command line parameter? I think Bob's increasing
timestamp fix should just be re-instated in core, I can't see how it
has any negative impact on anything.

> #@+node:<gnx>:<headline>
>
> Instead, they could have the form::
>
> #@+node::<headline>:<gnx>

I think this problem is unimportant, because I think that the
increasing timestamp is the smart way to fix the collisions that are
actually likely, that I'm not convinced uuid in gnxs need higher than
'wishlist' priority. The above seems reasonable if they are being
implemented. Also a get_gnx_timestamp() method to get the timestamp
from old user.timestamp.seq and new gnxs would be good.

Also take the opportunity to switch to a current time, rather than
session start time, timestamp, that would be nice.

I'm still not sure exposing people's network addresses in Leo files is
ideal
https://groups.google.com/d/msg/leo-editor/7HNA2mOqpJU/goulrkknvgwJ
but UUID version 1 is the only UUID that records timestamps and it also
records mac addresses. Of course, you can just give it a bogus MAC
address.

So... ha, funny thing, UUIDs... not necessarily unique?
http://en.wikipedia.org/wiki/Universally_unique_identifier#Version_1_.28MAC_address_.26_date-time.29

That wikipedia page says Python's implementation follows
http://tools.ietf.org/html/rfc4122 ... I assume that means it's (a)
fast and efficient, and (b) guaranteed to be unique on a particular
machine.

I'm not sure what impact a bogus MAC address would have on the
uniqueness guarantee, if Leo used say 0x123456789abc always, then it
would be degrading the uniqueness perhaps, but we could still avoid
exposing the real MAC by using g.app.db['fakemac'], which is set to
random.randint(0,0xffffffffffff) if it doesn't exist.

Cheers -Terry

[Terry Brown]

On Wed, 22 Oct 2014 10:12:19 -0500
"'Terry Brown' via leo-editor" <leo-e...@googlegroups.com> wrote:

> I'm still not sure exposing people's network addresses in Leo files is
> ideal

Just seen: http://en.wikipedia.org/wiki/MAC_address#Spying

Cheers -Terry

[Edward K. Ream]

On Wed, Oct 22, 2014 at 10:12 AM, 'Terry Brown' via leo-editor
<leo-e...@googlegroups.com> wrote:
>
> TL;DR: re-instate Bob's timestamp fix in core, doesn't make sense to be
> an off-by-default plug-in. UUID gnxs: wishlist priority, current time
> vs. session start time timestamps would be nice though.

Thanks for this summary and all your thoughtful comments.

Upon further reflection on my part, we are in almost total agreement.

> Ok, so your different invocations just refers to the fact that the current gnx is not guaranteed to be universally unique, and in the extremely rare case where it clashes *and* someone uses paste retaining clones, stuff breaks.

Correct.

> I think Leo should be around for a long time to come, but I'm not sure it will be around for the length of time it will take that to happen in the wild :-)

I agree. This fact makes Leo's job much easier. The singleton
NodeIndices object, g.app.nodeIndices, can delay allocating gnx's
until after each .leo file has been read.

Furthermore, as you suggested several days ago, a bit more cleverness
in NodeIndices class can eliminate the need for the post pass itself.
At the end of the load process (for each .leo file) we just allocate
the not-yet-allocated gnx's.

This is simple and safe, with no downsides.

> How is having to enable a plugin better than having to use a
> '--no-duplicate-gnx' command line parameter?

It's not. The unique_timestamps plugins was a wretched idea. It
implicitly dissed Bob's work flow. This is a recipe for further
problems for anyone who uses a similar work flow. Instead, everything
should *just work* for everybody.

> I think Bob's increasing timestamp fix should just be re-instated in core, I can't see how it has any negative impact on anything.

Happily, this is not needed now that we know that P1*...*P4 is so small.

> I think [the uuid representation in sentinels] problem is unimportant

I agree.

> Also take the opportunity to switch to a current time, rather than
> session start time, timestamp, that would be nice.

It's on the list. It fits in well with coming changes to the NodeIndices class.

This change might mask Bob's test, so I'll make this change only after
the delayed gnx allocation has passed Bob's tests.

> I'm still not sure exposing people's network addresses in Leo files is

> ideal...So... ha, funny thing, UUIDs... not necessarily unique?

Excellent. Let's forget all about uuid's :-)

===== Summary

Leo can ignore the possibility of global gnx clashes.

Delayed allocation of gnx's will replace the post pass.

A unique_timestamps plugin would cause problems anyone who uses Leo as
Bob does. There is nothing wrong with Bob's work flow!

uuid's are not need now, and probably not ever. There will be no
--uuid command line option and no change to Leo's sentinel lines.

There is still plenty of time for comments, but I think it is safe to
go ahead with the improved gnx allocation scheme discussed here.

Edward

[Edward K. Ream]

On Wednesday, October 22, 2014 10:24:11 AM UTC-5, Terry Brown wrote:

Just seen: http://en.wikipedia.org/wiki/MAC_address#Spying

Just retweeted it.

EKR

[SegundoBob]

On Wednesday, October 22, 2014 9:16:55 AM UTC-7, Edward K. Ream wrote:

On Wed, Oct 22, 2014 at 10:12 AM, 'Terry Brown' via leo-editor
<leo-e...@googlegroups.com> wrote:
>

> Also take the opportunity to switch to a current time, rather than
> session start time, timestamp, that would be nice.

It's on the list.  It fits in well with coming changes to the NodeIndices class.

I see the appeal of "current time," but making this switch almost certainly requires losing the benefit of minimizing the node index by only considering GNX's with the "session start time."  The change of time zone or daylight saving could cause times after the session start time to equal times in already existing GNX's.

Or do you want to do a complete scan to determine the minimum unused node index before every node creation that changes the time stamp?
---------
New Topic:

Early in this thread, you seemed concerned, once again, about one GNX being used in two different files.  This should not concern you.  It doesn't bother Leo-Editor at all.  It does happen all the time.  For example, I just copied a Leo-Editor file and then opened both copies in one Leo-Editor session.  Leo was happy.  I was happy.  When I copied a node from copy1 and pasted it into copy2 "retaining clones", it became a clone in copy2.  I was happy.  This is what I would expect.

For your benefit and the benefit of Leo-Editor users, you need to make this clearer in the Leo-Editor documentation.
 

[Edward K. Ream]

On Wed, Oct 22, 2014 at 5:51 PM, SegundoBob <segun...@gmail.com> wrote:

> I see the appeal of "current time," but making this switch almost certainly
> requires losing the benefit of minimizing the node index by only considering
> GNX's with the "session start time."

Having the gnx reflect the actual creation time of the node should
have no effect on the size of gnxs, except to make them slightly
smaller, because ni.lastIndex would be zero for most *new* gnxs.

> The change of time zone or daylight saving could cause times after the session start time to equal times in already existing GNX's.

Good point.

> Or do you want to do a complete scan to determine the minimum unused node index before every node creation that changes the time stamp?

We aren't going to do that. I'm happy sticking with a session timestamp.

> New Topic:
>
> Early in this thread, you seemed concerned, once again, about one GNX being used in two different files. This should not concern you.

It doesn't concern me *provided* that the id parts of two gnxs are
equal if and *only* if the same person actually created the two nodes!

This is true in the example you give, but it need not be true. In
that case, the clashing gnx's *are* going to cause data loss if the
paste-retaining-clones command is used. Happily, the case is very
rare, as shown by the P1*P2*P3*P4 calculation.

> For your benefit and the benefit of Leo-Editor users, you need to make this
> clearer in the Leo-Editor documentation.

Yup.

Edward

[Terry Brown]

On Wed, 22 Oct 2014 15:51:28 -0700 (PDT)
SegundoBob <segun...@gmail.com> wrote:

>
>
> On Wednesday, October 22, 2014 9:16:55 AM UTC-7, Edward K. Ream wrote:
> >
> > On Wed, Oct 22, 2014 at 10:12 AM, 'Terry Brown' via leo-editor
> > <leo-e...@googlegroups.com <javascript:>> wrote:
> > >
> >
> > > Also take the opportunity to switch to a current time, rather
> > > than session start time, timestamp, that would be nice.
> >
> > It's on the list. It fits in well with coming changes to the
> > NodeIndices class.
> >
> >
> I see the appeal of "current time," but making this switch almost
> certainly requires losing the benefit of minimizing the node index by
> only considering GNX's with the "session start time." The change of
> time zone or daylight saving could cause times after the session
> start time to equal times in already existing GNX's.

The current time would come if you were using uuid.uuid1(), but
ironically version 1 UUIDs a aren't really guaranteed unique. They
have a timestamp with a specified unit of "100 nanosecond intervals
since 00:00:00.00, 15 October 1582". Yes, 1582, the start of the
Gregorian calendar, not 1970, the start of the unix epoch. Plus they
have 14 random bits, but 2**14 is only 16384, so although the chance of
collision would be minuscule, not as minuscule as the content free
version 4 uuid, with about 120 random bits, or 2**120 =
1329227995784915872903807060280344576 values.

> Or do you want to do a complete scan to determine the minimum unused
> node index before every node creation that changes the time stamp?

No, I'm sure that would be too slow.

> ---------
> New Topic:
>
> Early in this thread, you seemed concerned, once again, about one GNX
> being used in two different files. This should not concern you. It
> doesn't bother Leo-Editor at all. It does happen all the time. For
> example, I just copied a Leo-Editor file and then opened both copies
> in one Leo-Editor session. Leo was happy. I was happy. When I
> copied a node from copy1 and pasted it into copy2 "retaining clones",
> it became a clone in copy2. I was happy. This is what I would
> expect.

The concern is for the extremely uncommon cases where what you describe
happens unexpectedly. Sure that doesn't trouble Leo at all, but if the
body contents differ, well, one of them will be lost.

To be honest I don't know what the intended behavior / use case for
paste-retaining-clone is. I'd assume that pasting a subtree
containing, internal to itself, clones, from outline A to outline B,
would lose the clones with plain paste, and keep them with
paste-retaining. But they're kept in either case.

Of course, given the three alternatives to clones Leo has,
I /personally/ would be fine with clone free Leo, but I know they're
Leo's killer feature for a lot of people.

> For your benefit and the benefit of Leo-Editor users, you need to
> make this clearer in the Leo-Editor documentation.

http://leoeditor.com/commands.html?highlight=retaining#cutting-pasting-and-deleting-nodes

The distinction between the top level node in the copied subtree and
its children seems confusing, but otherwise I'm not sure if the docs.
are complete / sufficient or not.

Cheers -Terry

[Terry Brown]

On Wed, 22 Oct 2014 18:34:45 -0500

"Edward K. Ream" <edre...@gmail.com> wrote:

> On Wed, Oct 22, 2014 at 5:51 PM, SegundoBob <segun...@gmail.com>
> wrote:

So - does the fact that Edward and my responses to Bob "clashed"
time wise (i.e. were made in parallel not sequentially) illustrate the
problem :-) I guess not :)

Cheers -Terry

[Matt Wilkie]

On Wed, Oct 22, 2014 at 4:35 PM, 'Terry Brown' via leo-editor <leo-e...@googlegroups.com> wrote:

Sure that doesn't trouble Leo at all, but if the
body contents differ, well, one of them will be lost.

if gnx == md5 sum or similar, there is no issue and no data loss possible. Identical gnx means identical content, period. No idea if that's computationally *affordable*, certainly not pretty for someone looking at the gnx and hoping to glean human sensical info out of it, but just thought that I'd throw that out there. The data de-duplication folks might have some interesting thoughts and algorithms on that.

-matt

[Matt Wilkie]

On Wed, Oct 22, 2014 at 7:44 PM, Matt Wilkie <map...@gmail.com> wrote:

Identical gnx means identical content

... belatedly realized this would mean a gnx changes with every single character entered, so, no, not useful. Sorry for the noise.   (I do like the idea of a program automatically creating clones, or more likely popping up a side bar of nodes whose content is >85% similar, or something though...)

-matt

[Edward K. Ream]

On Wed, Oct 22, 2014 at 9:44 PM, Matt Wilkie <map...@gmail.com> wrote:
>
> if gnx == md5 sum or similar, there is no issue and no data loss possible.
> Identical gnx means identical content, period.

We aren't interested in content, but identity. The gnx is supposed to
be unique and immutable.

Edward

[Todd Mars]

I'm sorry to jump in like this after long study of the problem by others, not me.

Why not just append a big enough range random number to the end of the timestamp, (a different field) so that the gnx starts off with a unique value, the random part of big enough range that probability of clash is close enough to 0?

[Edward K. Ream]

On Thu, Oct 23, 2014 at 12:59 PM, Todd Mars <tam...@gmail.com> wrote:

> Why not just append a big enough range random number to the end of the

> timestamp...

Because gnx's appear in external files and people already think they
are visually intrusive.

EKR

[Fidel N]

Couldn't two gnx be stored?

- a short one visually acceptable for inside the documents that the external user will see, just with a refference to the actual gnx it refers to. Maybe just an index of the place that the real gnx has within the Leo document.

- A second one, embedded in the Leo file, stored in a "list of gnx's", in a certain order that would be refered from the simple one. Just the position of the gnx's lists would need to be stored within the first one.

Well just a thought, maybe that would make things more complex.. But I liked the idea of extending the gnx's with big random number too.

[Edward K. Ream]

On Thu, Oct 23, 2014 at 3:59 PM, Fidel N <fidel...@gmail.com> wrote:

> Couldn't two gnx be stored?
> - a short one visually acceptable for inside the documents that the external

> user will see, just with a reference to the actual gnx it refers to. Maybe

> just an index of the place that the real gnx has within the Leo document.
> - A second one, embedded in the Leo file, stored in a "list of gnx's", in a

> certain order that would be referred from the simple one. Just the position

> of the gnx's lists would need to be stored within the first one.

I considered such a scheme many years ago. Not only is it more
complex, it's not robust enough. Sentinel comments must be
self-contained.

Edward

[Todd Mars]

OK great:

The fact that GNX is considered visually intrusive is a separate issue.

There should be a design to mitigate that.

My diagnosis: you are confusing two diseases as having the same cause.

1- There are bad GNX (functionality). If GNX is fixed it makes the visual worse.

2- There are visual problems with displaying them in external files (perhaps other problems related to this?)

So fix (2) and then (1) will be OK.

Todd.

So to fix (2), I would add something to their output format to distinguish 'new' from 'old'.

 If it is 'new', then I would display it differently in a nicer way (if possible, for example: perhaps an index to a table of real GNX at the end of the file).

If it is old then it is old.  So a way is needed to uniquely show 'new' that 'old' cannot 'masquerade' as (be confused as)

Todd.

[Edward K. Ream]

On Fri, Oct 24, 2014 at 3:00 PM, Todd Mars <tam...@gmail.com> wrote:

> The fact that GNX is considered visually intrusive is a separate issue.

No, it's not. gnx's are visible in sentinel comments, and they must
be stored internally in exactly the same form.

> My diagnosis: you are confusing two diseases as having the same cause.

I am frequently confused, but not about this.

I am going to change neither the format gnxs nor their placement in
external files.

The changes you suggest would cause compatibility problems and would
make reading external files much less reliable. Don't even think
about trying to change my mind about this.

Edward

[Edward K. Ream]

On Wed, Oct 22, 2014 at 6:35 PM, 'Terry Brown' via leo-editor
<leo-e...@googlegroups.com> wrote:

> Of course, given the three alternatives to clones Leo has, I /personally/ would be fine with clone free Leo, but I know they're Leo's killer feature for a lot of people.

Gnx's would be important even without clones: they are the only way to
create true identity and structure. Without gnx's permanent uA's are
impossible.

Bookmarks do not suffice: they can and do break. @auto is Leo without
gnx's, and that sucks ;-) Note that @persistence is a form of
bookmarking.

In short, gnx's, not clones, are *the* defining feature of Leo. This
became clear to me on today's walk.

Edward