Skip to first unread message
Edward K. Ream
Jul 5, 2014, 7:10:43 AM7/5/14
to leo-e...@googlegroups.com
I'm going to give the spatial versioning thread back to Kent :-) His work is important in its own right.
Here, I'm going to discuss thoughts related to supporting org mode, a file format that indicates outline structure without using gnx's. You could call this an Engineering Notebook post. These are notes to myself. Leo's core developers may be interested, but feel free to ignore this post.
I have given this subject much thought since my lengthy post in the other thread. The key conclusions:
1. This project will not affect existing Leo's position and vnode classes and related generators in *any* way.
2. Leo's core must *officially* support some kind of bookmark. This will be a generalization of Terry's bookmarks, using generalized unl's, xslt patterns, or whatever. Leo's icon box will contain a new visual element, indicating whether the node is bookmarked or not.
3. New commands will support @view nodes. uA's in these nodes will contain the generalized unl's that define the children of the @view node.
4. Unbookmarked (raw) clones **will not be allowed** inside @auto or @org-mode external files, but bookmarks, or **clones of bookmarks**, will be allowed.
5. I won't restart the failed "new @auto" project that I worked on earlier this year in Florida, except to allow bookmarked clones in @auto files.
6. All the big questions (and opportunities!) involve work flow. The big challenge is to make a virtue of necessity.
The implications of point 4 above are huge. Despite my brave words in the other post, decoupling clones from gnx's (immutable, unique identity) changes workflow in many ways. This is a high risk project. Happily, this design work involves no actual code, so the risk is manageable :-)
That's enough for now. A later post will discuss how I personally might live with the ban on unbookmarked (raw) cloned nodes in @auto and @org-mode files. The goal will be find a way for me personally to live with @org-mode files. To do this, I must find a reasonable workflow that allows me to get the *actual* benefits of clones without using raw clones. It could be a big ask, but glimmers of hope abound.
As always, your comments are appreciated.
Edward
P.S. Org mode doesn't have explicit support for sections and section references. We could ban such from @org-mode files, but adding an explicit (appearing in the actual outline) end-of-section comment would work. Like this::
<< my section >>
# <</ my section >>
Yes, it's ugly and error-prone, but I personally could live with it, and the read/write code could enforce the convention. Btw, it's easy to prove that an end-of-section delimiter is required.
EKR
Here, I'm going to discuss thoughts related to supporting org mode, a file format that indicates outline structure without using gnx's. You could call this an Engineering Notebook post. These are notes to myself. Leo's core developers may be interested, but feel free to ignore this post.
I have given this subject much thought since my lengthy post in the other thread. The key conclusions:
1. This project will not affect existing Leo's position and vnode classes and related generators in *any* way.
2. Leo's core must *officially* support some kind of bookmark. This will be a generalization of Terry's bookmarks, using generalized unl's, xslt patterns, or whatever. Leo's icon box will contain a new visual element, indicating whether the node is bookmarked or not.
3. New commands will support @view nodes. uA's in these nodes will contain the generalized unl's that define the children of the @view node.
4. Unbookmarked (raw) clones **will not be allowed** inside @auto or @org-mode external files, but bookmarks, or **clones of bookmarks**, will be allowed.
5. I won't restart the failed "new @auto" project that I worked on earlier this year in Florida, except to allow bookmarked clones in @auto files.
6. All the big questions (and opportunities!) involve work flow. The big challenge is to make a virtue of necessity.
The implications of point 4 above are huge. Despite my brave words in the other post, decoupling clones from gnx's (immutable, unique identity) changes workflow in many ways. This is a high risk project. Happily, this design work involves no actual code, so the risk is manageable :-)
That's enough for now. A later post will discuss how I personally might live with the ban on unbookmarked (raw) cloned nodes in @auto and @org-mode files. The goal will be find a way for me personally to live with @org-mode files. To do this, I must find a reasonable workflow that allows me to get the *actual* benefits of clones without using raw clones. It could be a big ask, but glimmers of hope abound.
As always, your comments are appreciated.
Edward
P.S. Org mode doesn't have explicit support for sections and section references. We could ban such from @org-mode files, but adding an explicit (appearing in the actual outline) end-of-section comment would work. Like this::
<< my section >>
# <</ my section >>
Yes, it's ugly and error-prone, but I personally could live with it, and the read/write code could enforce the convention. Btw, it's easy to prove that an end-of-section delimiter is required.
EKR
Edward K. Ream
Jul 5, 2014, 9:25:37 AM7/5/14
to leo-editor
On Sat, Jul 5, 2014 at 6:10 AM, Edward K. Ream <edre...@gmail.com> wrote:
> Here, I'm going to discuss thoughts related to supporting org mode, a file
> format that indicates outline structure without using gnx's.l
> Here, I'm going to discuss thoughts related to supporting org mode, a file
Terry, I'll be especially interested in your comments to what follows.
> 6. All the big questions (and opportunities!) involve work flow. The big
> challenge is to make a virtue of necessity.
completely unchanged, if not improved.
The trick is this: suppose the clone command automatically creates an
(official) bookmark node instead of a clone node inside @auto and
@org-mode trees.
Bookmarks *are* clones, but they aren't marked as clones unless the
user clones them later. In addition, bookmarks contains uA's, but I'm
getting ahead of myself: see points 3 and 5 below.
That's all there is to it!!!
Well, almost everything ;-) Here are the details:
1. [No duplicates in files]. Neither clones nor bookmarks may appear
more than once in an @auto or @org-mode file. The rationale for this
is that the read code would treat multiple bookmarks or clones as
separate pieces of code. This restriction *might* be lifted, but for
now let's assume it is enforced by the write code.
2. [Optional automatic moves]. Whenever I create a clone, the very
first thing I do is to move it to what is, in effect, an @view node.
Well, why not just do the move automatically?
Suppose a node (which must be outside any @auto or @org-mode tree) is
designated as the **current view node** In that case, cloning or
bookmarking a node will automatically move the new node to be a child
of the current view node. Not only does this save time, but it ensures
that newly-created bookmarks do not become duplicate bookmarks.
If there is no designated view node, creating a bookmark will create a
node following the bookmarked node, just as when creating a clone.
The mark-current-view-node command, or something like it, will
designate a node to which newly bookmarked (or cloned!) nodes will
automatically be moved.
Designated view nodes may not appear in @auto or @org-mode files, but
they *may* appear in @file nodes, provided that the root @file node
contains an @all directive. This will allow view nodes in
leoProjects.txt and leoNotes.txt.
Btw, there will likely be a dialog asking whether, when creating a
clone or bookmark, the user wants to move to the designated view node,
or remain in the outline.
3. [Bookmarks *are* clones, but are *not* marked as such]. Creating
(and perhaps automatically moving) a bookmark creates a new cloned
node, but the new bookmark is not *marked* as a clone. It will
probably be a good idea to mark the original node as the *target* of a
bookmark.
The clone command, when applied to a bookmark, will work as usual,
whether or not the bookmark has automatically been moved outside the
@org-mode or @auto tree. When explicitly cloned, the bookmark node
will be marked as a clone.
4. [Deleting bookmarks]. Deleting a bookmark will work just the same
as any other node, whether or not the bookmark is cloned. When only
the target of a bookmark remains, the bookmark will no longer be
marked as cloned.
5. [Bookmarks (and their targets) have uA's contain the bookmarks
pattern (s?)] Bookmarks are self-contained. They (and their clones)
may appear anywhere. Especially when bookmarks break, we'll want an
easy way to examine the patterns contained in the bookmark pattern,
say a show-bookmark-pattern command.
I said earlier that bookmark nodes may appear in @file nodes, provided
that the root @file node contains an @all directive. We want to allow
@file nodes to save bookmarks, because only @file trees can remember
the bookmark's uA's. Naturally, the write code will *not* write the
uA of a target of a bookmark if that target appears in an @auto or
@org-mode tree.
6. [Find can be aware of views and bookmarks]. If a find starts on or
within a view node, it might be good to limit the search to that view
node. Otherwise, find should probably skip all view nodes and their
children. This simple rule (or something like it) would reduce or
even eliminate most duplicate hits during finds.
===== Summary
All this is truly unexpected. Bookmark nodes will act just like any
other node! They be moved or cloned anywhere, except for the the
restriction in point 1. There will be some complications arising from
the distinction between a bookmark and its target, but my present
workflow will likely remain completely unchanged, except that I'll
take advantage of automatic moves of both clones and bookmarks.
Why the excitement, you ask? Because bookmarks provide a way of doing
without gnx's in @org-mode or @auto files, while retaining (or even
improving) existing workflow. That's a huge step forward for Leo.
Edward
Edward K. Ream
Jul 5, 2014, 3:59:22 PM7/5/14
to leo-e...@googlegroups.com
On Saturday, July 5, 2014 8:25:37 AM UTC-5, Edward K. Ream wrote:
> suppose the clone command automatically creates an (official) bookmark node instead of a clone node inside @auto and @org-mode trees.
> suppose the clone command automatically creates an (official) bookmark node instead of a clone node inside @auto and @org-mode trees.
The more I think about this, the simpler and clearer the picture becomes:
1. What I have been calling an "official bookmark" is nothing but a clone created using a pattern rather than a gnx. That's the only real difference--only the read and write code are affected. This is as bounded a change as can be imagined.
Let's use the term **bookmark clone** from now on to denote a clone created from a pattern instead of a gnx.
2. Automatically moving clones or bookmark clones to a designated view node is a separate matter. It should be worth doing. Ditto for having the find command treat bookmarks clones (and maybe other clones) differently from other nodes.
Let's see how to create bookmark clones from patterns. Consider the following unl:
<prefix> -> @<file> F -> W -> X - > Y -> Z
We can ignore the prefix, and search for the given @<file> F node. If not found, the unl fails to match. Otherwise, we search for a node Z in the @<file> tree. If not found, the unl fails. If there are several nodes Z, we pick the node Z with parent Y. And so on. The unl **matches** if a unique node is found. The unl **fails** if zero or more than one nodes are ultimately found. This algorithm will be straightforward and fast.
We could, in principle, generalize both unl's and the corresponding algorithm, but this is a good enough start for now.
Edward
1. What I have been calling an "official bookmark" is nothing but a clone created using a pattern rather than a gnx. That's the only real difference--only the read and write code are affected. This is as bounded a change as can be imagined.
Let's use the term **bookmark clone** from now on to denote a clone created from a pattern instead of a gnx.
2. Automatically moving clones or bookmark clones to a designated view node is a separate matter. It should be worth doing. Ditto for having the find command treat bookmarks clones (and maybe other clones) differently from other nodes.
Let's see how to create bookmark clones from patterns. Consider the following unl:
<prefix> -> @<file> F -> W -> X - > Y -> Z
We can ignore the prefix, and search for the given @<file> F node. If not found, the unl fails to match. Otherwise, we search for a node Z in the @<file> tree. If not found, the unl fails. If there are several nodes Z, we pick the node Z with parent Y. And so on. The unl **matches** if a unique node is found. The unl **fails** if zero or more than one nodes are ultimately found. This algorithm will be straightforward and fast.
We could, in principle, generalize both unl's and the corresponding algorithm, but this is a good enough start for now.
Edward
Fidel N
Jul 5, 2014, 5:00:21 PM7/5/14
to leo-e...@googlegroups.com
Hey Eward!
That seems very similar to what Terry and I did in our last update for unl's and recursiveunlfind function.
On the way we also added repeated names security (second node with same name would store its number in the UNL) as you know.
I really want to comment on this issue, I have been following those posts but not sure if I followed what was being commented, it was a bit technical for me.
So maybe this doesnt fall on the line of conversation, in that case please disregard the comment.
Just wanted to iterate how I would really like to have some "permanent" reference to a node. I would even pay the price of an attached "db per leo file" so those node references would become permanent.
There are two ways of making a permanent pointer to a node that I can think of. UNLs the way we implemented last time (and the way you just said) need one more thing for being "close" to perfect.
The idea would be not only to register their initial UNL and name, but also, whenever a node changes its header and/or position, that data should go to the DB.
That way a node could be tracked down from its initial point to the final current actual situation it is in right now.
Now I don't need permanent care-free node references often, but ever since I came to Leo I found many times in which I had to create many scripts just to track nodes up and down.
Think of that, whenever you want to reference a node, you need to consider all the positions it can be // will be, what can happen to it in EACH script you make.
It would be awesome to tell to a script "this node", and for Leo to be able to find it, no matter what, even in different files.
Hence, the database, referring to each node, their UNL, the changes on their UNL history, possibly a unique and permanent ID (gnx-like), etc.
Im also very glad towards where all recent discussions are heading, if Im understanding them well hehe.
Regards,
Fidel.
Edward K. Ream
Jul 6, 2014, 7:58:46 AM7/6/14
to leo-editor
On Sat, Jul 5, 2014 at 4:00 PM, Fidel N <fidel...@gmail.com> wrote:
> Hey Eward!
> That seems very similar to what Terry and I did in our last update for unl's
> and recursiveunlfind function.
Oh good. Great minds thinking alike :-) Could you tell me the link
> Hey Eward!
> That seems very similar to what Terry and I did in our last update for unl's
> and recursiveunlfind function.
to that conversation? I only vaguely remember it.
> Just wanted to iterate how I would really like to have some "permanent"
> reference to a node. I would even pay the price of an attached "db per leo
> file" so those node references would become permanent.
present caching machinery uses ~/.leo/db as a "master" db, with
subsidiary db's (per-leo-file db's) inside. So we have already paid
that price!
> There are two ways of making a permanent pointer to a node that I can think
> of. UNLs the way we implemented last time (and the way you just said) need
> one more thing for being "close" to perfect.
>
> The idea would be not only to register their initial UNL and name, but also,
> whenever a node changes its header and/or position, that data should go to
> the DB.
> That way a node could be tracked down from its initial point to the final
> current actual situation it is in right now.
Btw, git claims to be able to track file name renames without actually
keeping any per-file data. I can't seem to find the actual words,
just now. Does anyone know what I'm talking about? It's amazing how
git stimulates the imagination.
> Now I don't need permanent care-free node references often, but ever since I
> came to Leo I found many times in which I had to create many scripts just to
> track nodes up and down.
>
> Think of that, whenever you want to reference a node, you need to consider
> all the positions it can be // will be, what can happen to it in EACH script
> you make.
> It would be awesome to tell to a script "this node", and for Leo to be able
> to find it, no matter what, even in different files.
> Hence, the database, referring to each node, their UNL, the changes on their
> UNL history, possibly a unique and permanent ID (gnx-like), etc.
correctly, it would allow Leo to move to org-mode file format (no
gnx's) without sacrificing *anything*. We seem to be converging on
something truly important for Leo.
> Im also very glad towards where all recent discussions are heading, if Im
> understanding them well hehe.
Edward
Edward K. Ream
Jul 6, 2014, 9:22:45 AM7/6/14
to leo-e...@googlegroups.com
On Saturday, July 5, 2014 2:59:22 PM UTC-5, Edward K. Ream wrote:
The more I think about this, the simpler and clearer the picture becomes:
1. What I have been calling an "official bookmark" is nothing but a clone created using a pattern rather than a gnx. That's the only real difference--only the read and write code are affected. This is as bounded a change as can be imagined.
Let's use the term **bookmark clone** from now on to denote a clone created from a pattern instead of a gnx.
My present thinking is that bookmark clones will act *exactly* like all other clones (once they have been created). In other words, ignore these two complications from two earlier posts:
- Bookmarks aren't marked as clones unless the user clones them later.
- Unbookmarked (raw) clones **will not be allowed** inside @auto or @org-mode external files, but bookmarks, or **clones of bookmarks**, will be allowed.
At present, I'm not sure whether clones will be allowed in @auto or @org-mode files, but whatever the policy turns out to be, it will apply to both bookmarked and unbookmarked clones.
One more nit: There is no real need to change Leo's long-standing icon box. Following Terry's lead with the todo.py plugin, all we need is an small, official, easily recognized icon of a bookmark that can appear just to the right of the icon box. Leo's core will draw that icon whenever Leo's drawing code updates the icon box for a bookmarked node (cloned or not).
EKR
Edward K. Ream
Jul 6, 2014, 10:03:45 AM7/6/14
to leo-e...@googlegroups.com
On Sunday, July 6, 2014 6:58:46 AM UTC-5, Edward K. Ream wrote:
> That way a node could be tracked down from its initial point to the final
> current actual situation it is in right now.
Cool idea.
Btw, git claims to be able to track file name renames without actually
keeping any per-file data. I can't seem to find the actual words,
just now. Does anyone know what I'm talking about? It's amazing how
git stimulates the imagination.
Found it: http://git-scm.com/book/en/Git-Basics-Recording-Changes-to-the-Repository#Moving-Files
QQQ
Unlike many other VCS systems, Git doesn’t explicitly track file movement. If you rename a file in Git, no metadata is stored in Git that tells it you renamed the file. However, Git is pretty smart about figuring that out after the fact — we’ll deal with detecting file movement a bit later.
Thus it’s a bit confusing that Git has amv command...QQQ
EKR
Fidel N
Jul 6, 2014, 10:19:28 AM7/6/14
to leo-e...@googlegroups.com
Oh good. Great minds thinking alike :-) Could you tell me the link
to that conversation? I only vaguely remember it.
Hehehe, here is the link; Finding a node by the tail of the UNL. Current Leo working code has that loging working.
I quote from there:
If you search for it using priority_header == True, it will always find the node if its name exists, and will pick the longest matching UNL. It will do so only if it doesnt find the node header in its original position. This option is giving priority to UNL string over its position.
c.selectPosition(c.viewController.find_absolute_unl_node(unl,priority_header=True))Glad you liked the DB idea, back when we were updating the UNL code, we realized that would be the only way (we could think of) to finally create a permanent-inter-file unique node ID. Large copy / paste of outlines between files will give some headaches with the logic, but as long as the DB gets all the data, any node will be traceable from the moment he is born until the day he dies.
Sweet.
Edward K. Ream
Jul 6, 2014, 2:11:19 PM7/6/14
to leo-editor
On Sun, Jul 6, 2014 at 9:19 AM, Fidel N <fidel...@gmail.com> wrote:
> Hehehe, here is the link; Finding a node by the tail of the UNL. Current Leo
> working code has that loging working.
>
> https://github.com/leo-editor/leo-editor/pull/13#issuecomment-37282367
Thanks for that.
> Hehehe, here is the link; Finding a node by the tail of the UNL. Current Leo
> working code has that loging working.
>
> https://github.com/leo-editor/leo-editor/pull/13#issuecomment-37282367
> Glad you liked the DB idea, back when we were updating the UNL code, we
> realized that would be the only way (we could think of) to finally create a
> permanent-inter-file unique node ID. Large copy / paste of outlines between
> files will give some headaches with the logic, but as long as the DB gets
> all the data, any node will be traceable from the moment he is born until
> the day he dies.
> Sweet.
and only way I have ever used gnx's myself. I saw::
#@+node:ekr.20140102052259.16402: *5* vc.find_absolute_unl_node
Hahaha. I had forgotten all about this logic, including the fact that
I wrote it. It's good to see that parts of the "failed"
viewController logic have become useful.
Imo, it might be good to be able to retain the gnx above::
ekr.20140102052259.16402
Is this part of your present plan? If so, it would be totally awesome ;-)
Edward
Reply all
Reply to author
Forward
0 new messages