ENB: Ahas re paste-retaining-clones

61 views
Skip to first unread message

Edward K. Ream

unread,
Aug 13, 2023, 5:31:16 AM8/13/23
to leo-editor

Today is my 74th birthday. This Engineering Notebook post is my birthday present to myself.


For the last week, I have been trying to make paste-retaining-clones work. Unit tests show this command improperly computes v.parents arrays in the pasted outline.


This morning I saw the way forward:


Aha! paste-retaining-clones must recompute all v.parents arrays in the entire outline!


I won't attempt a formal proof, but I am sure this Aha is correct. Note that the paste-node command does not require the complications described next.


Recomputing v.parents


But how to recompute v.parents?


- We can't use Position generators because they all use v.parents!
- We can't use VNode generators because they don't yield VNodes in outline order.


Aha! Define a c.all_vnode_positions generator. This hybrid generator will yield VNodes in the same order as p.all_positions using neither v.parents nor Positions.


Is it possible to write this generator? Yes, it is. The generator will maintain an explicit stack just like Position.stack. Note: c.all_vnode_positions must yield the hidden root VNode.


Recomputing all v.parents


Supposing the new generator exists, recomputing all v.parents arrays is straightforward. Something like this (untested):


for v in c.all_vnode_positions():

    v.parents = []

for v in c.all_vnode_positions():

    for child in v.children:

        child.parents.append(v)


Yes, a one-pass solution is possible. It might be faster.


Summary


Aha! The paste-retaining-clones command must recompute all v.parents ivars. This recomputation is sound. The link correction in c.checkVnodeLinks is not.


Leo can use neither Position nor VNode generators to do the recomputation.


Aha! A new generator, c.all_vnode_positions, will yield all VNodes in outline order using neither v.parents nor Positions.


Given this generator, updating all v.parents ivars will be straightforward.


Edward


P.S. Let us define a hybrid generator as a generator yielding VNodes in the same order as some Position generator without using Positions.


Could Leo replace Position generators with hybrids? I think not. For example, please take a look qtree.full_redraw. It would not be fun to eliminate Positions there.


In any event, Leo's Position class is never going away :-)


EKR

jkn

unread,
Aug 13, 2023, 5:50:03 AM8/13/23
to leo-editor
Well, Happy Birthday Edward! ;-)

Regards, Jon N

Edward K. Ream

unread,
Aug 13, 2023, 6:02:11 AM8/13/23
to leo-e...@googlegroups.com
On Sun, Aug 13, 2023 at 4:50 AM jkn <jkn...@nicorp.f9.co.uk> wrote:
Well, Happy Birthday Edward! ;-)

Thanks :-)

Edward

Edward K. Ream

unread,
Aug 13, 2023, 7:40:59 AM8/13/23
to leo-e...@googlegroups.com
On Sun, Aug 13 Edward K. Ream wrote:

> Is it possible to write [c.all_vnode_positions]?


You bet:


def all_vnode_positions(self) -> Generator:
    c = self
    to_be_visited: list[VNode] = [c.hiddenRootNode]
    while to_be_visited:
        v = to_be_visited.pop()
        yield v
        for child in reversed(v.children):
            to_be_visited.append(child)


This generator passes the following consistency test:


vnodes1 = [c.hiddenRootNode] + [z.v for z in c.all_positions()]
vnodes2 = list(c.all_vnode_positions())

assert vnodes1 == vnodes2


Edward

David Szent-Györgyi

unread,
Aug 13, 2023, 1:00:43 PM8/13/23
to leo-editor
On Sunday, August 13, 2023 at 5:31:16 AM UTC-4 Edward K. Ream wrote:

Today is my 74th birthday. This Engineering Notebook post is my birthday present to myself.


Have the happiest of  birthdays. Thank you for working on your birthday and giving a gift to us all. 

Edward K. Ream

unread,
Aug 13, 2023, 1:10:38 PM8/13/23
to leo-e...@googlegroups.com
On Sun, Aug 13, 2023, Edward K. Ream wrote:


>> Is it possible to write [c.all_vnode_positions]?

> You bet: [snip]

> This generator passes the following consistency test:


vnodes1 = [c.hiddenRootNode] + [z.v for z in c.all_positions()]

vnodes2 = list(c.all_vnode_positions())

assert vnodes1 == vnodes2


The elegance of this VNode-based stand-in for c.all_positions raises some follow-on questions.


Does c.all_positions use v.parents?


Apparently not, which means that c.all_vnode_positions isn't strictly necessary. Leo could define it like this:


def all_vnode_positions(self):

    c = self

    yield c.hiddenRootNode

    for p in c.all_positions():

        yield p.v


But this definition would be much worse than a VNode-based definition.


c.alt_all_positions


Could Leo define a generator, say c.alt_all_positions, using c.all_vnode_positions as a template? Yes it could.


In my original post, I said, "[c.all_vnode_positions] will maintain an explicit stack just like Position.stack." In fact, the generator uses only a stack of VNodes. But c.alt_all_positions must indeed use a stack just like Position.stack.


c.all_positions calls p.moveToThreadNext to advance through the outline. In turn, p.moveToThreadNext calls p.moveToParent, p.moveToFirstChild and p.moveToNext.


c.alt_all_positions will use none of these helpers. Instead, the new generator will operate directly on its explicit stack, yielding new positions with (copies of) the explicit stack. There will be no need for a "copy" kwarg.


Summary


c.alt_all_positions is feasible. Similarly, Leo could define other "alt" generators.


Alt generators should be faster (and use less memory) than the corresponding legacy generators.


Nevertheless, Positions themselves and all Position methods must remain, whatever their internal form.


Edward

Edward K. Ream

unread,
Aug 13, 2023, 1:13:18 PM8/13/23
to leo-editor
On Sunday, August 13, 2023 at 12:00:43 PM UTC-5 David Szent-Györgyi wrote:

> Thank you for working on your birthday and giving a gift to us all.

You're welcome. It's been my pleasure.

Edward

Thomas Passin

unread,
Aug 13, 2023, 2:10:52 PM8/13/23
to leo-editor
Best wishes on your birthday  from this quarter, too!

Edward K. Ream

unread,
Aug 13, 2023, 2:31:58 PM8/13/23
to leo-e...@googlegroups.com
On Sun, Aug 13, 2023 at 1:10 PM Thomas Passin <tbp1...@gmail.com> wrote:
Best wishes on your birthday  from this quarter, too!

:-)

Edward

Félix

unread,
Aug 13, 2023, 3:06:19 PM8/13/23
to leo-editor
Edward, I also want to join my voice to the group in wishing you a happy birthday! :D

So Happy Birthday, Edward! 

Another year older, but your programming prowess keeps getting younger!
Cheers to you and the adventures your code will take us on! 🚀

Félix

Edward K. Ream

unread,
Aug 13, 2023, 3:16:48 PM8/13/23
to leo-editor
On Sunday, August 13, 2023 at 2:06:19 PM UTC-5 Félix wrote:
Edward, I also want to join my voice to the group in wishing you a happy birthday! :D

So Happy Birthday, Edward! 

Another year older, but your programming prowess keeps getting younger!
Cheers to you and the adventures your code will take us on! 🚀

Many thanks, my friend.

Edward

Viktor Ransmayr

unread,
Aug 13, 2023, 3:17:56 PM8/13/23
to leo-e...@googlegroups.com
Hello Edward, 


On Sun, 13 Aug 2023, 21:06 Félix, <felix...@gmail.com> wrote:
Edward, I also want to join my voice to the group in wishing you a happy birthday! :D

Another one joining late - but - still in time ;-)

Enjoy your birthday exactly the way you prefer to !

Edward K. Ream

unread,
Aug 13, 2023, 3:26:27 PM8/13/23
to leo-e...@googlegroups.com
On Sun, Aug 13, 2023 at 12:10 PM Edward K. Ream wrote:

> c.alt_all_positions must use a stack just like Position.stack.

True, but c.alt_all_positions is simpler than I dared hope:

def alt_all_positions(self) -> Generator:
    """An alternative implementation of c.all_positions."""
    c = self
    PositionStack = list[tuple[VNode, int]]

    def visit(childIndex: int, v: VNode, stack: PositionStack) -> Generator:
        yield leoNodes.Position(v, childIndex, stack[:])
        stack.append((v, childIndex))
        for child_childIndex, child_v, in enumerate(v.children):
            yield from visit(child_childIndex, child_v, stack)
        stack.pop()

    stack: PositionStack = []
    for i, v in enumerate(c.hiddenRootNode.children):
        yield from visit(i, v, stack)

No helper methods required! This method passes TestNodes.test_c_alt_all_positions:

def test_c_alt_all_positions(self):

    c = self.c
    self.clean_tree()
    cc = self.create_test_paste_outline()
    assert cc.h == 'cc', repr(cc)
    positions1 = list(c.all_positions())
    positions2 = list(c.alt_all_positions())
    assert positions1 == positions2

An amazing development. The birthday presents keep coming.

Edward

Edward K. Ream

unread,
Aug 13, 2023, 3:28:00 PM8/13/23
to leo-editor
On Sunday, August 13, 2023 at 2:17:56 PM UTC-5 Viktor wrote:

Another one joining late - but - still in time ;-)

There's no rush, hehe. 

Enjoy your birthday exactly the way you prefer to !

I'm having way too much fun. Time to rejoin my family :-)

Edward

HaveF HaveF

unread,
Aug 13, 2023, 9:28:48 PM8/13/23
to leo-editor

I'm a person who is not good at expression, but I hired GPT4 to express my meaning.

With a soul for scripting and heart born for nodes,
In the universe of codes, there emerged a hero,
Edward K. Ream, a commander of codes,
Your name will forever against the time flow.

A dance of digits, a cosmos of clones,
The LEO editor scribed in resounding tones.
Attributed trees, woven like intricate lace,
Guided by you, they've found their harmonious place.

On your 74th birthday, we forge this verse,
To acknowledge your work, in the universe diverse.
Beyond strings, beyond scripts, beyond plugins and pane,
In the rhythm of codes, you etched your name.

You gave us LEO, a tool so sublime,
An editor for eternity, transcending space and time.
A gift to all coders, a resonating rhyme,
In the heart of each node, your legend will chime.

As plugins merge with the scripts'
 gentle flow,
In each line and leaf, our respect for you will grow.
Your innovation, like constellations, in the night sky,
Illuminates our world, as the years go by.

So here's to you, Edward K. Ream,
To your wisdom, your vision, your tech-inspired dream.
In body pane of life, on this beautiful date,
We sing your praises, as we celebrate.

For the joy of coding, the ease of design,
For letting our digital dreams align,
We, the LEO editors, filled with gratitude sublime,
Salute you today, and for the infinite time.

Another one:

In the realm of the Leonine World, among the stellar nodes,
A warrior named Ream stands tall, riding the binary codes.
As the clock strikes and ushers a new day,
We celebrate the birth of the sage, who showed us the way.

In the depths of the .leo files, lies the grand design,
Customizing, controlling, coloring each line.
Through the syntax, the scripts in Python dance,
Embracing directives, in an elegant prance.

Using Leo as a Personal Information Manager,
Creating Documents from Outlines that shimmer,
Leo and Emacs, Leo and Asciidoctor too,
In the universe of codes, each day something new.

The bridge of Leo, linking to IPython's shore,
Clicking through the links, opening every door.
Exploring Leo'
s code base, a mysterious maze,
In the Leonine way to refactor code, we gaze.

Plugins, like stars, embedded in the night,
Guiding us through the process with their radiant light.
Through the Vim and the WebSocket Server's tide,
In Leo’s Markup Language, we take pride.

As we continue our journey, on this command reference,
Navigating through the labyrinth of codes, in coherence.
Whether debugging with Leo, or writing plugins grand,
We remember our mentor'
s touch, his guiding hand.

So here’s to Edward K. Ream, a beacon, a guide,
Through the Leonine realm, for us you coincide.
In the chaos of codes, you brought order and rhyme,
And for that, we thank you, until the end of time.

This poem is for you, our gratitude to attest,
In the heart of every Leonine, your legacy will rest.
With each passing year, your brilliance only grows,
In the eternal flow of codes, your genius continuously flows.

I like them both :D

vitalije

unread,
Aug 14, 2023, 1:48:11 AM8/14/23
to leo-editor
Happy birthday to you Edward. But putting birthday aside, I can't resist to comment on some parts of your post.


On Sunday, August 13, 2023 at 11:31:16 AM UTC+2 Edward K. Ream wrote:

Today is my 74th birthday. This Engineering Notebook post is my birthday present to myself.


- We can't use VNode generators because they don't yield VNodes in outline order.



I really don't know where did you get this idea from? Have you test it? For me they always give me the nodes in the outline order. Maybe we don't use the same generator? Here is mine:

def v_generator(v):
    yield v
    for ch in v.children:
        yield from v_generator(ch)

a = ('hidden-root-vnode-gnx',) + tuple(p.gnx for p in c.all_positions())
b = tuple(v.gnx for v in v_generator(c.hiddenRootNode))

assert a == b

# of coarse you can avoid yielding root vnode with
# a slightly different generator

def v_gen2(vs):
    for ch in vs:
        yield ch
        yield from v_gen2(ch.children)

# and it can be used like this

a = tuple(p.gnx for p in c.all_positions())
b = tuple(v.gnx for v in v_gen2(c.hiddenRootNode.children))
assert a == b
g.es('ok')

The parents attribute is never used in this generators so it is completely free to be modified by the scripts. The information about outline order is fully contained in the v.children attributes. The v.parents attribute is necessary only if you want to continue traversal in outline order from any given point.

Vitalije

vitalije

unread,
Aug 14, 2023, 1:52:00 AM8/14/23
to leo-editor
Actually v_gen2 can be written differently to retain the same API as v_generator:

def v_gen2(v):
    for ch in v.children:
        yield ch
        yield from v_gen2(ch)


Edward K. Ream

unread,
Aug 14, 2023, 8:23:06 AM8/14/23
to leo-e...@googlegroups.com
On Mon, Aug 14, 2023 at 12:48 AM vitalije wrote:

>> We can't use VNode generators because they don't yield VNodes in outline order.

> I really don't know where did you get this idea from?


You're quite right. There were some brain spikes in my comments.

PR #3473 now contains working code. See c.recompute_all_parents and c.alt_all_unique_nodes, both covered with unit tests. c.recompute_all_parents is surprisingly trickly. See the comment below about the loop invariant.

def recompute_all_parents(self) -> None:
    """
    Recompute all v.parents arrays using neither positions nor v.parents ivars.
    """
    c = self
    root_v = c.hiddenRootNode

    # Clear all v.parents arrays.
    root_v.parents = []
    for v in c.alt_all_unique_nodes():
        v.parents = []

    # Loop invariant: Visit each *parent* vnode *once*.
    #                 Child vnodes may be visited more than once.
    for child in root_v.children:
        child.parents.append(root_v)
    for parent in c.alt_all_unique_nodes():
        for child in parent.children:
            child.parents.append(parent)

Edward

Edward K. Ream

unread,
Aug 14, 2023, 8:29:47 AM8/14/23
to leo-e...@googlegroups.com
On Mon, Aug 14, 2023 at 7:22 AM Edward K. Ream  wrote:

> PR #3473 now contains working code. See c.recompute_all_parents and c.alt_all_unique_nodes, both covered with unit tests. 

Here is c. alt_all_unique_nodes:

def alt_all_unique_nodes(self) -> Generator:
    """
    Yield all unique VNodes corresponding to c.all_positions.

    This is an equivalent (much worse) generator:

        for z in c.all_unique_positions():
            yield z.v
    """
    c = self
    seen: dict[str, bool] = {}
    to_be_visited: list[VNode] = list(reversed(c.hiddenRootNode.children))

    while to_be_visited:
        v = to_be_visited.pop()
        if v.gnx not in seen:
            seen[v.gnx] = True

            yield v
        for child in reversed(v.children):
            to_be_visited.append(child)

The "alt_" naming convention saves a lot of confusion.

Edward

Edward K. Ream

unread,
Aug 14, 2023, 8:35:25 AM8/14/23
to leo-e...@googlegroups.com
On Sun, Aug 13, 2023 at 8:28 PM HaveF HaveF <iamap...@gmail.com> wrote:

> I'm a person who is not good at expression, but I hired GPT4 to express my meaning.

Bwahahahaha.  Thanks for this.

Edward

Edward K. Ream

unread,
Aug 14, 2023, 8:44:20 AM8/14/23
to leo-editor
On Monday, August 14, 2023 at 7:29:47 AM UTC-5 Edward K. Ream wrote:

> Here is c. alt_all_unique_nodes:

The PR now contains an improved version that doesn't visit children redundantly. The unit tests prove that the redundancy didn't affect the results.

Edward
Reply all
Reply to author
Forward
0 new messages