Three big Ahas re collaborating with Leo

[Edward K. Ream]

Aha 1: The key to collaborating with Leo is merging .leo files!

Aha 2: Leonistas need a way to resolve merge conflicts in.leo files!

Aha 3: git-diff provides the infrastructure to resolve such merge conflicts!

This is a long post, explaining all three Ahas in detail.

tl;dr: See the summary.

All three Ahas arose from the collaboration between Félix and me on the leojs project. This project is part of a plugin for vs code.

The motivating example

The leojs project uses this package.json file. The first thing I did was to expose its structure. Like this:

{
  << plugin info >>
  "contributes": {
    << contributes >>
  },
  "scripts": {
    << scripts >>
  },
  "devDependencies": {
    << devDependencies >>
  }
}

The << contributes >> section is the heart of package.json. It contains (long!) lists of commands, menu items, and key bindings. So it was natural to write scripts that would parse the sub-sections of <<contributes>> into separate nodes for each command, menu item, and key binding.

These scripts belong in leojs.leo. To make diffing easier, I moved those scripts into a separate file, scripts.txt. I use the .txt suffix to suppress pyflakes complaints. In other words, leojs.txt starts with:

- Startup

  - @settings

    - @file scripts.txt

So far, so good. You can see the first two scripts in scripts.txt.

Aside. You can also see the `@button backup` script that was the motivation for Leo issue #1780.

The motivating problem

Alas, "real" comments are never going to be allowed in .json files. See vs code issue #4482. Absent a major hack to Leo, this means that:

    leojs.leo must contain @clean package.json

Unlike with scripts.txt, we can expect merge conflicts in leojs.leo, due to merge conflicts in package.json.  That's because Leo stores outline structure for @clean files in the .leo file.

A workaround would be for Félix and me to agree to have only one of us modify package.json "at a time". But this would be clumsy, because changing package.json will likely be necessary in various branches simultaneously.

Furthermore, at present Félix uses @clean for all the .ts files in the project as well. So we can expect even more merge conflicts in leojs.leo. We can't reasonably restrict simultaneous changes to every @clean file!

To summarize Ahas 1 and 2 above:

    Félix and I can expect frequent merge conflicts in leojs.leo.

Aha 3: git-diff provides the infrastructure to resolve merge conflicts in .leo files!

Along with Leo's clone-find commands, Leo's git-diff command is one of Leo's killer features. The git-diff command produces Leonine diffs. That is, git-diff provides node-by node diffs. When diffing two versions of an eternal file (not .leo file), git-diff reports:

- Inserted nodes (nodes that only appear in version 2)

- Deleted nodes (nodes that only appear in version 2)

- Changed nodes

It is amazingly easy to provide node-by-node diffs! The diffs are done in the diffed outlines, rather than at the text level. It's a piece of cake.

The git diff command creates two hidden commanders, one for each version. A simple iteration (c.all_unique_positions) suffices to create a dictionary of nodes for each hidden commander. It is then trivial to compare the two dictionaries to determine the inserted, deleted, and changed nodes.

At present, git-diff works only on external files, but...

The Aha: we can use exactly the same strategy to diff .leo files!

There will be complications, but not many:

1. When creating the hidden commanders, Leo should not read @<file> nodes, because changes to external files do not affect the diff of the .leo file itself.

2. Because of conflict markers, leojs.leo may be corrupted while resolving merge conflicts. It's not a big deal. Git creates temp files containing the (uncorrupted) before/after versions of the .leo file.

Acknowledgment

Rebecca helped me see all three Ahas. She is such a great listener :-)

Summary

The three Ahas arose from problems that arose early during the collaboration on the leojs project. Such problems will arise whenever two or more people collaborate using @clean files.

The infrastructure underlying Leo's git-diff command can be repurposed to provide Leonine diffs of .leo files. These diffs will allow devs to resolve merge conflicts in .leo files by picking and choosing between nodes in the two .leo files.

Imo, an easy way of resolving merge conflicts in .leo files will make Leo much more suitable for collaboration. This is a supremely important step forward for Leo.

Edward

[Edward K. Ream]

On Saturday, December 12, 2020 at 7:09:52 AM UTC-6 Edward K. Ream wrote:

Aha 1: The key to collaborating with Leo is merging .leo files!

Aha 2: Leonistas need a way to resolve merge conflicts in.leo files!

Aha 3: git-diff provides the infrastructure to resolve such merge conflicts!

#1781 Is the corresponding issue. I'll do it during the sabbatical as part of the collaboration on leojs.

Edward

[tbp1...@gmail.com]

This is where I get uneasy because I have messed up too many merge attempts (never tried with Leo outlines).  Those merge tools can be so confusing as to what is going to happen!

On Saturday, December 12, 2020 at 10:06:53 AM UTC-5 Edward K. Ream wrote:

... 2. Because of conflict markers, leojs.leo may be corrupted while resolving merge conflicts. It's not a big deal. Git creates temp files containing the (uncorrupted) before/after versions of the .leo file.

[Edward K. Ream]

On Sat, Dec 12, 2020 at 10:59 AM tbp1...@gmail.com <tbp1...@gmail.com> wrote:

This is where I get uneasy because I have messed up too many merge attempts (never tried with Leo outlines).  Those merge tools can be so confusing as to what is going to happen!

Yes, things can get complicated, but that can't be helped.  Git doesn't create the conflicts, collaboration does.

The new git-pr command greatly clarifies the diffs in a pull request. Ditto for Leo's git diff command.

Only time will tell how easy it will be to handle merge conflicts in .leo files.

Edward

[Edward K. Ream]

On Saturday, December 12, 2020 at 7:09:52 AM UTC-6 Edward K. Ream wrote:

Aha 1: The key to collaborating with Leo is merging .leo files!

Aha 2: Leonistas need a way to resolve merge conflicts in.leo files!

Aha 3: git-diff provides the infrastructure to resolve such merge conflicts!

Recent experience with the leojs project has clarified matters.

I have just updated #1781: Leo's git-diff command and diff-pr script should support diffing .leo files. This *might* suffice. For sure we want Leonine diffs of .leo files in PR's.

Edward