Second try: Please test PR #2985

[Edward K. Ream]

Drat: I hit the "Post Message" button trying to format some text.  Here is the full post.

Leo issue #2983 suggests supporting so-called blackened sentinels, sentinel lines that start with "# @" instead of the usual "#@". Such lines can arise in external files created by @file after the user reformats the file with Black.

PR #2985 contains the changes, which involve inherently dangerous changes to Leo's read code. #2983 gives the rationale for accepting the risks involved, which I'll repeat here:

- Leo cannot prevent Leonistas from blackening external files unintentionally. Black's existence imposes a burden on Leo.

- Leo should not prevent Leonistas from blackening external files intentionally.
--black-sentinels can help avoid huge diffs in projects mandating Black.

- Any change to Leo's read/write code is inherently dangerous, but doing nothing may be more dangerous in the long run.

Summary of changes

Leo has (supposedly) been able to handle blackened sentinels for a long time. However, recent tests have exposed problems with Leo's legacy code. The new code fixes several bugs, some serious (potentially damaging data!) and some less so (crashers).

The PR adds support for  --black-sentinels (aka -b), which causes Leo to write blackened sentinels.

Testing status

I have strengthened several unit tests to test for both blackened and legacy sentinels.

All of Leo's files pass round-trip read/write tests, both with and without -b. This is the strongest test I know of.

I have been eating my own dog food without any serious problems.

Summary

PR #2985 contains inherently dangerous changes to Leo's read code.

I have tested the code as well as I can, but additional testing seems wise before merging PR #2985 into devel. Please test that branch now.

I'll wait about a week before doing the merge. Please report any problems immediately.

Edward

[Thomas Passin]

Using ekr-black2, I found the following bug:

Leo subtree:

@path c:/temp/leo

    @file a.py

    @file b.py

1. Select the a.py node.  Neither blacken-files nor blacken-files-diff find any file to blacken.

2. Select the @path node.  Blacken-files-diff only finds a.py, not b.py. (I assume blacken-files would do the same but did not test it).

The culprit seems to be g.findRootsWithPredicate(c, c.p).  I put a print statement in to verify 1) and 2).

Otherwise, using -b did produce # @xxx sentinels on write.  Test files with blackened sentinels did load into Leo with no warnings or apparent errors.

[Edward K. Ream]

On Sun, Dec 4, 2022 at 5:04 PM Thomas Passin <tbp1...@gmail.com> wrote:

Using ekr-black2, I found the following bug:

I have just created #2989 for this.

Otherwise, using -b did produce # @xxx sentinels on write.  Test files with blackened sentinels did load into Leo with no warnings or apparent errors.

Thanks for testing :-)

Edward

[Thomas Passin]

Alt-G goto-global-line is broken in branch ekr-black2:

 return line.startswith(sentinel1, sentinel2)
TypeError: slice indices must be integers or None or have an __index__ method

[Edward K. Ream]

On Mon, Dec 5, 2022 at 9:23 AM Thomas Passin <tbp1...@gmail.com> wrote:

Alt-G goto-global-line is broken in branch ekr-black2:

 return line.startswith(sentinel1, sentinel2)

Oops: should be `return line.startswith((sentinel1, sentinel2))

Edward

[Edward K. Ream]

On Mon, Dec 5, 2022 at 9:23 AM Thomas Passin <tbp1...@gmail.com> wrote:

Alt-G goto-global-line is broken in branch ekr-black2:

Time for a new unit test.

Edward

[Edward K. Ream]

On Mon, Dec 5, 2022 at 9:23 AM Thomas Passin <tbp1...@gmail.com> wrote:

Alt-G goto-global-line is broken in branch ekr-black2:

 return line.startswith(sentinel1, sentinel2)
TypeError: slice indices must be integers or None or have an __index__ method

Thanks. Fixed in a recent revs to the ekr-black2 branch.

Edward