Rev 789350d in devel: a big step for Leo's blacken and beautify commands

[Edward K. Ream]

This work completes #1325: Beautifiers should optionally preserve indentation of overindented comments.

Leo now sports two new settings:

@bool black-keep-comment-indentation
@bool orange-keep-comment-indentation

The former applies to the blacken-tree and blacken-node commands. The latter applies to the beautify-tree and beautify-node commands. Maybe it should be renamed beautify-keep-comment-indentation.


About 5 lines of code in this the SyntaxSanitizer class suffice to support the two new settings.

Significance

The way is now clear to finish #1322: beautify/blacken all of Leo's sources.

I plan to use black (blacken-tree) not orange (beautify-tree). This will provide a solid baseline for working on orange.

Black does not need to change in any way. Leo's blacken-commands use the SyntaxSanitizer class to shield black from Leo's syntax. There should be little or no need to apply the external version of black on external files!

Next steps

1. I'll use Leo's internal blacken-tree command  to beautify all of Leo's sources This will be done in a new test-black branch. As noted earlier, blacken-tree fails for one or two of Leo's sources.  I'll attempt to alter Leo's sources so that blacken-tree can be used everywhere.

2. After thorough testing, the test-black branch will be merged into the devel and beauty branches.

3. Finally, I'll complete #1266: (orange is the new black) in the beauty branch. A new test-orange branch will contain the results of using beautify-tree everywhere. This branch can then be compared against devel.

Summary

Leo now contains settings that allow black to beautify all of Leo's sources, using Leo's blacken-tree command.  A new setting prevent blacks from messing with hundreds of existing overindented comments.

Black does not need to change in any way! The blacken-tree command reflects Leo's philosophy: we never argue about preferences. Leonistas can use black as they choose, not as black's devs demand.

Work on orange (beautify-tree, enhanced with line breaking/joining) will continue.  At present, beautify-tree is four times faster than blacken-tree. The token-based code for beautify-tree is interesting in itself. Imo, it is significantly simpler than black's parse-tree based code.  For example, regularizing strings is very easy using tokens.  It's a nightmare using parse trees.

Edward

[Edward K. Ream]

On Wednesday, September 11, 2019 at 5:35:42 PM UTC-5, Edward K. Ream wrote:

> About 5 lines of code in this the SyntaxSanitizer class suffice to support the two new settings...

> Black does not need to change in any way...

> A new setting prevent blacks from messing with hundreds of existing overindented comments.

I want to brag a bit. The SyntaxSanitizer class is short, elegant and useful. It:

- allows Leo's devs to carry on, regardless of black's limitations, philosophy, whatever.

- allows the beautify-tree command to verify that it doesn't change the code's meaning (parse tree).

Leo's adaptation of black is way more elegant than the equivalent mods would be in black itself!

Edward