bugs/feature requests in the new python importer

[Edward K. Ream]

These bugs affect only the quality of the imported nodes. The files are imported correctly.

1. In some (rare?) cases, the @others of a class node is not indented. All child nodes contain the leading whitespace that should have preceded the @others.

This bug affects all classes in mypy/test-data/stdlib-samples/3.2/test/test_textwrap.py except the first class.

2. It appears that "async def" is not recognized as the start of a function.

See mypy/test-data/samples/crawl2.py.  A "...some declarations" node contains async def get_connection. There are many other examples.

3. Imo, "...some declarations" nodes should be merged into their following or preceding sibling, if such a sibling exists. In my experience, it's better to prefer the following sibling if both siblings exist.

That's all for now.

Edward

P.S. I shall soon add "@bool put-class-in-imported-headlines = True" to leoSettings.leo, so that the python importer will put 'class' in the headlines of class nodes. The "True" value conforms to legacy operation.

EKR

[vitalije]

I was absent for the last few days. Regarding async def problem, I believe that would be an easy fix/addition. I know that at-others some times can be at the zero indentation and most of the children have extra indentation. This is supposed to happen only if indenting at-others would lead to the indentation of under-indented comments and therefore would cause the difference between the source code and the code outline would produce. This was intentional. I will check to see why those files have this problem, and if it is caused by under indented comments I would rather let user manually fix this indentation problem. The next time the file is imported it would have usual form (indented at-others and not indented children). However, this might also be a matter of setting/preference too.

I'll investigate further and will report my findings later.

Vitalije

[vitalije]

> 1. In some (rare?) cases, the @others of a class node is not indented. All child nodes contain the leading whitespace that should have preceded the @others.

This bug affects all classes in mypy/test-data/stdlib-samples/3.2/test/test_textwrap.py except the first class.

Yes, indeed. These classes have some triple quoted strings which have zero indentation. If the at-others were indented, those strings would change their value, and that could be dangerous change. So, I believe it is better left to the user to decide what to do about those strings.  One possible solution would be to define those strings at the module level as constants and later in the body of the class, use those constants and not triple quoted strings or to replace triple quoted string with the single quoted string and add backslashes where necessary. Both decisions, I believe would be better left to user than let the importer guess what the user would like.

Or we could introduce a new kind of node (for example node whose headline is prefixed with '#') which would not be indented during the write operation. That would cause some changes in the read/write code, but perhaps this would be a good and sensible addition to the Leo's outline power.

Vitalije

[Edward K. Ream]

On Sat, Dec 11, 2021 at 10:31 AM vitalije <vita...@gmail.com> wrote:

I was absent for the last few days. Regarding async def problem, I believe that would be an easy fix/addition. I know that at-others some times can be at the zero indentation and most of the children have extra indentation. This is supposed to happen only if indenting at-others would lead to the indentation of under-indented comments and therefore would cause the difference between the source code and the code outline would produce. This was intentional.

I see. My preference is to base the indentation of @others on the first significant line that follows the "class" line. In that case, the python importer may be forced to insert the underindented escape sequence.

I will check to see why those files have this problem, and if it is caused by under indented comments I would rather let user manually fix this indentation problem. The next time the file is imported it would have usual form (indented at-others and not indented children). However, this might also be a matter of setting/preference too.

I am happy to "promote" underindented comments as required. In my importer, this is (would be?) done by not adding the underindented escape sequence before comment lines.

I'll investigate further and will report my findings later.

Thanks. I am enjoying this conversation. The python importer presents subtle problems, and it's good to see alternative solutions.

Edward