The new python importer: comments and questions

[Edward K. Ream]

The new python appears solid. It imports all mypy files perfectly.

My only question: I don't see any explicit perfect import checks. Why aren't the checks needed?

Edward

[vitalije]

The import process consists of two separate phases. In the first, toknize module is used to calculate all necessary indices, and then in the second phase, those calculated values are used to cut and paste chunks of text lines creating node bodies. Assuming there is no error in the calculated data, there isn't any possible point of failure later. And I think we can trust the python's own tokenizer to produce valid data.

In essence, importing can just change the indentation of a line. The order of lines isn't changed anywhere. The next node is generated after the previous one has been finished, and all the gaps between the consecutive definitions are filled with the declaration nodes which contain all the lines from the gap. Thus, the order of lines remains the same after the import.

The indentation of lines in the child nodes is reduced by the same amount the at-others directive is indented, thus the indentation in round trip must be the same. If you look at the mknode function, you can see that at-others is indented by `col-l_ind`, and from the body lines l_ind characters from the left are stripped, and then mknode is recursively called with the values `l_ind + col` for l_ind argument, and col with the column of the first line of method/class body. So, if col is greater than zero, this means at-others is indented by `col-l_ind` and all the lines in the children are stripped from the left by l_ind + col; the same amount. The sign in the expressions col - l_ind and col + l_ind are opposite because function `indent('@others\n')` adds the indentation to the left, and function `bodyLine` strips the indentation from the left. Have look at the explanation and the picture in my earlier post.

HTH Vitalije