Importers to-do

[Edward K. Ream]

I have declared #334: to be complete. In fact, Leo's importers are clearly as simple as possible. Not only that, they are documented to my satisfaction.

There are, however, several to-do items related to Leo's importers:

- Fix #342: @auto and @auto-md not equivalent for .Rmd file.

This has revealed ancient problems in Leo's importer infrastructure. The new importers, and the three remaining failing unit tests, are likely casualties of these problems. We shall see...

- C and Java importers: Backtrack to the start of signature lines, to eliminate empty headlines.

- Python importer: Rewrite python.findClass to support minor commands.

Bonus: write an @button script that creates a markdown TOC from an outline.

Heh, I expect these to take 1-3 days :-) Happily, none are mission critical.

Edward

[Edward K. Ream]

On Sunday, December 4, 2016 at 5:45:37 AM UTC-6, Edward K. Ream wrote:

There are, however, several to-do items related to Leo's importers:

...

Bonus: write an @button script that creates a markdown TOC from an outline.

Done at d2f5f0e.  It's in scripts.leo.

EKR

[Edward K. Ream]

On Sunday, December 4, 2016 at 5:45:37 AM UTC-6, Edward K. Ream wrote:

I have declared #334: to be complete. In fact, Leo's importers are clearly as simple as possible.

i.gen_lines now supports skip counts, and in a more flexible manner than used in several x.gen_lines overrides.  Rather than returning a skip count, helpers can set the self.skip ivar. This is a nice simplification and generalization.

- Fix #342: @auto and @auto-md not equivalent for .Rmd file.

Still open.  I have found an odious "if" statement in the atFile write logic:

   elif root.isAtAutoRstNode():

I trust that future maintainers will despise this kinds of hack as much as I do ;-)

Having said that, @auto-rst and @auto-md are supposed to force the correct importer to be used, when the file's extension is ambiguous.  So this item is a nit.
 

- C and Java importers: Backtrack to the start of signature lines, to eliminate empty headlines.

I am working extremely hard to complete the importers before Rebecca and I leave for Florida the Friday, December 9.

After a long day of work, the C importer is almost complete.  I expect to complete it this morning.  There were several false starts.  Instead of backtracking, the new code actually does parse C extern, class and function definitions using various regex's.

It's complex, but the complexity is inherent in the problem, not in the Importer infrastructure.  Furthermore, the complexity is limited to communication between two overrides of base class methods: c_i.starts_block and c_i.start_new_block. So while the complexity is unavoidable, it's all in one place where future maintainers can understand them fairly easily.

It's possible to imagine situations where the regex's will fail to parse properly.  For instance, if there were multi-line block comments in the middle of int foo (whatever) {.  I'm prepared to wait for the unlikely bug reports to fix this nit. A fix is surely possibly, but it would involve a secondary full C scanner, something like BaseScanner.startsHelper. It's not worth doing now.

The java importer will use a similar scheme.

Edward

[Edward K. Ream]

On Monday, December 5, 2016 at 3:10:15 AM UTC-6, Edward K. Ream wrote:

> So while the complexity is unavoidable, it's all in one place where future maintainers can understand them fairly easily.

In fact, the C_Importer class consists of only four methods: two override corresponding methods in the Importer class. The other two are pattern-matching helpers. Given the inherent complexity of parsing C, this surely is about as good as it possibly could be.

EKR