recent MarkDownFilter changes...

[jim]

The recent PR's that were merged is failing in my PR. I think it's
because I added smarter code to convert any empty TextUnits to
DocumentParts. So when the MarkDownFilter sends in single inline tags,
since they have no text, they are converted to DocumentParts. Normally
this is the behavior we want to reduce workbench clutter - but this will
be a problem for the MarkDownFilter.

Maybe this kind of aggressive cleanup shouldn't happen in the filter?
Adding a cleanup step would be one solution.

Any thoughts?


Jim

[Chase Tingley]

Is it definitely Markdown as opposed to MIF?  The MIF changes that I merged this morning from Denis did produce some additional segments to be extracted that contained only a single code, but I don't remember seeing anything like that in the markdown PRs (which had to do with paired code handling).

Centralizing code to convert tag-only segments to DPs seems ok, although I would want to double-check that there aren't any issues with subfiltered content that could break on merge.

Can you post the tests that failed?

--
You received this message because you are subscribed to the Google Groups "okapi-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to okapi-devel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/okapi-devel/5b56d63e-b577-e6bd-257f-2f1bcc2a97df%40gmail.com.

[jim]

The problem ended up being the code I added that converts more types of TextUnits to DPs. The MarkDownFilter doesn't like this as it depends on each inline tag being in a separate TextUnit (which my code is now smart enough to convert to DPs).

I now remember an earlier email where we discussed this and a logged issue by Mihai. It sounds like this won't be easy to fix on the MarkDownFilter side.

https://bitbucket.org/okapiframework/okapi/issues/928/the-markdown-filter-does-not-handle-html

Jim

To view this discussion on the web visit https://groups.google.com/d/msgid/okapi-devel/CAGRYq4jbH8rVC1%2B5yn_S4coyF5EiMm73yPe6SXRKwxibmWtGTw%40mail.gmail.com.

[Chase Tingley]

Interesting. The PRs we just merged resolve a lot of of the cases highlighted in that issue (although not the underlying architectural issue - that we pass the subfilter tokens in isolation).  Is the way to reproduce these failures to run the tests on your branch?

[jim]

Chase, I think I'm going to stash these current changes. It's not worth destabilizing another filter just to get the last few empty TU's. I can create a step to do the same which can then be used by all filters.

Subfiltering in general seems to be an issue with MarkDownFilter - the main problem being accumulating the content (HTML, YAML etc.) to send to the subfilter.  I can create a general ticket for this so we have it as a placeholder.

Jim