recent MarkDownFilter changes...

4 views
Skip to first unread message

jim

unread,
Nov 11, 2021, 3:52:19 PM11/11/21
to Group: okapi-devel
The recent PR's that were merged is failing in my PR. I think it's
because I added smarter code to convert any empty TextUnits to
DocumentParts. So when the MarkDownFilter sends in single inline tags,
since they have no text, they are converted to DocumentParts. Normally
this is the behavior we want to reduce workbench clutter - but this will
be a problem for the MarkDownFilter.

Maybe this kind of aggressive cleanup shouldn't happen in the filter?
Adding a cleanup step would be one solution.

Any thoughts?


Jim

Chase Tingley

unread,
Nov 11, 2021, 4:07:44 PM11/11/21
to okapi...@googlegroups.com
Is it definitely Markdown as opposed to MIF?  The MIF changes that I merged this morning from Denis did produce some additional segments to be extracted that contained only a single code, but I don't remember seeing anything like that in the markdown PRs (which had to do with paired code handling).

Centralizing code to convert tag-only segments to DPs seems ok, although I would want to double-check that there aren't any issues with subfiltered content that could break on merge.

Can you post the tests that failed?

--
You received this message because you are subscribed to the Google Groups "okapi-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to okapi-devel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/okapi-devel/5b56d63e-b577-e6bd-257f-2f1bcc2a97df%40gmail.com.

jim

unread,
Nov 11, 2021, 6:57:36 PM11/11/21
to okapi...@googlegroups.com, Chase Tingley
The problem ended up being the code I added that converts more types of TextUnits to DPs. The MarkDownFilter doesn't like this as it depends on each inline tag being in a separate TextUnit (which my code is now smart enough to convert to DPs).

I now remember an earlier email where we discussed this and a logged issue by Mihai. It sounds like this won't be easy to fix on the MarkDownFilter side.

https://bitbucket.org/okapiframework/okapi/issues/928/the-markdown-filter-does-not-handle-html

Jim

Chase Tingley

unread,
Nov 11, 2021, 8:57:48 PM11/11/21
to jim, okapi...@googlegroups.com
Interesting. The PRs we just merged resolve a lot of of the cases highlighted in that issue (although not the underlying architectural issue - that we pass the subfilter tokens in isolation).  Is the way to reproduce these failures to run the tests on your branch?

jim

unread,
Nov 12, 2021, 10:04:00 AM11/12/21
to Chase Tingley, okapi...@googlegroups.com
Chase, I think I'm going to stash these current changes. It's not worth destabilizing another filter just to get the last few empty TU's. I can create a step to do the same which can then be used by all filters.

Subfiltering in general seems to be an issue with MarkDownFilter - the main problem being accumulating the content (HTML, YAML etc.) to send to the subfilter.  I can create a general ticket for this so we have it as a placeholder.

Jim
Reply all
Reply to author
Forward
0 new messages