Comments ignored in okf_xml and okf_xmlstream

26 views
Skip to first unread message

Mihai Nita

unread,
Sep 29, 2021, 1:47:24 PM9/29/21
to okapi-users
Hi all,

For me it started with okf_xml-AndroidStrings

The xml comments are ignored, so something like this:
<resources xmlns:xliff="urn:oasis:names:tc:xliff:document:1.2">
  <!-- Some comment -->
  <string name="theId">This is the message</string>
</resources>
produces a TextUnit, but there is no note, not annotation, nothing

You can try tikal -x with -fc
I tried okf_xmlstream, okf_xml, and okf_xml-AndroidStrings

Compare with a test.properties
# Some comment
theId = This is the message
The resulting trans-unit  in the extracted xlf has a <note> with the comment.

I've tried adding support for comments to okf_xml using <its:translateRule>, but I didn't manage to make it work. I tried all kind of combinations, too many to list here (and didn't keep track what I tried, unfortunately)

Any ideas?

Thanks a lot!
Mihai

Yves Savourel

unread,
Sep 29, 2021, 11:59:32 PM9/29/21
to Mihai Nita, okapi-users

Hi Mihai,

 

The data category to use for notes in locNoteRule. The translateRule element is just to tell what is translatable or not  (if needed).

Use the selector attribute of locNoteRule to point to the translatable node you want to comment on and the locNotePointer attribute (or one of the other “locNote*” attributes) to tell where to get the comment from.

 

For example, in the ITS rules below we indicate that the text-unit extracted from //msg/data should use any comments that is a sibling node.

(There is no need to specify a translateRule here because <data> will be extracted by default):

 

<?xml version="1.0"?>

<Res>

  <prolog>

    <its:rules xmlns:its=http://www.w3.org/2005/11/its version="2.0">

      <its:locNoteRule locNoteType="description" selector="//msg/data" locNotePointer="../comment()"/>

    </its:rules>

  </prolog>

  <body>

    <msg id="FileNotFound">

      <!-- Indicates that the resource file {0} could not be loaded.-->

      <data>Cannot find the file {0}.</data>

    </msg>

  </body>

</Res>

 

So you get the following with “tikal -x test.xml”:

 

<trans-unit id="2" its:locNote=" Indicates that the resource file {0} could not be loaded." its:locNoteType="description">

<source xml:lang="en">Cannot find the file {0}.</source>

<target xml:lang="fr">Cannot find the file {0}.</target>

<note> Indicates that the resource file {0} could not be loaded.</note>

</trans-unit>

 

This example is adapted from the example 31 in the ITS 2.0 specification: https://www.w3.org/TR/its20/#locNote-implementation

I think the XML Filter does implement the full locNoteRule, even the things like locNoteRefPointer.

The tests results are shown in Localization Note section of the ITS 2.0 Test suite Dashboard (column “enlaso” is the Okapi implementation).

 

One more thing: Don’t forget to use namespaces in the selectors if your source file has namespaces. People often forget about them and wonder why the rule doesn’t work.

 

I hope this helps.

-yves

--
You received this message because you are subscribed to the Google Groups "okapi-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to okapi-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/okapi-users/CAK69zbkC2f4iJ6DbS8x2c0H%2BMXMmMQ7%3DGjmV01ebNzJ3j6kqhg%40mail.gmail.com.

Mihai Nita

unread,
Sep 30, 2021, 10:32:00 AM9/30/21
to Yves Savourel, okapi-users
Thank you Yves!

My description (in the email) was wrong, I was in fact using locNoteRule, I just copied the wrong tag from my editor.
And I got very darn close, at some point.
The mistake was that I was using "comment" instead of "comment()"

Next question: should that rule be there be default?
At least Android has no other documented way to add comments.

And no way to differentiate between comments for translators vs comments for developers.
Similar to Java .properties.

Thanks,
Mihai

phsteger

unread,
Sep 30, 2021, 12:07:41 PM9/30/21
to okapi-users
Hi all,

regarding the Android resource XML, I had similar issues when I tried to consider the comments in our Android resource files.
According to the its rules I tried

<its:locNoteRule selector="//string" locNotePointer="../comment()" locNoteType="description" />

but, surprisingly, this did not work as expected.
This rule found the very first comment of the file preceding a given <string> element but then applied this comment all over, i.e., to all subsequent <string> elements with a comment within the file.
After some research this is how I got it to work so that each comment was applied to/considered for its corresponding string element independently from its position in the file:

<its:locNoteRule selector="//string" locNotePointer="preceding-sibling::node()[not(self::text()[not(normalize-space())])][1][self::comment()]" locNoteType="description" />

(the normalization part might not be part of this solution)

My expectation would have been that the first locNoteRule above should work.
I'd have to investigate if the described behavior (keep the first comment found in "memory") is a bug and why/where this happens.
I run the XML filter from my IDE using <version>1.42.0-SNAPSHOT</version>.

As I said, I didn't have time yet to investigate further and make sure it really is a bug or myself applying the rules incorrectly.
If you think and can judge without further ado that this is a bug I could submit an issue with a detailed description in bitbucket.

Kind regards
Philipp

phsteger

unread,
Sep 30, 2021, 12:10:18 PM9/30/21
to okapi-users
Ah, @Mihai, 
and yes, as you can see, others would have found it quite beneficial if it was part of the default.
Definitely, it should be part of the default.

Kind regards,
Philipp

Reply all
Reply to author
Forward
0 new messages