Topodroid inserting spurious NULL chars before each char on label point texts

[Rodrigo Severo]

Hi Marco,


I was debugging the following issue in Mapiah:

When I tried to add some text to a point of type label created by Topodroid (tested with 6.5.5 and 6.5.22) which already had some text set on its text option, the text was completely replaced by the text I was trying to add instead of simply being suffixed.

I found out that Topodroid is adding stray NULL chars before each text char. According to Claude the issue seems to be that:

"tamboril-1G1-2p.th2 contains -text "\x00T\x00B\x00C\x00M" — a byte before every character, the classic signature of UTF-16 content that got written into a UTF-8 file. Reading that raw \u0000 into the text field is what confused the native text-input layer on focus/typing, producing the select-and-replace symptom"

I fixed Mapiah so it can deal with this kind of spurious NULL chars but I believe Topodroid should be fixed so it won't add these surious NULL chars.

I am sending one of the original th2 files Topodroid created.


Regards,

Rodrigo

[Marco Corvi]

thanks.

please provide also survey zip archives.

--
You received this message because you are subscribed to the Google Groups "TopoDroid" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topodroid+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/topodroid/CANpCgadMhVfcr6F2cOLGb%3DJiFGMDzyFmUWmoNRrJQQdqQvEhhg%40mail.gmail.com.

[Rodrigo Severo]

A zip archive with the issue. 

To view this discussion visit https://groups.google.com/d/msgid/topodroid/CAJiQYo2GOQzDbdmuM9JWof_XNA0TsEWwmxTQ%2B-0Eby7JaC214A%40mail.gmail.com.

[Marco Corvi]

thanks.

the label text in the topodroid tdr file are 12 bytes long.

exactly

c0 80 54 c0 80 42 c0 80 43 c0 80 4d

 

54 42 43 and 4d are T B C M

how did you managed to type the other bytes?

"c0 80" represent the nul character, and

when the string is formatted to the writer,

it is written 00 (utf-8).

i think it is ok that the program writes the text you entered.

therefore the problem is how that "c0 80" crept in.

To view this discussion visit https://groups.google.com/d/msgid/topodroid/CANpCgafx_t92aZrVdsG%2BGBwGJA7TESorYE7TVPB43eGhDN1j%2Bw%40mail.gmail.com.

[Rodrigo Severo]

Em ter., 22 de set. de 2026 às 14:35, Marco Corvi <marco...@gmail.com> escreveu:

thanks.

the label text in the topodroid tdr file are 12 bytes long.

exactly

c0 80 54 c0 80 42 c0 80 43 c0 80 4d

 

54 42 43 and 4d are T B C M

how did you managed to type the other bytes?

Just using Topodroid and typing the text label for the label I needed to create during a mapping session inside a cave. I didn't do anything fancy at all as far as I can tell.

It looks to me that Topodroid wrote my text label with those NULL char prefixes.

Isn´t Topodroid dealing with texts as UTF-16 inside it? AFAICT, Java char / String representation is UTF-16.

I believe you forgot to properly convert the internal UTF-16 representation to UTF-8 when writing the text label.

Please let me know if there is any other way I might help.

Rodrigo

[Marco Corvi]

On Tue, Sep 22, 2026, 22:16 Rodrigo Severo <rse...@gmail.com> wrote:

Em ter., 22 de set. de 2026 às 14:35, Marco Corvi <marco...@gmail.com> escreveu:

thanks.

the label text in the topodroid tdr file are 12 bytes long.

exactly

c0 80 54 c0 80 42 c0 80 43 c0 80 4d

 

54 42 43 and 4d are T B C M

how did you managed to type the other bytes?

Just using Topodroid and typing the text label for the label I needed to create during a mapping session inside a cave. I didn't do anything fancy at all as far as I can tell.

It looks to me that Topodroid wrote my text label with those NULL char prefixes.

Isn´t Topodroid dealing with texts as UTF-16 inside it?

topodroid uses android default encoding

AFAICT, Java char / String representation is UTF-16.

yes, internally java strings are encoded utf-16

I believe you forgot to properly convert the internal UTF-16 representation to UTF-8 when writing the text label.

Please let me know if there is any other way I might help.

what is your keyboard app ?

Rodrigo

--
You received this message because you are subscribed to the Google Groups "TopoDroid" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topodroid+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/topodroid/CANpCgafXsBU8ttW0bgy7ddPcpNGea6bprb3d4ucNVvJwsD7Kcw%40mail.gmail.com.

[Rodrigo Severo]

Em ter., 22 de set. de 2026 às 17:44, Marco Corvi <marco...@gmail.com> escreveu:

On Tue, Sep 22, 2026, 22:16 Rodrigo Severo <rse...@gmail.com> wrote:

Em ter., 22 de set. de 2026 às 14:35, Marco Corvi <marco...@gmail.com> escreveu:

thanks.

the label text in the topodroid tdr file are 12 bytes long.

exactly

c0 80 54 c0 80 42 c0 80 43 c0 80 4d

 

54 42 43 and 4d are T B C M

how did you managed to type the other bytes?

Just using Topodroid and typing the text label for the label I needed to create during a mapping session inside a cave. I didn't do anything fancy at all as far as I can tell.

It looks to me that Topodroid wrote my text label with those NULL char prefixes.

Isn´t Topodroid dealing with texts as UTF-16 inside it?

topodroid uses android default encoding

AFAICT, Java char / String representation is UTF-16.

yes, internally java strings are encoded utf-16

I believe you forgot to properly convert the internal UTF-16 representation to UTF-8 when writing the text label.

Please let me know if there is any other way I might help.

what is your keyboard app ?

For this particular topo I used a cell phone from a friend (which is in another city now) so I can't tell which one it is nor if he uses something exotic but I typed several other things in Topodroid on this same topo and same cell phone (team member names and station names for example) that didn't have this issue. It looks to me as if it's something restricted to the text label texts.

Rodrigo

 

[Rodrigo Severo]

Claude considerations on this issue:

------------------------------------------------
Found it: DrawingWindow.onKeyDown appends every key-down event to the label, including Shift, whose getUnicodeChar() returns 0 and gets inserted as \0 before capitals—explaining the \0T\0B\0C\0M pattern and why it's label-only, not a UTF-16 issue. Now checking the load path too.

  Ran 1 shell command

Now the fix. I'll skip non-printing key events, and also remove existing NULs when a .tdr is loaded so sketches that are already affected get cleaned up.

● Update(src/com/topodroid/TDX/DrawingWindow.java)
Added 1 line, removed 2 lines
        if ( mLabelPath != null ) {
          int code_point = ev.getUnicodeChar();
          // TDLog.v("key event unicode " + code_point );
     -     // TODO suppress invalid characters
          if ( code == KeyEvent.KEYCODE_DEL ) {
            mLabelPath.delTextChar( );
     -     } else {
     +     } else if ( code_point > 0 && ! Character.isISOControl( code_point ) ) { // skip non-printing key
s (eg, SHIFT gives 0), and dead keys (COMBINING_ACCENT flag makes it negative)
            mLabelPath.addTextChar( Character.toChars( code_point ) );
          }
          // mDrawingSurface.setBackgroundColor( 0 ); // this makes grey for all modes

● Update(src/com/topodroid/TDX/DrawingLabelPath.java)
Added 1 line, removed 1 line
      scale = dis.readInt( );
      if ( version > 401090 ) level = dis.readInt();
      if ( version > 401160 ) scrap = dis.readInt();
  -    text = dis.readUTF();
  +    text = dis.readUTF().replace( "\u0000", "" ); // drop NUL chars stored by older versions
      options = dis.readUTF();

      // TDLog.Log( TDLog.LOG_PLOT, "Label <" + text + " " + ccx + " " + ccy + " scale " + scale + " (" + op
tions + ")" );

  Ran 1 shell command

I found the bug and fixed it. The UTF-16 explanation turned out to be wrong: TopoDroid is adding the NULs itself when you type a label directly on the sketch. I haven't compiled or tested the change on a device.

Cause: DrawingWindow.onKeyDown (src/com/topodroid/TDX/DrawingWindow.java:8921) appends every key-down event to the label being typed, with no filter (there was even a // TODO suppress invalid characters there). Pressing Shift is a key-down event of its own, and ev.getUnicodeChar() returns 0 for it. So Character.toChars(0) added a \0 before each capital letter. Typing "TBCM" with Shift gives exactly \0T\0B\0C\0M.

This is why only labels are affected: this typing path is used only by DrawingLabelPath. The .th2 writer and escapeQuotes just write out whatever text they're given.

The same unfiltered call also had a second, latent bug. Dead keys (accent keys) return a negative value with the COMBINING_ACCENT flag set, and Character.toChars() throws an IllegalArgumentException on that.

What I changed:
1. DrawingWindow.java: a key's character is now appended only if it is greater than 0 and is not a control character. This skips Shift, Ctrl and similar keys, and dead keys. Delete still works as before.
2. DrawingLabelPath.java:98: NUL characters are removed from label text when a .tdr sketch is loaded. Sketches saved by 6.5.22 and earlier then export clean .th2 files the next time they're saved.

------------------------------------------------

I didn't test it because I don't have a working topodroid development environment running right now.

Rodrigo

[Balambér Hakapesz]

I tried it, and it works.
That must have been the problem; I had a null character in the export too.

B.

--

You received this message because you are subscribed to the Google Groups "TopoDroid" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topodroid+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/topodroid/CANpCgaeM-NyP-NADreEcYgEwPOjfnfZY6Uwacd0McSjgP_hQ_g%40mail.gmail.com.

[Marco Corvi]

thanks.

v. 6.5.23 includes the fix suggested by claude

claude is good at spotting the problem: although it started off, it was quick to correct and point to the target (and it admitted it).

the bug entered with in-canvas labels (v. 6.4.38 last may).

there was a TODO left in in the source code,

that i have never addressed in the meanwhile, and has now been solved.

To view this discussion visit https://groups.google.com/d/msgid/topodroid/CAPq666MrazMUH3knfHqdngL9QddjX3PyuUUcFAB5dJdDUeRN9w%40mail.gmail.com.