Issue 1113 in tungsten-replicator: Fixmysqlstrings.js filter does not correctly translate binary with trailing x'00' characters

[tungsten-...@googlecode.com]

Status: Accepted
Owner: robert.h...@continuent.com
Labels: Type-Defect Priority-High FoundIn-3.0.0 FixedIn-3.0.0

New issue 1113 by robert.h...@continuent.com: Fixmysqlstrings.js filter
does not correctly translate binary with trailing x'00' characters
https://code.google.com/p/tungsten-replicator/issues/detail?id=1113

What steps will reproduce the problem?

1. Set up master/slave replication with following filters enabled on
extract: colnames,fixmysqlstrings,pkey. (This si a common data warehouse
extraction pattern.)

2. Execute the following query on the master.

use test
set session binlog_format=row;
drop table if exists uuid_data;
create table uuid_data(id int primary key, d char(10), uuid binary(16));
insert into uuid_data values(1, 'hi', x'DD9D7EFFFBD247CAB3EF6167B0C4AD01');
insert into uuid_data values(2, 'bye', x'DD9D7EFFFBD247CAB3EF6167B0C4AD00');
insert into uuid_data values(3, 'bye', x'DD9D7EFFFBD247CAB3EF6167B0C4AD10');
insert into uuid_data values(4, 'bye', x'DD9D7EFFFBD247CAB300000000000000');
select id,d,hex(uuid) from uuid_data;

3. Check the values inserted into the THL for each transaction.

What is the expected output?

THL should show values like the following:

SEQ# = 112827 / FRAG# = 0 (last frag)
- TIME = 2015-04-09 11:40:12.0
- EPOCH# = 0
- EVENTID = mysql-bin.017264:0000000041379007;-1
- SOURCEID = logos1
- METADATA =
[mysql_server_id=1;dbms_type=mysql;service=batch;shard=test;strings=utf8]
- TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
- OPTIONS = [foreign_key_checks = 1, unique_checks = 1]
- SQL(0) =
- ACTION = INSERT
- SCHEMA = test
- TABLE = uuid_data
- ROW# = 0
- COL(1: id) = 2
- COL(2: d) = bye
- COL(3: uuid) = DD9D7EFFFBD247CAB3EF6167B0C4AD00
- KEY(1: id) = NULL

What do you see instead?

Trailing zeros on the uuid field are truncated.

What is the possible cause?

The operation to convert to hex strings uses
javax.xml.bind.DatatypeConverter.printHexBinary() does not handle trailing
null bytes.

What is the proposed solution?

Look up column metadata and pad the value out to the expected size for
BINARY but not VARBINARY.

Additional information

Fixed in maintenance 3.0 branch and trunk.
...

Use labels and text to provide additional information.


--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[tungsten-...@googlecode.com]

Comment #1 on issue 1113 by robert.h...@continuent.com: Fixmysqlstrings.js

filter does not correctly translate binary with trailing x'00' characters
https://code.google.com/p/tungsten-replicator/issues/detail?id=1113

This issue was updated by revision r2810.


Added support for padding BINARY column values to add trailing null
characters when converting to hex.

[tungsten-...@googlecode.com]

Comment #2 on issue 1113 by robert.h...@continuent.com: Fixmysqlstrings.js

filter does not correctly translate binary with trailing x'00' characters
https://code.google.com/p/tungsten-replicator/issues/detail?id=1113

This issue was updated by revision r2811.

Removed comment accidentally left in from previous fix.