Mojo::Log seems to encode twice a message with multibyte characters

[Geunyoung Park]

Hello,

I'm not so good at English, sorry.

While I was rebooting my database server, my Mojolicious server tried to connect it and failed. This is the output via STDERR:

---

$ cat log/app_err.log

DBIx::Class::Storage::DBI::catch {...} (): 

DBI Connection failed: DBI connect('dbname=webstore2;host=192.168.0.170','webstore',...) failed: 

could not connect to server: 호스트로 갈 루트가 없음

---

The last line contains a message in Korean ("No route to host" in English).

At the same time, the logger output the same message but Korean characters incorrectly.

---

$ cat log/production.log

[Fri Jan 19 07:41:29 2018] [error] DBIx::Class::Storage::DBI::catch {...} ():

DBI Connection failed: DBI connect('dbname=webstore2;host=192.168.0.170','webstore',...) failed:

could not connect to server: 호스트로 ê°ˆ 루트가 ì—†ì

---

I copied production.log and found that this is because a message which had been encoded using UTF-8 was encoded again:

---

$ cat test.pl

open my $fh, '<', 't.log';

my $line = <$fh>;

say $line;   # broken characters

my $once = decode('UTF-8', $line);

say $once;  # encoded byte stream

my $again = decode('UTF-8', $once); # decode decoded string again

say $again;  # now I get original Unicode string. (Wide character warning)

$ perl test.pl

[Fri Jan 19 07:41:29 2018] [error] DBIx::Class::Storage::DBI::catch {...} (): DBI Connection failed: DBI connect('dbname=webstore2;host=192.168.0.170','webstore',...) failed: could not connect to server: 호스트로 ê°ˆ 루트가 ì—†ì Œ

[Fri Jan 19 07:41:29 2018] [error] DBIx::Class::Storage::DBI::catch {...} (): DBI Connection failed: DBI connect('dbname=webstore2;host=192.168.0.170','webstore',...) failed: could not connect to server: 호스트로 갈 루트가 없음

Wide character in say at decode.pl line 14, <$fh> line 1.

[Fri Jan 19 07:41:29 2018] [error] DBIx::Class::Storage::DBI::catch {...} (): DBI Connection failed: DBI connect('dbname=webstore2;host=192.168.0.170','webstore',...) failed: could not connect to server: 호스트로 갈 루트가 없음

---

In my opinion, an original message from an external module(DBIx::Class in this case) may be a Unicode string or an encoded bytestream, depending on what the author of the module chose. So Mojo logger have to check whether the message is Unicode or not using a method like Encode::is_utf8.

Of course it may be that I'm wrong and misunderstand the problem.

Thanks.

[Dan Book]

You cannot determine in perl whether a string is intended to be bytes or characters. is_utf8 checks how the string is internally stored, which is irrelevant. You must always pass characters to Mojo::Log.

--
You received this message because you are subscribed to the Google Groups "Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mojolicious+unsubscribe@googlegroups.com.
To post to this group, send email to mojol...@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.