Mojo::JSON and utf8 troubles

[Ilya Skorik (RU)]

This code with russian string:

use utf8;
my $json = Mojo::JSON->new;
my $string = { string => "это тест на русском" };
my $jsonString = $json->encode($string);

$string->{string} is valid utf8 value. I have troubles with:

# Unicode
return b($string)->encode('UTF-8')->to_string;

in Mojo::JSON.pm module.

The string's encoding is break and instead of the Russian text I see
symbols like Обновить Ñ Ñ‚Ñƒ панÐ. If I remove a line
"b($string)->encode('UTF-8')->to_string;" , all works well.

This issue for git version, in cpan all works well.

[Sebastian Riedel]

> The string's encoding is break and instead of the Russian text I see
> symbols like Обновить Ñ Ñ‚Ñƒ панÐ. If I remove a line
> "b($string)->encode('UTF-8')->to_string;" , all works well.

Then you are double encoding it somewhere in *your* code which you've not shown here, Mojo::JSON generates valid UTF-8 encoded JSON which needs to be treated as binary data by you.

--
Sebastian Riedel
http://labs.kraih.com
http://mojolicious.org
http://twitter.com/kraih

[Ilya Skorik]

The identical code works in cpan versions and does not work in the version from git. Also works if to remove $b ($string)-> encode (' UTF-8 ')-> to_string.

2010/7/29 Sebastian Riedel <kra...@googlemail.com>

--
You received this message because you are subscribed to the Google Groups "Mojolicious" group.
To post to this group, send email to mojol...@googlegroups.com.
To unsubscribe from this group, send email to mojolicious...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mojolicious?hl=en.

[Sebastian Riedel]

> The identical code works in cpan versions and does not work in the version from git. Also works if to remove $b ($string)-> encode (' UTF-8 ')-> to_string.

There were no changes as far as i can see, could you please write a full unit test if you're sure that there's a bug?

[Ilya Skorik]

This is controller code:

package App::Locale;

use utf8;
use strict;
use warnings;

use base 'App::BaseController';

sub index {

    my $c = shift;

   
    my $json   = Mojo::JSON->new;
   

    my $string = { string => "Мама мыла раму" };

    my $jsonString = $json->encode($string);
   

    #$c->tx->res->headers->header('Content-Type' => "text/javascript; charset=utf-8;");
    $c->render_text($jsonString);
}

1;

With the git version in responce - {"string":"Мама мыла раму"}
With Mojolicious-0.999926 - {"string":"Мама мыла раму"}

2010/7/29 Ilya Skorik <ilya....@gmail.com>

[Sebastian Riedel]

> $c->render_text($jsonString);

Thats your problem, let me quote from the render_text documentation.

"Render the givent content as plain text, note that text will be encoded."

What you really want is render_data.

"Render binary data, similar to C<render_text> but data will not be encoded."

It's the old Perl chars vs bytes problem.

[Ilya Skorik]

Thank you!

The method render_data work well!

2010/7/29 Sebastian Riedel <kra...@googlemail.com>