Does Ruby 1.9 support Unicode normalization yet?

[Sam Grönblom]

In the process of upgrading from 1.8 to 1.9 we are getting a lot of
warnings about "Ruby 1.9 doesn't support Unicode normalization yet".
However the commit that added those lines is from 2008 and just
mentions "Ruby 1.9 compat: no Unicode normalization support yet"
without any references. Does anyone know whether this is still true
for ruby 1.9 and for which minor versions of 1.9?

[Aaron Patterson]

Ruby doesn't support unicode normalization natively, but you can use
this gem:

https://github.com/knu/ruby-unf

Hope that helps.

--
Aaron Patterson
http://tenderlovemaking.com/

[Sam Grönblom]

So using Iconv for transliteration is not working properly in 1.9 or?

[Aaron Patterson]

On Tue, Mar 13, 2012 at 09:10:34PM -0700, Sam Grönblom wrote:
> So using Iconv for transliteration is not working properly in 1.9 or?

You can still use it, but AFAIK iconv support is going to be dropped.
The encoding system in 1.9 will let you convert between encodings just
fine, but it doesn't do transliteration.

Probably the best path would either be to extract the iconv extension
from Ruby and provide it as a gem, or to use an existing gem like this
one:

https://github.com/rsl/stringex

I know it's not the ideal answer, but I hope it helps.

[NARUSE, Yui]

2012/3/14 Sam Grönblom <sgro...@gmail.com>:

> So using Iconv for transliteration is not working properly in 1.9 or?

iconv's //TRANSLIT is not Unicode normalization.

Moreover iconv translit works only with glibc and GNU libiconv,
and the actual behavior is not documented.

--
NARUSE, Yui  <nar...@airemix.jp>

[Aaron Patterson]

On Thu, Mar 15, 2012 at 04:47:21AM +0900, NARUSE, Yui wrote:
> 2012/3/14 Sam Grönblom <sgro...@gmail.com>:
> > So using Iconv for transliteration is not working properly in 1.9 or?
>
> iconv's //TRANSLIT is not Unicode normalization.
>
> Moreover iconv translit works only with glibc and GNU libiconv,
> and the actual behavior is not documented.

Ah. Thanks for the clarification!