Force UTF-8 encoding

239 views
Skip to first unread message

Garrett Lancaster

unread,
Jan 23, 2011, 4:21:06 PM1/23/11
to rubyonra...@googlegroups.com
I'm writing an upload script for contact management, but having encoding
issues:

#=> Encoding::UndefinedConversionError: "\xFA" from ASCII-8BIT to UTF-8

I will not always know the encoding supplied, and this is the result of
`some_string.encode("UTF-8")`

I have also tried .force_encoding("UTF-8") to no avail. For now I'm
simply trying to bypass invalid characters by replacing them with ? or
removing them completely so Postgres doesn't blow up on insert.

Any help or advice would be appreciated,
Garrett Lancaster

Mickael Gerard

unread,
Jan 23, 2011, 6:23:56 PM1/23/11
to rubyonra...@googlegroups.com
Hi,

In order to solve this issue, I use the chardet gem https://rubygems.org/gems/chardet to identify the file encoding and then I iconv it to utf-8.

Mickael 


--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To post to this group, send email to rubyonra...@googlegroups.com.
To unsubscribe from this group, send email to rubyonrails-ta...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.


Reply all
Reply to author
Forward
0 new messages