UTF-8 encoding and Mysql

[Rob G]

Hi all,

I've been wrestling with server responses today which don't seem to be
encoding output properly and was hoping someone with a bit more
experience could give some advice! The json response from porpoise was
mangling special characters (like the last letter of café) into two
separate (wrong) "unicode" characters, so I assume double encoding it
or something equally undesirable.

I've tracked down the cause to being the use of

$aResponse = utf8_encode_recursive($aResponse);

in poiserver.class.php. I'm pretty sure I'm missing something as I'm
not sure how it would work otherwise, but the following is my
understanding: When using the SQLPOIConnector, the connection is set
to UTF-8 before anything else happens, and if done correctly the data
should also be stored as UTF-8 meaning all strings received from the
database will be UTF-8 encoded. However, the above function is a
recursive form of the standard utf8_encode($string) function which
expects as input (to quote the php manual) "An ISO-8859-1 string".
This to me sounds like a bad combination.

I'd be grateful if anyone is able to point out what I'm not seeing,
and if there's a better way to fix this than either
a) Commenting out that line - which is my currently solution and
correctly renders the word "café"
Or b) overloading getPOIs to convert all POI data from the database to
ISO-8859-1 which seems completely backwards (I haven't actually tested
this but I assume it would work).

Thanks in advance,
Rob

PS. I checked the data from my DB using both mb_detect_encoding, and
rendering a proper webpage with utf-8 headers just containing a
straight echo of the data both of which showed it to be utf-8 as
expected so I'm pretty sure the issue isn't that end. Probably also
worth noting that I'm not using the PorPOIse dashboard for data input,
but my own web app which is full utf-8 and also displays all the data
correctly.

Additional info:
PHP v5.2.15
MySQL version 5.1.52
Collation used throughout DB is utf_unicode_ci

[Paul Lewis]

Hi Jens,

Can you help out here and answer this problem about the UTF encoding?

Thanks

Paul