Problems with utf8 characters in OrientDb Studio

[Marko Radinovic]

Hi,

i'm currently migrating my project from PostgreSql to OrientDB. I
created class CdGender with structure:

"name": "CdGender",
"properties":{
"idCdGender":{
"name": "idCdGender",
"type": "INTEGER",
"mandatory": false,
"notNull": true,
"min": null,
"max": null
},
"gender":{
"name": "gender",
"linkedType": "STRING",
"type": "EMBEDDEDMAP",
"mandatory": false,
"notNull": false,
"min": null,
"max": null
}

Property "gender" contains embeddedmap with i18n text values.

{
"@type": "d", "@rid": "#5:3", "@version": 0, "@class":
"CdGender",
"gender": {"en": "Male"},
"idCdGender": 1
},

When I insert new record in CdGender class, if value contains specific
chars (Serbian language Š,š,đ,Đ, ć, Ć...), orientdb studio reports
error on Query.

Query: select from CdGender

Error: Error: { "schema": {
"name": "CdGender",
"properties":{
"idCdGender":{
"name": "idCdGender",
"type": "INTEGER",
"mandatory": false,
"notNull": true,
"min": null,
"max": null
},
"gender":{
"name": "gender",
"linkedType": "STRING",
"type": "EMBEDDEDMAP",
"mandatory": false,
"notNull": false,
"min": null,
"max": null
}
}
},
"result": [{
"@type": "d", "@rid": "#5:2", "@version": 0, "@class":
"CdGender",
"gender": {"sr_Latn": "Muški", "en": "Male"},
"idCdGender":1
}, {
"@type": "d", "@rid": "#5:3", "@version": 0, "@class":
"CdGender",
"gender": {"en": "Female", "sr_Latn":"Ženski"},
"idCdGender": 2
}
]

When I execute query in console, i don't get any errors.

Is this bug in orientdb studio, or I need to configure orientdb to
support utf8 strings?

[Luca Garulli]

Hi,

what kind of error returns?

What release are you using?

Lvc@

[Marko Radinovic]

Hi,
in orientdb studio, I get in output field just json string :

Error: { "schema": {
"name": "CdGender",
"properties":{
"idCdGender":{
"name": "idCdGender",
"type": "INTEGER",

.........

without any exception log.

Also, I can't type in orientdb console any of Serbian unicode
characters.

In linux terminal I can type unicode characters and my environment
variables are properly configured.

I'm using Ubuntu 11.10, OrientDB 1.0rc7 SNAPSHOT.

On 31 окт, 15:27, Luca Garulli <l.garu...@gmail.com> wrote:
> Hi,
> what kind of error returns?
>
> What release are you using?
>
> Lvc@
>

[Luca Garulli]

Hi,

some weeks ago I've tried with Italian special characters and worked. Can you monitor the HTTP request? is it UTF-8?

Lvc@

[Marko Radinovic]

Hi,

I trace HTTP communication with Wireshark and here are results:

GET /query/social/sql/select+from+CdGender%2F20 HTTP/1.1
Host: localhost:2480
Connection: keep-alive
Authorization: Basic YWRtaW46YWRtaW4=
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML,
like Gecko) Chrome/15.0.874.106 Safari/535.2
Accept: */*
Referer: http://localhost:2480/studio/
Accept-Encoding: gzip,deflate,sdch
Accept-Language: sr,hr-HR;q=0.8,hr;q=0.6,en-US;q=0.4,en;q=0.2
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: OSESSIONID=OS13201557941975258201513135842167

HTTP/1.1 200 OK
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Date: Tue Nov 01 14:56:55 CET 2011
Content-Type: application/json
Server: OrientDB Server v.1.0rc7-SNAPSHOT (build 10765)
Connection: Keep-Alive
Set-Cookie: OSESSIONID=OS13201557941975258201513135842167; Path=/;
HttpOnly
Content-Length: 617

{ "schema": {
"name": "CdGender",
"properties":{
"idCdGender":{
"name": "idCdGender",
"type": "INTEGER",
"mandatory": false,
"notNull": true,
"min": null,
"max": null
},

"cdValue":{
"name": "cdValue",

"type": "EMBEDDEDMAP",
"mandatory": false,
"notNull": false,
"min": null,
"max": null
}
}
},
"result": [{

"@type": "d", "@rid": "#8:0", "@version": 0, "@class":
"CdGender",
"idCdGender": 1,
"cdValue": {"en": "Male", "sr_Latn": "Mu..ki"}
}
]
}

In server response i get this "Mu..ki" instead of "Muški".

Best regards

On 1 нов, 12:43, Luca Garulli <l.garu...@gmail.com> wrote:
> Hi,

> some weeks ago I've tried with Italian special characters and worked. Can
> you monitor the HTTP request? is it UTF-8?
>
> Lvc@
>

[Marko Radinovic]

I have also try on Windows 7, and I get the same problem.

[Luca Garulli]

Hi,

can you try to add this as parameter of the JVM?

-Dfile.Encoding = UTF8

Just edit server.bat and change the launch of java with:

call %JAVA% -server %JAVA_OPTS% -XX:+UseParallelGC -XX:+AggressiveOpts -XX:CompileThreshold=200 -Dfile.Encoding = UTF8 -Djava.util.logging.config.file="%LOG_FILE%" -Dorientdb.config.file="%CONFIG_FILE%" -Dorientdb.www.path="%WWW_PATH%" -Dlog.console.level=%LOG_CONSOLE_LEVEL% -Dlog.file.level=%LOG_FILE_LEVEL% -Dorientdb.build.number=@BUILD@ -cp "%ORIENTDB_HOME%\lib\*" com.orientechnologies.orient.server.OServerMain %CMD_LINE_ARGS%

Lvc@

[Marko Radinovic]

Hi,

I have tried that parameter, but it didn't help.

On 1 нов, 16:17, Luca Garulli <l.garu...@gmail.com> wrote:
> Hi,

> can you try to add this as parameter of the JVM?
>
> -Dfile.Encoding = UTF8
>
> Just edit server.bat and change the launch of java with:
>
> call %JAVA% -server %JAVA_OPTS% -XX:+UseParallelGC -XX:+AggressiveOpts

> -XX:CompileThreshold=200 *-Dfile.Encoding = UTF8*

> -Djava.util.logging.config.file="%LOG_FILE%"
> -Dorientdb.config.file="%CONFIG_FILE%" -Dorientdb.www.path="%WWW_PATH%"
> -Dlog.console.level=%LOG_CONSOLE_LEVEL% -Dlog.file.level=%LOG_FILE_LEVEL%
> -Dorientdb.build.number=@BUILD@ -cp "%ORIENTDB_HOME%\lib\*"
> com.orientechnologies.orient.server.OServerMain %CMD_LINE_ARGS%
>
> Lvc@
>

[Luca Garulli]

Hi,

by reading the W3C specs about HTTP I found a "charset" header in the response. I've changed the HTTP response to always return this new parameter ("UTF-8" by defalut), but you can change it as with other parameters: http://code.google.com/p/orient/wiki/PerformanceTuning#Parameters

SVN r4079.

Let me know if fixes.

Lvc@

[Marko Radinovic]

Hi,

I tried that (I downloaded new source and build and configure new
parameter in server configuration file) but no changes.

I tried today to execute query from "Raw access" option in orientdb
studio, and I get correct response.

"result": [{
"@type": "d", "@rid": "#5:0", "@version": 0, "@class": "CdGender",
"idCdGender": 1,
"cdValue": {"sr_Latn": "Muški", "en": "Male"}
}, {
"@type": "d", "@rid": "#5:1", "@version": 0, "@class": "CdGender",
"idCdGender": 2,
"cdValue": {"sr_Latn": "Ženski", "en": "Female"}
}

I think that problem isn't with server source but with orientdb studio
javascript code.

Also, in my project (web application using GWT framework), all strings
mapped in pojo classes are correct.

Best regards

On 2 нов, 09:59, Luca Garulli <l.garu...@gmail.com> wrote:
> Hi,

> by reading the W3C specs about HTTP I found a "charset" header in the
> response. I've changed the HTTP response to always return this new
> parameter ("UTF-8" by defalut), but you can change it as with other
> parameters:http://code.google.com/p/orient/wiki/PerformanceTuning#Parameters
>
> SVN r4079.
>
> Let me know if fixes.
>
> Lvc@
>

[Luca Garulli]

Hi,

at this point seems a problem of the Browser/Client configuration.

Lvc@