UTF-8 possible?

[Russell]

Hi,

I tried to insert some unicode characters such as Chinese or Japanese
data into the tables. When I do "select * " after that, those data are
all "????". Is there any wat to create a table using characterEncoding
parameter?

thx,

Russell

[Gabriel Wong]

Hi,
Could you please provide sample code?
Regards

[RH]

Hi,

I simply used the "jiql.jsp" downloaded from www.jiql.org to create table, insert into table, then select * from that table.

the sql I used could be something like following:

1. creating table:

create table ttest (id bigint, name varchar(128))

2. inserting to table:

insert into ttest (id, name) values(1, '測試名稱')

3. retrieving:

select * from ttable

the results of the "select *" are someting like following

id name
1 ????

the platform is google app engine. I coded the project on eclipse using google's GAE tools, then upload to google's appspot.com.

I tried to add the database.yml with the following contents under "WEB-INF" folder of my GAE project:

production:
adapter: jdbc
driver: org.jiql.jdbc.Driver
url: jdbc:jiql:local
username:
password:
encoding: utf8

It doesn't work either.

Thanks,

Russell

> --
> You received this message because you are subscribed to the Google Groups "jiql" group.
> To post to this group, send email to ji...@googlegroups.com.
> To unsubscribe from this group, send email to jiql+uns...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/jiql?hl=en.
>

[Gabriel Wong]

Hi,
That is a App Engine environment issue. Try converting value of name
to utf-8. Below is related link:
http://groups.google.com/group/google-appengine-java/search?group=google-appengine-java&q=utf-8+encoding
Regards

On Jul 15, 7:47 am, RH <newyor...@yahoo.com> wrote:
> Hi,
>

> I simply used the "jiql.jsp" downloaded fromwww.jiql.orgto create table, insert into table, then select * from that table.

[RH]

Hi, Gabriel:

Thanks for your response.

I used JDO on my GAE project and the UTF-8 encoding was OK both locally and remotely on appspot.com. It has problem only when I used jiql. (Actually, they were ok on my eclipse IDE locally. The Chinese characters were all messed up only when I deployed the project to the appspot.com). That's why I think jiql has encoding issues on appspot.com. When using jiql, I did try to convert the "name" to UTF-8 explicitly using the following codes on my jsp pages:

name = new String(name.getBytes(), "UTF-8");

It didn't work.

I went to that link you mentioned earlier. Well, I saw someone has the same problem similar to mine, but I didn't see anyone has the answer yet.

thx again! :)

Best regards,

Russell

[Gabriel Wong]

Your java code was probably compiled with UTF-8, so the input itself
was already UTF-8.
If you notice the printout in the jiql.jsp example (a printout of the
sql statement to the browser), the chinese characters are
already ????.
So even before any processing by jiql, the inputed text is already
garbled.

On Jul 16, 5:27 am, RH <newyor...@yahoo.com> wrote:
> Hi, Gabriel:
>
> Thanks for your response.
>
> I used JDO on my GAE project and the UTF-8 encoding was OK both locally and remotely on appspot.com. It has problem only when I used jiql. (Actually, they were ok on my eclipse IDE locally. The Chinese characters were all messed up only when I deployed the project to the appspot.com). That's why I think jiql has encoding issues on appspot.com. When using jiql, I did try to convert the "name" to UTF-8 explicitly using the following codes on my jsp pages:
>
> name = new String(name.getBytes(), "UTF-8");
>
> It didn't work.
>
> I went to that link you mentioned earlier. Well, I saw someone has the same problem similar to mine, but I didn't see anyone has the answer yet.
>
> thx again! :)
>
> Best regards,
>
> Russell
>
>
>
> > Hi,
> > That is a App Engine environment issue. Try converting value of name
> > to utf-8. Below is related link:

> >http://groups.google.com/group/google-appengine-java/search?group=goo...

> > Regards
> > On Jul 15, 7:47 am, RH <newyor...@yahoo.com> wrote:
> >> Hi,
>

> >> I simply used the "jiql.jsp" downloaded fromwww.jiql.orgtocreate table, insert into table, then select * from that table.

[RH]

Hi,

On jiql.jsp, it shows the results along with your last sql command line and my "insert" sql command with Chinese characters were printed on the page correctly with or without UTF-8 conversion (ex: name = new String(name.getBytes(), "UTF-8"); ). I believe it means the encoding on the jsp part is ok. Those Chinese characters were all "????" only when I did "select *" sql commands. That's why I believe jiql.jar/JIQL has encoding problem.

regards,

Russell

[Gabriel Wong]

In the App Engine Data Viewer, does it show the Chinese characters?

> >>>> I simply used the "jiql.jsp" downloaded fromwww.jiql.orgtocreatetable, insert into table, then select * from that table.

[RH]

Hi,

No, it shows "????". But if we modified the data on the Datastore Viewer's "Edit Entity" page, it shows the Chinese characters correctly after the modification. I believe Datastore Viewer uses JDO/JPA instead of JIQL.

regards,

Russell

[Gabriel Wong]

You must follow the suggestion here:
http://groups.google.com/group/google-appengine-java/browse_thread/thread/310cada27c1fe82b/eae5b8d869b539ab?lnk=gst&q=datastore+chinese+characters#eae5b8d869b539ab

e.g. ../appcfg.sh --compile_encoding=UTF-8 ....

On Jul 15, 7:47 am, RH <newyor...@yahoo.com> wrote:

> Hi,
>
> I simply used the "jiql.jsp" downloaded fromwww.jiql.orgto create table, insert into table, then select * from that table.

[RH]

Hi,

Really appreciate you time to answer my questions.

I believe all of my source codes are compiled with UTF-8 encoding on my Eclipse IDE, otherwise I'll even have problems on my JDO datastore functions. Currently, all of my JDO data storage functions of Chinese characters are ok. It has problem only when I use JIQL. I still think that JIQL has encoding problem.

regards,

Russell

> You must follow the suggestion here:

[Gabriel Wong]

You may want to manually compile with encoding flag as suggested.
Remove new String(name.getBytes(), "UTF-8");
Still don't believe me, then checkout:
http://jiqljsp.appspot.com/utf8jiql.jsp

On Jul 20, 5:29 am, RH <newyor...@yahoo.com> wrote:
> Hi,
>

> Really appreciate you time to answer my questions.
>
> I believe all of my source codes are compiled with UTF-8 encoding on my Eclipse IDE, otherwise I'll even have problems on my JDO datastore functions. Currently, all of my JDO data storage functions of Chinese characters are ok. It has problem only when I use JIQL. I still think that JIQL has encoding problem.
>
> regards,
>
> Russell
>
>
>
> > You must follow the suggestion here:

> >http://groups.google.com/group/google-appengine-java/browse_thread/th...

>
> > e.g. ../appcfg.sh --compile_encoding=UTF-8 ....
>
> > On Jul 15, 7:47 am, RH <newyor...@yahoo.com> wrote:
> >> Hi,
>

> >> I simply used the "jiql.jsp" downloaded fromwww.jiql.orgtocreate table, insert into table, then select * from that table.

[RH]

Hi,

Thanks a lot. After using your utf8jiql.jsp, I believe you just proved that jiql supports UTF-8 encoding, but the GAE project needs to be compiled that way. I am still trying to figure out how to do that in my Eclipse IDE. However, it does point me in the right direction. Really appreciate you help.

Best regards,

Russell