CFFILE Write character issue

309 views
Skip to first unread message

Completely Free Dating - Andrew

unread,
Jun 25, 2009, 6:41:01 PM6/25/09
to Railo
Hi,

I'm having a problem with CFFILE write and a character. The character
is question is curved apostrophe. If I get the data from the database
(UTF-8) and output it to the page it looks fine. However if I write it
to a file it appears as â instead.

I have set the charset on the cffile write to "utf-8" as well. I have
also tried it without setting the charset, but it does the same thing
(I assume the default is utf-8 anyway). All the charset settings in
the administrator our set to utf-8 as well. I'm running on Cent OS 5
Linux.

Maybe this is bug, I'm not sure.

Thanks.

Andrew.

Kris Jones

unread,
Jun 25, 2009, 9:32:16 PM6/25/09
to ra...@googlegroups.com
Yes, the charset attribute of CFFILE is absolutely necessary. You must
also set the appropriate header encoding info for the file. Try
adding <cfprocessingdirective pageencoding="utf-8"> in the first 1024
bytes of the template.

Cheers,
Kris

Completely Free Dating - Andrew

unread,
Jun 26, 2009, 5:04:15 AM6/26/09
to Railo
Hi Kris,

Thanks for the suggestion, but still the same problem. I think it
might be a bug to be honest.

Regards,

Andrew.

Michael Offner-Streit

unread,
Jun 26, 2009, 5:22:57 AM6/26/09
to ra...@googlegroups.com
Hi Andrew

i think this is more a configuration problem, can you provide a example.

tnx micha

Completely Free Dating - Andrew schrieb:

Completely Free Dating - Andrew

unread,
Jun 26, 2009, 5:44:55 AM6/26/09
to Railo
Hi Michael,

I can, what do you need in the way of example?

Andrew.

On Jun 26, 10:22 am, Michael Offner-Streit <michael.off...@railo.ch>
wrote:
> Hi Andrew
> i think this is more a configuration problem, can you provide a example.
> tnx micha
> Completely Free Dating - Andrew schrieb:Hi Kris, Thanks for the suggestion, but still the same problem. I think it might be a bug to be honest. Regards, Andrew. On Jun 26, 2:32 am, Kris Jones<krispl...@gmail.com>wrote:Yes, the charset attribute of CFFILE is absolutely necessary. You must also set the appropriate header  encoding info for the file. Try adding <cfprocessingdirective pageencoding="utf-8"> in the first 1024 bytes of the template. Cheers, KrisI'm having a problem with CFFILE write and a character. The character is question is curved apostrophe. If I get the data from the database (UTF-8) and output it to the page it looks fine. However if I write it to a file it appears as â instead.I have set the charset on the cffile write to "utf-8" as well. I have also tried it without setting the charset, but it does the same thing (I assume the default is utf-8 anyway). All the charset settings in the administrator our set to utf-8 as well. I'm running on Cent OS 5 Linux.Maybe this is bug, I'm not sure.-- Michael Offner-Streit CTO Railo Technologies GmbHmicha...@railo.chwww.getrailo.comMailing List (english):http://groups.yahoo.com/group/railo_talk/Mailing List (german):http://de.groups.yahoo.com/group/railo/Linked in:http://www.linkedin.com/e/gis/71368/0CF7D323BBC1Issue Tracker:http://jira.jboss.org/jira/browse/RAILOBlog:http://www.railo-technologies.com/blog

Gert Franz

unread,
Jun 26, 2009, 5:50:47 AM6/26/09
to ra...@googlegroups.com
Well just a code that reproduces the error. Something like:

<cfprocessingdirective...>

<cfset sOut = Chr(...)>

<cffile action="write" ...>

Greetings from Switzerland
Gert Franz

Railo Technologies Professional Open Source
skype: gert.franz ge...@getrailo.com
+41 76 5680 231 www.getrailo.ch


-----Ursprüngliche Nachricht-----
Von: ra...@googlegroups.com [mailto:ra...@googlegroups.com] Im Auftrag von
Completely Free Dating - Andrew
Gesendet: Freitag, 26. Juni 2009 11:45
An: Railo
Betreff: [railo] Re: CFFILE Write character issue

Completely Free Dating - Andrew

unread,
Jun 26, 2009, 6:27:44 AM6/26/09
to Railo
Hi Gert,

Below is the code, it is pretty straight forward:

<cfprocessingdirective pageencoding="utf-8">

<!--- ****************************************************************
--->
<!--- Get Members --->
<cfquery name="qry_Members" datasource="#Application.str_MembersDSN#">
SELECT field1, field2, field3, field4, field5, field6,
field7, field8, field9
FROM table
</cfquery>

Generating File<br>
<cfflush>
<!--- Create sitemap XML --->
<cfxml casesensitive="no" variable="GoogleBase">
<rss version="2.0">
<channel xmlns:g="http://base.google.com/ns/1.0">
<title>Completely Free Dating - Members</title>
<link>http://www.completelyfreedating.co.uk/</link>
<description>Completely Free Dating - 100% free online
dating for the UK</description>
<cfoutput query="qry_Members">
<item>
<title><![CDATA[#field1#]]></title>
<g:age>#field2#</g:age></cfif>
<description><![CDATA[#field3#]]></
description>
<g:gender>#field4#</g:gender>
<g:image_link>#XMLFormat
("#Application.baseURL#/memberimages/#field6#/thumb_large_#field5#")#</
g:image_link>
<g:interested_in>#XMLFormat(#field7#)
#</g:interested_in>
<link>#XMLFormat
("#Application.baseURL#/members/#URLEncodedFormat(field8)#")#</link>
<g:marital_status>single</
g:marital_status>
<g:location>#XMLFormat("#field9#")#,
United Kingdom</g:location>
</item></cfoutput>
</channel>
</rss>
</cfxml>

Writing File<br>
<cfflush>
<cffile action="WRITE"
file="#Application.rootDIR#/googleBase.xml"
output="#ToString(GoogleBase)#"
charset="utf-8">

Complete.

Kind regards,

Andrew.


On Jun 26, 10:50 am, "Gert Franz" <gert.fr...@railo.ch> wrote:
> Well just a code that reproduces the error. Something like:
>
> <cfprocessingdirective...>
>
> <cfset sOut = Chr(...)>
>
> <cffile action="write" ...>
>
> Greetings from Switzerland
> Gert Franz
>
> Railo Technologies      Professional Open Source
> skype: gert.franz       g...@getrailo.com
> +41 76 5680 231www.getrailo.ch
> GmbHmichael.off...@railo.chwww.getrailo.comMailingList
> (english):http://groups.yahoo.com/group/railo_talk/MailingList

Paul Klinkenberg

unread,
Jun 26, 2009, 6:32:39 AM6/26/09
to ra...@googlegroups.com
Hi Andrew,

The problem is probably the <cfprocessingdirective pageencoding="utf-8" />
which isn't explicitly set.
Wit the following code, if you omit this tag, then the string is not
correctly written, but when you do add it, then it is true.
Why, I'm not exactly sure, because Railo should always use utf-8 as
pageencoding, right?

IF ANYONE AT RAILO could shed some light on this...

I used the following test code:
(hope the curly quotes are preserved when you receive this message; they are
at the left and right of hi in string 'txt')
<cfprocessingdirective pageencoding="utf-8" />
<cfset txt = "“hi”" />
<cffile action="write" file="c:\testutf.txt" output="#txt#" addnewline="no"
/>
<cffile action="read" file="c:\testutf.txt" variable="txt_read" />
<cfoutput>
txt=#txt#<br />
txt_read=#txt_read#<br />
txt is txt_read = <strong>#txt_read is txt#</strong><br />
getCharNrs(txt) #getCharNrs(txt)#<br />
getCharNrs(txt_read) = #getCharNrs(txt_read)#
</cfoutput>

<cffunction name="getCharNrs" returntype="string">
<cfargument name="str" type="string" required="yes" />
<cfset var asc_arr = [] />
<cfset var index = 0 />
<cfloop from="1" to="#len(str)#" index="index">
<cfset ArrayAppend(asc_arr, asc(mid(str, index, 1))) />
</cfloop>
<cfreturn arrayToList(asc_arr, '-') />
</cffunction>

Paul

Paul Klinkenberg

unread,
Jun 26, 2009, 6:35:43 AM6/26/09
to ra...@googlegroups.com
Hi Andrew "completely free dating" ;-)

I guess if you remove the charset attribute from the [cffile action=write],
you should be fine.

Paul

-----Original Message-----
From: ra...@googlegroups.com [mailto:ra...@googlegroups.com] On Behalf Of

Completely Free Dating - Andrew

unread,
Jun 26, 2009, 6:51:07 AM6/26/09
to Railo
Nope, as I said in my first post, removing the charset attribute make
no difference, I believe utf-8 is the default if you don't specify it.

Andrew.

On Jun 26, 11:35 am, "Paul Klinkenberg" <p...@ongevraagdadvies.nl>
wrote:

Paul Klinkenberg

unread,
Jun 26, 2009, 7:14:11 AM6/26/09
to ra...@googlegroups.com
Hi Andrew,

Two fixes/workarounds for this problem:

<cffile action="write" file="..." output="#ToString(GoogleBase, 'utf-8')#"
charset="utf-8" />

And
<cffile action="write" file="..." output="#ToString(GoogleBase)#"
charset="utf-16" />

But it's a nice problem, I'm looking into it atm.

Completely Free Dating - Andrew

unread,
Jun 26, 2009, 8:27:42 AM6/26/09
to Railo
Hi Paul,

Thanks for the suggestion, but neither of them worked.

Kind regards,

Andrew.

On Jun 26, 12:14 pm, "Paul Klinkenberg" <p...@ongevraagdadvies.nl>

Michael Offner-Streit

unread,
Jun 26, 2009, 8:52:48 AM6/26/09
to ra...@googlegroups.com
Hi Andrew, Paul

First of all, mostly the problem is conversion from one charset to a other while reading or writing
in java everything is unicode, the question what is it when you read it write it.

ok let us go through your file and i will explain what is happeing (see below).

but i'm sure the problem is the database.

greetings micha


<cfprocessingdirective pageencoding="utf-8">
you define that the file should be loaded with utf-8 encoding, but the question is, is it really a utf-8 files, most files are not utf-8, most file have the default charset from the operation system.
the problem here is, when you read in a file you can not see if the charset (when you have no BOM).
if you not define the pageencoding railo use the encoding defined in the railo admin at Settings/Charset/Template Charset, by default this is the os default encoding.

but you do not have your special character in the file then this is not the problem.

<!--- ****************************************************************
--->
<!--- Get Members --->
<cfquery name="qry_Members" datasource="#Application.str_MembersDSN#">
        SELECT  field1, field2, field3, field4, field5, field6,
field7, field8, field9
        FROM    table
</cfquery>
the cfquery read in the data from the datasource, whitch datasouce type to you use, whitch encoding have you defined in the datasource ad the driver?
make a cfoutput of the content before you are writing to the file, if it is already wrong there, the problem is the datasource.
Completely Free Dating - Andrew schrieb: 
Hi Paul,

Thanks for the suggestion, but neither of them worked.

Kind regards,

Andrew.

On Jun 26, 12:14 pm, "Paul Klinkenberg" <p...@ongevraagdadvies.nl>
wrote:
  
Hi Andrew,

Two fixes/workarounds for this problem:

<cffile action="write" file="..." output="#ToString(GoogleBase, 'utf-8')#"
charset="utf-8" />
this makes not really sense, because the "utf-8" is only used when you have binary data, in this case you aleady have strings (in a dom tree)
-- 
Michael Offner-Streit
CTO
Railo Technologies GmbH

Completely Free Dating - Andrew

unread,
Jun 26, 2009, 11:48:11 AM6/26/09
to Railo
Hi Michael,

The database is MySQL 5.0 and the database is UTF-8, the table is
UTF-8 and the fields in question are UTF-8. I am using the "Other -
JDBC Driver" driver option and com.mysql.jdbc.Driver class, however
looking at it now I guess I should be using the MySQL option, but I
guess this would use the same JDBC connector anyway, so I guess this
will not really make any difference, unless of course if Railo
includes some additional options on the connection string for the
encoding?

Andrew.

On Jun 26, 1:52 pm, Michael Offner-Streit <michael.off...@railo.ch>
wrote:
> Hi Andrew, Paul
>
> First of all, mostly the problem is conversion from one charset to a
> other while reading or writing
> in java everything is unicode, the question what is it when you read it
> write it.
>
> ok let us go through your file and i will explain what is happeing (see
> below).
>
> but i'm sure the problem is the database.
>
> greetings micha
>
> > <cfprocessingdirective pageencoding="utf-8">
>
> *you define that the file should be loaded with utf-8 encoding, but the
> question is, is it really a utf-8 files, most files are not utf-8, most
> file have the default charset from the operation system.
> the problem here is, when you read in a file you can not see if the
> charset (when you have no BOM).
> if you not define the pageencoding railo use the encoding defined in the
> railo admin at Settings/Charset/Template Charset, by default this is the
> os default encoding.
>
> but you do not have your special character in the file then this is not
> the problem.*> <!--- ****************************************************************
> > --->
> > <!--- Get Members --->
> > <cfquery name="qry_Members" datasource="#Application.str_MembersDSN#">
> >         SELECT  field1, field2, field3, field4, field5, field6,
> > field7, field8, field9
> >         FROM    table
> > </cfquery>
>
> *the cfquery read in the data from the datasource, whitch datasouce type
> to you use, whitch encoding have you defined in the datasource ad the
> driver?
> make a cfoutput of the content before you are writing to the file, if it
> is already wrong there, the problem is the datasource.*
> *this makes not really sense, because the "utf-8" is only used when you
> have binary data, in this case you aleady have strings (in a dom tree)*
> michael.off...@railo.ch ...
>
> read more »

Oğuz Demirkapı

unread,
Jun 26, 2009, 2:26:38 PM6/26/09
to ra...@googlegroups.com

Try to use something like that ...


jdbc:mysql://127.0.0.1:3306/unicodetest?useUnicode=true&characterEncoding=UTF-8


Completely Free Dating - Andrew

unread,
Jun 26, 2009, 2:43:53 PM6/26/09
to Railo
Hi Oğuz,

Nope, that doesn't work either. I don't think it is a database issue,
as it writes it to the web browser just fine, it is only when I save
it into a file on the file system that it is a problem. I think it is
a bug with cffile write, there is no other explanation (that I can
think of).

Kind regards,

Andrew.

On Jun 26, 7:26 pm, Oğuz Demirkapı <demirk...@gmail.com> wrote:
> Try to use something like that ...
>
> *jdbc:mysql://
> 127.0.0.1:3306/unicodetest?useUnicode=true&characterEncoding=UTF-8
>
> *
>
> On Fri, Jun 26, 2009 at 8:48 AM, Completely Free Dating - Andrew <
>
> ...
>
> read more »

Oğuz Demirkapı

unread,
Jun 26, 2009, 2:44:44 PM6/26/09
to ra...@googlegroups.com
Let me try the code ...



2009/6/26 Completely Free Dating - Andrew <andrew...@gmail.com>

Oğuz Demirkapı

unread,
Jun 26, 2009, 3:39:21 PM6/26/09
to ra...@googlegroups.com
Andrew,

Can you try the attached file to generate content from the static content?

It works ok here with following versions.

ColdFusion 8,0,1,195765  
Railo 3.0.2.001 dev
Railo 3.1.0.015 beta

Both Railo servers has default installation.

Template: Cp1252
Web: UTF-8
Resource: Cp1252


Database level is another question. If this basic test works, we need to check something else on DB layer.


Sincerely,

Oğuz Demirkapı


PS: If you have BOM on your fıles, you do not need to have cfprocessingdirective on CF templates.




2009/6/26 Oğuz Demirkapı <demi...@gmail.com>
test_cffile.cfm
Reply all
Reply to author
Forward
0 new messages