"rake cucumber" error (win7): invalid byte sequence in UTF-8 (ArgumentError)

588 views
Skip to first unread message

erwann

unread,
Jan 30, 2011, 4:12:00 AM1/30/11
to Cukes
Hi,

I've a problem with cucumber running windows. I'm following the Rspec
book from Pragprog (chap. 19 with Rails integration) but I can't find
a solution for this issue:
invalid byte sequence in UTF-8 (ArgumentError)

http://pastie.org/1511071 for complete error message

According to https://rspec.lighthouseapp.com/projects/16211/tickets/354-cucumber-features-failed-on-windows
it's a win32console gem problem but it seems that Cucumber doesn't
rely on win32console anymore.

Any hint appreciated!

--
Win 7x64 - ruby 1.9.2p136 (2010-12-25) [i386-mingw32]
gem list
cucumber (0.10.0)
cucumber-rails (0.4.0.beta.1, 0.3.2)

Luis Lavena

unread,
Jan 30, 2011, 12:39:37 PM1/30/11
to cu...@googlegroups.com
On Sun, Jan 30, 2011 at 6:12 AM, erwann <erwa...@gmail.com> wrote:
> Hi,
>
> I've a problem with cucumber running windows. I'm following the Rspec
> book from Pragprog (chap. 19 with Rails integration) but I can't find
> a solution for this issue:
> invalid byte sequence in UTF-8 (ArgumentError)
>
> http://pastie.org/1511071 for complete error message
>
> According to https://rspec.lighthouseapp.com/projects/16211/tickets/354-cucumber-features-failed-on-windows
> it's a win32console gem problem but it seems that Cucumber doesn't
> rely on win32console anymore.
>

What is the console codepage you're starting cucumber from? (use chcp
in the console)

Because that line is this:
https://github.com/aslakhellesoy/cucumber/blob/master/lib/cucumber/formatter/unicode.rb#L12

`cmd /c chcp` =~ /(\d+)/

And if the current command prompt is not in a UTF-8 compatible
codebase, the result from cmd will trigger the failure.

--
Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exupéry

erwann

unread,
Jan 30, 2011, 2:19:26 PM1/30/11
to Cukes

Hi Luis,


> What is the console codepage you're starting cucumber from? (use chcp
> in the console)

it's 850

> Because that line is this:https://github.com/aslakhellesoy/cucumber/blob/master/lib/cucumber/fo...
>
> `cmd /c chcp` =~ /(\d+)/
>
> And if the current command prompt is not in a UTF-8 compatible
> codebase, the result from cmd will trigger the failure.

if I switch to chcp 65001 or chcp 1252, it's the same. Moreover, I'm
mainly using Rubymine IDE and I can't change the cp in their editor

Luis Lavena

unread,
Jan 30, 2011, 2:56:53 PM1/30/11
to cu...@googlegroups.com
On Sun, Jan 30, 2011 at 4:19 PM, erwann <erwa...@gmail.com> wrote:
>
>
>> What is the console codepage you're starting cucumber from? (use chcp
>> in the console)
>
> it's 850
>

So that is the console you're invoking cucumber from?

>
> if I switch to chcp 65001 or chcp 1252, it's the same. Moreover, I'm
> mainly using Rubymine IDE and I can't change the cp in their editor
>

Are you using the same command prompt you change the codepage before?

Please try this at the command line:

ruby -ve "`cmd /c chcp` =~ /(\d+)/; puts $1"

What is the output?

erwann

unread,
Jan 30, 2011, 3:58:41 PM1/30/11
to Cukes

> > it's 850
>
> So that is the console you're invoking cucumber from?

Yes, the windows console, whatever the cp, the error remains the same

> > if I switch to chcp 65001 or chcp 1252, it's the same. Moreover, I'm
> > mainly using Rubymine IDE and I can't change the cp in their editor
>
> Are you using the same command prompt you change the codepage before?

Under Rubymine, I can't change anything, I just tested it to check if
the result was the same

> Please try this at the command line:
>
> ruby -ve "`cmd /c chcp` =~ /(\d+)/; puts $1"
>
> What is the output?

C:\>ruby -ve "`cmd /c chcp` =~ /(\d+)/; puts $1"
ruby 1.9.2p136 (2010-12-25) [i386-mingw32]
850

Luis Lavena

unread,
Jan 30, 2011, 6:00:14 PM1/30/11
to cu...@googlegroups.com
On Sun, Jan 30, 2011 at 5:58 PM, erwann <erwa...@gmail.com> wrote:
>
>> > it's 850
>>
>> So that is the console you're invoking cucumber from?
>
> Yes, the windows console, whatever the cp, the error remains the same
>
>> > if I switch to chcp 65001 or chcp 1252, it's the same. Moreover, I'm
>> > mainly using Rubymine IDE and I can't change the cp in their editor
>>
>> Are you using the same command prompt you change the codepage before?
>
> Under Rubymine, I can't change anything, I just tested it to check if
> the result was the same
>

I don't know how to change RubyMine, so can't help you there.

Please invoke cucumber and rake from the console, *after* setting chcp
to 1252 at minimum

You can also change chcp permanently by editing the registry, but I
would love if you can execute from the command line first to determine
if the issue is RubyMine or not interfering with the right behavior.

erwann

unread,
Jan 31, 2011, 4:27:00 AM1/31/11
to Cukes

> I don't know how to change RubyMine, so can't help you there.

I'll ask them but I want to make it work on win console before

> Please invoke cucumber and rake from the console, *after* setting chcp
> to 1252 at minimum

That's what I made:
open console
chcp 1252
rake cucumber -> error
chcp 65001
rake cucumber -> error

> You can also change chcp permanently by editing the registry, but I
> would love if you can execute from the command line first to determine
> if the issue is RubyMine or not interfering with the right behavior.

I don't think RubyMine is interfering as I uninstalled it just to be
sure and the problem remains

Luis Lavena

unread,
Jan 31, 2011, 7:00:56 AM1/31/11
to cu...@googlegroups.com
On Mon, Jan 31, 2011 at 6:27 AM, erwann <erwa...@gmail.com> wrote:
>
>> I don't know how to change RubyMine, so can't help you there.
>
> I'll ask them but I want to make it work on win console before
>
>> Please invoke cucumber and rake from the console, *after* setting chcp
>> to 1252 at minimum
>
> That's what I made:
> open console
> chcp 1252
> rake cucumber -> error
> chcp 65001
> rake cucumber -> error
>

Can you also verify your cucumber .feature files are saved in UTF-8 encoding?

If this doesn't help, let's see if we can do a TeamViewer session
today to help you out.

Aslak Hellesøy

unread,
Jan 31, 2011, 7:44:26 AM1/31/11
to cu...@googlegroups.com
On Jan 31, 2011, at 12:01, Luis Lavena <luisl...@gmail.com> wrote:

> On Mon, Jan 31, 2011 at 6:27 AM, erwann <erwa...@gmail.com> wrote:
>>
>>> I don't know how to change RubyMine, so can't help you there.
>>
>> I'll ask them but I want to make it work on win console before
>>
>>> Please invoke cucumber and rake from the console, *after* setting chcp
>>> to 1252 at minimum
>>
>> That's what I made:
>> open console
>> chcp 1252
>> rake cucumber -> error
>> chcp 65001
>> rake cucumber -> error
>>
>
> Can you also verify your cucumber .feature files are saved in UTF-8 encoding?
>

If it turns out that saving the file as UTF-8 resolves the issue, I
can output an error message asking the user to verify that the file is
saved as UTF-8.

WDYT?

Aslak


> If this doesn't help, let's see if we can do a TeamViewer session
> today to help you out.
> --
> Luis Lavena
> AREA 17
> -
> Perfection in design is achieved not when there is nothing more to add,
> but rather when there is nothing more to take away.
> Antoine de Saint-Exupéry
>

> --
> You received this message because you are subscribed to the Google Groups "Cukes" group.
> To post to this group, send email to cu...@googlegroups.com.
> To unsubscribe from this group, send email to cukes+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
>

erwann

unread,
Jan 31, 2011, 7:52:35 AM1/31/11
to Cukes
Thanks for your help Luis, I'm usually obstinate but this time I'm
giving up and will forget cucumber for now

Luis Lavena

unread,
Jan 31, 2011, 8:11:08 AM1/31/11
to cu...@googlegroups.com
On Mon, Jan 31, 2011 at 9:52 AM, erwann <erwa...@gmail.com> wrote:
> Thanks for your help Luis, I'm usually obstinate but this time I'm
> giving up and will forget cucumber for now
>

Can you do me one small favor before giving up?

Provide me the output of the following:

SET LC_ALL
SET LC_CTYPE
SET LANG

These environment variables will affect how Ruby read files.

A workaround will be mark the encoding of all the cucumber files with
magic comment (#encoding: UTF-8)

erwann

unread,
Jan 31, 2011, 8:37:02 AM1/31/11
to Cukes

> Can you do me one small favor before giving up?
>
> Provide me the output of the following:
>
> SET LC_ALL
> SET LC_CTYPE
> SET LANG
>
> These environment variables will affect how Ruby read files.

Of course, these env variables are not defined:

C:\>SET LC_ALL
La variable d’environnement LC_ALL n’est pas définie.

C:\>SET LC_CTYPE
La variable d’environnement LC_CTYPE n’est pas définie.

C:\>SET LANG
La variable d’environnement LANG n’est pas définie.

erwann

unread,
Feb 5, 2011, 12:23:02 PM2/5/11
to Cukes

> Can you do me one small favor before giving up?
>
> Provide me the output of the following:
>
> SET LC_ALL
> SET LC_CTYPE
> SET LANG


Don't know if it's useful but when I run cucumber with a rails env set
to dev instead f test, I don't have the invalid byte error but a "rake
aborted!
Permission denied - db/test.sqlite3" one

Luis Lavena

unread,
Feb 5, 2011, 3:31:30 PM2/5/11
to cu...@googlegroups.com
On Sat, Feb 5, 2011 at 2:23 PM, erwann <erwa...@gmail.com> wrote:
>
>
> Don't know if it's useful but when I run cucumber with a rails env set
> to dev instead f test, I don't have the invalid byte error but a "rake
> aborted!
> Permission denied - db/test.sqlite3" one
>

It stops earlier than executing cucumber because you're running a
"rails server" which is using the db.

Can you ensure no background rails server is been executed, no
rspec/autotest or cucumber is also executed when you test?

I'm having problems to recreate your environment to help you better, sorry.

erwann

unread,
Feb 5, 2011, 4:35:26 PM2/5/11
to Cukes

> Can you ensure no background rails server is been executed, no
> rspec/autotest or cucumber is also executed when you test?

No rails server, no autotest installed but what do you mean by
"cucumber is also executed"?

> I'm having problems to recreate your environment to help you better, sorry.

I'll try to create the same env on my laptop to check if I get the
same error.

By the way, while installing RubyGems 1.5.0 I got this message:
RubyGems 1.5.0 installed
unable to convert U+0160 from UTF-8 to CP850 for History.txt, skipping
unable to convert U+0160 from UTF-8 to CP850 for History.txt, skipping

could it be related?

Luis Lavena

unread,
Feb 5, 2011, 7:22:11 PM2/5/11
to cu...@googlegroups.com
On Sat, Feb 5, 2011 at 6:35 PM, erwann <erwa...@gmail.com> wrote:
>
> By the way, while installing RubyGems 1.5.0 I got this message:
> RubyGems 1.5.0 installed
> unable to convert U+0160 from UTF-8 to CP850 for History.txt, skipping
> unable to convert U+0160 from UTF-8 to CP850 for History.txt, skipping
>
> could it be related?
>

No, that is RDoc issue, which can be solved by updating rdoc gem.

erwann

unread,
Feb 6, 2011, 9:05:58 AM2/6/11
to Cukes

I configured my laptop with the same programs (win 7 x64, latest Ruby
devkit and gems) and I have the same error

Luis Lavena

unread,
Feb 6, 2011, 11:08:08 AM2/6/11
to cu...@googlegroups.com

Can you jump on IRC? I'm on RubyInstaller channel (#rubyinstaller)

I would like to do a TeamViewer session with you if possible.

Please download the Quick Support package:

http://www.teamviewer.com/en/download/index.aspx

My nickname is "luislavena"

Luis Lavena

unread,
Feb 7, 2011, 9:52:43 AM2/7/11
to cu...@googlegroups.com
On Sun, Feb 6, 2011 at 1:08 PM, Luis Lavena <luisl...@gmail.com> wrote:
> On Sun, Feb 6, 2011 at 11:05 AM, erwann <erwa...@gmail.com> wrote:
>>
>> I configured my laptop with the same programs (win 7 x64, latest Ruby
>> devkit and gems) and I have the same error
>> invalid byte sequence in UTF-8 (ArgumentError)
>
> Can you jump on IRC? I'm on RubyInstaller channel (#rubyinstaller)
>
> I would like to do a TeamViewer session with you if possible.
>
> Please download the Quick Support package:
>
> http://www.teamviewer.com/en/download/index.aspx
>
> My nickname is "luislavena"
>

For the ones that want to know, we solved the issue setting
CUCUMBER_OUTPUT_ENCODING environment variable.

Aslak: seems to me that unicode.rb is not loaded in UTF-8 mode and
thus making the regexp around 'chcp' fail.

Will test it further and send you a pull request with the modifications.

Aslak Hellesøy

unread,
Feb 7, 2011, 3:13:04 PM2/7/11
to cu...@googlegroups.com
On Feb 7, 2011, at 10:53, Luis Lavena <luisl...@gmail.com> wrote:

> On Sun, Feb 6, 2011 at 1:08 PM, Luis Lavena <luisl...@gmail.com> wrote:
>> On Sun, Feb 6, 2011 at 11:05 AM, erwann <erwa...@gmail.com> wrote:
>>>
>>> I configured my laptop with the same programs (win 7 x64, latest Ruby
>>> devkit and gems) and I have the same error
>>> invalid byte sequence in UTF-8 (ArgumentError)
>>
>> Can you jump on IRC? I'm on RubyInstaller channel (#rubyinstaller)
>>
>> I would like to do a TeamViewer session with you if possible.
>>
>> Please download the Quick Support package:
>>
>> http://www.teamviewer.com/en/download/index.aspx
>>
>> My nickname is "luislavena"
>>
>
> For the ones that want to know, we solved the issue setting
> CUCUMBER_OUTPUT_ENCODING environment variable.
>
> Aslak: seems to me that unicode.rb is not loaded in UTF-8 mode and
> thus making the regexp around 'chcp' fail.
>
> Will test it further and send you a pull request with the modifications.
>

Thanks Luis!

Aslak

> --
> Luis Lavena
> AREA 17
> -
> Perfection in design is achieved not when there is nothing more to add,
> but rather when there is nothing more to take away.
> Antoine de Saint-Exupéry
>

Reply all
Reply to author
Forward
0 new messages