Not quite 100% passes against 2008

[Nick Tidey]

Now that I've been able to run the unit tests (against SQL Server
2008), I get one error, one failure:

1) Failure:
test_named_scopes_honor_current_scopes_from_when_defined
(NamedScopeTest)
<[...Posts...]> expected to be != to <[...Posts...]>

2) Error:
test_validate_uniqueness_with_limit_and_utf8(ValidationsTest):
ActiveRecord::StatementInvalid: DBI::DatabaseError: 22001 (8152)
[unixODBC][FreeTDS][SQL Server]String or binary data would be
truncated.: INSERT INTO [events] ([title]) VALUES('一二三四五')

Full output is here:
http://pastie.org/610635

The schema used to set up the Rails ActiveRecord tests contains:

create_table :events, :force => true do |t|
t.string :title, :limit => 5
end

So I'm not surprised that error is cropping up. I'm curious how anyone
is getting 100% success running these tests currently and if I'm doing
anything wrong. My previous post described the steps I took to get
this up and running:

http://groups.google.com/group/rails-sqlserver-adapter/browse_thread/thread/eeb1a50316d4872b

I'm running on Snow Leopard with ruby 1.8.7, freetds and unixODBC all
installed through Macports.

--
Nick

[Nick Tidey]

Hmm, I'm pretty baffled now. I manually edited 'vendor/rails/
activerecord/test/schema/schema.rb' and changed the declaration to:

create_table :events, :force => true do |t|

t.column :title, :nvarchar, :limit => 5
end

Once the tests were running, I confirmed, using SQL Server Management
Studio, that the column was indeed nvarchar. All good. However the
error still occurred. I then tried running the failing SQL "INSERT
INTO [events] ([title]) VALUES(N'一二三四五')" directly in SQLSMS and it
worked as expected.

--
Nick

On Sep 9, 3:06 pm, Nick Tidey <nickti...@gmail.com> wrote:
> Now that I've been able to run the unit tests (against SQL Server
> 2008), I get one error, one failure:
>
> 1) Failure:
> test_named_scopes_honor_current_scopes_from_when_defined
> (NamedScopeTest)
> <[...Posts...]> expected to be != to <[...Posts...]>
>
> 2) Error:
> test_validate_uniqueness_with_limit_and_utf8(ValidationsTest):
> ActiveRecord::StatementInvalid: DBI::DatabaseError: 22001 (8152)
> [unixODBC][FreeTDS][SQL Server]String or binary data would be
> truncated.: INSERT INTO [events] ([title]) VALUES('一二三四五')
>
> Full output is here:http://pastie.org/610635
>
> The schema used to set up the Rails ActiveRecord tests contains:
>
>         create_table :events, :force => true do |t|
>             t.string :title, :limit => 5
>         end
>
> So I'm not surprised that error is cropping up. I'm curious how anyone
> is getting 100% success running these tests currently and if I'm doing
> anything wrong. My previous post described the steps I took to get
> this up and running:
>

> http://groups.google.com/group/rails-sqlserver-adapter/browse_thread/...

[Ken Collins]

Hey Nick,

Just noticed your comment on my blog article, thanks! Also, It's been
awhile since I've run all the tests on 2000/2005/2008. So here are my
new results.

SQL Server 2000 => 100%
SQL Server 2005 => 100%
SQL Server 2008 => 100%

Typically I expect an odd deadlock issue on one test in 2005/2008. But
oddly I'm not seeing it in my new stack anymore. Could be some recent
changes to 2.3.x stable. I will say that your tests look like ones
where I am actually testing 1.8/1.9 char encodings and stressing the
UTF8 flag of the ruby ODBC library. When you compiled rb-odbc with
MacPorts, did you set the +utf8 variant?


- Ken

[Nick Tidey]

Hey Ken

Yeah I reinstalled rb-odbc with +utf8 when I read your blog post.

I jumped into isql to test the connection at a lower level and got the
following results, though having not dealt with unicode data before,
I'm not sure what to expect:

http://pastie.org/611754

Off to scan the unixODBC mailing lists for clues.

Thanks for all the help.
Nick

[Ken Collins]

Yes, please do followup with this and let us know.
Is anyone else able to pass all the tests? Maybe I've just gotten
lucky all this time?

- Ken

[Nick Tidey]

I've managed to get all tests running without error, but still see
that one failure in my original post. Here's what I've found:

FreeTDS assumes an ISO-8859-1 character encoding unless told
otherwise. I added "client charset = UTF-8" to my server definition in
freetds.conf and that allowed me to successfully run the isql
transcript from above.

I then re-ran the unit tests and encountered a new error on my error-
ing test from above. I reversed my changes to the ActiveRecord unit
test so that the column definition was again:

create_table :events, :force => true do |t|
t.string :title, :limit => 5
end

and was able to run that test successfully. Now only that one failing
test remains. About to look into that further now.

--
Nick

[Ken Collins]

Nick,

This is good info. I was reading this thread <http://lists.ibiblio.org/pipermail/freetds/2006q3/020396.html
> and when I run tsql I I see this.

locale is "en_US.utf-8"
locale charset is "utf-8"

It could be that on OS X the libiconv is configured that way
automatically. But great to know. I'll add these details to my page too!


- Thanks,
Ken

[Nick Tidey]

I just replied in detail and then had to delete it because the
response was based on a flawed premise. D'oh. I found and removed
"export LC_ALL='C'" in my .bash_profile - a legacy fix for an old
Subversion issue - which was muddying the waters. Now I see:

Nick@MacBook[~]$ tsql
locale is "en_AU.UTF-8"
locale charset is "UTF-8"

whereas before I was seeing "US-ASCII" character set. In spite of this
change, I can still only successfully run that unit test if I add the
"client charset" directive to freetds.conf.

--
Nick

[Ken Collins]

BTW, this is great to know. Added a few notes to my article too.

- Ken