Seeing oj-gem test failures with tcs ruby 193p231, but all tests pass on RubyInstaller ruby 1.9.3p194

[Jarmo Pertman]

Hello!

Thanks to the good work of Peter Ohler, fastest JSON parser so far in Ruby - Oj -  is now compiling on Windows!

All tests are passing on Ruby 193p194, which i have installed with RubyInstaller. I have however 5 failures when running tests against tcs Ruby 193p231.

C:\Users\jarmo\Documents\Projects\Ruby\oj>pushd ext\oj && ruby extconf.rb && make && popd

>>>>> Creating Makefile for tcs-ruby version 1.9.3 on i386-mingw32 <<<<<

creating Makefile

generating oj-i386-mingw32.def

compiling cache.c

<command-line>:0:4: warning: missing whitespace after the macro name

compiling cache8.c

<command-line>:0:4: warning: missing whitespace after the macro name

compiling dump.c

<command-line>:0:4: warning: missing whitespace after the macro name

compiling fast.c

<command-line>:0:4: warning: missing whitespace after the macro name

compiling load.c

<command-line>:0:4: warning: missing whitespace after the macro name

compiling oj.c

<command-line>:0:4: warning: missing whitespace after the macro name

linking shared-object oj.so

C:\Users\jarmo\Documents\Projects\Ruby\oj>ruby test\tests.rb

Run options:

# Running tests:

.....................................F......................EEE......................

Finished tests in 0.055003s, 1545.3703 tests/s, 2199.8800 assertions/s.

  1) Failure:

test_encode(Juice) [test/tests.rb:884]:

<"\u3074\u30FC\u305F\u30FC"> expected but was

<"\xE3\x81\xB4\xE3\x83\xBC\xE3\x81\x9F\xE3\x83\xBC">.

  2) Error:

test_range_compat(Juice):

NotImplementedError: Failed to dump 'Range' Object (09)

    test/tests.rb:609:in `dump'

    test/tests.rb:609:in `test_range_compat'

  3) Error:

test_range_null(Juice):

NotImplementedError: Failed to dump 'Range' Object (09)

    test/tests.rb:605:in `dump'

    test/tests.rb:605:in `test_range_null'

  4) Error:

test_range_object(Juice):

NotImplementedError: Failed to dump 'Range' Object (09)

    test/tests.rb:619:in `dump'

    test/tests.rb:619:in `test_range_object'

85 tests, 121 assertions, 1 failures, 3 errors, 0 skips

C:\Users\jarmo\Documents\Projects\Ruby\oj>ruby test\test_mimic.rb

Run options:

# Running tests:

.........E.......

Finished tests in 0.009000s, 1888.8889 tests/s, 2333.3333 assertions/s.

  1) Error:

test_load_proc(Mimic):

NotImplementedError: Calling a Proc with a block not supported in this version. Use func() {|x| } syntax instead.

    test/test_mimic.rb:79:in `load'

    test/test_mimic.rb:79:in `test_load_proc'

17 tests, 21 assertions, 0 failures, 1 errors, 0 skips

C:\Users\jarmo\Documents\Projects\Ruby\oj>ruby test\test_fast.rb

Run options:

# Running tests:

.................................

Finished tests in 0.009001s, 3666.2593 tests/s, 11665.3705 assertions/s.

33 tests, 105 assertions, 0 failures, 0 errors, 0 skips

Any ideas why Ruby from RubyInstaller has every test passing, but tcs does not?

You can get the oj from https://github.com/ohler55/oj and give it a try like this:

pushd ext\oj && ruby extconf.rb && make && popd

ruby test\tests.rb

ruby test\test_mimic.rb

ruby test\test_fast.rb

Thank You for help!

Jarmo Pertman

-----

IT does really matter - http://itreallymatters.net

[Jarmo Pertman]

After some further investigation it seems that most of the problems are in extconf.rb where RUBY_DESCRIPTION is used for detecting ruby stuff and tcs ruby has "tcs-ruby" instead of "ruby" in that variable.

Jarmo Pertman

-----

IT does really matter - http://itreallymatters.net

[Jon]

> After some further investigation it seems that most of the problems are in
> extconf.rb where RUBY_DESCRIPTION is used for detecting ruby stuff and tcs
> ruby has "tcs-ruby" instead of "ruby" in that variable.
>
> Jarmo Pertman

Good catch.

They need to switch to the more robust matching against `RbConfig::CONFIG['host_os']` way. Both official RubyInstaller and tcs-ruby both have `mingw32` for that value.

I'll also try to play with oj a bit later.

I don't have a 64bit machine (waiting for Win8 announce) to build against, but given the progress Luis, Hiroshi, and other have had building 64bit with the RubyInstaller recipes we should ensure any patch understands 32/64bit Rubies built with RubyInstaller recipes.

Jon

[Jon]

I toyed with oj with a mingw.org based 4.6.2 devkit on 1.9.3 ruby and tcs-ruby using win7 ultimate 32bit:

https://gist.github.com/3902755

I want to swing back and play with mingw-w64 4.7.2 and ruby 2.0.0 as well as try the full test suite...and my Arch 3.6.2 VM.

Jon

---
Fail fast. Fail often. Fail publicly. Learn. Adapt. Repeat.
http://thecodeshop.github.com | http://jonforums.github.com/
twitter: @jonforums

[Jon]

> > > After some further investigation it seems that most of the problems are in
> > > extconf.rb where RUBY_DESCRIPTION is used for detecting ruby stuff and tcs
> > > ruby has "tcs-ruby" instead of "ruby" in that variable.
> > >
> > > Jarmo Pertman
> >
> > Good catch.
> >
> > They need to switch to the more robust matching against `RbConfig::CONFIG['host_os']` way. Both official RubyInstaller and tcs-ruby both have `mingw32` for that value.
> >
> > I'll also try to play with oj a bit later.
> >
> > I don't have a 64bit machine (waiting for Win8 announce) to build against, but given the progress Luis, Hiroshi, and other have had building 64bit with the RubyInstaller recipes we should ensure any patch understands 32/64bit Rubies built with RubyInstaller recipes.
>
>
> I toyed with oj with a mingw.org based 4.6.2 devkit on 1.9.3 ruby and tcs-ruby using win7 ultimate 32bit:
>
> https://gist.github.com/3902755
>
> I want to swing back and play with mingw-w64 4.7.2 and ruby 2.0.0 as well as try the full test suite...and my Arch 3.6.2 VM.

When running the oj tests I also get failures in the test suites. I haven't looked closely, but they appear to be a repro of yours.

http://pastie.org/5071269

The problem is that the oj extconf.rb has been completely dumbfounded by the naming trickery I did for tcs-ruby and it generates a Makefile with CPPFLAGS that are causing key pieces of functionality to not be built into the .so.

With my original extconf.rb patch, here's the Makefile diff. Check out the CPPFLAGS diff between normal ruby and tcs-ruby.

https://gist.github.com/3902755#file_4_makefile_diffs.diff

I then hacked my original extconf.rb patch to include

type = type.delete('tcs-')

and both the build warnings and test failures disappeared when using tcs-ruby.

Would you confirm the extconf.rb patch also works for you?

[Jon]

Actually, the entire `type = type.delete('tcs-')` nonsense is too ugly to live.

`type` should be generated from existing config in rbconfig.rb, like `type = RbConfig::CONFIG['RUBY_BASE_NAME']` or some other relevant var (also applicable to rubinius) rather than parsing RUBY_DESCRIPTION.

[Jon]

> > > > > After some further investigation it seems that most of the problems are in
> > > > > extconf.rb where RUBY_DESCRIPTION is used for detecting ruby stuff and tcs
> > > > > ruby has "tcs-ruby" instead of "ruby" in that variable.
> > > > >
> > > > > Jarmo Pertman
> > > >
> > > > Good catch.
> > > >
> > > > They need to switch to the more robust matching against `RbConfig::CONFIG['host_os']` way. Both official RubyInstaller and tcs-ruby both have `mingw32` for that value.
> > > >
> > > > I'll also try to play with oj a bit later.
> > > >
> > > > I don't have a 64bit machine (waiting for Win8 announce) to build against, but given the progress Luis, Hiroshi, and other have had building 64bit with the RubyInstaller recipes we should ensure any patch understands 32/64bit Rubies built with RubyInstaller recipes.

Jarmo, thanks for bringing this up here and working the Windows issues over at Oj.

More WIP updates at https://github.com/ohler55/oj/issues/42

Jon

[Jarmo Pertman]

Jon, awesome work!

Thanks for taking the time to improve it :)

Jarmo