What's the status of compiler/compiling on windows?
Gonzalo Garramuño
I'm needing to embed ruby into some applications on windows. As such, I
need to match the ABI and compiler used for the application.
Specifically, I am looking for a full one-click ruby1.8 compiled with
MSVC7 and one with MSVC8.
My understanding is that one-click is still being compiled with MSVC6,
right? So... how hard is it these days to create your own ruby
compilation on windows from source?
--
Gonzalo Garramuño
gga...@advancedsl.com.ar
AMD4400 - ASUS48N-E
GeForce7300GT
Xubuntu Gutsy
Nobuyoshi Nakada
At Fri, 7 Dec 2007 17:26:12 +0900,
Gonzalo Garramuño wrote in [ruby-core:13908]:
> My understanding is that one-click is still being compiled with MSVC6,
> right? So... how hard is it these days to create your own ruby
> compilation on windows from source?
Have you given a glance at win32/REAME.win32 file?
--
Nobu Nakada
Gonzalo Garramuño
Yes. That just tells me how to get a bare-bones ruby install.
Does not mention anything about how gems would be handled if I compile
my own ruby version (I assume those are compiled for the one-click
installer version).
I basically want to distribute a vc7 or vc8 compiled ruby with my
application, but it must work seamlessly with all gems and anything
else. More so, I might even want to distribute something more akin to
the one-click installer rather than something barebones.
So, I ask again, what's the status of ruby on windows? Still a mess?
Being that gems is now part of ruby1.9, I think some more clarification
in the docs is in order.
Nobuyoshi Nakada
At Fri, 7 Dec 2007 20:21:34 +0900,
Gonzalo Garramuño wrote in [ruby-core:13914]:
> Does not mention anything about how gems would be handled if I compile
> my own ruby version (I assume those are compiled for the one-click
> installer version).
It's another story.
> I basically want to distribute a vc7 or vc8 compiled ruby with my
> application, but it must work seamlessly with all gems and anything
> else.
Pure ruby gems would work, but I don't know well about binary
gems.
> So, I ask again, what's the status of ruby on windows? Still a mess?
VC7, VC8 (and perhaps VC9) are supported as well as mingw and
VC6.
--
Nobu Nakada
Berger, Daniel
There are still two problematic libraries in the stdlib - OpenSSL and zlib.
Dan
This communication is the property of Qwest and may contain confidential or
privileged information. Unauthorized use of this communication is strictly
prohibited and may be unlawful. If you have received this communication
in error, please immediately notify the sender by reply e-mail and destroy
all copies of the communication and any attachments.
Luis Lavena
>
> Nobuyoshi Nakada wrote:
> > Hi,
> >
> > At Fri, 7 Dec 2007 17:26:12 +0900,
> > Gonzalo Garramuño wrote in [ruby-core:13908]:
> >> My understanding is that one-click is still being compiled with MSVC6,
> >> right? So... how hard is it these days to create your own ruby
> >> compilation on windows from source?
> >
> > Have you given a glance at win32/REAME.win32 file?
> >
>
> Yes. That just tells me how to get a bare-bones ruby install.
> Does not mention anything about how gems would be handled if I compile
> my own ruby version (I assume those are compiled for the one-click
> installer version).
>
pure ruby gems (Gem::Platform::RUBY) will work without problems.
Gems compiled with 'mswin32' platform will not be compatible due
runtime linking (msvcrt.dll versus msvcr80.dll, search the list about
this topic).
You will require to generate new gems based on Gem::Platform::CURRENT
(which is the best for this case).
> I basically want to distribute a vc7 or vc8 compiled ruby with my
> application, but it must work seamlessly with all gems and anything
> else. More so, I might even want to distribute something more akin to
> the one-click installer rather than something barebones.
>
I wish you luck, honestly. a VC8 or VC9 compiled ruby is way beyond be
considered "compatible" with One-Click installer, which grab the VC6
build from garbagecollect page (we don't build it ourself).
> So, I ask again, what's the status of ruby on windows? Still a mess?
>
Yes and no. Is a mess if you try to mix compilers. Ruby will with VC6
will allow extensions and gems be compiled with mingw, but VC7 or VC8
builds don't (untill we get a mingw version that link against the
updated runtimes).
The same fate faces other languages like Python, the mix of compilers
is a problem.
Also you need to compile all the dependencies (zlib, openssl,
readline, pdcurses) all with the same compiler, to provide the same
extensions that One-Click provides.
> Being that gems is now part of ruby1.9, I think some more clarification
> in the docs is in order.
RubyGems (in this repo trunk) works better with windows than 0.9.5
release. I don't know the exact state of the gems version included on
trunk, but guess is the same.
Latest RubyGems identify properly gems created with VC6 and gems
created with VC8.
Hope this information helps,
--
Luis Lavena
Multimedia systems
-
A common mistake that people make when trying to design
something completely foolproof is to underestimate
the ingenuity of complete fools.
Douglas Adams
Gonzalo Garramuño
Absolutely. That's exactly the info I was looking at. Someone should
add this to the README.win32 file, btw.
On a positive note, the compiling on windows on latest stable and
unstable branches has indeed improved. At least now the .dll and .lib's
generated carry the compiler version in them.
For what it is worth, it would also be ideal if configure.bat would
allow out of source builds, like autotools (or would do out of source
builds by default). That way you could avoid doing nmake clean each time.
Nobuyoshi Nakada
At Sat, 8 Dec 2007 14:57:07 +0900,
Gonzalo Garramuño wrote in [ruby-core:13941]:
> For what it is worth, it would also be ideal if configure.bat would
> allow out of source builds, like autotools (or would do out of source
> builds by default). That way you could avoid doing nmake clean each time.
Is "out of source" to build in different directories from the source directory?
It does allow, of course.
--
Nobu Nakada
Joe Swatosh
On Dec 7, 2007 10:41 AM, Luis Lavena <luisl...@gmail.com> wrote:
> On Dec 7, 2007 8:21 AM, Gonzalo Garramuño <gga...@advancedsl.com.ar> wrote:
> >
>
> Gems compiled with 'mswin32' platform will not be compatible due
> runtime linking (msvcrt.dll versus msvcr80.dll, search the list about
> this topic).
>
>
> I wish you luck, honestly. a VC8 or VC9 compiled ruby is way beyond be
> considered "compatible" with One-Click installer, which grab the VC6
> build from garbagecollect page (we don't build it ourself).
>
> > So, I ask again, what's the status of ruby on windows? Still a mess?
> >
>
> Yes and no. Is a mess if you try to mix compilers. Ruby will with VC6
> will allow extensions and gems be compiled with mingw, but VC7 or VC8
> builds don't (untill we get a mingw version that link against the
> updated runtimes).
>
I apologize for the late reply/threadjack, but I am curious about the
nature of the
problem on windows with extensions compiled with a different compiler than
Ruby itself.
Is the primary issue that Ruby objects created by the extension are created
in one heap then the interpreter attempts to destroy them in another heap (or
visa-versa)? If that is the case, there are some ideas we might be able to
borrow from the boost::shared_ptr<> implementation that could alleviate this
problem.
--
Thanks, Joe
Luis Lavena
>
> I apologize for the late reply/threadjack, but I am curious about the
> nature of the
> problem on windows with extensions compiled with a different compiler than
> Ruby itself.
>
This was discussed in other thread in ruby-talk, but just to summarize:
1) memory allocated by one runtime cannot be free'ed by another, that
ends in segmentation faults.
Example Scenario:
A) Ruby build with VC6 (the current official build).
B) Extension manually hacked to build with VC8 (this requires
rbconfig.rb modifications).
A is linked to MSVCRT.DLL, and B is linked to MSVCR80.DLL
You allloc something in you side that get free'ed by your extension
(calling C free()), that ends with a crash.
This is no a theorical scenario, happened to me when compiling libfcgi
and ruby-fcgi extension.
Another Scenario:
A) Ruby build with VC8 (your private build)
B) A pre-compiled extension created with VC6, like mongrel,
sqlite3-ruby, mysql, etc.
A is linked to MSVCR80.DLL and provides msvcr80-libruby.dll
B is linked to MSVCRT.DLL and link against msvcrt-libruby.dll
This is a more complex problem: the extension will not work, plain
simple, it will yell at you (msgbox) telling that there is a missing
dependency to get it working.
*Also*, it will have the cross-runtime alloc/free issues listed in the
previous scenario.
> Is the primary issue that Ruby objects created by the extension are created
> in one heap then the interpreter attempts to destroy them in another heap (or
> visa-versa)? If that is the case, there are some ideas we might be able to
> borrow from the boost::shared_ptr<> implementation that could alleviate this
> problem.
Yes. extensions will malloc on the linked CRT heap, and ruby GC will
try to free'ed it from their CRT heap... and vice-versa
The thing is Ruby will realy *love* some distutils compilation helper
like python has.
Anyway, the MinGW guys just commited a few changes that will allow you
target specific MSVCRT versions with the same compiler, removing the
pain to create extensions with gcc for ruby build with VC6/8/9.
Also, there is a optional mingw component that let you choose the
target CRT before the compile process, instead to link it with ld...
Regarding boost::shared_ptr I cannot comment, mostly since it seems
they use that practice to track reference counting, and that will
require some changes to how ruby works with it, and also extensions.
But please, correct me if I'm wrong or not so clever :-D
Regards,
Austin Ziegler
>
> 1) memory allocated by one runtime cannot be free'ed by another, that
> ends in segmentation faults.
>
> Example Scenario:
>
> A) Ruby build with VC6 (the current official build).
> B) Extension manually hacked to build with VC8 (this requires
> rbconfig.rb modifications).
>
> A is linked to MSVCRT.DLL, and B is linked to MSVCR80.DLL
> You allloc something in you side that get free'ed by your extension
> (calling C free()), that ends with a crash.
>
> This is no a theorical scenario, happened to me when compiling libfcgi
> and ruby-fcgi extension.
And the *reason* it's not a theoretical scenario is that Unix doesn't
have this problem (multiple runtimes) in > 99% of cases. Most Unixes
only have one runtime installed and cannot actually have another
runtime installed. Therefore, there's not multiple heaps involved.
Therefore, it's a common case to have an API that says "this C call
allocates memory; remember to free it when you're done with it." There
should be an library_api_free call for every library (e.g., zlib_free,
openssl_free, etc.) just in case, but it's Not An Issue on Unix.
> Anyway, the MinGW guys just commited a few changes that will allow you
> target specific MSVCRT versions with the same compiler, removing the
> pain to create extensions with gcc for ruby build with VC6/8/9.
Yay!
-austin
--
Austin Ziegler * halos...@gmail.com * http://www.halostatue.ca/
* aus...@halostatue.ca * http://www.halostatue.ca/feed/
* aus...@zieglers.ca
Nobuyoshi Nakada
At Thu, 13 Dec 2007 06:55:00 +0900,
Austin Ziegler wrote in [ruby-core:14040]:
> > This was discussed in other thread in ruby-talk, but just to summarize:
> >
> > 1) memory allocated by one runtime cannot be free'ed by another, that
> > ends in segmentation faults.
2) thread local variables like errno cannot be shared across
the runtimes.
> And the *reason* it's not a theoretical scenario is that Unix doesn't
> have this problem (multiple runtimes) in > 99% of cases. Most Unixes
> only have one runtime installed and cannot actually have another
> runtime installed. Therefore, there's not multiple heaps involved.
It's a half of the reason. Most Unixes shared library
mechanisms have just one namespace. Besides, a global variable
refers same address, if there were different libraries.
--
Nobu Nakada
Joe Swatosh
>
> And the *reason* it's not a theoretical scenario is that Unix doesn't
> have this problem (multiple runtimes) in > 99% of cases. Most Unixes
> only have one runtime installed and cannot actually have another
> runtime installed. Therefore, there's not multiple heaps involved.
> Therefore, it's a common case to have an API that says "this C call
> allocates memory; remember to free it when you're done with it." There
> should be an library_api_free call for every library (e.g., zlib_free,
> openssl_free, etc.) just in case, but it's Not An Issue on Unix.
If everyone used ruby_xmalloc and ruby_xfree, would this part of the binary
extension problem be addressed?
>
--
Joe
Joe Swatosh
> Hi,
>
> At Thu, 13 Dec 2007 06:55:00 +0900,
> Austin Ziegler wrote in [ruby-core:14040]:
> > > This was discussed in other thread in ruby-talk, but just to summarize:
> > >
> > > 1) memory allocated by one runtime cannot be free'ed by another, that
> > > ends in segmentation faults.
>
> 2) thread local variables like errno cannot be shared across
> the runtimes.
>
Gack. Is this common?
>
--
Joe
Luis Lavena
If memory plays well with my age, I think perl guys created a
PerlCRT.dll back in 2001 to workaround the CRT issues.
The extensions (even perl.exe) was linked against it and thus,
avoiding the msvcr* dilemma.
Nobuyoshi Nakada
At Thu, 13 Dec 2007 15:59:54 +0900,
Joe Swatosh wrote in [ruby-core:14061]:
> > > > This was discussed in other thread in ruby-talk, but just to summarize:
n> > > >
> > > > 1) memory allocated by one runtime cannot be free'ed by another, that
> > > > ends in segmentation faults.
> >
> > 2) thread local variables like errno cannot be shared across
> > the runtimes.
> >
>
> Gack. Is this common?
errno is quite common, I believe.
3) runtime internal variables refered from the outside, such as
the FILE structures table and the file descriptors table.
Anyway, this discussion has been raised many many times again
and again and again and again and again... and the conclusion
was always same, it's IMPOSSIBLE.
--
Nobu Nakada
Austin Ziegler
No, unfortunately. It would help when you're embedding Ruby in
something else, but most people do something like that anyway to
permit garbage collection, at least in their extension layer. IIRC. ;)
The problem is in the libraries that may be wrapped, such as ncurses or zlib.
Austin Ziegler
> On Dec 13, 2007 3:57 AM, Joe Swatosh <joe.s...@gmail.com> wrote:
> > On Dec 12, 2007 1:55 PM, Austin Ziegler <halos...@gmail.com> wrote:
> > > And the *reason* it's not a theoretical scenario is that Unix doesn't
> > > have this problem (multiple runtimes) in > 99% of cases. Most Unixes
> > > only have one runtime installed and cannot actually have another
> > > runtime installed. Therefore, there's not multiple heaps involved.
> > > Therefore, it's a common case to have an API that says "this C call
> > > allocates memory; remember to free it when you're done with it." There
> > > should be an library_api_free call for every library (e.g., zlib_free,
> > > openssl_free, etc.) just in case, but it's Not An Issue on Unix.
> > If everyone used ruby_xmalloc and ruby_xfree, would this part of the binary
> > extension problem be addressed?
> If memory plays well with my age, I think perl guys created a
> PerlCRT.dll back in 2001 to workaround the CRT issues.
>
> The extensions (even perl.exe) was linked against it and thus,
> avoiding the msvcr* dilemma.
Yikes. That doesn't seem pretty.
There are two real (long-term) solutions for this: Microsoft needs to
fix its behaviour so that you can have a MSVCRT (VC6) linked program
actually using MSVCRT80 (VC8) without knowing it; bugs need to be
filed against external libraries that don't provide their own resource
management operations (e.g., need
<library>_{malloc,free,errno,fopen,fclose}).
Joe Swatosh
On Dec 12, 2007 11:50 PM, Nobuyoshi Nakada <no...@ruby-lang.org> wrote:
> Hi,
>
> At Thu, 13 Dec 2007 15:59:54 +0900,
> Anyway, this discussion has been raised many many times again
> and again and again and again and again... and the conclusion
> was always same, it's IMPOSSIBLE.
>
Thanks Austin, Luis, and Nobu for taking me through it even though it
was just one more time around for you.
--
Joe
Austin Ziegler
No problem. I should make a blog post on this so that there's a
consistent place to point people about this particular problem.
I personally think that it's something Microsoft can and should fix,
but I can also see why they don't bother: most companies simply
recompile everything from source or wait for their vendor to provide
an updated version before they upgrade. (At work, we build our own
OpenSSL and zlib from source, as well as a number of other libraries
we incorporate.)