Exerb for mingw on github
Alexey Borzenkov
I just uploaded exerb-mingw on github:
http://github.com/snaury/exerb-mingw
It already includes a copy of eval.c from ruby 1.8.6 and ruby 1.8.7,
so you can just clone (or download) it, then execute:
$ rake
$ ruby setup.rb
This should build stubs, then install exerb, as well as create batch
files in ruby's bin directory.
Note, that upstream branch starts with exerb 4.2.0 and does not
include binaries in data/exerb directory, as well as ruby sources and
leftover temporary files in some releases. This was due to save space,
as well as because it is irrelevant to exerb-mingw (it uses the ruby's
own static library).
Gordon Thiesfeld
>
> Hello,
>
> I just uploaded exerb-mingw on github:
>
> http://github.com/snaury/exerb-mingw
>
Alexey,
I hope you won't mind answering some dumb questions for me.
I'm guessing the primary benefit to using this will be when your code
uses C extensions. Is that accurate?
The work you've done to get this working is completely over my head.
Do you think that I should have any concerns with stability of exerb-
mingw at this point?
I'm just trying to decide if switching pik over to this version is a
good idea at this point, and I'm trying to understand the implications
of doing so.
Either way, I'm definitely going to download this and start playing
with it. Thanks for taking the time to make this possible.
Gordon
Alexey Borzenkov
> I'm guessing the primary benefit to using this will be when your code uses C
> extensions. Is that accurate?
Yes, although I doubt it can make too much difference (though to be on
a safe side I'll be using my own build of exerb of course). The size
certainly did not benefit: msvc stub is ~500Kb, mingw stub is ~800Kb.
The only notable change in exerb cpp code itself is the use of
rb_provided, but that seems to be for ruby 1.8.7 lack of
enumerator.so.
> The work you've done to get this working is completely over my head. Do you
> think that I should have any concerns with stability of exerb-mingw at this
> point?
I'm not sure myself, since I haven't used it yet, besides obvious
tests like HelloWorld that requires 'zlib'. But since I didn't change
exerb cpp code it should probably be as stable as official exerb.
> I'm just trying to decide if switching pik over to this version is a good
> idea at this point, and I'm trying to understand the implications of doing
> so.
Well, the last exerb 1.8.6 version was 4.3.0, as long as you use that
you'll be on the safe side. Switching to exerb-mingw would mean that
on the downside you'll add ~300Kb to your final executable size, but
on the upside you'll be using ruby 1.8.6-p383 instead of -p110, that's
if you depend on bugfixes between p110 and p383.
> Either way, I'm definitely going to download this and start playing with it.
> Thanks for taking the time to make this possible.
Of course I primarily did it for myself. :) I often need to process
xml data at work (e.g. put translated text from lockit back to xml),
and I found it best to do with ruby+hpricot. The worst part was that
unlike python (with py2exe) I couldn't give my coworkers executables
that they could run themselves, instead of sending materials to me.
Well, now I hope I can. ;)
Alexey Borzenkov
> Well, the last exerb 1.8.6 version was 4.3.0
Sorry, 4.4.0 of course.
Luis Lavena
OMG, this is awesome Alexey:
# foo.rb
require 'zlib'
puts Zlib::ZLIB_VERSION
# foo.exy:
# Generated by mkexy
# on 2009-10-27 22:10
general:
startup: foo.rb
core: cui
kcode: utf8
file:
foo.rb:
zlib.so:
file: C:/Users/Luis/Tools/Ruby/ruby-1.8.6-p383-i386-mingw32/lib/ruby/1.8/i386-mingw32/zlib.so
type: extension-library
zlib1.dll:
file: C:/Users/Luis/Tools/Ruby/ruby-1.8.6-p383-i386-mingw32/bin/zlib1.dll
type: dynamic-library
And the generated EXE is 891KB
UPX'ed it to 347KB
And thanks to the manually added zlib1.dll, I no longer need to
include these DLLs with the executable
The trick is down to dlopen to load the DLL and exerb extracting it at runtime.
Now: lets compare for the following example:
# embed-sinatra.rb
require 'rubygems'
require 'sinatra'
get '/' do
"Hello from a simple Exerb sinatra application!"
end
===
Exerb:
* The generate executable is 1.77MB
* Fails due missing "rack/response":
sinatra/base.rb:35: no such file to load -- rack/response (LoadError)
Even manually adding it, still fails to load it
Ocra:
* Generated executable is 1.2MB
* Every time it's executed, requires approval of firewall permissions.
I think improving the detection mechanism of exerb to deal with the
requires will provide another awesome alternative for executable
packaging :-D
The 200KB penalty for linking with our static lib are not a big deal
after applying UPX to the executable ;-)
Thank you Alexey again for sharing your research and solution with us!
--
Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exupéry
Alexey Borzenkov
> And thanks to the manually added zlib1.dll, I no longer need to
> include these DLLs with the executable
>
> The trick is down to dlopen to load the DLL and exerb extracting it at runtime.
The beauty is that exerb is not extracting it at runtime. :) It has a
manual dll loader (like py2exe), it loads it just like it loads .so
files.
> Now: lets compare for the following example:
>
> # embed-sinatra.rb
> require 'rubygems'
> require 'sinatra'
>
> get '/' do
> "Hello from a simple Exerb sinatra application!"
> end
>
>
> ===
>
> Exerb:
>
> * The generate executable is 1.77MB
> * Fails due missing "rack/response":
> sinatra/base.rb:35: no such file to load -- rack/response (LoadError)
Fixed in the last commit. autoload requires files in variable.c, where
there's another call to rb_require_safe. Added patching now. BUT. Your
example still doesn't work. :( I think at_exit doesn't work. Seems to
be because caller_files is broken:
# sinatra/main.rb, ruby:
["embed-sinatra.rb"]
# sinatra/main.rb, exerb:
["sinatra/base.rb", "sinatra/main.rb"]
Alexey Borzenkov
> I think at_exit doesn't work.
Ooops. Forgot to remove after checking that at_exit DOES work. :)
Alexey Borzenkov
> there's another call to rb_require_safe. Added patching now. BUT. Your
> example still doesn't work. :( I think at_exit doesn't work. Seems to
> be because caller_files is broken:
>
> # sinatra/main.rb, ruby:
> ["embed-sinatra.rb"]
> # sinatra/main.rb, exerb:
> ["sinatra/base.rb", "sinatra/main.rb"]
And again copy/pasting from Parallels ate part of my email:
To workaround you can add Sinatra::Application.set :run, true at the
end of exerb-sinatra.rb...
Alexey Borzenkov
And even though caller(1) in exerb is broken (does not register
require calls), it is still sinatra's fault to assume that sinatra
paths start with /. While in ruby they usually do (e.g. ./sinatra.ru,
./sinatra/main.rb, etc.) in exerb they don't (e.g sinatra.rb,
sinatra/main.rb), and so sinatra needs to be patched:
--- base.rb.bak 2009-10-28 09:07:03.265625000 +0300
+++ base.rb 2009-10-28 10:15:14.078125000 +0300
@@ -599,7 +599,7 @@
return trace unless options.clean_trace?
trace.reject { |line|
- line =~ /lib\/sinatra.*\.rb/ ||
+ line =~ /(\A|lib\/)sinatra.*\.rb/ ||
(defined?(Gem) && line.include?(Gem.dir))
}.map! { |line| line.gsub(/^\.\//, '') }
end
@@ -951,7 +951,7 @@
public
CALLERS_TO_IGNORE = [
- /\/sinatra(\/(base|main|showexceptions|compat))?\.rb$/, # all
sinatra code
+ /(\A|\/)sinatra(\/(base|main|showexceptions|compat))?\.rb$/,
# all sinatra code
/\(.*\)/, # generated code
/custom_require\.rb$/, # rubygems require hacks
/active_support/, # active_support require hacks
Luis Lavena
>
> On Wed, Oct 28, 2009 at 9:49 AM, Alexey Borzenkov <sna...@gmail.com> wrote:
>> Seems to be because caller_files is broken
>
> And even though caller(1) in exerb is broken (does not register
> require calls), it is still sinatra's fault to assume that sinatra
> paths start with /. While in ruby they usually do (e.g. ./sinatra.ru,
> ./sinatra/main.rb, etc.) in exerb they don't (e.g sinatra.rb,
> sinatra/main.rb), and so sinatra needs to be patched:
>
I smell a patch will be submitted by you to sinatra ;-)
Alexey Borzenkov
>> And even though caller(1) in exerb is broken (does not register
>> require calls), it is still sinatra's fault to assume that sinatra
>> paths start with /. While in ruby they usually do (e.g. ./sinatra.ru,
>> ./sinatra/main.rb, etc.) in exerb they don't (e.g sinatra.rb,
>> sinatra/main.rb), and so sinatra needs to be patched:
> I smell a patch will be submitted by you to sinatra ;-)
Not by me, no. Please feel free to submit it yourself.
Luis Lavena
Well, was going to submit your patch, but did what you said: indicate
the force to run:
Sinatra::Application.set :run, true
Executed mkexy again, and got a working executable now!
(Backed by Webrick, but is a start) :-D
Alexey Borzenkov
>> And thanks to the manually added zlib1.dll, I no longer need to
>> include these DLLs with the executable
>>
>> The trick is down to dlopen to load the DLL and exerb extracting it at runtime.
> The beauty is that exerb is not extracting it at runtime. :) It has a
> manual dll loader (like py2exe), it loads it just like it loads .so
> files.
Ooops. I went to actually examine the code and can say now that it
DOES extract dlls at runtime to a temporary folder. Worse, exerb uses
pretty dirty patching to replace ruby-msvcrt*.dll name inside modules
with the executable name. It is NOTHING like py2exe's libary loader.
Sorry for spreading my confusion.