Idea: executable stubs to replace batch files

[Luis Lavena]

Hello,

I believe was last year when we discussed on this list with Alexey
Borzenkov (snaury) and Jon about changing batch files (like rake, irb
and others) and use executable files instead.

The purpose of use executable files was for two reasons:

1. To avoid the confusing "Terminate batch job" question that
sometimes appear and confuses when executing complicated things like
"rails server" (which spawns a child process which also ask to
terminate batch)

2. To have an identifiable process under Task manager instead of
countless "ruby.exe" ones.

For the first one I must say is one of the most confusing things to
answer to newcomers "what should I answer to this?"

The second one is not just for aesthetic purposes but also for
firewall permissions too (e.g. grant rackup public network access
while rails only private one)

The original idea was to use something similar to script-runner [1]
Alexey created, but targeted at Ruby. The script will call ruby.exe in
the background with the script name.

While nice, you will still have "ruby.exe" process there and you will
still need to grant permission to ruby.exe.

Instead, I'm exploring the idea of having this executable links and
use ruby API and DLL (e.g. msvcrt-ruby191.dll). Recreate what
"ruby.exe" does except it takes the script name from the executable
name and loads it.

The basic: recreate ruby.exe using Ruby's DLL is simple:

https://gist.github.com/2771839

From there:

1. It needs to obtain the executable name.
2. Find the script file in the same directory.
3. Load the script (maybe push it in the ARGV?)

There is a similar script in Ruby itself called stub.c [2] but the
code is a bit convoluted (besides, doesn't compile or install
properly)

The idea is get this executable and be able to ship it with
RubyInstaller, also changing RubyGems copy the executable stubs
instead of the batch files.

Anyone interested in tackle this mini-project? Anyone interested in
backing this mini-project?

Wanted to remind you guys that Ruby Masters Conference from Brazil
donated money to RubyInstaller last year. This means we have money to
back contributions, purchases of documentation, etc. that contributors
want or need.

Regards,

[1] https://github.com/snaury/script-runner
[2] https://github.com/ruby/ruby/blob/trunk/win32/stub.c
--
Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exupéry

[Boško Ivanišević]

Hi guys,

It's been a long time since I contributed to this project. My daily job took too much time but now I'm back (at least with this small contribution :)).

I just pushed initial commit to new project at http://github.com/bosko/gem-exefy. This is RubyGems plugin which adds new command - exefy. Executing

  gem exefy <gem_name>

will search for all executables in target Gem. For each executable it will try to find corresponding batch file which will, finally, be replaced with .exe file with the name of the Gem executable (batch file name).

Gem is still not ready for publishing. It is still missing DevKit's detection. BTW what would be the best way to detect if DevKit is available, to automatically expand PATH variable to include it while .exe file is created? John, Luis?

Nevertheless Gem is completely functional if you manually execute devkitvars.bat before executing:

  gem exefy...

Of course I would like to hear what you guys think about this.

--
Regards,
Boško Ivanišević

http://pragdevnotes.com

[Luis Lavena]

On Fri, Jun 8, 2012 at 8:05 AM, Boško Ivanišević
<bosko.iv...@gmail.com> wrote:
>
> Hi guys,
>
> It's been a long time since I contributed to this project. My daily job took
> too much time but now I'm back (at least with this small contribution :)).
>

We miss you!

> I just pushed initial commit to new project at
> http://github.com/bosko/gem-exefy. This is RubyGems plugin which adds new
> command - exefy. Executing
>
>   gem exefy <gem_name>
>
> will search for all executables in target Gem. For each executable it will
> try to find corresponding batch file which will, finally, be replaced with
> .exe file with the name of the Gem executable (batch file name).
>
> Gem is still not ready for publishing. It is still missing DevKit's
> detection. BTW what would be the best way to detect if DevKit is available,
> to automatically expand PATH variable to include it while .exe file is
> created? John, Luis?
>

You can:

begin
require "devkit"
rescue LoadError
# ...
end

A proper installation of DevKit will place a "devkit.rb" in the
$LOAD_PATH, you can use that to detect if is present or fail.

I just sent you a pull request with minor changes.

https://github.com/bosko/gem-exefy/pull/1

> Nevertheless Gem is completely functional if you manually execute
> devkitvars.bat before executing:
>
>   gem exefy...
>
> Of course I would like to hear what you guys think about this.
>

This is awesome! thank you for taking the time to work on this!

I noticed that with every gem the compilation of the executable is
triggered, perhaps we would like to cache the executable inside the
Gem datadir and compile only once?

I'll investigate that a bit more and get back to you, in the
meantime... welcome back and thank you!

Cheers,

[Boško Ivanišević]

On Fri, Jun 8, 2012 at 2:47 PM, Luis Lavena <luisl...@gmail.com> wrote:

On Fri, Jun 8, 2012 at 8:05 AM, Boško Ivanišević
<bosko.iv...@gmail.com> wrote:
>
> Hi guys,
>
> It's been a long time since I contributed to this project. My daily job took
> too much time but now I'm back (at least with this small contribution :)).
>

We miss you!

Thanks! Nice to hear that ;-)

 

 

> I just pushed initial commit to new project at
> http://github.com/bosko/gem-exefy. This is RubyGems plugin which adds new
> command - exefy. Executing
>
>   gem exefy <gem_name>
>
> will search for all executables in target Gem. For each executable it will
> try to find corresponding batch file which will, finally, be replaced with
> .exe file with the name of the Gem executable (batch file name).
>
> Gem is still not ready for publishing. It is still missing DevKit's
> detection. BTW what would be the best way to detect if DevKit is available,
> to automatically expand PATH variable to include it while .exe file is
> created? John, Luis?
>

You can:

begin
 require "devkit"
rescue LoadError
 # ...
end

A proper installation of DevKit will place a "devkit.rb" in the
$LOAD_PATH, you can use that to detect if is present or fail.

I knew there was a way to do it. I added it to execute method and pushed to master. Thanks!

I just sent you a pull request with minor changes.

https://github.com/bosko/gem-exefy/pull/1

Merged.

 

> Nevertheless Gem is completely functional if you manually execute
> devkitvars.bat before executing:
>
>   gem exefy...
>
> Of course I would like to hear what you guys think about this.
>

This is awesome! thank you for taking the time to work on this!

The idea you gave is excellent. I just did small implementation ;)

 

I noticed that with every gem the compilation of the executable is
triggered, perhaps we would like to cache the executable inside the
Gem datadir and compile only once?

Executable can be packed in the gem and just copied to target directory with different names so if DevKit is not found we can ask user whether batch file should be replaced with precompiled exe. In the case DevKit is loaded we can make executable in gem-exefy's install directory instead of temporary one and use it later whenever exefy command is issued.

 

I'll investigate that a bit more and get back to you, in the
meantime... welcome back and thank you!

Thanks again. I'll wait till you get back with new ideas. In the mean time it would be good if others test this gem too, so we can release it once we clear up problem with multiple compilation.

[Luis Lavena]

On Fri, Jun 8, 2012 at 10:40 AM, Boško Ivanišević
<bosko.iv...@gmail.com> wrote:
>
> On Fri, Jun 8, 2012 at 2:47 PM, Luis Lavena <luisl...@gmail.com> wrote:
>>
>> I noticed that with every gem the compilation of the executable is
>> triggered, perhaps we would like to cache the executable inside the
>> Gem datadir and compile only once?
>>
>
> Executable can be packed in the gem and just copied to target directory with
> different names so if DevKit is not found we can ask user whether batch file
> should be replaced with precompiled exe. In the case DevKit is loaded we can
> make executable in gem-exefy's install directory instead of temporary one
> and use it later whenever exefy command is issued.
>

That will not work between Ruby 1.8.x and 1.9.x.

I think that firing compilation and placing the result in the gem data
directory will be enough.

Since gems are installed/placed in a directory that follows Ruby API
versioning (e.g. 1.8, 1.9.1) linking to the right ruby dll will be no
problem.

>>
>> I'll investigate that a bit more and get back to you, in the
>> meantime... welcome back and thank you!
>>
> Thanks again. I'll wait till you get back with new ideas. In the mean time
> it would be good if others test this gem too, so we can release it once we
> clear up problem with multiple compilation.
>

Short term goal: make gem-exefy hook gem installation and replace
windows scripts (.bat) with pre-generated .exe file.

Long term goal: Make this part of RubyInstaller and replace Ruby's
.bat with .exe files, also hooking RubyGems to replace batch files.

Really long term goal: conquer the world.

:o)

[Boško Ivanišević]

On Fri, Jun 8, 2012 at 5:01 PM, Luis Lavena <luisl...@gmail.com> wrote:

On Fri, Jun 8, 2012 at 10:40 AM, Boško Ivanišević
<bosko.iv...@gmail.com> wrote:
>
> On Fri, Jun 8, 2012 at 2:47 PM, Luis Lavena <luisl...@gmail.com> wrote:
>>

>> I noticed that with every gem the compilation of the executable is
>> triggered, perhaps we would like to cache the executable inside the
>> Gem datadir and compile only once?
>>
>
> Executable can be packed in the gem and just copied to target directory with
> different names so if DevKit is not found we can ask user whether batch file
> should be replaced with precompiled exe. In the case DevKit is loaded we can
> make executable in gem-exefy's install directory instead of temporary one
> and use it later whenever exefy command is issued.
>

That will not work between Ruby 1.8.x and 1.9.x.

I think that firing compilation and placing the result in the gem data
directory will be enough.

Since gems are installed/placed in a directory that follows Ruby API
versioning (e.g. 1.8, 1.9.1) linking to the right ruby dll will be no
problem.

I agree. I'll add this within next few days.
 

>>
>> I'll investigate that a bit more and get back to you, in the
>> meantime... welcome back and thank you!
>>
> Thanks again. I'll wait till you get back with new ideas. In the mean time
> it would be good if others test this gem too, so we can release it once we
> clear up problem with multiple compilation.
>

Short term goal: make gem-exefy hook gem installation and replace
windows scripts (.bat) with pre-generated .exe file.

That was my initial idea but then I decided to let users to choose whether they want to keep batch files or replace them with exe counterparts. I'll have to find out a way to hook gem-exefy to gem installation. Probably do it in a similar way as DevKit, right?

Long term goal: Make this part of RubyInstaller and replace Ruby's
.bat with .exe files, also hooking RubyGems to replace batch files.

It can be also short term. I think we have almost everything implemented. It would be easy to create additional rake tasks which will replace Ruby's .bat with .exe files. Although I would like to find solution which would not force us to have duplicated code. One in gem-exefy and the other in RubyInstaller recipes. We should think about that.

Really long term goal: conquer the world.
 

:o)

Why wait so long :D

[Luis Lavena]

On Fri, Jun 8, 2012 at 4:06 PM, Boško Ivanišević
<bosko.iv...@gmail.com> wrote:
>
> On Fri, Jun 8, 2012 at 5:01 PM, Luis Lavena <luisl...@gmail.com> wrote:
>>
>> I think that firing compilation and placing the result in the gem data
>> directory will be enough.
>>
>> Since gems are installed/placed in a directory that follows Ruby API
>> versioning (e.g. 1.8, 1.9.1) linking to the right ruby dll will be no
>> problem.
>>
>
> I agree. I'll add this within next few days.
>

I have an idea for this, can't code it right now (deploying some
features to one of our applications) but will get on it later today.

>> >>
>> >> I'll investigate that a bit more and get back to you, in the
>> >> meantime... welcome back and thank you!
>> >>
>> > Thanks again. I'll wait till you get back with new ideas. In the mean
>> > time
>> > it would be good if others test this gem too, so we can release it once
>> > we
>> > clear up problem with multiple compilation.
>> >
>>
>> Short term goal: make gem-exefy hook gem installation and replace
>> windows scripts (.bat) with pre-generated .exe file.
>>
> That was my initial idea but then I decided to let users to choose whether
> they want to keep batch files or replace them with exe counterparts. I'll
> have to find out a way to hook gem-exefy to gem installation. Probably do it
> in a similar way as DevKit, right?
>

You need to implement it as a Gem plugin, hooking part of the gem
installation process.

What DevKit does is provide the "operating_system" default override,
but both DevKit *and* gem-exefy wouldn't be able to co-exist in the
same file (without stepping into each other)

I (personally) find the batch files uber annoying :P

>> Long term goal: Make this part of RubyInstaller and replace Ruby's
>> .bat with .exe files, also hooking RubyGems to replace batch files.
>>
> It can be also short term. I think we have almost everything implemented. It
> would be easy to create additional rake tasks which will replace Ruby's .bat
> with .exe files. Although I would like to find solution which would not
> force us to have duplicated code. One in gem-exefy and the other in
> RubyInstaller recipes. We should think about that.
>

Well, I'm thinking we use gem-exefy right now as proof of concept,
even "bundle" it in the installer (as installed gem)

In the future we might want to make gem-exefy part of Ruby, who knows ;-)

[Boško Ivanišević]

On Fri, Jun 8, 2012 at 9:40 PM, Luis Lavena <luisl...@gmail.com> wrote:

On Fri, Jun 8, 2012 at 4:06 PM, Boško Ivanišević
<bosko.iv...@gmail.com> wrote:
>
> On Fri, Jun 8, 2012 at 5:01 PM, Luis Lavena <luisl...@gmail.com> wrote:
>>

>> I think that firing compilation and placing the result in the gem data
>> directory will be enough.
>>
>> Since gems are installed/placed in a directory that follows Ruby API
>> versioning (e.g. 1.8, 1.9.1) linking to the right ruby dll will be no
>> problem.
>>
>
> I agree. I'll add this within next few days.
>

I have an idea for this, can't code it right now (deploying some
features to one of our applications) but will get on it later today.

I also can't do it right now and since you already have an idea I'll wait for your solution. I gave you write permissions to the repository so you don't have to create pull requests and can work directory on gem-exefy repository.
 

>> >>
>> >> I'll investigate that a bit more and get back to you, in the
>> >> meantime... welcome back and thank you!
>> >>
>> > Thanks again. I'll wait till you get back with new ideas. In the mean
>> > time
>> > it would be good if others test this gem too, so we can release it once
>> > we
>> > clear up problem with multiple compilation.
>> >
>>
>> Short term goal: make gem-exefy hook gem installation and replace
>> windows scripts (.bat) with pre-generated .exe file.
>>
> That was my initial idea but then I decided to let users to choose whether
> they want to keep batch files or replace them with exe counterparts. I'll
> have to find out a way to hook gem-exefy to gem installation. Probably do it
> in a similar way as DevKit, right?
>

You need to implement it as a Gem plugin, hooking part of the gem
installation process.

What DevKit does is provide the "operating_system" default override,
but both DevKit *and* gem-exefy wouldn't be able to co-exist in the
same file (without stepping into each other)

Ok. Will investigate it a little bit and implement it after that.
 

I (personally) find the batch files uber annoying :P

Me too ;-)
 

>> Long term goal: Make this part of RubyInstaller and replace Ruby's
>> .bat with .exe files, also hooking RubyGems to replace batch files.
>>
> It can be also short term. I think we have almost everything implemented. It
> would be easy to create additional rake tasks which will replace Ruby's .bat
> with .exe files. Although I would like to find solution which would not
> force us to have duplicated code. One in gem-exefy and the other in
> RubyInstaller recipes. We should think about that.
>

Well, I'm thinking we use gem-exefy right now as proof of concept,
even "bundle" it in the installer (as installed gem)

I agree.
 

In the future we might want to make gem-exefy part of Ruby, who knows ;-)

That would be nice :)

[Jon]

> >> Short term goal: make gem-exefy hook gem installation and replace
> >> windows scripts (.bat) with pre-generated .exe file.
> >>
> > That was my initial idea but then I decided to let users to choose whether
> > they want to keep batch files or replace them with exe counterparts. I'll
> > have to find out a way to hook gem-exefy to gem installation. Probably do it
> > in a similar way as DevKit, right?
> >
>
> You need to implement it as a Gem plugin, hooking part of the gem
> installation process.
>
> What DevKit does is provide the "operating_system" default override,
> but both DevKit *and* gem-exefy wouldn't be able to co-exist in the
> same file (without stepping into each other)

Hey Boško, good to hear from you again :)

I'm way late to the discussion and haven't looked at gem-exefy yet, but can you guard DevKit and gem-exefy from stepping on each other with something creative in https://github.com/oneclick/rubyinstaller/blob/master/resources/devkit/dk.rb.erb#L48-54

In particular check for presence of https://github.com/oneclick/rubyinstaller/blob/master/resources/devkit/dk.rb.erb#L72 or other ENV trickery.

I'll try to look at gem-exefy this weekend.

Jon

---
Fail fast. Fail often. Fail publicly. Learn. Adapt. Repeat.
http://thecodeshop.github.com | http://jonforums.github.com/
twitter: @jonforums

[Boško Ivanišević]

On Fri, Jun 8, 2012 at 10:09 PM, Jon <jon.f...@gmail.com> wrote:

> >> Short term goal: make gem-exefy hook gem installation and replace
> >> windows scripts (.bat) with pre-generated .exe file.
> >>
> > That was my initial idea but then I decided to let users to choose whether
> > they want to keep batch files or replace them with exe counterparts. I'll
> > have to find out a way to hook gem-exefy to gem installation. Probably do it
> > in a similar way as DevKit, right?
> >
>
> You need to implement it as a Gem plugin, hooking part of the gem
> installation process.
>
> What DevKit does is provide the "operating_system" default override,
> but both DevKit *and* gem-exefy wouldn't be able to co-exist in the
> same file (without stepping into each other)

Hey Boško, good to hear from you again :)

Hey Jon. It's good to be back :-) I'm sorry I missed all fun parts about speeding up Ruby on Windows, which I was following very carefully, btw, but there might be some work there too for me to be done in my free time ;-)
 

I'm way late to the discussion and haven't looked at gem-exefy yet, but can you guard DevKit and gem-exefy from stepping on each other with something creative in https://github.com/oneclick/rubyinstaller/blob/master/resources/devkit/dk.rb.erb#L48-54

You are not late. I always like to hear you opinions and suggestions.

In particular check for presence of https://github.com/oneclick/rubyinstaller/blob/master/resources/devkit/dk.rb.erb#L72 or other ENV trickery.

I think gem-exefy will hook up in RubyGem's post_install hook so it will, by default, replace all .bat files with .exe counterparts. That way it will not override DevKit.

Latest commit which requires devkit (according to Luis suggestion) detects it quite well so we do not need ENV trickery (unless I missed something).

I'll try to look at gem-exefy this weekend.

Please do. I would like to hear what you think about this.

[Justin Baker]

Oh! I can do that!

But, alas I can't run "generate_exe" because it's an instance method. :'(

https://github.com/bosko/gem-exefy/blob/02bfaf0db185afc6329594381a7b11c855531d7f/lib/rubygems/commands/exefy_command.rb#L54

Also, it would be nice if we could extract copying batch files out of that

method so instead the parameters are (executable, target) since on an

install there wouldn't be any batch files to backup.

I'll open some issues!

 

>> Long term goal: Make this part of RubyInstaller and replace Ruby's
>> .bat with .exe files, also hooking RubyGems to replace batch files.
>>
> It can be also short term. I think we have almost everything implemented. It
> would be easy to create additional rake tasks which will replace Ruby's .bat
> with .exe files. Although I would like to find solution which would not
> force us to have duplicated code. One in gem-exefy and the other in
> RubyInstaller recipes. We should think about that.
>

Well, I'm thinking we use gem-exefy right now as proof of concept,
even "bundle" it in the installer (as installed gem)

I agree.
 

In the future we might want to make gem-exefy part of Ruby, who knows ;-)

That would be nice :)

So I'm thinking writing a pre_install hook which requires a file that overwrites the

.bat creation method. That's why I want the generate_exe to not even mess

with the .bat files... Thoughts?

Current Windows stub creation method I would overwrite:

https://github.com/ruby/ruby/blob/trunk/lib/rubygems/installer.rb#L268

Justin

[Luis Lavena]

Justin, take a look to pull request #2.

Sorry for top posting. Sent from mobile.

--
You received this message because you are subscribed to the Google Groups "RubyInstaller" group.
To post to this group, send email to rubyin...@googlegroups.com.
To unsubscribe from this group, send email to rubyinstalle...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rubyinstaller?hl=en.

[Jon]

 

I'll try to look at gem-exefy this weekend.

Please do. I would like to hear what you think about this.

Looks nice. I scanned master@2fda77f and have the following feedback:

* `gem_exe.c:L60` - IIRC, `xmalloc` exits (doesn't return NULL) if mem alloc has problems, but if there's any chance that `xmalloc` ever returns NULL, `memset` on L61 will SEGV
* `exefy_command.rb:link` - is the `libruby_dir` and `libruby` combo correct since `CONFIG['libdir']` have only `*.dll.a` and `*-static.a`? Should `CONFIG['bindir']` be used instead so the link is against the DLL?
* `exefy_command.rb:execute` - safer to backup the .bat's by default than count on the user to remember `-b`. Naming the backups something like `*.bat.orig` might be clearer than `*.bcp`
* `exefy_command.rb:L25` - update the wording similar to "...executed only on a RubyInstaller Windows OS installation"? Similarly, update the `post_install_message` and `exefy_command.rb:initialize`?
* allow gem install only if `RbConfig::CONFIG['host_os'] =~ /mingw/`? Making it hard to accidentally install on `mswin` should save support emails.

I'll try to break away from the big time suck that is my new Snow Leopard VM and MacPorts and play with it a bit more ;)

Jon

[Luis Lavena]

libdir is correct. We link against the import library.

Link against a dll is a MinGW feature that not always work.

Agree about the platform. Perhaps the gem can be forced to be mingw32.

Sorry for top posting. Sent from mobile.

--

You received this message because you are subscribed to the Google Groups "RubyInstaller" group.

To view this discussion on the web visit https://groups.google.com/d/msg/rubyinstaller/-/LFXrENJgLFgJ.

[Jon]

> libdir is correct. We link against the import library.
>
> Link against a dll is a MinGW feature that not always work.

gem-exefy depends upon the MinGW DevKit, so directly linking against the dll is usually preferred for memory and time benefits.

I've never had direct link failures using MinGW. Where have you seen failures?

[Luis Lavena]

On Sun, Jun 10, 2012 at 10:05 PM, Jon <jon.f...@gmail.com> wrote:
>> libdir is correct. We link against the import library.
>>
>> Link against a dll is a MinGW feature that not always work.
>
> gem-exefy depends upon the MinGW DevKit, so directly linking against the dll is usually preferred for memory and time benefits.
>

When you tell ld to link with .dll.a you are telling it to link against the DLL.

Also, it is consistent with how Ruby builds everything.

> I've never had direct link failures using MinGW. Where have you seen failures?
>

Yes, back in Ruby 1.8.5, 1.8.6, don't quite remember the details but
made me avoid linking to DLLs and always use the import libraries
instead.

[Jon]

> >> libdir is correct. We link against the import library.
> >>
> >> Link against a dll is a MinGW feature that not always work.
> >
> > gem-exefy depends upon the MinGW DevKit, so directly linking against the dll is usually preferred for memory and time benefits.
> >
>
> When you tell ld to link with .dll.a you are telling it to link against the DLL.
>
> Also, it is consistent with how Ruby builds everything.
>
> > I've never had direct link failures using MinGW. Where have you seen failures?
> >
>
> Yes, back in Ruby 1.8.5, 1.8.6, don't quite remember the details but
> made me avoid linking to DLLs and always use the import libraries
> instead.

If you recall the specifics, let me know as it's definitely worth remembering.

gem-exefy isn't complex so it's likely a don't-care, but see if the "direct linking to a dll" section of

http://sourceware.org/binutils/docs/ld/WIN32.html

causes you to favor a direct dll link rather than a `libmsvcrt-ruby191.dll.a` import lib link.

[Luis Lavena]

Have you tried it? My proposed way of linking follows the same rules ruby links itself.

I believe my issue with linking was function decorators and stdcall used in the msvcrt replacement functions provided by ruby. Again, that was long time ago and I believe wasted lot of time on that, so wanted to stay away and burn again.

Sorry for top posting. Sent from mobile.

--
You received this message because you are subscribed to the Google Groups "RubyInstaller" group.

[Boško Ivanišević]

On Mon, Jun 11, 2012 at 12:24 AM, Jon <jon.f...@gmail.com> wrote:

 

I'll try to look at gem-exefy this weekend.

Please do. I would like to hear what you think about this.

Looks nice. I scanned master@2fda77f and have the following feedback:

* `gem_exe.c:L60` - IIRC, `xmalloc` exits (doesn't return NULL) if mem alloc has problems, but if there's any chance that `xmalloc` ever returns NULL, `memset` on L61 will SEGV

Good point. Fixed.

 

* `exefy_command.rb:link` - is the `libruby_dir` and `libruby` combo correct since `CONFIG['libdir']` have only `*.dll.a` and `*-static.a`? Should `CONFIG['bindir']` be used instead so the link is against the DLL?

I don't think it will make any difference in this case since this is really simple application but I tried both ways of linking on Ruby 1.9.2 and there are small differences. Executable created with direct linking is slightly bigger but it imports one method less. Imported methods from msvcrt-ruby191.dll are:

getenv <---- When we link against the import library

ruby_init

ruby_init_stack

ruby_options

ruby_run_node

ruby_sysinit

ruby_xmalloc

I think we should stay on the safe side and link against import library but keep on our minds that we can switch to direct linking if we found performances to be bad.

 

* `exefy_command.rb:execute` - safer to backup the .bat's by default than count on the user to remember `-b`. Naming the backups something like `*.bat.orig` might be clearer than `*.bcp`

Personally I use both versions but .bat.orig is probably better. I've changed this too.

 

* `exefy_command.rb:L25` - update the wording similar to "...executed only on a RubyInstaller Windows OS installation"? Similarly, update the `post_install_message` and `exefy_command.rb:initialize`?

I changed existing messages and added one more sentence in post-install message. I'm not sure I made it clearer so if you have any suggestion for post-install message I'll be glad to change this one.

* allow gem install only if `RbConfig::CONFIG['host_os'] =~ /mingw/`? Making it hard to accidentally install on `mswin` should save support emails.

Still trying to find a way to do this. I can't even set up Gem::Specification's platform through Hoe.spec so platform in built gem is always 'ruby'.

I'll try to break away from the big time suck that is my new Snow Leopard VM and MacPorts and play with it a bit more ;)

Jon

--

You received this message because you are subscribed to the Google Groups "RubyInstaller" group.

To view this discussion on the web visit https://groups.google.com/d/msg/rubyinstaller/-/LFXrENJgLFgJ.

To post to this group, send email to rubyin...@googlegroups.com.
To unsubscribe from this group, send email to rubyinstalle...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rubyinstaller?hl=en.

[Boško Ivanišević]

I merged Azolo's pre_install_hook branch to the branch with same name. Branch was rebased on current master so it includes changes suggested by Jon. I  fixed handling non-mingw installation and added uninstallation of generated .exe files. It would be good if you guys can review changes in pre_install_hook branch and if there are no errors or remarks I would merge it to the master.

[Luis Lavena]

On Mon, Jun 11, 2012 at 4:40 AM, Boško Ivanišević
<bosko.iv...@gmail.com> wrote:
> On Mon, Jun 11, 2012 at 12:24 AM, Jon <jon.f...@gmail.com> wrote:
>
>> * allow gem install only if `RbConfig::CONFIG['host_os'] =~ /mingw/`?
>> Making it hard to accidentally install on `mswin` should save support
>> emails.
>>
> Still trying to find a way to do this. I can't even set up
> Gem::Specification's platform through Hoe.spec so platform in built gem is
> always 'ruby'.
>

platform is not exposed through Hoe, you need to use `spec_extras` in
the Hoe.spec block:

spec_extras[:platform] = Gem::Platform::CURRENT

[Boško Ivanišević]

On Mon, Jun 11, 2012 at 3:46 PM, Luis Lavena <luisl...@gmail.com> wrote:

On Mon, Jun 11, 2012 at 4:40 AM, Boško Ivanišević
<bosko.iv...@gmail.com> wrote:
> On Mon, Jun 11, 2012 at 12:24 AM, Jon <jon.f...@gmail.com> wrote:
>

>> * allow gem install only if `RbConfig::CONFIG['host_os'] =~ /mingw/`?
>> Making it hard to accidentally install on `mswin` should save support
>> emails.
>>
> Still trying to find a way to do this. I can't even set up
> Gem::Specification's platform through Hoe.spec so platform in built gem is
> always 'ruby'.
>

platform is not exposed through Hoe, you need to use `spec_extras` in
the Hoe.spec block:

 spec_extras[:platform] = Gem::Platform::CURRENT

Great! I didn't know about this. Should we add this so we only build mingw gem?

[Luis Lavena]

Yes please, want this be an exclusive feature ;-)

[Boško Ivanišević]

On Mon, Jun 11, 2012 at 4:53 PM, Luis Lavena <luisl...@gmail.com> wrote:

On Mon, Jun 11, 2012 at 10:57 AM, Boško Ivanišević
<bosko.iv...@gmail.com> wrote:
>
> On Mon, Jun 11, 2012 at 3:46 PM, Luis Lavena <luisl...@gmail.com> wrote:
>>

>> platform is not exposed through Hoe, you need to use `spec_extras` in
>> the Hoe.spec block:
>>
>>  spec_extras[:platform] = Gem::Platform::CURRENT
>>
>
> Great! I didn't know about this. Should we add this so we only build mingw
> gem?
>

Yes please, want this be an exclusive feature ;-)

:-) Added to the pre_install_hook branch. I am optimist and believe we will merge it  to the master ;-)

[Boško Ivanišević]

Is this silence due to the lack of time to check new branch :( or you guys do not have any remarks and I can merge pre_install_hook to the master :-)? Do you agree with me that we are near, or even better, at the point where I can release gem?

Of course if you do not agree with me let's see what else we can/should improve/fix/implement in order to reach 0.0.1 release.

[Luis Lavena]

On Tue, Jun 12, 2012 at 5:25 AM, Boško Ivanišević
<bosko.iv...@gmail.com> wrote:
>
> Is this silence due to the lack of time to check new branch :( or you guys
> do not have any remarks and I can merge pre_install_hook to the master :-)?

Sorry, was traveling. I'll check the branch later today.

[Jon]

> > Is this silence due to the lack of time to check new branch :( or you guys
> do not have any remarks and I can merge pre_install_hook to the master :-)?
> Do you agree with me that we are near, or even better, at the point where I
> can release gem?

The former. Finishing a new client meeting today and will test out the new branch during some downtime at the airport.

[Boško Ivanišević]

Luis, Jon

Thanks guys! I am waiting for your remarks and suggestions.

[Jon]

> Luis, Jon
>
> Thanks guys! I am waiting for your remarks and suggestions.

Here's some quick test results before the airport lounge cocktails kick in and everything goes south.

Bottom line
-----------
* need a bit more helpful info/error messages
* `data/gemstub.exe` needs to be stripped before using it as the .exe template
* backup .bat's by default; don't make a user remember `-b`
* consider adding `-a, --all` to exefy all gems at once



C:\Users\Jon\Documents>ruby --version
ruby 1.9.3p194 (2012-04-20) [i386-mingw32]

C:\Users\Jon>gem i gem-exefy-0.0.1-x86-mingw32.gem
**************************************************

Thank you for installing gem-exefy-0.0.1!

This gem will work only on RubyInstaller versions of Ruby with installed DevKit.

**************************************************
Successfully installed gem-exefy-0.0.1-x86-mingw32
1 gem installed


# what commands are available with gem-exefy? [DevKit is *not* installed]
C:\Users\Jon\Documents>gem help commands
ERROR: Loading command: exefy (LoadError)
cannot load such file -- devkit
ERROR: While executing gem ... (NameError)
uninitialized constant Gem::Commands::ExefyCommand


# OK, installed DevKit...what new commands exist?
C:\Users\Jon>gem help commands
Temporarily enhancing PATH to include DevKit...
GEM commands are:

build Build a gem from a gemspec
...
exefy Replaces Gem's batch file with executable file
(RubyInstaller installation only)


# how do I use `gem exefy`?
C:\Users\Jon>gem help exefy
Temporarily enhancing PATH to include DevKit...
Usage: gem exefy [options]
...


# Eh? the help command didn't tell me I needed to select a specific gem!
# Hey, where's the `-a, --all` option to exefy all my gems in one shot
C:\Users\Jon>gem exefy
Temporarily enhancing PATH to include DevKit...
ERROR: While executing gem ... (Gem::CommandLineError)
Please specify at least one gem name (e.g. gem build GEMNAME)


# Hey, this isn't very verbose...did anything happen?
C:\Users\Jon>gem exefy -v eventmachine
Temporarily enhancing PATH to include DevKit...


# Ok, now somthing happened...oh nooo, you deleted my .bat file by default :(
C:\Users\Jon>gem exefy thor
Temporarily enhancing PATH to include DevKit...
Generating executable 'C:/ruby-test/193/lib/ruby/gems/1.9.1/gems/gem-exefy-0.0.1-x86-mingw32/data/gemstub.exe'...
Creating executable as 'rake2thor.exe'
Removing batch file 'rake2thor.bat'
Creating executable as 'thor.exe'
Removing batch file 'thor.bat'


# remember to use `-b` to backup .bat
C:\Users\Jon>gem exefy erubis -b -v
Temporarily enhancing PATH to include DevKit...
Creating executable as 'erubis.exe'
Creating backup of 'erubis.bat' batch file

# manually delete `erubis.exe` and try again without `-v`
# did anything happen?...manually check...Ok, I see the new `erubis.exe`
C:\Users\Jon>gem exefy erubis -b
Temporarily enhancing PATH to include DevKit...


# Let's try it again with `thor`...>:->
# did anything happen?...i can't tell
C:\Users\Jon>gem exefy -v thor
Temporarily enhancing PATH to include DevKit...


# BOGUS: I see bin/testrb.bat...let's exefy it!
C:\Users\Jon>gem exefy testrb
Temporarily enhancing PATH to include DevKit...
Cannot exefy. Gem testrb not found


# quick smoke test .exe vs. bat cmd line option handling
C:\Users\Jon>gem exefy tilt
Temporarily enhancing PATH to include DevKit...

C:\Users\Jon>echo "Answer: <%= 2 + n %>" | tilt -t erb --vars="{n: 100}"
"Answer: 102"


# hey, those gemstub .exe's look too porky...strip 'em
C:\Users\Jon>dir \ruby-test\193\bin\thor.exe
06/12/2012 02:30 PM 142,487 thor.exe

C:\Users\Jon>strip -s \ruby-test\193\bin\thor.exe

C:\Users\Jon>dir \ruby-test\193\bin\thor.exe
06/12/2012 02:50 PM 29,198 thor.exe

[Justin Baker]

Here's some quick test results before the airport lounge cocktails kick in and everything goes south.

Bottom line
-----------
 * need a bit more helpful info/error messages
 * `data/gemstub.exe` needs to be stripped before using it as the .exe template
 * backup .bat's by default; don't make a user remember `-b` 

 * consider adding `-a, --all` to exefy all gems at once

I don't know about making a backup by default. 

I would even go as far as to make the argument to not allow them to be backed up at all.

I think I would rather have a revert option on `gem exefy` itself then another option that lets you elect to 

not remove the *.bat* scripts (in case you depend on them).

But I think that backing them up could cause left over scripts after an uninstall, plus we have to mess with

actually keeping track of the back-ups.

Basically, I think that options through the `gem` utility itself instead of creating back-ups of files would be

better.

Besides stripping the stub, which I don't feel like I have enough experience to make a comment about,

everything else looks good. I like your detailed examples too, thanks for that. =)

@Bosko 

I think you should go ahead and merge it, I think it's that's the direction everyone wants the gem to go anyway. 

At least that way we can go ahead and keep developing on it and moving it forward. 

We can always change it or revert it if we need to. I'm young therefore I get to think carelessly like that.

Justin

[Boško Ivanišević]

These are excellent and detailed testing results. Thanks a lot, Jon!

My opinion about removing .bat files by default is same as Justin's. If someone installs gem-exefy then we can assume he wants to get rid of old batch files and wants to use .exe versions instead. Moreover making batch files backups by default would not be consistent with gems installed after gem-exefy installation since gem-exefy will install only .exe file. In the case of creating backups by default we would have to install batch files always, replace them with .exe and make backups. It seems to me as unnecessary complication. Further comments are below.

You are right about this. I  improved help message, improved missing DevKit handling and avoid loading DevKit when it is not needed (i.e. on 'gem help commands' there is no need to load DevKit).

# Hey, this isn't very verbose...did anything happen?
C:\Users\Jon>gem exefy -v eventmachine
Temporarily enhancing PATH to include DevKit...

You have to use '-V' (capital letter 'V') not '-v'.

 

# Ok, now somthing happened...oh nooo, you deleted my .bat file by default :(
C:\Users\Jon>gem exefy thor
Temporarily enhancing PATH to include DevKit...
Generating executable 'C:/ruby-test/193/lib/ruby/gems/1.9.1/gems/gem-exefy-0.0.1-x86-mingw32/data/gemstub.exe'...
Creating executable as 'rake2thor.exe'
Removing batch file 'rake2thor.bat'
Creating executable as 'thor.exe'
Removing batch file 'thor.bat'


# remember to use `-b` to backup .bat
C:\Users\Jon>gem exefy erubis -b -v
Temporarily enhancing PATH to include DevKit...
Creating executable as 'erubis.exe'
Creating backup of 'erubis.bat' batch file

See beginning of message.

 

# manually delete `erubis.exe` and try again without `-v`
# did anything happen?...manually check...Ok, I see the new `erubis.exe`
C:\Users\Jon>gem exefy erubis -b
Temporarily enhancing PATH to include DevKit...


# Let's try it again with `thor`...>:->
# did anything happen?...i can't tell
C:\Users\Jon>gem exefy -v thor
Temporarily enhancing PATH to include DevKit...

See above comment 

# BOGUS: I see bin/testrb.bat...let's exefy it!
C:\Users\Jon>gem exefy testrb
Temporarily enhancing PATH to include DevKit...
Cannot exefy. Gem testrb not found

Of course. File testrb is not part of gem. This batch file is part of Ruby distribution. I hope we can move this implementation to Ruby core later but let's do it one step at the time.

 

# quick smoke test .exe vs. bat cmd line option handling
C:\Users\Jon>gem exefy tilt
Temporarily enhancing PATH to include DevKit...

C:\Users\Jon>echo "Answer: <%= 2 + n %>" | tilt -t erb --vars="{n: 100}"
"Answer: 102"


# hey, those gemstub .exe's look too porky...strip 'em
C:\Users\Jon>dir \ruby-test\193\bin\thor.exe
06/12/2012  02:30 PM           142,487 thor.exe

C:\Users\Jon>strip -s \ruby-test\193\bin\thor.exe

C:\Users\Jon>dir \ruby-test\193\bin\thor.exe
06/12/2012  02:50 PM            29,198 thor.exe

I agree. File gemstub.exe is stripped now.

All changes are still in the pre_install_hook branch. Please test latest changes once again so we are sure I haven't broken anything. I did test it but we can never be too careful;)

@Justin

I agree with you. I think we all agree that this is right direction. However I will wait for few days to give Luis time to send his remarks. In the mean time I will continue fixes and new implementation in this branch. Hopefully I will soon merge this branch to master and release 0.0.1 version :-)

[Luis Lavena]

One last comment: we need to test scripts located in different GEM_PATH than GEM_HOME.

The other test is installation in a non-ascii directory.

Sorry for top posting. Sent from mobile.

--
You received this message because you are subscribed to the Google Groups "RubyInstaller" group.

[Boško Ivanišević]

On Wed, Jun 13, 2012 at 1:54 PM, Luis Lavena <luisl...@gmail.com> wrote:

One last comment: we need to test scripts located in different GEM_PATH than GEM_HOME.

The other test is installation in a non-ascii directory.

I just tried:

  gem i bundler --user-install

with previously installed gem-exefy and .exe file was correctly placed in the installed bin directory within my home folder. After that I created two folders D:\Ruby\boško.ivanišević and D:\Ruby\бошко.иванишевић and tried:

  gem i bundler --install-dir D:\Ruby\boško.ivanišević

  gem i bundler --install-dir D:\Ruby\бошко.иванишевић

but both commands failed with same message:

D:/Ruby/Ruby192/lib/ruby/site_ruby/1.9.1/rubygems/config_file.rb:171:in `block in initialize': invalid byte sequence in UTF-8 (ArgumentError)

        from D:/Ruby/Ruby192/lib/ruby/site_ruby/1.9.1/rubygems/config_file.rb:166:in `map'

        from D:/Ruby/Ruby192/lib/ruby/site_ruby/1.9.1/rubygems/config_file.rb:166:in `initialize'

        from D:/Ruby/Ruby192/lib/ruby/site_ruby/1.9.1/rubygems/gem_runner.rb:78:in `new'

        from D:/Ruby/Ruby192/lib/ruby/site_ruby/1.9.1/rubygems/gem_runner.rb:78:in `do_configuration'

        from D:/Ruby/Ruby192/lib/ruby/site_ruby/1.9.1/rubygems/gem_runner.rb:51:in `run'

        from D:/Ruby/Ruby192/bin/gem:30:in `<main>'

Obviously error is not caused by gem-exefy.

Are these test what you meant or I missed something?

[Luis Lavena]

Thanks for testing, meant either ruby or gems relocated to a directory with non-ascii directory.

I've seen this done by some users. Is not gem installation itself but later.

Sorry for top posting. Sent from mobile.

[Boško Ivanišević]

On Wed, Jun 13, 2012 at 2:26 PM, Luis Lavena <luisl...@gmail.com> wrote:

Thanks for testing, meant either ruby or gems relocated to a directory with non-ascii directory.

I've seen this done by some users. Is not gem installation itself but later.

Seems like I didn't understand previous message. Do you mean I should move one of my installed Ruby versions to a folder with non-ascii directory and than try to run

  gem exefy

on some of installed gems?

[Luis Lavena]

Sorry, limited mobile device is no excuse for my limited description.

Meant: install ruby, exefy scripts, relocate then test.. :-D

Thank you.

Sorry for top posting. Sent from mobile.

[Boško Ivanišević]

On Wed, Jun 13, 2012 at 2:43 PM, Luis Lavena <luisl...@gmail.com> wrote:

Sorry, limited mobile device is no excuse for my limited description.

Meant: install ruby, exefy scripts, relocate then test.. :-D

Tested with code page 65001, 1250 and 437 and it does not work. But invoking scripts directly does not work too:

D:\>bundle help

Script d:\boško.ivaniševic\Ruby192\bin\bundle is missing!

D:\>ruby D:\boško.ivanišević\Ruby192\bin\bundle helpruby: No such file or directory -- D:/bo�ko.ivani�evic/Ruby192/bin/bundle (LoadError)

So this obviously is not problem of gem-exefy but Ruby.

[Luis Lavena]

Can you test against 1.9.3?

1. 9.2 is broken for unicode files and directories.

Sorry for top posting. Sent from mobile.

[Boško Ivanišević]

On Wed, Jun 13, 2012 at 3:03 PM, Luis Lavena <luisl...@gmail.com> wrote:

Can you test against 1.9.3?

1. 9.2 is broken for unicode files and directories.

Same.

D:\projects\ruby\gem-exefy>ruby -v

ruby 1.9.3p194 (2012-04-20) [i386-mingw32]

D:\projects\ruby\gem-exefy>where ruby

d:\boško.ivanišević\Ruby193\bin\ruby.exe

D:\projects\ruby\gem-exefy>bundle help

Script d:\boško.ivaniševic\Ruby193\bin\bundle is missing!

D:\projects\ruby\gem-exefy>ruby d:\boško.ivanišević\Ruby193\bin\bundle help

<internal:gem_prelude>:1:in `require': cannot load such file -- rubygems.rb (LoadError)

        from <internal:gem_prelude>:1:in `<compiled>'

[Boško Ivanišević]

While we are still on this development branch I would like to ask you guys about opinion how exefying all gems feature should be implemented. I think it should work like update command. This means if gem name is given then only for that gem batch file is replaced with exe. On the other hand if no name is given:

gem exefy

batch files for all gems will be replaced. This way we would be consistent with existing implementation. Maybe we should ask user for confirmation before we start replacing batch files for all gems?

[Jon]

I like the consistency with `gem update` idea (rather than `-a, --all`
option) and asking for confirmation when replacing batch files for all
gems.

[Jon]

> My opinion about removing .bat files by default is same as Justin's. If
> someone installs gem-exefy then we can assume he wants to get rid of old
> batch files and wants to use .exe versions instead. Moreover making batch
> files backups by default would not be consistent with gems installed after
> gem-exefy installation since gem-exefy will install only .exe file. In the
> case of creating backups by default we would have to install batch files
> always, replace them with .exe and make backups. It seems to me as
> unnecessary complication. Further comments are below.

You both are quite persuasive and I don't fundamentally disagree.

However, as exefy is a destructive operation, we don't want to
accidentally brick a user's gem install if the .exe doesn't work for
their environment. If exefy does brick a gem's install, the current
fix is likely an uninstall/install cycle. It's debatable whether
that's easier than manually deleting the .exe and renaming *.bat.org
-> *.bat.

I've gotten used to how Arch deals with pacman (pkg mgr) updates. It
always creates a *.pacnew for both pacman.conf (user config) and it's
mirrors list (not usually user edited so more similar to our .bat
situation). It's very conservative and doesn't run the risk of users
(rightly or wrongly) claiming "pacman brick'd my system".

Whatever the final decision, recovery should be (a) readily apparent,
and (b) easy for a user.

That said, I don't view the current behavior as a showstopper and will
play with the uninstall/install behavior a bit more.

[Boško Ivanišević]

I must say you really have good points. Now I'm rethinking about it once again :-) Luis, what do you think about this (and the whole branch, of course)?

[Luis Lavena]

On Wed, Jun 13, 2012 at 2:36 PM, Boško Ivanišević
<bosko.iv...@gmail.com> wrote:
>
> I must say you really have good points. Now I'm rethinking about it once
> again :-) Luis, what do you think about this (and the whole branch, of
> course)?
>

I'm with limited access to internet, but:

gem exefy should work like pristine:

gem pristine foo

For a single gem, or:

gem pristine --all

For all.

To revert:

gem exefy foo --revert

Or:

gem exefy --revert --all

That will make operation really simple.

Revert doesn't need to have the original .bat file, we can find the
executables and create the right one using RubyGems original windows
script method, so we don't need to keep them all along.

Just my thoughts, sorry for the lack of "timing" but I'm on limited
internet access as I'm traveling right now.

Cheers,

[Justin Baker]

> I must say you really have good points. Now I'm rethinking about it once
> again :-) Luis, what do you think about this (and the whole branch, of
> course)?

I'm with limited access to internet, but:

gem exefy should work like pristine:

gem pristine foo

For a single gem, or:

gem pristine --all

For all.

To revert:

gem exefy foo --revert

Or:

gem exefy --revert --all

That will make operation really simple.

Revert doesn't need to have the original .bat file, we can find the
executables and create the right one using RubyGems original windows
script method, so we don't need to keep them all along.

Just my thoughts, sorry for the lack of "timing" but I'm on limited
internet access as I'm traveling right now.

I like this. It makes me think we may be over thinking the entire batch file thing.

If you're using `gem-exefy` you probably know what you're getting into.

That being said, we could also take the `gem rdoc` approach instead of `pristine`.

Meaning we could provide `--[no-]exe` and `--[no-]bat` options.

But I think like Luis' idea better.

Justin

[Boško Ivanišević]

On Wed, Jun 13, 2012 at 8:27 PM, Luis Lavena <luisl...@gmail.com> wrote:

On Wed, Jun 13, 2012 at 2:36 PM, Boško Ivanišević
<bosko.iv...@gmail.com> wrote:
>
> I must say you really have good points. Now I'm rethinking about it once
> again :-) Luis, what do you think about this (and the whole branch, of
> course)?
>

I'm with limited access to internet, but:

gem exefy should work like pristine:

gem pristine foo

For a single gem, or:

gem pristine --all

For all.

To revert:

gem exefy foo --revert

Or:

gem exefy --revert --all

That will make operation really simple.

Revert doesn't need to have the original .bat file, we can find the
executables and create the right one using RubyGems original windows
script method, so we don't need to keep them all along.

Just my thoughts, sorry for the lack of "timing" but I'm on limited
internet access as I'm traveling right now.

I like your idea. I'll merge branch tomorrow and implement --all and --revert actions. That way we will not have to keep backup by default, so we will keep that part. Jon is that OK with you?

[Jon]

On Wed, Jun 13, 2012 at 3:00 PM, Boško Ivanišević

Looks superb.

If I didn't think +1 usage was so lame, I'd give it a +7.

[Boško Ivanišević]

Just wanted to check before I continue with implementation. If revert feature is implemented do we really need batch files backup? I would skip these backups because it will be simple to get them back.

[Jon]

On Wed, Jun 13, 2012 at 4:34 PM, Boško Ivanišević

I also see no need for the FS batch backups if revert can recreate them.