This was asked, and asked, and asked too many times, so here we go.
Since I don't want to hold anyone to try it, I'll like to share some progress I've made in this area. I'll like to make a few points clear before:
1) VC6 vs. VC8 vs. MinGW have no room in this discussion. I took the MinGW path since was the only one that could be bootstrapped from scratch.
2) This is a work in progress. Even that I'm 110% busy with work, interest on this seems it deserve be shared, no mater how "alpha" the whole process is.
3) Don't blame me if isn't work or your Rails application don't work either. Isn't complete!. Instead: grab the code, hack it and send me your patches.
4) Make code talk for you. I've heard this too many times on other projects, so please move from simple comments to share your ruby-fu.
5) Part of this project is bring a new build procedure to One-Click Installer, if we succeed, everybody will enjoy the benefits.
For those who aren't aware, I'm the new maintainer of OCI (One-Click Ruby Installer for Windows). Curt give me the kingdom keys a few months back and the latest release was completely build in my office computer.
OCI, in its current state is a 'Package of Ruby Tools and libraries', not a Installer. There are plans to change that.
This is a pure-ruby solution that download and setup a mingw+msys environment to get Ruby build.
It uses some rake recipes to download all the needed packages At this time Ruby 1.8.6 p111 is targeted, but targeting Ruby 1.9 will no have mayor issues.
Dependencies are not compiled, just downloaded for the time being. Tested all the scenarios (pre-built and compile from source) and there are issues with them which I reported previously.
To bootstrap and use this project, you need: - Ruby 1.8.5 or greater - Rake 0.7.3 or greater - Tar [1] and Unzip [2] binary packages from GnuWin32
If someone can provide me a way to unpack 4MB zip, tgz and tar.bz2 files without external dependencies (just ruby) -- you're welcome.
The rake tasks available: rake check # Run tests for the interpreter in the sandbox. rake clean # Remove any temporary products. rake clobber # Remove any generated file. rake compile # Compile the interpreter. rake configure # Run the configure process for the interpreter. rake download # Download all components. rake extract # Extract all downloaded components. rake install # Install the interpreter in the sandbox. rake prepare # Prepare the freshly extracted components.
clean will remove the sandbox and clobber will remove all the downloaded files.
-- Luis Lavena Multimedia systems - A common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity of complete fools. Douglas Adams
>> 1) VC6 vs. VC8 vs. MinGW have no room in this discussion. I took the >> MinGW path since was the only one that could be bootstrapped from >> scratch.
> Just to clarify... What does "bootstrapped from scratch" mean > in this context?
As low as he (we) can go. At the moment, it's in ruby (as per the code presented and requirements), but some of my ideas had most of the stack from a .cmd file - although that may be excessive.
> > 1) VC6 vs. VC8 vs. MinGW have no room in this discussion. I took the > > MinGW path since was the only one that could be bootstrapped from > > scratch.
> Just to clarify... What does "bootstrapped from scratch" mean > in this context?
Don't know if that is the right terminology (since I'm not a native english speaker) but this means:
> >> 1) VC6 vs. VC8 vs. MinGW have no room in this discussion. I took the > >> MinGW path since was the only one that could be bootstrapped from > >> scratch.
> > Just to clarify... What does "bootstrapped from scratch" mean > > in this context?
> As low as he (we) can go. At the moment, it's in ruby (as per the code > presented and requirements), but some of my ideas had most of the > stack from a .cmd file - although that may be excessive.
> js or .vb via WSH anyone?
The idea is Bootstrap Ruby with Ruby, like Rubinius guys are doing.
I must admit that have extensive batch files that I use to bootstrap ruby with VC6 at office, but maintain .CMD and workaround its limitations don't worth the effort.
Also, we are using Ruby for the fun of it, right? ;-) -- Luis Lavena
>>>> 1) VC6 vs. VC8 vs. MinGW have no room in this discussion. I took >>>> the >>>> MinGW path since was the only one that could be bootstrapped from >>>> scratch.
>>> Just to clarify... What does "bootstrapped from scratch" mean >>> in this context?
>> As low as he (we) can go. At the moment, it's in ruby (as per the >> code >> presented and requirements), but some of my ideas had most of the >> stack from a .cmd file - although that may be excessive.
>> js or .vb via WSH anyone?
> The idea is Bootstrap Ruby with Ruby, like Rubinius guys are doing.
> I must admit that have extensive batch files that I use to bootstrap > ruby with VC6 at office, but maintain .CMD and workaround its > limitations don't worth the effort.
> Also, we are using Ruby for the fun of it, right? ;-)
Right, shame one can't do the whole implement Jump, branch and then move on up thing though... ;-)
> This is a pure-ruby solution that download and setup a mingw+msys > environment to get Ruby build.
> It uses some rake recipes to download all the needed packages > At this time Ruby 1.8.6 p111 is targeted, but targeting Ruby 1.9 will > no have mayor issues.
> Dependencies are not compiled, just downloaded for the time being. > Tested all the scenarios (pre-built and compile from source) and there > are issues with them which I reported previously.
> To bootstrap and use this project, you need: > - Ruby 1.8.5 or greater > - Rake 0.7.3 or greater > - Tar [1] and Unzip [2] binary packages from GnuWin32
Aren't they in MSys?
[snip]
> I suggest to those who are interested, help me get this working.
This does look like fun. I have a spare Windows XP license I'm not using and a Linux hosted VMware Workstation 6. :) Are the compiler, linker, "tar" and zip/gzip/bzip2 the only external dependencies you have at the moment?
On 4 ene, 03:03, "M. Edward (Ed) Borasky" <zn...@cesmail.net> wrote:
> Luis Lavena wrote:
> [snip]
> > To bootstrap and use this project, you need: > > - Ruby 1.8.5 or greater > > - Rake 0.7.3 or greater > > - Tar [1] and Unzip [2] binary packages from GnuWin32
> Aren't they in MSys?
Chicken-egg situation:
They are inside the compressed tar.gz and tar.bz2 files... how I'm supposed to get them if I don't decompress them first? :-D
So: if someone have a pure-ruby (with aditional gem or extension) that allow us extract content from:
All without these dependencies, feel free to modify rake/ extracttask :-)
> This does look like fun. I have a spare Windows XP license I'm not using > and a Linux hosted VMware Workstation 6. :) Are the compiler, linker, > "tar" and zip/gzip/bzip2 the only external dependencies you have at the > moment?
You don't need to manually download MinGW or MSYS, only you need get your hands at tar and unzip. (also, have a working ruby, which you could use OCI for that) ;-)
The compiler, linker and dependencies are downloaded automatically.
Right now only two "dependencies" are in place:
- Zlib 1.2.3 - Readline 5.0
I stopped at readline since I got some serious issues with Readline.readline, as posted before to this list and ruby-core.
The thing is that those worked in the past, but right know they don't.
Each time I add a dependency (to get the bundled extension built), I do a 'test-all' procedure and see if it's passing their self-tests.
> On 4 ene, 03:03, "M. Edward (Ed) Borasky" <zn...@cesmail.net> wrote: > > Luis Lavena wrote:
> > [snip]
> > > To bootstrap and use this project, you need: > > > - Ruby 1.8.5 or greater > > > - Rake 0.7.3 or greater > > > - Tar [1] and Unzip [2] binary packages from GnuWin32
> > Aren't they in MSys?
> Chicken-egg situation:
> They are inside the compressed tar.gz and tar.bz2 files... how I'm > supposed to get them if I don't decompress them first? :-D
> So: if someone have a pure-ruby (with aditional gem or extension) that > allow us extract content from:
> All without these dependencies, feel free to modify rake/ > extracttask :-)
If youcan afford relying on ruby stdlib, there is zlib which can do deflate and gunzip.
There is minitar hosted on rubyforge under ruwiki (does not work on unseekable files like stdin but works on gzipped files by using zlib). There is rubyzip (untested, uses zlib for compression). bzip2 remains to be solved but hopefully the basic stuff can be obtained as gzipped archives.
> On 04/01/2008, Luis Lavena <luislav...@gmail.com> wrote: > > On 4 ene, 03:03, "M. Edward (Ed) Borasky" <zn...@cesmail.net> wrote: > > > Luis Lavena wrote:
> > > [snip]
> > > > To bootstrap and use this project, you need: > > > > - Ruby 1.8.5 or greater > > > > - Rake 0.7.3 or greater > > > > - Tar [1] and Unzip [2] binary packages from GnuWin32
> > > Aren't they in MSys?
> > Chicken-egg situation:
> > They are inside the compressed tar.gz and tar.bz2 files... how I'm > > supposed to get them if I don't decompress them first? :-D
> > So: if someone have a pure-ruby (with aditional gem or extension) that > > allow us extract content from:
> > All without these dependencies, feel free to modify rake/ > > extracttask :-)
> If youcan afford relying on ruby stdlib, there is zlib which can do > deflate and gunzip.
> There is minitar hosted on rubyforge under ruwiki (does not work on > unseekable files like stdin but works on gzipped files by using zlib). > There is rubyzip (untested, uses zlib for compression). bzip2 remains > to be solved but hopefully the basic stuff can be obtained as gzipped > archives.
On 4 ene, 09:35, Michal Suchanek <hramr...@centrum.cz> wrote:
> If youcan afford relying on ruby stdlib, there is zlib which can do > deflate and gunzip.
> There is minitar hosted on rubyforge under ruwiki (does not work on > unseekable files like stdin but works on gzipped files by using zlib). > There is rubyzip (untested, uses zlib for compression). bzip2 remains > to be solved but hopefully the basic stuff can be obtained as gzipped > archives.
Since I was moving from a batch-build process to a ruby/rake build process, a few external tools remains as requirement.
MinGW packages are distributed in tar.gz format [1] (which I guess map perfect for Minitar project), but MSYS provides tar.bz2 files [2], which will still need the external tool to uncompress.
In this case, tar.bz2 files can be uncompressed with bsdtar, which is the best implementation I've tested that work with all the file formats. (tgz, tar.gz and tar.bz2)
There is also the memory usage and speed situation, extract 8MB files can take a bit slow using a pure-ruby implementation.
> All without these dependencies, feel free to modify rake/ > extracttask :-)
.gz and .tar.gz files aren't hard to deal with since a working Ruby 1.8.5 is a prerequisite. Look at minitar, and the zlib library is part of a working Ruby 1.8.5 install.
> On 4 ene, 09:35, Michal Suchanek <hramr...@centrum.cz> wrote:
> > If youcan afford relying on ruby stdlib, there is zlib which can do > > deflate and gunzip.
> > There is minitar hosted on rubyforge under ruwiki (does not work on > > unseekable files like stdin but works on gzipped files by using zlib). > > There is rubyzip (untested, uses zlib for compression). bzip2 remains > > to be solved but hopefully the basic stuff can be obtained as gzipped > > archives.
> Since I was moving from a batch-build process to a ruby/rake build > process, a few external tools remains as requirement.
> MinGW packages are distributed in tar.gz format [1] (which I guess map > perfect for Minitar project), but MSYS provides tar.bz2 files [2], > which will still need the external tool to uncompress.
> In this case, tar.bz2 files can be uncompressed with bsdtar, which is > the best implementation I've tested that work with all the file > formats. (tgz, tar.gz and tar.bz2)
> There is also the memory usage and speed situation, extract 8MB files > can take a bit slow using a pure-ruby implementation.
The decompression is handled by zlib so this should not be that much of an issue. Still people extracting on slow machines could see the difference, and there should be no problem with including these unpack tools in the archive containing the build scripts. On the other hand, the minitar might be useful if you wanted to do something special with symlinks (currently it does not extract them at all). Maintaining a patched minitar might be easier than maintaining a patched bsdtar binary.
The rubyzip implementation does not include a working command line tool as far as I can tell. A poor man's unzip using the library is simple but I am not sure it is of much use. Perhaps an external unzip tool would be easier.
Thanks
Michal
#unzip.rb # The example: #Zip::ZipInputStream::open("my.zip") { # |io| # # while (entry = io.get_next_entry) # puts "Contents of #{entry.name}: '#{io.read}'" # end # }
require 'zip/zip'
def open_wr fn elts = fn.split("/") dir = "" elts[0...-1].each{|d| raise RuntimeError, "Invalid filename '#{fn}'." if d == "" or d == ".." dir << d << "/" Dir.mkdir dir unless File.directory? dir } raise RuntimeError, "File exists '#{fn}'." if File.exist? fn File.open(fn,"wb"){|f| yield f} end
Zip::ZipInputStream::open(ARGV[0]) { |io| while (entry = io.get_next_entry) STDERR.puts entry.name # directories are of type file, they just have a trailing slash next if entry.name[-1] == '/'[-1] open_wr(entry.name){|out| out << io.read} end
> On 04/01/2008, Austin Ziegler <halosta...@gmail.com> wrote: >> On Jan 4, 2008 5:34 AM, Luis Lavena <luislav...@gmail.com> wrote: >>> They are inside the compressed tar.gz and tar.bz2 files... how I'm >>> supposed to get them if I don't decompress them first? :-D
>>> So: if someone have a pure-ruby (with aditional gem or extension) >>> that >>> allow us extract content from:
>>> All without these dependencies, feel free to modify rake/ >>> extracttask :-)
>> .gz and .tar.gz files aren't hard to deal with since a working Ruby >> 1.8.5 is a prerequisite. Look at minitar, and the zlib library is >> part >> of a working Ruby 1.8.5 install.
> Yes, it just does not handle bzip2 which is used to compress the mingw > base that contains tar.
> So you still need bzip2, and you can as well include a full > tar/gzip/bzibp2 suite then as it will probably make the decomression > somewhat faster and less memory intensive. It might waste a bit of > space as the minitar source would probably compress a few kilobytes > smaller than the tar binary. Didn't do any benchmarks, though.
> On Jan 4, 2008 5:34 AM, Luis Lavena <luislav...@gmail.com> wrote: > > They are inside the compressed tar.gz and tar.bz2 files... how I'm > > supposed to get them if I don't decompress them first? :-D
> > So: if someone have a pure-ruby (with aditional gem or extension) that > > allow us extract content from:
> > All without these dependencies, feel free to modify rake/ > > extracttask :-)
> .gz and .tar.gz files aren't hard to deal with since a working Ruby > 1.8.5 is a prerequisite. Look at minitar, and the zlib library is part > of a working Ruby 1.8.5 install.
Yes, it just does not handle bzip2 which is used to compress the mingw base that contains tar.
So you still need bzip2, and you can as well include a full tar/gzip/bzibp2 suite then as it will probably make the decomression somewhat faster and less memory intensive. It might waste a bit of space as the minitar source would probably compress a few kilobytes smaller than the tar binary. Didn't do any benchmarks, though.
On 4 ene, 12:44, James Tucker <jftuc...@gmail.com> wrote:
> 7za.exe from the 7z distribution is both open source, and really small.
> Whilst it may not be a 'normal' tool, it very happily deals with all > of these issues in a single app, and common command line interface.
> It's also pretty swift.
Until 'we' switch to other alternatives, either pure-ruby or externals tools, I've uploaded a zip file which contains the needed files to get you running:
On Jan 4, 1:17 pm, Luis Lavena <luislav...@gmail.com> wrote:
> [...]
> Until 'we' switch to other alternatives, either pure-ruby or externals > tools, I've uploaded a zip file which contains the needed files to get > you running:
> [...]
No more!
(bumping this).
Thanks to Dennis Ranke Installer3 nows uses a pure-ruby solution to download the required extracting tools using RubyZip code.
There is no need to install rubyzip gem!
You can grab exported code (revision 36) at the same URL: