During a discussion on IRC, I started to wonder if Ruby's install
scripts are state of the art, what could be done better and how.
Ruby's mkmf.rb and Aoki's setup.rb probably have their roots in the
oldest pieces of Ruby source still in use. While setup.rb had some
changes in the latter time, mkmf.rb more or less stayed the same.
I have looked into how other languages install source and compile
extensions, and the library I liked best so far is Python's distutils.
I'm not very familiar with Python, but I like the general approach and
the essence of API. Basically, you create a file, setup.py, like
this:
from distutils.core import setup
setup (name = "Distutils",
version = "0.1.1",
description = "Python Module Distribution Utilities",
author = "Greg Ward",
author_email = "gw...@python.net",
url = "http://www.python.org/sigs/distutils-sig/",
packages = ['distutils', 'distutils.command'])
In Ruby, this would maybe look like that:
require 'package'
setup {
name "Distutils"
version "0.1.1"
description "Python Module Distribution Utilities"
author "Greg Ward"
author_email "gw...@python.net"
url "http://www.python.org/sigs/distutils-sig/"
packages ['distutils', 'distutils/command']
}
Given this file, we can simply run:
python setup.py install
and the files will get installed where they belong to. distutils can
also handle different prefixes, installing into home directories, and
complex cases like putting scripts to /usr/bin, but libraries to
/opt/local and whatever.
Python's distutils also handles compiling extensions:
name = 'DateTime.mxDateTime.mxDateTime'
src = 'mxDateTime/mxDateTime.c'
setup (
...
ext_modules =
[(name,
{ 'sources': [src]
'include_dirs': ['mxDateTime'] }
)]
)
Here, something like this would be possible in Ruby (I'm not yet sure
about exact semantics of the Python version):
setup {
# ...
extension("DateTime/mxDateTime/mxDateTime") {
sources "mxDateTime/mxDateTime.c"
include_dirs "mxDateTime"
}
}
Of course, more complex build descriptions can be represented too:
extension("foolib") {
sources "foo.c", "bar.c"
if have_library("foo", "fooble")
define "HAVE_FOO_H"
cflags << `foo-config --cflags`
ldflags << `foo-config --libs`
else
fail "foolib is needed"
end
}
Whether this will generate a Makefile (like mkmf.rb), a Rakefile
or compile directly (like distutils) is still an open question.
To allow for an easy conversion of setup.rb usage, Package will
provide convenience methods that will make it behave like setup.rb
with respect to the directory structure.
Package doesn't try to conquer the world, however, it aims to be just
a tool that would be useful if it was standard and everyone could
build on due to it's policy-neutrality
What advantages will Package have over setup.rb and mkmf.rb, as
they are now?
* simple, clean and consistent working
* unified library to handle both extensions and libraries
* lightweight approach (if included in the standard library)
* easy adaption
* more flexible directory layout: especially small projects
profit from this, as setup.rb's directory layout is quite
bulky by default and not very customizable
* easier packaging by third-party packagers due to simple
but flexible and standardized invocation
What do we need to get a wide adoption of Package?
* inclusion in the standard library so it doesn't need to be
shipped with every package (as setup.rb unfortunately is).
* backing from the community to make use of Package.
* acceptance from packaging projects like RPA, RubyGems and
distributions like Debian, FreeBSD and PLD.
Coding of Package has not started yet (the name is also not set into
stone yet, so if you have better ideas, please tell me) because it
would be pointless without a strong feedback from the community. I
expect to get a first version done rather quickly, possibly borrowing
code from setup.rb and mkmf.rb, but Package will not depend on these
both. If anyone is interested in helping development, please mail me;
helpful hands are always of use. Also, there will be need for testers
on all and even the most weird platforms.
But now, I'll ask you: Are you satisfied with the way installing Ruby
extensions and libraries works? Do you think there is a place for
Package? Do you have further improvements or can provide alternative
ideas?
--
Christian Neukirchen <chneuk...@gmail.com> http://chneukirchen.org
I think that there is room for improvement, definitely. However, if
you're going to do extension support with this, it absolutely *must*
work perfectly well on Windows. It has to work better than setup.rb,
and setup.rb works mostly well for that. Gems, much less so.
-austin
--
Austin Ziegler * halos...@gmail.com
* Alternate: aus...@halostatue.ca
> In Ruby, this would maybe look like that:
>
> require 'package'
>
> setup {
> name "Distutils"
> version "0.1.1"
> description "Python Module Distribution Utilities"
> author "Greg Ward"
> author_email "gw...@python.net"
> url "http://www.python.org/sigs/distutils-sig/"
>
> packages ['distutils', 'distutils/command']
> }
If you have #name(arg) do the same thing as #name=(arg), how
does one get access to the instance variable?
> Package doesn't try to conquer the world, however, it aims to be just
> a tool that would be useful if it was standard and everyone could
> build on due to it's policy-neutrality
> What advantages will Package have over setup.rb and mkmf.rb, as
> they are now?
I find mkmf.rb very useful to create a Makefile when I am building a C wrapper
to some library. Will Package do the same thing, or will mkmf still
be needed?
--
Jim Freeze
Theory and practice are the same, in theory. -- Ryan Davis
No. setup.rb is OK but inflexible. Package management (gems etc.)
nor build management (Rant, Rake) should not be in any way incorporated
in 'Package', either.
Might be a good idea to have the extension builder as a completely
separate tool, too.
Just a simple, easy packager/setup utility, please!
(I will help, too, if I can!)
>Christian Neukirchen
E
--
template<typename duck>
void quack(duck& d) { d.quack(); }
That isn't too hard to implement.
def variable(*args)
@variable = args.first unless args.empty?
@variable
end
Now we can be even more like Smalltalk!
Seriously, though, I think a Hash would both make more sense there (if
I understand correctly what it's doing) and be more Ruby-like than
hacking the setter syntax in a block. Though I guess maybe people don't
like typing equals signs. I don't know.
> I started to wonder if Ruby's install
> scripts are state of the art, what could be done better and how.
clap, clap.
this is wonderfull: we need it.
> Basically, you create a file, setup.py
I can suggest you to look at scons (http://www.scons.org)
A great thing would be to replace:
- configure make and make install
As said on this mailing list to rake author you could:
work on a configure script template like configure.in.
and provide macro as ruby functions like:
- Ac.init_automake("package", 1.0.1)
and Makefile.am that generate a Makefile.in and and a Makefile.rb
- Am.subdirs(src)
- Am.sources("package", [files])
This approach will led you to a great success, imo.
--
>here are more things in heaven and earth,
horatio, than are dreamt of in your philosophy.
> On 6/4/05, Christian Neukirchen <chneuk...@gmail.com> wrote:
>> But now, I'll ask you: Are you satisfied with the way installing Ruby
>> extensions and libraries works? Do you think there is a place for
>> Package? Do you have further improvements or can provide alternative
>> ideas?
>
> I think that there is room for improvement, definitely. However, if
> you're going to do extension support with this, it absolutely *must*
> work perfectly well on Windows. It has to work better than setup.rb,
> and setup.rb works mostly well for that. Gems, much less so.
I don't think making the installer work on windows "perfectly" would
be a hard job. For building extensions, things look a bit
different... I'd certainly need a win32 expert for that.
How well does extconf.rb work on win32?
> -austin
> * Christian Neukirchen <chneuk...@gmail.com> [2005-06-05 07:59:15 +0900]:
>
>> Package doesn't try to conquer the world, however, it aims to be just
>> a tool that would be useful if it was standard and everyone could
>> build on due to it's policy-neutrality
>
>> What advantages will Package have over setup.rb and mkmf.rb, as
>> they are now?
>
> I find mkmf.rb very useful to create a Makefile when I am building a C wrapper
> to some library. Will Package do the same thing, or will mkmf still
> be needed?
Package will try to provide a more clean (no icky globals, for
example) API for the things mkmf.rb does. I think I'll start with a
recent mkmf.rb and refactor it heavily.
> Jim Freeze
SCons is a full blown build tool. AFAIK they use the distutils
(setup.py) package for SCons installation.
Stefan
Package will not handle dependencies on it's own (You are strongly
recommended to check for libraries in the build part, though).
Package will not provide general "build management", but does generate
instructions how to build extensions. That is, if you before used an
extconf.rb, you'll use Package, if you used a custom Rakefile, you'll
continue to do so.
It would be imaginable that Package could generate Rake tasks
dynamically, however, Rake is not (yet?) included in the Ruby standard
library.
> Might be a good idea to have the extension builder as a completely
> separate tool, too.
I'll try to make it easy to use outside of Package.
> Just a simple, easy packager/setup utility, please!
> (I will help, too, if I can!)
Thank you, I'll come back on that.
> E
A clean API sounds good, it would be nice if it could be used as a
library to allow easy integration into other tools, e.g. Rant.
OTOH if Package uses Rant or Rake it is easier to make it portable
and there would be less dependencies on external (non-Ruby) tools.
Stefan
>> I started to wonder if Ruby's install
>> scripts are state of the art, what could be done better and how.
>
> clap, clap.
> this is wonderfull: we need it.
>
>
>> Basically, you create a file, setup.py
>
> I can suggest you to look at scons (http://www.scons.org)
>
>
> A great thing would be to replace:
> - configure make and make install
Please note that Package aims to specifically help Ruby developers
installing *Ruby* libraries and extensions. Package is not trying to
be a general purpose tool for other languages, especially not autoconf
or automake.
> This approach will led you to a great success, imo.
How about you write the autotools in Ruby and have your own great
success? :-)
I agree. However, Package should not have any dependencies that are
not in the Ruby standard libraries.
> Stefan
>> I started to wonder if Ruby's install
>> scripts are state of the art, what could be done better and how.
d> clap, clap.
d> this is wonderfull: we need it.
>> Basically, you create a file, setup.py
d> I can suggest you to look at scons (http://www.scons.org)
d> A great thing would be to replace:
d> - configure make and make install
d> As said on this mailing list to rake author you could:
Sorry but the "autoconf", "automake" and "libtools" are the worst
things i've ever seen in building libraries. I don't think we should
not write even one line to support these 15 year old c library problem
workaround tool.
Use ANSI-C and platform _IF_MY_SYSTEM_ specific precompiler conditions.
If this can't solve your problem change your code or use another
library or restrict the platforms on which your extensions runs.
--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ruby-ide.com
CTO Scriptolutions Ruby, PHP, Python IDE 's
Very nice.
> * more flexible directory layout: especially small projects
> profit from this, as setup.rb's directory layout is quite
> bulky by default and not very customizable
I am still not sure that this is a disadvantage. Probably you are
referring to the lib/<foo-lib>/ thing, to have to create that
subdirectory everytime. While this may seem bulky, it is easy for the
mind. You just know that lib/* is copied to whatever ruby libdir is
configured. The same holds for data/, conf/, etc.
> * easier packaging by third-party packagers due to simple
> but flexible and standardized invocation
This is a big advantage. I've seen RPA make a small step in the same
direction, being able to create the debian/ subdirectory for RPA's
metadata file. While the result almost always has to be tweaked, it's a
big help as starting point.
I think that I can speak for the Debian Ruby Maintainers Team that we
will back this effort.
> What do we need to get a wide adoption of Package?
>
> * inclusion in the standard library so it doesn't need to be
> shipped with every package (as setup.rb unfortunately is).
> * backing from the community to make use of Package.
> * acceptance from packaging projects like RPA, RubyGems and
> distributions like Debian, FreeBSD and PLD.
Yeah, maybe the name is misleading. It does nothing for packaging, it
stays completely out of the packaging scope (although it's a bit
relevant :)). Something like distutils is also not really covering it,
it does not aid in distribution, it is a system for
configuring/building/installing. Maybe Setup/Installer is a better them,
it is associated with the act of copying the files.
> Coding of Package has not started yet (the name is also not set into
> stone yet, so if you have better ideas, please tell me) because it
> would be pointless without a strong feedback from the community. I
> expect to get a first version done rather quickly, possibly borrowing
> code from setup.rb and mkmf.rb, but Package will not depend on these
> both. If anyone is interested in helping development, please mail me;
> helpful hands are always of use. Also, there will be need for testers
> on all and even the most weird platforms.
I'll help you in both areas if and when I can!
> But now, I'll ask you: Are you satisfied with the way installing Ruby
> extensions and libraries works?
Yes. I usually use setup.rb for this, for it's simplicitly. I have
never done C bindings and stuff, so it always has been pure Ruby for me.
There are many methods to install stuff these days, supplied Rakefiles,
Rantfiles, setup.rb, extconf.rb, mkmf.rb etc. I never understood why
some part hasn't been abstracted from and put in a library yet.
> Do you think there is a place for Package?
I strongly do.
> Do you have further improvements or can provide alternative ideas?
Well, I reckon that some things have to be thought out yet, there are
many good things in setup.rb, mkmf.rb etc. and those things can be
combined IMO. We'll see how it goes.
Just one thing. I think it's a good thing if Package should use the
paths specified by Ruby's rbconfig, but that one has to be cleaned up.
I hate to say it, but (at least from the perpective of a 3rd party Ruby
distributor) it has become a mess. So something has to be changed
in that area as well.
Good stuff, great!
Regards,
Paul
--
Student @ Eindhoven | email: pau...@debian.org
University of Technology, The Netherlands | JID: pa...@luon.net
>>> Using the Power of Debian GNU/Linux <<< | GnuPG key ID: 0x50064181
> On Sun, Jun 05, 2005 at 07:59:15AM +0900, Christian Neukirchen wrote:
>> What advantages will Package have over setup.rb and mkmf.rb, as
>> they are now?
>>
>> * simple, clean and consistent working
>> * unified library to handle both extensions and libraries
>> * lightweight approach (if included in the standard library)
>> * easy adaption
>
> Very nice.
>
>> * more flexible directory layout: especially small projects
>> profit from this, as setup.rb's directory layout is quite
>> bulky by default and not very customizable
>
> I am still not sure that this is a disadvantage. Probably you are
> referring to the lib/<foo-lib>/ thing, to have to create that
> subdirectory everytime. While this may seem bulky, it is easy for the
> mind. You just know that lib/* is copied to whatever ruby libdir is
> configured. The same holds for data/, conf/, etc.
True, but it does no harm if you can easily change it.
There may be lots of reasons not to use a directory layout like that.
>> * easier packaging by third-party packagers due to simple
>> but flexible and standardized invocation
>
> This is a big advantage. I've seen RPA make a small step in the same
> direction, being able to create the debian/ subdirectory for RPA's
> metadata file. While the result almost always has to be tweaked, it's a
> big help as starting point.
> I think that I can speak for the Debian Ruby Maintainers Team that we
> will back this effort.
I've been talking to #debian-ruby yesterday already. Cooperation with
packagers is important for success. Packagers of other distributions
and projects are welcome to mail me.
>> What do we need to get a wide adoption of Package?
>>
>> * inclusion in the standard library so it doesn't need to be
>> shipped with every package (as setup.rb unfortunately is).
>> * backing from the community to make use of Package.
>> * acceptance from packaging projects like RPA, RubyGems and
>> distributions like Debian, FreeBSD and PLD.
>
> Yeah, maybe the name is misleading. It does nothing for packaging, it
> stays completely out of the packaging scope (although it's a bit
> relevant :)). Something like distutils is also not really covering it,
> it does not aid in distribution, it is a system for
> configuring/building/installing. Maybe Setup/Installer is a better them,
> it is associated with the act of copying the files.
I was just thinking, it *defines* what is in an Package, doesn't it?
>> Coding of Package has not started yet (the name is also not set into
>> stone yet, so if you have better ideas, please tell me) because it
>> would be pointless without a strong feedback from the community. I
>> expect to get a first version done rather quickly, possibly borrowing
>> code from setup.rb and mkmf.rb, but Package will not depend on these
>> both. If anyone is interested in helping development, please mail me;
>> helpful hands are always of use. Also, there will be need for testers
>> on all and even the most weird platforms.
>
> I'll help you in both areas if and when I can!
Thank you, I'll come back on you.
>> Do you have further improvements or can provide alternative ideas?
>
> Well, I reckon that some things have to be thought out yet, there are
> many good things in setup.rb, mkmf.rb etc. and those things can be
> combined IMO. We'll see how it goes.
>
> Just one thing. I think it's a good thing if Package should use the
> paths specified by Ruby's rbconfig, but that one has to be cleaned up.
> I hate to say it, but (at least from the perpective of a 3rd party Ruby
> distributor) it has become a mess. So something has to be changed
> in that area as well.
Being not a Ruby core developer, I don't have influence into that.
However, I think rbconfig can (and should) be cleaned up a lot.
Maybe you do want to bring up this issue on ruby-core?
For a start, rbconfig is useful enough for getting standard paths,
though. Package will provide support to make use of alternative
paths, just like distutils do it.
> Good stuff, great!
> Regards,
Thanks a lot!
> Paul
Pretty well, actually.
Occasionally some minor glitches, like I was trying to build
rbtree-0.1.3 on win32 the other day, and it was mysteriously
failing during extconf... Turned out one line in rbtree's
extconf.rb was the problem:
$CFLAGS << ' -std=c89 -pedantic -Wall -Wno-long-long'
..Unconditionally adding gnu-specific flags to $CFLAGS
regardless of platform. After commenting out that line,
rbtree built just fine on windows.
Regards,
Bill
> From: "Christian Neukirchen" <chneuk...@gmail.com>
>>
>> I don't think making the installer work on windows "perfectly" would
>> be a hard job. For building extensions, things look a bit
>> different... I'd certainly need a win32 expert for that.
>>
>> How well does extconf.rb work on win32?
>
> Pretty well, actually.
>
> Occasionally some minor glitches, like I was trying to build
> rbtree-0.1.3 on win32 the other day, and it was mysteriously
> failing during extconf... Turned out one line in rbtree's
> extconf.rb was the problem:
>
> $CFLAGS << ' -std=c89 -pedantic -Wall -Wno-long-long'
>
> ...Unconditionally adding gnu-specific flags to $CFLAGS
> regardless of platform. After commenting out that line,
> rbtree built just fine on windows.
That is good to know, I guess you used MSVC then?
> Regards,
>
> Bill
Re: naming. Since 'Package', as mentioned, may be ever so slightly
misleading, perhaps call it 'Kit' or something? If you want to get
fancier, my pick for my own package manager (whenever I got around
to it) was 'Oblong' (extra points for the reference).
>>>But now, I'll ask you: Are you satisfied with the way installing Ruby
>>>extensions and libraries works? Do you think there is a place for
>>>Package? Do you have further improvements or can provide alternative
>>>ideas?
>>
>> No. setup.rb is OK but inflexible. Package management (gems etc.)
>> nor build management (Rant, Rake) should not be in any way incorporated
>> in 'Package', either.
>
>Package will not handle dependencies on it's own (You are strongly
>recommended to check for libraries in the build part, though).
>
>Package will not provide general "build management", but does generate
>instructions how to build extensions. That is, if you before used an
>extconf.rb, you'll use Package, if you used a custom Rakefile, you'll
>continue to do so.
>
>It would be imaginable that Package could generate Rake tasks
>dynamically, however, Rake is not (yet?) included in the Ruby standard
>library.
>
>> Might be a good idea to have the extension builder as a completely
>> separate tool, too.
>
>I'll try to make it easy to use outside of Package.
>
>> Just a simple, easy packager/setup utility, please!
>> (I will help, too, if I can!)
>
>Thank you, I'll come back on that.
Just let me know what you need done.
>> E
definitely fine.
But it would be great to have a builtin way to trick it so that
extensions can be built with a different compiler than the one used for
ruby (tipically mingw extensions for mswin ruby)
Here's the information of the when and where:
When:
June 6, 2005
Where:
Panera @ Eagle Run Shopping Center
13410 West Maple Road
Omaha, NE 68164
------------------------------------------------
Blaine Buxton, Mad Scientist In Training
Emperor of Planet Alto Dorado
http://www.blainebuxton.com
_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today - it's FREE!
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
I think that's an issue pure Ruby code can't change... can it?
> Le 5/6/2005, "Christian Neukirchen" <chneuk...@gmail.com> a écrit:
>>ES <rub...@magical-cat.org> writes:
>>
>>> Le 4/6/2005, "Christian Neukirchen" <chneuk...@gmail.com> a écrit:
>
> Re: naming. Since 'Package', as mentioned, may be ever so slightly
> misleading, perhaps call it 'Kit' or something? If you want to get
> fancier, my pick for my own package manager (whenever I got around
> to it) was 'Oblong' (extra points for the reference).
While I agree that Package is not a perfect name, Kit isn't better IMO.
It should be a word where everyone can imagine what it is... but there
is no concept of a "kit" in Ruby.
Still looking for alternatives...
> E
Seeing as Aoki-san combined his efforts to setup.rb,
'Install' or 'Installer' would be available.
I personally do not think 'Package' is a bad name (because that is
what it is), but it may cause confusion since the existing package
_managers_ are ambiguously named. Perhaps you should just brave
those waters and go with it.
A less-bureaucratic and more-dastardly pet name could of course
come in handy. That, or a cute logo (which is known to drastically
improve the quality of software).
>> E
>--
>>>How well does extconf.rb work on win32?
>>>
>>
>>definitely fine.
>>But it would be great to have a builtin way to trick it so that
>>extensions can be built with a different compiler than the one used
>>for ruby (tipically mingw extensions for mswin ruby)
>
>
> I think that's an issue pure Ruby code can't change... can it?
I don't know.
I am guessing there is some place where extconf.rb/mkmf.rb check values
from rbconfig and generate the makefile stuff based on it,
the fake-mingw32 module seem to basically change this values.
I am thinking of some thing like
if ruby_built_with_mingw
output gcc code
elsif ruby_built_with_msvc
output cl code
end
and thinking of adding one additional check like
if suggested_from_cli_compiler then output code for suggested compiler
fake-ming32 is a C module, so it may confirm your impression.
OTOH, it may be a C module just because extconf/mkmf are already in
place, and faking must happen at a lower level.
Sorry for my inability to give real useful informations :/
Please, do not reply to a random mail to skip the step of having to type
the address. It distorts threads by inserting non-related mail for mail
clients that do support them.
Thanks,
Paul
--
Student @ Eindhoven | email: pa...@luon.net
> Christian Neukirchen ha scritto:
>
>>>>How well does extconf.rb work on win32?
>>>>
>>>
>>>definitely fine.
>>>But it would be great to have a builtin way to trick it so that
>>>extensions can be built with a different compiler than the one used
>>>for ruby (tipically mingw extensions for mswin ruby)
>> I think that's an issue pure Ruby code can't change... can it?
>
> I don't know.
> I am guessing there is some place where extconf.rb/mkmf.rb check
> values from rbconfig and generate the makefile stuff based on it,
> the fake-mingw32 module seem to basically change this values.
>
> I am thinking of some thing like
> if ruby_built_with_mingw
> output gcc code
> elsif ruby_built_with_msvc
> output cl code
> end
But would it be possible to run extensions built with mingw32 on a
MSVC ruby?
> and thinking of adding one additional check like
> if suggested_from_cli_compiler then output code for suggested compiler
>
> fake-ming32 is a C module, so it may confirm your impression.
>
> OTOH, it may be a C module just because extconf/mkmf are already in
> place, and faking must happen at a lower level.
>
> Sorry for my inability to give real useful informations :/
--
> But would it be possible to run extensions built with mingw32 on a
> MSVC ruby?
I think nobu said in a recent thread that if ruby is built with msvc7 it
does not work with mingw extensions (*yet*).
Anyway in the past they almost always worked flawlessly.