[ANN] [RFC] Proposing a Ruby Packaging Standard
Christian Neukirchen
earlier this month I wrote up a draft for a specification on how to
structure Ruby projects. It started at http://gist.github.com/361451
and there was lots of discussion there, which I now want to move to
ruby-talk to gain a larger audience.
The specification now resides at
http://chneukirchen.github.com/rps/
and is generated from http://github.com/chneukirchen/rps.
Please keep discussion on RPS to this thread, so people which usually
don't follow ruby-talk can discuss too.
Thank you,
--
Christian Neukirchen <chneuk...@gmail.com> http://chneukirchen.org
Intransition
wrote:
> Hello,
>
> earlier this month I wrote up a draft for a specification on how to
> and there was lots of discussion there, which I now want to move to
> ruby-talk to gain a larger audience.
>
> The specification now resides at
>
> http://chneukirchen.github.com/rps/
>
>
> Please keep discussion on RPS to this thread, so people which usually
> don't follow ruby-talk can discuss too.
I don't understand. Do you want us to discuss it here or not?
Intransition
> I don't understand. Do you want us to discuss it here or not?
Ok. Just read last post from github and it indicates to discuss here.
Also, I noticed the the Google archive knocked off the [RFC] portion
of the subject. Hope that's not problem.
Intransition
I carried on development of that project here: http://proutils.github.com/setup.
It can now run as ordinary bin, or, of course, it can be included as a
project script when specifically necessary.
Point responses to zimbatm's post
> * data/ is copied in #{PREFIX_DIR}/share (like /usr/share). Why not name it "share" then ?
I would agree, but rbconfig.rb refers to it as 'datadir'.
> * conf/ is copied in /etc. Why not name it "etc" then ?
The latest versions of setup.rb have renamed that to etc/.
> * setup.rb multi-project layout is probably a bad idea. Package management should handle this
Latest versions of setup.rb no longer support multi-project layout.
The most problematic issue I had with working on setup.rb was with
documentation. FHS standards would indicate that the packages
documentation be copied to /usr/share/doc/{name}/ (using standard
system prefix), However, unless Ruby projects use doc/{name}/ (just
like lib/{name}/) then it's not very feasible b/c there is no standard
way to get the name of the project. Right now it looks in
either .setup/name or .meta/name (or without dot) as a stop-gap
measure for lack of any other option. If anything comes out this
process, beyond just writing down current common practices, maybe it
would be this doc/{name}/ convention.
On the other hand, maybe it's time to give setup.rb and the whole FHS
style of program installation the kiss of death.
Intransition
1) Change "Executables SHOULD NOT require 'rubygems' or modify the
$LOAD_PATH" to a "MUST NOT". There's no reason to do this in an
executable.
2) Change "Libraries SHOULD NOT require code of the project that are
outside of lib/" and "Executables SHOULD NOT require code of the
project that are outside of lib/" to a "MUST NOT". Otherwise other
package mangers like setup.rb and rip will not work.
3) I have basically given up on using data/ directory. I now put all
such files in lib/. It is much easier to use relative lookup then to
use rbconfig.rb and Config::CONFIG['datadir']. While some think this
is a "no no", I don't see any worthy advantage of using data/. I know
the noble idea is that data/ is then accessible by other libraries,
but honestly it almost never happens. In a case where it is likely,
then ok use data/, but I don't think it should be mandatory. So I
suggest removing "Ruby library files MUST end with .rb." altogether.
4) Test locations might be a little too framework-oriented for any
kind of RPS. Test::Unit and MiniTest use test/, RSpec used spec/,
Cucumber uses features/. I ran into this question when developing QED
and in the end I just settled for qed/.
Joel VanderWerf
> 1) Change "Executables SHOULD NOT require 'rubygems' or modify the
> $LOAD_PATH" to a "MUST NOT". There's no reason to do this in an
> executable.
Except for the _gem_ executable itself, or any other executable that
needs to operate on gems. SHOULD NOT was the right call.
Suraj Kurapati
I've only ever used RubyGems for packaging my projects, so I'm kind of
scared of setup.rb's style of separating my project's directories (i.e.
lib/ is no longer next to bin/ and so on). In particular, I am confused
about the following:
I want to run the man executable (pointing it to my project's man/
directory) from my project's executable in bin/. With RubyGems, I can
rely on bin/ being next to man/. But with setup.rb I don't know how to
find my project's man/ directory from my project's executable in bin/!
Any suggestions?
Also, if I provide a man/ directory in my gem/tarball package like this:
man/
└── man1
├── ember.1
└── ember.1.html
1 directory, 2 files
Will it be copied into /usr/share/man/ by RubyGems/setup.rb/Rip and
friends?
Thanks for your consideration.
--
Posted via http://www.ruby-forum.com/.
James Britt
> Final comments for now, I promise ;-)
>
> 1) Change "Executables SHOULD NOT require 'rubygems' or modify the
> $LOAD_PATH" to a "MUST NOT". There's no reason to do this in an
> executable.
I have a JRuby desktop app that clears out and then sets up $: The
reason is that I need to know that it is not loading anything other than
what is bundled up with the application.
If there is a way to restrict the load path without altering $: I'd be
interested to know what it is.
--
James Britt
www.jamesbritt.com - Playing with Better Toys
www.ruby-doc.org - Ruby Help & Documentation
www.rubystuff.com - The Ruby Store for Ruby Stuff
www.neurogami.com - Smart application development
Luis Lavena
wrote:
> Hello,
>
> earlier this month I wrote up a draft for a specification on how to
> and there was lots of discussion there, which I now want to move to
> ruby-talk to gain a larger audience.
>
> The specification now resides at
>
> http://chneukirchen.github.com/rps/
>
>
> Please keep discussion on RPS to this thread, so people which usually
> don't follow ruby-talk can discuss too.
>
My comments:
"Extensions" Section:
1) "Extensions SHOULD reside in ext/"
There are project that include more than one extension, I've
encouraged project follow the setup.rb folder structure, exemplified
here:
http://github.com/luislavena/rake-compiler
(under structure)
2) "Extensions SHOULD be buildable with ruby extconf.rb; make."
That is not true, there are extension that even using extconf they
generate Rakefiles, because they depend on mkrf and not mkmf
That can't be enforced on these developers, if done a lot of project
could break.
3) "Extensions SHOULD be installed into an architecture-specific
directory."
RubyGems doesn't do that, it just relocate binaries from ext directory
to lib, adding the platform directory will make impossible for pre-
built binary gems that bundle 1.8 and 1.9 extension.
It also means that RubyGems will need to add RUBY_PLATFORM (or
RbConfig::CONFIG['host_os']) of each gem into $LOAD_PATH.
This has been requested in the past in RubyGems:
http://rubyforge.org/tracker/index.php?func=detail&aid=14943&group_id=126&atid=575
Installations not using RubyGems will need to adapt their libraries to
try loading from RUBY_PLATFORM directory and under failure try from
other options?
If no RubyGems and no developers implementing those tricks, means Ruby
will need to be modified to handle that.
Setup.rb already handles that putting binaries *outside* the package
and inside site_ruby for each RUBY_PLATFORM
3) Files ending with .exe
AFAIK: "Exe"cutables, not extensions or libraries
My two cents
--
Luis Lavena
Intransition
Clearly the gem command itself is necessary exception --it *is*
rubygems. I would suggest any other library that "operates on gems"
still use a lib/ file and require that in their executable. But the
rule applies even in lib/, so here again it is an exceptional case
when one is specifically extending/augmenting rubygems. The rules
intent is to address all other cases. So perhaps the best phrasing is
still "MUST" but with a qualification concerning libraries designed
for extending/augmenting rubygems.
Intransition
On Apr 20, 1:12 pm, Suraj Kurapati <sun...@gmail.com> wrote:
> Hi,
>
> I've only ever used RubyGems for packaging my projects, so I'm kind of
> scared of setup.rb's style of separating my project's directories (i.e.
> lib/ is no longer next to bin/ and so on). In particular, I am confused
> about the following:
>
> I want to run the man executable (pointing it to my project's man/
> directory) from my project's executable in bin/. With RubyGems, I can
> rely on bin/ being next to man/. But with setup.rb I don't know how to
> find my project's man/ directory from my project's executable in bin/!
>
> Any suggestions?
The standard approach would be something like:
require 'rbconfig'
dir = Config::CONFIG['datadir']
file = File.join(dir, 'man/man1/ember.1')
B/c of RubyGems there is an extension method that redirects to the the
appropriate place.
Config.datadir("ember")
But I think it might be a problem to tell which is which for locating
the man/ directory.
> Also, if I provide a man/ directory in my gem/tarball package like this:
>
> man/
> └── man1
> ├── ember.1
> └── ember.1.html
Can man handle .html? I glanced at the entries in my system and I
noticed they are all gzipped (ending in .gz).
> Will it be copied into /usr/share/man/ by RubyGems/setup.rb/Rip and
> friends?
Setup.rb does. RubyGems does not. Not sure about Rip.
ghorner
> directory) from my project's executable in bin/. With RubyGems, I can
> rely on bin/ being next to man/. But with setup.rb I don't know how to
> find my project's man/ directory from my project's executable in bin/!
>
> Any suggestions?
Just use gem-man, http://github.com/defunkt/gem-man:
$ gem install gem-man
$ gem man ember
> Will it be copied into /usr/share/man/ by RubyGems/setup.rb/Rip and
> friends?
The next version of rip should handle installing man pages.
Suraj Kurapati
> On Apr 20, 1:12 pm, Suraj Kurapati <sun...@gmail.com> wrote:
>> find my project's man/ directory from my project's executable in bin/!
>
>
> require 'rbconfig'
> dir = Config::CONFIG['datadir']
> file = File.join(dir, 'man/man1/ember.1')
Thanks. Based on this example, I found that there is a
Config::CONFIG['mandir'] which tells us exactly where the
system's man/ directory is located.
> B/c of RubyGems there is an extension method that redirects to the the
> appropriate place.
>
> Config.datadir("ember")
RubyGems does not seem to provide that functionality on my system:
sun@yantram ~> irb
## ruby 1.9.1p378 (2010-01-10 revision 26273) [i686-linux]
>> require 'rbconfig'
=> true
>> Config.datadir("ember")
NoMethodError: undefined method `datadir' for RbConfig:Module
from (irb):2
from /usr/bin/irb:12:in `<main>'
>> require 'rubygems'
=> false
>> Config.datadir("ember")
NoMethodError: undefined method `datadir' for RbConfig:Module
from (irb):4
from /usr/bin/irb:12:in `<main>'
>> Gem::VERSION
=> "1.3.6"
> But I think it might be a problem to tell which is which for locating
> the man/ directory.
I guess RubyGems provides a Config.mandir(the_gem_name) method to do
this?
>> Also, if I provide a man/ directory in my gem/tarball package like this:
>>
>> man/
>> └── man1
>> ├── ember.1
>> └── ember.1.html
>
> Can man handle .html?
Nope. That's just there as a backup, in case a user is on Windoze
or their system somehow lacks the `man` command. `man` does warn me
about it though when searching for manpages in my man/ directory:
man: warning: man/man1/ember.1.html: ignoring bogus filename
But I think it's worthy trade-off to have a HTML version available.
> I glanced at the entries in my system and I
> noticed they are all gzipped (ending in .gz).
Thanks for the tip. I've gzipped my Roff manpage now.
>> Will it be copied into /usr/share/man/ by RubyGems/setup.rb/Rip and
>> friends?
>
> Setup.rb does. RubyGems does not. Not sure about Rip.
Will RubyGems ever follow the FHS style of installation like setup.rb?
Will setup.rb ever follow the keep-everything-together RubyGems style?
What installation style will this Ruby Packaging Standard follow?
So many questions... Life was easier when I only used RubyGems. :-/
Suraj Kurapati
>> I want to run the man executable (pointing it to my project's man/
>
Excellent tip! Thank you.
>> Will it be copied into /usr/share/man/ by RubyGems/setup.rb/Rip
>
> The next version of rip should handle installing man pages.
This is great news. I hope it also gzips man pages (if they're not
already) during package installation to save space.
Lucas Nussbaum
I think that what is important is that we have a clear set of guidelines
for Ruby library developers to follow, that allow to do both
rubygems-style stuff and setup.rb-style stuff. The goal of this
"standard" should not be to push one or the other solution.
--
| Lucas Nussbaum
| lu...@lucas-nussbaum.net http://www.lucas-nussbaum.net/ |
| jabber: lu...@nussbaum.fr GPG: 1024D/023B3F4F |
Suraj Kurapati
> On 21/04/10 at 14:01 +0900, Suraj Kurapati wrote:
>> Will RubyGems ever follow the FHS style of installation like setup.rb?
>> Will setup.rb ever follow the keep-everything-together RubyGems style?
>> What installation style will this Ruby Packaging Standard follow?
>
> guidelines for Ruby library developers to follow, that allow to do
> both rubygems-style stuff and setup.rb-style stuff.
It's burdensome on Ruby library developers to support both ways.
Perhaps this Ruby Packaging Standard could provide a tiny Ruby
library to ease that burden, by making the FHS/setup.rb style
installation appear like RubyGems style installation (i.e. how all
files are organized inside a release package and when checked out
from a source code repository)?
> The goal of this "standard" should not be to push one or the other
> solution.
Good point. Thanks for clarifying this.
Lucas Nussbaum
> Lucas Nussbaum wrote:
> > On 21/04/10 at 14:01 +0900, Suraj Kurapati wrote:
> >> Will RubyGems ever follow the FHS style of installation like setup.rb?
> >> Will setup.rb ever follow the keep-everything-together RubyGems style?
> >> What installation style will this Ruby Packaging Standard follow?
> >
> > I think that what is important is that we have a clear set of
> > guidelines for Ruby library developers to follow, that allow to do
> > both rubygems-style stuff and setup.rb-style stuff.
>
> It's burdensome on Ruby library developers to support both ways.
Not really. Both solutions are largely compatible. You just need to
organize the files correctly, and it just works.
> Perhaps this Ruby Packaging Standard could provide a tiny Ruby
> library to ease that burden, by making the FHS/setup.rb style
> installation appear like RubyGems style installation (i.e. how all
> files are organized inside a release package and when checked out
> from a source code repository)?
I'm not sure I understand your point. The point of setup.rb is to
respect FHS, which contradicts installing everything in the same place.
But the starting point of a setup.rb installation is the same as a
well-organized gem: if you put executables in bin/, libraries in lib/,
etc, it just works.
Suraj Kurapati
> On 21/04/10 at 14:55 +0900, Suraj Kurapati wrote:
>> Perhaps this Ruby Packaging Standard could provide a tiny Ruby
>> like RubyGems style installation
>
> respect FHS, which contradicts installing everything in the same place.
My concern is about the Ruby code in bin/ and lib/ being able to
find the rest of the files that originally came in the package. If
this RPS would define that related-file-finding code, then I don't
need to reinvent it (or copy/paste) for all of my Ruby libraries.
Dmitry Borodaenko
> My concern is about the Ruby code in bin/ and lib/ being able to
> find the rest of the files that originally came in the package. If
> this RPS would define that related-file-finding code, then I don't
> need to reinvent it (or copy/paste) for all of my Ruby libraries.
Isn't that precisely what Config::CONFIG['datadir'] is for?
--
Dmitry Borodaenko
Christian Neukirchen
> 3) Files ending with .exe
>
> AFAIK: "Exe"cutables, not extensions or libraries
Afaict do libraries have that extension on VMS.
Suraj Kurapati
> On Wed, Apr 21, 2010 at 9:36 AM, Suraj Kurapati <sun...@gmail.com>
> wrote:
>> My concern is about the Ruby code in bin/ and lib/ being able to
>> find the rest of the files that originally came in the package.
>
Yes, but that value does not reflect RubyGems style on my system:
>> Config::CONFIG['datadir']
=> "/usr/share"
>> `gem env`.lines.grep(/install/i)
=> [" - INSTALLATION DIRECTORY: /usr/lib/ruby/gems/1.9.1\n"]
It needs to reflect both styles to work correctly under either one.
I am using RubyGems 1.3.6 with Ruby 1.9.1p378 on i686-linux.
Damian Janowski
<chneuk...@gmail.com> wrote:
> Hello,
>
> earlier this month I wrote up a draft for a specification on how to
> structure Ruby projects. It started at http://gist.github.com/361451
> and there was lots of discussion there, which I now want to move to
> ruby-talk to gain a larger audience.
Under Tests, should there be a standard for running the entire test
suite? Say, `rake`?
Luis Lavena
> [...]
>
> Under Tests, should there be a standard for running the entire test
> suite? Say, `rake`?
That would trigger other dependencies developers might have in their
rake tasks (like coverage, documentation, mutation testing, etc.)
I simple "ruby -Ilib test/test*.rb" should suffice, right?
--
Luis Lavena
Intransition
Depends on the testing framework. Some do not support autorun. Some
tests aren't located in test/. For most 'rake test' is the common
practice, but as someone who does not use rake I have at times
wondered how best to approach this myself. It's not a huge issue
really, since only developers tend to actually run the tests and
whatever system used it's not too hard for developers to figure out.
On occasion though I have provided a ruby executable called 'script/
test' to handle it.
James Britt
I have some Rakefiles where the default task is to list the tasks.
In other cases a I set it to run the test tool of choice (sometimes
bacon, something rspec, it varies).
If I were to want a default default (so to speak) I'd prefer the task
listing rather than kicking off something. It's easy enough to change
to suit one's personal preferences.
Or leave the Rakefile with no default task.
James Edward Gray II
> If I were to want a default default (so to speak) I'd prefer the task listing rather than kicking off something.
I don't think I'm for recommending a default, but listing the tasks definitely wouldn't be my first choice. That's a function of Rake itself (rake -T), so I would hate to waste a task on it. That's just my opinion.
James Edward Gray II
John Barnette
The purpose of Christian's is to provide a minimal, consistent, and *tool agnostic* set of guidelines for organizing your Ruby library. If the changes or additions you're suggesting assume the presence of any tool *other than Ruby itself*, you're almost certainly bikeshedding.
Please give your suggestions some heavy thought before posting.
~ j.
James Britt
Yeah, it's not like it isn't already there, it's just that if I type
'rake', I'd rather something useful but benign happen instead of a
message about there being no 'default' defined.
But either way I don't want to inadvertently kick off some potentially
lengthy process.
Joshua Peek
> 2) "Extensions SHOULD be buildable with ruby extconf.rb; make."
>
> That is not true, there are extension that even using extconf they
> generate Rakefiles, because they depend on mkrf and not mkmf
>
> That can't be enforced on these developers, if done a lot of project
> could break.
I'd like to see mkrf formally spec'd here. I'm not to familiar with
mkrf or know to many gems using it. Do you still use extconf.rb but
require mkrf instead?
Also, whats your opinion on make + copy vs "make install" with
sitearchdir. Both seem to be compatible with all Makefiles generated
with mkmf. The problem with copying is that you have to manually
extract the target_prefix from the Makefile.
Intransition
On Apr 21, 2:06 am, Lucas Nussbaum <lu...@lucas-nussbaum.net> wrote:
> I'm not sure I understand your point. The point of setup.rb is to
> respect FHS, which contradicts installing everything in the same place.
> But the starting point of a setup.rb installation is the same as a
> well-organized gem: if you put executables in bin/, libraries in lib/,
> etc, it just works.
All true. But it does effect how one is able to access files from
other files within a package, e.g. relative requires. If it were not
for the limitations that the FHS design brings to bare we would have
many fewer rules and worries.
My personal opinion falls in line with Hisham H. Muhammad's in <a
href="http://www.gobolinux.org/?page=doc/articles/clueless">I am not
clueless</a>. I feel that the FHS has become largely arcane. If I had
my druthers I'd much rather see the Ruby community make a concerted
motion toward throwing setup.rb/FHS support right out on it's arse.
And that's despite the fact that I personally spent considerable time
modernizing the setup.rb code base. I believe the FHS is largely
holding back progress in this area. Both Windows and MacOS X use
directory package models. Linux lags behind primarily b/c it is a open/
democratic system tied to entrenched customs. As such, it can not be
changed from the top down. The change must come from the bottom up --
which is much harder to do. I think scripting languages and their
modern packaging systems, like RubyGems, are (no doubt inadvertently)
leading the way in this "quiet" revolution. Hisham si one of few
people that I think sees the writing on the wall is preparing for it
(see http://mwh.geek.nz/2009/07/23/an-overview-of-systemaliens/)
As far as shared data and configuration goes, btw, when it is needed,
the XDG directory standard offers a much better approach.
Suraj Kurapati
> I feel that the FHS has become largely arcane. If I had
> my druthers I'd much rather see the Ruby community make a concerted
> motion toward throwing setup.rb/FHS support right out on it's arse.
Hurray! Well said. +1 :-)
> As far as shared data and configuration goes, btw, when it is needed,
> the XDG directory standard offers a much better approach.
Here's a link about XDG-DS in Ruby context for lazy people like myself:
Cheers.
Intransition
On Apr 22, 2:14 pm, Suraj Kurapati <sun...@gmail.com> wrote:
> Thomas Sawyer wrote:
> > I feel that the FHS has become largely arcane. If I had
> > my druthers I'd much rather see the Ruby community make a concerted
> > motion toward throwing setup.rb/FHS support right out on it's arse.
>
> Hurray! Well said. +1 :-)
>
> > As far as shared data and configuration goes, btw, when it is needed,
> > theXDGdirectory standard offers a much better approach.
>
> Here's a link aboutXDG-DS in Ruby context for lazy people like myself:
>
> http://xdg.rubyforge.org/
Project has since moved to GitHub:
http://github.com/rubyworks/xdg
http://rubyworks.github.com/xdg
Suraj Kurapati
> Suraj Kurapati wrote:
>> http://xdg.rubyforge.org/
>
> Project has since moved to GitHub:
>
> http://github.com/rubyworks/xdg
> http://rubyworks.github.com/xdg
Thanks for the update. For posterity, I suggest
adding a .htaccess to the old RubyForge site thus:
RedirectPermanent / http://rubyworks.github.com/xdg
Evgeniy Dolzhenko
during say a development of a gem?
On Apr 20, 11:27 am, Christian Neukirchen <chneukirc...@gmail.com>
wrote:
> Hello,
>
> earlier this month I wrote up a draft for a specification on how to
> structure Ruby projects. It started athttp://gist.github.com/361451
> and there was lots of discussion there, which I now want to move to
> ruby-talk to gain a larger audience.
>
> The specification now resides at
>
> http://chneukirchen.github.com/rps/
>
> and is generated fromhttp://github.com/chneukirchen/rps.
>
> Please keep discussion on RPS to this thread, so people which usually
> don't follow ruby-talk can discuss too.
>
> Thank you,
> --
> Christian Neukirchen <chneukirc...@gmail.com> http://chneukirchen.org