Test-time dependencies.
Dmitri Tikhonov
Dear fellow Perl module authors:
I have a test suite for my distribution (RT-Client-REST) that requires
some modules that the module itself does not require (Test::Exception,
for example). Since it is not listed as a dependency, some people's
tests fail[1,2].
Question: is it a good idea to put Test::Exception in as a dependency?
My problem with this is that the module itself, once installed, does
not use Test::Exception, so it is not really a dependency. On the other
hand, I want all tests to run (most of them test exceptions), so I do
not want to 'skip' any tests.
Thanks for your input,
- Dmitri.
1. http://www.nntp.perl.org/group/perl.cpan.testers/340611
2. http://www.nntp.perl.org/group/perl.cpan.testers/340620
David Golden
> I have a test suite for my distribution (RT-Client-REST) that requires
> some modules that the module itself does not require (Test::Exception,
> for example). Since it is not listed as a dependency, some people's
> tests fail[1,2].
>
> Question: is it a good idea to put Test::Exception in as a dependency?
> My problem with this is that the module itself, once installed, does
> not use Test::Exception, so it is not really a dependency. On the other
> hand, I want all tests to run (most of them test exceptions), so I do
> not want to 'skip' any tests.
Is it a good idea? "It depends."
There's an argument to be made for not forcing people to download extra
modules. But most of the "build_requires" type approaches don't seem to
work well in practice. Module::Install can bundle test dependencies in
an inc/ directory, but then you're on the Module::Install treadmill and
people are potentially downloading modules multiple times instead of
just installing them once.
Personally, I don't really worry too much about including test
dependencies as long as the modules are (a) fairly portable and (b)
likely to be used by lots of other common modules. I find that
preferable to skipping lots of tests or other things. All the
dependencies are listed in the META.yml, so if people want to skip my
module for having too many dependencies, well, fine.
That said, I will try to minimize test modules a bit if it doesn't
sacrifice the ease and convenience of tests. For example, while I
wouldn't give up IPC::Run3 for testing scripts, I can (and often do)
without Test::Exception.
While the syntax of Test::Exception is nice, I don't find it too
terrible to just write:
eval { lots_of_code() };
is( $@, '', 'Lives' );
Or:
eval { code_dies() };
like( $@, qr/error message/, 'Got an error' );
I originally did this because Test::Exception relies on Sub::Uplevel,
which had some problems with other code that uses Sub::Uplevel, but I've
taken over and patched Sub::Uplevel, so it should now be fine. So,
personally, if you like Test::Exception, I'd go ahead and use it.
Regards,
David Golden
Ken Williams
On Aug 3, 2006, at 9:01 PM, David Golden wrote:
> Dmitri Tikhonov wrote:
>> I have a test suite for my distribution (RT-Client-REST) that
>> requires
>> some modules that the module itself does not require
>> (Test::Exception,
>> for example). Since it is not listed as a dependency, some people's
>> tests fail[1,2].
>> Question: is it a good idea to put Test::Exception in as a
>> dependency?
>> My problem with this is that the module itself, once installed, does
>> not use Test::Exception, so it is not really a dependency. On the
>> other
>> hand, I want all tests to run (most of them test exceptions), so I do
>> not want to 'skip' any tests.
>
> Is it a good idea? "It depends."
>
> There's an argument to be made for not forcing people to download
> extra modules. But most of the "build_requires" type approaches
> don't seem to work well in practice.
My intention when I added 'build_requires' to Module::Build was not
to let people avoid downloading & building those dependencies, but
rather to indicate that there was no runtime dependency, so if people
are tracking their modules using RPMs or Debs or whatever, they could
know what really depended on what.
-Ken
Jonathan Rockway
"build_requires" that should do what you want. People installing from
source via CPAN will have to download the build_requires, because
they're building the packages, but users of pre-compiled (and tested)
binary distributions won't be asked to install the modules that your
test depends on.
If you're not using Module::Install, though, I would include
Test::Exception as a regular dependency anyway. You'll get better
feedback from the CPAN testers that way, and that's probably more
important than worrying about whether or not someone needs to install
T::E. It's a pretty standard module; they'll need it sooner or later :)
Regards,
Jonathan Rockway
Johan Vromans
> My intention when I added 'build_requires' to Module::Build was not
> to let people avoid downloading & building those dependencies, but
> rather to indicate that there was no runtime dependency, ...
For this to function properly, the 'build_requires' modules (and their
respecitve dependencies!) should all be downloaded and installed in a
temporary location (e.g., blib) and not be installed permanently.
-- Johan
Adrian Howard
I don't think that's the necessary conclusion. Personally I prefer
build requirements to be installed permanently - so I don't have to
build them repeatedly when multiple modules require the same build
requirements.
What build_requires does is enable different folk to make different
decisions depending on what their needs are.
Adrian
Johan Vromans
> On 4 Aug 2006, at 14:04, Johan Vromans wrote:
>> For this to function properly, the 'build_requires' modules (and their
>> respecitve dependencies!) should all be downloaded and installed in a
>> temporary location (e.g., blib) and not be installed permanently.
>
> I don't think that's the necessary conclusion. Personally I prefer
> build requirements to be installed permanently - so I don't have to
> build them repeatedly when multiple modules require the same build
> requirements.
Apparently we differ in opinion.
As already mentioned by other people in other threads, if you prepare
install kits for many systems, you would not want to install all the
build-time requirements, only the run-time requirements. Every single
module that gets installed on a production system increases the
maintenance load.
-- Johan
A. Pagaltzis
> Adrian Howard <adr...@quietstars.com> writes:
> > On 4 Aug 2006, at 14:04, Johan Vromans wrote:
> >> For this to function properly, the 'build_requires' modules
> >> (and their respecitve dependencies!) should all be
> >> downloaded and installed in a temporary location (e.g.,
> >> blib) and not be installed permanently.
> >
> > I don't think that's the necessary conclusion. Personally I
> > prefer build requirements to be installed permanently - so I
> > don't have to build them repeatedly when multiple modules
> > require the same build requirements.
>
> Apparently we differ in opinion.
How?
If the packager installs the build dependencies in order to build
a package, what’s the difference? The package won’t contain
build-time dependencies any more than it will contain runtime
dependencies, and since build-time dependencies are listed
separately from runtime dependencies it’s easy to create a
package that lists only the latter as its dependencies.
I don’t see why any of this requires installing to `blib`.
Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>
Johan Vromans
> If the packager installs the build dependencies in order to build
> a package, what’s the difference? The package won’t contain
> build-time dependencies any more than it will contain runtime
> dependencies, and since build-time dependencies are listed
> separately from runtime dependencies it’s easy to create a
> package that lists only the latter as its dependencies.
>
> I don’t see why any of this requires installing to `blib`.
Because the next module that you build will not let you know that it
requires modules that you have already installed on your build system
as part of the build-time dependencies of (yet another) module. So it
increases the probability that modules will be forgotten to be
included in the install kit. Oh yes, there's always a test install
that will (hopefully) complain.
I didn't say it's wrong, I just said it increases the maintenance
load.
-- Johan
A. Pagaltzis
> Because the next module that you build will not let you know
> that it requires modules that you have already installed on
> your build system as part of the build-time dependencies of
> (yet another) module.
That seems wrong. Every module you install contains lists of
dependencies, one list for runtime, one for build-time.
The CPAN shell won’t ask questions about missing dependencies,
that is true. But why does that matter? You have the dependency
lists anyway.
And why would build-time dependencies be a problem by that token,
but not runtime dependencies? If you rely on the shell, wouldn’t
you have to install each module in a pristine Perl install so the
shell will notify you of its dependencies?
James E Keenan
> Dear fellow Perl module authors:
>
> I have a test suite for my distribution (RT-Client-REST) that requires
> some modules that the module itself does not require (Test::Exception,
> for example). Since it is not listed as a dependency, some people's
> tests fail[1,2].
>
> Question: is it a good idea to put Test::Exception in as a dependency?
> My problem with this is that the module itself, once installed, does
> not use Test::Exception, so it is not really a dependency. On the other
> hand, I want all tests to run (most of them test exceptions), so I do
> not want to 'skip' any tests.
>
I've taken a number of approaches to this question:
1. Include distributions in the PREREQ_PM key in Makefile.PL, which
forces installation just as it would for modules needed for the .pm files.
2. Include the needed modules (or just enough of them as are needed for
the testing to work) under a t/lib/ or t/testlib/ directory.
3. Put the individual tests which need the modules inside SKIP blocks.
Other people -- who are more fluent in writing Makefiles than I -- have
taken this approach:
4. Do #3 above, but write Makefile.PL such that it prompts the user to
install the needed modules.
#3 is my current practice. TMTOWTDI.
Jim Keenan
Johan Vromans
> That seems wrong. Every module you install contains lists of
> dependencies, one list for runtime, one for build-time.
>
> The CPAN shell won’t ask questions about missing dependencies,
> that is true. But why does that matter? You have the dependency
> lists anyway.
Let's try to work out an example. You have a build system, and a
series of production systems. The production systems need module
Foo::Simple.
Module Foo::Simple requires Foo::Heavy at run-time, and
Test::Ridiculous at build time.
A mere build (not install, just build) of Foo::Simple causes
Foo::Heavy and Test::Ridiculous to be installed on the build system[1].
After install of Foo::Simple, three new modules got added to the build
system.
If Test::Ridiculous were not installed, I could chase down the perl
installation to find out what files were added[2], and distribute
these to the production systems. Now I have to manually weed out the
files that are not needed for production.
As I said, it increases the maintenance load.
-- Johan
[1] So if you decide not to use Foo::Heavy after all, you can't just
throw away the build results since the build system has already
been permanently modified.
[2] Module::Build at least provides a target fakeinstall that will
provide this information. Unfortunately, most modules are still
MakeMaker based.
Ken Williams
That's really up to the user to decide. They may wish to install
them permanently just so they save themselves the bother of building
& installing them again next time.
-Ken
Ken Williams
On Aug 4, 2006, at 10:21 AM, Johan Vromans wrote:
> As already mentioned by other people in other threads, if you prepare
> install kits for many systems, you would not want to install all the
> build-time requirements, only the run-time requirements.
If the "install kit" is essentially a binary distribution that won't
build & test, just essentially copy files, then yeah. Don't include
build-time dependencies in it. That would indeed be a waste.
-Ken
Ken Williams
On Aug 5, 2006, at 4:27 AM, Johan Vromans wrote:
> If Test::Ridiculous were not installed, I could chase down the perl
> installation to find out what files were added[2], and distribute
> these to the production systems. Now I have to manually weed out the
> files that are not needed for production.
>
> [...]
>
> [2] Module::Build at least provides a target fakeinstall that will
> provide this information. Unfortunately, most modules are still
> MakeMaker based.
A trivial patch to MakeMaker would let you do this with MakeMaker
too, since it's just a pass-through to ExtUtils::Install that's doing
the real work.
Essentially the Makefile would need this change:
--- Makefile~ 2006-08-03 22:10:19.000000000 -0500
+++ Makefile 2006-08-05 12:45:26.000000000 -0500
@@ -215,7 +215,8 @@
ECHO_N = echo -n
UNINST = 0
VERBINST = 0
-MOD_INSTALL = $(PERLRUN) -MExtUtils::Install -e 'install({@ARGV},
'\''$(VERBINST)'\'', 0, '\''$(UNINST)'\'');'
+NONONO = 0
+MOD_INSTALL = $(PERLRUN) -MExtUtils::Install -e 'install({@ARGV},
'\''$(VERBINST)'\'', $(NONONO), '\''$(UNINST)'\'');'
DOC_INSTALL = $(PERLRUN) "-MExtUtils::Command::MM" -e perllocal_install
UNINSTALL = $(PERLRUN) "-MExtUtils::Command::MM" -e uninstall
WARN_IF_OLD_PACKLIST = $(PERLRUN) "-MExtUtils::Command::MM" -e
warn_if_old_packlist
and then you'd run:
% make pure_install NONONO=1
That wouldn't show you the man page installs, but in an environment
like the one you're describing maybe you don't care about man pages
anyway?
-Ken
Nicholas Clark
> "A. Pagaltzis" <paga...@gmx.de> writes:
>
> > That seems wrong. Every module you install contains lists of
> > dependencies, one list for runtime, one for build-time.
> >
> > that is true. But why does that matter? You have the dependency
> > lists anyway.
>
> Let's try to work out an example. You have a build system, and a
> series of production systems. The production systems need module
> Foo::Simple.
>
> Module Foo::Simple requires Foo::Heavy at run-time, and
> Test::Ridiculous at build time.
>
> A mere build (not install, just build) of Foo::Simple causes
> Foo::Heavy and Test::Ridiculous to be installed on the build system[1].
> After install of Foo::Simple, three new modules got added to the build
> system.
>
> If Test::Ridiculous were not installed, I could chase down the perl
> installation to find out what files were added[2], and distribute
> these to the production systems. Now I have to manually weed out the
> files that are not needed for production.
>
> As I said, it increases the maintenance load.
This is true if you're installing all modules required in one block.
Whereas I believe that the counter arguments come from people who are
considering the case of packaging up each module in turn, and marking
that package as only depending on the packages (modules) that it stated
as run-time requirements, basing that dependency information on the
metadata from the modules (instead of what got installed)
Nicholas Clark
Sam Vilain
This is what Module::Install does when you use 'auto_include'.
What I've done for recent modules is to bundle Test::Depends by making
it the only build_requires line. Then, from the test scripts I can just;
use Test::Depends qw(Some::Module), Other::Module => [ @import_args ];
Of course it means that the test suite doesn't get run fully, but I
think the output makes that obvious enough.
Sam.
Adrian Howard
On 4 Aug 2006, at 18:00, Johan Vromans wrote:
[snip]
> I didn't say it's wrong, I just said it increases the maintenance
> load.
And that was my (obviously badly stated ;-) point.
For people who care about keeping the number of installed modules on
a production platform to a minimum not installing build/test modules
is the "right thing".
For people who care about reducing build time installing build/test
is the "right thing".
Or, for example, I might want to manage a separate tree of non-
production test/build modules so I can get the best of both worlds.
build_requires is useful now, even though the installation tools
don't support all possible use cases yet. M::B is doing the right
thing letting us talk about different kinds of prerequisites. If
enough people itch CPANPLUS and friends will grow to allow people to
use the information in more interesting ways.
TIMTOWTDI and all that :-)
Cheers,
Adrian