Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Ignore lists for a specific PAUSE id + tester

3 views
Skip to first unread message

Karen Etheridge

unread,
Jan 27, 2013, 12:36:04 PM1/27/13
to cpan-teste...@perl.org

I have seen some mention of ignore lists - are these for specific
combinations of authors + testers? How do I update this list?

I have been receiving a series of failure reports, for more than one
distribution, where only the configure_requires dependencies have been
installed but none of the runtime or testing requirements have been. These
reports are nothing but spam, as obviously a distribution is going to fail
if its prerequisites have not been properly installed first.

Past efforts to engage with this author have not proved fruitful.
I cannot easily filter this email out, as the tester's identity is not
included in the email itself, only the web form of the report. Therefore
I feel that adding an entry to an ignore list is my only recourse.

If there is any information available as to how this list works, and how it
may be updated by authors, I would greatly appreciate it.

thanks in advance,

Karen Etheridge, et...@cpan.org

David Cantrell

unread,
Jan 29, 2013, 11:15:46 AM1/29/13
to cpan-teste...@perl.org
On Sun, Jan 27, 2013 at 09:36:04AM -0800, Karen Etheridge wrote:

> I have seen some mention of ignore lists - are these for specific
> combinations of authors + testers? How do I update this list?

You don't. The only person who can update an individual tester's
blacklists is that tester himself.

> I have been receiving a series of failure reports, for more than one
> distribution, where only the configure_requires dependencies have been
> installed but none of the runtime or testing requirements have been. These
> reports are nothing but spam, as obviously a distribution is going to fail
> if its prerequisites have not been properly installed first.

If you can point us at a couple of example failure reports, maybe
someone here can figure out what's going wrong. In my experience it is
almost always the case that things like this are caused by an error in
the distribution under test.

--
David Cantrell | Bourgeois reactionary pig

" In My Egotistical Opinion, most people's ... programs should be
indented six feet downward and covered with dirt. "
--Blair P. Houghton

Karen Etheridge

unread,
Jan 30, 2013, 12:38:39 PM1/30/13
to cpan-teste...@perl.org


> If you can point us at a couple of example failure reports, maybe
> someone here can figure out what's going wrong. In my experience it is
> almost always the case that things like this are caused by an error in
> the distribution under test.

Here's the most recent example of such failures:
http://www.cpantesters.org/cpan/report/cb01432c-6737-11e2-992e-e7c00a4a7996

As you can see, only the configure_requires dependencies have been
installed. *Some* of the runtime requirements are in @INC via their blib
directories, but not all -- and the compilation tests fail immediately
because Moose is one of the missing dependencies. Moose is clearly
identified as a prereq in the metadata, both a v1.4 META.yml and a v2.0
META.json file and Makefile.PL.

FWIW, this type of failure was also recently reported here:
http://blogs.perl.org/users/joel_berger/2013/01/alienbase---progress-and-problems.html


Andreas Koenig

unread,
Jan 31, 2013, 2:00:36 AM1/31/13
to Karen Etheridge, NHO...@cpan.org, cpan-teste...@perl.org
Nigel, your smoker seems severly misconfigured. My first supect goes
against trust_test_report_history config variable. It should hardly ever
be true for a smoker. Can you post the output of 'o conf' of the smoker
involved in the report cited by Karen?

Thanks,
--
andreas

Andreas Koenig

unread,
Feb 1, 2013, 2:07:05 AM2/1/13
to Karen Etheridge, NHO...@cpan.org, cpan-teste...@perl.org
Andreas Koenig <andreas.koe...@franz.ak.mind.de> writes:

> Karen Etheridge <et...@cpan.org> writes:
>
>>> If you can point us at a couple of example failure reports, maybe
>>> someone here can figure out what's going wrong. In my experience it is
>>> almost always the case that things like this are caused by an error in
>>> the distribution under test.
>>
>> Here's the most recent example of such failures:
>> http://www.cpantesters.org/cpan/report/cb01432c-6737-11e2-992e-e7c00a4a7996

I have contacted Nigel and he sent me his CPAN.pm configuration and I
cannot see anything wrong in it. Especially my suspect about
trust_test_report_history was wrong, trust_test_report_history is off.

>> As you can see, only the configure_requires dependencies have been
>> installed.

I fear this is a misleading sentence, because fact is only that the
PREREQUISITES section of the report is too short. Normally
CPAN::Reporter tells us more about installed prerequisites. From this
fact we cannot make generalized assumptions about Nigel's installation.

>> *Some* of the runtime requirements are in @INC via their blib
>> directories, but not all -- and the compilation tests fail immediately
>> because Moose is one of the missing dependencies. Moose is clearly
>> identified as a prereq in the metadata, both a v1.4 META.yml and a v2.0
>> META.json file and Makefile.PL.

Well, I have tried to get my perl 5.8.1 to install Moose and it fails
early because Moose demands 5.8.3. After that CPAN::Reporter refused to
write a test report because of missing prerequisites. And this is what
we expect. At the moment I have no further ideas which bug or
misconfiguration might have let the report escape that you encountered.

Ideas, anybody?

--
andreas

David Golden

unread,
Feb 1, 2013, 5:50:19 AM2/1/13
to Andreas Koenig, Karen Etheridge, NHO...@cpan.org, cpan-teste...@perl.org
On Fri, Feb 1, 2013 at 2:07 AM, Andreas Koenig
<andreas.koe...@franz.ak.mind.de> wrote:
>
> I fear this is a misleading sentence, because fact is only that the
> PREREQUISITES section of the report is too short. Normally
> CPAN::Reporter tells us more about installed prerequisites. From this
> fact we cannot make generalized assumptions about Nigel's installation.

Something is going wrong either in CPAN.pm detection of prereqs or in
the report generation (or both). Relevant CPAN::Reporter code shows
that it's calling prereq_pm on the CPAN::Distribution object.

# Extract requires/build_requires from CPAN dist
my $prereq_pm = $dist->prereq_pm;
if ( ref $prereq_pm eq 'HASH' ) {
# is it the new CPAN style with requires/build_requires?
if (join(q{ }, sort keys %$prereq_pm) eq "build_requires requires") {
$need{requires} = $prereq_pm->{requires}
if ref $prereq_pm->{requires} eq 'HASH';
$need{build_requires} = $prereq_pm->{build_requires}
if ref $prereq_pm->{build_requires} eq 'HASH';
}
else {
$need{requires} = $prereq_pm;
}
}

I suggest Nigel should disable CPAN::Reporter (o conf test_report 0)
and try it manually and see what happens.

David

--
David Golden <x...@xdg.me>
Take back your inbox! → http://www.bunchmail.com/
Twitter/IRC: @xdg

Karen Etheridge

unread,
Feb 1, 2013, 10:39:23 AM2/1/13
to cpan-teste...@perl.org

> Well, I have tried to get my perl 5.8.1 to install Moose and it fails
> early because Moose demands 5.8.3. After that CPAN::Reporter refused to
> write a test report because of missing prerequisites. And this is what
> we expect. At the moment I have no further ideas which bug or
> misconfiguration might have let the report escape that you encountered.

It's not just a perl version thing... there are more reports for later
versions of perl, where Moose is capable of installing quite happily. e.g.:

http://www.cpantesters.org/cpan/report/788c0bbe-59e4-11e2-992e-e7c00a4a7996
http://www.cpantesters.org/cpan/report/3fac9ade-6a06-11e2-992e-e7c00a4a7996
http://www.cpantesters.org/cpan/report/cf194c38-6a1c-11e2-992e-e7c00a4a7996

Serguei Trouchelle

unread,
Feb 1, 2013, 11:59:49 PM2/1/13
to Andreas Koenig, Karen Etheridge, NHO...@cpan.org, cpan-teste...@perl.org
Andreas Koenig wrote:
>>>> If you can point us at a couple of example failure reports, maybe
>>>> someone here can figure out what's going wrong. In my experience it is
>>>> almost always the case that things like this are caused by an error in
>>>> the distribution under test.
>>> Here's the most recent example of such failures:
>>> http://www.cpantesters.org/cpan/report/cb01432c-6737-11e2-992e-e7c00a4a7996
> I have contacted Nigel and he sent me his CPAN.pm configuration and I
> cannot see anything wrong in it. Especially my suspect about
> trust_test_report_history was wrong, trust_test_report_history is off.
At least two of Nigel's smoker setups generate errors because they use NFS, and everything that uses IPC::Run3 fails
because File::Temp cannot guarantee successful unlinking on NFS. Probably, some modules that use File::Temp, may fail too.
In my opinion, these results are useless for module authors. I, as an author, do not benefit from reports like this:
http://www.cpantesters.org/cpan/report/835621d4-6b45-11e2-a82e-ca470f55919d as it has nothing to do with the module
itself. It's not even a problem with architecture (this module has no problems running on arm), it's a problem with
smoker setup.

--
S.T.

Gabor Szabo

unread,
Feb 2, 2013, 2:20:21 AM2/2/13
to Serguei Trouchelle, Andreas Koenig, Karen Etheridge, NHO...@cpan.org, cpan-teste...@perl.org
If a real world user can also have similar setup using NFS, then this
is a legitimate setup, isn't it?
Isn't this exactly the same situation as Windows vs Linux, where
Windows might not support all
the features Linux does?

Are those only test failures, or is the module (partially) broken on a
system using NFS?

If the module does not fully work over NFS then shouldn't it recognize
the situation
and disallow those features?

If only the tests fail on NFS, shouldn't those be skipped?

Gabor

David Golden

unread,
Feb 2, 2013, 9:00:21 AM2/2/13
to Gabor Szabo, Serguei Trouchelle, Andreas Koenig, Karen Etheridge, NHO...@cpan.org, cpan-teste...@perl.org
On Sat, Feb 2, 2013 at 2:20 AM, Gabor Szabo <ga...@szabgab.com> wrote:
> If a real world user can also have similar setup using NFS, then this
> is a legitimate setup, isn't it?
> Isn't this exactly the same situation as Windows vs Linux, where
> Windows might not support all
> the features Linux does?

+1

I've seen similar issues on Win32 depending on whether someone's
filesystem is FAT or NTFS.

The more salient question is whether we can identify such things (NFS)
in the report, so, say, something like Andreas' analysis sites could
clearly show the problem is NFS.

Separately -- if this is a IPC::Run3/File::Temp vs NFS issue, maybe it
should be fixed to deal more gracefully with an unlinking problem?

Marvin Humphrey

unread,
Feb 2, 2013, 10:00:50 AM2/2/13
to David Golden, Gabor Szabo, Serguei Trouchelle, Andreas Koenig, Karen Etheridge, NHO...@cpan.org, cpan-teste...@perl.org
On Sat, Feb 2, 2013 at 6:00 AM, David Golden <x...@xdg.me> wrote:
On Sat, Feb 2, 2013 at 2:20 AM, Gabor Szabo <ga...@szabgab.com> wrote:
>> If a real world user can also have similar setup using NFS, then this
>> is a legitimate setup, isn't it?
>> Isn't this exactly the same situation as Windows vs Linux, where
>> Windows might not support all
>> the features Linux does?
>
> +1

As with extending support to other operating systems, sometimes NFS
portability is easy and sometimes it's difficult or impossible.

If you have to assume that every file system operation might involve an NFS
mount, that imposes heavy burdens on certain fields of endeavor.

> The more salient question is whether we can identify such things (NFS)
> in the report, so, say, something like Andreas' analysis sites could
> clearly show the problem is NFS.

In a perfect world, CPAN distributions would be able to opt out of supporting
NFS as cleanly as they can with operating systems. :P

> Separately -- if this is a IPC::Run3/File::Temp vs NFS issue, maybe it
> should be fixed to deal more gracefully with an unlinking problem?

Dealing with NFS can be a real bear. Cache coherency, absence of
delete-on-last-close filehandle semantics, oi...

And unlike the OS, you can't pull information out of %Config::Config and
change up behavior as necessary.

I don't know the specifics of the IPC::Run3/File::Temp issue with NFS, but I
have some idea how crazy dealing with race conditions on NFS can be and
that's File::Temp's bailiwick. I'll bet the developers have been all over
this issue for a long time.

Marvin Humphrey

David Golden

unread,
Feb 2, 2013, 1:07:52 PM2/2/13
to Marvin Humphrey, Gabor Szabo, Serguei Trouchelle, Andreas Koenig, Karen Etheridge, NHO...@cpan.org, cpan-teste...@perl.org, Ricardo Signes
> I don't know the specifics of the IPC::Run3/File::Temp issue with NFS, but I
> have some idea how crazy dealing with race conditions on NFS can be and
> that's File::Temp's bailiwick. I'll bet the developers have been all over
> this issue for a long time.

I've suggested an easy fix in this RT ticket:
https://rt.cpan.org/Ticket/Display.html?id=82720

(Note that Nigel was the original requestor.)

RJBS: tl;dr: Unless Tim fixes File::Temp quickly, have IPC::Run3 use
File::Temp->new instead of tempfile() and IPC::Run3 will stop failing
on NFS filesystems, which will make lots of people happy. (DESTROY
doesn't have the same fatal problem with failing unlink.)

David

yary

unread,
Feb 2, 2013, 1:14:44 PM2/2/13
to cpan-teste...@perl.org
On Sat, Feb 2, 2013 at 10:00 AM, Marvin Humphrey <mar...@rectangular.com>wrote:

> > The more salient question is whether we can identify such things (NFS)
> > in the report, so, say, something like Andreas' analysis sites could
> > clearly show the problem is NFS.
>
> In a perfect world, CPAN distributions would be able to opt out of
> supporting
> NFS as cleanly as they can with operating systems. :P



It would be nice to have something like $OSNAME for filesystem type
discovery... Beef up Sys::Filesystem and put it in the core? Or at least
encourage its use where helpful...

-y

Michiel Beijen

unread,
Feb 2, 2013, 1:23:14 PM2/2/13
to yary, cpan-teste...@perl.org
On Sat, Feb 2, 2013 at 7:14 PM, yary <not...@gmail.com> wrote:
> It would be nice to have something like $OSNAME for filesystem type
> discovery... Beef up Sys::Filesystem and put it in the core? Or at least
> encourage its use where helpful...

That would only be helpful in limited cases, and in a very different
way than $OSNAME is. If your test runs somewhere in
/home/user/perl5/... and this is an NFS share, it could be it fails
because of the filesystem type, but it could be you actually want your
Perl code to run in /opt... which is on a BTRFS device, or the other
way around.
So the best way is to try and handle errors more gracefully, and this
could involve the use of Sys::Filesystem, maybe. Davids fix for using
File::Temp->new seems quite reasonable and more elegant, though.
--
Mike

Serguei Trouchelle

unread,
Feb 2, 2013, 3:08:36 PM2/2/13
to Gabor Szabo, Andreas Koenig, Karen Etheridge, NHO...@cpan.org, cpan-teste...@perl.org
Gabor Szabo wrote:
> If a real world user can also have similar setup using NFS, then this is a legitimate setup, isn't it?
I can hardly imagine a real system with /tmp being on NFS, but yes, it's possible, and I don't argue with this. My point
is, once problem is found, do authors need more reports with the same issue if this issue has nothing to do with the
module functionality? I think no. Problem appears when I call Test::Script::script_compiles, it calls IPC::Run3, which
in turn calls File::Temp, and "unlink0" function cannot guarantee actual removal of file on NFS. "script_compiles" is a
very basic sanity check and if it fails, then something is absolutely wrong.
Yes, most likely, Test::Script should have a test for this and refuse to install, but I don't see how *I* can do
something about it.
> Isn't this exactly the same situation as Windows vs Linux, where
> Windows might not support all
> the features Linux does?
Not really. If module fails on Windows, it fails every time "make test" is executed on Windows machine.
If module cannot be compiled on this machine (because of missing library or something), it will get UNKNOWN grade. It's
not the same situation.
I'm fine with getting UNKNOWN. I'm not fine with getting FAIL when it's a problem with testing infrastructure that my
modules are not using during real-world tasks.
> Are those only test failures, or is the module (partially) broken on a
> system using NFS?
Which module? If you're talking about my module, it's not.
File::Temp, which is used by *test* harness, is somewhat broken.
That's my point, when problem is found and located, then affected smoker should be stopped to prevent entropy increase.
> If the module does not fully work over NFS then shouldn't it recognize
> the situation
> and disallow those features?
Well, you cannot actually know is it NFS or not on when rusing library interfaces without hacks. And I don't like hacks.
> If only the tests fail on NFS, shouldn't those be skipped?
The only way I see it's skipped is to skip when $ENV{'HOME'} =~ m!^/home/njh! but it's extremely counter-productive. I'd
rather live with false positives :)

--
S.T.

Serguei Trouchelle

unread,
Feb 2, 2013, 3:20:11 PM2/2/13
to yary, cpan-teste...@perl.org
yary wrote:

>>> The more salient question is whether we can identify such things (NFS)
>>> in the report, so, say, something like Andreas' analysis sites could
>>> clearly show the problem is NFS.

>> In a perfect world, CPAN distributions would be able to opt out of
>> supporting
>> NFS as cleanly as they can with operating systems. :P

> It would be nice to have something like $OSNAME for filesystem type
> discovery... Beef up Sys::Filesystem and put it in the core? Or at least
> encourage its use where helpful...

One can use different filesystems for different files.
NFS is usually used in home directories, and that's where people keep smokers.

"Enterprise level Perl installations" will not use NFS, I believe.

--
S.T.

David Golden

unread,
Feb 2, 2013, 3:24:30 PM2/2/13
to Serguei Trouchelle, Karen Etheridge, CPAN Testers Discuss, Andreas Koenig, NHO...@cpan.org, Gabor Szabo
By default, tempfile creates in cwd, which could easily be on NFS.

David

Eirik Berg Hanssen

unread,
Feb 2, 2013, 4:14:08 PM2/2/13
to David Golden, Serguei Trouchelle, Karen Etheridge, CPAN Testers Discuss, Andreas Koenig, NHO...@cpan.org, Gabor Szabo
On Sat, Feb 2, 2013 at 9:24 PM, David Golden <da...@autopragmatic.com>wrote:

> By default, tempfile creates in cwd, which could easily be on NFS.


Eh? Oh, that kind of default ...

$ perldoc File::Temp | grep -B5 -A1 ', as specified' ; perl -mFile::Spec -E
'say "On my system, ", File::Spec->tmpdir'

$fh = tempfile();
($fh, $filename) = tempfile();

Create a temporary file in the directory specified for temporary
files, as specified by the tmpdir() function in File::Spec.

On my system, /tmp
$

But, File::Spec->tmpdir returns "the current directory if no writable
temporary directories are found."

Okay, that's a default, in a sense.

But it is also a pretty flaky system, with cwd on NFS and no writable
temporary directory provided.


Eirik

yary

unread,
Feb 2, 2013, 7:33:45 PM2/2/13
to CPAN Testers Discuss
Veering off topic here- seems like the original issue is taken care of, and
I'm glad there's a workaround in sight that's lightweight.

Anyway, I've been in enterprises where NFS is in wide use, and I also used
a boot-from-network diskless server that had /tmp mounted via NFS... and as
for "One can use different filesystems for different files" of course
Sys::Filesystem is path-specific, I only mentioned $OSNAME to reference
something even more fundamental to the core than a core module.

David Cantrell

unread,
Feb 4, 2013, 10:00:05 AM2/4/13
to cpan-teste...@perl.org
On Fri, Feb 01, 2013 at 08:59:49PM -0800, Serguei Trouchelle wrote:

> At least two of Nigel's smoker setups generate errors because they use NFS,
> and everything that uses IPC::Run3 fails because File::Temp cannot
> guarantee successful unlinking on NFS. Probably, some modules that use
> File::Temp, may fail too.
> In my opinion, these results are useless for module authors.

I sympathise. However, if Nigel isn't paying attention, it would seem
that the best solution would be to file a bug report for File::Temp,
that being the root of these problems.

--
David Cantrell | even more awesome than a panda-fur coat

Blessed are the pessimists, for they test their backups

David Cantrell

unread,
Feb 4, 2013, 10:13:28 AM2/4/13
to cpan-teste...@perl.org
On Sat, Feb 02, 2013 at 01:14:44PM -0500, yary wrote:

> It would be nice to have something like $OSNAME for filesystem type
> discovery... Beef up Sys::Filesystem and put it in the core? Or at least
> encourage its use where helpful...

That's tricky. What you're really interested in isn't the filesystem
but the filesystem's capabilities. Consider, for example, NFS which
ends up on a FAT filesystem has different capabilities from NFS which
has an ext4 backend. And in the general case you can't tell what those
capabilities are without just trying them.

So, riffing off of Devel::CheckOS, you'd want an API something like
this ...

my $fs = Devel::CheckFS->new($filename);

print "$filename is on an FS that is case sensitive\n"
if($fs->supports('CaseSensitive'));

and because you want this to work on Windows as well as Unix, you need
to jump through all the painful hoops of making things like figuring out
what filesystem a file is on portable.

I'd love to see someone other than me write this code!

--
David Cantrell | Pope | First Church of the Symmetrical Internet

THIS IS THE LANGUAGE POLICE
PUT DOWN YOUR THESAURUS
STEP AWAY FROM THE CLICHE

yary

unread,
Feb 4, 2013, 10:24:06 AM2/4/13
to David Cantrell, cpan-teste...@perl.org
On Mon, Feb 4, 2013 at 10:13 AM, David Cantrell <da...@cantrell.org.uk>wrote:

> On Sat, Feb 02, 2013 at 01:14:44PM -0500, yary wrote:
>
> > It would be nice to have something like $OSNAME for filesystem type
> > discovery... Beef up Sys::Filesystem and put it in the core? Or at least
> > encourage its use where helpful...
>
> That's tricky. What you're really interested in isn't the filesystem
> but the filesystem's capabilities. Consider, for example, NFS which
> ends up on a FAT filesystem has different capabilities from NFS which
> has an ext4 backend. And in the general case you can't tell what those
> capabilities are without just trying them.
>
> So, riffing off of Devel::CheckOS, you'd want an API something like
> this ...
>
> my $fs = Devel::CheckFS->new($filename);
>
> print "$filename is on an FS that is case sensitive\n"
> if($fs->supports('CaseSensitive'));
>
> and because you want this to work on Windows as well as Unix, you need
> to jump through all the painful hoops of making things like figuring out
> what filesystem a file is on portable.
>
> I'd love to see someone other than me write this code!


Good points! I'd love to have time and money enough to allow me to work on
such code! lol, we know it sounds good, but not quite enough pain to be
bothered...

Reini Urban

unread,
Feb 4, 2013, 10:57:04 AM2/4/13
to cpan-teste...@perl.org
On 02/04/2013 09:13 AM, David Cantrell wrote:
> On Sat, Feb 02, 2013 at 01:14:44PM -0500, yary wrote:
>
>> It would be nice to have something like $OSNAME for filesystem type
>> discovery... Beef up Sys::Filesystem and put it in the core? Or at least
>> encourage its use where helpful...
>
> That's tricky. What you're really interested in isn't the filesystem
> but the filesystem's capabilities. Consider, for example, NFS which
> ends up on a FAT filesystem has different capabilities from NFS which
> has an ext4 backend. And in the general case you can't tell what those
> capabilities are without just trying them.
>
> So, riffing off of Devel::CheckOS, you'd want an API something like
> this ...
>
> my $fs = Devel::CheckFS->new($filename);
>
> print "$filename is on an FS that is case sensitive\n"
> if($fs->supports('CaseSensitive'));
>
> and because you want this to work on Windows as well as Unix, you need
> to jump through all the painful hoops of making things like figuring out
> what filesystem a file is on portable.

It's more like identifying the mountpoint per path,
and then retrieving the mount options per mount.
It has almost nothing to do with $^O.

I started refactoring File::Spec::case_tolerant to
take a mountpoint as optional argument for cygwin perl,
but there was never a reply on p5p.

--
Reini

Working towards a true Modern Perl.
Slim, functional, unbloated, compile-time optimizable

David Cantrell

unread,
Feb 4, 2013, 11:59:07 AM2/4/13
to cpan-teste...@perl.org
On Mon, Feb 04, 2013 at 09:57:04AM -0600, Reini Urban wrote:
> On 02/04/2013 09:13 AM, David Cantrell wrote:
> >and because you want this to work on Windows as well as Unix, you need
> >to jump through all the painful hoops of making things like figuring out
> >what filesystem a file is on portable.
> It's more like identifying the mountpoint per path,
> and then retrieving the mount options per mount.
> It has almost nothing to do with $^O.

That's actually not easy to do portably. See Sys-Filesystem.
Then you have to figure out what those options mean on each OS.

And the mount options won't tell you whether your NFS volume is
backed by something sensible like UFS or ext4, or whether the admin has
done something crazy like export a FAT filesystem or re-export something
from Samba. This is important if you want to know things like whether
hard links are supported, or whether it is case-sensitive vs
case-preserving vs case-smashing.

--
David Cantrell | Reality Engineer, Ministry of Information

Fashion label: n: a liferaft for personalities
which lack intrinsic buoyancy

Marvin Humphrey

unread,
Feb 4, 2013, 12:16:40 PM2/4/13
to David Cantrell, cpan-teste...@perl.org
On Mon, Feb 4, 2013 at 8:59 AM, David Cantrell <da...@cantrell.org.uk> wrote:
> And the mount options won't tell you whether your NFS volume is
> backed by something sensible like UFS or ext4, or whether the admin has
> done something crazy like export a FAT filesystem or re-export something
> from Samba. This is important if you want to know things like whether
> hard links are supported, or whether it is case-sensitive vs
> case-preserving vs case-smashing.

Say that your CPAN module depends on one of those features, e.g. hard linking.
When a CPAN tester runs your module's test suite on a file system where hard
linking is not supported, what should happen? Once testing is underway, the
only options are to either A) fail or B) skip, possibly obscuring true
failures.

Is hard linking not a legitimate use case in the judgment of CPAN testers
because it is not sufficiently portable? Is it the responsibility of module
authors to "fix" their distributions to accommodate the lowest common
denominator of file system features?

Marvin Humphrey

David Cantrell

unread,
Feb 4, 2013, 12:49:09 PM2/4/13
to cpan-teste...@perl.org
On Mon, Feb 04, 2013 at 09:16:40AM -0800, Marvin Humphrey wrote:
> On Mon, Feb 4, 2013 at 8:59 AM, David Cantrell <da...@cantrell.org.uk> wrote:
> > And the mount options won't tell you whether your NFS volume is
> > backed by something sensible like UFS or ext4, or whether the admin has
> > done something crazy like export a FAT filesystem or re-export something
> > from Samba. This is important if you want to know things like whether
> > hard links are supported, or whether it is case-sensitive vs
> > case-preserving vs case-smashing.
> Say that your CPAN module depends on one of those features, e.g. hard linking.
> When a CPAN tester runs your module's test suite on a file system where hard
> linking is not supported, what should happen? Once testing is underway, the
> only options are to either A) fail or B) skip, possibly obscuring true
> failures.
>
> Is hard linking not a legitimate use case in the judgment of CPAN testers
> because it is not sufficiently portable?

Testers are happy to test modules that are specific to weird legacy
proprietary OSes, so portability isn't a problem for us. We merely
encourage people to declare their non-portability in advance.

> Is it the responsibility of module
> authors to "fix" their distributions to accommodate the lowest common
> denominator of file system features?

No, I think it's authors' responsibility, if they don't want to fall
foul of the tester running the wrong OS or whatever, to declare their
pre-requisites, and, if they forget to declare some of them, to be
prepared to put up with the occasional test failure report.

For something that is so rarely a problem as working hard links, I'd be
inclined to not care, no matter whether I'm wearing my testing hat or
my author hat, until it becomes a problem for me. As it happens, I do
have one project that depends on working hard links, and when it gets to
the stage where that matters to my tests, I'll do something about it,
and if I'm feeling generous I'll do it in such a way that others can add
their own filesystem capability checks.

--
David Cantrell | London Perl Mongers Deputy Chief Heretic

Graecum est; non legitur

David Golden

unread,
Feb 4, 2013, 1:16:47 PM2/4/13
to David Cantrell, cpan-teste...@perl.org
On Mon, Feb 4, 2013 at 10:00 AM, David Cantrell <da...@cantrell.org.uk> wrote:
>
> I sympathise. However, if Nigel isn't paying attention, it would seem
> that the best solution would be to file a bug report for File::Temp,
> that being the root of these problems.

In case you missed it in the thread, already done:

https://rt.cpan.org/Ticket/Display.html?id=82720

David Golden

unread,
Feb 4, 2013, 1:19:35 PM2/4/13
to David Cantrell, cpan-teste...@perl.org
On Mon, Feb 4, 2013 at 10:13 AM, David Cantrell <da...@cantrell.org.uk> wrote:
>
> my $fs = Devel::CheckFS->new($filename);

I didn't get the point of $filename until later when I realized how
elegantly it solves the filesystem problem.

E.g. create a tempdir, test a file in that tempdir. If things work,
then tests that create and do work in a tempdir should work, and so
on.

Though for tests, probably File::Spec->tmpdir and "." (relative to
Makefile.PL) are the only things of interest.

Easy to write a hacky version, but a bitch to write well, though.

David

Marvin Humphrey

unread,
Feb 4, 2013, 2:42:20 PM2/4/13
to David Cantrell, cpan-teste...@perl.org
On Mon, Feb 4, 2013 at 9:49 AM, David Cantrell <da...@cantrell.org.uk> wrote:
> Testers are happy to test modules that are specific to weird legacy
> proprietary OSes, so portability isn't a problem for us. We merely
> encourage people to declare their non-portability in advance.

Well, what about NFS, then?

http://www.time-travellers.org/shane/papers/NFS_considered_harmful.html

While NFS is a reasonable approach to the problem of remote access to
disks, it fails to do so in a way that preserves Unix file semantics.
Using NFS for complex networking issues is often a poorly thought out
attempt to use a simple solution to a complex problem.

NFS suffers from a fundamental design flaw, attempting to find a
compromise between the desire to deal with the network environment
efficiently and preserve existing access semantics. The advertised ability
of NFS to treat network resources as if they were local plainly does not
measure up under close scrutiny.

All the projects I work on are NFS compatible, but the engineering cost to get
them there was way out of scale. And I hate recieving patches that touch NFS
functionality -- because NFS is deceptively difficult to understand, and
because concurrency is hard and concurrency is where NFS fails most subtly,
patches which seem "simple" to the contributor are in fact exhausting to
review.

> No, I think it's authors' responsibility, if they don't want to fall
> foul of the tester running the wrong OS or whatever, to declare their
> pre-requisites, and, if they forget to declare some of them, to be
> prepared to put up with the occasional test failure report.

How does one declare to CPAN testers that a module does not support NFS?

Marvin Humphrey

David Cantrell

unread,
Feb 4, 2013, 6:56:07 PM2/4/13
to cpan-teste...@perl.org
On Mon, Feb 04, 2013 at 01:19:35PM -0500, David Golden wrote:
> On Mon, Feb 4, 2013 at 10:13 AM, David Cantrell <da...@cantrell.org.uk> wrote:
> > my $fs = Devel::CheckFS->new($filename);
> I didn't get the point of $filename until later when I realized how
> elegantly it solves the filesystem problem.
> E.g. create a tempdir, test a file in that tempdir. If things work,
> then tests that create and do work in a tempdir should work, and so
> on.

Thinking more about it, you don't really care where the filesystem is
mounted, you just care about the capabilities that apply in a particular
directory. And given that, to cope with shenanigans such as someone
re-exporting a Samba FS over NFS you need to check those capabilities by
actually trying to do stuff, then you might as well just try to do stuff
and ignore mount points and whatnot altogether.

I'm still not gonna write this sucker until I need it though :-)

--
David Cantrell | Minister for Arbitrary Justice

Seven o'clock in the morning is something that
happens to those less fortunate than me

David Cantrell

unread,
Feb 4, 2013, 7:05:31 PM2/4/13
to cpan-teste...@perl.org
On Mon, Feb 04, 2013 at 11:42:20AM -0800, Marvin Humphrey wrote:
> On Mon, Feb 4, 2013 at 9:49 AM, David Cantrell <da...@cantrell.org.uk> wrote:
> > Testers are happy to test modules that are specific to weird legacy
> > proprietary OSes, so portability isn't a problem for us. We merely
> > encourage people to declare their non-portability in advance.
> Well, what about NFS, then?

Ultimately it's no different from any other non-perl-module dependency.

> > No, I think it's authors' responsibility, if they don't want to fall
> > foul of the tester running the wrong OS or whatever, to declare their
> > pre-requisites, and, if they forget to declare some of them, to be
> > prepared to put up with the occasional test failure report.
> How does one declare to CPAN testers that a module does not support NFS?

There is no tool yet for doing this easily, but I suggest you start by
reading this:
http://wiki.cpantesters.org/wiki/CPANAuthorNotes

I recommend that you write some code involving Sys::Filesystem to figure
out if your tests are trying to use an NFS volume and then either skip
tests or bail out completely at Makefile.PL-time.

--
David Cantrell | semi-evolved ape-thing

Perl: the only language that makes Welsh look acceptable
0 new messages