Should I be reporting some of these modules I am adding to my disabled.yml
Martin J. Evans
e.g., killing off ones which prompt or loop after a time etc) but I just
use CPAN::Reporter::Smoker as I like to be able to see exactly what is
going on. My list of modules which prompt or loop or consume all memory
etc is growing like mad (over 180 now). I can't help feeling I should be
doing more than just ignoring these. The reasons I have gathered so far are:
o prompts unnecessarily (sometimes just to warn you about something
e.g., there is one module which forces you to acknowledge it is
unsupported).
o prompts for location of library X
o prompts to ask if you want feature X
o prompts for something else without using "prompt"
e.g., can't start firefox - start it manually and press a key
o loops forever (often repeatedly saying can't do something only to do
it again)
o hangs
o consumes all memory until my machine grinds to a halt
o fails with deep recursion but for some reason the test does not spot
this and it hangs
Should I be reporting any of these?
In most cases, I note the module and why I will add it to disabled.yml
then ctrl/c it to move on to the next one. This of course, means that
not even a report is sent - is there another way to do this?
Martin
Slaven Rezic
[...]
> o loops forever (often repeatedly saying can't do something only to do
> it again)
> o hangs
> o consumes all memory until my machine grinds to a halt
> o fails with deep recursion but for some reason the test does not spot
> this and it hangs
>
> Should I be reporting any of these?
Best is to set resource limits to avoid problems with resource hungry or
looping processes. I use the following lines before starting my smoker
--- sorry, the "limits" command is BSD-specific, but you should be able
to use the perl module BSD::Resource on non-BSD systems instead:
my $data_limit = 500*1024*1024;
my $cputime_limit = 3600*1;
my $openfiles_limit = 2048;
limits -t $cputime_limit -d $data_limit -v $data_limit -n $openfiles_limit ...
A distribution which is killed because of exceeding resource limits will
probably get a FAIL report in this case.
> In most cases, I note the module and why I will add it to disabled.yml
> then ctrl/c it to move on to the next one. This of course, means that
> not even a report is sent - is there another way to do this?
If I am sure that a test suite hangs (i.e. after inspecing run time with
ps and maybe also the last system call using strace/truss), then I
"kill -9" the process. This also causes a FAIL report.
Regards,
Slaven
--
Slaven Rezic - slaven <at> rezic <dot> de
tkruler - Perl/Tk program for measuring screen distances
http://ptktools.sourceforge.net/#tkruler
Chris 'BinGOs' Williams
> I don't know how many people do fully automated smoking (unattended,
> e.g., killing off ones which prompt or loop after a time etc) but I just
> use CPAN::Reporter::Smoker as I like to be able to see exactly what is
> going on. My list of modules which prompt or loop or consume all memory
> etc is growing like mad (over 180 now). I can't help feeling I should be
> doing more than just ignoring these. The reasons I have gathered so far are:
>
> o prompts unnecessarily (sometimes just to warn you about something
> e.g., there is one module which forces you to acknowledge it is
> unsupported).
> o prompts for location of library X
> o prompts to ask if you want feature X
> o prompts for something else without using "prompt"
> e.g., can't start firefox - start it manually and press a key
> o loops forever (often repeatedly saying can't do something only to do
> it again)
> o hangs
> o consumes all memory until my machine grinds to a halt
> o fails with deep recursion but for some reason the test does not spot
> this and it hangs
>
Been there, done that, got the T-Shirt >:)
These are the very reasons that prompted (pardon the pun) me to wrap smoke
testing in POE in the first place.
POE::Component::SmokeBox and the front-end for it minismokebox, are pretty much
the accumulation of 4 years would of CPAN Testing experience.
It handles all of the above cases and I get on with my life without having to
worry about it.
> Should I be reporting any of these?
I've not bothered myself after finding I was ignored the first couple of times I
reported things, ( or found Andreas had reported it already and it had been ignored ).
But I have been thinking recently that I have this data ( PoCo-SmokeBox does indicate whether
it has killed off a smoke job ) and I haven't been collecting it, but I might start to.
Cheers,
--
Chris Williams
aka BinGOs
PGP ID 0x4658671F
http://www.gumbynet.org.uk
==========================
Barbie
> On Mon, Nov 23, 2009 at 08:21:24PM +0000, Martin J. Evans wrote:
> > I don't know how many people do fully automated smoking (unattended,
> > e.g., killing off ones which prompt or loop after a time etc) but I just
> > use CPAN::Reporter::Smoker as I like to be able to see exactly what is
> > going on. My list of modules which prompt or loop or consume all memory
> > etc is growing like mad (over 180 now). I can't help feeling I should be
> > doing more than just ignoring these. The reasons I have gathered so far are:
> >
> > o prompts unnecessarily (sometimes just to warn you about something
> > e.g., there is one module which forces you to acknowledge it is
> > unsupported).
> > o prompts for location of library X
> > o prompts to ask if you want feature X
> > o prompts for something else without using "prompt"
> > e.g., can't start firefox - start it manually and press a key
> > o loops forever (often repeatedly saying can't do something only to do
> > it again)
> > o hangs
> > o consumes all memory until my machine grinds to a halt
> > o fails with deep recursion but for some reason the test does not spot
> > this and it hangs
> >
>
> Been there, done that, got the T-Shirt >:)
Likewise :)
I even did a YAPC talk about it back in 2005 & 2007 [1]
[1] http://birmingham.pm.org/talks/barbie/cpanready2
> > Should I be reporting any of these?
>
> I've not bothered myself after finding I was ignored the first couple
> of times I reported things
Again likewise. I even sent patches on a few occasions, but I never
noticed them getting implemented.
If you have the time, then I'd say report them, as at least others are
then aware that they may prove irritating to install without knowing the
settings (or understanding them in some cases).
> But I have been thinking recently that I have this data (
> PoCo-SmokeBox does indicate whether it has killed off a smoke job )
> and I haven't been collecting it, but I might start to.
I'd be happy to post it on the stats site somewhere :)
Cheers,
Barbie.
--
Birmingham Perl Mongers <http://birmingham.pm.org>
Memoirs Of A Roadie <http://barbie.missbarbell.co.uk>
CPAN Testers Blog <http://blog.cpantesters.org>
YAPC Conference Surveys <http://yapc-surveys.org>
Andreas J. Koenig
>> I've not bothered myself after finding I was ignored the first couple
>> of times I reported things
> Again likewise. I even sent patches on a few occasions, but I never
> noticed them getting implemented.
Key is how old the distro is. I observe a high number of tickets being
resolved on everything that is younger than four weeks or so. The
attention of the authors is still awake so short after the release. Or
they still have a console open in the source directory:)
> If you have the time, then I'd say report them, as at least others are
> then aware that they may prove irritating to install without knowing the
> settings (or understanding them in some cases).
Agree.
--
andreas
David Cantrell
> o prompts unnecessarily (sometimes just to warn you about something
> e.g., there is one module which forces you to acknowledge it is
> unsupported).
> o prompts for location of library X
> o prompts to ask if you want feature X
Solved using expect in CPAN Distroprefs. Andreas distributes some
examples with CPAN.pm. eg
http://cpansearch.perl.org/src/ANDK/CPAN-1.94_52/distroprefs/MI.yml
And (if you share your distroprefs between different machines) you can
customise per machine, eg
http://cpansearch.perl.org/src/ANDK/CPAN-1.94_52/distroprefs/DCANTRELL.Devel-CheckOS.yml
> o prompts for something else without using "prompt"
> e.g., can't start firefox - start it manually and press a key
I'd disable that one, or perhaps test it manually once. Maybe notify
the author and suggest that he check for AUTOMATED_TESTING.
> o loops forever (often repeatedly saying can't do something only to do
> it again)
> o hangs
> o consumes all memory until my machine grinds to a halt
> o fails with deep recursion but for some reason the test does not spot
> this and it hangs
I'd disable the current version of those and report them to the author
manually. If future versions still have the bug, I'd just disable
testing for all versions of that distribution.
--
David Cantrell | Reality Engineer, Ministry of Information
It's my experience that neither users nor customers can articulate
what it is they want, nor can they evaluate it when they see it
-- Alan Cooper
David Cantrell
> Best is to set resource limits to avoid problems with resource hungry or
> looping processes. I use the following lines before starting my smoker
> --- sorry, the "limits" command is BSD-specific, but you should be able
> to use the perl module BSD::Resource on non-BSD systems instead:
ulimit should be available in most places, although the exact
command-line arguments vary wildly.
Incidentally, if you're testing on shared machines, I recommend using
nice -n 19 so that if something *does* go into an infinite loop, you
don't piss off all the other users. Yes, this means that other users
can make your tests hang when *their* processes go mad, but you you can
sleep soundly in the knowledge that you at least tried to play nicely
with others :-)
--
David Cantrell | Reality Engineer, Ministry of Information
When a man is tired of London, he is tired of life
-- Samuel Johnson
Chris Marshall
> "Martin J. Evans" <martin...@easysoft.com> writes:
>
> [...]
>> o loops forever (often repeatedly saying can't do something only to do
>> it again)
>> o hangs
>> o consumes all memory until my machine grinds to a halt
>> o fails with deep recursion but for some reason the test does not spot
>> this and it hangs
>>
>> Should I be reporting any of these?
As an author who it trying to improve modules I am contributing
to, I definitely would prefer to know if there is some problem
with the CPAN Testing of PDL or OpenGL.
For example, both PDL and OpenGL have large XS files leading to
high memory usage during compile which could definitely fit the
"consumes all memory until my machine grinds to a halt" category
if the allocated memory is low leading to swapping or out-of-memory
failures.
This situation can also come up if the smoker starts to build
PDL on all of its test environments at the same time---even if
there is enough memory for one to build at a time.
--Chris
PDL and POGL developer
> Best is to set resource limits to avoid problems with resource hungry or
> looping processes. I use the following lines before starting my smoker
> --- sorry, the "limits" command is BSD-specific, but you should be able
> to use the perl module BSD::Resource on non-BSD systems instead:
>
> my $data_limit = 500*1024*1024;
> my $cputime_limit = 3600*1;
> my $openfiles_limit = 2048;
> limits -t $cputime_limit -d $data_limit -v $data_limit -n $openfiles_limit ...
>
> A distribution which is killed because of exceeding resource limits will
> probably get a FAIL report in this case.
>
>> In most cases, I note the module and why I will add it to disabled.yml
>> then ctrl/c it to move on to the next one. This of course, means that
>> not even a report is sent - is there another way to do this?
>
> If I am sure that a test suite hangs (i.e. after inspecing run time with
> ps and maybe also the last system call using strace/truss), then I
> "kill -9" the process. This also causes a FAIL report.
>
> Regards,
> Slaven
>
>
>
> ------------------------------------------------------------------------
>
>
> No virus found in this incoming message.
> Checked by AVG - www.avg.com
> Version: 8.5.425 / Virus Database: 270.14.79/2522 - Release Date: 11/23/09 19:45:00
>