CPAN TESTERS ALERT -- throttle your smokers NOW
David Golden
CPAN Testers have cracked 10 reports a minute (a rate of about 400,000
reports per month), which is threatening to overwhelm the Perl NOC's
email system.
If we can throttle our rate down significantly, Robert is willing to
give us until March 1, 2010 to launch CPAN Testers 2.0 and shut down
email transport and the NNTP backing store. If not, we'll be cut off
sooner.
So we now have a deadline -- which we didn't have before -- and we
need to get focused on finishing up CPAN Testers 2.0. I'm about done
with my work on blead and can refocus on it I'll probably put
together a mini project plan and be requesting volunteers for pieces
of work to help get it done.
In the meantime, please slow down testing -- put sleep loops in the
smoker code or something. Whatever you have to do -- we need to get
back down to the 100,000 reports/month level (or further) to take some
pressure off the NOC.
On the positive side -- we've been wonderfully successful attracting
new participants and testing new platforms, so kudos. But now we need
to make this sucker scale better.
-- David
George Greer
> In the meantime, please slow down testing -- put sleep loops in the
> smoker code or something. Whatever you have to do -- we need to get
> back down to the 100,000 reports/month level (or further) to take some
> pressure off the NOC.
In the spirit of the season, I'll give my smoker a holiday vacation.
--
George Greer
Martin Evans
I stopped a week ago since the majority of my reports were being
returned with date too far in the past i.e., I guess they failed to send
but sendmail continued to try for 5 days then the other end threw them
out when they did get through.
Martin
--
Martin J. Evans
Easysoft Limited
http://www.easysoft.com
Andreas J. Koenig
> In the meantime, please slow down testing -- put sleep loops in the
> smoker code or something.
I'd like to continue smoking blead on newly uploaded distros only. I've
disabled all other perls now.
--
andreas
Serguei Trouchelle
> In the meantime, please slow down testing -- put sleep loops in the
> smoker code or something. Whatever you have to do -- we need to get
> back down to the 100,000 reports/month level (or further) to take some
> pressure off the NOC.
Most valuable tests (IMO) are FAIL tests, because they indicate that something's not right and raise author's attention.
We had about 30,000-40,000 FAIL tests monthly during last 6 months, and it's less than 100,000.
It may be a good idea to send FAIL/UNKNOWN reports and hold on with OK/NA.
I estimated my smoking volume comparing to other testers and started to send FAIL/UNKNOWN not faster than 1 report/min
rate and OK/NA at 1 report/10 mins.
--
Serguei Trouchelle
Andreas J. Koenig
> Most valuable tests (IMO) are FAIL tests, because they indicate that
> something's not right and raise author's attention. We had about
> 30,000-40,000 FAIL tests monthly during last 6 months, and it's less
> than 100,000.
> It may be a good idea to send FAIL/UNKNOWN reports and hold on with
> OK/NA.
I disagree. I'd start from the other end: the least interesting tests
are those that have been reported several hundred times.
If everybody sends only fails we lose the ability to analyse the reasons
for the fails. Once in a while every distro gets a fail and after a
while the whole CPAN would look bad and we would not be able to
tell if it is true.
If you're able to limit your smpokers to uploads within the last 6
months you're most likely throttling well.
And before somebody argues that even old uploads may be interesting: I
wholeheartedly agree. It's a pity if we have to throttle and I believe
we do the least harm by concentrating on recent uploads.
--
andreas
Chris Marshall
>
> If everybody sends only fails we lose the ability to analyse the reasons
> for the fails. Once in a while every distro gets a fail and after a
> while the whole CPAN would look bad and we would not be able to
> tell if it is true.
The proportion of PASS/FAIL is useful to ms as an indicator of
how serious a problem is and whether is is likely in the
general code or platform specific sections.
Also, modules with many/all FAILs may be perceived as being of low
quality. Many FAILs or FAILs on one's platform type can indicate
that the module is not robust and may be difficult to configure
and install.
--Chris
Andreas J. Koenig
> In the meantime, please slow down testing -- put sleep loops in the
> smoker code or something. Whatever you have to do -- we need to get
> back down to the 100,000 reports/month level (or further) to take some
> pressure off the NOC.
We have not yet throttled enough to reach 100000/month. I pulled the
stats file from devel.cpantesters.org and saw excellent reductions:
Total report per day:
14206 20091213
27881 20091214
19905 20091215
10092 20091216
15618 20091217
8355 20091218
8217 20091219
6250 20091220
But 6000*30 is still 180000:(
Top reporters on 20091220:
2198 bingos
1343 apocal
746 Oliver Paukstadt
562 david@cantrell
298 randir
187 salvacpan
175 stro
127 potatohead
123 andreas.koenig.gmwojprw
Please ACK,
--
andreas
pe...@0ne.us
I've given my smokers a holiday treat - they are going to be offline
until at least 2010 :)
When I come back I'll continue to throttle the smokers. My emailer
was configured to send 1 report every minute. But this means 3600
reports a day, and that might still be too much :( By then I'll catch up
with the mailing list and see if I should throttle myself even more...
Thanks again and have a great holidays everyone!
Barbie
been working on recently:
http://www.cpantesters.org/home/status
This page is still beta, and there is more work on it planned.
The first two 1 week charts highlight the drop per day of report
submissions. The number of requests isn't reports alone, but they do
make up the bulk of the requests.
In the new year I plan to work on stats to show report submissions per
hour and per day, to help give a more realtime feel to the sites. This
will then give testers a feel for whether we are hitting the right
numbers.
Cheers,
Barbie.
--
Birmingham Perl Mongers <http://birmingham.pm.org>
Memoirs Of A Roadie <http://barbie.missbarbell.co.uk>
CPAN Testers Blog <http://blog.cpantesters.org>
YAPC Conference Surveys <http://yapc-surveys.org>
David Cantrell
> Andreas J. Koenig wrote:
> >We have not yet throttled enough to reach 100000/month. I pulled the
> >stats file from devel.cpantesters.org and saw excellent reductions:
> until at least 2010 :)
I'm (slowly) catching up with mail after being offline for a few weeks
over Christmas. I'm gonna stop testing 5.10.* on Linux cos let's face
it, EVERYONE tests that.
--
David Cantrell | Enforcer, South London Linguistic Massive
Languages for which ISO-Latin-$n is not necessary, #1 in a series:
Latin
M W487
since cygwin and Windows seem to account for less than 4% of incoming
reports now, are there any objections to my starting again?
Burak Gürsoy
Turning off a Windows smoker is a bad idea (IMHO), as you said it's quite a
rarity :)
David Golden
>> I shut down my (cygwin and Windows) smoking upon your request, but
>> since cygwin and Windows seem to account for less than 4% of incoming
>> reports now, are there any objections to my starting again?
>
> Turning off a Windows smoker is a bad idea (IMHO), as you said it's quite a
> rarity :)
And Windows smokers are generally very slow, anyway, because of all
the perl processes that get spawned for every Makefile command. (E.g.
"fire up perl to copy a file... fire up perl to copy the next file...
ugh...")
-- David
Barbie
We were doing fine keeping below 5k reports a day, so adding some of
yours wouldn't have been a problem.
However, apo...@cpan.org is currently submitting nearly 5k a day alone,
so we're getting back up to 10k reports a day.
APOCAL can you please throttle your reports.
Nigel Horne
"blacklisted envelope" problem that CPAN testers
seems to suffer from:
CPAN::Reporter: Test result is 'pass', All tests successful.
CPAN::Reporter: preparing a CPAN Testers report for Email-Find-0.10
CPAN::Reporter: sending test report with 'pass' to cpan-t...@perl.org
CPAN::Reporter: Test::Reporter: error from
'Test::Reporter::Transport::Net::SMTP
:'
Net::SMTP: 5.7.1 <cpan-t...@perl.org>... Relaying denied. Proper
authentication required.
-Nigel
David Golden
> Net::SMTP: 5.7.1 <cpan-t...@perl.org>... Relaying denied. Proper
> authentication required.
You might be in the "wait for March/April" category.
I can't wait until we're finally free of email for reports.
David