New analytical statistics site asking for feedback
Andreas J. Koenig
few weeks on. During the course I wrote over 200 RT tickets and there is
plenty of material for several hundreds more.
At the moment this is just about running regression analysis between the
variables provided by cpantesters. About 2500 critical releases of the
last three years are examined with all due care. The homepage is a huge
table with plenty of links to zoom in.
http://pause.perl.org:3000/ # main table
http://pause.perl.org:3000/about # blurb
It's running on pause with a low priority catalyst server process, so
please be patient and take it as a private showing. I'll have to work
some more to get this application down to use less memory and CPU. But
of course it's much more fun to get it to *do* more.
Your feedback is highly appreciated. In other words, what would *you*
expect from analytics.cpantesters.org?
--
andreas
PS: If you have a spare box to host the thing I'm looking forward to your
offer. Pause is not the ideal place to keep it on.
David Landgren
> Today I'm proud to present what I spent most of my time over the last
> few weeks on. During the course I wrote over 200 RT tickets and there is
> plenty of material for several hundreds more.
>
> At the moment this is just about running regression analysis between the
> variables provided by cpantesters. About 2500 critical releases of the
> last three years are examined with all due care. The homepage is a huge
> table with plenty of links to zoom in.
>
> http://pause.perl.org:3000/ # main table
> http://pause.perl.org:3000/about # blurb
>
> It's running on pause with a low priority catalyst server process, so
> please be patient and take it as a private showing. I'll have to work
> some more to get this application down to use less memory and CPU. But
> of course it's much more fun to get it to *do* more.
>
> Your feedback is highly appreciated. In other words, what would *you*
> expect from analytics.cpantesters.org?
I think I understand what this is telling me, but there's a lot of
information and it's difficult to pin-point what is important. Looking
at the File-Path distro, with 300+ passes to 3 or so fails, I can see
what appears to influence the likelihood of a fail.
Then again: looking at BSD-Process, which will fail anywhere except on
FreeBSD, I can't see where it highlights the fact that, well hey, this
module tends to fail with surprising regularity on Linux.
I need a summary that tells me more clearly "this modules tends to fail
in these circumstances".
It wouldn't have to be pretty graphics. Just using color, with red is
bad to green is good would be useful.
Later,
David
--
it's an anthem in a vacuum on a hyperstation, day dreaming days in a
daydream nation
Andreas J. Koenig
> Hi Andreas,
> BTW I think the site could make very interesting reading, however, I
> suggest you add something on the about page to explain what the columns
> mean for the regression tables.
> I have no ideas what the 3 number columns represent, and the R^2, N and
> K are meaningless to me I'm afraid. I suspect they have mathematical
> meanings, but I don't have that high level background, and suspect there
> are others who would be as mystified. If you can include references and
> links (Wikipedia maybe?) that would help.
Sorry for that. I beefed the "about" page up a little and added column
names to all tables. I was too deep into the matter to think of the
trivial things. I still think this needs a few iterations of
improvement.
You can read the diff at
http://repo.or.cz/w/andk-cpan-tools.git/commitdiff/f07c51aef089e40b77e5cfd80eb85d95471eb3cb
>>>>> On Wed, 02 Dec 2009 22:23:52 +0100, David Landgren <da...@landgren.net> said:
> I think I understand what this is telling me, but there's a lot of
> information and it's difficult to pin-point what is important. Looking
> at the File-Path distro, with 300+ passes to 3 or so fails, I can see
> what appears to influence the likelihood of a fail.
It's really just like a grep over all results without downloading them.
Next step for you is to click on the title of the
Regression qr:(Failed test\s+\S+.*)
table and get links to the individual tests.
> Then again: looking at BSD-Process, which will fail anywhere except on
> FreeBSD, I can't see where it highlights the fact that, well hey, this
> module tends to fail with surprising regularity on Linux.
I reprint a few lines from the
Regression conf:archame+osvers
table:
Name Theta StdErr T-test
[0='const'] 0.0000 0.0888 0.00
[2='eq_amd64-freebsd 6.2-release'] 0.6667 0.1356 4.92
[7='eq_i386-freebsd 6.1-release'] 0.5000 0.1538 3.25
[12='eq_i386-freebsd-64int 6.0-stable'] 1.0000 0.1985 5.04
[13='eq_i386-freebsd-64int 6.1-release'] 1.0000 0.1985 5.04
[14='eq_i386-freebsd-64int 7.2-release'] 0.0000 0.1538 0.00
[15='eq_i386-freebsd-thread-multi 6.1-release'] 1.0000 0.1985 5.04
[18='eq_i686-linux 2.4.27-2-386'] 0.0000 0.1985 0.00
[20='eq_i686-linux 2.6.24.3'] 0.0000 0.1356 0.00
[21='eq_i686-linux-64int 2.6.22-1-k7'] 0.0000 0.1538 0.00
First of all, the const line has a Theta of zero, so we start with 0 and
look for positive values. (It might come the opposite direction as well,
with const being 1 and influencing factors having negative impact on the
success).
Where Theta is one (and T-test is >0) we may expect success. Where Theta
is zero we must expect no success. Where Theta is in between we know
that we had both success and failure on this operating system/version.
Again, if you click on the link at the top of the regression table, you
find the three tests for amd64-freebsd 6.2-release:
1094946 FAIL
715474 PASS
714951 PASS
That gives 2/3 PASS => 0.0000+0.6667 expected value of the success.
> I need a summary that tells me more clearly "this modules tends to
> fail in these circumstances".
I'll work this example into the about page later. Thanks for asking:)
--
andreas