SKIPs Are Now a Code Smell

[Chromatic]

I just made two checkins to unskip a couple of reasonably valid tests.
Currently, there are around 450 skipped tests for all of Parrot, which is a
sizable number when compared to the total number of tests in the system.

This concerns me, for at least three reasons.

First, the ratio of skips to non-skips is much higher than with any other
project I know (except possibly Pugs, which does something I consider worse
in principle, but manages to avoid other problems).

Second, I haven't noticed anyone exploring whether those skips are truly
necessary anymore. With the tests I fixed, it only took a few minutes of
debugging and a bit of massaging to bring crufty old syntax in line with the
current state of Parrot to improve things. However, there are a couple of
other skipped tests I didn't fix because I couldn't tell what the point was.
(See the skipped test in t/library/streams.t, for example, which looks to me
like it wants to assume that you can store an instance method in a Sub PMC,
invoke it, and magically get the proper invocant somehow.)

Maybe I'm wrong and people are checking these occasionally. I doubt that,
however, because, third, some of the syntax and features are woefully out of
date. Skipped tests don't even run, so there's no chance for Parrot to look
at the old syntax and say "What are you doing? That opcode doesn't even
exist anymore!"

That makes it much more difficult to *fix* those tests, unless you have a
degree in comparative Parrot archaeology.

Granted, some of those skips are there from those dark times before
Parrot::Test supported TODO, but I don't even remember *when* I added that
feature, so it's been around for a while.

Thus, I suggest that we collectively unskip all of the tests that might
possibly pass on all platforms, tighten the skip conditions to their minimum
necessities, use TODO instead of SKIP absolutely everywhere possible, and
delete the tests that don't matter anymore.

(Adding more tests is also a good way to improve the ratio. Pretend I said
that too.)

I've just checked in a CAGE task for this.

-- c

[Chip Salzenberg]

On Sat, Aug 19, 2006 at 07:17:03PM -0700, chromatic wrote:
> Thus, I suggest that we collectively unskip all of the tests that might
> possibly pass on all platforms, tighten the skip conditions to their minimum
> necessities, use TODO instead of SKIP absolutely everywhere possible, and
> delete the tests that don't matter anymore.

I'm Chip Salzenberg, and I approve of this message.

> I've just checked in a CAGE task for this.

Thanks.
--
Chip Salzenberg <ch...@pobox.com>