Re: 700 tests for PGE
Patrick R. Michaud
> Here are 700 tests for pge, autoconverted from 're_tests' file, from
> perl5 source.
>
> If there are any significant errors in these tests, please tell, and I
> can correct the script. Original perl5-tests are in comments so it's
> quite easy to see if there has been an error in converting.
I'm getting errors from the test script itself, there are lines such as
# 65: ^ abc y $&
p6rule_like('abc', '^', qr/0: <\Q\E @/, 're_tests 35/$0 (#35)');
# 66: $ abc y $&
p6rule_like('abc', '$', qr/0: <\Q\E @/, 're_tests 36/$0 (#36)');
which cause my test script to produce...
$ perl t/harness t/p6rules/re_tests.t
t/p6rules/re_tests....syntax error at t/p6rules/re_tests.t line 106, near "0: <\Q\E @"
syntax error at t/p6rules/re_tests.t line 108, near "0: <\Q\E @"
syntax error at t/p6rules/re_tests.t line 236, near "1: <\Q\E @"
syntax error at t/p6rules/re_tests.t line 253, near "1: <\Q\E @"
syntax error at t/p6rules/re_tests.t line 321, near "0: <\Q\E @"
syntax error at t/p6rules/re_tests.t line 354, near "1: <\Q\E @"
syntax error at t/p6rules/re_tests.t line 428, near "1: <\Q\E @"
syntax error at t/p6rules/re_tests.t line 432, near "2: <\Q\E @"
syntax error at t/p6rules/re_tests.t line 444, near "2: <\Q\E @"
syntax error at t/p6rules/re_tests.t line 579, near "0: <\Q\E @"
t/p6rules/re_tests.t has too many errors.
I suppose the problem could be with my perl installation (5.8.0, RH9)
Also, the PGE test harness itself isn't sacred -- if there are functions
or other features we could add to make this sort of testing easier, I'm
all for it.
Pm
Markus Laire
>
> I'm getting errors from the test script itself, there are lines such as
>
> # 65: ^ abc y $&
> p6rule_like('abc', '^', qr/0: <\Q\E @/, 're_tests 35/$0 (#35)');
> # 66: $ abc y $&
> p6rule_like('abc', '$', qr/0: <\Q\E @/, 're_tests 36/$0 (#36)');
>
> which cause my test script to produce...
>
> $ perl t/harness t/p6rules/re_tests.t
> t/p6rules/re_tests....syntax error at t/p6rules/re_tests.t line 106, near "0: <\Q\E @"
> syntax error at t/p6rules/re_tests.t line 108, near "0: <\Q\E @"
> syntax error at t/p6rules/re_tests.t line 236, near "1: <\Q\E @"
>
> I suppose the problem could be with my perl installation (5.8.0, RH9)
I don't get those errors with 5.8.3, SuSe 9.1 Personal
If those are caused by '@/' which seems a bit like array, then it would
be easily fixed by addind space between @ and /.
Other 'special' syntax which I invented was for tests which only test a
position, not value:
# 131: ()ef def y $-[0] 1
# 132: ()ef def y $+[0] 3 # SKIP
p6rule_like('def', '()ef', qr/0: <.* @ 1>/, 're_tests 97/$0 (#98)');
# 133: ()ef def y $-[1] 1
# 134: ()ef def y $+[1] 1 # SKIP
p6rule_like('def', '()ef', qr/1: <.* @ 1>/, 're_tests 98/$1 (#99)');
> Also, the PGE test harness itself isn't sacred -- if there are
> functions or other features we could add to make this sort of testing
> easier, I'm all for it.
Currently the script is written in such a way that it's trivial to
output tests in different syntax, should that be changed.
I was wondering if it would make sense to add the original 're_tests'
file to parrot distribution, with a script which autogenerates
're_tests.t' from it. This way it would be possible to update the script
if testing-format is changed, or if some bigger mistakes are seen.
Of course small errors in 're_tests.t' file could be fixed manually, but
if testing-format it changed, then those changes would be lost when file
was autogenerated again.
--
Markus Laire
<Jam. 1:5-6>
Patrick R. Michaud
>
> I was wondering if it would make sense to add the original 're_tests'
> file to parrot distribution, with a script which autogenerates
> 're_tests.t' from it. This way it would be possible to update the script
> if testing-format is changed, or if some bigger mistakes are seen.
>
> Of course small errors in 're_tests.t' file could be fixed manually, but
> if testing-format it changed, then those changes would be lost when file
> was autogenerated again.
After thinking about this a bit...
Although it's probably worthwhile for us to find a good way to encode
p6rule tests in a way that allows us to go to other formats in the future,
I'm not so sure we buy a lot in being able to do that for the original
're_tests'. As I'll note in my next message, the semantics of regexs
and rules are just different enough that I don't think that
a one-to-one correspondence for 're_tests' is worth trying to
preserve or code around. At some point I think it's just better to
use the autogenerating script to create re_tests.t, manually fix re_tests.t
for the things that don't convert, and then go from there.
Plus, I suspect that p5's 're_tests' is mature enough that it doesn't
change often, and the places where it might change in the future are
esoteric enough that it'd be easier to add+translate those changes manually
than to update the conversion script.
Note that for this I'm only talking about ease of building and maintaining
the p6rules test suite. Having a script to automatically convert any
arbitrary p5 regex to its p6 equivalent *is* important, useful, and
definitely worth pursuing, but I'm not sure it's worth the effort (at
this stage) to worry about keeping the p6rules test suite consistent
with that. And the p5->p6 regex converter will probably deserves its
own test suite, once p6rules is sufficiently advanced.
Pm
Patrick R. Michaud
> On Mon, Dec 20, 2004 at 01:59:57PM +0200, Markus Laire wrote:
> >
> > I was wondering if it would make sense to add the original 're_tests'
> > file to parrot distribution, with a script which autogenerates
> > 're_tests.t' from it. This way it would be possible to update the script
> > if testing-format is changed, or if some bigger mistakes are seen.
I forgot to mention in my previous mail -- if (when?) we change the
testing format we're probably going to want a way to convert all of
the other tests in p6rules (i.e., the ones that didn't come from
're_tests'). It will probably be just as easy, if not easier, to
convert the manually edited re_tests.t file using whatever converter
we come up with then.
Just a thought.
Pm
Jared Rhine
Markus> I was wondering if it would make sense to add the original
Markus> 're_tests' file to parrot distribution, with a script which
Markus> autogenerates 're_tests.t' from it.
To me, it certainly would. I would note, however, that my
implementation of something like this approach was rejected prior to
the release of PGE as too complicated, despite a working code base.
Please sync up carefully with Patrick and Dan on your approach to this
automation to avoid rework. It may be that it's still preferred to
just maintain your "cooker" script outside the tree and check in only
the results.
(Briefly, my approach was to have a "p5_re_tests.t" script, which
would generate 1 PGE test for each line in the re_tests source file,
but do that testing on-the-fly, not "cooked", so no regeneration would
ever be necessary.)
The argument for avoiding this approach didn't really make sense to me
at the time, as I saw a future with thousands of PGE tests. I
believed (and still believe) that given more time, it would be
realized that automating the infrastructure so people didn't have to
type in or port repetitive regexp/rule tests would be beneficial and
result in more, better tests.
If you're on a roll, Markus, I'd like to point you to:
svn://software.wordzoo.com/p6ge-test-harness/trunk
where I gathered lots of additional tests (including perl 5's re_tests
file) that were intended to be ported to PGE. The codebase is
abortive at best, as I simply dropped it when Patrick indicated he was
going a different direction. So I've since applied my open-source
time in other directions, but you are welcome to browse this obsolete
codebase for ideas.
"Better to be of a rare breed than a long line." -- TDK