What's needed for a new languages/t/*?

1 view
Skip to first unread message

Amir Karger

unread,
Aug 4, 2005, 8:52:39 AM8/4/05
to perl6-i...@perl.org
I'm about to commit an updated version of leo's Z-code-to-PIR
translator. I'm wondering what I should do about t.

I have a test script that runs 85 tests (and will run many more once I
write more opcodes. Luckily, I developed it already when I was doing
plotz). I could easily modify it to output "ok n" and "not ok" with a
comment about what went wrong. However, because it's a big Z-code
file, it would be hard to write a Test::Simple script that calls ok()
85 times. In quickly looking at languages/* I saw that there seem to
be a number of different ways of doing tests, some of which use
Parrot::Test.

So if all I want to do is, essentially, perl -e 'chdir languages/Z and
system("parrot z3.imc t/test.z3")' and let the script print out a
bunch of (not )?ok's for Test::Harness to read, what should I do in my
t directory? (Also, how do I make sure it'll find parrot before make
install has been done?) I didn't see a languages.pod with this info,
but maybe I just wasn't looking in the right place.

-Amir Karger

Will Coleda

unread,
Aug 4, 2005, 2:46:41 PM8/4/05
to Amir Karger, perl6-i...@perl.org
Excellent questions. Perhaps I can whip up a languages.pod once 0.2.3
is out the door, based on partcl and the current state of a few other
languages out there.

Right now, the "unified language testing harness", such as it is,
would rather you had a script called "harness" that took a --files
option to figure out what .t files to run. The .t files don't have to
be perl scripts, but it helps: we don't have a lot of requirements
for parrot at the moment, so for portability perl is the best. (but
you could make it a shell script, or whatever. Just be careful with
$Test::Harness::Switches).

Finding parrot before make install is done is somewhat tricky: this
is another reason to use the builtin Parrot::Test modules. Right now,
several languages provide a language specific class this can dispatch
to, e.g. "Parrot::Test::Tcl". {{ this implementation is a little
muddy at the moment and could probably use some cleanup }}

But, for now, if you have a .t script that works, I would say commit
it (after the code freeze, of course); we can have you be standalone
for now and integrate you into the unified lang testing later. Many
of the existing languages are in various states of disrepair in re:
testing, so it won't be evil.

amirk...@gmail.com

unread,
Aug 4, 2005, 3:29:33 PM8/4/05
to

Will Coleda wrote:
> Right now, the "unified language testing harness", such as it is,
> would rather you had a script called "harness" that took a --files
> option to figure out what .t files to run. The .t files don't have to
> be perl scripts, but it helps: we don't have a lot of requirements
> for parrot at the moment, so for portability perl is the best. (but
> you could make it a shell script, or whatever. Just be careful with
> $Test::Harness::Switches).

Actually, I'm fine with making it a Perl script. The problem is that I
want that script to run my Zcode program, which will print 'ok N' o
stdout to be read by Test::Harness (which languages/testall is using)
rather than having a script that uses Test::Simple (or I guess
Parrot::Test) to call ok() which prints 'ok'.

> Finding parrot before make install is done is somewhat tricky: this
> is another reason to use the builtin Parrot::Test modules. Right now,
> several languages provide a language specific class this can dispatch
> to, e.g. "Parrot::Test::Tcl". {{ this implementation is a little
> muddy at the moment and could probably use some cleanup }}
>
> But, for now, if you have a .t script that works, I would say commit
> it (after the code freeze, of course); we can have you be standalone
> for now and integrate you into the unified lang testing later. Many
> of the existing languages are in various states of disrepair in re:
> testing, so it won't be evil.

I guess the problem is, I'm looking for a compromise. AFAICT, if we can
find a way to find parrot before install, then I *will* be able to be a
part of the unified lang testing, even though I'm not using
Parrot::Test. Think it could work?

-Amir

Bernhard Schmalhofer

unread,
Aug 4, 2005, 3:02:53 PM8/4/05
to Amir Karger, perl6-i...@perl.org
Amir Karger schrieb:

>I have a test script that runs 85 tests (and will run many more once I
>write more opcodes. Luckily, I developed it already when I was doing
>plotz). I could easily modify it to output "ok n" and "not ok" with a
>comment about what went wrong. However, because it's a big Z-code
>file, it would be hard to write a Test::Simple script that calls ok()
>85 times. In quickly looking at languages/* I saw that there seem to
>be a number of different ways of doing tests, some of which use
>Parrot::Test.
>
>So if all I want to do is, essentially, perl -e 'chdir languages/Z and
>system("parrot z3.imc t/test.z3")' and let the script print out a
>bunch of (not )?ok's for Test::Harness to read, what should I do in my
>t directory?
>

AFAIK there are no strict testing requirements for language implementations.
I think that the most important thing is to try to be nice.

There are three things that I want to suggest:

i. Z/t/z3.t should be a Perl5 script that prints output in Test Anything
Protcol.
So put your 'system("parrot z3.imc t/test.z3")' into Z/t/z3.t

ii. Z/t/harness should behave like most other 't/harness'. It executes
the t/*.t files and prints
a harness report of the output.

iii. 'Z/t/harness --files' returns the list of test files.

You could also try to put 'Z' into @unified_testable_languages of
'languages/testall' and add support for Z in
'config/gen/makefiles/languages.in'. Calling 'make languages-test' could
then test 'Z' along the other languages.

> (Also, how do I make sure it'll find parrot before make
>install has been done?)
>

I would expect that an install parrot is only found when the
executable 'parrot' is in $PATH.
Before installation I usually use $FindBin::Bin for building up the
relevant paths.

>I didn't see a languages.pod with this info,
>but maybe I just wasn't looking in the right place.
>
>

I think there in no such documentation. New insight could be added to
'docs/compiler_faq.pod'.

CU, Bernhard

Chromatic

unread,
Aug 5, 2005, 3:37:27 AM8/5/05
to Amir Karger, perl6-i...@perl.org
On Thu, 2005-08-04 at 08:52 -0400, Amir Karger wrote:

> I'm about to commit an updated version of leo's Z-code-to-PIR
> translator. I'm wondering what I should do about t.

I hope to have a variant of Test::More written in pure PIR in the near
future (though it probably requires the new calling conventions code to
reach the trunk) and a port of Parrot::Test shortly after that. That
ought to let you write tests either in PIR or, with the appropriate
calling code, in the language of your choice.

-- c

Amir Karger

unread,
Aug 5, 2005, 12:54:03 PM8/5/05
to

Bernhard Schmalhofer wrote:
> Amir Karger schrieb:
>
> >I have a test script that runs 85 tests (and will run many more once I
> >write more opcodes.
> >
> >So if all I want to do is, essentially, perl -e 'chdir languages/Z and
> >system("parrot z3.imc t/test.z3")' and let the script print out a
> >bunch of (not )?ok's for Test::Harness to read, what should I do in my
> >t directory?
> >
> AFAIK there are no strict testing requirements for language implementations.
> I think that the most important thing is to try to be nice.

Or, as a wise comedian once said, "Be Excellent to each other".

> There are three things that I want to suggest:
>
> i. Z/t/z3.t should be a Perl5 script that prints output in Test Anything
> Protcol.
> So put your 'system("parrot z3.imc t/test.z3")' into Z/t/z3.t

Done (ish).

> ii. Z/t/harness should behave like most other 't/harness'. It executes
> the t/*.t files and prints
> a harness report of the output.

I think I can copy someone else's pretty much verbatim

> iii. 'Z/t/harness --files' returns the list of test files.

Looks like I can perl -pe 's/urm/Zcode/' urm/t/harness >
Zcode/t/harness

> You could also try to put 'Z' into @unified_testable_languages of
> 'languages/testall' and add support for Z in
> 'config/gen/makefiles/languages.in'. Calling 'make languages-test' could
> then test 'Z' along the other languages.

I think I'll try that once I can get make test to work.

> > (Also, how do I make sure it'll find parrot before make
> >install has been done?)
> >
> I would expect that an install parrot is only found when the
> executable 'parrot' is in $PATH.
> Before installation I usually use $FindBin::Bin for building up the
> relevant paths.

OK. I see others say PARROT=../../parrot

Thanks,

-Amir

Amir Karger

unread,
Aug 6, 2005, 10:22:10 PM8/6/05
to perl6-i...@perl.org
[Argh! Posted to google groups again!]

Bernhard Schmalhofer wrote:
> Amir Karger schrieb:

> >I have a test script that runs 85 tests (and will run many more once I
> >write more opcodes.

> >So if all I want to do is, essentially, perl -e 'chdir languages/Z and
> >system("parrot z3.imc t/test.z3")' and let the script print out a
> >bunch of (not )?ok's for Test::Harness to read, what should I do in my
> >t directory?


> AFAIK there are no strict testing requirements for language implementations.
> I think that the most important thing is to try to be nice.

Or, as a wise comedian once said, "Be Excellent to each other".


> There are three things that I want to suggest:

> i. Z/t/z3.t should be a Perl5 script that prints output in Test Anything
> Protcol.
> So put your 'system("parrot z3.imc t/test.z3")' into Z/t/z3.t

Done (ish).


> ii. Z/t/harness should behave like most other 't/harness'. It executes
> the t/*.t files and prints
> a harness report of the output.

I think I can copy someone else's pretty much verbatim

> iii. 'Z/t/harness --files' returns the list of test files.

Looks like I can perl -pe 's/urm/Zcode/' urm/t/harness >
Zcode/t/harness

> You could also try to put 'Z' into @unified_testable_languages of
> 'languages/testall' and add support for Z in

> 'config/gen/makefiles/language­s.in'. Calling 'make languages-test' could

> then test 'Z' along the other languages.

I think I'll try that once I can get make test to work.

> > (Also, how do I make sure it'll find parrot before make
> >install has been done?)

> I would expect that an install parrot is only found when the
> executable 'parrot' is in $PATH.
> Before installation I usually use $FindBin::Bin for building up the
> relevant paths.

OK. I see others say PARROT=../../parrot

Thanks,


-Amir

Reply all
Reply to author
Forward
0 new messages