It's only in my local svk repository. I'll push it so others can work
on the bug (and try it out on multiple platforms). I temporarily
added a test file language/punie/t/problematic.t that isolates the
failing test (makes it easier to filter through the debugging output).
Allison
But, I've encountered two major problems. On darwin, I can't finish
past_node.t, first parrot takes over 100 megs of ram, then perl(5.8.7)
wants 180 megs. On freebsd, it's actually worse, but more confusing.
It fails with past_*.t and post_*.t. But past.t is fine. It's
essentially a lot of out of memory errors(yet swap space isn't touched,
and there's free ram at that moment). Darwin essentially starts
"stalling" when freebsd just gives up on allocating more memory. Line
numbers aren't given in the print out, except for src/memory.c line 92.
Like others, problematic.t seems to runs okay on my system
(Linux x86_64). :-( Maybe I can get access to a platform on
which it fails...?
Pm
This is a known bug. As far as we can tell so far, it's not the Punie
code, but Parrot::Test causing the problem. On both platforms, try
running the generated .pir files in languages/punie/t directly (e.g.
t/past_node_2.pir). Everyone who's tried it so far has reported that
they have no memory errors running the test code directly, and only
get errors running the code through the test harness.
Allison
Long ago we had an OS X box available to developers. Is that still
around? If not, I can temporarily set up a laptop on an externally
accessible IP.
Allison
> It's only in my local svk repository. I'll push it so others can work
> on the bug (and try it out on multiple platforms). I temporarily added
> a test file language/punie/t/problematic.t that isolates the failing
> test (makes it easier to filter through the debugging output).
Both on OS/X darwin and x86/linux 'make test' as well as
./parrot -p languages/punie/punie.pbc languages/punie/t/problematic_1.p1
are succeeding here. (The latter is created by setting env
POSTMORTEM=1, if someone wonders).
> Allison
leo
The error is gone here too now. Not sure if it's Leo's fix or unrelated.
Allison
> Apparently I have a 267 megabyte past_node_5.out file... And if
> past_op_2.pir and past_val_2.pir were printed to a file, I imagine
> it'd
> do the same(printing a lot of spaces). Seems to be more than just
> Parrot::Test for me.
Could you send me the first 50 lines of the past_node_5.out file?
That'll help me figure out where the extra bytes are coming from. I
just ran it and got a file 11 lines long and less than 1k.
past_op_2.out and past_val_2.out are each 6 lines long.
Also, run:
parrot past_node_5.pir > past_node_5.out
and let me know if you still end up with a 267 megabyte file.
Allison
[languages/punie/t] jisom% xxd past_node_5.out | head
0000000: 2020 2020 2020 2020 2020 2020 2020 2020
0000010: 2020 2020 2020 2020 2020 2020 2020 2020
0000020: 2020 2020 2020 2020 2020 2020 2020 2020
0000030: 2020 2020 2020 2020 2020 2020 2020 2020
0000040: 2020 2020 2020 2020 2020 2020 2020 2020
0000050: 2020 2020 2020 2020 2020 2020 2020 2020
0000060: 2020 2020 2020 2020 2020 2020 2020 2020
0000070: 2020 2020 2020 2020 2020 2020 2020 2020
0000080: 2020 2020 2020 2020 2020 2020 2020 2020
0000090: 2020 2020 2020 2020 2020 2020 2020 2020
[languages/punie/t] jisom%
I don't think there's a line ending. It's the same if I run it
directly. Got up to 20 megs in a couple seconds. Since that's
obviously rather useless, here's a trace.
0 load_bytecode "languages/punie/lib/"
0 newclass P0, "PAST::Node" - P0=PMCNULL,
3 addattribute P0, "source" -
P0=Class=PAST::Node:PMC(0x50ba50),
6 addattribute P0, "pos" - P0=Class=PAST::Node:PMC(0x50ba50),
9 addattribute P0, "children" -
P0=Class=PAST::Node:PMC(0x50ba50),
12 set_returns PMC_C[6]
14 returncc
2 new P1, "PAST::Node" - P1=PMCNULL,
18 new P1, 38 - P1=PMCNULL,
21 new P2, 31 - P2=PMCNULL,
24 new P3, 38 - P3=PMCNULL,
27 setattribute P0, "source", P1 -
P0=Object(PAST::Node)=PMC(0x50ba68), , P1=PerlUndef=PMC(0x50b888)
31 setattribute P0, "pos", P2 -
P0=Object(PAST::Node)=PMC(0x50ba68), , P2=Integer=PMC(0x50b870: 0)
35 setattribute P0, "children", P3 -
P0=Object(PAST::Node)=PMC(0x50ba68), , P3=PerlUndef=PMC(0x50b858)
39 set_returns PMC_C[6]
41 returncc
5 new P0, "PAST::Node" - P0=PMCNULL,
18 new P1, 38 - P1=PMCNULL,
21 new P2, 31 - P2=PMCNULL,
24 new P3, 38 - P3=PMCNULL,
27 setattribute P0, "source", P1 -
P0=Object(PAST::Node)=PMC(0x50b8a0), , P1=PerlUndef=PMC(0x50b828)
31 setattribute P0, "pos", P2 -
P0=Object(PAST::Node)=PMC(0x50b8a0), , P2=Integer=PMC(0x50b810: 0)
35 setattribute P0, "children", P3 -
P0=Object(PAST::Node)=PMC(0x50b8a0), , P3=PerlUndef=PMC(0x50b7f8)
39 set_returns PMC_C[6]
41 returncc
8 set_args PMC_C[7] (4), P0, "b", 9, P2 - ,
P0=Object(PAST::Node)=PMC(0x50b8a0), , , P2=PMCNULL
14 get_results PMC_C[12]
16 callmethodcc P0, "set_node" -
P0=Object(PAST::Node)=PMC(0x50b8a0),
42 get_params PMC_C[14] (4), P0, S0, I0, P1 - , P0=PMCNULL,
, I0=2178912, P1=PMCNULL
48 getattribute P2, P0, "source" - P2=PMCNULL,
P0=Object(PAST::Node)=PMC(0x50b8a0),
52 set P2, S0 - P2=PerlUndef=PMC(0x50b828), S0="b"
55 getattribute P2, P0, "pos" - P2=PerlString=PMC(0x50b828),
P0=Object(PAST::Node)=PMC(0x50b8a0),
59 set P2, I0 - P2=Integer=PMC(0x50b810: 0), I0=9
62 setattribute P0, "children", P1 -
P0=Object(PAST::Node)=PMC(0x50b8a0), , P1=PMCNULL
66 set_returns PMC_C[6]
DOD
GC
68 returncc
19 new P2, 46 - P2=PMCNULL,
22 push P2, P0 - P2=PerlArray=PMC(0x50b840),
P0=Object(PAST::Node)=PMC(0x50b8a0)
25 set_args PMC_C[7] (4), P1, "foo", 42, P2 - ,
P1=Object(PAST::Node)=PMC(0x50ba68), , , P2=PerlArray=PMC(0x50b840)
31 get_results PMC_C[12]
33 callmethodcc P1, "set_node" -
P1=Object(PAST::Node)=PMC(0x50ba68),
42 get_params PMC_C[14] (4), P0, S0, I0, P1 - , P0=PMCNULL,
, I0=9, P1=PMCNULL
48 getattribute P2, P0, "source" - P2=PMCNULL,
P0=Object(PAST::Node)=PMC(0x50ba68),
52 set P2, S0 - P2=PerlUndef=PMC(0x50b888), S0="foo"
55 getattribute P2, P0, "pos" - P2=PerlString=PMC(0x50b888),
P0=Object(PAST::Node)=PMC(0x50ba68),
59 set P2, I0 - P2=Integer=PMC(0x50b870: 0), I0=42
62 setattribute P0, "children", P1 -
P0=Object(PAST::Node)=PMC(0x50ba68), , P1=PerlArray=PMC(0x50b840)
66 set_returns PMC_C[6]
68 returncc
36 set_args PMC_C[8] (1), P1 - ,
P1=Object(PAST::Node)=PMC(0x50ba68)
39 get_results PMC_C[12]
41 callmethodcc P1, "dump" -
P1=Object(PAST::Node)=PMC(0x50ba68),
102 get_params PMC_C[29] (2), P0, I0 - , P0=PMCNULL,
I0=5289976
106 repeat S0, " ", I0 - , , I0=5289976
110 add I0, 1 - I0=5289976,
DOD
GC
113 typeof S1, P0 - , P0=Object(PAST::Node)=PMC(0x50ba68)
116 print S0 - S0=" "
Okay, the problem seems to be that it's getting garbage values in
"optional" integer parameters if the parameters aren't passed in. So,
the PIR code is calling the 'dump' method with no parameters:
node1.dump()
But inside the 'dump' method, get_params is retrieving a value of
5289976 into the integer parameter 'level' (I0).
.param int level
Then, 'level' is used as a multiplier for the number of spaces that
should be used to indent the displayed content of the AST node:
indent = repeat " ", level
That explains all the spaces. It's indenting each line of the output
by 21,159,904 spaces (5289976 * 4).
The question is, where's the garbage value in the parameter list
coming from? Anyone have any suggestions?
(Still curious if you tried running it outside the Parrot::Test
framework. Mostly to determine if this is the same bug as was
reported before or a new bug.)
Allison
> On Feb 4, 2006, at 16:51, Joshua Isom via RT wrote:
>> 41 callmethodcc P1, "dump" -
>> P1=Object(PAST::Node)=PMC(0x50ba68),
>> 102 get_params PMC_C[29] (2), P0, I0 - , P0=PMCNULL,
>> I0=5289976
>> 106 repeat S0, " ", I0 - , , I0=5289976
> Okay, the problem seems to be that it's getting garbage values in
> "optional" integer parameters if the parameters aren't passed in. So,
> the PIR code is calling the 'dump' method with no parameters:
.sub "dump" method
.param int level
The level argument isn't optional at all. Turning on argument count
checks would prevent such errors.
It has to be:
.sub "dump" method
.param int level :optional
> Allison
leo
Okay, thanks, changed. Joshua, let me know how it goes. Particle,
could you check and see if this fixes your problem as well?
What's the difference between :optional and :opt_flag? I found a few
lines of documentation on these once I knew what to grep for, but
that's all.
Marking optional parameters is a good move forward. But... retrieving
garbage values into parameters that aren't passed an argument is not
a good design for a stable system. It leads to thorny bugs like this
one. (I've been looking for it for over a month, but could never
reproduce it on any of my machines.) Can you prevent it from
retrieving garbage values, even when :optional isn't set?
Allison
>> ... Turning on argument count
>> checks would prevent such errors.
> What's the difference between :optional and :opt_flag? I found a few
> lines of documentation on these once I knew what to grep for, but
> that's all.
Seed pdd03. :optional is the argument. :opt_flag is 1/0 if the argument
was passed or not.
> Marking optional parameters is a good move forward. But... retrieving
> garbage values into parameters that aren't passed an argument is not a
> good design for a stable system.
Of course. The plan is to turn on argument/return value count checks on.
See some recent discusson on p6i, e.g. "Param count checks" ;)
You can turn it on manually with the errorson opcode too, see
t/op/calling.t for examples.
> Allison
leo
D:\usr\local\parrot\trunk\languages\punie>prove t/
t\base_cond....ok
t\base_if......ok
t\base_lex.....ok
t\base_pat.....ok
t\base_term....ok
t\io_print.....ok
t\op_goto......ok
t\past.........ok
t\past_node....ok
t\past_op......ok
t\past_val.....ok
t\post.........ok
t\post_node....ok
t\post_op......ok
t\post_val.....ok
All tests successful.
Files=15, Tests=41, 14 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU)
this change made languages-smoke possible on win32 again.
i'll submit one as soon as i can.
~jerry
Ah, need a case insensitive grep for that one. (pdd03 doesn't contain
the strings "opt" or "optional" anywhere, only "OPT" and "OPTIONAL".)
>> Marking optional parameters is a good move forward. But... retrieving
>> garbage values into parameters that aren't passed an argument is
>> not a
>> good design for a stable system.
>
> Of course. The plan is to turn on argument/return value count
> checks on.
> See some recent discusson on p6i, e.g. "Param count checks" ;)
On by default? Or on-all-the-time-you-can't-turn-it-off-even-if-you-
want-to?
If it's the former, the problem still needs to be solved.
> You can turn it on manually with the errorson opcode too, see
> t/op/calling.t for examples.
I'll use this to ferret out the places where Punie and TGE use
optional parameters. But I'm more concerned about the overall design
question than I am about Punie. (Punie is an exercise to explore what
we want out of the AST interface and find potential obstacles to
other language implementations.)
Allison
I too had seen this memory problem before on Solaris/SPARC, but I'm
pretty sure I saw it even when running t/past_node_5.pir directly.
However, trying again today, I'm happy to report that that particular
problem seems to be gone. Of course an awful lot of the tests still
fail, but I don't know if that's expected or not.
Failed 6/15 test scripts, 60.00% okay. 16/41 subtests failed, 60.98% okay.
Failed Test Stat Wstat Total Fail Failed List of Failed
-------------------------------------------------------------------------------
t/past.t 6 1536 7 6 85.71% 2-7
t/past_op.t 2 512 2 2 100.00% 1-2
t/past_val.t 2 512 2 2 100.00% 1-2
t/post.t 2 512 3 2 66.67% 2-3
t/post_op.t 2 512 2 2 100.00% 1-2
t/post_val.t 2 512 2 2 100.00% 1-2
*** Error code 2
make: Fatal error: Command failed for target `test'
--
Andy Dougherty doug...@lafayette.edu
Excellent.
> Of course an awful lot of the tests still
> fail, but I don't know if that's expected or not.
All of the test should be passing (at least, they are on other
platforms). Could you send me more details on the failing tests? Some
of the generated .out and .pir files would be a good start (running
with POSTMORTEM=1).
Thanks!
Allison
I don't pretend to remotely understand what the code is attempting to
do, let alone how it's failing, but the failures vaguely seem to be fairly
similar. It looks like 'node.set_node' doesn't actually end up doing
anything. So for t/past_2.pir, for example, we have
.sub _main
load_bytecode 'languages/punie/lib/PAST.pir'
.local pmc node
node = new 'PAST::Code'
$P0 = new PerlString
$P0 = 'bar'
$P1 = new PerlArray
push $P1, $P0
node.set_node('foo', 42, $P1)
$P1 = getattribute node, 'source'
print $P1
print "\n"
$P1 = getattribute node, 'pos'
print $P1
print "\n"
$P2 = getattribute node, 'children'
$P3 = $P2[0]
print $P3
print "\n"
.return ()
.end
and for t/past_2.out, we have
1
1
bar
The other one that looks significantly different is t/past_op_1.pir. The
pir file is
.sub _main
load_bytecode 'languages/punie/lib/PAST.pir'
.local pmc node
node = new 'PAST::Op'
node.set_node('foo', 42, 'bar')
$P1 = getattribute node, 'source'
print $P1
print "\n"
$P1 = getattribute node, 'pos'
print $P1
print "\n"
$P1 = getattribute node, 'op'
print $P1
print "\n"
.return ()
.end
and the output file is
Null PMC access in get_string()
current instr.: '_main' pc 20 (/home/doughera/src/parrot/parrot-andy/languages/punie/t/past_op_1.pir:7)
Here's the full output of 'make test'. It accurately reflects the .out
files (well, I didn't exhaustively check them all, but the ones I did
check matched up.)
perl5.6 -Ilib t/harness t/*.t
t/base_cond....ok
t/base_if......ok
t/base_lex.....ok
t/base_pat.....ok
t/base_term....ok
t/io_print.....ok
t/op_goto......ok
t/past.........
# Failed test (t/past.t at line 44)
# got: '1
# 1
# bar
# '
# expected: 'foo
# 42
# bar
# '
# Failed test (t/past.t at line 44)
# got: '1
# 1
# bar
# '
# expected: 'foo
# 42
# bar
# '
# Failed test (t/past.t at line 44)
# got: '1
# 1
# bar
# '
# expected: 'foo
# 42
# bar
# '
# Failed test (t/past.t at line 44)
# got: '1
# 1
# bar
# '
# expected: 'foo
# 42
# bar
# '
# Failed test (t/past.t at line 44)
# got: '1
# 1
# bar
# '
# expected: 'foo
# 42
# bar
# '
# Failed test (t/past.t at line 44)
# got: '1
# 1
# bar
# '
# expected: 'foo
# 42
# bar
# '
# Looks like you failed 6 tests of 7.
dubious
Test returned status 6 (wstat 1536, 0x600)
DIED. FAILED tests 2-7
Failed 6/7 tests, 14.29% okay
t/past_node....ok
t/past_op......
# Failed test (t/past_op.t at line 7)
# got: 'Null PMC access in get_string()
# current instr.: '_main' pc 20 (/home/doughera/src/parrot/parrot-andy/languages/punie/t/past_op_1.pir:7)
# '
# expected: 'foo
# 42
# bar
# '
# './parrot "/home/doughera/src/parrot/parrot-andy/languages/punie/t/past_op_1.pir"' failed with exit code 43
# Failed test (t/past_op.t at line 30)
# got: '<PAST::Op> => {
# 'source' => undef,
# 'pos' => undef,
# 'op' => 'bar',
# 'children' => []
# }
# '
# expected: '<PAST::Op> => {
# 'source' => 'foo',
# 'pos' => '42',
# 'op' => 'bar',
# 'children' => []
# }
# '
# Looks like you failed 2 tests of 2.
dubious
Test returned status 2 (wstat 512, 0x200)
DIED. FAILED tests 1-2
Failed 2/2 tests, 0.00% okay
t/past_val.....
# Failed test (t/past_val.t at line 7)
# got: '42
# 42
# bar
# '
# expected: 'foo
# 42
# bar
# '
# Failed test (t/past_val.t at line 30)
# got: '<PAST::Val> => {
# 'source' => '42',
# 'pos' => '42',
# 'value' => 'bar',
# 'valtype' => undef,
# }
# '
# expected: '<PAST::Val> => {
# 'source' => 'foo',
# 'pos' => '42',
# 'value' => 'bar',
# 'valtype' => undef,
# }
# '
# Looks like you failed 2 tests of 2.
dubious
Test returned status 2 (wstat 512, 0x200)
DIED. FAILED tests 1-2
Failed 2/2 tests, 0.00% okay
t/post.........
# Failed test (t/post.t at line 44)
# got: '1
# 1
# bar
# '
# expected: 'foo
# 42
# bar
# '
# Failed test (t/post.t at line 44)
# got: '1
# 1
# bar
# '
# expected: 'foo
# 42
# bar
# '
# Looks like you failed 2 tests of 3.
dubious
Test returned status 2 (wstat 512, 0x200)
DIED. FAILED tests 2-3
Failed 2/3 tests, 33.33% okay
t/post_node....ok
t/post_op......
# Failed test (t/post_op.t at line 7)
# got: 'Null PMC access in get_string()
# current instr.: '_main' pc 20 (/home/doughera/src/parrot/parrot-andy/languages/punie/t/post_op_1.pir:7)
# '
# expected: 'foo
# 42
# bar
# '
# './parrot "/home/doughera/src/parrot/parrot-andy/languages/punie/t/post_op_1.pir"' failed with exit code 43
# Failed test (t/post_op.t at line 30)
# got: '<POST::Op> => {
# 'source' => undef,
# 'pos' => undef,
# 'op' => 'bar',
# 'children' => []
# }
# '
# expected: '<POST::Op> => {
# 'source' => 'foo',
# 'pos' => '42',
# 'op' => 'bar',
# 'children' => []
# }
# '
# Looks like you failed 2 tests of 2.
dubious
Test returned status 2 (wstat 512, 0x200)
DIED. FAILED tests 1-2
Failed 2/2 tests, 0.00% okay
t/post_val.....
# Failed test (t/post_val.t at line 7)
# got: '42
# 42
# bar
# '
# expected: 'foo
# 42
# bar
# '
# Failed test (t/post_val.t at line 30)
# got: '<POST::Val> => {
# 'source' => '42',
# 'pos' => '42',
# 'value' => 'bar',
# 'valtype' => undef,
# }
# '
# expected: '<POST::Val> => {
# 'source' => 'foo',
# 'pos' => '42',
# 'value' => 'bar',
# 'valtype' => undef,
# }
# '
# Looks like you failed 2 tests of 2.
dubious
Test returned status 2 (wstat 512, 0x200)
DIED. FAILED tests 1-2
Failed 2/2 tests, 0.00% okay
Allison
-J
--
> [doughera - Mon Feb 13 09:10:50 2006]:
> andy/languages/punie/t/post_op_1.pir:7)
> # '
> # expected: 'foo
> # 42
> # bar
> # '
> # './parrot "/home/doughera/src/parrot/parrot-