Cygwin issues may not be ICU related
Joshua Gatcomb
While there are definately ICU issues on Cygwin, I
have gotten it to link to parrot both statically and
dynamically several different ways. The problem is
that the resulting parrot.exe coredumps upon
execution. I had always assumed that the problem was
related to ICU since it was immediately preceded by
the linking.
This morning it got me wondering if that was the case
at all. Is it possible that parrot is coredumping for
some other reason? The parrot.exe is around 3MB. I
know that it isn't dying immediately upon execution
because if the data file isn't in the directory it is
supposed to be, invoking parrot.exe will come back and
say "Can't find data file aparently in '<dir name>'"
instead of coredumping.
Is there any trickery I could do to rule out ICU or
determine if it is actually the problem? Does ANYONE
have it working on Cygwin? Am I alone on this island?
Cheers
__________________________________
Do you Yahoo!?
Win a $20,000 Career Makeover at Yahoo! HotJobs
http://hotjobs.sweepstakes.yahoo.com/careermakeover
Jeff Clites
> Is it possible that parrot is coredumping for
> some other reason? The parrot.exe is around 3MB. I
> know that it isn't dying immediately upon execution
> because if the data file isn't in the directory it is
> supposed to be, invoking parrot.exe will come back and
> say "Can't find data file aparently in '<dir name>'"
> instead of coredumping.
>
> Is there any trickery I could do to rule out ICU or
> determine if it is actually the problem?
If you run it in a debugger (gdb or whatever), you should be able to
see where it's crashing.
JEff
Joshua Gatcomb
--- Jeff Clites <jcl...@mac.com> wrote:
> If you run it in a debugger (gdb or whatever), you
> should be able to
> see where it's crashing.
Ok, after spending about 5 minutes figuring out what
gdb was and how to use it (did I mention I was
clueless) it looks like it isn't ICU at all!!!!
Program received signal SIGSEGV, Segmentation fault.
0x00419f6c in new_pmc_header (interpreter=0x100d1d48,
flags=1024) at src/headers.c:251
251 *((Dead_PObj*)pmc)->arena_dod_flag_ptr
|=
Ok, so I naively tried --gc=libc and --gc=malloc and
it didn't seem to help.
So in retrospect, it appears that there are a number
of ways getting ICU playing nice on Cygwin but until
this other issue is resolved they don't matter. If
anyone is interested in a complete list of recipes I
have used then let me know.
Leopold Toetsch
> Program received signal SIGSEGV, Segmentation fault.
> 0x00419f6c in new_pmc_header (interpreter=0x100d1d48,
> flags=1024) at src/headers.c:251
> 251 *((Dead_PObj*)pmc)->arena_dod_flag_ptr
> |=
(gdb) bt
(gdb) p pmc
/gdb) p *((Dead_PObj*)pmc)
would be good.
Which program did you run?
Are all segfaulting?
Does running the program with -G option help?
> Ok, so I naively tried --gc=libc and --gc=malloc and
> it didn't seem to help.
Above error is only possible, when ARENA_DOD_FLAGS is turned on. This
switch relies on the presence of a working memalign function. The
C<--gc> option isn't related to this. But you can turn off
ARENA_DOD_FLAGS in include/parrot/pobj.h line 25.
> Cheers
leo
Joshua Gatcomb
--- Leopold Toetsch <l...@toetsch.at> wrote:
> (gdb) bt
> (gdb) p pmc
> /gdb) p *((Dead_PObj*)pmc)
> would be good.
flags=1024) at src/headers.c:251
251 *((Dead_PObj*)pmc)->arena_dod_flag_ptr
|=
(gdb) bt
(interpreter=0x100d1d68, flags=1024) at
src/headers.c:251
#1 0x0042e600 in get_new_pmc_header
(interpreter=0x100d1d68, base_type=34, flags=1024) at
src/pmc.c:110
#2 0x0042e830 in pmc_new_noinit
(interpreter=0x100d1d68, base_type=34) at
src/pmc.c:187
#3 0x0042e47b in pmc_new (interpreter=0x100d1d68,
base_type=34) at src/pmc.c:69
#4 0x00423977 in enter_nci_method
(interpreter=0x100d1d68, type=26, func=0x481670,
name=0x53f8d9 "thread1", proto=0x53f8d4 "vIOP") at
src/interpreter.c:2668
#5 0x0053fb75 in Parrot_ParrotInterpreter_class_init
(interp=0x100d1d68, entry=26)
at classes/parrotinterpreter.c:707
#6 0x00537494 in Parrot_initialize_core_pmcs
(interp=0x100d1d68) at src/core_pmcs.c:92
#7 0x00464693 in init_world (interpreter=0x100d1d68)
at src/global_setup.c:75
#8 0x00412f5d in Parrot_init (interpreter=0x100d1d68)
at src/embed.c:84
#9 0x00422464 in make_interpreter (parent=0x0,
flags=NO_FLAGS) at src/interpreter.c:1867
#10 0x00412f19 in Parrot_new (parent=0x0) at
src/embed.c:47
#11 0x0040292b in main (argc=1, argv=0x100d10f8) at
imcc/main.c:409
(gdb) p pmc
$1 = (PMC *) 0x1810020
(gdb) p *((Dead_PObj*)pmc)
$2 = {free_list_ptr = 0x1810000,
object_buflen_dont_use = 0, object_flags_dont_use = 0,
arena_dod_flag_ptr = 0x0, flag_shift = 0}
(gdb)
>
> Which program did you run?
> Are all segfaulting?
> Does running the program with -G option help?
gdb ./parrot
The -G option is of no help
FYI - the make never finishes. Right after linking
parrot (which is why I thought this was exclusively an
ICU issue), it bombs out on config_lib.pasm
> But you can
> turn off
> ARENA_DOD_FLAGS in include/parrot/pobj.h line 25.
Will try that and post results - hopefully in the next
20 minutes as I have to go pick up someone from the
airport.
Joshua Gatcomb
> is turned on. This
> switch relies on the presence of a working memalign
> function. The
> C<--gc> option isn't related to this. But you can
> turn off
> ARENA_DOD_FLAGS in include/parrot/pobj.h line 25.
>
Well, at least the make finishes without problems. I
am running make test right now but doubt it will
finish before I have to leave.
I know you can't see it, but this is me doing my happy
dance.
Is this a work-around or the actual fix? I ask
because I plan on drafting up some notes on the
various ways to get ICU working on Cygwin as well.
Cheers - Joshua
Leopold Toetsch
> --- Leopold Toetsch <l...@toetsch.at> wrote:
>> (gdb) bt
>> (gdb) p pmc
>> /gdb) p *((Dead_PObj*)pmc)
>> would be good.
> Program received signal SIGSEGV, Segmentation fault.
> #5 0x0053fb75 in Parrot_ParrotInterpreter_class_init
Early in program startup.
> (gdb) p *((Dead_PObj*)pmc)
> $2 = {free_list_ptr = 0x1810000,
> object_buflen_dont_use = 0, object_flags_dont_use = 0,
> arena_dod_flag_ptr = 0x0, flag_shift = 0}
^^^
Somethings is *really* wrong here. I don't know exactly what's happening
here, but I assume that the memalign function is buggy. To nail that
further down:
1) Determine the used memalign function:
$ grep memalign lib/Parrot/Config.pm
'memalign' => 'memalign',
It could be 'posix_memalign' too.
2) Copy either config/auto/memalign/ test_c.in or test_c2.in somewhere
as e.g. align.c
3) Replace
#include <${malloc_header}>
with <malloc.h>
or whatever is the correct header file. see e.g. "man memalign"
4) compile and run it:
$ cc -Wall -g align.c -o align && ./align
ok
5) Now the easy part: Vary the alignment of the function and the test,
if actually the lo bits are zero.
BTW Kind of this "stress" test is very welcome for the config system.
Google around, if you can find signs of a broken memalign function on
your system.
> Cheers
leo
Leopold Toetsch
> --- Leopold Toetsch <l...@toetsch.at> wrote:
>> turn off
>> ARENA_DOD_FLAGS in include/parrot/pobj.h line 25.
> Is this a work-around or the actual fix?
More the former.
> because I plan on drafting up some notes on the
> various ways to get ICU working on Cygwin as well.
Good. Thanks.
> Cheers - Joshua
leo
Joshua Gatcomb
It is still unclear to me if turning off
ARENA_DOD_FLAGS in include/parrot/pobj.h is required
yet or not. Leo suspects a broken memalign function
but since parrot was working prior to April 7th and
nothing changed except parrot, I am not too sure. As
soon as this posts I will start digging around there.
You will know if you need to do this if parrot
coredumps every time you use it ;-)
Using the ICU that is bundled with parrot:
1. make realclean
2. cvs update -dP
3. add link => 'c++', to config/init/hints/cygwin.pl
4. remove --disable-shared from config/gen/icu.pl
5. add <parrot root dir>/blib/lib to your PATH env
variable
6. make
It will bomb out when linking to ICU - that's ok
7. chmod 755 blib/lib/*
8. c++ -o parrot.exe -L/usr/local/lib -g imcc/main.o
blib/lib/libparrot.a blib/lib/libicuuc.dll
blib/lib/libicudata.dll -lcrypt
You could mess with the config/gen/icu.pl and
config/gen/makefiles/root.in but why bother
9. make
FYI - the make process builds ICU twice which I doubt
is supposed to happen.
I have very little "free" time, planning a wedding an
all, but if I get a chance I will fix
the make process for this.
Using a system ICU built from source > 2.6
1. make realclean
2. cvs update -dP
3. add link => 'c++', to config/init/hints/cygwin.pl
4. Depending on the system version of ICU and the
--datadir option, you may have to:
Modify $icudatadir = '' to the proper path
(usually <system prefix>/share/icu/<version>
if there is no .dat file in that directory you can
skip this step
5. perl Configure.pl
--icuheaders='<system prefix>/include' (usally
/usr/local/include)
--icushared='-L<system prefix>/lib -l<uc library>
-l<data library'
if using /usr/local, the default, the -L is
not required
1st library will usually be -licuuc, but if
built statically and a newer version, -lsicuuc
2nd library will usually be -licudata, but if
built with a new version, -licudt and if static
-lsicudt
4. If it is not already, add <system prefix>/lib to
the PATH env variable
5. chmod 755 <system prefix>/lib/*icu*
6. make
Using a system ICU built from a binary download
(icu-2.8-Win32_msvc7.zip)
1. make realclean
2. cvs update -dP
3. add link => 'c++', to config/init/hints/cygwin.pl
4. If it is not already, add <system prefix>/bin to
the PATH env variable
5. Download the msvcr70.dll to <system prefix>/bin
http://www.dll-files.com/dllindex/dll-files.shtml?msvcr70
6. chmod 755 <system prefix>/bin/*.dll
7. perl COnfigure.pl
--icuheaders='<system prefix>/include'
--icushared='-L<system prefix>/bin -licuuc28
-licudt28l'
8. make
These instructions still may contain errors but there
are no huge ommissions. If I ever get some "free
time",
I intend to provide some patches that makes this much
less of a PITA. If anyone else feels like doing this
I
certainly would have no hard feelings.
Finally, if you haven't done much development on
Cygwin and you get a "Unable to remap" error about
some
.dll, then you will need to run the rebaseall utility.
I do not believe this has anything to do with parrot
but I can't be sure since I needed to do it for
IO::Socket::SSL to work properly much earlier on.
Cheers
Joshua Gatcomb
a.k.a. Limbic~Region
Joshua Gatcomb
> Somethings is *really* wrong here. I don't know
> exactly what's happening
> here, but I assume that the memalign function is
> buggy. To nail that
> further down:
>
> 1) Determine the used memalign function:
>
> $ grep memalign lib/Parrot/Config.pm
> 'memalign' => 'memalign',
>
> It could be 'posix_memalign' too.
> ...
> 4) compile and run it:
> $ cc -Wall -g align.c -o align && ./align
> ok
yes - it kicked back "ok" not "nix"
> 5) Now the easy part: Vary the alignment of the
> function and the test,
> if actually the lo bits are zero.
I didn't understand. I assume you wanted me to mess
around if I didn't get "ok"?
> Google around, if you can find signs of a broken
> memalign function on
> your system.
I am not exactly sure what to be looking for.
Everything, including parrot, was working fine prior
to April 7th. I returned from vacation and parrot was
no longer working. Nothing on the system changed
except parrot.
> leo
Joshua Gatcomb
Extend test #12 was disabled because it was hanging on
Cygwin and you had to kill the process in another
window for the test suite to complete. This is now
happening on #13 as well.
Failed Test Stat Wstat Total Fail Failed
List of Failed
-------------------------------------------------------------------------------
t/pmc/freeze.t 1 256 13 1 7.69%
13
t/pmc/managedstruct.t 3 768 5 3 60.00%
2-4
t/pmc/nci.t 16 4096 33 16 48.48% 8
14-26 31 33
t/pmc/object-meths.t 1 256 19 1 5.26%
19
t/pmc/sub.t 3 768 73 3 4.11%
60-62
t/pmc/tqueue.t 1 256 1 1 100.00% 1
t/src/extend.t 1 256 13 1 7.69%
13
t/src/io.t 2 512 20 2 10.00% 4
7
2 tests and 58 subtests skipped.
Failed 8/102 test scripts, 92.16% okay. 28/1522
subtests failed, 98.16% okay.
Cal Henderson
http://code.iamcal.com/parrot/sha1.imc
works on a few tested 32 bit platforms.
fails on 64 bits, due to what looks like a bug with
constants and/or sprintf (the low words of the final
ints are correct, but trying to clear the high words
causes wierdness - thomason is writing a test case
for this).
--cal
Leopold Toetsch
> #4 0x00423977 in enter_nci_method
> (interpreter=0x100d1d68, type=26, func=0x481670,
> name=0x53f8d9 "thread1", proto=0x53f8d4 "vIOP") at
Does the recent change related to NCI cure the problem?
leo
Leopold Toetsch
> I am not sure where to go from here. Any suggestions?
Ok, here is a sample debugging session:
$ cat hello.pasm
print "hello\n"
end
$ parrot hello.pasm
hello
$ gdb parrot
...
(gdb) b new_pmc_header
(gdb) r hello.pasm
Breakpoint 1, new_pmc_header (interpreter=0x82e5668, flags=1024)
at src/headers.c:244
244 pool = flags & PObj_constant_FLAG ?
(gdb) n
(gdb) n
249 if (flags & PObj_is_PMC_EXT_FLAG) {
(gdb) p *pool
...
(gdb) p *pool->last_Arena
...
(gdb) n
251 *((Dead_PObj*)pmc)->arena_dod_flag_ptr |=
(gdb) n
256 pmc->pmc_ext = new_pmc_ext(interpreter);
(gdb)
When it passes the first few times without problems then you could disable
the breakpoint:
(gdb) dis 1
and continue until the program fails:
(gdb) c
Then (p)rint again *pool, pmc, *pmc, *pool->last_Arena.
PMCs are taken from the free_list which points to the end of the arena:
(gdb) p pmc
$9 = (PMC *) 0x40b0c018
(gdb) c
(gdb) n
(gdb) n
(gdb) p pmc
the next one is at old - 0x18:
$10 = (PMC *) 0x40b0c000
(gdb) p sizeof(PMC)
$11 = 24
(gdb) p /x sizeof(PMC)
$12 = 0x18
(gdb)
(gdb) p *pool
$15 = {last_Arena = 0x40b00000, object_size = 24,
objects_per_alloc = 21843, total_objects = 2048, num_free_objects = 2046,
skip = 0, replenish_level = 614, free_list = 0x40b0bfe8 ...
(gdb) p *(PMC*)pool->free_list
...
(gdb) p *(Dead_PObj*)pmc
$17 = {free_list_ptr = 0x40b0bfe8, object_buflen_dont_use = 0,
object_flags_dont_use = 0, arena_dod_flag_ptr = 0x82f0f04,
flag_shift = 24}
The arena_dod_flag_ptr must point somewhere into arena->dod_flags.
Actual adresses will of course differ.
leo
Leopold Toetsch
> Program received signal SIGSEGV, Segmentation fault.
> 0x00419f6c in new_pmc_header (interpreter=0x100d1d68,
> flags=1024) at src/headers.c:251
> 251 *((Dead_PObj*)pmc)->arena_dod_flag_ptr
One more idea: When you set a breakpoint at C<new_pmc_header> and (s)tep
into the next function call, you should have arreived in
C<get_free_object_df> A very similar line like the segfaulting one, is
used there, to reset the on_free_list flag of the PMC.
I can't imagine, how C<arena_dod_flag_ptr could be valid in
C<get_free_object_df> and NULL some instructions later.
BTW: Which compiler and version are use using?
leo
Joshua Gatcomb
I assume you wanted this done with the ARENA_DOD_FLAGS
enabled. With them enabled it blows up (not passing)
at the last step you had me do:
(gdb) b new_pmc_header
Breakpoint 1 at 0x419ed4: file src/headers.c, line
244.
(gdb) r hello.pasm
Starting program: /perl/parrot/parrot.exe hello.pasm
Breakpoint 1, new_pmc_header (interpreter=0x100d1d20,
flags=1024) at src/headers.c:244
244 pool = flags & PObj_constant_FLAG ?
(gdb) n
247 pmc = pool->get_free_object(interpreter,
pool);
(gdb) n
249 if (flags & PObj_is_PMC_EXT_FLAG) {
(gdb) p *pool
$1 = {last_Arena = 0x10200000, object_size = 32,
objects_per_alloc = 16382, total_objects = 2048,
num_free_objects = 2047, skip = 0, replenish_level =
614, free_list = 0x10210000, align_1 = 0,
add_free_object = 0x4700f0 <add_free_object>,
get_free_object = 0x470110 <get_free_object>,
alloc_objects = 0x470580 <alloc_objects>,
more_objects = 0x470050 <more_traceable_objects>,
mem_pool = 0x0, start_arena_memory = 270532672,
end_arena_memory = 271056896, name = 0x41a6cd "pmc"}
(gdb) p *pool->last_Arena
$2 = {used = 2048, total_objects = 16382, object_size
= 32, dod_flags = 0x100fc668, pool = 0x100f2778,
live_objects = 0, prev = 0x0, next = 0x0,
start_objects = 0x10200040}
(gdb) n
251 *((Dead_PObj*)pmc)->arena_dod_flag_ptr
|=
(gdb) n
Program received signal SIGSEGV, Segmentation fault.
0x00419f6c in new_pmc_header (interpreter=0x100d1d20,
flags=1024) at src/headers.c:251
251 *((Dead_PObj*)pmc)->arena_dod_flag_ptr
|=
I assume this is blowing up earlier than you expected?
Leopold Toetsch
> (gdb) p *pool
> $1 = {last_Arena = 0x10200000, object_size = 32,
> objects_per_alloc = 16382, total_objects = 2048,
> num_free_objects = 2047, skip = 0, replenish_level =
> 614, free_list = 0x10210000, align_1 = 0,
> add_free_object = 0x4700f0 <add_free_object>,
> get_free_object = 0x470110 <get_free_object>,
That's the problem. The darn thing is just calling the wrong funtion.
Or I'm dumb:
On your system sizeof(Dead_PObj) must be greater then sizeof(PMC). How that?
Seems that you got sizeof(UINTVAL) == 8 ???
Can you please supply the output of:
(gdb) p sizeof(PMC)
24
(gdb) p sizeof(Dead_PObj)
20
(gdb) p sizeof(void *)
4
(gdb) p sizeof(UINTVAL)
4
(gdb) p sizeof(UINTVAL*)
4
thanks for helping to track it down,
leo
Joshua Gatcomb
(gdb) p sizeof(PMC)
$1 = 32
(gdb) p sizeof(Dead_PObj)
$2 = 40
(gdb) p sizeof(void *)
$3 = 4
(gdb) p sizeof(UINTVAL)
$4 = 8
(gdb) p sizeof(UINTVAL*)
$5 = 4
> thanks for helping to track it down,
> leo
>
I don't know how much help I am, but I am able bodied
and willing (albeit feeble minded).
Leopold Toetsch
>> Seems that you got sizeof(UINTVAL) == 8 ???
> (gdb) p sizeof(PMC)
> $1 = 32
> (gdb) p sizeof(Dead_PObj)
> $2 = 40
> (gdb) p sizeof(void *)
> $3 = 4
> (gdb) p sizeof(UINTVAL)
> $4 = 8
Aargh yes. So here we go:
$ make realclean
$ perl Configure.pl --intval=long --opcode=long
Then please verify that INTVAL_SIZE in include/parrot/config.h is 4.
If INTVAL_SIZE is still 8, then use "int" instead of "long".
Is your perl compiled with "long long" IVs?
$ perl -V
...
ivtype=long, ivsize=4
Anyway, the config system *should not* use perl5's types. Patches
welcome. The same problem is, if perl5 is compiled with "long double"s.
leo
Joshua Gatcomb
--- Leopold Toetsch <l...@toetsch.at> wrote:
> Aargh yes. So here we go:
>
> $ make realclean
> $ perl Configure.pl --intval=long --opcode=long
>
This was the ticket!!!! Thanks. For my own
edification, can you explain to me what in the code I
isolated that changed on the 15th accounts for this?
>
> Is your perl compiled with "long long" IVs?
> $ perl -V
> ...
> ivtype=long, ivsize=4
>
ivtype='long long', ivsize=8
Now this is 5.8.4 built myself from the source
recently, the 5.8.2 version that ships with Cygwin may
be different.
> Anyway, the config system *should not* use perl5's
> types. Patches
> welcome. The same problem is, if perl5 is compiled
> with "long double"s.
>
> leo
I think this is something I could do if I knew what it
*should be* using/getting the values from.
Thanks again
Joshua Gatcomb
$ cvs update -dP
$ perl Configure.pl --intval=long --opcode=long
-licudt'
$ make
$ make test
....
All tests successful, 2 tests and 57 subtests skipped.
Files=102, Tests=1522, 395 wallclock secs (230.74 cusr
+ 149.40 csys = 380.13 CPU)
Cheers!!!!
Joshua Gatcomb
lightening fast in comparison to what it was only a
few short hours ago. What a difference after:
--intval=long --opcode=long
$ time parrot primes2.pasm
N primes calculated to 5000 is 670
last is: 4999
real 0m4.148s
user 0m3.905s
sys 0m0.045s
This was about 6s prior to Leo's Configure.pl
suggestion.
parrot -j <anything>
coredumped previously
$ time parrot -j primes2.pasm
N primes calculated to 5000 is 670
last is: 4999
real 0m2.182s
user 0m2.155s
sys 0m0.015s
I am very very very happy. Thanks Leo, George R, and
anyone/everyone else that helped.
Cheers
Leopold Toetsch
> --- Leopold Toetsch <l...@toetsch.at> wrote:
>> $ make realclean
>> $ perl Configure.pl --intval=long --opcode=long
> This was the ticket!!!! Thanks.
Great.
> For my own
> edification, can you explain to me what in the code I
> isolated that changed on the 15th accounts for this?
It was the assumption that a PMC is not smaller then a Dead_PObj
structure, which obviously is wrong. Other code paths that deal with
Dead_PObj do check the size. But at the place, where you got the
segfault, there wasn't such a check because of that assumption.
The Dead_PObj structure had another problem, which is fixed now: I
thought that pobj->buflen is an UINTVAL. Actually it's a size_t, which
might not be the same length.
BTW: STRINGs have an UINTVAL strlen and a size_t buflen, which might not
be very useful.
> ivtype='long long', ivsize=8
[ get rid of perl5 types in config ]
> I think this is something I could do if I knew what it
> *should be* using/getting the values from.
Good question. But normally we should have:
sizeof(void *) == sizeof(opcode) == sizeof(INTVAL)
The sizeof(void *) is the "natural" size of the system. When we allow
sizeof(INTVAL) == 8 on 32bit system, we need a big code cleanup.
U?INTVALs are used all over the place just as plain ints.
This was discussed earlier several times, but isn't really solved IMHO.
> Thanks again
Welcome.
> Joshua Gatcomb
leo