Re: CURRENT (r249438): (devel/libiconv)./unistd.h:686:5: error: invalid token at start of a preprocessor expression : #if @GNULIB_EUIDACCESS@

[Jan Beich]

Dimitry Andric <d...@FreeBSD.org> writes:

> On Apr 16, 2013, at 00:42, Jan Beich <jbe...@tormail.org> wrote:
>
>> "O. Hartmann" <ohar...@zedat.fu-berlin.de> writes:
>>> ./unistd.h:694:5: error: invalid token at start of a preprocessor
>>> expression
>>> #if @GNULIB_EUIDACCESS@
>>> ^
>>> 1 error generated.
>>
>> Maybe -O3 overoptimizes regex in libc e.g.,
>>
>> $ echo '#if @GNULIB_EUIDACCESS@' | sed 's/@GNULIB_EUIDACCESS@/0/'
>> #if @GNULIB_EUIDACCESS@
>>
>> $ echo 'aaaaaaaaaaaaaaaaxxxaaaa' | sed 's/aaaaaaaaaaaaxxxaaaa//'
>> aaaaaaaaaaaaaaaaxxxaaaa
>
> How did you arrive at this result?

1/ chroot into poudriere jail for /head amd64
2/ echo CFLAGS+=-O3 >> /etc/make.conf
3/ make -j2 (in /usr/src/lib/libc)
4/ prepend LD_LIBRARY_PATH=. before sed(1)
5/ rebuild regcomp.o, regcomp.So with -O2 to confirm

> I have recompiled both libc and sed
> with -O3, but it works just fine here.

> Maybe -march=native is the clue, so which kind of CPU do you have?

sse2 is available on amd64 even without -march=. But I can't reproduce
the issue on i386 even with -march=native.

> To see what CPU llvm detects, try:
>
> tblgen -version | grep CPU

Host CPU: penryn

>
> Note that -O3 turns on clang's vectorizer, so you might have run into an
> optimizer bug, or some kind of undefined behavior which now falls over.

My luck is poor even with -O2 e.g., firefox20 crash on stackoverflow.com

[New LWP 101222]
[New Thread 801b02400 (LWP 101222)]
[New Thread 81060e800 (LWP 101372 StreamTrans #1)]
[New Thread 810ee8800 (LWP 101373 DOM Worker)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 801b02400 (LWP 101222)]
PodCopy<JS::Value> (dst=<optimized out>, nelem=<optimized out>,
src=<optimized out>, dst=<optimized out>, src=<optimized out>,
nelem=<optimized out>) at ../../../js/src/jsutil.h:238
238 PodAssign(dst, src);
(gdb) bt
#0 PodCopy<JS::Value> (dst=<optimized out>, nelem=<optimized out>,
src=<optimized out>, dst=<optimized out>, src=<optimized out>,
nelem=<optimized out>) at ../../../js/src/jsutil.h:238
#1 getCallFrame (cx=<optimized out>, flags=<optimized out>, fun=<optimized out>,
report=(unknown: 2266652928), this=<optimized out>, args=..., script=...)
at ../../../js/src/vm/Stack-inl.h:454
#2 getFixupFrame (cx=<optimized out>, initial=<optimized out>,
fun=<optimized out>, ncode=<optimized out>, dummy=0, report=<optimized out>,
this=<optimized out>, cx=<optimized out>, report=<optimized out>, args=...,
fun=<optimized out>, script=..., ncode=<optimized out>,
initial=<optimized out>, stackLimit=<optimized out>)
at ../../../js/src/vm/Stack-inl.h:513
#3 js::mjit::stubs::FixupArity (f=..., nactual=4294945312)
at /wrkdirs/usr/ports/www/firefox/work/mozilla-release/js/src/methodjit/InvokeHelpers.cpp:213
#4 0x00000008019c48c2 in ?? ()
#5 0x00000008019b56b8 in ?? ()
#6 0x0000000000000001 in ?? ()
#7 0x0000000000000000 in ?? ()

>
> -Dimitry
_______________________________________________
freebsd...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-curre...@freebsd.org"

[Dimitry Andric]

On Apr 17, 2013, at 07:31, Jan Beich <jbe...@tormail.org> wrote:
> Dimitry Andric <d...@FreeBSD.org> writes:
> On Apr 16, 2013, at 00:42, Jan Beich <jbe...@tormail.org> wrote:

...

>>> Maybe -O3 overoptimizes regex in libc e.g.,
>>>
>>> $ echo '#if @GNULIB_EUIDACCESS@' | sed 's/@GNULIB_EUIDACCESS@/0/'
>>> #if @GNULIB_EUIDACCESS@
>>>
>>> $ echo 'aaaaaaaaaaaaaaaaxxxaaaa' | sed 's/aaaaaaaaaaaaxxxaaaa//'
>>> aaaaaaaaaaaaaaaaxxxaaaa
>>
>> How did you arrive at this result?
>
> 1/ chroot into poudriere jail for /head amd64
> 2/ echo CFLAGS+=-O3 >> /etc/make.conf
> 3/ make -j2 (in /usr/src/lib/libc)
> 4/ prepend LD_LIBRARY_PATH=. before sed(1)
> 5/ rebuild regcomp.o, regcomp.So with -O2 to confirm

I have been able to reproduce this on amd64, with -O3, but not on i386.
It seems regcomp() is either miscompiled at -O3, or it contains some bug
triggered only by the vectorizer. I am still investigating.

This is a completely different issue. Is there any way you can reduce
the test case? Or maybe upstream has already fixed it?

[Jung-uk Kim]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2013-04-17 06:07:47 -0400, Dimitry Andric wrote:
> On Apr 17, 2013, at 07:31, Jan Beich <jbe...@tormail.org> wrote:
>> Dimitry Andric <d...@FreeBSD.org> writes: On Apr 16, 2013, at
>> 00:42, Jan Beich <jbe...@tormail.org> wrote:
> ...
>>>> Maybe -O3 overoptimizes regex in libc e.g.,
>>>>
>>>> $ echo '#if @GNULIB_EUIDACCESS@' | sed
>>>> 's/@GNULIB_EUIDACCESS@/0/' #if @GNULIB_EUIDACCESS@
>>>>
>>>> $ echo 'aaaaaaaaaaaaaaaaxxxaaaa' | sed
>>>> 's/aaaaaaaaaaaaxxxaaaa//' aaaaaaaaaaaaaaaaxxxaaaa
>>>
>>> How did you arrive at this result?
>>
>> 1/ chroot into poudriere jail for /head amd64 2/ echo CFLAGS+=-O3
>> >> /etc/make.conf 3/ make -j2 (in /usr/src/lib/libc) 4/ prepend
>> LD_LIBRARY_PATH=. before sed(1) 5/ rebuild regcomp.o, regcomp.So
>> with -O2 to confirm
>
> I have been able to reproduce this on amd64, with -O3, but not on
> i386. It seems regcomp() is either miscompiled at -O3, or it
> contains some bug triggered only by the vectorizer. I am still
> investigating.

...

With "-fno-vectorize", this problem doesn't seem to happen.

FYI,

Jung-uk Kim
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (FreeBSD)

iQEcBAEBAgAGBQJRbvUHAAoJECXpabHZMqHOekEIAIml2te9436LzTFsr794y82E
Vmytl95H+vW9Nj0qK5X/DkB/0MSepL5FZqKF5CSNTXFoNJoVFewYRIH/H5oICSpZ
jfS4evF9i2mEDOScTyC/XaucvcVWupLE9Kf7FHEk5YIhDMs4r4nzwMFGkzffEqPK
yLkV/Cpc8xjvi28OuXd1KaPIcX3S8Z9vEmWPyljtseRv9WlC5gT44fSz18hmqYmv
fWSiML4YKKkDRAPOCy/Shpf5QUcygOul7Jz8RiDBx3O4R5goGW8Ee8Napn7UulSL
nAXTHy8dcSbiAqqPKeXhmZGPCotj++P9s3jEvunOxL7lvrdjfy3WtGedcp02ia8=
=jwYS
-----END PGP SIGNATURE-----

[Dimitry Andric]

On Apr 17, 2013, at 21:16, Jung-uk Kim <jk...@FreeBSD.org> wrote:
> On 2013-04-17 06:07:47 -0400, Dimitry Andric wrote:
>> On Apr 17, 2013, at 07:31, Jan Beich <jbe...@tormail.org> wrote:
>>> Dimitry Andric <d...@FreeBSD.org> writes: On Apr 16, 2013, at
>>> 00:42, Jan Beich <jbe...@tormail.org> wrote:
>> ...
>>>>> Maybe -O3 overoptimizes regex in libc e.g.,
>>>>>
>>>>> $ echo '#if @GNULIB_EUIDACCESS@' | sed
>>>>> 's/@GNULIB_EUIDACCESS@/0/' #if @GNULIB_EUIDACCESS@
>>>>>
>>>>> $ echo 'aaaaaaaaaaaaaaaaxxxaaaa' | sed
>>>>> 's/aaaaaaaaaaaaxxxaaaa//' aaaaaaaaaaaaaaaaxxxaaaa
>>>>
>>>> How did you arrive at this result?
>>>
>>> 1/ chroot into poudriere jail for /head amd64 2/ echo CFLAGS+=-O3
>>>>> /etc/make.conf 3/ make -j2 (in /usr/src/lib/libc) 4/ prepend
>>> LD_LIBRARY_PATH=. before sed(1) 5/ rebuild regcomp.o, regcomp.So
>>> with -O2 to confirm
>>
>> I have been able to reproduce this on amd64, with -O3, but not on
>> i386. It seems regcomp() is either miscompiled at -O3, or it
>> contains some bug triggered only by the vectorizer. I am still
>> investigating.
> ...
>
> With "-fno-vectorize", this problem doesn't seem to happen.

After some more investigation, I submitted LLVM PR 15830, with a test
case reduced from our regcomp.c. It got diagnosed and fixed pretty
quickly, and I have pulled in the fix in r249817. Please verify it, by
rebuilding libc with your original -O3 settings; the sed commands listed
above should now work correctly.

[O. Hartmann]

Both the ports devel/libiconv and graphics/jasper do not show the
reported error anymore after this patch!

Thanks for helping fixing this that fast!

With regards,
Oliver