grsecurity interfering with the parrot JIT/build
Christian Jaeger
I'm using Linux/x86 with the Grsecurity.org patch applied, which is
enforcing page execution permissions (PAX) unless you turn them off
on a binary using the "chpax" userspace tool.
This means - unless you turn it off - an executable that is executing
code in a page which is not marked as executable is killed by the
kernel, since this could be some attacker trying to execute code he
had put into the process by abusing some buffer overflow or the like.
As I said, you can turn it off by modifying the executable with the
chpax utility; but "perl Configure.pl" is compiling and executing
binaries without leaving the user a chance to run this utility. The
result is that parrot is (somewhat "correctly") built without JIT.
I've tried to find out where I could put a call to "chpax -s test"
into the configuration process, but given up so far, that looks quite
complicated.
The correct solution would be to mark the respective memory pages
executable. I guess this would be done by calling mprotect(2). I'd
welcome tips on whether this would be easy to do, or how - if I get
enough confidence to get this work without too much work I'll do it
and provide a patch.
Here's the output of perl Configure.pl --optimize:
...
Determining if your C library has a working S_ISREG...................done.
Determining architecture, OS and JIT
capability.......................p = 0x8050000 PAGE_SIZE = 4096
(0x1000)
failure: Permission denied
done.
Running CPU specific stuff............................................done.
...
and here what the kernel log is telling:
Oct 15 21:19:05 elvis kernel: PAX: From 129.132.126.110: execution
attempt in: <anonymous mapping>, 08050000-08051000 00006000
Oct 15 21:19:05 elvis kernel: PAX: From 129.132.126.110: terminating
task: /COMPILE/parrot-0.1.1/test(test):23401, uid/euid: 1000/1000,
PC: 08050000, SP: 5c994f3c
Oct 15 21:19:05 elvis kernel: PAX: bytes at PC: b8 01 00 00 00 c3 00
00 00 00 00 00 f9 0f 02 00 00 00 00 00
(.. repeating same thing several times.)
Christian.
Leopold Toetsch
> Hello
> I'm using Linux/x86 with the Grsecurity.org patch applied, which is
> enforcing page execution permissions (PAX) unless you turn them off
> on a binary using the "chpax" userspace tool.
[ ... ]
> The correct solution would be to mark the respective memory pages
> executable. I guess this would be done by calling mprotect(2).
We should already do that - or better the hooks are there.
> welcome tips on whether this would be easy to do, or how - if I get
> enough confidence to get this work without too much work I'll do it
> and provide a patch.
Ok. There is a test in config/auto/jit/test_exec_openbsd.in, which is
run from jit.pl. This should result in $PConfig{has_exec_protect} = 1.
You can verify that by e.g.
$ perl Configure.pl --verbose-step=JIT
> Here's the output of perl Configure.pl --optimize:
> ...
> Determining if your C library has a working S_ISREG...................done.
> Determining architecture, OS and JIT
> capability.......................p = 0x8050000 PAGE_SIZE = 4096
> (0x1000)
> failure: Permission denied
But it seems that the test has the $osname in it, and the test isn't
running. And the next test (for fcomip) will fail too. That needs either
duplicating the test or running it for bsd and linux.
Anyway, JIT memory is allocate by mem_alloc_executable(), which should
do the right thing i.e. call mprotect().
> Christian.
Thanks,
leo
Christian Jaeger
>Anyway, JIT memOk. There is a test in config/auto/jit/test_exec_openbsd.in,
Ok, I've looked at the test_exec_linux source, and tried it out
separately; it's clear what happens to me now: I've enabled the
following GrSecurity option, which makes the mprotect system call
fail with a permission error - the test even outputs this as the line
"failure: Permission denied".
CONFIG_GRKERNSEC_PAX_MPROTECT
Enabling this option will prevent programs from
- changing the executable status of memory pages that were
not originally created as executable,
- making read-only executable pages writable again,
- creating executable pages from anonymous memory.
You should say Y here to complete the protection provided by
the enforcement of non-executable pages.
NOTE: you can use the 'chpax' utility to control this
feature on a per file basis. chpax is available at
<http://pax.grsecurity.net>
I've not yet hat the time to reboot that machine with a kernel w/o
that option, so I'm not sure it'd work with the other grsecurity/PAX
features. (In any case, the problem remains that one cannot build the
JIT if GRKERNSEC_PAX_MPROTECT is enabled because the user has no
chance running chpax; it should be possible to switch PAX off for
some path using GrSecurity's ACL rules but I don't know if that reads
the inodes into the kernel as LIDS does, if so then it's a real
conflict.)
But I've built parrot now on another machine which is not running
GrSecurity. This machine is running a not up-to-date Debian Sarge
(gcc 3.3.2) and has a Intel PentiumIII 1Ghz CPU.
I did perl Configure.pl --optimize. I still get a parrot that is
about 4 times slower than perl5, and for which none of the
-f,-g,-P,-S,-C,-j options does seem to make a significant difference
in speed. (see data below).
I checked the Makefile, it has:
CFLAGS = -D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN
-I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O3
-DDISABLE_GC_DEBUG=1 -DNDEBUG -g -Dan_Sugalski -Larry -Wall
-Wstrict-prototypes -Wmissing-prototypes -Winline -Wshadow
-Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings
-Waggregate-return -Winline -W -Wno-unused -Wsign-compare
-Wformat-nonliteral -Wformat-security -Wpacked
-Wdisabled-optimization -mno-accumulate-outgoing-args -Wno-shadow
-falign-functions=16 -I./include -I./blib/include -DHAS_JIT -DI386
-DHAVE_COMPUTED_GOTO
and include/parrot/config.h has:
#define PARROT_JIT_CAPABLE 1
Below are some timings, and the output of the configuration.
Does this indicate that something is wrong or not?
Thanks
Christian.
time perl -w fib.pl
fib(28) = 317811
real 0m2.193s user 0m2.180s sys 0m0.010s
time python fib.py
fib(28) = 317811
real 0m1.723s user 0m1.700s
time ../../parrot fib.imc
fib(28) = 317811 11.031775s
real 0m11.113s user 0m8.130s sys 0m0.030s
with -f:
real 0m17.399s user 0m8.060s
real 0m8.540s user 0m7.900s
with -g:
real 0m8.904s user 0m7.590s
with -P:
real 0m11.088s user 0m8.860s
with -S:
real 0m8.607s user 0m8.220s
with -j:
real 0m7.884s user 0m7.860s
real 0m11.019s user 0m9.040s
with -j -O:
real 0m8.292s user 0m8.280s
with -j -O8:
real 0m7.873s user 0m7.870s
with -j --optimize=9:
real 0m7.401s user 0m7.340s
time perl -w oo6.pl
500000
real 0m2.567s user 0m2.550s
time python oo6.py
500000
real 0m2.679s user 0m2.560s
time ../../parrot oo6.imc
500000
real 0m7.539s user 0m7.470s
with -j:
real 0m8.080s user 0m8.040s
with -C:
real 0m7.345s user 0m7.190s
real 0m7.619s user 0m7.360s
with -j:
real 0m9.734s user 0m7.630s
Output from perl Configure.pl --optimize --verbose-step=JIT :
...
Determining architecture, OS and JIT capability.......................
Setting Configuration Data:
(
archname => 'i386-linux-thread-multi',
cpuarch => 'i386',
osname => 'linux',
);
-e "jit/i386/core.jit" = yes
Setting Configuration Data:
(
asmfun_o => '',
);
Setting Configuration Data:
(
jitarchname => 'i386-linux',
jitcpuarch => 'i386',
jitcpu => 'I386',
jitosname => 'LINUX',
jitcapable => '1',
cc_hasjit => ' -DHAS_JIT -DI386',
TEMP_jit_h => '$(INC)/jit.h',
TEMP_jit_o => '$(SRC)/jit$(O) $(SRC)/jit_cpu$(O)
$(SRC)/jit_debug$(O) $(SRC)/jit_debug_xcoff$(O)',
);
Setting Configuration Data:
(
TEMP_exec_h => '$(INC)/jit.h $(INC)/exec.h $(INC)/exec_dep.h
$(INC)/exec_save.h',
TEMP_exec_o => '$(SRC)/exec$(O) $(SRC)/exec_cpu$(O)
$(SRC)/exec_save$(O)',
execcapable => '1',
);
(has_exec_protect cc -D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS
-DDEBIAN -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64 -O3 -DDISABLE_GC_DEBUG=1 -DNDEBUG -I./include
-c test.c
c++ -L/usr/local/lib -Wl,-E test.o -o test -ldl -lm -lpthread -lcrypt
./test 0
ok
no) cc -D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN
-I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O3
-DDISABLE_GC_DEBUG=1 -DNDEBUG -I./include -c test.c
c++ -L/usr/local/lib -Wl,-E test.o -o test -ldl -lm -lpthread -lcrypt
./test
ok
Setting Configuration Data:
(
jit_i386 => 'fcomip',
);
...done.
Running CPU specific
stuff............................................Setting
Configuration Data:
(
verbose => undef,
);
done.
Verifying that the compiler supports function pointer casts...........done.
...
Leopold Toetsch
> Ok, I've looked at the test_exec_linux source, and tried it out
> separately; it's clear what happens to me now: I've enabled the
> following GrSecurity option, which makes the mprotect system call fail
> with a permission error - the test even outputs this as the line
> "failure: Permission denied".
>
> CONFIG_GRKERNSEC_PAX_MPROTECT
> Enabling this option will prevent programs from
> - changing the executable status of memory pages that were
> not originally created as executable,
With that option enabled we'd need a different
mem_alloc_executable/mem_free_executable pair. Can you try to replace
the allocation to use mmap with the PROT_EXEC bit set?
> I did perl Configure.pl --optimize. I still get a parrot that is about 4
> times slower than perl5,
... with the fib test. Yes. That's the reason for the ongoing discussion
for a different calling scheme.
> ... and for which none of the -f,-g,-P,-S,-C,-j
> options does seem to make a significant difference in speed. (see data
> below).
That was a bug that is already fixed. Please update from CVS.
leo