Guard page on Linux/GCC?

[Alex Snaps]

Hey all, 

Not quite sure this is the best place for this, but I thought some here may have ideas/experience with this:

I'm trying to "gracefully" deal with stack overflows, by having  signal handler (with its own small stack) test against the siginfo_t's si_code on SIGSEGV. On OS X/clang I get a SEGV_ACCERR, as the main thread gets a (i.e. one) guard page below the stack.

To my surprise though, I couldn't find a way to achieve the same on Linux (and gcc 5.4). I can't think of any other approach, and I don't want to "artificially" spawn another thread with such a guard page in place to address the issue. 

All my experiments with trying to mmap (PROT_NONE, MAP_FIXED) such a page myself didn't succeeded either.

Has anyone ever dealt with such a case?

Thanks!
Alex

--

Alex Snaps

Twitter: @alexsnaps

[Gil Tene]

This is a common technique, and (at least) JVMs tend to use it. I.e. placing a PROT_NONE page at the end of the stack and lifting protection in the SEGV handler. You need to make sure the signal handler uses an alternate stack [by specifying it with sigaction(2) and signalstack(2)], or this will [obviously] not work.

 

There are all sorts of variations to how this stuff is done. Including "stack banging" (touching the stack e.g. 8KB ahead in calls) techniques that can sometimes avoid an alternate stack, and variations on how ones reserves stack virtual space to make sure the stack can be grown (note that the protection scheme is only "safe" as long as stack frames do not exceed the protected space size). There are also variations that do stack-chaining (where the stack is not contiguous in memory and extends somewhere else when needed). But the simplest seems to be the protect+extend-in-SEGV.

[Alex Snaps]

Thanks Gil. My stuff uses its own stack, as it does make indeed sense this wouldn't work otherwise. 

But for some reason, on Linux I have not been able to report a SEGV_ACCERR when handling the SIGSEGV, i.e. the "variation" I had in mind was just to protect a page "below" the stack (PROT_NONE), so that any access would then be reported as a SEGV_ACCERR. But I couldn't get that work, it seems to "move" the offset at which the error gets reported. So what I do for now is just look at the address the error occurred, and if it's within a "page's distance" from the end of the stack, I consider that a stack overflow. Yuck.

I'm not very surprised the JVM uses similar trickery though... I guess my best shot is reading through that code base, should have thought of it! 

Thanks again!

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alex Snaps]

Just in case someone tumbles upon this later or anyone's interested now already: 

I found the explanation in the JVM source code wrt my issue on linux and the main thread's stack "behaviour"

http://hg.openjdk.java.net/jdk8/jdk8/hotspot/file/tip/src/os/linux/vm/os_linux.cpp#l3036

Thanks again Gil, while I'm not working with the JVM anymore, having a copy of the hotspot source code around remains useful!