pthread_attr_getstack(), pthread_attr_getstackaddr()
dick....@gmail.com
The only discussion I could find on this group related to setting the
stack. I need to find the start/size in order to anticipate whether
a recursive function is about to overflow, and terminate gracefully
rather than getting a segfault on the guard page.
http://www.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_09.html#tag_02_09_08
Sample code I've written fails on both AIX and Solaris, so it's likely
I don't understand the interface fully. My understanding is:
1) use pthread_create, pointing at a pthread_attr_t
2) Use the same pthread_attr used to create the thread, and you can
retrieve the assigned stack address and the size.
On AIX, an initialized pthread_attr, before calling pthread_create,
will return the default stack size and 0 as a stack address.
I expected that.
Following pthread_create, an attempt to return the thread stack address
still returns zero ... I didn't expect that.
The code returns 0 for both values on Solaris.
Can anyone spot what I've done wrong here?
Note; Solaris hasn't implemented the pthread_attr_getstack() routine,
so
I'm using *getstackaddr(), to be more portable.
/* /opt/SUNWspro/bin/cc -mt -lrt -lpthread -o stk8 stk8.c */
#include <pthread.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <stdlib.h>
void*
Proc(void* param) {
sleep(3);
return 0;
}
int
main(int argc, char *argv[]) {
pthread_attr_t Attr;
pthread_t Id;
void *Stk;
size_t Siz;
int err;
if ((err = pthread_attr_init(&Attr)) != 0)
printf("pthread_attr_init failed with <%d> error\n", err);
if ((err = pthread_attr_getstacksize(&Attr, &Siz)) != 0)
printf("pthread_attr_getstacksize failed with <%d> error\n",
err);
if ((err = pthread_attr_getstackaddr(&Attr, &Stk)) != 0)
printf("pthread_attr_getstackaddr failed with <%d> error\n",
err);
printf("Default: Addr=%08x Size=%d\n", Stk, Siz);
if ((err = pthread_create(&Id, &Attr, Proc, NULL)) != 0) {
printf("pthread_create failed with <%d> error\n", err);
perror("pthread_create"); }
if ((err = pthread_attr_getstackaddr(&Attr, &Stk)) != 0)
printf("pthread_attr_getstackaddr failed with <%d> error\n",
err);
printf("Stack : Addr=%08x Size=%d\n", Stk, Siz);
return 0;
}
loic...@gmx.net
> I'm having trouble find the defined thread stack in a portable way.
> The only discussion I could find on this group related to setting the
> stack. I need to find the start/size in order to anticipate whether
> a recursive function is about to overflow, and terminate gracefully
> rather than getting a segfault on the guard page.
to my knowledge, POSIX doesn't offer an API which returns the start
address and size of the stack used by a thread.
> Sample code I've written fails on both AIX and Solaris, so it's likely
> I don't understand the interface fully. My understanding is:
>
> 1) use pthread_create, pointing at a pthread_attr_t
> 2) Use the same pthread_attr used to create the thread, and you can
> retrieve the assigned stack address and the size.
No, that's not correct. Notice that accordingly to IEEE Std 1003.1,
2004 Edition, the prototype for pthread_create() is:
int pthread_create(pthread_t *restrict thread,
const pthread_attr_t *restrict attr,
void *(*start_routine)(void*), void *restrict arg);
The qualifier const tells us that the structure pointed by attr isn't
modified by pthread_create(). This new prototype underlines a property
that holds since Posix.1c.
> On AIX, an initialized pthread_attr, before calling pthread_create,
> will return the default stack size and 0 as a stack address.
> I expected that.
>
> Following pthread_create, an attempt to return the thread stack address
> still returns zero ... I didn't expect that.
Not surprising, since thread attribute isn't modified by
pthread_create().
> The code returns 0 for both values on Solaris.
AFAICS, there is no restriction in the standard to use the value "0" to
mean: "default stack size".
> Can anyone spot what I've done wrong here?
> Note; Solaris hasn't implemented the pthread_attr_getstack() routine,
> so
> I'm using *getstackaddr(), to be more portable.
Which version of Solaris are you using? You might need to compile with
the _XOPEN_SOURCE=600 flag in order to access the
pthread_attr_getstack() function.
I have no idea yet how to determine the default stack size in a
portable fashion. The only possibility I see to your problem would be
to manage the thread's stack by yourself. You would then exactly knows
where it starts and how big it is.
HTH,
Loïc.
David Butenhof
> Hi Dick,
>
>> I'm having trouble find the defined thread stack in a portable way.
>> The only discussion I could find on this group related to setting the
>> stack. I need to find the start/size in order to anticipate whether
>> a recursive function is about to overflow, and terminate gracefully
>> rather than getting a segfault on the guard page.
>
> to my knowledge, POSIX doesn't offer an API which returns the start
> address and size of the stack used by a thread.
POSIX has no standard support for debugging and virtually none for what
might be called "diagnostic introspection". These are useful, and even
critical; but the business of POSIX is specifically to standardize
"common industry practice", and the interfaces and models used for such
things simply vary too widely. (Do we specify ptrace, /proc filesystem;
and would /proc be single level or hierarchical, binary or textual...
and so forth.)
On most platforms there'll be a way to determine the stack region of the
running thread. (For example, on Tru64 UNIX and OpenVMS the pthread_t
value points to a "Thread Environment Block" defined in <sys/types.h>
that would provide this information.) The compilers use this to check
for stack overflow for large variable stack frames, rather than testing
each page looking for the guard page.
>> Sample code I've written fails on both AIX and Solaris, so it's likely
>> I don't understand the interface fully. My understanding is:
>>
>> 1) use pthread_create, pointing at a pthread_attr_t
>> 2) Use the same pthread_attr used to create the thread, and you can
>> retrieve the assigned stack address and the size.
>
> No, that's not correct. Notice that accordingly to IEEE Std 1003.1,
> 2004 Edition, the prototype for pthread_create() is:
>
> int pthread_create(pthread_t *restrict thread,
> const pthread_attr_t *restrict attr,
> void *(*start_routine)(void*), void *restrict arg);
>
> The qualifier const tells us that the structure pointed by attr isn't
> modified by pthread_create(). This new prototype underlines a property
> that holds since Posix.1c.
pthread_attr_t is an extendable input arguments block for thread create.
There's no information in it that you didn't PUT in it, so there's
rarely any point in trying to get something out of it. It's common
practice to reuse attributes objects to create multiple threads -- the
design intent was something like a "thread subclass" mechanism -- so
it'd make no sense to store information about a particular thread.
>> On AIX, an initialized pthread_attr, before calling pthread_create,
>> will return the default stack size and 0 as a stack address.
>> I expected that.
The initial value of all the stack attributes, AND the relationship
between them, is completely unspecified by the standard. While some
implementations have inferred that "stacksize" should be the SIZE of an
application-managed stack specified with "stackaddr", this relationship
is illusory, and was never intended (much less specified) in the
standard. You EITHER set stacksize, to control the SIZE of the stack
allocated automatically by the implementation OR you set stackaddr to
specify "the address" of a stack you've allocated and want the
implementation to use. (And the standard says nothing about what this
means across machines where the stack grows up or down, or is post- or
pre-incremented; you need to figure out what the particular
implementation expects.)
>> Following pthread_create, an attempt to return the thread stack address
>> still returns zero ... I didn't expect that.
>
> Not surprising, since thread attribute isn't modified by
> pthread_create().
>
>> The code returns 0 for both values on Solaris.
>
> AFAICS, there is no restriction in the standard to use the value "0" to
> mean: "default stack size".
The default value is completely undefined. On some implementations,
stacksize might be the default stack size; but it might also be 0. I was
actually surprised when I discovered Solaris had done that, but after
some thought I realized it didn't violate any rules... and it does have
some advantages. (For the implementation, that is!) The standard says
nothing about what you can expect, or what it means. Any assumptions are
erroneous and nonportable!
>> Can anyone spot what I've done wrong here?
>> Note; Solaris hasn't implemented the pthread_attr_getstack() routine,
>> so
>> I'm using *getstackaddr(), to be more portable.
I find this vaguely amusing, though it's inevitable. POSIX introduced
pthread_attr_getstack() specifically because there's no remotely
portable way to use or interpret the stackaddr attribute. It's
inherently broken. It never should have existed, since sigstack(), which
has the same problems, had already been replaced by sigaltstack(), which
doesn't. But someone really wanted it, nobody else in the POSIX working
group did, and instead of arguing about the syntax we just made it an
option and forgot about it. (Always, ALWAYS, a mistake. And indeed some
years later the Single UNIX Specification came along and said "this
option shall be supported", and we couldn't avoid the stupid thing any
more; so we finally got around to fixing it.)
Anyway, even though it's still possible that the stackaddr attribute may
be supported more places than the stack attribute, it is NOT more
portable; it's absolutely inherently non-portable. If all you've got is
stackaddr, you might as well give up right there and go back to bed.
(You'll accomplish just as much and be far more comfortable. ;-) )
Not, of course, that EITHER of them help for what you're trying to
accomplish, anyway!
> Which version of Solaris are you using? You might need to compile with
> the _XOPEN_SOURCE=600 flag in order to access the
> pthread_attr_getstack() function.
>
> I have no idea yet how to determine the default stack size in a
> portable fashion. The only possibility I see to your problem would be
> to manage the thread's stack by yourself. You would then exactly knows
> where it starts and how big it is.
Yes, IF you control all the threads in which this code might run, you
could use pthread_attr_setstack() to use your own application-managed
stacks. (A DIFFERENT stack for each thread, please!) You could remember
the address and size of each, identify (either by thread ID or simply by
the current stack pointer) which you're using, and then easily determine
its size. Of course if you're using application-managed stacks you won't
have a guard page unless you make it yourself. (And again you need to be
aware of whether stacks go UP or DOWN in order to figure out which
page(s) to protect.)
If you don't create all the threads, you'll have to determine the
non-portable mechanism to determine thread stack extents for each
platform you want to support.
--
Dave Butenhof, David.B...@hp.com
HP Utility Pricing software, POSIX thread consultant
Manageability Solutions Lab (MSL), Hewlett-Packard Company
110 Spit Brook Road, ZK2/3-Q18, Nashua, NH 03062
dick.dunbar
to the same conclusion. Why anybody would want to use setstack
followed by getstack to find
out what they just set is beyond me. I couldn't find a single example
of usage, except the posixtestsuite.
I've given up hope to solve this with thread api's ... I know how big
my stacks are, just not the starting (or ending) point. So, the simple
solution:
- Initial thread function will store the current stack pointer in
thread local storage
void* Stk = &Stk;
- Recursive function will calculate the current stack address in the
same way,
and determine whether there is enough stack space for the next
recursion.
- Algorithm will be tuned knowing the stack size, the amount of stack
space needed for each recursion, and some overhead values, depending on
the OS implementation.
Incidently, most implementations don't actually check for a guard page,
they just use mprotect (or similar OS function) to set Read-Only
attribute on the page. If anyone tries to carve off a new stack that
touches the guard page, they'll segfault.
My question has been answered, and I appreciate the thoughtful
responses of clarification. Thanks.
David Butenhof
> Incidently, most implementations don't actually check for a guard page,
> they just use mprotect (or similar OS function) to set Read-Only
> attribute on the page. If anyone tries to carve off a new stack that
> touches the guard page, they'll segfault.
This is OK as long as the compiler knows that the stack frame can NEVER
"skip" a guard page. However any variable sized (or known large) frame
(that is, with large or variable local data) may scribble over other
writable data if the "end" gets written before the part that maps into
the guard.
So skipping any explicit stack overflow check is just fine for any frame
the compiler knows is too small to possibly skip a known guard. If the
compiler doesn't claim to know the guard size, or if the frame might be
large, it MUST check explicitly -- or it's broken.
The standard algorithm is to break down the allocation into chunks that
cannot skip a guard page (that might be a single page, or a multiple if
it knows the guard is more than one page) and probe each to provoke a
segv during the prologue, or it can mean finding the actual guard region
and doing a comparison of the tentative new stack pointer against that
value. If you KNOW the allocation is large (potentially "many times the
guard"), then the comparison is usually a lot more efficient. (Presuming
of course that the compiler can easily acquire the necessary information.)
Torsten Robitzki
dick....@gmail.com wrote:
> I'm having trouble find the defined thread stack in a portable way.
> The only discussion I could find on this group related to setting the
> stack. I need to find the start/size in order to anticipate whether
> a recursive function is about to overflow, and terminate gracefully
> rather than getting a segfault on the guard page.
did you take changing you algorithm from recursing to iteration into
account too? If you use just a loop and emulate a stack in a separate
data structure allocated on the heap, you might get the most portable
solution.
regards
Torsten
dick.dunbar
the original code, so changing the "elegance of recursion" to iteration
is a tough sell.
I presented the following point of view, which was rebuffed:
- Allocating large stacks ... especially in a thread pool ... just
because we might have a large recursion is wasteful of valuable address
space.
- Large thread stacks mean we cannot scale beyond a fixed number of
users, without moving to 64-bit support.
- Thread stacks are a single, unshared resource ... the heap is a
shared resource, which we should be using for this purpose.
- Any recursion can be expressed in terms of iteration.
Good suggestion, Torsten. My view is that detecting stack overflow is
at best a stop-gap solution that solves an immediate issue with a
system crash, but isn't a good overall solution. It's necessary, not
sufficient.
dick.dunbar
Agreed. If your goal is speed, and you're coding in C++, I would
rather rely on the application knowledge of the programmer to determine
these things (large allocations skip guard pages), rather than the
brute-force, belt-and-suspenders assurance of having the compiler check
... repeatedly ... for an event that never happens.
The implementations I've looked at have a default number of guard
pages (typically 8K), and also permit you to allocate additional guard
pages. I would only do this in a product assurance test, not in
production.
Function calls aren't the only way one can subvert the checks ... it's
quite easy to use alloca() to increase your allocated stack frame.
Windows has replaced _alloca() with _malloca(), for more security
checks. Their implementation of alloca() and the stacks allow on to
detect stack overflow with compiler checks.