Stack overflow: how to monitor stack usage?

[Mateusz Viste]

Hello,

Just wondering - is there any mechanism that allows to monitor the
current usage of the stack?

I am battling with some stack overflow situations within a Turbo C
program. I know how to increase the stack size using the stklen global
variable under Turbo C, but I couldn't find any information about how to
check the current usage of the stack... Should I look directly at the
value of the SP register?

regards,
Mateusz

[Johann Klammer]

Are you perhaps calling libc routines from inside an interrupt context?
At least the watcom compiler checks the STACK address against some
internal variables. Others possibly do, too. Compact model clib may
expect the stack to be in the same data seg as everything else. This is
not necessarily true for interrupt stacks. clib is not necessarily
reentrant. Especially the parts that call Dos for I/O -> Don't call clib
from int context.

If it's in application context, just look for array auto vars and
replace them by ptrs+malloc. reading the SP may be possible with some
inline assembly. just doing
void * s;
__asm{
mov ax,sp
mov s,ax
}
might work...(not quite sure myself.. not a Borland person)

[Sjouke Burry]

Just make an asm function containing mov ax,sp ,that
then returns 16 bit stack size.
And print that value somewhere on the screen.
Thats how I did it 20 years ago, and found c++ leaking
memory all over the place.
Switched to C and NO MALLOC PLEASE, and all problems gone.
A lot of programmers are very sloppy with freeing memory,
and get away with that in small programs, or programs with
short runtimes, and then thinking they did it right.......

[Rod Pemberton]

On Thu, 24 Jul 2014 15:20:13 -0400, Mateusz Viste
<mateusz.viste@localhost> wrote:

> Hello,
>
> Just wondering - is there any mechanism that allows to monitor the
> current usage of the stack?

There is no official way to do this since C officially does not
require a stack.

On a POSIX system, you should be able to use alloca() to monitor
changes to stack allocation. I doubt that Turbo C has it since
it's for DOS, but it might. So, you should check.

You could have a function that declares an a procedure local
variable (auto) and returns or displays it's address. This
acts like the address that would be returned by alloca() with
a small value, like 2 or 4. So, your variable will be allocated
at the top of the stack space. So, the variable's address
represents an address that's a bit larger than what is currently
in use for the stack. You'd call the function each time you want
an update. The problem is this only gives an address near the
top of the current stack. To determine the amount of stack in
use, you need to subtract an initial address from the current
address. You'll need to call your function just after entering
main() and save an early value for the low value to subtract.


Rod Pemberton

[Mateusz Viste]

Hi,

On 07/24/2014 10:08 PM, Johann Klammer wrote:> Are you perhaps calling

libc routines from inside an interrupt context?
> At least the watcom compiler checks the STACK address against some
> internal variables. Others possibly do, too. Compact model clib may
> expect the stack to be in the same data seg as everything else. This
> is not necessarily true for interrupt stacks. clib is not necessarily
> reentrant. Especially the parts that call Dos for I/O -> Don't call
> clib from int context.

Well, I do use an interrupt handler, but no, I don't use any libc calls
from within it (not that I am aware of, at least). But truth is that my
stack overflow troubles begun since I started playing with the PIT
interrupt.. so it's not impossible is wrong with my code. However, it's
so simple code that I don't really know what could be wrong there. The
actual code is hosted here:

http://sourceforge.net/p/dosmid/code/HEAD/tree/timer.c

What I do is that I reprogram the PIT to be much faster than the default
18.2hz, and hook my own interrupt handler on the PIT IRQ to increment a
long counter I use for timing.

> If it's in application context, just look for array auto vars and
> replace them by ptrs+malloc.

Yes, that's a theory I'm aware of, and of course I never try to allocate
too much on the stack.

cheers,
Mateusz

[Mateusz Viste]

On 07/25/2014 12:24 AM, Sjouke Burry wrote:
> Just make an asm function containing mov ax,sp ,that
> then returns 16 bit stack size.

But won't the SP register simply point to the top of my stack? How do I
know then how far down it can grow?

> A lot of programmers are very sloppy with freeing memory,

Yes, I know how malloc/free works, I use them daily in my 'real life'
work. I am however a bit lost sometimes when it comes to specific x86
and real mode stuff - on my usual compiler (gcc) I never have to worry
about running out of stack, or being interrupted by an IRQ, etc...

Anyway, how could a sloppy malloc() be responsible for a stack overflow
situation? (can it? the stack space is supposed to be separated from the
heap, but maybe things are different under 16bit?)

cheers,
Mateusz

[Mateusz Viste]

Hi Rod,

On 07/25/2014 07:42 AM, Rod Pemberton wrote:
> On a POSIX system, you should be able to use alloca() to monitor
> changes to stack allocation. I doubt that Turbo C has it since
> it's for DOS, but it might. So, you should check.

Yes, there's no alloca() in Turbo C indeed.

> The problem is this only gives an address near the
> top of the current stack. To determine the amount of stack in
> use, you need to subtract an initial address from the current
> address. You'll need to call your function just after entering
> main() and save an early value for the low value to subtract.

This sounds like a good idea! I haven't thought about it. You're right,
I should probably read the stack pointer position (here I could just as
well use the SP register directly, as suggested by others, instead of
playing with alloca-like functions), and compare it to an early value of
it. Since I know how much stack I am supposed to have in total, this
shall provide me a good way to measure how much stack is left. Maybe +/-
the amount of stack consumed by local main() variables, but should be
still good enough for my needs.

cheers,
Mateusz

[Rod Pemberton]

On Fri, 25 Jul 2014 02:20:23 -0400, Mateusz Viste
<mateusz.viste@localhost> wrote:

> Well, I do use an interrupt handler, but no, I don't use any libc calls
> from within it (not that I am aware of, at least). But truth is that my
> stack overflow troubles begun since I started playing with the PIT
> interrupt.. so it's not impossible is wrong with my code. However, it's
> so simple code that I don't really know what could be wrong there. The
> actual code is hosted here:
>

> [link to timer.c]

From reading your posted and linked to code, I'm not sure
what is incorrect here or with your XMS call.

Basically, I've not used Turbo C. I do use OpenWatcom
which is similar in some respects.

If you don't have it, start by grabbing the Turbo C documentation:
http://bitsavers.trailing-edge.com/pdf/borland/turbo_c/

Next, some things I'd check:
1) (in timer.c) try setting the PIT to the default speed
2) (in timer.c) try not calling oldfunc()
3) (in timer.c) try moving "static int callmod = 0;" to just
below "static void interrupt (*oldfunc)(void);"
4) (in timer.c) try removing all the 'static' keywords,
after moving callod as in #3
5) (in xms_move) try not calling xmsdrv()
6) (in xms_move) try changing 'far *' to 'huge *'
7) (in xms_move) check if the xmsdrv() pointer is correct
8) (in xms_move) check if xmsdrv() is called and returned from

#1 checks if the code works correctly without a timing change.
(You'll likely need to comment out the 'callmod%=64' line, etc.)
#2 checks if it's an issue with calling the original interrupt
#3 checks the placement of callmod, intended to be used with #4
#4 'static' shouldn't be needed
#5 checks if it's an issue with calling the xmsdrv() or XMS
#6 checks if it's an issue with segment:offset boundaries
#7 checks if the xmsdrv() function pointer is correct
#8 checks if the xmsdrv() is calling XMS and returning correctly

I don't know what you've done or tested so far, or if any of
these will produce any useful results. It's just stuff I'd
check first. You can put 'static' etc back in after you've
located the problem.


Rod Pemberton

[Johann Klammer]

I see you are calling the previous handler to keep sys timing
consistent. I believe you are supposed to use chain_intr or sthg
instead... because of the iret which takes a bit more off the stack than
just a return address... register image... (It' ll _under_run)

[Mateusz Viste]

On 07/25/2014 12:07 PM, Johann Klammer wrote:
> I see you are calling the previous handler to keep sys timing
> consistent. I believe you are supposed to use chain_intr or sthg
> instead... because of the iret which takes a bit more off the stack than
> just a return address... register image... (It' ll _under_run)

What you say makes a lot of sense. I can see indeed how daisy-chaining
interrupt routines could lead to stack corruption.

I looked into my Turbo C reference documentation, and unfortunately it
seem there is no 'chaining' function that would allow to call another
INT handler in a clean way.

Also, all INT-handling functions I could find on the internet (and
written for Turbo C) do exactly like I do - calling the original handler
'as-is'. So I wonder if maybe Turbo C is supposed to detect such
situation and apply some hidden magic...

Here I found a sad statement:
http://stanislavs.org/helppc/_chain_intr.html

"no corresponding function in Turbo C"

But again, I'm not sure how to understand this... Does it mean
"impossible to do", or "should work anyway"? Or maybe I am supposed to
manually correct the stack pointer and/or base pointer myself? This I
can only keep guessing. :/

Mateusz

[Mateusz Viste]

Hello Rod,

Thanks for your analysis. Yes, as you guessed I already performed plenty
of tests and tries (most of them, or maybe all, being in your list).
Haven't keps a log of them though, so I am not sure anymore what I tried
and what I haven't, so I will go through your list and check it one by
one all over. Thanks!

Mateusz

[Johann Klammer]

Not true.

Diomidis Spinellis DOS TRACE utility makes use of it...
You can find the code on the net...

[Mateusz Viste]

On 07/25/2014 01:18 PM, Johann Klammer wrote:
> Not true.
>
> Diomidis Spinellis DOS TRACE utility makes use of it...
> You can find the code on the net...

Well, I don't say that such instruction doesn't exist :) It clearly
does. But unfortunately Turbo C doesn't know about it, and I couldn't
find any alternative in the Turbo C reference documentation.

I looked up the "DOS trace" program, and it appears that it has been
written for Microsoft C, not Turbo C. At least that's what I found on
the 'open-watcom' github project:

"This is the DOS trace utility, originally written by Diomidis
Spinellis. It was ported to Open Watcom (original was designed for
Microsoft C/C++ 7.0)"

cheers,
Mateusz

[Johann Klammer]

Interesting, I seem to have downloaded TC3 just to compile it a while
ago... was not aware of that port...
not sure if it worked.. anyway, doing a text search on the TC3
installation shows it's in DOS.H line 448...

...
void _Cdecl _chain_intr ( void interrupt (far *__target)( ));^M
...

[Johann Klammer]

On 07/25/2014 02:06 PM, Mateusz Viste wrote:

> On 07/25/2014 01:18 PM, Johann Klammer wrote:
>> Not true.
>>
>> Diomidis Spinellis DOS TRACE utility makes use of it...
>> You can find the code on the net...
>
> Well, I don't say that such instruction doesn't exist :) It clearly
> does. But unfortunately Turbo C doesn't know about it, and I couldn't
> find any alternative in the Turbo C reference documentation.
>

That thelp.com thing? It's in there...
http://members.aon.at/~aklamme4/scratch/ci.png

[Mateusz Viste]

On 07/25/2014 02:48 PM, Johann Klammer wrote:
> Interesting, I seem to have downloaded TC3 just to compile it a while
> ago... was not aware of that port...
> not sure if it worked.. anyway, doing a text search on the TC3
> installation shows it's in DOS.H line 448...
>
> ...
> void _Cdecl _chain_intr ( void interrupt (far *__target)( ));^M
> ...

In fact, I did the same thing on my side already, and found nothing with
the word "chain" in any of the *.h files shipped with my Turbo C copy.

The difference is that you seem to check against Turbo C 3.x, while I am
developing under Turbo C 2.01. On version 2.01, there is no function
with the word "chain" in it, inside dos.h, nor in any other *.h file
shipped with the compiler.

Nonetheless, it's interesting to know that TC3 have such utility - I
will get TC3 as well, and try to compile my buggy code with it, using
_chain_intr for INT chaining. Who knows - maybe this is the silver
bullet it needs :)

Thanks for your investigations!

cheers,
Mateusz

[Mateusz Viste]

On 07/25/2014 02:59 PM, Johann Klammer wrote:
> That thelp.com thing? It's in there...
> http://members.aon.at/~aklamme4/scratch/ci.png

No, I don't use the 'electronic' manual, I was checking inside the paper
version of the "Turbo C reference guide" from 1988. I mirrored a copy of
it below (just in case you'd be interested in such a prehistoric
documentation):

gopher://gopher.viste-family.net/1/docs/programming

But as said in my previous message, I see now that we are checking on
different versions of the TC compiler, so that explains why we see
different things :)

cheers,
Mateusz

[Mateusz Viste]

Hi!

Following Johann's hint, I went and tried Turbo C 3.0, and replaced the
daisy-chaining inside my IRQ0 interrupt routine with a _chain_intr()
call. And - magically - all my stack overflow troubles went away.

But in fact, it's not the '_chain_intr()' that fixed it. Later I
replaced it again with a raw call to the original timing interrupt
handler, and the stack overflow didn't happened anymore. So without any
code modification, the 'stack overflow' stopped. Simply by using a
different compiler.

So either TC3 is handling such daisy-chaining in an inherently better
way than Turbo C 2, or... no idea - TC3 is handling stack differently,
or something.. Anyway, since the stack seems to behave well now, the
problem is somehow solved - I guess I will just stick to TC3 now.

cheers!
Mateusz

On 07/24/2014 09:20 PM, Mateusz Viste wrote:

[Rod Pemberton]

On Fri, 25 Jul 2014 15:26:01 -0400, Mateusz Viste
<mateusz.viste@localhost> wrote:

> So either TC3 is handling such daisy-chaining in an inherently better
> way than Turbo C 2, or... no idea - TC3 is handling stack differently,
> or something.. Anyway, since the stack seems to behave well now, the
> problem is somehow solved - I guess I will just stick to TC3 now.
>

If you're curious, you could always compare the disassembly to
see what's different. Once you know, you could make it work
for TC2 too.


Rod Pmeberton