Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

stack clobbering

7 views
Skip to first unread message

andrey the giant

unread,
Sep 24, 2009, 7:48:54 AM9/24/09
to

Given function foo(u32 x) and bar(u32 x), can you do this:
; Assume y is a local u32
mov edx, y
push edx
call foo
cmp eax, ...
j.. if_foo
call bar
cmp eax, ....
add esp, 4
j.. if_bar
...
if_foo:
add esp, 4
...

without worrying about esp clobber?

Tim Roberts

unread,
Sep 26, 2009, 3:26:37 PM9/26/09
to

Sure, as long as you remember that esp was modified. For example, if "y"
is defined as an offset from [ebp], then no problem. But if it is defined
as an offset from [esp], then you need to remember that esp has changed.

On the other hand, there's no real advantage to deferring the adjustment.

This kind of thing is the reason that calling conventions in x86 often use
a "callee cleans up" standard (using the "ret 4" instruction) instead of
"caller cleans up".
--
Tim Roberts, ti...@probo.com
Providenza & Boekelheide, Inc.

Robert Redelmeier

unread,
Sep 27, 2009, 11:32:26 AM9/27/09
to

Tim Roberts <ti...@munged.microcosmotalk.com> wrote in part:


"callee cleans up" might look nice, but what about routines that take a
variable (unspecified) number/type of params? Think `c` printf()
Only the caller knows for sure how many params were pushed.

The callee could guess from the fmt string, but any error will return the
wrong ESP. With stack frames, in many cases this will pass unnoticed,
but could give some hard-to-track errors.

-- Robert

ArarghMai...@not.at.arargh.com

unread,
Sep 27, 2009, 4:34:45 PM9/27/09
to

On 27 Sep 2009 15:32:26 GMT, Robert Redelmeier
<red...@ev1.net.invalid> wrote:
<snip>

>"callee cleans up" might look nice, but what about routines that take a
>variable (unspecified) number/type of params? Think `c` printf()
>Only the caller knows for sure how many params were pushed.

Either pass the length of the parms pushed, or pass a pointer to a
list of the parms. More work, but either would work.

>The callee could guess from the fmt string, but any error will return the
>wrong ESP. With stack frames, in many cases this will pass unnoticed,
>but could give some hard-to-track errors.

True -- bad idea. :-)
--
ArarghMail909 at [drop the 'http://www.' from ->] http://www.arargh.com
BCET Basic Compiler Page: http://www.arargh.com/basic/index.html

To reply by email, remove the extra stuff from the reply address.

Tim Roberts

unread,
Sep 28, 2009, 11:12:05 PM9/28/09
to

Robert Redelmeier <red...@ev1.net.invalid> wrote:

>
>Tim Roberts <ti...@munged.microcosmotalk.com> wrote:
>
>> This kind of thing is the reason that calling conventions in x86 often use
>> a "callee cleans up" standard (using the "ret 4" instruction) instead of
>> "caller cleans up".
>
>"callee cleans up" might look nice, but what about routines that take a
>variable (unspecified) number/type of params? Think `c` printf()
>Only the caller knows for sure how many params were pushed.

Of course there are exceptions. This was sufficiently obvious that I
didn't think it was worth the trouble to type it.

Tim Roberts

unread,
Sep 28, 2009, 11:12:51 PM9/28/09
to

Robert Redelmeier <red...@ev1.net.invalid> wrote:
>
>Tim Roberts <ti...@munged.microcosmotalk.com> wrote:
>
>> This kind of thing is the reason that calling conventions in x86 often use
>> a "callee cleans up" standard (using the "ret 4" instruction) instead of
>> "caller cleans up".
>
>"callee cleans up" might look nice, but what about routines that take a
>variable (unspecified) number/type of params? Think `c` printf()
>Only the caller knows for sure how many params were pushed.

Of course there are exceptions. This was sufficiently obvious that I


didn't think it was worth the trouble to type it.

James Harris

unread,
Sep 29, 2009, 8:56:54 AM9/29/09
to

On 29 Sep, 04:12, Tim Roberts <t...@MUNGED.microcosmotalk.com> wrote:
> Robert Redelmeier <red...@ev1.net.invalid> wrote:
>
> >Tim Roberts <t...@munged.microcosmotalk.com> wrote:
>
> >> This kind of thing is the reason that calling conventions in x86 often=
use
> >> a "callee cleans up" standard (using the "ret 4" instruction) instead =

of
> >> "caller cleans up".
>
> >"callee cleans up" might look nice, but what about routines that take a
> >variable (unspecified) number/type of params? =A0Think `c` printf()

> >Only the caller knows for sure how many params were pushed.
>
> Of course there are exceptions. =A0This was sufficiently obvious that I

> didn't think it was worth the trouble to type it.

What would you do here?

1. Use caller-remove when calling variadic functions (but still keep
to callee remove when calling functions with a fixed number of
arguments).

2. Have the callee remove any fixed parameters and the caller remove
any others it may have pushed.

Any preferences - especially where performance is important?

James

Robert Redelmeier

unread,
Sep 29, 2009, 9:33:44 AM9/29/09
to

Tim Roberts <ti...@munged.microcosmotalk.com> wrote in part:

Not quite so obvious: Just _how_ do you make exceptions?
Do you want prototypes required for everything, and those
prototypes tied to machine architectures (FASTCALL)?

Sure, you can get a parser to recognize `printf()`, but what happens
when you want to do your own foo_print(), maybe as a wrapper?

ASM is great because of course you can do whatever you like,
and have whatever conventions suit a function. Just so long
as you keep them straight. Much more difficult for HLLs
which are required to be portable.


-- Robert

Robert Redelmeier

unread,
Sep 29, 2009, 9:34:03 AM9/29/09
to

ArarghMai...@not.at.arargh.com wrote in part:

> Either pass the length of the parms pushed,

IIRC PASCAL uses this.

> or pass a pointer to a list of the parms.

Sounds like FORTRAN :)

> More work, but either would work.

True, but more housekeeping that ASMers love :)
Rather more than the savings of RET 4 vs add esp,4

-- Robert

James Harris

unread,
Sep 29, 2009, 1:17:30 PM9/29/09
to

On 27 Sep, 21:34, ArarghMail909NOS...@NOT.AT.Arargh.com wrote:
> On 27 Sep 2009 15:32:26 GMT, Robert Redelmeier<red...@ev1.net.invalid> wr=

ote:
>
> <snip>
>
> >"callee cleans up" might look nice, but what about routines that take a
> >variable (unspecified) number/type of params? =A0Think `c` printf()

> >Only the caller knows for sure how many params were pushed.
>
> Either pass the length of the parms pushed, or pass a pointer to a
> list of the parms. =A0More work, but either would work.

For a number of reasons passing the length of the parameters pushed
should be avoided. Apart from the extra register usage the return
address will be on the stack with the parameters below it. You'd
effectively have to pop the return address, then adjust the stack and
then jump to the address that you popped before you adjusted the
stack.

The ret N instruction does this but only works with a constant number
of bytes to drop from the stack.

The other suggestion, a pointer to a parameter block, is better. If
the pointer is in a register a simple ret will return. If the pointer
is on the stack a ret 4 or similar would do.

James

Jerry Coffin

unread,
Sep 29, 2009, 2:17:08 PM9/29/09
to

In article <4ac20ccb$0$4958$9a6e...@unlimited.newshosting.com>,
red...@ev1.net.invalid says...

>
> ArarghMai...@not.at.arargh.com wrote in part:
> > Either pass the length of the parms pushed,
>
> IIRC PASCAL uses this.

Not any Pascal compiler I've seen (or at least examined the code it
produced). The compiler generates the code for both the call and the
return, and generates code so they match. Since nothing (except
built-in things like readln and writeln) accepts a variable number of
parameters, the called code knows the number of parameters
implicitly.



> > or pass a pointer to a list of the parms.
>
> Sounds like FORTRAN :)

Most Fortran compilers use pretty much the same calling convention as
most Pascal compilers. For example, at one time Microsoft C accepted
both 'pascal' and 'fortran' as keywords to specify calling
conventions for functions -- and the two were synonyms.

--
Later,
Jerry.

ArarghMai...@not.at.arargh.com

unread,
Sep 29, 2009, 8:29:15 PM9/29/09
to

On 29 Sep 2009 18:17:08 GMT, Jerry Coffin
<jerryv...@MUNGED.microcosmotalk.com> wrote:

>
>In article <4ac20ccb$0$4958$9a6e...@unlimited.newshosting.com>,=20
>red...@ev1.net.invalid says...
>>=20
>> ArarghMai...@not.at.arargh.com wrote in part:
>> > Either pass the length of the parms pushed,=20
>>=20
>> IIRC PASCAL uses this.
>
>Not any Pascal compiler I've seen (or at least examined the code it=20
>produced). The compiler generates the code for both the call and the=20
>return, and generates code so they match. Since nothing (except=20
>built-in things like readln and writeln) accepts a variable number of=20
>parameters, the called code knows the number of parameters=20
>implicitly.
>=20
>> > or pass a pointer to a list of the parms. =20
>>=20
>> Sounds like FORTRAN :)
>
>Most Fortran compilers use pretty much the same calling convention as=20
>most Pascal compilers. For example, at one time Microsoft C accepted=20
>both 'pascal' and 'fortran' as keywords to specify calling=20


>conventions for functions -- and the two were synonyms.

Probably accepted 'basic' as well. All three have the same calling
conventions in microsoft 16-bit compiler products.

From alang.hlp (view in a fixed font):

C SYSCALL STDCALL BASIC FORTRAN PASCAL
+-----------------------------------------------+
Leading Underscore =A6 X =A6 =A6 X =A6 =A6 =A6=
=A6
+-------+-------+-------+-------+-------+-------=A6
Capitalize All =A6 =A6 =A6 =A6 X =A6 X =A6=
X =A6
+-------+-------+-------+-------+-------+-------=A6
Args Left to Right =A6 =A6 =A6 =A6 X =A6 X =A6=
X =A6
+-------+-------+-------+-------+-------+-------=A6
Args Right to Left =A6 X =A6 X =A6 X =A6 =A6 =A6=
=A6
+-------+-------+-------+-------+-------+-------=A6
Caller Stack Cleanup =A6 X =A6 =A6 * =A6 =A6 =A6=
=A6
+-------+-------+-------+-------+-------+-------=A6
BP Saved =A6 =A6 =A6 =A6 X =A6 X =A6=
X =A6
+-------+-------+-------+-------+-------+-------=A6
:VARARG Allowed =A6 X =A6 X =A6 X =A6 =A6 =A6=
=A6
+-----------------------------------------------+
* The STDCALL language type uses caller stack cleanup if the :VARARG
parameter is used. Otherwise, the called routine must clean up the
stack.
--=20

Tim Roberts

unread,
Oct 1, 2009, 4:12:22 AM10/1/09
to

James Harris <james.h...@MUNGED.microcosmotalk.com> wrote:
>
>What would you do here?
>
>1. Use caller-remove when calling variadic functions (but still keep
>to callee remove when calling functions with a fixed number of
>arguments).
>
>2. Have the callee remove any fixed parameters and the caller remove
>any others it may have pushed.
>
>Any preferences - especially where performance is important?

Option 1. Use callee-remove for fixed argument functions, and
caller-remove for variadic functions. That's only slightly more
complicated than the always-caller-remove solution.

The performance argument is a bit of a red herring -- after all, we're
talking about one additional instruction.

0 new messages