You can get it here:
I'd like some design advice. Everything is implemented to C90
requirements (as far as I know) except for one thing - setjmp
and longjmp. These aren't implemented at all really.
Amazingly, everything so far has been able to be implemented
using C. I'm using mingw gcc as the compiler, and using the
Windows API I haven't needed to resort to coding in gnu's
assembler.
Ideally I want to use a compiler other than gcc in the future, e.g.
replace it with Microsoft C. So I'd prefer not to use GNU-specific
embedded assembler to implement setjmp.
Is there a Windows API that will allow me to implement this
functionality (basically saving and restoring registers), or do I need
to use the dreaded gasp after all? Any alternatives?
Note - obviously if I'm replacing mingw with my own library, I can't
use its version or CRTDLL (Microsoft's version). I am restricted
to kernel32.
Thanks. Paul.
> Is there a Windows API that will allow me to implement this
> functionality (basically saving and restoring registers), or do I need
> to use the dreaded gasp after all? Any alternatives?
>
You can try GetThreadContext()/SetThreadContext(), but I don't know if this
would actually get the job done. setjmp()/longjmp() can be implemented by
naively replacing register contents, but while this meets the letter of the
standard, it's not a very useful implementation (most "real" implementations
save and restore stack frames, so you can jump back in a function which has
already returned -- something a naive implementation can't do without
leaving you with a corrupt stack). Also, this would only be portable in the
sense that it's Win32 -- it's still architecture-specific. In short, I'm
glad I don't have to do it.
--
J.
Thanks for the pointer.
This:
http://msdn.microsoft.com/en-us/library/ms679362(VS.85).aspx
says:
"If you call GetThreadContext for the current thread, the function
returns successfully; however, the context returned is not valid."
What's that supposed to mean? I'm exactly after the current thread.
> setjmp()/longjmp() can be implemented by
> naively replacing register contents, but while this meets the letter of the
> standard, it's not a very useful implementation
I only want to meet the letter of the standard. It's taken 15 years
to
get this far and setjmp/longjmp is the last thing stopping even the
letter being met.
> (most "real" implementations
> save and restore stack frames, so you can jump back in a function which has
> already returned -- something a naive implementation can't do without
> leaving you with a corrupt stack). Also, this would only be portable in the
> sense that it's Win32 -- it's still architecture-specific. In short, I'm
> glad I don't have to do it.
Sure, I'm only after Win32. I have a setjmp/longjmp already for MVS
and CMS (and they are about to be rewritten to NOT save the entire
stack since it is not needed and a general waste.
Regardless - is there a way to save registers from within C or do I
need to break out the assembler?
Thanks. Paul.
The next bit is even trickier -- the context returned by GetThreadContext()
isn't what you can literally use for SetThreadContext(), because then you'd
just force the thread into the wait routine, which cannot possibly complete
successfully (for one thing, the stack won't be valid). So you'll need to
walk the stack to find the real return location, and somehow patch into the
context a way of indicating that you're returning from a longjmp() -- you
can probably make this work by making the wait call return WAIT_OBJECT_0 +
1. You need to know every architecture's way of organizing stack frames and
return values for this -- this is what I meant when I said you can't do this
portably, at least not platform-independently. Anyway, on x86, we'd be
patching EAX. If SignalObjectAndWait() returns WAIT_OBJECT_0, your worker
thread has returned from setjmp(). If it returns WAIT_OBJECT_0 + 1, you've
just come from a longjmp().
> Regardless - is there a way to save registers from within C or do I
> need to break out the assembler?
>
See above. I think it can be made to work. Maybe. :-)
--
J.
Crikey. If it comes down to manipulating registers, I'll write in
assembler.
Before I break out gasp, which is a non-masm-compatible assembler,
I wonder if it is possible to write masm-compatible code using
wasm (Watcom), so that it's actually in a standard language,
and then mix that with gcc-compiled .o files?
> > Regardless - is there a way to save registers from within C or do I
> > need to break out the assembler?
>
> See above. I think it can be made to work. Maybe. :-)
One last option with your threads. Since setjmp can be implemented
as a macro, would there be scope to get it to do something like:
#define setjmp(x) \
__setupThread(x), __returnThread(x)
Such that the setup thread does some of the things you were talking
about, while the return thread at the same time prepares for a return
from longjmp?
BFN. Paul.
> Before I break out gasp, which is a non-masm-compatible assembler,
> I wonder if it is possible to write masm-compatible code using
> wasm (Watcom), so that it's actually in a standard language,
> and then mix that with gcc-compiled .o files?
>
This is going to be tricky if it's possible. I don't know how you'd make
Watcom produce compatible object files, if this can be done at all. Stuffing
the .o in a compatible .lib should be doable; I think you can make GCC
cooperate with VC++ that way at least (though I've never done this myself).
On the whole, cobbling together a compilation with different compilers is
tricky business.
>>> Regardless - is there a way to save registers from within C or do I
>>> need to break out the assembler?
>> See above. I think it can be made to work. Maybe. :-)
>
> One last option with your threads. Since setjmp can be implemented
> as a macro, would there be scope to get it to do something like:
>
> #define setjmp(x) \
> __setupThread(x), __returnThread(x)
>
> Such that the setup thread does some of the things you were talking
> about, while the return thread at the same time prepares for a return
> from longjmp?
>
Sure. This is actually pretty attractive because __returnThread(x) boils
down to a single WaitForSingleObject() call, checking the return value.
--
J.
Ok, given that that isn't a loony idea, can you explain how it
could work like this?
I was thinking along the lines that the setupThread() would create
a new thread. Then both threads would return, but with different
values.
Those different values would then determine whether returnThread
is called or not.
But actually, given that new threads are being created anyway,
why is all this complication required in the first place?
Why not just have the new thread wait to see if a longjmp is
every signalled, and then continue on? Why the need to
manipulate registers when you've got an entire thread with
the required registers?
One thing though - to avoid problems with repeated calls to
setjmp that have no corresponding longjmp, I want the ability
for setjmp to see if there is already a waiting thread, and if
so, tell it to terminate without doing anything (like closing
files!). Is that feasible?
BFN. Paul.
However, the new thread has none of the stack or registers of the original.
It cannot continue "as if" it were the original thread -- any access to
*any* variables would fail, returning from the function it's currently in
would fail. Such an implementation would be worse than useless, and I'm
pretty sure it doesn't actually meet the requirements of the standard either.
--
J.
A new thread is the equivalent of a fork(), isn't it? If so, I'm
pretty
sure that means that it gets a copy of the stack *as it was at
that point*, so can return. And the standard allows the local
stack variables in the calling function to be unreliable. And the
global variables should be available too - that's why when accessing
them you need to use semaphore locks normally, right?
However, in the end, I think you're right. The local variables of the
caller of the caller may have been manipulated (via pointer) and so
a copy of the stack is useless.
Ok, it sounds like I need to break out the assembler. I think what
I will do is first port pdpclib to Visual Studio 8, which I now have
installed for another reason, so that I can use masm to implement
setjmp and longjmp.
BFN. Paul.
No, it isn't. That's exactly the problem. A new thread gets a brand new
stack. You could copy the stack, taking care to omit the stack frame for the
function you're in, but by that point the solution hardly seems simpler than
what I proposed -- and you need access to the stack pointer anyway.
> If so, I'm pretty sure that means that it gets a copy of the stack *as it
> was at that point*, so can return. And the standard allows the local
> stack variables in the calling function to be unreliable.
Are you sure? I don't have a copy of the C90 standard available, but I do
have a draft C99 standard, and that says:
"All accessible objects have values, and all other components of the
abstract machine have state, as of the time the longjmp function was called,
except that the values of objects of automatic storage duration that are
local to the function containing the invocation of the corresponding setjmp
macro that do not have volatile-qualified type and have been changed between
the setjmp invocation and longjmp call are indeterminate."
Note the exceptions for local variables that are volatile, and local
variables that haven't changed. If C90 is more conservative on this point,
you're OK, but otherwise...
> And the global variables should be available too - that's why when
> accessing them you need to use semaphore locks normally, right?
>
Global variables will be available because they're stored at
stack-independent addresses.
> However, in the end, I think you're right. The local variables of the
> caller of the caller may have been manipulated (via pointer) and so
> a copy of the stack is useless.
>
Well, not if you copy the entire stack at the point of issuing longjmp()
(rather then setjmp()) -- but you'd need some tricky synchronization to make
sure the other thread gets it correctly.
Even if you can get it to work, though, there are still two problems. First,
all pointers to stack-based variables will be wrong unless you go to the
trouble of rebasing them (which you can't do reliably since you can't know
what's a pointer). They're not that common, but still.
Second, this solution requires ending the thread that called longjmp(). This
will get the program in trouble if it synchronizes on this thread, or does
anything else that requires thread identity (like APCs). Threads aren't
covered by the C standard so they can't occur in a strictly conforming
program and you're technically free to mess with them however you wish, but
you'd still need to put up a disclaimer.
> Ok, it sounds like I need to break out the assembler. I think what
> I will do is first port pdpclib to Visual Studio 8, which I now have
> installed for another reason, so that I can use masm to implement
> setjmp and longjmp.
>
Good luck. Make sure to make it work on both x86-32 and x86-64. And Itanium,
if you're up for it. :-)
--
J.
Pretty sure. You just quoted the relevant bit (which is the same
in C90 - the draft of C90 is available on the net).
> I don't have a copy of the C90 standard available, but I do
> have a draft C99 standard, and that says:
>
> "All accessible objects have values, and all other components of the
> abstract machine have state, as of the time the longjmp function was called,
> except that the values of objects of automatic storage duration that are
> local to the function containing the invocation of the corresponding setjmp
> macro that do not have volatile-qualified type and have been changed between
> the setjmp invocation and longjmp call are indeterminate."
>
> Note the exceptions for local variables that are volatile, and local
> variables that haven't changed. If C90 is more conservative on this point,
> you're OK, but otherwise...
Those exceptions are to what is UNRELIABLE. Normal local variables
in that function are INDETERMINATE. Only the volatile ones will be
reliable. (ie using the volatile keyword).
> Well, not if you copy the entire stack at the point of issuing longjmp()
> (rather then setjmp()) -- but you'd need some tricky synchronization to make
> sure the other thread gets it correctly.
Ok. Too much trickiness to save a bit of assembler. :-)
> > Ok, it sounds like I need to break out the assembler. I think what
> > I will do is first port pdpclib to Visual Studio 8, which I now have
> > installed for another reason, so that I can use masm to implement
> > setjmp and longjmp.
>
> Good luck. Make sure to make it work on both x86-32 and x86-64. And Itanium,
> if you're up for it. :-)
I'll be quite satisified if I can get it to work on x86-32!
I spent today trying to get this to work. First problem is that I
include
windows.h. I used to be using the mingw version of this, which was
working fine.
However, now I tried to switch to what came with VC + SDK. I would
have though that the SDK was something that all compilers could
use, and not be VC-specific.
However, the SDK seems to include VC headers, and my own C
library was conflicting with that. Some of the types were different,
e.g. their time_t was 64-bit, and they expected wcscpy to exist,
which is not part of C90.
However, after hacking around them so that I could link, I found
out that the VC compiler apparently generates code that requires
some support object code, some sort of security thing, plus
floating point. But I can't let it link those in, otherwise I get
heaps
of duplicate symbols.
I didn't have this problem with mingw because the compiler support
stuff was limited to one library, so I only needed that plus kernel32.
So I went back to mingw but took a look at the assemblers that
were now available. jwasm is new, and I spent time investigating
why there was no real mode executable for it, and then created
one.
Tomorrow hopefully I will find out if the non-real-mode version is
capable of producing a .o that can be used with gcc.
BFN. Paul.
Yes.
> Normal local variables in that function are INDETERMINATE. Only the
> volatile ones will be reliable. (ie using the volatile keyword).
No. Indeterminate are exactly these: local variables that are not
volatile-qualified which have changed between the calls. Reversing this
condition, this means that all volatile-qualified variables *and* all
variables that have not changed (volatile or not) are *not* indeterminate.
!(!volatile && changed) == volatile || !changed.
> I spent today trying to get this to work. First problem is that I
> include
> windows.h. I used to be using the mingw version of this, which was
> working fine.
>
> However, now I tried to switch to what came with VC + SDK. I would
> have though that the SDK was something that all compilers could
> use, and not be VC-specific.
>
Nope. A Win32 platform consists of compiler, headers and libraries, all
specific to that compiler. It's much like the C library in this regard. This
is one of the reasons mingw has its own set of headers (the other major one
being copyright, of course).
> However, after hacking around them so that I could link, I found
> out that the VC compiler apparently generates code that requires
> some support object code, some sort of security thing, plus
> floating point. But I can't let it link those in, otherwise I get
> heaps of duplicate symbols.
>
You should be able to turn most of this off, with various options. The
security stuff should disappear with /GS-. Not sure about the floating point
bits.
--
J.
I went back and see where the confusion happened. Yes, if the
local variables haven't changed, they will still be there. Thus, if
you have copied the entire stack with fork(), there's no problem.
That's what I was trying to say. However, what I actually said
was that local variables were unreliable, and as you correctly
pointed out, that is not true. So long as they haven't changed
(plus some other bonuses), an application can rely on them
still being there.
> Nope. A Win32 platform consists of compiler, headers and libraries, all
> specific to that compiler. It's much like the C library in this regard.
Ok, thanks for the info.
> You should be able to turn most of this off, with various options. The
> security stuff should disappear with /GS-. Not sure about the floating point
> bits.
Ok, thanks for that info too. The fltused is probably used to drag in
various floating point stuff, so I could probably have just defined
the
variable and ignored it, since in pdpclib, that is always dragged in
anyway.
However, I've since moved on. I found out that mingw is generating
coff format object files, and that both masm and jwasm are capable
of generating coff format files. As such, I was able to write some
masm-compatible assembler and get it to be linked in, and my
initial code fragment works:
public ___setj
___setj proc
mov eax, 77
ret
___setj endp
_TEXT ends
end
So now I need to think about how to preserve these values:
push ebp
mov ebp, esp
push esi
push edi
push edx
push ecx
push ebx
and I should be set!
BFN. Paul.
Took a long time to get that right!
But it all seems to work now. jwasm, masm and poasm can all
handle this code, used in conjunction with gcc:
winsupa.asm:
; winsupa.asm - support code for C programs for Win32
;
; This program written by Paul Edwards
; Released to the public domain
.386
.model flat,c
_DATA segment dword 'DATA'
_DATA ends
_BSS segment dword 'BSS'
_BSS ends
_TEXT segment 'CODE'
public __setj
__setj proc env:dword
mov eax, env
push ebx
push ebp
mov ebx, esp
push ebx ; esp
mov [eax + 0], ebx
mov [eax + 4], ecx
mov [eax + 8], edx
mov [eax + 12], edi
mov [eax + 16], esi
pop ebx
mov [eax + 20], ebx ; esp
pop ebx
mov [eax + 24], ebx ; ebp
mov ebx, [ebp + 4] ; return address
mov [eax + 28], ebx ; return address
pop ebx
mov eax, 0
ret
__setj endp
public __longj
__longj proc env:dword
mov eax, env
mov ebp, [eax + 20]
mov esp, ebp
mov ebp, [eax + 24]
pop ebx ; position of old ebp
pop ebx ; position of old ebx
pop ebx ; another old ebp
pop ebx ; position of old return address
mov ebx, [eax + 28] ; return address
push ebx
mov ebx, [eax + 24] ; ebp saved as normal
push ebx
mov ebx, [eax + 0]
mov ecx, [eax + 4]
mov edx, [eax + 8]
mov edi, [eax + 12]
mov esi, [eax + 16]
mov eax, [eax + 32] ; return value
ret
__longj endp
_TEXT ends
end
setjmp.c:
/
*********************************************************************/
/*
*/
/* This Program Written by Paul Edwards.
*/
/* Released to the Public Domain
*/
/*
*/
/
*********************************************************************/
/
*********************************************************************/
/*
*/
/* setjmp.c - implementation of stuff in setjmp.h
*/
/*
*/
/
*********************************************************************/
#include "setjmp.h"
#if defined(__MVS__) || defined(__CMS__)
int __saver(jmp_buf env);
int __loadr(jmp_buf env);
#elif defined(__WIN32__)
int __longj(void *);
#endif
#if !defined(__WIN32__)
int setjmp(jmp_buf env)
{
#if defined(__MVS__) || defined(__CMS__)
__saver(env);
#endif
return (0);
}
#endif
void longjmp(jmp_buf env, int val)
{
if (val == 0)
{
val = 1;
}
env[0].retval = val;
/* load regs */
#if defined(__MVS__) || defined(__CMS__)
__loadr(env);
#elif defined(__WIN32__)
__longj(env);
#endif
return;
}
setjmp.h:
/
*********************************************************************/
/*
*/
/* This Program Written by Paul Edwards.
*/
/* Released to the Public Domain
*/
/*
*/
/
*********************************************************************/
/
*********************************************************************/
/*
*/
/* setjmp.h - setjmp header file.
*/
/*
*/
/
*********************************************************************/
#ifndef __SETJMP_INCLUDED
#define __SETJMP_INCLUDED
typedef struct {
#if defined(__MVS__) || defined(__CMS__)
int saveptr; /* pointer to stack savearea */
int savelng; /* length of save area */
int savestk; /* where to put it */
int saver13; /* Where to leave it pointing to */
int saver14; /* and return address */
#elif defined(__WIN32__)
int ebx;
int ecx;
int edx;
int edi;
int esi;
int esp;
int ebp;
int retaddr;
#endif
int retval;
} jmp_buf[1];
#if defined(__WIN32__)
int __setj(void *);
#define setjmp(x) (__setj(x))
#else
int setjmp(jmp_buf env);
#endif
void longjmp(jmp_buf env, int val);
#endif
pdptest.c:
#include <stdio.h>
#include <setjmp.h>
#include <stdlib.h>
int main(void)
{
jmp_buf env;
int i;
printf("this program should print 0 then 2\n");
i = setjmp(env);
printf("i = %d\n", i);
if (i != 0) exit(0);
longjmp(env, 2);
printf("Does this line get printed?\n");
return (0);
}
Thanks for your help Jeroen!
BFN. Paul.
If you are happy with returning to an upper (on the stack) setjmp, then
SEH might be the way to go. If you you need more, using Fibers ought to
be the right thing. Implementing a kind of context switch by oneself is
difficult, which is the main reason Fibers exists.
SEH was my first idea, but the OP said he was using GCC, which AFAIK
has no SEH support. There have been third-party efforts, but using
those just to so you can build setjmp()/longjmp() on top of it (which
will still not give you full support for them) seems overkill.
Using fibers is an interesting idea, hadn't thought of that. However,
this has problems with thread identity -- you either need to create a
new fiber or convert the existing thread to a fiber, either of which
could easily trip up a client. The additional difficulty here is that
the code is in a standard library implementation, which can't assume
full control over the environment. Well, it can, but then the quality
suffers.
Or perhaps, "mostly right". :-)
I've made a fix for the Windows version, and also added similar code
for OS/2, MSDOS, PDOS, MVS and CMS and hopefully I will have
an official release soon which will presumably be the first C90
compliant public domain (explicit notice, and not GPL etc) C
runtime library for Win32.
In the meantime, here's the slightly modified assembler:
http://pdos.cvs.sourceforge.net/viewvc/pdos/pdos/pdpclib/winsupa.asm?view=markup
BFN. Paul.