Exception propagation in C

[James Harris]

Below is a suggestion for handling - more accurately, propagating -
exceptions in C. It is intended for overall program performance though
it goes against a preference of many including me. Can anyone come up
with a faster or better way to propagate exceptions? What other ways
are there to handle exceptions in C? What do you do? (I tend to detect
exceptions where they occur by checking return values but I have never
had a good way in C to get them to propagate.)

The issue is the desire to have exception propagation using *standard*
C. I'm not talking principally about detecting exceptions but of a way
for nested functions to signal an exception and have that exception
propagate back along the call chain to a function that is set up to
handle it.

* Option 1. Use setjmp/longjmp. Not too good as, to be compatible,
they need to be used in an if or a switch statement IIRC which can be
inconvenient. Don't they also need a separate jump buffer for each
active setjmp? I have used them in the past but found them awkward.

* Option 2. Traditional exception handling. As used in other languages
this unwinds the stack of function calls looking for a handler. I
cannot think of a way to implement that in C. Furthermore, it might be
slow for two reasons: 1) the need to push and pop sufficient context
for the exception handler to recognise where a given exception should
be caught, 2) the consequent loss of sync between the stack and the
CPU's branch prediction for returns. (A return would go back to
somewhere other than where it was called from potentially impacting
the performance of the rest of the calls on the call stack.) The
latter is OK if exceptions are infrequent but is not generally
applicable.

* Option 3. The function detecting an exception creates an exception
object and sets an exception type in a thread-specific variable. Any
calling functions, after each call (or group of calls) test that
variable. If non-zero they jump to a local exception handler if there
is one or jump to a return from the current function if not. This does
involve a test and branch for each call but that is very fast (fusible
and single cycle) and it avoids the need for exception handling
context to be pushed and popped. It would also preserve the branch
prediction of calls and returns.

Unfortunately option 3 would mean use of goto. I haven't used goto in
C code for many years and don't like the idea of starting now but I
can't think of anything quicker. In C this could be

temp = func();
if (thread_excep) goto exception_handler;
x = temp;

In x86-32 this should become something like

cmp [thread_excep], 0
jne exception_handler
mov x, eax

If set up to do so the exception handler would handle the exception
itself and clear thread_excep. If not or if it decided to propagate
the exception it would return to its caller with the thread_excep word
still holding the exception value. Or it could alter thread_excep and
return.

There are subtleties but option 3 is the basic idea. Anyone know of
something better?

James

[jacob navia]

Le 29/12/11 18:52, James Harris a écrit :

> * Option 2. Traditional exception handling. As used in other languages
> this unwinds the stack of function calls looking for a handler. I
> cannot think of a way to implement that in C.

This can be done in C of course. Look at the exception handling code in
gcc. It implements an abstract machine that gets opcodes from a series
of tables generated by the compiler.

For more information look at the DWARF debugging info specs, where
the stack unwinding machine is described in detail. These are the
specs, not to be confused with the implementation of those specs
by gcc (the only compiler that uses this monster) that are
slightly different.

You should also look at the code of gcc. Specifically look in the
"except.c" file and in the unwind-xx-fde files. There you will
find the implementation

> Furthermore, it might be
> slow for two reasons: 1) the need to push and pop sufficient context
> for the exception handler to recognise where a given exception should
> be caught,

That is the exception handling object. But before you go any further,
think a bit:

SPEED IS NOT IMPORTANT HERE

The program has just crashed, and most often than not this is NOT the
regular path of the application. So, stop worrying about speed now.

2) the consequent loss of sync between the stack and the
> CPU's branch prediction for returns.

So what? The pipeline will be flushed, but that is not important at all
since (I repeat) SPEED IS NOT IMPORTANT HERE.

> (A return would go back to
> somewhere other than where it was called from potentially impacting
> the performance of the rest of the calls on the call stack.) The
> latter is OK if exceptions are infrequent but is not generally
> applicable.
>

Look. This is a quite researched field, and several full
implementations exist. Before starting spewing ideas that
are nothing more than a few hours of thought

1) Read the literature and documentations of the existing
implementations. Do NOT think you will run into the
GREAT idea that all others didn't see.

2) Study the code of gcc and the open implementations of the
exception handling implementations published by Microsoft and
Apple. (Apple is mostly gcc anyway).

3) Choose one model implementation and implement that. See how
it works within your context.

If you are writing an OS forget about C and remember that your OS
should support several languages smoothly, not only just C.

[BGB]

On 12/29/2011 11:52 AM, James Harris wrote:
> Below is a suggestion for handling - more accurately, propagating -
> exceptions in C. It is intended for overall program performance though
> it goes against a preference of many including me. Can anyone come up
> with a faster or better way to propagate exceptions? What other ways
> are there to handle exceptions in C? What do you do? (I tend to detect
> exceptions where they occur by checking return values but I have never
> had a good way in C to get them to propagate.)
>
> The issue is the desire to have exception propagation using *standard*
> C. I'm not talking principally about detecting exceptions but of a way
> for nested functions to signal an exception and have that exception
> propagate back along the call chain to a function that is set up to
> handle it.
>

it depends on what exactly one means by "standard".
I will assume here "can work with a typical C compiler on a typical
target without the need/ability to modify said compiler to make it work".

> * Option 1. Use setjmp/longjmp. Not too good as, to be compatible,
> they need to be used in an if or a switch statement IIRC which can be
> inconvenient. Don't they also need a separate jump buffer for each
> active setjmp? I have used them in the past but found them awkward.
>

yes. however, partially useful may be the ideas/workings of
setjmp/longjmp, rather than the specific interface.

main thing it does:
captures call state;
restores call state upon invocation.

one could either wrap setjmp/longjmp, or create something similar, which
implements a nested exception handler.

> * Option 2. Traditional exception handling. As used in other languages
> this unwinds the stack of function calls looking for a handler. I
> cannot think of a way to implement that in C. Furthermore, it might be
> slow for two reasons: 1) the need to push and pop sufficient context
> for the exception handler to recognise where a given exception should
> be caught, 2) the consequent loss of sync between the stack and the
> CPU's branch prediction for returns. (A return would go back to
> somewhere other than where it was called from potentially impacting
> the performance of the rest of the calls on the call stack.) The
> latter is OK if exceptions are infrequent but is not generally
> applicable.
>

IRL, this is not likely to actually make all that big of a difference.
the CPU doesn't do any sort of large-scale code analysis, but only
really tends to predict/pipeline within a fairly small context (the next
several instructions).
after the (probably reasonably minor) cost of the non-local jump/...,
there is unlikely to be any further costs.


as for implementing in C:
one option is, of-course, to fold the try, catch, and finally (if
applicable) blocks into their own functions, and then handle the whole
thing something like:
fooTryCatchFinally(myapp_try, myapp_catch, myapp_finally);

but, this interface is awkward (I have tried it).


also possibly workable:
if(!fooBeginTry())
{
try-block stuff...
fooEndTry(); //unregister handler
}else
{
exception-handling stuff...
}

so, the "fooBeginTry()" registers the exception handler frame (similar
to a "setjmp()"), returning 0 for normal execution, and 1 if an
exception has taken place.

"fooEndTry()", of course, unregisters the exception handler frame (it is
not necessary if an exception is thrown before it is reached).

handling could be something like:
FooExceptionObject *ex;
ex=fooGetExceptionObject();
if(fooExceptionIs(ex, "MyApp_SomeException"))
{
...
}
...
finally stuff...
...
fooThrow(ex); //continue unwinding
//otherwise, one just falls out the end and continues executing


note that almost invariably one will end up either dealing with OS
specific mechanisms or ASM in terms of implementing the mechanism itself
(or faking it with "longjmp" or similar).

> * Option 3. The function detecting an exception creates an exception
> object and sets an exception type in a thread-specific variable. Any
> calling functions, after each call (or group of calls) test that
> variable. If non-zero they jump to a local exception handler if there
> is one or jump to a return from the current function if not. This does
> involve a test and branch for each call but that is very fast (fusible
> and single cycle) and it avoids the need for exception handling
> context to be pushed and popped. It would also preserve the branch
> prediction of calls and returns.
>

TLS variables are not standard C (but are supported by MSVC and GCC).
potentially though one could implement them as status-getter function
calls, which is allowed in standard C.

if(fooGetExceptionStatus())
...

> Unfortunately option 3 would mean use of goto. I haven't used goto in
> C code for many years and don't like the idea of starting now but I
> can't think of anything quicker. In C this could be
>
> temp = func();
> if (thread_excep) goto exception_handler;
> x = temp;
>
> In x86-32 this should become something like
>
> cmp [thread_excep], 0
> jne exception_handler
> mov x, eax
>

don't need goto's.

more so, assuming one had TLS variables, one can't typically access them
in a single instruction.

more typically it is some magic like:
mov ecx, [fs:magicOffset]
mov edx, [__myVariable_index]
mov eax, [ecx+edx*4]

and eax, eax
jnz ...


on some systems, they amount to an internal function call.

> If set up to do so the exception handler would handle the exception
> itself and clear thread_excep. If not or if it decided to propagate
> the exception it would return to its caller with the thread_excep word
> still holding the exception value. Or it could alter thread_excep and
> return.
>

unclear how this would work exactly.

[BGB]

On 12/29/2011 12:40 PM, jacob navia wrote:
> Le 29/12/11 18:52, James Harris a écrit :
>> * Option 2. Traditional exception handling. As used in other languages
>> this unwinds the stack of function calls looking for a handler. I
>> cannot think of a way to implement that in C.
>
> This can be done in C of course. Look at the exception handling code in
> gcc. It implements an abstract machine that gets opcodes from a series
> of tables generated by the compiler.
>
> For more information look at the DWARF debugging info specs, where
> the stack unwinding machine is described in detail. These are the
> specs, not to be confused with the implementation of those specs
> by gcc (the only compiler that uses this monster) that are
> slightly different.
>
> You should also look at the code of gcc. Specifically look in the
> "except.c" file and in the unwind-xx-fde files. There you will
> find the implementation
>

simpler is, of course, to do it more like Win32 SEH, which basically
just captures the state (stack position, registers, ...) at each handler
frame (state is captured, linked into a linked-list, and unlinked on
return).

in this case, passing control to a frame essentially restores the state
at this point, and invokes the handler logic.


proper frame unwinding (like used with GCC and Win64) does work, and is
a little cheaper (when no exceptions are thrown), but is a bit more
complex and requires compiler support (debug-info, or, in the case of
Win64, specially-formed frame prologues/epilogues and some special info
located in a table within a special section).


I had written some (never really used) ideas for my own code-generators,
namely using a prologue/epilogue system similar to Win64 and having
linked annotation metadata (exploiting the use of multi-byte NOP
instructions, which can also be used to encode pointers to things).

more practically though, I just stuck with a SEH-like system, since then
the unwinder code does not have to know anything about how to unwind the
stack or similar (or cross language/compiler boundaries), it just
saves/restores whatever.

>> Furthermore, it might be
>> slow for two reasons: 1) the need to push and pop sufficient context
>> for the exception handler to recognise where a given exception should
>> be caught,
>
> That is the exception handling object. But before you go any further,
> think a bit:
>
> SPEED IS NOT IMPORTANT HERE
>
> The program has just crashed, and most often than not this is NOT the
> regular path of the application. So, stop worrying about speed now.
>

yep, agreed.
despite some people using exceptions as a control-flow feature, this
isn't really their intended or ideal use.

> 2) the consequent loss of sync between the stack and the
>> CPU's branch prediction for returns.
>
> So what? The pipeline will be flushed, but that is not important at all
> since (I repeat) SPEED IS NOT IMPORTANT HERE.
>

I think he was expecting something like "all branch predictions for all
subsequent returns will fail", which will not actually happen (given how
the CPU works).

it will flush the pipeline, maybe effecting several instructions and
costing several clock cycles, but really?... a mispredicted "if()", a
call through a function pointer, or invoking a "switch()" which uses a
jump-table, all have the same sorts of costs here.

>> (A return would go back to
>> somewhere other than where it was called from potentially impacting
>> the performance of the rest of the calls on the call stack.) The
>> latter is OK if exceptions are infrequent but is not generally
>> applicable.
>>
>
> Look. This is a quite researched field, and several full
> implementations exist. Before starting spewing ideas that
> are nothing more than a few hours of thought
>
> 1) Read the literature and documentations of the existing
> implementations. Do NOT think you will run into the
> GREAT idea that all others didn't see.
>
> 2) Study the code of gcc and the open implementations of the
> exception handling implementations published by Microsoft and
> Apple. (Apple is mostly gcc anyway).
>
> 3) Choose one model implementation and implement that. See how
> it works within your context.
>

yep.

> If you are writing an OS forget about C and remember that your OS
> should support several languages smoothly, not only just C.
>

this doesn't entirely follow.

ideally, one can have an interface which works fairly well for whatever
languages work on the system.

however, sadly, IRL it is often not so nice:
typically language-specific exceptions and OS exceptions are disjoint.

say, Win32 SEH vs C++ or Java exceptions, ...
typical result is then that they don't really cleanly cross language
boundaries or show up in a "sane" manner (things don't necessarily
unwind in a sane order, often one system is invisible to another, ...).

at least on Win64, IIRC Win64 SEH / MSVC++ / .NET all use the same basic
mechanism.

typically, this means some amount of ugliness to deal with these cases.


luckily, at least, C is sort of a "least common denominator".

better would probably be a solid ABI-level mechanism (which applies to C
and any other language which is compiled to native code on the OS).

but, alas...

[Kaz Kylheku]

On 2011-12-29, James Harris <james.h...@googlemail.com> wrote:
> Below is a suggestion for handling - more accurately, propagating -
> exceptions in C. It is intended for overall program performance though
> it goes against a preference of many including me. Can anyone come up
> with a faster or better way to propagate exceptions? What other ways
> are there to handle exceptions in C? What do you do? (I tend to detect
> exceptions where they occur by checking return values but I have never
> had a good way in C to get them to propagate.)
>
> The issue is the desire to have exception propagation using *standard*
> C. I'm not talking principally about detecting exceptions but of a way
> for nested functions to signal an exception and have that exception
> propagate back along the call chain to a function that is set up to
> handle it.
>
> * Option 1. Use setjmp/longjmp. Not too good as, to be compatible,
> they need to be used in an if or a switch statement IIRC which can be
> inconvenient. Don't they also need a separate jump buffer for each
> active setjmp? I have used them in the past but found them awkward.
>
> * Option 2. Traditional exception handling. As used in other languages
> this unwinds the stack of function calls looking for a handler. I
> cannot think of a way to implement that in C.

Option 2 can be done using Option 1, with some macros.

It can be be done and I have done it, as well as others. Years ago I wrote an
exception handling module that came to be used in Ethereal (now called
Wireshark) which is written in C. Maybe that code is still used. If a packet
is captured by Wireshark which is truncated (does not parse properly according
to its protocol) the packet dissectors throw an exception.

More recently, I made setjmp-based exception handling in a language
called TXR, more sophisticated than that old one.

See here:

http://www.kylheku.com/cgit/txr/tree/unwind.c
http://www.kylheku.com/cgit/txr/tree/unwind.h

This can't be easily lifted into other C programs, though, because
of the reliance on the "val" type, which is a dynamically typed value.

> the need to push and pop sufficient context
> for the exception handler to recognise where a given exception should
> be caught, 2) the consequent loss of sync between the stack and the
> CPU's branch prediction for returns.

Any exception handling that isn't integrated into your compiler is going to be
slow in some way compared to a "state of the art" implementation that is
properly integrated into a language. That is almost a given.

> * Option 3. The function detecting an exception creates an exception
> object and sets an exception type in a thread-specific variable. Any
> calling functions, after each call (or group of calls) test that
> variable.

Testing a global error status variable is not exception handling,
even if you call that variable "current_exception" or whatever
gives you a feeling that the approach is "exceptional".

Exception handling was invented specifically to avoid crufty
code like this.

It becomes an exception handling strategy if you can hide those tests from the
programmer (i.e. have the compiler generate them).

> If non-zero they jump to a local exception handler if there
> is one or jump to a return from the current function if not. This does
> involve a test and branch for each call but that is very fast (fusible
> and single cycle) and it avoids the need for exception handling
> context to be pushed and popped. It would also preserve the branch
> prediction of calls and returns.

But, at least, functions which are not interested in exception handling do not
have to push and pop contexts (unless they have local resources that need
unwinding!)

Under Option 3, all functions have to cooperate, because any normal return is
suspected of occuring in an exceptional situation. If an exceptional return is
happening, then subsequent statements in a function must not be executed. Every
function must check and return.

This is likely going to be a loser in the average cases, not to mention a
nightmare to maintain.

Furthermore, your scheme won't work when the call stack contains frames from
third-party libraries, even if those libraries don't have any local resources
that need cleanup. Longjmp-based exceptions can work through third party call
frames when those frames have no local resources that must be cleaned up.

In any good exception handling implementation, functions that are not involved
with exception handling or unwinding do not "pay" any cost.

The tradeoff is played out between the cost of searching for the exit point
and doing the control transfer, and the cost of setting up a catch or unwind
handler.

In general, if you make it very cheap to set up the catch, then the machine has
to work harder in processing a throw, to identify where to go and to restore
the state properly.

> There are subtleties but option 3 is the basic idea. Anyone know of
> something better?

I can't think of anything that isn't better.

Oh, here is one: it's not worse than a big hammer on a spring popping out of
your computer's case and whacking you on the head when an error occurs!

The really silly thing is that you're worried about shaving machine cycles on
comparisons or branch prediction, but you're overlooking what it costs to do a
thread-local lookup, which this scheme requires at every function call
boundary, just in case an exception is happening.

(You can pass the address of that variable as an extra argument among your
functions wherever that is feasible, I suppose: yet more cruft.)

[Rod Pemberton]

"James Harris" <james.h...@googlemail.com> wrote in message
news:99fbedd4-b37f-4ced...@m7g2000vbc.googlegroups.com...

> Below is a suggestion for handling - more accurately, propagating -
> exceptions in C. It is intended for overall program performance though
> it goes against a preference of many including me. Can anyone come up
> with a faster or better way to propagate exceptions? What other ways
> are there to handle exceptions in C? What do you do? (I tend to detect
> exceptions where they occur by checking return values but I have never
> had a good way in C to get them to propagate.)
>

I dropped c.l.c. Please don't add c.l.c. back while
my comments are included in a reply ...

> The issue is the desire to have exception propagation using *standard*
> C. I'm not talking principally about detecting exceptions but of a way
> for nested functions to signal an exception and have that exception
> propagate back along the call chain to a function that is set up to
> handle it.
>

C is a single thread. Exceptions disrupt the control-flow of C. So, I
wouldn't expect a truly elegant solution ...

I'd look at these:

1) custom assembly
2) setjmp and longjmp
3) check status and exit from procedures repeatedly until exception can be
handled
4) check status and call handler prior to exiting a procedure

Good and bad:

1) is non-portable
2) is somewhat portable or easily fixable
3) can skip completion of an in-process procedure requiring recalling or
restarting them
4) can a bit of code and may still disrupt execution

> * Option 1. Use setjmp/longjmp. Not too good as, to be compatible,
> they need to be used in an if or a switch statement IIRC which can be
> inconvenient. Don't they also need a separate jump buffer for each
> active setjmp? I have used them in the past but found them awkward.

...

> * Option 2. Traditional exception handling. As used in other languages
> this unwinds the stack of function calls looking for a handler.

How does it "unwind"? By tracing stackframes or by exiting procedures?

> I cannot think of a way to implement that in C.

Tracing stackframes would result in non-portable or non-standard C code. C
is not required to use a stack.

Checking a status prior to exiting a procedure or post exit can be done
though. Checking prior to exit has the issues I mentioned above for my #3
and #4. I didn't mention checking when returning from calling a procedure
above. It should have the same or similar issues.

> Furthermore, it might be
> slow for two reasons: 1) the need to push and pop sufficient context
> for the exception handler to recognise where a given exception should
> be caught,

This seems to be recreating setjmp and longjmp except for an exception
handler ...

> 2) the consequent loss of sync between the stack and the
> CPU's branch prediction for returns. (A return would go back to
> somewhere other than where it was called from potentially impacting
> the performance of the rest of the calls on the call stack.) The
> latter is OK if exceptions are infrequent but is not generally
> applicable.
>

...

> * Option 3. The function detecting an exception creates an exception
> object and sets an exception type in a thread-specific variable. Any
> calling functions, after each call (or group of calls) test that
> variable.

I think that is describing my #3.

> If non-zero they jump to a local exception handler if there
> is one or jump to a return from the current function if not.

A local exception handler can repeat a large amount of code.

> This does
> involve a test and branch for each call but that is very fast (fusible
> and single cycle) and it avoids the need for exception handling
> context to be pushed and popped. It would also preserve the branch
> prediction of calls and returns.
>

...

> Unfortunately option 3 would mean use of goto. I haven't used goto in
> C code for many years and don't like the idea of starting now but I
> can't think of anything quicker. In C this could be
>

A goto with local handling is quick, but it:

1) prevents restructuring the code
2) may skip executing some code due to goto-ing the exception
3) may require code duplication for the local handler

> temp = func();
> if (thread_excep) goto exception_handler;
> x = temp;
>
> In x86-32 this should become something like
>
> cmp [thread_excep], 0
> jne exception_handler
> mov x, eax
>

...

> If set up to do so the exception handler would handle the exception
> itself and clear thread_excep. If not or if it decided to propagate
> the exception it would return to its caller with the thread_excep word
> still holding the exception value. Or it could alter thread_excep and
> return.
>
> There are subtleties but option 3 is the basic idea. Anyone know of
> something better?
>

In pseudo-C code, you might try something like the following naive
implementation, which would be my #4:

int exception=0; /* file scope or "global" */

struct exceptions
{
int condition;
char data[];
}

void exception_handler(struct exceptions)
{
exception=0; /* clear exception state */
/* if need be, you could stack and unstack state here ... */

/* hopefully, only a few exceptions are required */
switch(exceptions.condition)
condition 1:
do stuff;
break;
condition 2:
do stuff;
break;
default:
do stuff;
break;
}

blah some_proc(blah blah, blah blahblah)
{

bunch of code ...

/* in each procedure */
/* wherever a condition can be detected */
if (condition)
{
exception=1;
exceptions.condition = 0x02;
memcpy(exceptions.data,error_data);
}

more code ...

/* just prior to return() */
/* in any procedure that can set an exception */
if(exception)
exception_handler(exceptions);

return();
}

It doesn't skip executing any code. Since the handler is it's own
procedure, it eliminates duplicating much code. It doesn't requiring saving
current execution context and and restarting. It doesn't require recalling
a bunch of nested procedures after restoring data state. It doesn't use
goto's.


Rod Pemberton

[BartC]

"James Harris" <james.h...@googlemail.com> wrote in message
news:99fbedd4-b37f-4ced...@m7g2000vbc.googlegroups.com...

> * Option 1. Use setjmp/longjmp. Not too good as, to be compatible,

> * Option 2. Traditional exception handling. As used in other languages

> * Option 3. The function detecting an exception creates an exception

What sort of syntax did you have in mind to make use of this:

Presumably something will be needed at the top level, to let the exception
handler know where to return to. And something in the lower level to invoke
the exception (I assume this will be the result of some test or failure in
software).

I don't have any ideas, just interested in how this might be done myself.
Particularly of what clean-up is (or isn't) done of the intermediate
functions which may have allocated or opened resources that need taking care
of.

> Unfortunately option 3 would mean use of goto.

I suspect some of the solutions will involve a lot worse than goto.

(FWIW I achieve something similar (error recovery) by using inline assembly
then:

o Saving Stack, Frame registers, and storing a recovery label address (in
this same function) in a global variable

o Starting the process (calling the functions) where the error might occur

o When the error occurs, jumping to the recovery point (more inline
assembly), where the Stack and Frame registers are restored.

No resources are released here either, but the program is usually about to
end anyway so is not critical.)

--
Bartc

[jacob navia]

Le 29/12/11 22:02, Kaz Kylheku a écrit :

> On 2011-12-29, James Harris<james.h...@googlemail.com> wrote:

>> There are subtleties but option 3 is the basic idea. Anyone know of
>> something better?
>
> I can't think of anything that isn't better.
>

> The really silly thing is that you're worried about shaving machine cycles on
> comparisons or branch prediction, but you're overlooking what it costs to do a
> thread-local lookup, which this scheme requires at every function call
> boundary, just in case an exception is happening.

This is a consequence of ignoring the literature, not studying the
subject matter and think that you are going to find the smart idea
that all others missed.

The crucial point (that you mention) is that needs to be done in the
compiler, not in each user program. Since he has chosen a "lightweight"
C compiler (gcc) he doesn't need to do anything if he uses the one
in gcc, but probably he is not aware of that fact, or, for unknown
reasons, he doesn't want to use gcc's.

[James Harris]

On Dec 29, 7:07 pm, BGB <cr88...@hotmail.com> wrote:
> On 12/29/2011 11:52 AM, James Harris wrote:

...

> > The issue is the desire to have exception propagation using *standard*
> > C. I'm not talking principally about detecting exceptions but of a way
> > for nested functions to signal an exception and have that exception
> > propagate back along the call chain to a function that is set up to
> > handle it.
>
> it depends on what exactly one means by "standard".
> I will assume here "can work with a typical C compiler on a typical
> target without the need/ability to modify said compiler to make it work".

Yes, that's the intention.

...

> > * Option 3. The function detecting an exception creates an exception
> > object and sets an exception type in a thread-specific variable. Any
> > calling functions, after each call (or group of calls) test that
> > variable. If non-zero they jump to a local exception handler if there
> > is one or jump to a return from the current function if not. This does
> > involve a test and branch for each call but that is very fast (fusible
> > and single cycle) and it avoids the need for exception handling
> > context to be pushed and popped. It would also preserve the branch
> > prediction of calls and returns.
>
> TLS variables are not standard C (but are supported by MSVC and GCC).
> potentially though one could implement them as status-getter function
> calls, which is allowed in standard C.

I can't see TLS as C or not C. Wouldn't TLS be a characteristic of the
environment in which the C code runs?

...

> more so, assuming one had TLS variables, one can't typically access them
> in a single instruction.

I was thinking of thread_excep being at a fixed location so it could
literally be checked with one instruction:

cmp [thread_excep], 0

> more typically it is some magic like:
> mov ecx, [fs:magicOffset]
> mov edx, [__myVariable_index]
> mov eax, [ecx+edx*4]
>
> and eax, eax
> jnz ...
>
> on some systems, they amount to an internal function call.

As shown, with static linking

if (thread_excep)

should compile to

cmp [thread_excep], 0

> > If set up to do so the exception handler would handle the exception
> > itself and clear thread_excep. If not or if it decided to propagate
> > the exception it would return to its caller with the thread_excep word
> > still holding the exception value. Or it could alter thread_excep and
> > return.
>
> unclear how this would work exactly.

I'll try to explain what I have in mind. There would be a machine word
(which I've called thread_excep above) which would be initialised to
zero. A non-zero value would be set in it to indicate that an
exception had occurred. I'm not sure how best to use the word but one
option is to use different bits to indicate exception classes: data
exception, index exception, io exception, user exception etc.

In the main body of the code exceptions would be detected by

if (thread_excep) goto handler_2;

Within a function there would be as many handlers as are needed. Say
we wanted to deal with only index-type exceptions in this module. We
could code

handler_2:
if (thread_excep & EXCEP_INDEX) {
/* Deal with an index exception */
thread_excep &= ~EXCEP_INDEX;
/* Delete the exception object */
/* Next step: return, go on to next iteration, whatever */
}
else return 0;

In the else path the value of thread_excep has not been changed so
whatever called this routine will get a chance to handle the
exception(s).

Does that make sense? I'd welcome corrections or improvements.

James

[James Harris]

On Dec 30, 1:07 am, "BartC" <b...@freeuk.com> wrote:
> "James Harris" <james.harri...@googlemail.com> wrote in message

>
> news:99fbedd4-b37f-4ced...@m7g2000vbc.googlegroups.com...
>
> > * Option 1. Use setjmp/longjmp. Not too good as, to be compatible,
> > * Option 2. Traditional exception handling. As used in other languages
> > * Option 3. The function detecting an exception creates an exception
>
> What sort of syntax did you have in mind to make use of this:

No special syntax. Just standard C.

> Presumably something will be needed at the top level, to let the exception
> handler know where to return to.

Not as such. Each routine would decide what to do. In the general case
there would need to be some default response at the top level.

> And something in the lower level to invoke
> the exception (I assume this will be the result of some test or failure in
> software).

Yes, the lower level that detected the exception would create an
object to describe it. I've no idea yet what format the exception
object would take. I need to find out what others have done.

> I don't have any ideas, just interested in how this might be done myself.
> Particularly of what clean-up is (or isn't) done of the intermediate
> functions which may have allocated or opened resources that need taking care
> of.

The beauty of it is that each function would be responsible for
cleaning up its own resources.

No, the main beauty of it is that it will work with different
languages and even between languages. That's particularly important
for what I have in mind.

Check out the example code I posted in response to BGB.

> > Unfortunately option 3 would mean use of goto.
>
> I suspect some of the solutions will involve a lot worse than goto.

:-) I'm beginning to see this is very true.

James

[James Harris]

On Dec 29, 9:02 pm, Kaz Kylheku <k...@kylheku.com> wrote:
> On 2011-12-29, James Harris <james.harri...@googlemail.com> wrote:

...

> http://www.kylheku.com/cgit/txr/tree/unwind.c
> http://www.kylheku.com/cgit/txr/tree/unwind.h
>
> This can't be easily lifted into other C programs, though, because
> of the reliance on the "val" type, which is a dynamically typed value.

Nevertheless, ISTM that code is very good.

...

> Testing a global error status variable is not exception handling,
> even if you call that variable "current_exception" or whatever
> gives you a feeling that the approach is "exceptional".
>
> Exception handling was invented specifically to avoid crufty
> code like this.
>
> It becomes an exception handling strategy if you can hide those tests from the
> programmer (i.e. have the compiler generate them).

We have to work with what we have. I'm not looking to alter a language
or a compiler.

In fact a great thing about the scheme, IMHO, is that it will work
between languages. A routine in language A could call a routine in
language B ... could call a routine in language Z. Each could work
with the exception indication of the others. None would have to know
what other languages were involved.

> > If non-zero they jump to a local exception handler if there
> > is one or jump to a return from the current function if not. This does
> > involve a test and branch for each call but that is very fast (fusible
> > and single cycle) and it avoids the need for exception handling
> > context to be pushed and popped. It would also preserve the branch
> > prediction of calls and returns.
>
> But, at least, functions which are not interested in exception handling do not
> have to push and pop contexts (unless they have local resources that need
> unwinding!)

Right, there is no pushing and popping of contexts.

> Under Option 3, all functions have to cooperate, because any normal return is
> suspected of occuring in an exceptional situation. If an exceptional return is
> happening, then subsequent statements in a function must not be executed. Every
> function must check and return.

Well, the exception status could be checked whenever needed, probably
after each call but if appropriate it could be checked after a group
of calls as it would be 'sticky'.

Yes, every function would have to check.

> This is likely going to be a loser in the average cases, not to mention a
> nightmare to maintain.

It is much faster than pushing and popping contexts.

> Furthermore, your scheme won't work when the call stack contains frames from
> third-party libraries, even if those libraries don't have any local resources
> that need cleanup.  Longjmp-based exceptions can work through third party call
> frames when those frames have no local resources that must be cleaned up.

True. The exception status would pass through such code, though.

...

> In general, if you make it very cheap to set up the catch, then the machine has
> to work harder in processing a throw, to identify where to go and to restore
> the state properly.

True. IIRC there's a good write-up on the tradeoffs in books such as
Programming Language Pragmatics and Principles of Programming
Languages.

...

> The really silly thing is that you're worried about shaving machine cycles on
> comparisons or branch prediction, but you're overlooking what it costs to do a
> thread-local lookup, which this scheme requires at every function call
> boundary, just in case an exception is happening.

I don't think you understand what I have in mind. If I have it right
checking the exception indication would be just one instruction
testing a word that is probably cached in L1. It could hardly be
faster.

James

[Eric Sosman]

On 12/29/2011 12:52 PM, James Harris wrote:
> Below is a suggestion for handling - more accurately, propagating -
> exceptions in C. It is intended for overall program performance though
> it goes against a preference of many including me. Can anyone come up
> with a faster or better way to propagate exceptions? What other ways
> are there to handle exceptions in C? What do you do? (I tend to detect
> exceptions where they occur by checking return values but I have never
> had a good way in C to get them to propagate.)

Maybe this is clear to you already, but the words you use leave
me in some doubt: In pondering the performance of an exception scheme
one should not worry about how fast exceptions are handled, but about
how fast non-exceptions are not-handled. If you make a million calls
to malloc() and get an exception on one of them, it's the performance
of the other 999,999 that you need to consider.

They're called "exceptions," not "usuals."

> The issue is the desire to have exception propagation using *standard*
> C. I'm not talking principally about detecting exceptions but of a way
> for nested functions to signal an exception and have that exception
> propagate back along the call chain to a function that is set up to
> handle it.
>
> * Option 1. Use setjmp/longjmp. Not too good as, to be compatible,
> they need to be used in an if or a switch statement IIRC which can be
> inconvenient. Don't they also need a separate jump buffer for each
> active setjmp? I have used them in the past but found them awkward.

This is how all the "pure C" exception frameworks I've seen have
worked. Yes, it requires a dedicated jmp_buf for each target. It
can also require that some variables in the setjmp() caller be made
`volatile', which is easy to forget to do *and* may prevent some of
the optimizer's more imaginative transformations -- that is, it may
slow down the more important, non-exception path.

> * Option 2. Traditional exception handling. As used in other languages
> this unwinds the stack of function calls looking for a handler. I
> cannot think of a way to implement that in C. Furthermore, it might be
> slow for two reasons: 1) the need to push and pop sufficient context
> for the exception handler to recognise where a given exception should
> be caught, 2) the consequent loss of sync between the stack and the
> CPU's branch prediction for returns. (A return would go back to
> somewhere other than where it was called from potentially impacting
> the performance of the rest of the calls on the call stack.) The
> latter is OK if exceptions are infrequent but is not generally
> applicable.

The only portable ways to unwind the stack are to return or to
use longjmp(). With the latter there's no way to unwind the stack
frames one by one, nor to inspect what's being unwound: You just
longjmp() to the next-higher handler, do whatever local cleanup's
needed, longjmp() again to a handler higher still, and so on until
somebody's local cleanup says "This exception has propagated far enough;
I'm absorbing it."

Again, I feel you're worrying about performance in a situation
where performance is not very important.

> * Option 3. The function detecting an exception creates an exception
> object and sets an exception type in a thread-specific variable. Any
> calling functions, after each call (or group of calls) test that
> variable. If non-zero they jump to a local exception handler if there
> is one or jump to a return from the current function if not. This does
> involve a test and branch for each call but that is very fast (fusible
> and single cycle) and it avoids the need for exception handling
> context to be pushed and popped. It would also preserve the branch
> prediction of calls and returns.

Isn't this just an `errno' equivalent, with a pointer instead
of an `int'? In "The Standard C Library," PJ Plauger writes

Nobody likes errno or the machinery that it implies. I can't
recall anybody defending this approach to error reporting,
not in two dozen or more meetings of X3J11, the committee
that developed the C Standard. Several alternatives were
proposed over the years. At least one faction favored simply
discarding errno.

I'd hesitate before imitating an interface nobody likes ...

Also, the performance probably suffers. For all its drawbacks,
the function-returns-a-distinguishable-value-on-failure approach has
the advantage that the value is very likely in a CPU register when
you need to test it, not in a global variable somewhere off in RAM.

> [...] Anyone know of
> something better?

I've seen various exception-like frameworks overlaid on C, some
down-and-dirty, some fairly elaborate. All of them, however useful
for the particular task at hand, suffered from lack of support by
the language itself: There's just no way to tell the compiler that
a function might throw an exception, which means the compiler can't
tell the programmer when he's forgotten to catch one. Somebody opens
a file, obtains values from various accessor functions and writes them
to the file, closes the stream, and returns -- except that one of the
accessors throws an exception that gets caught somewhere way up the
stack, and the file-writing function never closes its stream ...

Errors of this kind can be prevented by programmer discipline,
but I've seen no scheme that will help keep the programmer on the
strait and narrow path. This is sad, because it means an exception
mechanism that was supposed to make the programmer's job easier can
wind up making it more burdensome -- and more perilous.

Exceptions aren't something you can just bolt on to a language.
Well, actually, you *can*: people have bolted them on to C, over and
over again. Yet none of the various schemes has caught on, which I
think should cause you to ponder. Maybe it's because bolting on is
a poor substitute for baking in.

--
Eric Sosman
eso...@ieee-dot-org.invalid

[BGB]

TLS is typically stored separately from typical global variables, and is
generally indicated via the use of special modifiers ("__thread" or
"__declspec(thread)", say, in GCC or MSVC), however, doing so is not
allowed in standard C (these are extensions).

typically, the threading library will also provide function-calls to
get/set TLS variables, which are a little more standard.

>> more so, assuming one had TLS variables, one can't typically access them
>> in a single instruction.
>
> I was thinking of thread_excep being at a fixed location so it could
> literally be checked with one instruction:
>
> cmp [thread_excep], 0
>

if it is a global though, it is not a TLS.

otherwise, I guess it would need to be changed around by the scheduler
or something (register variables with scheduler, and it saves/restores
their values on context switches or something?...).

>> more typically it is some magic like:
>> mov ecx, [fs:magicOffset]
>> mov edx, [__myVariable_index]
>> mov eax, [ecx+edx*4]
>>
>> and eax, eax
>> jnz ...
>>
>> on some systems, they amount to an internal function call.
>
> As shown, with static linking
>
> if (thread_excep)
>
> should compile to
>
> cmp [thread_excep], 0
>

except if it were a proper/traditional TLS variable, which are typically
accessed indirectly (on Windows and Linux, there is a special per-thread
structure, generally accessed via a segment register).


I guess, if one is making an OS, other possibilities also exist (special
magic TLS sections?).

say, if one has a section ".tls" section, where the scheduler
automatically saves/restores the contents of anything stored within
these sections. the advantage could be very fast access to TLS variables
at the cost of potentially more expensive context switches.

>>> If set up to do so the exception handler would handle the exception
>>> itself and clear thread_excep. If not or if it decided to propagate
>>> the exception it would return to its caller with the thread_excep word
>>> still holding the exception value. Or it could alter thread_excep and
>>> return.
>>
>> unclear how this would work exactly.
>
> I'll try to explain what I have in mind. There would be a machine word
> (which I've called thread_excep above) which would be initialised to
> zero. A non-zero value would be set in it to indicate that an
> exception had occurred. I'm not sure how best to use the word but one
> option is to use different bits to indicate exception classes: data
> exception, index exception, io exception, user exception etc.
>

FOURCC?...

'IOX ', 'GPF ', 'PAGE', ...

many systems use objects though, where the object will typically hold
information, such as the location, captured register state, ...

> In the main body of the code exceptions would be detected by
>
> if (thread_excep) goto handler_2;
>
> Within a function there would be as many handlers as are needed. Say
> we wanted to deal with only index-type exceptions in this module. We
> could code
>
> handler_2:

> if (thread_excep& EXCEP_INDEX) {

> /* Deal with an index exception */

> thread_excep&= ~EXCEP_INDEX;

> /* Delete the exception object */
> /* Next step: return, go on to next iteration, whatever */
> }
> else return 0;
>
> In the else path the value of thread_excep has not been changed so
> whatever called this routine will get a chance to handle the
> exception(s).
>
> Does that make sense? I'd welcome corrections or improvements.
>

I don't think flags are the ideal strategy here.
flags don't as easily allow user-defined exceptions, and if one has only
a few bits for a user-defined exception class or similar, then it
creates a bit of a burden WRT avoiding clashes.

one "could" potentially use interned strings and magic index numbers
though, say, 10 or 12 bits being plenty to allow a range of named
exception classes (potentially being keyed via a hash-table or similar).

this is actually how my dynamic type-system works (types are identified
by name, but encoded via hash indices).

technically, one can combine indices with FOURCC values though, since if
one restricts the valid FOURCC values to being ASCII printable
characters, there is a large range of non-ASCII values which can be used
as index values (512M in the range of 0x00000000..0x1FFFFFFF, and 2G in
the range 0x80000000..0xFFFFFFFF, and possibly another 16M in the range
0x7F000000..0x7FFFFFFF).

IIRC, I once devised a TLV format based on this strategy, but I don't
remember ever making use of this (many of my TLV formats use a
variable-length encoding more like that used in the Matroska/MKV format).


or such...

[Scott Wood]

[crosspost snipped]

On 2011-12-29, James Harris <james.h...@googlemail.com> wrote:

> Below is a suggestion for handling - more accurately, propagating -
> exceptions in C. It is intended for overall program performance though
> it goes against a preference of many including me. Can anyone come up
> with a faster or better way to propagate exceptions? What other ways
> are there to handle exceptions in C? What do you do? (I tend to detect
> exceptions where they occur by checking return values but I have never
> had a good way in C to get them to propagate.)

Use C++? :-)

> The issue is the desire to have exception propagation using *standard*
> C. I'm not talking principally about detecting exceptions but of a way
> for nested functions to signal an exception and have that exception
> propagate back along the call chain to a function that is set up to
> handle it.
>
> * Option 1. Use setjmp/longjmp. Not too good as, to be compatible,
> they need to be used in an if or a switch statement IIRC which can be
> inconvenient. Don't they also need a separate jump buffer for each
> active setjmp? I have used them in the past but found them awkward.

Awkward and slow -- need to save state every time you enter a scope that you
*might* need to unwind to.

It's probably the only option for "standard C" -- if that truly is a
constraint you have to live with, I'd just consider exceptions a tool that
is unavailable, except for a few rare cases where it's worth the pain of
setjmp/longjmp.

> * Option 2. Traditional exception handling. As used in other languages
> this unwinds the stack of function calls looking for a handler. I
> cannot think of a way to implement that in C. Furthermore, it might be
> slow for two reasons: 1) the need to push and pop sufficient context
> for the exception handler to recognise where a given exception should
> be caught,

This information does not need to be pushed on the stack -- it is implicit
in the call trace combined with compiler-generated metadata.

There is some lost optimization potential due to additional code flow
possibilities, but you'd have that with explicit error checks as well.
Other than that, this sort of exception handling should be pretty much
cost-free when an exception is not taken.

> 2) the consequent loss of sync between the stack and the
> CPU's branch prediction for returns. (A return would go back to
> somewhere other than where it was called from potentially impacting
> the performance of the rest of the calls on the call stack.) The
> latter is OK if exceptions are infrequent but is not generally
> applicable.

Not a big deal -- you get this every time you context switch too. The
effect should be short-lived, and only affect returns from calls already
made.

> * Option 3. The function detecting an exception creates an exception
> object and sets an exception type in a thread-specific variable. Any
> calling functions, after each call (or group of calls) test that
> variable.

Not really exceptions, as others pointed out. Still, could be a better way
to do explicit error checking. Depending on how efficient thread-local
lookups are in your implementation, might want to pass this as a return
value instead.

> Unfortunately option 3 would mean use of goto. I haven't used goto in
> C code for many years and don't like the idea of starting now but I
> can't think of anything quicker.

Goto is unfairly maligned -- it is useful for error handling in C. It's
widely used for this purpose in the Linux kernel, for example.

-Scott

[s_dub...@yahoo.com]

On Dec 30, 3:38 pm, James Harris <james.harri...@googlemail.com>
wrote:

Yeah, these sound like state flags. The example is rather abstract
from the problem, so it is hard to suggest C or I.

A couple of decades ago I had a commecial dish service that provided
delayed stock data to a terminal with a serial port which could be
connected to a PC and with commercial software, a screen full of
stock, bond, options, mutual funds and news headlines, could be
followed. Of course the PC software was an addon expense with a hefty
monthy charge to follow a handful of additional securities above what
the terminal provided. So I wrote my own in C.

The serial output was the raw data feed at 9600 baud, the data used a
poor man's encryption that wasn't to hard to figure out after snooping
the output. But the output records for a security could vary from
complete (all the fields of the record were transmitted) to sparse
(one or more 'update' fields were in the stream). They were 15-20
minute quote delayed, so the vast majority of records were just
partial updates to things like last quote, new high/low, volume, etc.
And the fields would vary by security type, of course. It was
relatively easy to see that each record transmitted for a 'new'
security started with SOH`byte followed by SecurityNumber`word. The
word was transmitted little endian as I recall. There after, various
control codes, like SOH, told for which field or security type, etc.
The SecurityNumber wasn't the cusip number, it was just a sequence
number and the outfit published a directory every so often showing
which number went to which security, The only help was that
securities were group by number ranges so say bonds were numbers >
50,000. But there were small number ranges inbetween that were
unassigned also.

My program development was stepwise and somewhat haphazard because I
didn't know, I had to deduce, the data stream. I wrote this on and
for a PC XT clone that ran at 10mhz, and its serial port card used an
i8250, that's an early model with a one byte tx/rx buffer, so the
handler can't dally or you lose data.

I wrote the C code in two main chunks, part one to handle the input
stream to a ring buffer, and when the status port said RX register was
empty, part 2 extracted data off the ring buffer like a secondary
stream treating non-printables like SOH (control codes) as state flags
within a for(;;) block - those were control codes for the meaning of
the following data in the stream. That's alot like 'exceptions'
without the overhead in that the input stream is random and not
repeatable. Also a difference is that the control codes were like
trees, one lead to check for a limited number of succession leaves
(controls for individual update fields, if present).

Nowadays you could use threads, one per control codes like SOH to
decode for the next data, but you need a 'sink' thread to catch
undefined or errant data. In my case with the state flags in the
for(;;) block, I discarded errant and unknown records until I figured
out what they meant, then added in a state flag and handler for them.

I also did something silly, which was to do random writes to my data
file on a ramdisk. IIRC is was a 256k ramdisk, so the file held only
a small subset of the securities. Every so often the ram disk data
file would be copied to the hard disk.

It actually worked well with surprisingly few bad records due to data
loss.

I used char flags for the data controls like SOH instead of dealing
with bit shifts to extract control data, so they; ie. SOH_flg, was
either null or non-zero.

Pretty definitely standard C, fwiw, without setjmp, longjmp, raise,
signal or even malloc for that matter.

Good for a chuckle eh?

Steve

[Rod Pemberton]

"James Harris" <james.h...@googlemail.com> wrote in message

news:541969d7-e4ab-43ba...@m4g2000vbc.googlegroups.com...
>
> There would be a machine word [...]

He said: "machine word"... Ok, that screams "volatile" keyword to me.

Are these x86 exceptions or x86 interrupts or events external to C, e.g.,
for your OS, that are being passed to C via a machine word? I suspect they
are, but you probably wanted to keep things topical for c.l.c. ...

At a minimum, I'd suggest reading Eric Sosman's post and also
my post on a.o.d. ;-)

You said Kaz's setjmp/longjmp code looks good to you, so I'd take that as an
intuitive proxy that you should use it. I'm sure others on c.l.c. could
help you fix whatever issues you have with it.


Rod Pemberton

[Rod Pemberton]

<s_dub...@yahoo.com> wrote in message
news:f47cf54b-07c6-4896...@p16g2000yqd.googlegroups.com...
>

<way OT for c.l.c, fine for alt.*>

> A couple of decades ago I had a commecial dish service

> that provided delayed stock data ...
> [...]

> They were 15-20 minute quote delayed

Quote feeds are delayed 15 minutes for non-market makers.

> My program development was stepwise and somewhat haphazard

> because I didn't know, I had to deduce, the [stock quote] data stream.

They probably weren't using NASDAQ or NYSE protocols for the
satellite feed. You weren't subscribed to an official NASDAQ or NYSE quote
feed. If you had been, you could've gotten the protocols for the quote
stream after you (i.e., your company's legal department) signed a
non-disclosure agreement. I worked at a brokerage a decade or so ago
programming on their stock trading application and also software for a
couple of their feeds: SOES and SuperSOES (obsolete). The system had
numerous other computers and related market feeds: quote feeds, market
feeds, alternate markets, ECNs, Instinet, etc. I was so complicated that no
one knew how it was all connected. As part of a merger, new management
wanted to know. It took numerous departments months to obtain all the info
and produce a diagram. Apparently, they thought they could reduce costs by
replacing the system ... laugh.

> I wrote this on and for a PC XT clone that ran at 10mhz,
> and its serial port card used an i8250, that's an early model
> with a one byte tx/rx buffer, so the handler can't dally
> or you lose data.

That's tough to do on a PC ... Market volume was probably much less back
then. I'm assuming this was late 1980's from the PC and dish service. You
probably weren't getting a full quote feed anyway. We had mainframes
handling the quote feed data during early 2000's: four T1s. These T1s were
completely maxed out while the market was open. After a mistake in our
networking department, we got connected to all the T1's available at the
time for NYSE, all fourteen of them ... I think it was NYSE, but it may
have been for NASDAQ. The company relocated around that time to an
adjacent building, and then we started losing quotes ... "Where is the
opening quote for security XYZ?" The extra distance on the rerouted T1s
was causing the data to time out. That was all about a decade ago when
market volume was much lower than it is today.

> IIRC is was a 256k ramdisk, so the file held only
> a small subset of the securities.

Yes ...

We also used a software ramdisk on the mainframes which had some of the
fastest memory manufactured at the time. They were considering hardware
ramdisks, but found out the hardware ramdisks used the same memory as in the
mainframes ... So, they stayed with software ramdisks.

> Every so often the ram disk data
> file would be copied to the hard disk.

Start of day ... end of day. Batch processing, to produce reports required
for NASDAQ and NYSE, ran from about 8PM to 5AM ... Another department
burned it all to laser disks to comply with securities and other Federal
laws. We also had to submit a huge amount of data to another firm that
generated compliance reports.

> Pretty definitely standard C, fwiw, without setjmp, longjmp,
> raise, signal or even malloc for that matter.
>
> Good for a chuckle eh?

Looping works!


Rod Pemberton

[Jens Gustedt]

Am 12/30/2011 10:38 PM, schrieb James Harris:
> On Dec 29, 7:07 pm, BGB <cr88...@hotmail.com> wrote:
>> On 12/29/2011 11:52 AM, James Harris wrote:

>> TLS variables are not standard C (but are supported by MSVC and GCC).

Officially they are since some weeks now, they come with C11

> I can't see TLS as C or not C. Wouldn't TLS be a characteristic of the
> environment in which the C code runs?

according to C11 thread local variables and atomics are part of the
language (new keywords _Thread_local and _Atomic) and functions
associated to threads and atomic operations form optional components
of the C library for hosted environments.

And these tools clearly ease the programming of some try/catch mechnism
with macros in C.

Jens

[BGB]

On 12/31/2011 6:40 AM, Jens Gustedt wrote:
> Am 12/30/2011 10:38 PM, schrieb James Harris:
>> On Dec 29, 7:07 pm, BGB<cr88...@hotmail.com> wrote:
>>> On 12/29/2011 11:52 AM, James Harris wrote:
>
>>> TLS variables are not standard C (but are supported by MSVC and GCC).
>
> Officially they are since some weeks now, they come with C11
>

ok, I hadn't looked much at C11 recently...

who knows how long until it is really well supported though, as it has
been long enough just waiting for MSVC to support C99 stuff...

>> I can't see TLS as C or not C. Wouldn't TLS be a characteristic of the
>> environment in which the C code runs?
>
> according to C11 thread local variables and atomics are part of the
> language (new keywords _Thread_local and _Atomic) and functions
> associated to threads and atomic operations form optional components
> of the C library for hosted environments.
>

and alas they couldn't just reuse GCC's keywords?...
well, in any case, probably macro magic.

> And these tools clearly ease the programming of some try/catch mechnism
> with macros in C.
>

yep.


I had forgot about a trick I had actually used before:
one can fake TLS using a macro wrapping a function call.

say:
int *foo_errno_fcn()
{ ... }

#define foo_errno (*foo_errno_fcn())

granted, this still doesn't allow a single operation to access it (as in
the OP), but really, who really needs to care?...


one case, something was due to me having initially used straightforward
TLS variables, but then running into a case where they "didn't really
work so well" on WinXP (top-level declared TLS variables are fussy /
prone-to-breaking on WinXP and earlier), but a wrapped function call was
more reliable.

not that I think status codes located within TLS are really a sane
exception mechanism (they can be sanely used, but IMO can't be sanely
called exceptions...).


most often, I pass state in passed context structs, but in some cases
have used "context binding" where one may need a TLS variable mostly to
hold a pointer to the context.

nevermind the potential need for TLS to implement things like nested
exceptions.


hmm:
in C one could use macros to be like:
BEGIN_TRY
...
END_TRY
BEGIN_CATCH(FooException, ex)
...
END_CATCH
BEGIN_FINALLY
...
END_FINALLY

nevermind if the syntax is horrid...
(technically, this could probably be implemented via if/else magic).

#define BEGIN_TRY if(fooBeginTry()) {
#define END_TRY fooEndTry();}

#define BEGIN_CATCH(ext, exn) else if(fooCatchException(#ext)) { \
ext exn=fooGetException();
#define END_CATCH }

#define BEGIN_FINALLY else{
#define END_FINALLY fooThrowCurrentException();}

(one may forsake using all caps...).


or such...

[Nick Keighley]

On Dec 31 2011, 5:20 am, "s_dubrov...@yahoo.com"

interesting war story. Um but where are the exceptions? Sounds like a
for(;;) loop with a big switch statement. It's kind of "well,how else
would you write it?"

[James Harris]

On Dec 31 2011, 10:53 am, "Rod Pemberton"

<do_not_h...@noavailemail.cmm> wrote:
> "James Harris" <james.harri...@googlemail.com> wrote in message

...

> > There would be a machine word [...]
>
> He said: "machine word"...  Ok, that screams "volatile" keyword to me.

May be a bad term. I mean a 16-bit word on a 16-bit machine, a 32-bit
word on a 32-bit machine, a 64-bit word on a 64-bit machine. I
sometimes avoid the plain term "word" as it has been somewhat hijacked
by the 16-bit brigade. I come from an older era where the word size
depended on the machine.

> Are these x86 exceptions or x86 interrupts or events external to C, e.g.,
> for your OS, that are being passed to C via a machine word?  I suspect they
> are, but you probably wanted to keep things topical for c.l.c. ...

This fully applies to C (or another language for that matter). The
machine word is one that could be set, altered and cleared all in the
language or in a called module whether in the same language or not.
That said, it could just as easily be modified by the OS or something
else. Perhaps you could think of it and the operations as a protocol.
Whatever raises an exception that it doesn't handle itself sets the
word and returns to its caller. Whatever does handle outstanding
exceptions clears the word. And the word is checked whenever
appropriate.

> At a minimum, I'd suggest reading Eric Sosman's post and also
> my post on a.o.d.  ;-)

LOL! You think I wouldn't read all the replies?

> You said Kaz's setjmp/longjmp code looks good to you, so I'd take that as an
> intuitive proxy that you should use it.  I'm sure others on c.l.c. could
> help you fix whatever issues you have with it.

My comment was a general one. It looked like good code but it's not
what I was looking for. In fact I wasn't looking for a solution as
such but for feedback on my preferred option (which I got). Some good
points were made and I have some replies to get round to making.

James

[Rod Pemberton]

"James Harris" <james.h...@googlemail.com> wrote in message

news:6c6a43c1-53d3-4386...@p42g2000vbt.googlegroups.com...

> On Dec 31 2011, 10:53 am, "Rod Pemberton"
> <do_not_h...@noavailemail.cmm> wrote:
> > "James Harris" <james.harri...@googlemail.com> wrote in message
...

> > There would be a machine word [...]
>
> > He said: "machine word"... Ok, that screams "volatile" keyword to me.
>
> May be a bad term. I mean a 16-bit word on a 16-bit machine, a 32-bit
> word on a 32-bit machine, a 64-bit word on a 64-bit machine. I
> sometimes avoid the plain term "word" as it has been somewhat hijacked
> by the 16-bit brigade. I come from an older era where the word size
> depended on the machine.
>

[ ... continuing the thought ...]

> > Are these x86 exceptions or x86 interrupts or events external to C,
> > e.g., for your OS, that are being passed to C via a machine word?
> > I suspect they are, but you probably wanted to keep things topical
> > for c.l.c. ...
>
> This fully applies to C (or another language for that matter). The
> machine word is one that could be set, altered and cleared all in the
> language or in a called module whether in the same language or not.
> That said, it could just as easily be modified by the OS or something
> else. Perhaps you could think of it and the operations as a protocol.
> Whatever raises an exception that it doesn't handle itself sets the
> word and returns to its caller. Whatever does handle outstanding
> exceptions clears the word. And the word is checked whenever
> appropriate.

Ok, that screams "volatile" keyword to me. Anything modified outside of C
and used by C is unknown to C so needs a volatile so it's not optimized away
or to ensure it's reloaded from memory instead of being cached in a register
since it's value could changed by non-C code ...

> > At a minimum, I'd suggest reading Eric Sosman's post and
> > also my post on a.o.d. ;-)
>
> LOL! You think I wouldn't read all the replies?

No. First, I wasn't sure you were reading from a.o.d. Second, it was just
easier than saying I didn't agree with some comments of other post-ers
without starting an argument with them. My experience with c.l.c people
over the past decade is that many of them are insanely hostile, intolerant
of any criticism, and will argue to their death that they are correct when
in fact they are provably wrong ...


Rod Pemberton

[Scott Wood]

On 2012-01-02, Rod Pemberton <do_no...@noavailemail.cmm> wrote:
> "James Harris" <james.h...@googlemail.com> wrote in message
> news:6c6a43c1-53d3-4386...@p42g2000vbt.googlegroups.com...
>> On Dec 31 2011, 10:53 am, "Rod Pemberton"
>> <do_not_h...@noavailemail.cmm> wrote:
>> > "James Harris" <james.harri...@googlemail.com> wrote in message
> ...

>> > Are these x86 exceptions or x86 interrupts or events external to C,
>> > e.g., for your OS, that are being passed to C via a machine word?
>> > I suspect they are, but you probably wanted to keep things topical
>> > for c.l.c. ...
>>
>> This fully applies to C (or another language for that matter). The
>> machine word is one that could be set, altered and cleared all in the
>> language or in a called module whether in the same language or not.
>> That said, it could just as easily be modified by the OS or something
>> else. Perhaps you could think of it and the operations as a protocol.
>> Whatever raises an exception that it doesn't handle itself sets the
>> word and returns to its caller. Whatever does handle outstanding
>> exceptions clears the word. And the word is checked whenever
>> appropriate.
>
> Ok, that screams "volatile" keyword to me. Anything modified outside of C
> and used by C is unknown to C so needs a volatile so it's not optimized away
> or to ensure it's reloaded from memory instead of being cached in a register
> since it's value could changed by non-C code ...

The relevant factor isn't "C" versus "non-C", it's synchronous versus
asynchronous (or perhaps, outside this instance of the single-threaded C
execution model).

If it's modified by code during what the C code sees as an ABI-conformant
function call (which seems to be the case here -- I don't think he was
talking about recovering from hardware exceptions), it doesn't matter what
language the function is implemented in, as long as it is indeed
ABI-conformant -- volatile is not needed.

OTOH, if the variable is modified from an interrupt/signal handler, another
thread, etc. you'll need (at a minimum) either volatile or some
implementation-specific optimization barrier, even if the code that actually
modifies the variable is also written in C.

-Scott

[Joe keane]

In article <99fbedd4-b37f-4ced...@m7g2000vbc.googlegroups.com>,

James Harris <james.h...@googlemail.com> wrote:
>Can anyone come up with a faster or better way to propagate exceptions?
>What other ways are there to handle exceptions in C? What do you do?

I worked at a place where the coding guidelines said
'every function returns an int, and that's the error code'.

[Kleuske]

On Mon, 02 Jan 2012 09:49:32 +0000, Joe keane saw fit to publish the
following:

So have I, and i must say I much preferred it to the place i worked
where it was deemed acceptable to silently fail and return nothing. At
least the former delivered stable code which was easy to trace and debug.

Hurray for returning error codes!

--
A Vulcan can no sooner be disloyal than he can exist without breathing.
-- Kirk, "The Menagerie", stardate 3012.4

[Jens Gustedt]

Am 12/31/2011 05:40 PM, schrieb BGB:
> On 12/31/2011 6:40 AM, Jens Gustedt wrote:
>> Officially they are since some weeks now, they come with C11

> who knows how long until it is really well supported though, as it has
> been long enough just waiting for MSVC to support C99 stuff...

I don't think that this will take as long as for C99 until the majors
will implement most of C11. As for one thing most of the things that
go in C11 are already there in most compilers (or OS for threads). And
then, they learnt from the experience and made some features optional.

> and alas they couldn't just reuse GCC's keywords?...
> well, in any case, probably macro magic.

yes, for tls the replacement by macros is trivial

>> And these tools clearly ease the programming of some try/catch mechnism
>> with macros in C.

> I had forgot about a trick I had actually used before:
> one can fake TLS using a macro wrapping a function call.

yes, in particular with the POSIX like per thread keys, that also come
with C11

> hmm:
> in C one could use macros to be like:
> BEGIN_TRY
> ...
> END_TRY
> BEGIN_CATCH(FooException, ex)
> ...
> END_CATCH
> BEGIN_FINALLY
> ...
> END_FINALLY

I think it is easy to do that even nicer, where exceptions would throw
an integer value

TRY {
...
} CATCH(identifier) {
case 1: ...

default:
printf("uncaught exception %d\n", identifier);
RETHROW;
}

I'll probably implement something in that vein in the next weeks on
top of P99.

Jens

[BGB]

On 1/2/2012 9:49 AM, Jens Gustedt wrote:
> Am 12/31/2011 05:40 PM, schrieb BGB:
>> On 12/31/2011 6:40 AM, Jens Gustedt wrote:
>>> Officially they are since some weeks now, they come with C11
>
>> who knows how long until it is really well supported though, as it has
>> been long enough just waiting for MSVC to support C99 stuff...
>
> I don't think that this will take as long as for C99 until the majors
> will implement most of C11. As for one thing most of the things that
> go in C11 are already there in most compilers (or OS for threads). And
> then, they learnt from the experience and made some features optional.
>

yes, but it also took until 2010 for MSVC to add "stdint.h", which
hardly seemed like a huge/complex issue.

I still can't just use "%lld" as a printf modifier, one has to type
"%I64d" with MSVC to get the expected results (but then have this not
work on Linux). (I created a function "gcrlltoa()" which does little
more than to print a long-long into a buffer and return a pointer to a
temporary string, so I can use "%s" instead.)

...

>> and alas they couldn't just reuse GCC's keywords?...
>> well, in any case, probably macro magic.
>
> yes, for tls the replacement by macros is trivial
>

yeah, probably.

>>> And these tools clearly ease the programming of some try/catch mechnism
>>> with macros in C.
>
>> I had forgot about a trick I had actually used before:
>> one can fake TLS using a macro wrapping a function call.
>
> yes, in particular with the POSIX like per thread keys, that also come
> with C11
>

ok.

>> hmm:
>> in C one could use macros to be like:
>> BEGIN_TRY
>> ...
>> END_TRY
>> BEGIN_CATCH(FooException, ex)
>> ...
>> END_CATCH
>> BEGIN_FINALLY
>> ...
>> END_FINALLY
>
> I think it is easy to do that even nicer, where exceptions would throw
> an integer value
>
> TRY {
> ...
> } CATCH(identifier) {
> case 1: ...
>
> default:
> printf("uncaught exception %d\n", identifier);
> RETHROW;
> }
>
> I'll probably implement something in that vein in the next weeks on
> top of P99.
>

the main issue with directly using braces is that it doesn't give
anywhere to un-register the exception.

catch with an integer could also use a FOURCC for well-known / named
exceptions.


my current exception system uses dynamically-typed references
(generally, type-checking would be done as part of catching an
exception). generally, my type-system identifies types by type-name
strings (additionally, class/instance objects and certain C structs may
also be used, as these are built on top of the dynamic type system).


or such...

[Dr Nick]

BGB <cr8...@hotmail.com> writes:

> On 1/2/2012 9:49 AM, Jens Gustedt wrote:
>> Am 12/31/2011 05:40 PM, schrieb BGB:
>>> On 12/31/2011 6:40 AM, Jens Gustedt wrote:
>>>> Officially they are since some weeks now, they come with C11
>>
>>> who knows how long until it is really well supported though, as it has
>>> been long enough just waiting for MSVC to support C99 stuff...
>>
>> I don't think that this will take as long as for C99 until the majors
>> will implement most of C11. As for one thing most of the things that
>> go in C11 are already there in most compilers (or OS for threads). And
>> then, they learnt from the experience and made some features optional.
>>
>
> yes, but it also took until 2010 for MSVC to add "stdint.h", which
> hardly seemed like a huge/complex issue.
>
> I still can't just use "%lld" as a printf modifier, one has to type
> "%I64d" with MSVC to get the expected results (but then have this not
> work on Linux). (I created a function "gcrlltoa()" which does little
> more than to print a long-long into a buffer and return a pointer to a
> temporary string, so I can use "%s" instead.)

Easier, probably, to take a hint from stdint and do something like this:

#if ..MSVC test..
#define PRINT_LLD "I64d"
#else
#define PRINT_LLD "lld"
#endif

...
long long var = 3872984822;

printf("this is a big number %" PRINT_LLD ", isn't it?",var);
--
Online waterways route planner | http://canalplan.eu
Plan trips, see photos, check facilities | http://canalplan.org.uk

[Rod Pemberton]

"Scott Wood" <sc...@buserror.net> wrote in message
news:slrnjg2p6e...@snotra.buserror.net...

Sorry, "non-C" is inaccurate. I think of C as "a single-threaded execution
model." By that, I mean there is no other C code executing in parallel,
even though that's *not* the correct definition for single-threaded.
Obviously, there could be C code executing in parallel with each modifying
the same machine word. So, "non-C" should be "everything other than" the C
code deemed to be active.

> The relevant factor is [...] synchronous versus asynchronous [...]

I don't see this as being relevant. His C code could be 100% synchronous
and he could still need a volatile. His code could be single-threaded,
single-core, single-processor, serial, non-parallel, not use setjmp/longjmp,
and be entirely interrupt free. If hardware modifies his machine word that
his C code is accessing, he still needs a volatile keyword for it.

AFAICT, James never fully stated what exactly is modifying his machine word.
It could be other software - in C or in assembly. Or, it could be hardware.
From his description, all I know is that it's "in the cloud" somewhere ...
I.e., it might not even be on his own computer. Maybe it's only accessable
from a another computer by a remote procedure call over a network. Without
knowing exactly what is modifying his machine word, I'm not sure how he is
going to guarantee non-volatility and/or atomicity.

> (or perhaps, outside this instance of the single-threaded C
> execution model).
>

Yes. Or, I like that better, although technically neither of us is being
precise enough. I'm not entirely sure of the correct terminology.
Single-threaded doesn't specifically exclude parallel processing for the
single-thread. Single-threaded just means a single command. E.g., one or
more parts of a single-threaded and in parallel C application could optimize
away use of the variable because it's unaware of the other parts' usage of
it.

> If it's modified by code during what the C code sees as an ABI-conformant
> function call (which seems to be the case here -- I don't think he was
> talking about recovering from hardware exceptions), it doesn't matter what
> language the function is implemented in, as long as it is indeed
> ABI-conformant -- volatile is not needed.

...

> OTOH, if the variable is modified from an interrupt/signal handler,
> another thread, etc. you'll need (at a minimum) either volatile or some
> implementation-specific optimization barrier, even if the code that
> actually modifies the variable is also written in C.
>

Yes.


Rod Pemberton

[Rod Pemberton]

"Kleuske" <kle...@nowhere.net> wrote in message
news:jds27t$423$3...@dont-email.me...

> On Mon, 02 Jan 2012 09:49:32 +0000, Joe keane saw fit to publish the
> following:
>
> > In article
> > <99fbedd4-b37f-4ced...@m7g2000vbc.googlegroups.com>,
> > James Harris <james.h...@googlemail.com> wrote:
> >>Can anyone come up with a faster or better way to propagate exceptions?
> >>What other ways are there to handle exceptions in C? What do you do?
> >
> > I worked at a place where the coding guidelines said 'every function
> > returns an int, and that's the error code'.
>
> So have I, and i must say I much preferred it to the place i worked
> where it was deemed acceptable to silently fail and return nothing. At
> least the former delivered stable code which was easy to trace and debug.
>
> Hurray for returning error codes!
>

I'm curious. In response to not being able to return other items, what did
you guys do? Did you guys also pass in an additional variable(s) to every
procedure or almost every procedure? E.g., an error structure, or a
function number?


Rod Pemberton

[ImpalerCore]

On Jan 2, 2:02 pm, BGB <cr88...@hotmail.com> wrote:
> On 1/2/2012 9:49 AM, Jens Gustedt wrote:
>
> > Am 12/31/2011 05:40 PM, schrieb BGB:
> >> On 12/31/2011 6:40 AM, Jens Gustedt wrote:
> >>> Officially they are since some weeks now, they come with C11
>
> >> who knows how long until it is really well supported though, as it has
> >> been long enough just waiting for MSVC to support C99 stuff...
>
> > I don't think that this will take as long as for C99 until the majors
> > will implement most of C11. As for one thing most of the things that
> > go in C11 are already there in most compilers (or OS for threads). And
> > then, they learnt from the experience and made some features optional.
>
> yes, but it also took until 2010 for MSVC to add "stdint.h", which
> hardly seemed like a huge/complex issue.
>
> I still can't just use "%lld" as a printf modifier, one has to type
> "%I64d" with MSVC to get the expected results (but then have this not
> work on Linux). (I created a function "gcrlltoa()" which does little
> more than to print a long-long into a buffer and return a pointer to a
> temporary string, so I can use "%s" instead.)

To workaround that particular problem, I rely on the PRI[diouxX]64
modifier to print 64-bit integers. I have this in my stdint.h +
inttypes.h wrapper that you may find useful.

\code snippet
/*
* Even if the Microsoft Runtime library is linked against a C99
* compiler, the printf modifier "ll" will likely not work. Use the
"I64"
* modifier to declare the correct PRI[di]64 fprintf modifier macros.
*/
#if (defined(__STDC__) && defined(__STDC_VERSION__))
# if (__STDC__ && __STDC_VERSION__ >= 199901L)
# define inttypes_prid64_defined
# if defined(_WIN32) || defined(__WIN32__)
/* Microsoft Runtime does not support the "ll" modifier. */
# define PRId64 "I64d"
# define PRIi64 "I64i"
# else
# define PRId64 "lld"
# define PRIi64 "lli"
# endif
# endif
#endif

/*
* If one lacks a C99 compiler, but allows 64-bit integer extensions,
* one can still define the set of PRI[di]64 fprintf modifier macros.
*/
#if !defined(inttypes_prid64_defined) && !defined(ANSI_C_PSTDINT)
# if defined(_WIN32) || defined(__WIN32__)
# define PRId64 "I64d"
# define PRIi64 "I64i"
# else
# if (INT64_MAX == LLONG_MAX)
# define PRId64 "lld"
# define PRIi64 "lli"
# elif (INT64_MAX == LONG_MAX)
# define PRId64 "ld"
# define PRIi64 "li"
# elif (INT64_MAX == INT_MAX)
# define PRId64 "d"
# define PRIi64 "i"
# else
# error "Platform not supported"
# endif
# endif
#endif
\endcode

Best regards,
John D.

[ImpalerCore]

On Jan 2, 2:53 pm, Dr Nick <3-nos...@temporary-address.org.uk> wrote:

I'd probably go with PRIdLL or PRIuLL, just to keep in style with the
other fprintf modifiers, if the PRI prefix isn't reserved (although
I'd still be quite tempted to use them).

Best regards,
John D.

[s_dub...@yahoo.com]

On Dec 31 2011, 5:19 am, "Rod Pemberton"
<do_not_h...@noavailemail.cmm> wrote:
> <s_dubrov...@yahoo.com> wrote in message

>
> news:f47cf54b-07c6-4896...@p16g2000yqd.googlegroups.com...
>
>
>
> <way OT for c.l.c, fine for alt.*>
>
> > A couple of decades ago I had a commecial dish service
> > that provided delayed stock data ...
> > [...]
> > They were 15-20 minute quote delayed
>
> Quote feeds are delayed 15 minutes for non-market makers.
>
> > My program development was stepwise and somewhat haphazard
> > because I didn't know, I had to deduce, the [stock quote] data stream.
>
> They probably weren't using NASDAQ or NYSE protocols for the
> satellite feed.  You weren't subscribed to an official NASDAQ or NYSE quote
> feed.  If you had been, you could've gotten the protocols for the quote
> stream after you (i.e., your company's legal department) signed a
> non-disclosure agreement.  I worked at a brokerage a decade or so ago

No, I wasn't subcribed to an official Nasdaq or NYSE newsfeed,
presumeably the provider was, and repackaged those feeds for delayed
and 'consolidated' quotes streaming. Probably, they didn't do the
quote consolidation themselves, but purchased that type of feed from
the exchanges, massaged it for KU band transmittion at 9600 baud. For
those who don't know, consolidated quotes are not the 'time and
sales' quotes of price, quantity, exchange, and market maker or
specialist firm for each crossing trade. Consolidated quotes are
amalgamated or combined results - again, not the individual trades,
probably on the order of what you'ld get from yahoo finance nowadays.

> programming on their stock trading application and also software for a
> couple of their feeds: SOES and SuperSOES (obsolete).  The system had
> numerous other computers and related market feeds: quote feeds, market
> feeds, alternate markets, ECNs, Instinet, etc.  I was so complicated that no
> one knew how it was all connected.  As part of a merger, new management
> wanted to know.  It took numerous departments months to obtain all the info
> and produce a diagram.  Apparently, they thought they could reduce costs by
> replacing the system ... laugh.
>

[snip]
>
> Looping works!

Agreed, it worked for me for limited resources and a narrow
application, that's my point.

There were already existing multitasking, multithreaded, OS's for the
XT that might have worked as well or alittle better, but I'll never
know.

The shelf life of that KU band technology was about 18-24 months
before being greatly diminished by the growth of affordable internet
access and availability. It zenith was over.

> Rod Pemberton

- to touch on Nick's comments -

Sure, the problem lends itself to multi-tasking, multithreads; one
task to respond RX data from the serial input, one to process the
data, with the processing task devoting a thread to handle each
control value in the input buffer, or setting up each control value as
an 'exception' to handle in the random input stream. But this is a
programming paradigm, a way of programming, that is of debateable
use. The results go to a file, which is an atomic process, anyway, in
this case.

The resources needed context switching weren't affordable, as the
ramdisk took available free memory.

Now though, well what would be your approach to processing other
'control' languages like XML or HTML? -or postscript, etc.

Steve

[James Harris]

On Dec 31 2011, 2:24 am, BGB <cr88...@hotmail.com> wrote:
> On 12/30/2011 3:38 PM, James Harris wrote:

I've snipped the bits about how TLS is generally accessed. Thanks for
explaining that info. It was new to me.

...

> > I was thinking of thread_excep being at a fixed location so it could
> > literally be checked with one instruction:
>
> >    cmp [thread_excep], 0
>
> if it is a global though, it is not a TLS.
>
> otherwise, I guess it would need to be changed around by the scheduler
> or something (register variables with scheduler, and it saves/restores
> their values on context switches or something?...).

Yes. The task switcher would include code along the lines of

push [thread_excep]

for the outgoing task and, for the incoming task,

pop [thread_excep]

Since the word would be checked many times between task switches this
is much faster than following a structure off FS or similar.

...

> I guess, if one is making an OS, other possibilities also exist (special
> magic TLS sections?).
>
> say, if one has a section ".tls" section, where the scheduler
> automatically saves/restores the contents of anything stored within
> these sections. the advantage could be very fast access to TLS variables
> at the cost of potentially more expensive context switches.

As thread_excep would be accessed very frequently it would be changed
explicitly on a context switch. That doesn't apply to the bulk of the
thread info, though, which might be wanted by the task. Most of that
info would be changed simply by updating a pointer. For example, as
well as there being

thread_excep: resd 1

in a page that the user-mode task can update there would also be

thread_data_p: resd 1

in a page which, to the task, was read-only. On a context switch both
words would be updated. This keeps thread switching minimal yet allows
the user-mode program easy and fast access to the kinds of things it
may want info on. For example, if the program wanted its thread id it
might find it at thread_data_p->id.

Incidentally, these could be wrapped in getter and setter functions
but, alternatively, for performance they can be linked to fixed
locations.

> I don't think flags are the ideal strategy here.
> flags don't as easily allow user-defined exceptions, and if one has only
> a few bits for a user-defined exception class or similar, then it
> creates a bit of a burden WRT avoiding clashes.

I suppose it depends on the normal first level of selecting
exceptions. The best way might be whatever the handler uses to first
distinguish them. Most code I've seen or written is interested, at the
top level, in which *type* of exceptions to catch and which to ignore
and it generally does that by looking at the exception type:
indexerror, valueerror, computationerror etc. That coupled with the
fact that multiple exceptions can be outstanding at the same time
suggested the use of a bit array but it's not the only option.

You mention user-defined exceptions. I planned only one bit for them
(as there is an arbitrary number of them). To distinguish one user
exception from another would probably require a call to a routine that
examines the detail.

> one "could" potentially use interned strings and magic index numbers
> though, say, 10 or 12 bits being plenty to allow a range of named
> exception classes (potentially being keyed via a hash-table or similar).
>
> this is actually how my dynamic type-system works (types are identified
> by name, but encoded via hash indices).

I see your point. The problem is that this allows only one exception
at a time to be signalled-but-not-yet-handled. Which one? I suppose
the most critical one or the first one or the last one could be chosen
to be marked in the indicator word.

<<END
Just on this point, and bringing it back specifically to C, if TLS is
hard to obtain or slow to access there are two other possibilities
that I don't think have been mentioned yet that spring to mind for use
in C.

1. If only running a single thread simply use a global. Job done.

2. Use errno. Set it to zero at the start and check it where
appropriate. On a user-detected exception that does not already set
errno set it to a value which is outside the normal range (and create
or append to the detailed exception object, as before).

Using errno does only allow one exception to be indicated at a time
but that's the same as what I understand your suggestion to be. It
also doesn't work between languages (unless they obtain the address of
errno) but I think it could work well for C.
END

At the end of the day, any scheme could be used. For performance the
idea is that the exception-indicating word is either zero or non-zero.
If it's non-zero one or more exceptions has/have occurred and must be
dealt with.

James

[Kleuske]

On Mon, 02 Jan 2012 16:01:26 -0500, Rod Pemberton saw fit to publish the

following:

> "Kleuske" <kle...@nowhere.net> wrote in message
> news:jds27t$423$3...@dont-email.me...
>> On Mon, 02 Jan 2012 09:49:32 +0000, Joe keane saw fit to publish the
>> following:
>>
>> > In article
>> > <99fbedd4-b37f-4ced...@m7g2000vbc.googlegroups.com>,
>> > James Harris <james.h...@googlemail.com> wrote:
>> >>Can anyone come up with a faster or better way to propagate
>> >>exceptions? What other ways are there to handle exceptions in C? What
>> >>do you do?
>> >
>> > I worked at a place where the coding guidelines said 'every function
>> > returns an int, and that's the error code'.
>>
>> So have I, and i must say I much preferred it to the place i worked
>> where it was deemed acceptable to silently fail and return nothing. At
>> least the former delivered stable code which was easy to trace and
>> debug.
>>
>> Hurray for returning error codes!
>>
>>
> I'm curious. In response to not being able to return other items, what
> did you guys do?

Given the problem domain (controlling hugely expensive equipment very, very,
VERY accurately), there wasn't any big need to return all kinds of
stuff, but there's always the possibility to pass a pointer in the argument
list and return results through that. Stability and correctness was the prime
concern, everything else was rather less important.

> Did you guys also pass in an additional variable(s) to
> every procedure or almost every procedure? E.g., an error structure, or
> a function number?

Error structures? Nope. Not really. Other variables passed by pointer in order
to return results? Occasionally, but there wasn't any big need for that. The
flow of information was a rather one-way affair.

I should have mentioned it wasn't your average piece of software and should
have qualified my remarks to reflect that.

--
It would seem that evil retreats when forcibly confronted.
-- Yarnek of Excalbia, "The Savage Curtain", stardate 5906.5

[James Harris]

On Dec 30 2011, 10:53 pm, Eric Sosman <esos...@ieee-dot-org.invalid>
wrote:

> On 12/29/2011 12:52 PM, James Harris wrote:
>
> > Below is a suggestion for handling - more accurately, propagating -
> > exceptions in C. It is intended for overall program performance though
> > it goes against a preference of many including me. Can anyone come up
> > with a faster or better way to propagate exceptions? What other ways
> > are there to handle exceptions in C? What do you do? (I tend to detect
> > exceptions where they occur by checking return values but I have never
> > had a good way in C to get them to propagate.)
>
>      Maybe this is clear to you already, but the words you use leave
> me in some doubt: In pondering the performance of an exception scheme
> one should not worry about how fast exceptions are handled, but about
> how fast non-exceptions are not-handled.  If you make a million calls
> to malloc() and get an exception on one of them, it's the performance
> of the other 999,999 that you need to consider.
>
>      They're called "exceptions," not "usuals."

I'm principally thinking of overall performance. I did say that but I
can see my later comments could be taken to mean the opposite. There
is small caveat below but unless an exception is repeatedly generated
in a loop them the time it takes to generate or propagate tends to
zero and becomes insignificant.

Perhaps it's better expressed as: what's the fastest way in standard C
to ignore non-exceptions (still not a good phrase but you know what I
mean) while still reliably propagating and handling exceptions.

The caveat is that the design of C and its libraries may overuse
return values where exceptions, if available, would be a better
choice. So it may be hard for me to convince people on a C newsgroup
that invalid return values are not the best thing since sliced bread!

Exceptions should probably always be unusual but there are cases where
they could be beneficially used instead of invalid values.

...

>      The only portable ways to unwind the stack are to return or to
> use longjmp().  With the latter there's no way to unwind the stack
> frames one by one, nor to inspect what's being unwound: You just
> longjmp() to the next-higher handler, do whatever local cleanup's
> needed, longjmp() again to a handler higher still, and so on until
> somebody's local cleanup says "This exception has propagated far enough;
> I'm absorbing it."

For me, this paragraph sums it up. While a single longjmp back to main
would be very fast, in the general case a hierarchy of setjmps and
longjmps would be far too slow.

The only other portable option of returning can be made fast (very
fast) using option 3.

>      Again, I feel you're worrying about performance in a situation
> where performance is not very important.
>
> > * Option 3. The function detecting an exception creates an exception
> > object and sets an exception type in a thread-specific variable. Any
> > calling functions, after each call (or group of calls) test that
> > variable. If non-zero they jump to a local exception handler if there
> > is one or jump to a return from the current function if not. This does
> > involve a test and branch for each call but that is very fast (fusible
> > and single cycle) and it avoids the need for exception handling
> > context to be pushed and popped. It would also preserve the branch
> > prediction of calls and returns.
>
>      Isn't this just an `errno' equivalent, with a pointer instead
> of an `int'?  In "The Standard C Library," PJ Plauger writes
>
>         Nobody likes errno or the machinery that it implies. I can't
>         recall anybody defending this approach to error reporting,
>         not in two dozen or more meetings of X3J11, the committee
>         that developed the C Standard. Several alternatives were
>         proposed over the years. At least one faction favored simply
>         discarding errno.
>
> I'd hesitate before imitating an interface nobody likes ...

This is not the same as errno and does not use the same 'machinery'.
errno can be ignored and provides no instance data (such as what
specifically went wrong?). The value of errno, if potentially wanted,
must be saved before another call is made as it may be overwriten. An
exception, on the other hand, is something that should record specific
data and cause an immediate change in the processing path.exceptions

I chose TLS since that's the 'right' place for an exception condition.
Once an exception occurs the thread itself needs to sit up and take
notice, not just one routine, not just part of the thread, and not the
whole process.

As mentioned in reply to BGB, though, if not running multiple threads
a global could be used instead of TLS. I'm not advocating general use
of globals. They carry state between routines and break interfaces
which is a very bad idea. This, however, is an exception indication
that applies to the locus of control wherever it happens to be in the
single-threaded process. The alternative of adding an explicit
exception-return parameter from each called routine is also good but I
doubt most C programmers would like it as it's so unfamiliar.

>      Also, the performance probably suffers.  For all its drawbacks,
> the function-returns-a-distinguishable-value-on-failure approach has
> the advantage that the value is very likely in a CPU register when
> you need to test it, not in a global variable somewhere off in RAM.

I disagree here. As with the case at the top, performance is only an
issue if code is executed repeatedly. If being run repeatedly the
variable would normally be in L1 cache. Consider the proposed x86-32
code following a call to a routine.

cmp [thread_excep], 0
jnz exception_handler

Given this being executed in a loop the only normal way for
thread_excep not to be cached would be if there was extreme contention
for cache capacity. If that were the case a lot of things would be
slow and, again, this sequence would tend to become insignificant.

As well as thread_excep being cached, the branch would nearly always
fall through (be not taken) so is an ideal case for branch prediction
to correctly predict as not taken. Finally, the two instructions may
be fusible into one operation.

In other words, there's almost zero cost to this sequence.

The main negative to me is that, in C, it uses a goto but this hasn't
elicited the chorus of disapproval that I thought it might.

> > [...] Anyone know of
> > something better?
>
>      I've seen various exception-like frameworks overlaid on C, some
> down-and-dirty, some fairly elaborate.  All of them, however useful
> for the particular task at hand, suffered from lack of support by
> the language itself: There's just no way to tell the compiler that
> a function might throw an exception, which means the compiler can't
> tell the programmer when he's forgotten to catch one.  Somebody opens
> a file, obtains values from various accessor functions and writes them
> to the file, closes the stream, and returns -- except that one of the
> accessors throws an exception that gets caught somewhere way up the
> stack, and the file-writing function never closes its stream ...
>
>      Errors of this kind can be prevented by programmer discipline,
> but I've seen no scheme that will help keep the programmer on the
> strait and narrow path.  This is sad, because it means an exception
> mechanism that was supposed to make the programmer's job easier can
> wind up making it more burdensome -- and more perilous.
>
>      Exceptions aren't something you can just bolt on to a language.
> Well, actually, you *can*: people have bolted them on to C, over and
> over again.  Yet none of the various schemes has caught on, which I
> think should cause you to ponder.  Maybe it's because bolting on is
> a poor substitute for baking in.

Agreed. Inbuilt is better in so many ways. However, we have to work
with what there is available. C is C, after all, and I don't want to
mess with C-with-extensions whether they are plusses or something
else!

James

[James Harris]

On Jan 2, 10:51 am, Kleuske <kleu...@nowhere.net> wrote:
> On Mon, 02 Jan 2012 09:49:32 +0000, Joe keane saw fit to publish the
> following:
>
> > In article

> > <99fbedd4-b37f-4ced-8759-4ba965625...@m7g2000vbc.googlegroups.com>,

> > James Harris  <james.harri...@googlemail.com> wrote:
> >>Can anyone come up with a faster or better way to propagate exceptions?
> >>What other ways are there to handle exceptions in C? What do you do?
>
> > I worked at a place where the coding guidelines said 'every function
> > returns an int, and that's the error code'.
>
> So have I, and i must say I much preferred it to the place i worked
> where it was deemed acceptable to silently fail and return nothing. At
> least the former delivered stable code which was easy to trace and debug.
>
> Hurray for returning error codes!

What did you guys return if an exception occurred in an exception
handler?

James

[Kleuske]

On Tue, 03 Jan 2012 03:59:27 -0800, James Harris saw fit to publish the

following:

> On Jan 2, 10:51 am, Kleuske <kleu...@nowhere.net> wrote:
>> On Mon, 02 Jan 2012 09:49:32 +0000, Joe keane saw fit to publish the
>> following:
>>
>> > In article
>> > <99fbedd4-b37f-4ced-8759-4ba965625...@m7g2000vbc.googlegroups.com>,
>> > James Harris  <james.harri...@googlemail.com> wrote:
>> >>Can anyone come up with a faster or better way to propagate
>> >>exceptions? What other ways are there to handle exceptions in C? What
>> >>do you do?
>>
>> > I worked at a place where the coding guidelines said 'every function
>> > returns an int, and that's the error code'.
>>
>> So have I, and i must say I much preferred it to the place i worked
>> where it was deemed acceptable to silently fail and return nothing. At
>> least the former delivered stable code which was easy to trace and
>> debug.
>>
>> Hurray for returning error codes!
>
> What did you guys return if an exception occurred in an exception
> handler?

Lacking exception handlers, the question did not arise. In case of error
in that subsystem, the machine just freezes, in order to prevent damage
to the expensive (think jet-fighter price-range) equipment.

I remember celebrating the first time the software ran for ten seconds
without freezing the machine. I got home a bit wobbly on that occasion.

--
Warp 7 -- It's a law we can live with.

[James Harris]

On Jan 2, 8:59 pm, "Rod Pemberton" <do_not_h...@noavailemail.cmm>
wrote:

...

> AFAICT, James never fully stated what exactly is modifying his machine word.
> It could be other software - in C or in assembly.  Or, it could be hardware.
> From his description, all I know is that it's "in the cloud" somewhere ...
> I.e., it might not even be on his own computer.  Maybe it's only accessable
> from a another computer by a remote procedure call over a network.  Without
> knowing exactly what is modifying his machine word, I'm not sure how he is
> going to guarantee non-volatility and/or atomicity.

Nothing special would modify the word. It is intended just as a way
for a callee to return an exception indication to its caller.

To make an example,

callee
if ((temp = malloc(SIZ)) == NULL) {
... record the details ...
thread_excep |= EXCEP_MEMSPACE;
return;
}
... further processing and return to caller ...

caller
p1 = callee(SIZ1);
if (thread_excep) goto handler;
p2 = callee(SIZ2);
if (thread_excep) goto handler;
... normal processing ...

As I understand it, even though the callee could modify thread_excep
it would not need to be declared as volatile.

James

[James Harris]

On Jan 2, 6:50 am, "Rod Pemberton" <do_not_h...@noavailemail.cmm>
wrote:

...

> My experience with c.l.c people
> over the past decade is that many of them are insanely hostile, intolerant
> of any criticism, and will argue to their death that they are correct when
> in fact they are provably wrong ...

A lot of the antagonists have moved on. IME clc is a far better place
than it was a few years ago.

James

[Keith Thompson]

And a well-constructed killfile can make it even more pleasant.

--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
Will write code for food.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[BGB]

On 1/3/2012 4:19 AM, James Harris wrote:
> On Dec 31 2011, 2:24 am, BGB<cr88...@hotmail.com> wrote:
>> On 12/30/2011 3:38 PM, James Harris wrote:
>
> I've snipped the bits about how TLS is generally accessed. Thanks for
> explaining that info. It was new to me.
>
> ...
>
>>> I was thinking of thread_excep being at a fixed location so it could
>>> literally be checked with one instruction:
>>
>>> cmp [thread_excep], 0
>>
>> if it is a global though, it is not a TLS.
>>
>> otherwise, I guess it would need to be changed around by the scheduler
>> or something (register variables with scheduler, and it saves/restores
>> their values on context switches or something?...).
>
> Yes. The task switcher would include code along the lines of
>
> push [thread_excep]
>
> for the outgoing task and, for the incoming task,
>
> pop [thread_excep]
>
> Since the word would be checked many times between task switches this
> is much faster than following a structure off FS or similar.
>

possibly, however the issue then partly becomes:
how does the task switcher know where the variable is?...

possible options:
do it like the BIOS, with every task having a dedicated kernel area
which is also accessible to the userspace process;
the shared-object or DLL holding the variable is "owned" by the
OS/kernel (the kernel then depends on the presence of the library and on
the location of the variable within the library).

> ...
>
>> I guess, if one is making an OS, other possibilities also exist (special
>> magic TLS sections?).
>>
>> say, if one has a section ".tls" section, where the scheduler
>> automatically saves/restores the contents of anything stored within
>> these sections. the advantage could be very fast access to TLS variables
>> at the cost of potentially more expensive context switches.
>
> As thread_excep would be accessed very frequently it would be changed
> explicitly on a context switch. That doesn't apply to the bulk of the
> thread info, though, which might be wanted by the task. Most of that
> info would be changed simply by updating a pointer. For example, as
> well as there being
>
> thread_excep: resd 1
>
> in a page that the user-mode task can update there would also be
>
> thread_data_p: resd 1
>
> in a page which, to the task, was read-only. On a context switch both
> words would be updated. This keeps thread switching minimal yet allows
> the user-mode program easy and fast access to the kinds of things it
> may want info on. For example, if the program wanted its thread id it
> might find it at thread_data_p->id.
>
> Incidentally, these could be wrapped in getter and setter functions
> but, alternatively, for performance they can be linked to fixed
> locations.
>

FWIW:
if one uses PE/COFF DLLs, typically all accesses to imported global
variables is indirect (and requires explicit declaration);
if one uses ELF, potentially nearly all access to global variables is
indirect (the typical strategy is to access them via the GOT except in
certain special cases).

also, segment overrides are fairly cheap.
the cheapest option then would probably be to include whatever magic
state directly into the TEB/TIB/whatever, then be like:
mov eax, [fs:address]

in some cases, accessing a variable this way could in-fact be
potentially cheaper than accessing a true global.

x86-64 is a little better, since the CPU defaults to relative
addressing, but is messed up some by ELF and the SysV/AMD64 ABI by
demanding that all access to globals still be via the GOT. the slight
advantage though is that one doesn't need dllimport/dllexport
annotations to import/export functions and variables, but overall I
prefer PE/COFF DLLs more.

back when I was developing an OS (so long ago), I also used PE/COFF.


a possible idea (for an alternative to both a GOT and the traditional
DLL imports, if developing a custom format for binaries, or just
tweaking PE/COFF) could be the use of a relocation table for any imports
(potentially in a semi-compressed form). the main issue is mostly on
x86-64, how to best deal with the 2GB window issue for global variables
if using this strategy (besides just requiring all libraries to be in
the low 2GB, or using indirect addressing if the compiler and/or linker
can't determine if the variable will be within the 2GB window).

this would probably be mostly applicable in the case where one is
reusing an existing compiler but writing a custom linker, and probably
while using a non-ELF object format (such as COFF).

possible nifty features one could add in such a case: fat-binaries,
partial late-binding (sadly, non-trivial cases would require compiler
support), ...

>> I don't think flags are the ideal strategy here.
>> flags don't as easily allow user-defined exceptions, and if one has only
>> a few bits for a user-defined exception class or similar, then it
>> creates a bit of a burden WRT avoiding clashes.
>
> I suppose it depends on the normal first level of selecting
> exceptions. The best way might be whatever the handler uses to first
> distinguish them. Most code I've seen or written is interested, at the
> top level, in which *type* of exceptions to catch and which to ignore
> and it generally does that by looking at the exception type:
> indexerror, valueerror, computationerror etc. That coupled with the
> fact that multiple exceptions can be outstanding at the same time
> suggested the use of a bit array but it's not the only option.
>
> You mention user-defined exceptions. I planned only one bit for them
> (as there is an arbitrary number of them). To distinguish one user
> exception from another would probably require a call to a routine that
> examines the detail.
>

probably ok for a single app, but is dubious for an OS or general mechanism.

>> one "could" potentially use interned strings and magic index numbers
>> though, say, 10 or 12 bits being plenty to allow a range of named
>> exception classes (potentially being keyed via a hash-table or similar).
>>
>> this is actually how my dynamic type-system works (types are identified
>> by name, but encoded via hash indices).
>
> I see your point. The problem is that this allows only one exception
> at a time to be signalled-but-not-yet-handled. Which one? I suppose
> the most critical one or the first one or the last one could be chosen
> to be marked in the indicator word.
>

errm... typical exception mechanisms only allow a single exception at a
time. as soon as an exception is thrown, it is handled immediately. no
delays or queuing are allowed. if an exception occurs within a handler,
this may either throw the new exception (the prior one is forgotten), or
simply kill the app.

Windows does this: if something goes wrong and an exception can't be
thrown in-app, Windows will simply kill the process. if something goes
wrong in-kernel, it is a blue-screen / BSOD.

likewise for the CPU:
if an exception occurs within an interrupt handler, this is a
"doublefault" and generally leads to a reboot.


an exception is not a status code...

> <<END
> Just on this point, and bringing it back specifically to C, if TLS is
> hard to obtain or slow to access there are two other possibilities
> that I don't think have been mentioned yet that spring to mind for use
> in C.
>

it is slow, yes, but typically not enough to care about.
for most things people don't worry about a few clock-cycles here and
there (generally, it is big algorithm-level stuff that kills
performance, and not a lack of sufficient micro-optimization).

> 1. If only running a single thread simply use a global. Job done.
>

except when globals are slower.

> 2. Use errno. Set it to zero at the start and check it where
> appropriate. On a user-detected exception that does not already set
> errno set it to a value which is outside the normal range (and create
> or append to the detailed exception object, as before).
>

FWIW, depending on the C-library, errno may be in-fact a function call
wrapped in a macro.

> Using errno does only allow one exception to be indicated at a time
> but that's the same as what I understand your suggestion to be. It
> also doesn't work between languages (unless they obtain the address of
> errno) but I think it could work well for C.
> END
>
> At the end of the day, any scheme could be used. For performance the
> idea is that the exception-indicating word is either zero or non-zero.
> If it's non-zero one or more exceptions has/have occurred and must be
> dealt with.
>

except that "exceptions as a status word" aren't really "exceptions" in
the traditional sense.

may as well just call them status-codes, and leave the whole mess of
exception-handling out of this.

[Joe keane]

In article <99fbedd4-b37f-4ced...@m7g2000vbc.googlegroups.com>,
James Harris <james.h...@googlemail.com> wrote:

>* Option 3. The function detecting an exception creates an exception
>object and sets an exception type in a thread-specific variable. Any
>calling functions, after each call (or group of calls) test that
>variable. If non-zero they jump to a local exception handler if there
>is one or jump to a return from the current function if not. This does
>involve a test and branch for each call but that is very fast (fusible
>and single cycle) and it avoids the need for exception handling
>context to be pushed and popped. It would also preserve the branch
>prediction of calls and returns.

I'm baffled as to why one would do that.

It's like using a global variable to pass information between a function
and a function it calls or the reverse. Then you realize that it
doesn't work for multi-threaded and make it more complicated.

It still doesn't work when you get an exception, then the cleanup code
itself gets an exception. [Of course, in some cases you just throw up
your hands and call abort(), but this often can be handled.] Presumably
the callers want the -first- error code but you overwrote it.

If the stack works [or registers] why not use that?

[James Harris]

On Dec 29 2011, 11:38 pm, "Rod Pemberton"
<do_not_h...@noavailemail.cmm> wrote:

...

> I dropped c.l.c.  Please don't add c.l.c. back while
> my comments are included in a reply ...

...

> I'd look at these:
>
> 1) custom assembly
> 2) setjmp and longjmp
> 3) check status and exit from procedures repeatedly until exception can be
> handled
> 4) check status and call handler prior to exiting a procedure
>
> Good and bad:
>
> 1) is non-portable
> 2) is somewhat portable or easily fixable
> 3) can skip completion of an in-process procedure requiring recalling or
> restarting them
> 4) can a bit of code and may still disrupt execution

...

> > * Option 2. Traditional exception handling. As used in other languages
> > this unwinds the stack of function calls looking for a handler.
>
> How does it "unwind"?  By tracing stackframes or by exiting procedures?

Perhaps it depends on the language implementation. I don't know of a
universal mechanism.

> > I cannot think of a way to implement that in C.
>
> Tracing stackframes would result in non-portable or non-standard C code.  C
> is not required to use a stack.

So I've heard but in my book C, being recursive, has to use a stack of
function calls. They may not go on a hardware stack but they have to
form a stack. Otherwise a given activation would not know where to
return to. That said, I suppose it could be implemented as a linked
list that's only added to or deleted from at one end. Being a LIFO I'd
call that at least stack-ish!

> Checking a status prior to exiting a procedure or post exit can be done
> though.

If you mean at the end of a procedure prior to normal exit that's an
unusual place to check for exceptions!

>  Checking prior to exit has the issues I mentioned above for my #3
> and #4.  I didn't mention checking when returning from calling a procedure
> above.  It should have the same or similar issues.

...

> > * Option 3. The function detecting an exception creates an exception
> > object and sets an exception type in a thread-specific variable. Any
> > calling functions, after each call (or group of calls) test that
> > variable.
>

> I think that is describing my #3.

Sounds like it.

I don't understand the point you are making here.

...

> > Unfortunately option 3 would mean use of goto. I haven't used goto in
> > C code for many years and don't like the idea of starting now but I
> > can't think of anything quicker. In C this could be
>
> A goto with local handling is quick, but it:
>
> 1) prevents restructuring the code

True, goto is a pain in a number of ways.

> 2) may skip executing some code due to goto-ing the exception

Don't you normally want to skip some code if you get an exception?

...

> In pseudo-C code, you might try something like the following naive
> implementation, which would be my #4:
>
> int exception=0; /* file scope or "global" */
>
> struct exceptions
> {
>    int condition;
>    char data[];
>
> }
>
> void exception_handler(struct exceptions)
> {
>     exception=0; /* clear exception state */
> /* if need be, you could stack and unstack state here ... */
>
> /* hopefully, only a few exceptions are required */
>     switch(exceptions.condition)
>       condition 1:
>             do stuff;
>             break;
>       condition 2:
>             do stuff;
>             break;
>       default:
>             do stuff;
>             break;
>
> }
>
> blah some_proc(blah blah, blah blahblah)
> {
>
>    bunch of code ...
>
> /* in each procedure */
> /* wherever a condition can be detected */
>    if (condition)
>    {
>       exception=1;
>       exceptions.condition = 0x02;
>       memcpy(exceptions.data,error_data);
>    }
>
>    more code ...
>
> /* just prior to return() */
> /* in any procedure that can set an exception */
>    if(exception)
>      exception_handler(exceptions);
>
>    return();
>
> }
>
> It doesn't skip executing any code.  Since the handler is it's own
> procedure, it eliminates duplicating much code.  It doesn't requiring saving
> current execution context and and restarting.  It doesn't require recalling
> a bunch of nested procedures after restoring data state.  It doesn't use
> goto's.

Your code seems to continue the normal processing path in the event of
an exception, which is in line with the preference you mentioned above
but it seems an odd approach.

It's the "more code" part and the placement of it that surprises me.
Granted there can be times when you want to carry on to the next
instruction after an exception has occurred such as when the faulting
instruction is just before a finally block but that's not the norm is
it?

James

[James Harris]

On Jan 3, 8:04 pm, BGB <cr88...@hotmail.com> wrote:
> On 1/3/2012 4:19 AM, James Harris wrote:
> > On Dec 31 2011, 2:24 am, BGB<cr88...@hotmail.com>  wrote:

...

> > Yes. The task switcher would include code along the lines of
>
> >    push [thread_excep]
>
> > for the outgoing task and, for the incoming task,
>
> >    pop [thread_excep]
>
> > Since the word would be checked many times between task switches this
> > is much faster than following a structure off FS or similar.
>
> possibly, however the issue then partly becomes:
> how does the task switcher know where the variable is?...

Isn't this easy if you have a flat address space: just put the
variable at a fixed location?

> possible options:
> do it like the BIOS, with every task having a dedicated kernel area
> which is also accessible to the userspace process;
> the shared-object or DLL holding the variable is "owned" by the
> OS/kernel (the kernel then depends on the presence of the library and on
> the location of the variable within the library).

...

> FWIW:
> if one uses PE/COFF DLLs, typically all accesses to imported global
> variables is indirect (and requires explicit declaration);
> if one uses ELF, potentially nearly all access to global variables is
> indirect (the typical strategy is to access them via the GOT except in
> certain special cases).

Does this mean that the global, G, in the following would not be
accessed directly?

int G;
void sub() {
while (G > 0) {
G -= 1;
sub();
}
}
int main(int argc, char **argv) {
G = argc;
sub();
return 0;

}

> also, segment overrides are fairly cheap.

True. It obviously depends on the CPU but in my tests, while segment
loading cost a bit, segment overrides were completely free.

> the cheapest option then would probably be to include whatever magic
> state directly into the TEB/TIB/whatever, then be like:
> mov eax, [fs:address]
>
> in some cases, accessing a variable this way could in-fact be
> potentially cheaper than accessing a true global.

As mentioned, I would expect a file-scope global to be accessed
directly - as a machine word at a location that gets filled-in by the
linker. Am I wrong? Are you are thinking of library code?

...

> > You mention user-defined exceptions. I planned only one bit for them
> > (as there is an arbitrary number of them). To distinguish one user
> > exception from another would probably require a call to a routine that
> > examines the detail.
>
> probably ok for a single app, but is dubious for an OS or general mechanism.

Why is this no good? There can be any number of user exception types
so it's not possible to have one bit for each. The best I could think
of was to have one bit to indicate a user exception and functions to
examine as much other detail as needed.

> >> one "could" potentially use interned strings and magic index numbers
> >> though, say, 10 or 12 bits being plenty to allow a range of named
> >> exception classes (potentially being keyed via a hash-table or similar).
>
> >> this is actually how my dynamic type-system works (types are identified
> >> by name, but encoded via hash indices).
>
> > I see your point. The problem is that this allows only one exception
> > at a time to be signalled-but-not-yet-handled. Which one? I suppose
> > the most critical one or the first one or the last one could be chosen
> > to be marked in the indicator word.
>
> errm... typical exception mechanisms only allow a single exception at a
> time. as soon as an exception is thrown, it is handled immediately. no
> delays or queuing are allowed. if an exception occurs within a handler,
> this may either throw the new exception (the prior one is forgotten), or
> simply kill the app.

If only one exception at a time is needed that makes things
significantly easier. But is only one needed? I was thinking of
exceptions caused in exception handlers. For example, the exception
handler divides by the number it first thought of and triggers an
overflow or it checks an array index and finds it out of bounds. What
does the system do?

It could leave the first exception in place - after all that was the
root cause. Or it could replace the exception information with its own
- after all it has an internal fault. Neither seems obviously best. It
might be best to keep both indications until both have been either
resolved or reported ... but it would be easier to handle if only one
was kept.

> Windows does this: if something goes wrong and an exception can't be
> thrown in-app, Windows will simply kill the process.

If an exception "can't be thrown"? What would prevent it being thrown?

> if something goes
> wrong in-kernel, it is a blue-screen / BSOD.

Not very good, though, is it! In reality I think that Windows or any
OS will handle almost all exceptions. A BSOD should only occur on a
hardware failure such as a machine-check exception. (It looks like
Windows does run third-party drivers as privileged this increasing its
vulnerability many-fold.)

> likewise for the CPU:
> if an exception occurs within an interrupt handler, this is a
> "doublefault" and generally leads to a reboot.
>
> an exception is not a status code...

Following your analogy of a CPU you could perhaps think of each
exception condition as an interrupt line that gets asserted until the
condition causing the interrupt has been resolved. I know, as an
analogy it's not perfect. :-(

> > <<END
> > Just on this point, and bringing it back specifically to C, if TLS is
> > hard to obtain or slow to access there are two other possibilities
> > that I don't think have been mentioned yet that spring to mind for use
> > in C.
>
> it is slow, yes, but typically not enough to care about.
> for most things people don't worry about a few clock-cycles here and
> there (generally, it is big algorithm-level stuff that kills
> performance, and not a lack of sufficient micro-optimization).
>
> > 1. If only running a single thread simply use a global. Job done.
>
> except when globals are slower.

I'd like to understand why they would be. I tried compiling the code
above with gcc and it seems to refer to G directly such as in sub()
the compiler generates

mov DWORD PTR G, %eax

> > 2. Use errno. Set it to zero at the start and check it where
> > appropriate. On a user-detected exception that does not already set
> > errno set it to a value which is outside the normal range (and create
> > or append to the detailed exception object, as before).
>
> FWIW, depending on the C-library, errno may be in-fact a function call
> wrapped in a macro.

I know. I found that in some systems there is a call which returns the
address of errno. In that case the compiler should (hopefully)
remember that address for each use in a function.

> > Using errno does only allow one exception to be indicated at a time
> > but that's the same as what I understand your suggestion to be. It
> > also doesn't work between languages (unless they obtain the address of
> > errno) but I think it could work well for C.
> > END
>
> > At the end of the day, any scheme could be used. For performance the
> > idea is that the exception-indicating word is either zero or non-zero.
> > If it's non-zero one or more exceptions has/have occurred and must be
> > dealt with.
>
> except that "exceptions as a status word" aren't really "exceptions" in
> the traditional sense.
>
> may as well just call them status-codes, and leave the whole mess of
> exception-handling out of this.

Don't forget the original intention was speed. The point of a status
word, as you call it, is to allow exception handling (or, non-
handling) quickly. It is not an end in itself but a means of adding
exceptions to C in a way that recognises exceptions in the right
places while having the lowest possible impact on overall performance.

James

[Jens Gustedt]

Am 01/02/2012 08:02 PM, schrieb BGB:
> On 1/2/2012 9:49 AM, Jens Gustedt wrote:
>> I think it is easy to do that even nicer, where exceptions would throw
>> an integer value
>>
>> TRY {
>> ...
>> } CATCH(identifier) {
>> case 1: ...
>>
>> default:
>> printf("uncaught exception %d\n", identifier);
>> RETHROW;
>> }
>>
>> I'll probably implement something in that vein in the next weeks on
>> top of P99.
>>
>
> the main issue with directly using braces is that it doesn't give
> anywhere to un-register the exception.

The 'TRY' can hide a lot of things. In particular you can do scope
bound resource manangement by using "for" scope variables. See e.g

http://gustedt.wordpress.com/2010/08/14/scope-bound-resource-management-with-for-scopes/

> catch with an integer could also use a FOURCC for well-known / named
> exceptions.

FOURCC ??

> my current exception system uses dynamically-typed references
> (generally, type-checking would be done as part of catching an
> exception). generally, my type-system identifies types by type-name
> strings (additionally, class/instance objects and certain C structs may
> also be used, as these are built on top of the dynamic type system).

Throwing void* instead of int would also be an option, indeed. You
then could easily develop a system that uses some sort of struct to
transfer the exception information.

But, I'd prefer the solution I indicated over that. I would be
basically a (hidden) switch statement for the CATCH block, things that
look familiar to C programmers. And when throwing pointers to objects
you'd always have the problem of the responsability for the allocated
space.

Jens

[Scott Wood]

On 2012-01-02, Rod Pemberton <do_no...@noavailemail.cmm> wrote:
> "Scott Wood" <sc...@buserror.net> wrote in message
> news:slrnjg2p6e...@snotra.buserror.net...

>> > Ok, that screams "volatile" keyword to me. Anything modified outside of
>> > C and used by C is unknown to C so needs a volatile so it's not
>> > optimized away or to ensure it's reloaded from memory instead of being
>> > cached in a register since it's value could changed by non-C code ...
>>
>> The relevant factor isn't "C" versus "non-C",
>
> Sorry, "non-C" is inaccurate. I think of C as "a single-threaded execution
> model." By that, I mean there is no other C code executing in parallel,
> even though that's *not* the correct definition for single-threaded.
> Obviously, there could be C code executing in parallel with each modifying
> the same machine word. So, "non-C" should be "everything other than" the C
> code deemed to be active.
>
>> The relevant factor is [...] synchronous versus asynchronous [...]
>
> I don't see this as being relevant. His C code could be 100% synchronous
> and he could still need a volatile. His code could be single-threaded,
> single-core, single-processor, serial, non-parallel, not use setjmp/longjmp,
> and be entirely interrupt free. If hardware modifies his machine word that
> his C code is accessing, he still needs a volatile keyword for it.

Usually such an access by hardware would be asynchronous DMA -- but yes,
another use case for volatile could be if the compiler doesn't fully
understand the effects of things that the program does (though often in such
cases an implementation-specific barrier would be more appropriate). If
this happens without the program invoking behavior undefined by C, such as
disallowed aliases, poking at arbitrary addresses, or using an
implementation-defined facility, then that's a bug in the C implementation.

> AFAICT, James never fully stated what exactly is modifying his machine word.
> It could be other software - in C or in assembly. Or, it could be hardware.
> From his description, all I know is that it's "in the cloud" somewhere ...
> I.e., it might not even be on his own computer. Maybe it's only accessable
> from a another computer by a remote procedure call over a network. Without
> knowing exactly what is modifying his machine word, I'm not sure how he is
> going to guarantee non-volatility and/or atomicity.

It seemed clear from the context that it was to be set by the called
function or some descendent in the call stack -- that is, implementing
functionality similar to what other languages call exceptions. The thread
subject isn't "network messaging in C", or "device drivers in C". :-)

If it were to be set anywhere other than during a function call, how would
you know when to check, even with volatile?

>> (or perhaps, outside this instance of the single-threaded C
>> execution model).
>>
>
> Yes. Or, I like that better, although technically neither of us is being
> precise enough. I'm not entirely sure of the correct terminology.
> Single-threaded doesn't specifically exclude parallel processing for the
> single-thread. Single-threaded just means a single command. E.g., one or
> more parts of a single-threaded and in parallel C application could optimize
> away use of the variable because it's unaware of the other parts' usage of
> it.

Could you be more specific? I have a hard time picturing something that
doesn't involve either a broken C environment or something that I wouldn't
describe as single-threaded.

-Scott

[Scott Wood]

On 2012-01-03, James Harris <james.h...@googlemail.com> wrote:
> On Dec 31 2011, 2:24 am, BGB <cr88...@hotmail.com> wrote:
>> On 12/30/2011 3:38 PM, James Harris wrote:
>
> I've snipped the bits about how TLS is generally accessed. Thanks for
> explaining that info. It was new to me.
>
> ...
>
>> > I was thinking of thread_excep being at a fixed location so it could
>> > literally be checked with one instruction:
>>
>> >    cmp [thread_excep], 0
>>
>> if it is a global though, it is not a TLS.
>>
>> otherwise, I guess it would need to be changed around by the scheduler
>> or something (register variables with scheduler, and it saves/restores
>> their values on context switches or something?...).
>
> Yes. The task switcher would include code along the lines of
>
> push [thread_excep]
>
> for the outgoing task and, for the incoming task,
>
> pop [thread_excep]
>
> Since the word would be checked many times between task switches this
> is much faster than following a structure off FS or similar.

You'd have to maintain a different set of page tables on each CPU
in order to keep the address constant...

If you want exceptions, why not use a language with exceptions?

If you're OK with explicit error checking, use the return value or a
pointer argument.

-Scott

[Rod Pemberton]

"James Harris" <james.h...@googlemail.com> wrote in message

news:2e4cacd5-735c-4692...@z17g2000vbe.googlegroups.com...

On Dec 29 2011, 11:38 pm, "Rod Pemberton"
<do_not_h...@noavailemail.cmm> wrote:

...

> > > I cannot think of a way to implement that in C.
>
> > Tracing stackframes would result in non-portable or non-standard
> > C code. C is not required to use a stack.
>
> So I've heard but in my book C, being recursive, has to use a stack
> of function calls.

A stack makes it easy to implement recursion, but it's not required.

See "The Basic Issues" of "The C Language Calling Sequence" by D.M. Ritchie
http://cm.bell-labs.com/cm/cs/who/dmr/clcs.html

See "10.5 Functions and the calling sequence" and "11.1 Mapping and the user
program" of "Portability of C Programs and the UNIX System" by S.C. Johnson
and D.M. Ritchie.
http://cm.bell-labs.com/who/dmr/portpap.html

D.A. Gwyn on stacks in C:
http://groups.google.com/group/comp.lang.c.moderated/msg/fa067b2088bc759c

D.A. Gwyn on activation records:
http://groups.google.com/group/comp.lang.c/msg/788eb205f41f310e

> They may not go on a hardware stack but they have to
> form a stack.

Haven't you discussed "activation records" a handful of times on various
groups? I thought you did ...

The stack frame or activation records must be "connected", e.g., perhaps by
a linked list, but do not need to be contiguous like if they were stored on
a stack.

> I don't understand the point you are making here.

Sorry, I'm not quite sure which context you are asking about. Was it about
me just stating it was like #3? Or, was it about #3?

> > 2) may skip executing some code due to goto-ing the exception
>
> Don't you normally want to skip some code if you get an exception?

You may ... If continuing execution would cause some sort of failure, I'd
say yes, definately. I didn't take an exception to mean that current
processing must halt, only that the exception must be handled. If you do
skip, then you can expect the skipped code to not have been completed, even
if the data being processed by that procedure was correct. I.e., after the
exception, you may have to backup to a prior execution state and call the
procedure(s) again with the original data. Or, you may need assembly
routines to save and restore the exact execution state, so that it can be
started and stopped at will, i.e., by the exception or your debugger etc.
Or, you may have to save the execution state and relevant data somewhere
prior to calling that procedure, so you can resume after the execution.

> Your code seems to continue the normal processing path in the event of
> an exception, which is in line with the preference you mentioned above
> but it seems an odd approach.

Well, you'll have to decide what you want to do to restart or continue
execution after handling the interruption caused the exception. The
exception changes the control-flow. If you placed a line like "if(flag)
foo();" when foo() gets called, it would change the control-flow much like
an exception, but at a more controlled point in the execution. If calling
foo() doesn't affect the remainder of the procedure or stacked data, that's
great! If you placed a line like "if(flag) foo(); goto end_of_procedure",
then execution of some code gets skipped. That may be necessary, but should
be avoided (IMO). The result is whatever was going to execute without the
exception will not execute. After handling the exception, how do you resume
execution at the point where the exception occurred? Or, how do you resume
execution in a way that the program returns to that point with the same
data? That's the issue.

http://en.wikipedia.org/wiki/Software_transactional_memory


Rod Pemberton

[Rod Pemberton]

"Scott Wood" <sc...@buserror.net> wrote in message

news:slrnjgaem8...@snotra.buserror.net...
...

> If it were to be set anywhere other than during a function call, how would
> you know when to check, even with volatile?
>

Polling.

> >> (or perhaps, outside this instance of the single-threaded C
> >> execution model).
> >>
> >
> > Yes. Or, I like that better, although technically neither of us is
> > being precise enough. I'm not entirely sure of the correct terminology.
> > Single-threaded doesn't specifically exclude parallel processing for the
> > single-thread. Single-threaded just means a single command. E.g., one
> > or more parts of a single-threaded and in parallel C application could
> > optimize away use of the variable because it's unaware of the other
> > parts' usage of it.
>
> Could you be more specific? I have a hard time picturing something that
> doesn't involve either a broken C environment or something that I wouldn't
> describe as single-threaded.
>

Parallel.


Rod Pemberton

[James Harris]

On Jan 5, 11:43 am, "Rod Pemberton" <do_not_h...@noavailemail.cmm>
wrote:

> "James Harris" <james.harri...@googlemail.com> wrote in message

> On Dec 29 2011, 11:38 pm, "Rod Pemberton"

...

> > They may not go on a hardware stack but they have to
> > form a stack.
>
> Haven't you discussed "activation records" a handful of times on various
> groups?  I thought you did ...
>
> The stack frame or activation records must be "connected", e.g., perhaps by
> a linked list, but do not need to be contiguous like if they were stored on
> a stack.

I was trying to show that this is purely semantics. The data structure
to support C's procedure calling has to be a LIFO because C is
recursive. Even if implemented as a linked list it is still a LIFO at
the function call level. You challenged the term that I used
initially, "stack of function calls," and I was just explaining why I
used the term.

> > I don't understand the point you are making here.
>
> Sorry, I'm not quite sure which context you are asking about.  Was it about
> me just stating it was like #3?  Or, was it about #3?

:-) Sorry, ignore that. It must have been extraneous text left from a
draft of the reply.

> > > 2) may skip executing some code due to goto-ing the exception
>
> > Don't you normally want to skip some code if you get an exception?
>
> You may ...  If continuing execution would cause some sort of failure, I'd
> say yes, definately.  I didn't take an exception to mean that current
> processing must halt, only that the exception must be handled.  If you do
> skip, then you can expect the skipped code to not have been completed, even
> if the data being processed by that procedure was correct.  I.e., after the
> exception, you may have to backup to a prior execution state and call the
> procedure(s) again with the original data.

Yes, or fix the data if it was wrong. For example, say Program A was
told to read a file which did not exist. That program called a
subroutine Sub B which called subroutine Sub C and Sub C tried to open
the file. There's not a lot any of these can do to correct the error.
They cannot read from the file if it doesn't exist. So it would
normally be desirable for each to abandon what it was trying to do,
clean up any resources it had allocated and return to its caller.

I say "normally" as the program might want to sit in a loop
periodically trying to open the file until it existed. In which case
the exception in Sub C (which tried to open the file) would be handled
locally and not propagated. Or the condition could be propagated to
Sub B which would periodically call Sub C again until the file did
exist.

In any case it would be unusual to expect a routine to continue
through to the end of its normal execution and then look at what
exceptions had happened which is what you seemed to be suggesting. I
may have misunderstood your intent, though.

...

> > Your code seems to continue the normal processing path in the event of
> > an exception, which is in line with the preference you mentioned above
> > but it seems an odd approach.
>
> Well, you'll have to decide what you want to do to restart or continue
> execution after handling the interruption caused the exception.  The
> exception changes the control-flow.  If you placed a line like "if(flag)
> foo();" when foo() gets called, it would change the control-flow much like
> an exception, but at a more controlled point in the execution.

I like the idea, Rod, but when foo() finishes it returns to whatever
called it doesn't it? If so, how does it affect the control flow?

Something else to think about: What happens if foo() cannot recover
from the specific exception that caused it to be called?

>  If calling
> foo() doesn't affect the remainder of the procedure or stacked data, that's
> great!  If you placed a line like "if(flag) foo(); goto end_of_procedure",
> then execution of some code gets skipped.  That may be necessary, but should
> be avoided (IMO).

I agree to avoid skipping code if the exception can be recovered from
but not if the exception means the routine cannot continue. And if you
are going to recover from the exception it's probably more normal to
put both the op and the recovery in a loop such as,

for (x = NULL; x;) {
x = op();
if (flag) foo();
}

And you are still left with the issue of what to do if this loop
experiences an exception that foo cannot recover from.

> The result is whatever was going to execute without the
> exception will not execute.  After handling the exception, how do you resume
> execution at the point where the exception occurred?  Or, how do you resume
> execution in a way that the program returns to that point with the same
> data?  That's the issue.

Understood.

Continuing the example above, if you have an exception handler that
knows what to do with a file open exception but gets hit with
something else that it is not built to handle would you agree it
should take a sharp exit until it gets back to something that can deal
with the exception condition that happened?

The whole point of exception propagation is to find the right place to
handle the exceptional condition. That place can then take recover
action and can repeat the processing if appropriate. This massively
saves every routine being written to handle every type of exception
that can occur.

James

[James Harris]

On Jan 5, 7:25 am, Scott Wood <sc...@buserror.net> wrote:

...

> >> >    cmp [thread_excep], 0

...

> You'd have to maintain a different set of page tables on each CPU
> in order to keep the address constant...

I don't follow. The machine word cannot be shared among processes
executing in parallel. In fact each thread would have to have its own
copy.

> If you want exceptions, why not use a language with exceptions?

1. Exceptions are not the only requirement.

2. The scheme proposed will work between languages. I don't intend the
use of many languages but as an example, a Pascal routine could call
an assembly routine which could call a C routine which could call
another assembly routine and all of them could follow the same, very
fast, exception propagation protocol. (More accurately, the point is
that each would very quickly skip non-exceptions while still handling
exceptions.)

> If you're OK with explicit error checking, use the return value or a
> pointer argument.

This is definitely a possibility. The regularity of calling routines
a() and b() as follows is quite tempting.

if (ex = a()) {if (handler(&ex)) return ex;)
if (ex = b()) {if (handler(&ex)) return ex;}

Naturally, handler() would return non-zero if it did not handle the
exception and the normal returns from a() and b() would be &vars.
Perhaps the main things against it are,

1. It adds a parameter to each call. There is the question of whether
a return be added to routines which cannot generate an exception just
to maintain the regularity. The form is not used in system calls or C
library routines and is unfamiliar to many.

2. It wouldn't look quite so regular when the arguments are included.
Nor would it look so neat when many single-line statements had to be
converted to two lines because of the length of the text.

3. It doesn't allow a sticky exception state such as

a();
b();
if (thread_excep) /* An exception occurred in a() or b() */

If I wanted to add a sticky state I suppose I could write

if (ex |= b()) ...

but that puts the OR instruction (and possibly a memory update) into
the main non-exception processing path so would be slower in the
normal processing case.

I'm not ruling it out completely. It does do away with the goto.

James

[Ed Coran]

"Rod Pemberton" <do_no...@noavailemail.cmm> wrote in message
news:je428e$2fc$1...@speranza.aioe.org...

>
> "James Harris" <james.h...@googlemail.com> wrote in message
> news:2e4cacd5-735c-4692...@z17g2000vbe.googlegroups.com...

>> Your code seems to continue the normal processing path in the event of
>> an exception, which is in line with the preference you mentioned above
>> but it seems an odd approach.
>
> Well, you'll have to decide what you want to do to restart or continue
> execution after handling the interruption caused the exception.

The concept you both seem to be ignorant of is of resumptive vs.
termination exception handling semantics. If the suggestion is for
resumption semantics, then that would indeed be "odd", for most of the
industry has rejected that in favor of termination semantics, for better
or worse. There are exactly 3 choices for an implementation of exception
handling: resumption, termination, both (either applied as appropriate in
context). Stroustrup's "D&E" gives a brief introduction to the debate,
but note that the book is dated. Internet search on the other terms
should give loads of results.

[Rod Pemberton]

"James Harris" <james.h...@googlemail.com> wrote in message

news:b0b03ad9-12ca-4356...@m4g2000vbc.googlegroups.com...

> On Jan 5, 11:43 am, "Rod Pemberton" <do_not_h...@noavailemail.cmm>
> wrote:
> > "James Harris" <james.harri...@googlemail.com> wrote in message
> > On Dec 29 2011, 11:38 pm, "Rod Pemberton"
...

> For example, say Program A was
> told to read a file which did not exist. That program called
> a subroutine Sub B which called subroutine Sub C and
> Sub C tried to open the file. There's not a lot any of these
> can do to correct the error.

Then do this:

> They cannot read from the file if it doesn't exist. So it would
> normally be desirable for each to abandon what it was trying to do,
> clean up any resources it had allocated and return to its caller.

Yes.

> I say "normally" as the program might want to sit in a loop
> periodically trying to open the file until it existed.

Or, exited ...

But, that consumes cpu cycles on a task which is now known to be impossible
to complete. Hopefully, it would loop until a user cancels the action, or
well placed code determines the file doesn't exist and attempts to handle
the situation.

> In which case the exception in Sub C (which tried to open the file)
> would be handled locally and not propagated. Or the condition could
> be propagated to Sub B which would periodically call Sub C again
> until the file did exist.

I don't see these as being exceptions. Exceptions indicate an abnormal,
unexpected change to the program flow.

> In any case it would be unusual to expect a routine to continue
> through to the end of its normal execution and then look at what
> exceptions had happened which is what you seemed to be suggesting. I
> may have misunderstood your intent, though.

In that case, you have a situation which directly impacts the data or
operations to be performed. Let's say you are processing a financial
transaction. Sub C is what attempts to save the completed transaction. At
just that moment, "Klutz the Trader", accidentally disconnects his keyboard.
So, you get a keyboard exception interrupting Sub C just prior to saving the
critical data. The two are unrelated or independent of each other. What do
you? You need to make sure to continue or return somehow into Sub C to save
that critical data. If you can't, you're in some serious job related
trouble.

> I like the idea, Rod, but when foo() finishes it returns to whatever
> called it doesn't it? If so, how does it affect the control flow?

It was just intended as an example in C of how an exception interrupts a
routine at some random location. If foo() is only called for an
"exception", it "disrupts" the "normal" control flow which is without it.
An actual exception handler may or may not return and will likely interrupt
at a random assembly instruction, i.e., not at a completed C operation,
which I think they call a "sequence point" or somesuch ...

> Something else to think about: What happens if foo() cannot
> recover from the specific exception that caused it to be called?

Then, you have a problem. It may be trivial, i.e., missing mouse movement
or keyboard data, which can be ignored for the most part. Hopefully, the
user notices and corrects such errors. Or, it may be really serious, i.e.,
lost financial data. Truly serious stuff results in fines and legal
actions.

> I agree to avoid skipping code if the exception can be recovered
> from but not if the exception means the routine cannot continue.

...

> And if you are going to recover from the exception it's probably
> more normal to put both the op and the recovery in a loop such as,
>
> for (x = NULL; x;) {
> x = op();
> if (flag) foo();
> }
>
> And you are still left with the issue of what to do if this loop
> experiences an exception that foo cannot recover from.

You could ignore it and think of it as being 'fault tolerant'. It depends
on how serious the exception is and whether terminating the program is
acceptable or not. That's how it's coded now, since only op() can set x to
NULL after the first initialization of x.

Alternately, you could assume it's serious and abort, but you could lose
data:

for (x = NULL; x;) {
x = op();

if (flag) x = foo(x);
}
exit(0);

Perhaps, foo(&x) so x is passed in and changed to NULL if an error? x would
be preserved for no error in foo(). I.e., op(), foo(), and the
initialization of x can set x to NULL, but only op() can set x to non-NULL.

Without knowing all exceptions and ranking them as to severity, you're
probably not going to be able to preemptively identify all issues, e.g.,
memory corruption. It's not likely your application is running checksums on
memory blocks to ensure data integrity. The hardware is just expected to
work correctly, and if it's fault tolerant hardware, it's probably doing
error correction and detection for you.

> Continuing the example above, if you have an exception handler that
> knows what to do with a file open exception but gets hit with
> something else that it is not built to handle would you agree it
> should take a sharp exit until it gets back to something that can deal
> with the exception condition that happened?

What do you mean by "sharp exit"? return(err)? exit(0)?

If it's truly:

"something else that it is not built to handle"

then there is not:

"something that can deal with the exception condition that happened"

...

So, no I would not agree to that it is always best to take a "sharp exit" -
whatever you meant by that. However, passively ignoring the exception is
probably the best simple solution for a more 'fault tolerant' approach, if
you know that most exceptions will be something non-serious or ignorable.
If you know in advance that something really serious could occur, you should
attempt to handle it.

> The whole point of exception propagation is to find the right place

> to handle the exceptional condition. That place can then take recover[y]

> action and can repeat the processing if appropriate. This massively
> saves every routine being written to handle every type of exception
> that can occur.

...


Rod Pemberton

[Rod Pemberton]

"Ed Coran" <sys...@new.net> wrote in message
news:je4rcd$v5f$1...@speranza.aioe.org...

> "Rod Pemberton" <do_no...@noavailemail.cmm> wrote in message
> news:je428e$2fc$1...@speranza.aioe.org...
> > "James Harris" <james.h...@googlemail.com> wrote in message
> >
news:2e4cacd5-735c-4692...@z17g2000vbe.googlegroups.com...
>
> >> Your code seems to continue the normal processing path in the event of
> >> an exception, which is in line with the preference you mentioned above
> >> but it seems an odd approach.
> >
> > Well, you'll have to decide what you want to do to restart or continue
> > execution after handling the interruption caused the exception.
>
> The concept you both seem to be ignorant of is of resumptive vs.
> termination exception handling semantics.

Thanks. Those terms are nice, IMO.

> If the suggestion is for resumption semantics, then that would indeed
> be "odd", for most of the industry has rejected that in favor of
> termination semantics, for better or worse.

I can tell you that that's definately not the case where legal requirements
prohibit the application from being unavailable. Fault tolerance is
required for various industries by law in the USA: banking, securities
trading, medical devices, automotive ICMs, etc.


Rod Pemberton

[James Harris]

On Jan 6, 12:05 am, "Rod Pemberton" <do_not_h...@noavailemail.cmm>
wrote:

...

> An actual exception handler may or may not return and will likely interrupt
> at a random assembly instruction, i.e., not at a completed C operation,
> which I think they call a "sequence point" or somesuch ...

There is an important conceptual issue here. There is no way an
exception or an exception handler should interrupt a random assembly
instruction (your phrase). A signal, perhaps, but not an exception.
Exceptions should be synchronous, IMHO, i.e. occurring in lock-step
with the instruction or call that led to them.

> > Something else to think about: What happens if foo() cannot
> > recover from the specific exception that caused it to be called?
>
> Then, you have a problem.  It may be trivial, i.e., missing mouse movement
> or keyboard data, which can be ignored for the most part.  Hopefully, the
> user notices and corrects such errors.  Or, it may be really serious, i.e.,
> lost financial data.  Truly serious stuff results in fines and legal
> actions.

You don't have a problem if the default action is to return to the
caller. The handler can still catch anything it knows how to deal with
and wants to deal with.

Silent failure is pernicious. It is far better to report an issue (and
possibly to allow some form of continuation) than to proceed silently
with corrupt data. As you say, missing mouse packets or keystrokes are
unimportant or self-correcting due to the feedback loop and, if they
do in fact generate exceptions, they can be explicitly allowed for but
if your code is blissfully ignoring errors thinking only of mouse
packets or lost keystrokes and an important and significant exception
hits your code will be unaware of it.

...

> > And you are still left with the issue of what to do if this loop
> > experiences an exception that foo cannot recover from.
>
> You could ignore it and think of it as being 'fault tolerant'.  It depends
> on how serious the exception is and whether terminating the program is
> acceptable or not.

The key point about making exceptions propagate *by default* is
protection. You can always code to ignore harmless exceptions knowing
if something you have not anticipated strikes that the program will
say so.

> That's how it's coded now, since only op() can set x to
> NULL after the first initialization of x.
>
> Alternately, you could assume it's serious and abort, but you could lose
> data:

Far better than giving the answer 42 when it should be 53!

>    for (x = NULL; x;) {
>      x = op();
>      if (flag) x = foo(x);
>    }
>    exit(0);
>
> Perhaps, foo(&x) so x is passed in and changed to NULL if an error?

The problem with this is it makes foo() specific to the allocation to
x or the result from op() but that's maybe OK, depending on the rest
of the code.

...

> > Continuing the example above, if you have an exception handler that
> > knows what to do with a file open exception but gets hit with
> > something else that it is not built to handle would you agree it
> > should take a sharp exit until it gets back to something that can deal
> > with the exception condition that happened?
>
> What do you mean by "sharp exit"?  return(err)?  exit(0)?

I mean a hasty return to the caller, and for that program, if it
doesn't know how to deal with the exception that occurred, to make a
similar hasty return to its caller etc until we get to a routine that
is set up to handle the issue. This might be the OS in which case the
app will have been terminated (with an exception report).

By "a sharp exit until it gets back to something that can deal with
the exception" I didn't mean the exit() call. It can be a pain, making
a routine uncallable if we want to be sure to get control back!

> If it's truly:
> "something else that it is not built to handle"
> then there is not:
> "something that can deal with the exception condition that happened"
> ...
>
> So, no I would not agree to that it is always best to take a "sharp exit" -
> whatever you meant by that.

Not "always" but "by default." There should be a sharp exit to the
caller by default. Then we still have the option of handling as many
exceptions as we want knowing that any we don't anticipate will
ultimately generate a report rather than being silently ignored.

> However, passively ignoring the exception is
> probably the best simple solution for a more 'fault tolerant' approach, if
> you know that most exceptions will be something non-serious or ignorable.

I can't agree. If you knew that *all* exceptions that could occur at a
certain point were benign this would be fine but if only *most*
exceptions would be ignorable there's always the chance that a
malignant exception happens. (You can still code to ignore the
harmless exceptions.) If you ignore the harmful exceptions too the
program could do something undefined and, more to the point, do it
*silently*. A program that fails silently to give the correct results
can be used for years without anyone being aware that what it is
saying is wrong. Exception handling should protect us from that.

> If you know in advance that something really serious could occur, you should
> attempt to handle it.

You know the term fail-safe. AIUI the original meaning of that is that
is something failed it would fail to a safe condition rather than
failing to either an unsafe condition or an indeterminate condition. I
guess I like a fail-safe system whereas you are an optimist. ;-)

At least I understand your view more now.

James

[James Harris]

On Jan 6, 12:06 am, "Rod Pemberton" <do_not_h...@noavailemail.cmm>
wrote:

> "Ed Coran" <sys...@new.net> wrote in message

...

> > If the suggestion is for resumption semantics, then that would indeed
> > be "odd", for most of the industry has rejected that in favor of
> > termination semantics, for better or worse.
>
> I can tell you that that's definately not the case where legal requirements
> prohibit the application from being unavailable.  Fault tolerance is
> required for various industries by law in the USA: banking, securities
> trading, medical devices, automotive ICMs, etc.

Applications are made fault tolerant by recognising fault conditions
and responding to the causes of the faults, not by ignoring the faults
and carrying on as if they weren't happening!!!

James

[Ed Coran]

"Rod Pemberton" <do_no...@noavailemail.cmm> wrote in message

news:je5dqs$9aj$1...@speranza.aioe.org...

> "Ed Coran" <sys...@new.net> wrote in message
> news:je4rcd$v5f$1...@speranza.aioe.org...
>> "Rod Pemberton" <do_no...@noavailemail.cmm> wrote in message
>> news:je428e$2fc$1...@speranza.aioe.org...
>> > "James Harris" <james.h...@googlemail.com> wrote in message
>> >
> news:2e4cacd5-735c-4692...@z17g2000vbe.googlegroups.com...
>>
>> >> Your code seems to continue the normal processing path in the event
>> >> of
>> >> an exception, which is in line with the preference you mentioned
>> >> above
>> >> but it seems an odd approach.
>> >
>> > Well, you'll have to decide what you want to do to restart or
>> > continue
>> > execution after handling the interruption caused the exception.
>>
>> The concept you both seem to be ignorant of is of resumptive vs.
>> termination exception handling semantics.
>
> Thanks. Those terms are nice, IMO.

(Aside: Save for my inconsistent use of the English language: I used both
"ive" and "ion"... but it "sounded right" without thinking about it. I
think I need to learn how to conjugate verbs better or something. Or read
more of the literature so that I can relay it correctly. Not that I don't
add any value to the information I process though!).

>
>> If the suggestion is for resumption semantics, then that would indeed
>> be "odd", for most of the industry has rejected that in favor of
>> termination semantics, for better or worse.
>
> I can tell you that that's definately not the case where legal
> requirements
> prohibit the application from being unavailable. Fault tolerance is
> required for various industries by law in the USA: banking, securities
> trading, medical devices, automotive ICMs, etc.
>

I see you have not looked up the concepts of "resumption" and
"termination". Specifically, I'll bet that you were thinking when you
wrote the above that "termination EH semantics" means aborting or
terminating a program, or something similar to that. Well, it doesn't.
The concepts refer to a "micro" level rather than a "macro" level, so to
speak.

On a separate note, EH and fault tolerance are not the same thing at all,
but I wouldn't go so far as to call them orthogonal necessarily, but I do
think that they can indeed be so ("obviously"). I say "obviously" because
when I think of fault tolerance, I think of orthogonal systems! For the
newbs, that means, at the simplest, redundant independently developed
systems, and there is a range of possibility between the jet fighter
control system and the corporate database system, but those are not the
endpoints, of course. But let's not delve int fault tolerance in this
thread, for it is off topic: the topic is exception handling.

[Ed Coran]

You are using "fault tolerance" as kinda synonymous with "exception
handling", apparently (yes?). Apples and meat (i.e., not even the same
food group). You have to change something in your above statement to make
it even close to being correct. Fault tolerance is a separate subject and
not germane to the thread you started in your OP. Fault tolerance is EASY
to drop from this thread pronto, and I suggest that be done to bring the
thread back on track. I say "easy" because you and RP seem to be throwing
around "fault tolerance" in place of "exception handling", and they are
not the one in the same. Again though, fault tolerance is not germane to
the topic of the thread you started, so save that for another day
(suggestion).

Do you (and you too RP) agree? I.e., that the thread topic is "exception
handling" (whatever that is)? You don't have to agree that it is not
about "fault tolerance" (whatever that is), of course, because you
conflate that concept with EH. I find this thread interesting, because it
is a problem I solved years ago under NDA. The first thing I did was
learn the concepts (no laughter please!). I guestimate that I have a few
years of my existence with EH on my mind and, while not an academic, I'm
satisfied with my understanding of problem and the solutions to it. It's
definitely not something someone can learn in a day. It takes time to
absorb it and then time to become comfortable with it to the point that
you are confident in implementing or using it. While time may be a bitch,
she not necessarily a harsh mistress (save for the adolescent boys
"wishing to jump her bones"!).

[Ed Coran]

Rod Pemberton wrote:
> "James Harris" <james.h...@googlemail.com> wrote in message
> news:b0b03ad9-12ca-4356...@m4g2000vbc.googlegroups.com...

>> They cannot read from the file if it doesn't exist. So it would
>> normally be desirable for each to abandon what it was trying to do,
>> clean up any resources it had allocated and return to its caller.
>
> Yes.

Those are ("that is"?) termination semantics. Before you were on the
resumption semantics bandwagon?

> I don't see these as being exceptions. Exceptions indicate an
> abnormal, unexpected change to the program flow.

That is so incorrect that I have to post that it is incorrect. In an area
where there is a lot of confusion by many, it serves no one to just shoot
off a message without thinking about its correctness. As in most of
programming, one must go backward for a time (sometimes a loooong time)
before one can move forward. With that statement, you put the thread at
the proverbial "square one". I.e., "What is an "exception""?. You cannot
have a discussion about "exceptions" until you define what an "exception"
is. That's a bit of a fib, for you can have a discussion once there is
AGREEMENT of what an "exception" is for the discussion at hand. It's the
new year though, and everyone and their brother knows by now what an
"exception" is, so how come you don't? ;)

(Aside: EM is what separates the ... from the ... in SW dev). <-
politically correct attempt. Now, are you a programmer or a software
developer?

>
>> In any case it would be unusual to expect a routine to continue
>> through to the end of its normal execution and then look at what
>> exceptions had happened which is what you seemed to be suggesting. I
>> may have misunderstood your intent, though.
>
> In that case, you have a situation which directly impacts the data or
> operations to be performed. Let's say you are processing a financial
> transaction. Sub C is what attempts to save the completed
> transaction. At just that moment, "Klutz the Trader", accidentally
> disconnects his keyboard. So, you get a keyboard exception
> interrupting Sub C just prior to saving the critical data. The two
> are unrelated or independent of each other. What do you? You need
> to make sure to continue or return somehow into Sub C to save that
> critical data. If you can't, you're in some serious job related
> trouble.

Did you say "continue OR return"? Resume or terminate? ;)

>> I like the idea, Rod, but when foo() finishes it returns to whatever
>> called it doesn't it? If so, how does it affect the control flow?
>
> It was just intended as an example in C of how an exception
> interrupts a routine at some random location.

No, no! Now you are confusing synchronous and asynchronous scenarios (or
at least, throwing around the well-defined terms with reckless abandon)!

[Snipped the rest, for the above is enough to not consider the rest.]

>> Something else to think about: What happens if foo() cannot
>> recover from the specific exception that caused it to be called?

Now that hints at what separates "architects" from "programmers". This is
an easy one to teach. Use C++: double-throw will send you packin'. Ha!
Apparently some rules are good. Analogy (or metaphor, I never bothered to
learn the diff between those, wouldn't I find it funny how someday my
life would revolve around those simple "definitions"? I'm just kidding):
choo-choo train not very good without choo-choo train tracks! Random acts
of software, similarly, are not good.

>> I agree to avoid skipping code if the exception can be recovered
>> from but not if the exception means the routine cannot continue.

BTW, what is an "exception"? ;)

> ...
>
>> And if you are going to recover

Define "recover". (But not before you define "exception"!).

>> from the exception it's probably
>> more normal to put both the op and the recovery in a loop such as,

Are you thinking like a programmer rather than like an architect?

[Snipped studental ponderances ;)]

>> Continuing the example above, if you have an exception handler

What is an "exception handler"? What framework are you talking about?

> If you know in advance that
> something really serious could occur, you should attempt to handle it.

"Wrong answer" (i.e., from a client perspective). Software companies are
engaged to deliver reliable software as per spec, rather than "you
should, um, maybe, kinda, handle that".

>
>> The whole point of exception propagation is to find the right place
>> to handle the exceptional condition.

No, that's not correct.

>> That place can then take
>> recover[y] action and can repeat the processing if appropriate.

That would be resumption, albeit, a bit different than what I said
before.

>> This
>> massively saves every routine being written to handle every type of
>> exception that can occur.

Ha!! You cannot say that without addressing the issue of "neophytes"
"throwing it over the wall"!

*****

(I addressed both JH and RP in this post)

[Ed Coran]

James Harris wrote:
> On Jan 6, 12:05 am, "Rod Pemberton" <do_not_h...@noavailemail.cmm>
> wrote:
>
> ...
>
>> An actual exception handler may or may not return and will likely
>> interrupt at a random assembly instruction, i.e., not at a completed
>> C operation, which I think they call a "sequence point" or somesuch
>> ...
>
> There is an important conceptual issue here.

Indeed. It is that it is incorrect to have this much convo when there is
not an elementary agreement and understanding of terminology and
concepts. (Aside: I miss the ACM).

> There is no way an
> exception or an exception handler

What are those things?

> should interrupt a random assembly
> instruction (your phrase).

He did, and I jumped all over that.

> A signal, perhaps, but not an exception.
> Exceptions should be synchronous,

Ya think? What if my framework exploits Windows SEH and now asynch is now
too under my control (read, at my own "peril")?

> You don't have a problem if the default action is to return to the
> caller. The handler can still catch anything it knows how to deal with
> and wants to deal with.

Apparently you have a framework (a context) in mind. AOD S/N: low now.

>
> Silent failure is pernicious. It is far better to report an issue (and
> possibly to allow some form of continuation) than to proceed silently
> with corrupt data.

K&R C! If RP put this thread back to definition of "exception", you just
put it back to ... 1969?

> As you say, missing mouse packets or keystrokes are
> unimportant or self-correcting due to the feedback loop and, if they
> do in fact generate exceptions, they can be explicitly allowed for but
> if your code is blissfully ignoring errors thinking only of mouse
> packets or lost keystrokes and an important and significant exception
> hits your code will be unaware of it.
>

Have you (both) considered that fresh out of school lads (there are no
girls in IT), already know all this stuff you are pondering about?

> The key point about making exceptions propagate *by default* is
> protection.

What is an "exception"? What is "exception propagation"?

> You can always code to ignore harmless exceptions

What is a "harmless exception"?

> knowing
> if something you have not anticipated strikes that the program will
> say so.

Did you just say: "I'm going to live forever, so I'm not gonna worry
about it."?

>>> Continuing the example above, if you have an exception handler that
>>> knows what to do with a file open exception but gets hit with
>>> something else that it is not built to handle would you agree it
>>> should take a sharp exit until it gets back to something that can
>>> deal with the exception condition that happened?
>>
>> What do you mean by "sharp exit"? return(err)? exit(0)?
>
> I mean a hasty return to the caller,

A ha! Termination semantics! ;) Tis no haste though (a topic for another
NG).

>> If it's truly:
>> "something else that it is not built to handle"
>> then there is not:
>> "something that can deal with the exception condition that happened"
>> ...
>>
>> So, no I would not agree to that it is always best to take a "sharp
>> exit" - whatever you meant by that.
>
> Not "always" but "by default."

There is no need in the area of EM for "sharp exit" though.

> There should be a sharp exit

Gibberish. "sharp exit" is unknown and unnaccepted in the vernacular, but
I will bet that it will be rejected if proposed.

> You know the term fail-safe.

No, he does not. Do you? (I don't know ENTIRELY it either (only realtime
SW devs know that), but I don't need to because I am not a realtime
software developer). Oh, I need to know it, it's just not my priority
right now. But you throw it around like a dish of salsa.

> AIUI the original meaning of that is that
> is something failed it would fail to a safe condition rather than
> failing to either an unsafe condition or an indeterminate condition. I
> guess I like a fail-safe system whereas you are an optimist. ;-)
>

Off topic.

[Rod Pemberton]

"James Harris" <james.h...@googlemail.com> wrote in message

news:8d3900c2-05d7-483a...@j9g2000vby.googlegroups.com...

The hardware can be fault tolerant.

You're correct in that software faults have to be found and fixed when
noticed, but that can be done long after the fact, if the environment
supports it. I.e., you can save the data and application's entire state (or
"core") and debug it or test it on a development system instead of a
production system.


Rod Pemberton

[Rod Pemberton]

"Ed Coran" <sys...@new.net> wrote in message

news:je604q$97g$1...@speranza.aioe.org...
>
...

> That is so incorrect that I have to post that it is incorrect. In an area
> where there is a lot of confusion by many, it serves no one to just shoot
> off a message without thinking about its correctness. As in most of
> programming, one must go backward for a time (sometimes a loooong time)
> before one can move forward. With that statement, you put the thread at
> the proverbial "square one". I.e., "What is an "exception""?. You cannot
> have a discussion about "exceptions" until you define what an "exception"
> is. That's a bit of a fib, for you can have a discussion once there is
> AGREEMENT of what an "exception" is for the discussion at hand.

I started programming in 1981. AFAIK, there is nothing wrong with what I
stated. My terminology may not match CS usage since I'm from an EE
background. Wikipedia has been helpful in providing names for what I know.
So, if I've forgotten something over time, or don't know the modern
terminology, or am using obsolete terminology, you'll just have to forgive
me.

> Now, are you a programmer or a software developer?

I was and am both.

> No, no! Now you are confusing synchronous and asynchronous
> scenarios (or at least, throwing around the well-defined terms
> with reckless abandon)!

Given my EE background with experience in both digital and analog
electronics, I most definately am not.

> (I addressed both JH and RP in this post)

We noticed. It's awkward to follow. Stop it. Please.


Rod Pemberton

[Rod Pemberton]

"Ed Coran" <sys...@new.net> wrote in message

news:je62hn$d2j$1...@speranza.aioe.org...
>
> Off topic.

Nothing is off-topic in an alt.* heirarchy Usenet group.

comp.lang.c is not involved in this part of the thread either.

You certainly have the *tone* of our anonymous friend "Fritz Weuhler" ...
"Fritz" is that you?

Hmm ...

MSOE on XPSP3 via AIOE.org. How did you come across AIOE? One of my posts?


RP

[Rod Pemberton]

"Ed Coran" <sys...@new.net> wrote in message

news:je5t2l$4bb$1...@speranza.aioe.org...
>
...

> I see you have not looked up the concepts of
> "resumption" and "termination".

Provide a link to either on Wikipedia. I'll take a look.

How do you see that exactly ... ?

> But let's not delve int fault tolerance in this
> thread, for it is off topic: the topic is exception handling.

The topic is whatever James is willing to discuss. He is the only one
posting to a.o.d. lately.


Rod Pemberton

[Rod Pemberton]

"James Harris" <james.h...@googlemail.com> wrote in message

news:87e6a259-4d7d-490a...@z1g2000vbx.googlegroups.com...

> On Jan 6, 12:05 am, "Rod Pemberton" <do_not_h...@noavailemail.cmm>
> wrote:
...

> Silent failure is pernicious.

It depends on the failure.

> > Alternately, you could assume it's serious and abort, but you could
> > lose data:
>
> Far better than giving the answer 42 when it should be 53!

If "fault-tolerance" is the goal or the program cannot be terminated until a
certain criteria, e.g., 4:15pm, then aborting and exiting is not a choice.
You have to use STM (software transactional memory) or other fault tolerant
methods to preserve data. So, what are your goals?

> > for (x = NULL; x;) {
> > x = op();
> > if (flag) x = foo(x);
> > }
> > exit(0);
> >
> > Perhaps, foo(&x) so x is passed in and changed to NULL if an error?
>
> The problem with this is it makes foo() specific to the allocation to
> x or the result from op() but that's maybe OK, depending on the rest
> of the code.

I don't know what you mean ...

Nothing indicated where you allocated x. I.e., it's indeterminable whether
x is an an auto with procedure scope or a "global" with file scope, etc, not
that it matters. And, you have a choice as to how x is passed:

foo(&x); /* address passed */
x=foo(x); /* value passed and returned */

Yes, both were there previously. There is no need to look back ... ;-)

> > > [...] take a sharp exit [...]
>
> > What do you mean by "sharp exit"? [...]

>
> I mean a hasty return to the caller, and for that program, if it
>doesn't know how to deal with the exception that occurred,
> to make a similar hasty return to its caller etc until we get to
> a routine that is set up to handle the issue. This might be the
> OS in which case the app will have been terminated
> (with an exception report).

Is there something wrong with logging whatever is known about the unknown
exception and continuing execution? I'm assuming the exception and
execution are independent of each other, at least I think that's more
typical than an all out failure which must be aborted.

> There should be a sharp exit to the caller by default.

It depends on your goals.

> > However, passively ignoring the exception is probably the best
> > simple solution for a more 'fault tolerant' approach, if you know
> > that most exceptions will be something non-serious or ignorable.
>
> I can't agree. If you knew that *all* exceptions that could occur at a
> certain point were benign this would be fine but if only *most*
> exceptions would be ignorable there's always the chance that a
> malignant exception happens.

What do you intend to do about a "malignant" exception when it happens if
you don't know what it is or how to code code to handle it, assuming that
you even recognized that it occurred? The program has no method to contain
any complications such an exception could cause, if in fact it does cause
them. So, which do you think is more likely: a) it's "benign" b) it's
"malignant"? That will decide your solution: a) exit b) ignore.

> If you ignore the harmful exceptions too the
> program could do something undefined and, more to
> the point, do it *silently*.

So?

If you exit(0), that's done *explosively* ...

Your focus seems to be on only one side of the coin so to speak,
when the opposite side can have the same theoretical issues.

If you silently ignore, it's possible that data corruption will occur. If
you explosively exit, it's possible that data corruption will occur too.
The data error when silently ignoring could be due to "corrupted" code
overwriting the data. The error when explosively exiting could be due to
not flushing memory buffers to disk.

> A program that fails silently to give the correct results
> can be used for years without anyone being aware that
> what it is saying is wrong.

Yes, that's true. I've dealt with that in the code of others ...
as well as my own.

At the brokerage I worked for about a decade ago, I found and fixed one bug
that hadn't been located in over twenty-five years. I was also given the
task of fixing another that had been in place for ten years before it was
noticed, i.e., negative numbers on the financial reports. That one was
simply a missing paren for a block of code that had been "cut-n-pasted".

In my own code, one such example was a simple directory search program. I
extended it at some point later on to handle other directory related things.
It would work correctly for shorter directory paths, but fail for some of
the longer ones. Obviously, all test cases were against short directories.
It took quite a while and a bit of use to notice that not all directories
where being recursed. The program itself did not generate any catchable
errors or exceptions. It did what it was coded to do. This was a coding
error on my part.

If you've got a low-use program or a program which works correctly 90%+
percent of the time, you might not ever find or notice an error. For
personal C programs, I frequently skip checks for returned NULLs on the
various string functions. It can take quite a bit of extra code to make
them 100% "fail-safe". I expect the input data to match what I coded the
routine to handle. Occasionally, it doesn't and I have to fix it. E.g., if
I'm searching for '\r', I expect the input string to have an '\r', instead
of not having one and so returning a NULL which is then dereferenced.
Sometimes it's easier to pad, e.g., with '\r' when searching for it, than it
is to check for a NULL.

> Exception handling should protect us from that.

Lol ...

> You know the term fail-safe. AIUI the original meaning of that is that
> is something failed it would fail to a safe condition rather than
> failing to either an unsafe condition or an indeterminate condition.

If the fail-safe perspective contradicts the fault-tolerance perspective,
how do you reconcile the two? Do you preference fail-safe or
fault-tolerance? It's your choice as to which way to go.

If I get an NMI in my OS, I know I don't have code in place to fix whatever
triggered it. In fact, it took quite a bit of work just to determine what
could possibly trigger NMIs. I now know many of the things that will
supposedly trigger it. I accumulated a list of 31 or so items. Most of
them are non-critical, some are user events, some are unfixable via
software, etc. I'm not aware of any standard and current (at least post
486...) hardware that uses NMI (yet). There is bunch of stuff on the list
such as old PCJr and PS/2 hardware, old DRAM memory failures, (supposedly)
DMA timeouts, local bus timeouts, watchdog cards, some soundcards, etc. So,
I've ignored NMI errors. If PCI uses it upon timeout or somesuch, I may
have to change that. That's clearly not fail-safe as you've described it.
As you've described it, fail-safe would probably require a complete reboot
when an NMI occurs to eliminate any potential hardware sources of the
problem causing futher NMI errors. Fault-tolerant however can ignore it and
continue execution, probably without issue in 99% of the situations. For
things like double-faults, I still need to review the manuals to see what is
recoverable and what isn't since I haven't worked on it in a few years.

> I guess I like a fail-safe system whereas you are an optimist. ;-)

I worked on fault-tolerant systems for a while ... You couldn't legally
"exit(0)" the program. If you did, "you" (the company) were fined for
breaking the law. The program had to continue execution - preferably
correct execution - even if it experienced an "unrecoverable" and "unknown"
exception or signal or whatever. Fail-safe - as you've described it -
wasn't an option. We used software that did something similar to STM. I
think, perhaps, that fault-tolerant concepts corresponded to my own style
of programming.


Rod Pemberton

[BGB]

On 1/4/2012 6:26 AM, James Harris wrote:
> On Jan 3, 8:04 pm, BGB<cr88...@hotmail.com> wrote:
>> On 1/3/2012 4:19 AM, James Harris wrote:
>>> On Dec 31 2011, 2:24 am, BGB<cr88...@hotmail.com> wrote:
>
> ...
>
>>> Yes. The task switcher would include code along the lines of
>>
>>> push [thread_excep]
>>
>>> for the outgoing task and, for the incoming task,
>>
>>> pop [thread_excep]
>>
>>> Since the word would be checked many times between task switches this
>>> is much faster than following a structure off FS or similar.
>>
>> possibly, however the issue then partly becomes:
>> how does the task switcher know where the variable is?...
>
> Isn't this easy if you have a flat address space: just put the
> variable at a fixed location?
>

if the task switcher is part of the OS kernel (it generally is), then it
does not have direct visibility of variables declared in a user-space
binary.

>> possible options:
>> do it like the BIOS, with every task having a dedicated kernel area
>> which is also accessible to the userspace process;
>> the shared-object or DLL holding the variable is "owned" by the
>> OS/kernel (the kernel then depends on the presence of the library and on
>> the location of the variable within the library).
>
> ...
>
>> FWIW:
>> if one uses PE/COFF DLLs, typically all accesses to imported global
>> variables is indirect (and requires explicit declaration);
>> if one uses ELF, potentially nearly all access to global variables is
>> indirect (the typical strategy is to access them via the GOT except in
>> certain special cases).
>
> Does this mean that the global, G, in the following would not be
> accessed directly?
>
> int G;
> void sub() {
> while (G> 0) {
> G -= 1;
> sub();
> }
> }
> int main(int argc, char **argv) {
> G = argc;
> sub();
> return 0;
> }
>

it depends on the target (CPU mode, compilation mode, and object format).

if compiling for PE/COFF (EXE or DLL), it will be accessed directly.


for ELF, in this case it will generally be accessed indirectly via the
GOT (since the compiler will not know at this point where the variable
is declared, a "common" variable could be either within the local ".bss"
segment, or potentially imported from another SO, but this is only known
to the linker).

if it were initialized or static:
static int G;
or:
int G=0;

then the compiler would know that it was defined in the same compilation
unit, in which case:
on 32-bit x86, it will depend on whether or not this is PIC code (which
will generally depend on whether or not this is a Shared-Object).

on x86-64, the CPU has RIP-relative addressing, and so will presumably
use this to access the variable (for both SO's and non-SO binaries).

>> also, segment overrides are fairly cheap.
>
> True. It obviously depends on the CPU but in my tests, while segment
> loading cost a bit, segment overrides were completely free.
>

yep.

>> the cheapest option then would probably be to include whatever magic
>> state directly into the TEB/TIB/whatever, then be like:
>> mov eax, [fs:address]
>>
>> in some cases, accessing a variable this way could in-fact be
>> potentially cheaper than accessing a true global.
>
> As mentioned, I would expect a file-scope global to be accessed
> directly - as a machine word at a location that gets filled-in by the
> linker. Am I wrong? Are you are thinking of library code?
>
> ...
>

I am thinking for the case where an OS scheduler needs access to a
userspace global variable.

the kernel can't easily know about this case, unless something "special"
is done.

>>> You mention user-defined exceptions. I planned only one bit for them
>>> (as there is an arbitrary number of them). To distinguish one user
>>> exception from another would probably require a call to a routine that
>>> examines the detail.
>>
>> probably ok for a single app, but is dubious for an OS or general mechanism.
>
> Why is this no good? There can be any number of user exception types
> so it's not possible to have one bit for each. The best I could think
> of was to have one bit to indicate a user exception and functions to
> examine as much other detail as needed.
>

exceptions don't generally work this way.
yes, "one bit per type for every type" is absurd, but using one bit
per-type for certain exceptions but not others, isn't much better.

most exception handler systems will simply ignore the first and rethrow
the next.

some systems (those with Resumable Exceptions, such as Win32 SEH),
generally nest them sort of like a stack, so (AFAIK) exceptions are
handled in LIFO order provided all are resumable exceptions. if a
non-resumable exception occurs, then it will unwind and presumably there
is no way to handle prior exceptions.

>> Windows does this: if something goes wrong and an exception can't be
>> thrown in-app, Windows will simply kill the process.
>
> If an exception "can't be thrown"? What would prevent it being thrown?
>

stack overflow;
crash within dispatcher;
(Win64) if function prologues/epilogues are missing or illegible;
...

>> if something goes
>> wrong in-kernel, it is a blue-screen / BSOD.
>
> Not very good, though, is it! In reality I think that Windows or any
> OS will handle almost all exceptions. A BSOD should only occur on a
> hardware failure such as a machine-check exception. (It looks like
> Windows does run third-party drivers as privileged this increasing its
> vulnerability many-fold.)
>

nope.

apparently the Windows kernel treats pretty much any CPU exception
within the kernel as a BSOD-causing offense.

this leads to more BSODs, potentially, but the argument is that if
something has gone wrong in the kernel, it is better to try to kill the
system "gracefully" than to continue on in a potentially inconsistent state.

AFAIK, Linux tries a little harder, displaying a register dump and
similar and tries to recover, or failing this does a "kernel panic".


user-space exceptions can be handled more gracefully (raising signals or
invoking exception handlers, ...).

interestingly, the "This program has performed an illegal operation and
must be shut down." message is also an exception handler. if one crashes
the application hard enough, one will not see this, and instead the
process simply/silently dies/disappears.

in the past, this could leave dead/phantom windows (don't redraw, can't
be moved or closed, ...) but I suspect newer versions have addressed
this (any windows owned by the process are automatically destroyed even
if the process dies rather non-gracefully...).

I could be wrong on the details here, mostly speaking from old memories.

>> likewise for the CPU:
>> if an exception occurs within an interrupt handler, this is a
>> "doublefault" and generally leads to a reboot.
>>
>> an exception is not a status code...
>
> Following your analogy of a CPU you could perhaps think of each
> exception condition as an interrupt line that gets asserted until the
> condition causing the interrupt has been resolved. I know, as an
> analogy it's not perfect. :-(
>

flags and similar are used here.

so, normal interrupt occurs:
transfers to handler (and temporarily disables other interrupts);
handler does its thing;
iret (return from interrupt, restores prior CPU state, although GPRs/...
generally need to be manually saved/restored).

if things go bad in the first handler, then a double-fault occurs (a
dedicated special interrupt).

if the DF handler itself fails, the CPU reboots.


AFAIK, older CPUs would essentially just ignore whatever (maskable)
interrupts occurred during the handler, but newer ones will essentially
queue them and handle them in a FIFO order.


my memory gets a bit foggy here, as it has been a long time since I have
messed with this stuff...

>>> <<END
>>> Just on this point, and bringing it back specifically to C, if TLS is
>>> hard to obtain or slow to access there are two other possibilities
>>> that I don't think have been mentioned yet that spring to mind for use
>>> in C.
>>
>> it is slow, yes, but typically not enough to care about.
>> for most things people don't worry about a few clock-cycles here and
>> there (generally, it is big algorithm-level stuff that kills
>> performance, and not a lack of sufficient micro-optimization).
>>
>>> 1. If only running a single thread simply use a global. Job done.
>>
>> except when globals are slower.
>
> I'd like to understand why they would be. I tried compiling the code
> above with gcc and it seems to refer to G directly such as in sub()
> the compiler generates
>
> mov DWORD PTR G, %eax
>

but, with which OS and what compiler settings.

if this was Linux-i386 and "-shared" or "-fpic" was given, one would not
likely see a global being loaded this way.

if it was on Win32, then the above is what one would expect to see.

>>> 2. Use errno. Set it to zero at the start and check it where
>>> appropriate. On a user-detected exception that does not already set
>>> errno set it to a value which is outside the normal range (and create
>>> or append to the detailed exception object, as before).
>>
>> FWIW, depending on the C-library, errno may be in-fact a function call
>> wrapped in a macro.
>
> I know. I found that in some systems there is a call which returns the
> address of errno. In that case the compiler should (hopefully)
> remember that address for each use in a function.
>

this depends...

in most cases, it probably wont, since the compiler can't be sure that
the function wont return a different address each time (though I wont
rule out the possibility of some sort of a "this function will always
return the same value" modifier/attribute).

>>> Using errno does only allow one exception to be indicated at a time
>>> but that's the same as what I understand your suggestion to be. It
>>> also doesn't work between languages (unless they obtain the address of
>>> errno) but I think it could work well for C.
>>> END
>>
>>> At the end of the day, any scheme could be used. For performance the
>>> idea is that the exception-indicating word is either zero or non-zero.
>>> If it's non-zero one or more exceptions has/have occurred and must be
>>> dealt with.
>>
>> except that "exceptions as a status word" aren't really "exceptions" in
>> the traditional sense.
>>
>> may as well just call them status-codes, and leave the whole mess of
>> exception-handling out of this.
>
> Don't forget the original intention was speed. The point of a status
> word, as you call it, is to allow exception handling (or, non-
> handling) quickly. It is not an end in itself but a means of adding
> exceptions to C in a way that recognises exceptions in the right
> places while having the lowest possible impact on overall performance.
>

it is debatable that this is actually faster...

the problem is if the status is checked in each function (possibly
multiple times) then this may infact cost more overall than even
registering/unregistering exception handlers.

handlers capable of reflective unwinding (the GCC/DWARF strategy, Win64
SEH), only have a cost if an exception occurs, and are free otherwise
(apart from debatable "possibly contrivance" in the prologue/epilogue
case WRT Win64 SEH).


in my own (mostly unsused) ABI, I had done something like:
push ebp
mov ebp, esp
push ... ;push-register sequence, like in Win64
sub esp, ... ;space for locals/...
...
lea esp, [ebp-offs]
pop ...
pop ebp
ret
nop [magic_info] ;function metadata (optional)

this preserved a property that is desirable from typical with cdecl, but
is lost with both Win64 and SysV/AMD64: the ability to backtrack simply
using an EBP/RBP chain (with Win64, one has to unwind using awkward
logic, and on SysV/AMD64 one may be SOL apart from using DWARF as the
ABI does not mandate any sort of standardized frame pointer).

in the 32-bit case, my ABI would have been otherwise nearly identical to
cdecl, only differing in the use of special prologues and epilogues.


the 64-case was mostly similar to the 32-bit case and Win64, differing
from Win64 mostly in that the first 4 arguments are always passed on the
stack (and RBP being reserved as the frame pointer).

(gluing between them would generally be little more than a few 'mov'
instructions or similar).


in practice, I had mostly just used the Win64 and SysV/AMD64 ABIs (and a
custom name-mangling scheme). my implementation of the SysV/AMD64 ABI
was technically kind of crappy though, as I didn't bother much with
complex edge cases (passing/returning structs in registers, ...), and
due to technical reasons, arguments were stored to the stack and loaded
into registers just before the call operation.

otherwise, cross-ABI calls would require the use of costly
transfer-thunks (which basically save/restore registers and re-pack
argument lists, so a poorly-implemented SysV/AMD64 was at least a little
faster).

or such...

[BGB]

On 1/4/2012 11:01 AM, Jens Gustedt wrote:
> Am 01/02/2012 08:02 PM, schrieb BGB:
>> On 1/2/2012 9:49 AM, Jens Gustedt wrote:
>>> I think it is easy to do that even nicer, where exceptions would throw
>>> an integer value
>>>
>>> TRY {
>>> ...
>>> } CATCH(identifier) {
>>> case 1: ...
>>>
>>> default:
>>> printf("uncaught exception %d\n", identifier);
>>> RETHROW;
>>> }
>>>
>>> I'll probably implement something in that vein in the next weeks on
>>> top of P99.
>>>
>>
>> the main issue with directly using braces is that it doesn't give
>> anywhere to un-register the exception.
>
> The 'TRY' can hide a lot of things. In particular you can do scope
> bound resource manangement by using "for" scope variables. See e.g
>
> http://gustedt.wordpress.com/2010/08/14/scope-bound-resource-management-with-for-scopes/
>
>

didn't think of this...

>> catch with an integer could also use a FOURCC for well-known / named
>> exceptions.
>
> FOURCC ??
>

http://en.wikipedia.org/wiki/FOURCC

basically, 4 ASCII characters are encoded into a 32-bit integer, and
used as a magic number.

if one only allows printable ASCII characters, then one has all numbers
less than 0x20202020, or 538976288, which may be used as raw integer
values. similarly, all numbers >= 0x7F000000 are also outside the valid
printable ASCII range (this includes the entire set of negative integers
as well).

>> my current exception system uses dynamically-typed references
>> (generally, type-checking would be done as part of catching an
>> exception). generally, my type-system identifies types by type-name
>> strings (additionally, class/instance objects and certain C structs may
>> also be used, as these are built on top of the dynamic type system).
>
> Throwing void* instead of int would also be an option, indeed. You
> then could easily develop a system that uses some sort of struct to
> transfer the exception information.
>

what I am doing is a bit more involved than this.
note "dynamically typed": this isn't something nearly so simple as a raw
void pointer to a struct.


in my case though, it "may" be a pointer, but the type on the other end
depends on what is pointed-to.

one could potentially just pass the FOURCC as a fixnum, although a TWOCC
or THREECC would be better (since a fixnum doesn't have the full 32-bit
range on a 32-bit system, and any value outside of a 28-bit range in my
system would require using a boxed value, which is more expensive).

so, a THREECC would pass an exception name like 'GPF', 'IOX', 'BND', ...

> But, I'd prefer the solution I indicated over that. I would be
> basically a (hidden) switch statement for the CATCH block, things that
> look familiar to C programmers. And when throwing pointers to objects
> you'd always have the problem of the responsability for the allocated
> space.
>

only if it is a pointer to some sort of malloc()'ed memory or similar.

other options:
using a garbage collector (allocate and forget, the GC may reclaim it
later);
the reference doesn't actually point to any existent memory (it is an
abstract value, such as a handle or fixnum);
ownership doesn't transfer: some other 3rd party maintains
responsibility (such as the exception-dispatch subsystem);
...

[jacob navia]

Le 06/01/12 21:05, BGB a écrit :

> it is debatable that this is actually faster...
>
> the problem is if the status is checked in each function (possibly
> multiple times) then this may infact cost more overall than even
> registering/unregistering exception handlers.
>
> handlers capable of reflective unwinding (the GCC/DWARF strategy, Win64
> SEH), only have a cost if an exception occurs, and are free otherwise
> (apart from debatable "possibly contrivance" in the prologue/epilogue
> case WRT Win64 SEH).
>

I was the first to answer Mr Harris post (Dec 29th)

He never answered or acknowldged my post even if I pointed him to:

<quote from my message from Dec 29th>
Look at the exception handling code in
gcc. It implements an abstract machine that gets opcodes from a series
of tables generated by the compiler.

... snip ...

You should also look at the code of gcc. Specifically look in the
"except.c" file and in the unwind-xx-fde files. There you will
find the implementation
<end quote>

He ignored my advice and tried to defend his completely crazy schema
with some "machine word" global variable.

This is surprising since he declared (in another thread) that "gcc" was
his "leight weight" compiler that he would use.

Either:

1 Mr Harris doesn't *really* know what he is talking about

2 He wants to eliminate the exception handling mechanism of his
preferred compiler and figure out a new one AND hack gcc to
implement that...

Since the second looks like a LOT of development time (I would guess at
least 6 months full time) I would rather say that option (1) is more
likely, unless of curse, he doesn't KNOW about DWARF/gcc/exception
handling what would confirm (1) but make comprehensible why he is
proposing (2).

[Rod Pemberton]

"BGB" <cr8...@hotmail.com> wrote in message
news:je7m5e$7qg$1...@news.albasani.net...
...
<OT for c.l.c>

> >> catch with an integer could also use a FOURCC for well-known /
> >> named exceptions.
> >
> > FOURCC ??
> >
>
> http://en.wikipedia.org/wiki/FOURCC
>

Even if one is unfamiliar with FOURCC, we are familiar with four character
ID3 Tags:
http://en.wikipedia.org/wiki/Id3

> basically, 4 ASCII characters are encoded into a 32-bit integer, and
> used as a magic number.
>

Interesting idea ... at least for a.o.d. ... When I first read that, I had
no idea what you were rambling about. The 32-bit integer could function as
both a unique identifier, or hash-like value, and provide information since
it'd be readable as ASCII too. I like it. 0xDEADBEEF on "steroids" ...


Rod Pemberton
(FYI, follow-ups set to a.o.d.)

[Ed Coran]

Oh, a politician you are?

[Ed Coran]

Rod Pemberton wrote:
> "Ed Coran" <sys...@new.net> wrote in message
> news:je62hn$d2j$1...@speranza.aioe.org...
>>
>> Off topic.
>
> Nothing is off-topic in an alt.* heirarchy Usenet group.

Well isn't that just fine and dandy... for you.

>
> comp.lang.c is not involved in this part of the thread either.

I just posted. I didn't look at the destination(s).

>
> You certainly have the *tone* of our anonymous friend "Fritz Weuhler"
> ... "Fritz" is that you?
>
> Hmm ...

Interesting. You are afraid of being wrong on the fundamental
information? Uh oh. You are why I am not a teacher? Because how can one
teach anyone who knows everything? :p Nuff said, let's get back on topic,
hmm?

[Ed Coran]

Rod Pemberton wrote:
> I started programming in 1981.

Well excuuuuuuuse me.

[Ed Coran]

Rod Pemberton wrote:
> "James Harris" <james.h...@googlemail.com> wrote in message
> news:87e6a259-4d7d-490a...@z1g2000vbx.googlegroups.com...
>> On Jan 6, 12:05 am, "Rod Pemberton" <do_not_h...@noavailemail.cmm>
>> wrote:
> ...
>
>> Silent failure is pernicious.
>
> It depends on the failure.
>
>>> Alternately, you could assume it's serious and abort, but you could
>>> lose data:
>>
>> Far better than giving the answer 42 when it should be 53!
>
> If "fault-tolerance" is the goal or the program cannot be terminated
> until a certain criteria, e.g., 4:15pm, then aborting and exiting is
> not a choice. You have to use STM (software transactional memory) or
> other fault tolerant methods to preserve data. So, what are your
> goals?

"name dropper", for sure. Define the words you use. Game over? I.e.,
backstreets and alleys are not credible approaches to launching space
shuttles. So curb it.

[Ed Coran]

Rod Pemberton wrote:
> I started programming in 1981.

Now we know why the astronauts died?

[Ed Coran]

jacob navia wrote:
> Le 06/01/12 21:05, BGB a écrit :
>> it is debatable that this is actually faster...
>>
>> the problem is if the status is checked in each function (possibly
>> multiple times) then this may infact cost more overall than even
>> registering/unregistering exception handlers.
>>
>> handlers capable of reflective unwinding (the GCC/DWARF strategy,
>> Win64 SEH), only have a cost if an exception occurs, and are free
>> otherwise (apart from debatable "possibly contrivance" in the
>> prologue/epilogue case WRT Win64 SEH).
>>
>
> I was the first to answer Mr Harris post (Dec 29th)

And that is significant why?

>
> He never answered or acknowldged my post even if I pointed him to:

Uh oh, you think newsgroup posts are akin to talking to people for real?

>
> He ignored my advice

You are his advisor? I have done a little bit of consulting and know that
all that matters in the legal substrate is the contract. So, what are you
all up about, Jack? Are you in love with him? Is this beyond "the law"?

> Either:
>
> 1 Mr Harris doesn't *really* know what he is talking about

Or he does not accept you as "his master"? Have you considered that you
are unimportant to him?

>
> 2 He wants to eliminate the exception handling mechanism of his
> preferred compiler and figure out a new one AND hack gcc to
> implement that...

And if he solved that problem, your toil and labor would be all for
naught? Really now. His disregard for emperialism is, surely, at the
highest disregard, I mean we are all the same huh, but he breached that?

You seek a "kindred spirit", surely. Surely I'm just here to fuck you up.
Ya think?

[jacob navia]

Le 07/01/12 05:24, Ed Coran a �crit :

Mr Coran:

[snip personal attacks]

Nothing in your message goes to the problem I have been repeating for a
while here:

Exception handling is a wide field of research that has been studied
for years by many bright people with some implementations like gcc's
open for study with their source code.

The idea of replacing the fruit of all those efforts with something
like a global variable that must be constantly POLLED by user code
is such a completely NUTS idea that I recommended mr Harris to READ THE
LITERATURE about the subject before proposing such a ridiciulous
"solution".

Your personal attacks do not change anything about this state of affairs
and really only display your total lack of arguments.

Yes, what else can you say than:

<quote>

Surely I'm just here to fuck you up

<end quote>

THAT is the level of your arguemnts.

Well, to be able to insult me, Sir, it is first necessary that I give
some importantce to what you say.

Yours sincerely

Jacob Navia

[Rod Pemberton]

"Ed Coran" <sys...@new.net> wrote in message

news:je8eua$f4f$1...@speranza.aioe.org...

> Rod Pemberton wrote:
> > "Ed Coran" <sys...@new.net> wrote in message
> > news:je5t2l$4bb$1...@speranza.aioe.org...
...

> > The topic is whatever James is willing to discuss.
> > He is the only one posting to a.o.d. lately.
> >
>
> Oh, a politician you are?
>

Well, no ...

Seriously, what interests you Ed? So far, all we know about you is that
your humor is somewhat off ... slightly zany. If you want to be polite, you
could start another thread on some topic and OT it.

I've got no problem discussing just about anything, time and life and
interest permitting, with prudence for a few rare topics. I have quite a
broad an extreme range of interests: cars, automotive restoration, drag
racing, EVs, ICEs, Maglev, sterling engines, "overunity" devices,
generators, solar power, geo-thermal power, off-grid power, wind power,
water power, rust removal, guns, home assembled guns, home made guns, weapon
registration, Bill of Rights, cats, dogs, ferrets, fish, God, Lucifer or
Satan, Ten Commandments, KJV Revelations, KJV Job, KJV Genesis, Doomsday,
Christianity, Islam, Judaism, Jews, Nazis, socialism, communism, economics,
"Big Brother", A51, aliens, top secret, military drones, NSA, CIA, KKK,
Black Panthers, Presidents, conspiracy theories, electronic tracking,
thrash, metalcore, New York subway, DJ Trump, casino's, programming, C,
assembly, parsing, lexing, compilers, microprocessors, software licenses,
copyrights, action movies, spy movies, westerns, spy devices, hidden
camera's, laser microphones, eavesdropping, electronics, martial arts, Jeet
Kune Do, Sambo (martial art), Krav Maga, Gracie Jiu-Jitsu, Ninja, Kenjutsu,
Nunchaku, martial art causes of death, central nervous system, arteries,
vital points, combat techniques for dog kills, 45-70 gov't., 458 Socom,
AK-47s, AR-15s, 1911s, 2011s, nature, squirrels, deer, healthy foods, bombs,
weapons, nuclear weapons, laser weapons, satellites, internet protocols,
telephone system, high voltage electronics - can crushers - rail guns - coin
shrinkers - jacob ladders, encryption, filesystems, hash functions,
operating systems, securities, stocks, bonds, market orders, strips,
straddles, puts, calls, derivatives, physics, calculus, geometry, algebra,
business, wealth, profit, manufacturing, hidden knowledge, secret
organizations, organized crime, drug gangs, C-Walk, biker gangs, Ace of
Spades, Kill Light, corrupt cops, cop murders by taser electrocution, auto
theft rings, drug manufacturing, chemistry, women, common sex, common sexual
perversions, abortion, mortgages, home construction, home repair, painting,
taxes, national identity cards, automotive industry, die casting, stamping,
powdered metal, slitters, screw machines, wave soldering machines, 401Ks,
IRAs, cement, cinder blocks, glass block, mining, forestry, oil, countries,
wide variety of TV shows ... That's just to start. I'll talk about just
about everything except politics, English grammar, the law, sports such as
golf, boxing, baseball, basketball, and movie or music genres I'm not
interested in, because they all bore me. And, I won't openly discuss
extreme sex or "secret" knowledge about sex or sexual behavior, e.g., Red
and Purple Wings, bestiality, molestation, pedophilia, S&M, DP, or far far
more extreme crap, or certain methods of combat or martial arts killing ...


Rod Pemberton

[James Harris]

On Jan 6, 10:54 pm, jacob navia <ja...@spamsink.net> wrote:
> Le 06/01/12 21:05, BGB a écrit :
>
> > it is debatable that this is actually faster...
>
> > the problem is if the status is checked in each function (possibly
> > multiple times) then this may infact cost more overall than even
> > registering/unregistering exception handlers.
>
> > handlers capable of reflective unwinding (the GCC/DWARF strategy, Win64
> > SEH), only have a cost if an exception occurs, and are free otherwise
> > (apart from debatable "possibly contrivance" in the prologue/epilogue
> > case WRT Win64 SEH).
>
> I was the first to answer Mr Harris post (Dec 29th)
>
> He never answered or acknowldged my post even if I pointed him to:

Just because I didn't reply to you directly don't think I didn't read
your post. I did, more than once, so don't be offended. It is not
feasible to reply to everyone!

James

[James Harris]

On Jan 6, 8:05 pm, BGB <cr88...@hotmail.com> wrote:

...

> >>> Since the word would be checked many times between task switches this
> >>> is much faster than following a structure off FS or similar.
>
> >> possibly, however the issue then partly becomes:
> >> how does the task switcher know where the variable is?...
>
> > Isn't this easy if you have a flat address space: just put the
> > variable at a fixed location?
>
> if the task switcher is part of the OS kernel (it generally is), then it
> does not have direct visibility of variables declared in a user-space
> binary.

The intention is the other way round. Rather than the OS accessing
task space the tasks can access a fixed location in the kernel space.
For example, say the indicator variable is called task_excep and is
actually at location 4100. With a flat address space the linker should
be able to resolve the label to that address shouldn't it? (Though see
my response below to your point about PIC.)

...

Thanks for pointing this out. Normal elf32 from GCC and was fine. It
just issued a simple access. When I added the -fpic option you
mentioned below it tried to access it via the GOT. That would be a bit
of a pain, given my desire for speed. That's a pretty cumbersome
mechanism for a variable at a fixed location in the address space but
there's maybe no way to tell the compiler enough that it understands.
For the code, I'm not sure that PIC would be necessary. Do you know
what else it does on GCC, especially as x86 code is generally position-
independent anyway? Branches and jumps are usually relative without
specifying PIC. Is it just for data accesses, for example?

> on x86-64, the CPU has RIP-relative addressing, and so will presumably
> use this to access the variable (for both SO's and non-SO binaries).

The thing is, the program would not necessarily be loaded at the same
address in each address space so the RIP-relative offset to a fixed
location doesn't work. OT but RIP-relative data addressing is bearable
for reading constants but otherwise is a pretty awful addition, IMHO.

None of these things should be a problem, though. The default settings
in C seemed fine.

...

> I am thinking for the case where an OS scheduler needs access to a
> userspace global variable.
>
> the kernel can't easily know about this case, unless something "special"
> is done.

As mentioned, it's the other way round. Each task needs access to a
machine word that belongs to the kernel. I can't see anything special
being needed. Am I missing something? The idea of a symbolic name
rather than an address is that it can be resolved by the link step.

...

> > Why is this no good? There can be any number of user exception types
> > so it's not possible to have one bit for each. The best I could think
> > of was to have one bit to indicate a user exception and functions to
> > examine as much other detail as needed.
>
> exceptions don't generally work this way.
> yes, "one bit per type for every type" is absurd, but using one bit
> per-type for certain exceptions but not others, isn't much better.

OK

...

> >> an exception is not a status code...
>
> > Following your analogy of a CPU you could perhaps think of each
> > exception condition as an interrupt line that gets asserted until the
> > condition causing the interrupt has been resolved. I know, as an
> > analogy it's not perfect. :-(
>
> flags and similar are used here.
>
> so, normal interrupt occurs:
> transfers to handler (and temporarily disables other interrupts);
> handler does its thing;
> iret (return from interrupt, restores prior CPU state, although GPRs/...
> generally need to be manually saved/restored).
>
> if things go bad in the first handler, then a double-fault occurs (a
> dedicated special interrupt).
>
> if the DF handler itself fails, the CPU reboots.

It seems you are thinking principally of the CPU exception interrupts.
I was thinking more of the hardware interrupt lines but it's not too
important. It wasn't a very good analogy anyway.

...

> > I tried compiling the code
> > above with gcc and it seems to refer to G directly such as in sub()
> > the compiler generates
>
> >    mov     DWORD PTR G, %eax
>
> but, with which OS and what compiler settings.
>
> if this was Linux-i386 and "-shared" or "-fpic" was given, one would not
> likely see a global being loaded this way.

As above, you are right about -fpic. -shared seemed to make no
difference.

> if it was on Win32, then the above is what one would expect to see.

Do you mean that a compiler using Win32's version of position
independent code would access G directly?

...

> > I found that in some systems there is a call which returns the
> > address of errno. In that case the compiler should (hopefully)
> > remember that address for each use in a function.
>
> this depends...
>
> in most cases, it probably wont, since the compiler can't be sure that
> the function wont return a different address each time (though I wont
> rule out the possibility of some sort of a "this function will always
> return the same value" modifier/attribute).

OK. How about at the top of a function,

int *erloc = &errno;

Then after each call,

if (*erloc)

I've not worked this through. Just a thought. As Kirk says, I like to
think there are always options. :-)

...

> > Don't forget the original intention was speed. The point of a status
> > word, as you call it, is to allow exception handling (or, non-
> > handling) quickly. It is not an end in itself but a means of adding
> > exceptions to C in a way that recognises exceptions in the right
> > places while having the lowest possible impact on overall performance.
>
> it is debatable that this is actually faster...
>
> the problem is if the status is checked in each function (possibly
> multiple times) then this may infact cost more overall than even
> registering/unregistering exception handlers.

I don't see your mathematics on this. I can't think of a perfect
analogy but it is similar to taking a C program ending with

return;

and changing it to

return 1;

How many people would complain that that slowed down their code?

Your PIC example would not be so fast, if written in C, where it goes
via the GOT. Position-independent assembly language would not be a
problem. Non-PIC code in either language doesn't see to be a problem
either.

The only thing I could think of which would be ostensibly faster is
just to test the carry flag on return

call subroutine
jc exception

However, that is no good for these reasons

1. Time would still be required to set or clear carry in the
subroutine.
2. C does not support access to the carry flag.

So, overall, it would not be faster and may be slower in some cases,
and could not be written in C.

> handlers capable of reflective unwinding (the GCC/DWARF strategy, Win64
> SEH), only have a cost if an exception occurs, and are free otherwise
> (apart from debatable "possibly contrivance" in the prologue/epilogue
> case WRT Win64 SEH).
>
> in my own (mostly unsused) ABI, I had done something like:
> push ebp
> mov ebp, esp
> push ... ;push-register sequence, like in Win64
> sub esp, ... ;space for locals/...
> ...
> lea esp, [ebp-offs]
> pop ...
> pop ebp
> ret
> nop [magic_info] ;function metadata (optional)

OK. This seems similar to a standard prologue and epilogue.

> this preserved a property that is desirable from typical with cdecl, but
> is lost with both Win64 and SysV/AMD64: the ability to backtrack simply
> using an EBP/RBP chain (with Win64, one has to unwind using awkward
> logic, and on SysV/AMD64 one may be SOL apart from using DWARF as the
> ABI does not mandate any sort of standardized frame pointer).

OK. FWIW if you used a test-on-return as per my proposal you could
build the stack trace on return, even without use of ebp-style stack
frames. That's what I plan to do. Of course, it will only build a
stack trace report between where the exception occurs and where it is
handled.

James

[jacob navia]

Le 08/01/12 23:20, James Harris a écrit :

>
> As mentioned, it's the other way round. Each task needs access to a
> machine word that belongs to the kernel. I can't see anything special
> being needed. Am I missing something?

Yes, you are missing everything:

1) Many people including me have told you that each thread needs an
exception handler, in a multi-threaded environment. It can't be
just a machine word somewhere. It must be thread local storage

2) Accessing that machine word by polling, as you propose can't be done
in one instruction since you need the thread ID to be able to access
the thread local storage.

3) You need exclusive access to that machine word. You do not want other
process to write to it when you are reading. Remember that today's
CPUs have several cores. Having a thread gain exclusive access would
BLOCK all other threads since they are constantly polling that
famous machine word... If a thread gains exclusive access to that
address and then dies without releasing the lock the PROCESS STOPS.
All threads are blocked.

I could go on but what would be the point?

Can you explain why the existing schemas that have been developed
through many years of usage and debugging do not please you?

Why do you feel compelled to reinvent the wheel by proposing a
square wheel?

HINT: Wheels should be round :-)

HINT: Exception handlers are invoked by the OS or a throw in a given
thread and walk the stack seeking a handler using the tables
generated by the compiler

[Scott Wood]

On 2012-01-08, James Harris <james.h...@googlemail.com> wrote:
> The thing is, the program would not necessarily be loaded at the same
> address in each address space so the RIP-relative offset to a fixed
> location doesn't work. OT but RIP-relative data addressing is bearable
> for reading constants but otherwise is a pretty awful addition, IMHO.

From glancing at its output, it seems GCC is able to make pretty extensive
use of RIP-relative addressing, and not just for constants.

-Scott

[jacob navia]

Le 09/01/12 00:29, Scott Wood a écrit :

Using Macintosh OS X you have ONLY RIP addressing! I had to rewrite
most of lcc's backend to make it run there.

Apple has always been fan of relative addressing.

[James Harris]

On Jan 8, 11:19 pm, jacob navia <ja...@spamsink.net> wrote:
> Le 08/01/12 23:20, James Harris a écrit :
>
>
>
> > As mentioned, it's the other way round. Each task needs access to a
> > machine word that belongs to the kernel. I can't see anything special
> > being needed. Am I missing something?

The above relates to the initial option 3 so, sticking to that....

> Yes, you are missing everything:

Are you sure it's not that you totally misunderstand the idea?

> 1) Many people including me have told you that each thread needs an
>     exception handler, in a multi-threaded environment. It can't be
>     just a machine word somewhere. It must be thread local storage

This is TLS. I stated that in the initial post.

> 2) Accessing that machine word by polling, as you propose can't be done
>     in one instruction since you need the thread ID to be able to access
>     the thread local storage.

No, for option 3 the word is changed on a thread switch. I have
explained this too.

Checking back, if you view this discussion as a hierarchy see what is
currently the seventh message, a reply I made to BGB.

> 3) You need exclusive access to that machine word. You do not want other
>     process to write to it when you are reading. Remember that today's
>     CPUs have several cores. Having a thread gain exclusive access would
>     BLOCK all other threads since they are constantly polling that
>     famous machine word... If a thread gains exclusive access to that
>     address and then dies without releasing the lock the PROCESS STOPS.
>     All threads are blocked.

The word is specific to a thread, not to a process, not to a CPU, not
to a machine.

> I could go on but what would be the point?

Agreed. Please don't. :-)

James

[Scott Wood]

On 2012-01-09, James Harris <james.h...@googlemail.com> wrote:
> On Jan 8, 11:19 pm, jacob navia <ja...@spamsink.net> wrote:
>> 1) Many people including me have told you that each thread needs an
>>     exception handler, in a multi-threaded environment. It can't be
>>     just a machine word somewhere. It must be thread local storage
>
> This is TLS. I stated that in the initial post.
>
>> 2) Accessing that machine word by polling, as you propose can't be done
>>     in one instruction since you need the thread ID to be able to access
>>     the thread local storage.
>
> No, for option 3 the word is changed on a thread switch. I have
> explained this too.

And it's been explained to you that "changed on a thread switch" doesn't
work unless this is CPU-local storage by some other mechanism. More than
one thread can be running at the same time -- no switching, no opportunity
to swap the magic variable.

-Scott

[Ed Coran]

"jacob navia" <ja...@spamsink.net> wrote in message
news:je8uuc$9nf$1...@speranza.aioe.org...
> Le 07/01/12 05:24, Ed Coran a écrit :

>
> Mr Coran:
>
> [snip personal attacks]
>
> Nothing in your message goes to the problem I have been repeating for a
> while here:
>
> Exception handling is a wide field of research that has been studied
> for years by many bright people with some implementations like gcc's
> open for study with their source code.

Surely by some of those lovely people that gave us C and C++! (See what's
wrong with your stance in the above statement now?)

>
> The idea of replacing the fruit of all those efforts with something
> like a global variable that must be constantly POLLED by user code
> is such a completely NUTS idea that I recommended mr Harris to READ THE
> LITERATURE about the subject before proposing such a ridiciulous
> "solution".

I must have missed the part where he suggested polling as some kind of
ultimate general solution. Indeed, wasn't his OP surveying the landscape
of ideas on the subject, and in particular, as a retrofit for C while
still being 100% legal C? You wouldn't be contriving something in attempt
to seem somehow superior to him, now would you?

(OK, noted: no more remote humour when replying to your thread. It's hard
to assess whether posters just know literal English or the "richer"
language that it can be).

[Ed Coran]

"Rod Pemberton" <do_no...@noavailemail.cmm> wrote in message
news:je99cp$v9m$1...@speranza.aioe.org...

> "Ed Coran" <sys...@new.net> wrote in message
> news:je8eua$f4f$1...@speranza.aioe.org...
>> Rod Pemberton wrote:
>> > "Ed Coran" <sys...@new.net> wrote in message
>> > news:je5t2l$4bb$1...@speranza.aioe.org...
> ...
>
>> > The topic is whatever James is willing to discuss.
>> > He is the only one posting to a.o.d. lately.
>> >
>>
>> Oh, a politician you are?
>>
>
> Well, no ...
>
> Seriously, what interests you Ed? So far, all we know about you is
> that
> your humor is somewhat off ... slightly zany.

Off-topic discourse: anything personal, politics, religion, sex. But
those are just in general. Seeing as how this is a.o.d, some relevance to
that, or the larger domain of software and system development, including
languages and such, is all I care to entertain in here.

[Ed Coran]

James Harris wrote:
> On Jan 8, 11:19 pm, jacob navia <ja...@spamsink.net> wrote:

>> 1) Many people including me have told you that each thread needs an
>> exception handler, in a multi-threaded environment. It can't be
>> just a machine word somewhere. It must be thread local storage
>
> This is TLS. I stated that in the initial post.

He seems to be limited in his thinking to the existing implementations he
has seen or read about. I.e., there is no indication of out-of-the-box
thinking. His dialog seems to hint toward a C++-like implementation of
exceptions as the be-all and end-all, bottom line, last answer on the
topic of exception handling, which is like much of the rationale often
spewed for C++ features: absurd. His last sentence above is obviously
wrong also.

[Ian Collins]

What is the "C++-like implementation of exceptions"? There are many.

--
Ian Collins

[Nomen Nescio]

Scott Wood <sc...@buserror.net> wrote:

> From glancing at its output, it seems GCC is able to make pretty extensive
> use of RIP-relative addressing, and not just for constants.

^^^

Thanks to Scott now we get the tombstone question!

[Ed Coran]

That which behaves, from the language user perspective, not the language
implementor perspective, like C++ exceptions do. Close enough. It would
be incorrect (for lack of a better term which escapes me right now) to
scrutinize what I wrote from a mostly or only detailed implementation
level instead of some higher level perspective of the behavioral
characteristics.

[James Harris]

On Jan 9, 5:41 am, Scott Wood <sc...@buserror.net> wrote:

...

> >> 2) Accessing that machine word by polling, as you propose can't be done
> >>     in one instruction since you need the thread ID to be able to access
> >>     the thread local storage.
>
> > No, for option 3 the word is changed on a thread switch. I have
> > explained this too.
>
> And it's been explained to you that "changed on a thread switch" doesn't
> work unless this is CPU-local storage by some other mechanism.

If you mean your comment four days ago, I did reply. I said that I
didn't understand what you meant but you said no more. Not what I
would call 'explained' to me! :-(

> More than
> one thread can be running at the same time -- no switching, no opportunity
> to swap the magic variable.

I've been puzzling over this. The only thing I can think that might
cause a problem is multiple threads running in the same address space
on different CPUs. Is that what you mean?

James

[James Harris]

:-) Nice one - sorry to be burning up the airwaves! Then again, those
who are not interested in a certain topic can just resist the urge to
read posts on it, I hope!

James

[Scott Wood]

On 2012-01-09, James Harris <james.h...@googlemail.com> wrote:

> On Jan 9, 5:41 am, Scott Wood <sc...@buserror.net> wrote:
>
> ...
>
>> >> 2) Accessing that machine word by polling, as you propose can't be done
>> >>     in one instruction since you need the thread ID to be able to access
>> >>     the thread local storage.
>>
>> > No, for option 3 the word is changed on a thread switch. I have
>> > explained this too.
>>
>> And it's been explained to you that "changed on a thread switch" doesn't
>> work unless this is CPU-local storage by some other mechanism.
>
> If you mean your comment four days ago, I did reply. I said that I
> didn't understand what you meant but you said no more. Not what I
> would call 'explained' to me! :-(

Hmm, sorry about that -- I thought I replied, but I can't find it now. I
may have had the reply open when my computer crashed. Something in the
Linux kernel branched right into the middle of an instruction (probably
corrupted memory, maybe rotting hardware). :-(

>> More than one thread can be running at the same time -- no switching, no
>> opportunity to swap the magic variable.
>
> I've been puzzling over this. The only thing I can think that might
> cause a problem is multiple threads running in the same address space
> on different CPUs. Is that what you mean?

Yes, that's what I mean. Swapping on thread switch is only viable for
per-CPU storage, and if the whole point is to have a fixed virtual address,
that means you need a separate MMU context -- which is doable, but at the
point that I'd question whether it's really worthwhile.

A summary of what I recall of the rest of the reply:

> 2. The scheme proposed will work between languages. I don't intend the
> use of many languages but as an example, a Pascal routine could call
> an assembly routine which could call a C routine which could call
> another assembly routine and all of them could follow the same, very
> fast, exception propagation protocol. (More accurately, the point is
> that each would very quickly skip non-exceptions while still handling
> exceptions.)

It won't skip them as quickly as native compiler exception handling -- but
yes, you'd need some glue code around language transitions unless all the
languages involved implement the same exception mechanism (asm could be made
to, others maybe with compiler extensions?). How often are potentially
cross-language calls made, compared to component-internal calls?

If you need something that works in a wide variety of languages without glue
or other hassle, it sounds like simple return values are the way to go. You
can combine errors and successful return values as long as they have
disjoint ranges (often negative for errors, non-negative for success), so
for many functions you may not need an extra pointer parameter. Perhaps you
could stick a full error report struct in true TLS (you wouldn't need to
access it in the no-error case, so it's not as speed-sensitive), but be
careful that such structs can't leak.

> if (ex = a()) {if (handler(&ex)) return ex;)
> if (ex = b()) {if (handler(&ex)) return ex;}

What would go in handler()? I'd think you'd usually want to make the
decision locally about whether/how to handle or propagate and clean up from
the error.

> 3. It doesn't allow a sticky exception state such as
>
> a();
> b();
> if (thread_excep) /* An exception occurred in a() or b() */

I've only occasionally felt the need to do this -- often I'd want to clean
up differently, or I wouldn't want to call b() if a() failed.

> If I wanted to add a sticky state I suppose I could write
>
> if (ex |= b()) ...
>
> but that puts the OR instruction (and possibly a memory update) into
> the main non-exception processing path so would be slower in the
> normal processing case.

Doing a load from memory followed by a compare and conditional branch will
also slow down the normal processing case. A few ORs is well worth
eliminating a bunch of loads.

> I'm not ruling it out completely. It does do away with the goto.

Don't fear the goto -- it's one of the nicer ways of doing error cleanup in
C that I've seen. The Linux kernel uses it extensively.

-Scott

[Joe keane]

In article <slrnjgnnf0...@snotra.buserror.net>,

Scott Wood <sc...@buserror.net> wrote:
>If you need something that works in a wide variety of languages without glue
>or other hassle, it sounds like simple return values are the way to go.

I guess the goal is to get rid of the compare-and-branch for each
operation? Then you have C++-style function calls [assume succesful,
then fix things up with funky extra code if there is an error]. I think
if this is too costly you have other problems.

Also the machine already does something a lot like this. Assume the
branch is not taken, smash it, later fix the machine state back if you
find there was a mistake.

Maybe the real problem is that you have too many function calls?
Refactor the code, or use macros [or use inline], find the can't-fail
operations...

#define FOO_GET_X(FOO, X) ((X) = (FOO) >> 4 & 0x1)
#define FOO_SET_X(FOO, X) ((FOO) |= ((X) & 0x1) << 4)

[James Harris]

On Jan 10, 6:42 am, Scott Wood <sc...@buserror.net> wrote:

...

> >> And it's been explained to you that "changed on a thread switch" doesn't
> >> work unless this is CPU-local storage by some other mechanism.
>
> > If you mean your comment four days ago, I did reply. I said that I
> > didn't understand what you meant but you said no more. Not what I
> > would call 'explained' to me! :-(
>
> Hmm, sorry about that -- I thought I replied, but I can't find it now.  I
> may have had the reply open when my computer crashed.

...

No worries. Thanks for replying.

...

> > I've been puzzling over this. The only thing I can think that might
> > cause a problem is multiple threads running in the same address space
> > on different CPUs. Is that what you mean?
>
> Yes, that's what I mean.  Swapping on thread switch is only viable for
> per-CPU storage, and if the whole point is to have a fixed virtual address,
> that means you need a separate MMU context -- which is doable, but at the
> point that I'd question whether it's really worthwhile.

To answer I'll have to ditch the term "thread" which I used as it was
something people would be familiar with and instead use the terms I
really use, "task" and "address space." This starts to get into the
detail of the design and could go way off topic so I'll limit the info
to what hopefully will be enough to make sense of the exception-
indicator word which is the point being discussed.

In these terms, an "address space" is hopefully fairly self
explanatory. A "task" is basically a unit of scheduling. Unlike
threads in a process, tasks, by default, do not share resources.

When an address space is supporting just one task there is no
possibility of interference as there's nothing to interfere with.
Another task in another address space would have its own exception
word.

Where tasks execute in different address spaces they cannot interfere
with each other's exception word as it is in different memory. Address
X in one is not the same as address X in another.

Where a single address space supports multiple tasks things get a bit
more interesting.

If the multiple tasks are kept on one CPU they cannot interfere
because the task switcher changes the exception word on a task switch.
This is my preferred model. The address space is kept on one CPU
partly to avoid the cost of transferring updated data between CPU
caches across the system bus.

(In that scenario, although all the tasks in one address space are
restricted to running on one CPU at a time, the other CPUs would not
need to be idle as they could still be busy executing active tasks in
other address spaces.)

What about the final scenario where it really is a good idea to have
multiple tasks in the same address space running concurrently on
different CPUs? I can think of two potential solutions.

1. Use a different address for the exception word on each CPU. This
does involve pinning tasks to CPUs, more specifically the code of
tasks would need to include the specific address assigned for that
CPU. Fortunately my primary execution model uses a very late link step
- data values are resolved at load time - so the task_excep word and
its ilk could be resolved after the CPU has been chosen by the OS.

2. Alternatively, let the tasks share most address space but keep the
task-local data er, local to a CPU. If it would work this sounds quite
attractive.

As mentioned, my preferred option is to keep an address space on one
CPU at a time but the last situation may be resolvable where
necessary. Someone will probably tell me in no uncertain terms if it
is not.

> A summary of what I recall of the rest of the reply:
>
> > 2. The scheme proposed will work between languages. I don't intend the
> > use of many languages but as an example, a Pascal routine could call
> > an assembly routine which could call a C routine which could call
> > another assembly routine and all of them could follow the same, very
> > fast, exception propagation protocol. (More accurately, the point is
> > that each would very quickly skip non-exceptions while still handling
> > exceptions.)
>
> It won't skip them as quickly as native compiler exception handling

That's what I thought at the start but considering the time that might
be needed to establish stack frames that the native exception handler
can unwind I'm having my doubts. A lot possibly depends on how many
try-type blocks there are as to which option would come out on top.

It is a moot point anyway as the original intention was to devise
something that would work in standard C. Changing a compiler or using
any compiler-specific facilities is out of scope.

> -- but
> yes, you'd need some glue code around language transitions unless all the
> languages involved implement the same exception mechanism (asm could be made
> to, others maybe with compiler extensions?).  How often are potentially
> cross-language calls made, compared to component-internal calls?

Not very often, it's true. As a general construct, cross-language
calls are irrelevant. They may be very important for some of what I am
doing, though, where some of the code calls assembly routines. And
they simplify matters by making the interfaces consistent. (If only I
could do similarly for the calling conventions, eh!)

> If you need something that works in a wide variety of languages without glue
> or other hassle, it sounds like simple return values are the way to go.  You
> can combine errors and successful return values as long as they have
> disjoint ranges (often negative for errors, non-negative for success), so
> for many functions you may not need an extra pointer parameter.  Perhaps you
> could stick a full error report struct in true TLS (you wouldn't need to
> access it in the no-error case, so it's not as speed-sensitive), but be
> careful that such structs can't leak.

Yes, an object that explains what went wrong would have to be in true
TLS.

> >   if (ex = a()) {if (handler(&ex)) return ex;)
> >   if (ex = b()) {if (handler(&ex)) return ex;}
>
> What would go in handler()?  I'd think you'd usually want to make the
> decision locally about whether/how to handle or propagate and clean up from
> the error.

The handlers wouldn't need to be the same. They could be the same or
different depending on need.

> > 3. It doesn't allow a sticky exception state such as
>
> > a();
> > b();
> > if (thread_excep) /* An exception occurred in a() or b() */
>
> I've only occasionally felt the need to do this -- often I'd want to clean
> up differently, or I wouldn't want to call b() if a() failed.

True in many cases. I have a piece of code I wrote just the other day
(using the exception word idea) which needed multiple pieces of
memory. It naturally fell out as

a = mem_alloc...
b = mem_alloc...
if (task_excep) goto excep_handler_mem;

Such a form may not be all that uncommon. Obtain a bunch of things
then check for an exception before using them.

(I have issues with what to represent as an exception if more than one
mem_alloc fails, though!)

> > If I wanted to add a sticky state I suppose I could write
>
> >   if (ex |= b()) ...
>
> > but that puts the OR instruction (and possibly a memory update) into
> > the main non-exception processing path so would be slower in the
> > normal processing case.
>
> Doing a load from memory followed by a compare and conditional branch will
> also slow down the normal processing case.  A few ORs is well worth
> eliminating a bunch of loads.

I don't agree with this at all! The exception word will normally load
from cache, L1 cache at that. The branch predictor, because a taken
exception-branch is rare, will predict not taken and the CPU will
carry on with the next instruction (which it already has read-in and
decoded). The sequence

cmp [task_excep], 0
jne handler

should thus virtually disappear in the normal processing case.

> > I'm not ruling it out completely. It does do away with the goto.
>
> Don't fear the goto -- it's one of the nicer ways of doing error cleanup in
> C that I've seen.  The Linux kernel uses it extensively.

Someone mentioned that - maybe you. I'm not sure I'd take the Linux
kernel as a good model (I don't like a lot of conditional compilation,
for example) but in general the reaction to my suggestion of using
goto has been far more positive than I expected.

Another thought just occurs to me. Perhaps the above ideas could be
combined so that instead of

<operation potentially causing an exception>
if (task_excep) goto handler_x;

we could have in some cases

<operation potentially causing an exception>
if (task_excep && handler_x()) return;

though that fails to skip following statements. Another option could
be some variation of

<operation potentially causing an exception>
if (task_excep) if (handler_x()) return;

with appropriate else clauses as required. Not sure. Although I
dislike using goto it does express the thought succinctly and if used
consistently would become idiomatic and thus easy to read.

James

[Scott Wood]

[snipped comp.lang.c, as this is getting rather off-topic for there]

On 2012-01-11, James Harris <james.h...@googlemail.com> wrote:
> On Jan 10, 6:42 am, Scott Wood <sc...@buserror.net> wrote:
>
> ...
>

> Where a single address space supports multiple tasks things get a bit
> more interesting.
>
> If the multiple tasks are kept on one CPU they cannot interfere
> because the task switcher changes the exception word on a task switch.
> This is my preferred model. The address space is kept on one CPU
> partly to avoid the cost of transferring updated data between CPU
> caches across the system bus.

That's fine if you have a busy system with more address spaces than CPUs
runnable at any given time. It's not making the best use of resources if
you have only one address space actively running, with multiple tasks, on a
system with multiple CPUs.

Also consider hyperthreading, where you're sharing a cache anyway.

The ability to take advantage of multiple CPUs in this manner is one of the
main reasons people use threads^H^H^H^H^H^Hmultiple tasks in an address
space. It's not the only reason, but it's a significant benefit. It's up
to the application to not toss cache lines around so much that it's worse
than running on one CPU (or to explicitly ask the OS to keep the tasks on a
single CPU if it's beneficial to that app).

> (In that scenario, although all the tasks in one address space are
> restricted to running on one CPU at a time, the other CPUs would not
> need to be idle as they could still be busy executing active tasks in
> other address spaces.)

But what if they're not busy?

This is something that all modern mainstream OSes support, so I wouldn't
dismiss it as unnecessary unless there are reasons specific to what you're
trying to accomplish.

> What about the final scenario where it really is a good idea to have
> multiple tasks in the same address space running concurrently on
> different CPUs? I can think of two potential solutions.
>
> 1. Use a different address for the exception word on each CPU. This
> does involve pinning tasks to CPUs, more specifically the code of
> tasks would need to include the specific address assigned for that
> CPU.

Oh dear. That's... not the direction I expected you to go. :-)

> Fortunately my primary execution model uses a very late link step data
> - values are resolved at load time - so the task_excep word and

> its ilk could be resolved after the CPU has been chosen by the OS.

Dynamically migrating threads between CPUs to balance load is another thing
that is expected of a modern OS. You don't want to do it too often because
it's costly, and you may want a way to *optionally* pin a thread to a CPU,
but not supporting it at all would be bad.

> 2. Alternatively, let the tasks share most address space but keep the
> task-local data er, local to a CPU. If it would work this sounds quite
> attractive.

You can do that. It's what I meant by keeping separate page tables (you
could share everything but the toplevel page directory). If you do it on a
per-CPU basis, it'll complicate migrating tasks between CPUs. If you do it
on a per-thread basis, it'll consume more memory and be less efficient when
switching between one address space's tasks on the same CPU. I'd strongly
suggest just going with a more conventional error handling approach. But
what's a hobby OS without a little crazy here and there? :-)

Just be open to revisiting design decisions down the road if you no longer
feel you're getting enough benefit relative to the cost.

> As mentioned, my preferred option is to keep an address space on one
> CPU at a time but the last situation may be resolvable where
> necessary. Someone will probably tell me in no uncertain terms if it
> is not.
>
>> A summary of what I recall of the rest of the reply:
>>
>> > 2. The scheme proposed will work between languages. I don't intend the
>> > use of many languages but as an example, a Pascal routine could call
>> > an assembly routine which could call a C routine which could call
>> > another assembly routine and all of them could follow the same, very
>> > fast, exception propagation protocol. (More accurately, the point is
>> > that each would very quickly skip non-exceptions while still handling
>> > exceptions.)
>>
>> It won't skip them as quickly as native compiler exception handling
>
> That's what I thought at the start but considering the time that might
> be needed to establish stack frames that the native exception handler
> can unwind I'm having my doubts. A lot possibly depends on how many
> try-type blocks there are as to which option would come out on top.

Have you benchmarked it, or looked at the output of the compiler for
exceptions versus the same error handling and cleanup using manual error
checking? Or are you just guessing at what the compiler's doing?

Try looking at the compiler output from the following C++ program:

================================
extern int func_returns(int x);
extern int func_throws(int x);

extern int get_resource(int x);
extern int release_resource(int x);

extern int do_something_good(int x);

class stack_resource {
int x_;

public:
stack_resource(int x)
{
x_ = x;
get_resource(x);
}

~stack_resource()
{
release_resource(x_);
}
};

int caller_c_like(int x)
{
int ret;

get_resource(x);

ret = func_returns(x);
if (ret)
goto out;

ret = do_something_good(x);

out:
release_resource(x);
return ret;
}

int caller_destructor(int x)
{
stack_resource res(x);
int ret;

ret = func_returns(x);
if (ret)
return ret;

return do_something_good(x);
}

int caller_exception(int x)
{
stack_resource res(x);

try {
func_throws(x);
} catch (int err) {
return err;
}

return do_something_good(x);
}

================================

I built with g++ 4.6.1 -O3, and caller_destructor() produced the exact same
code as caller_c_like() except for the addition of a few instructions
*after* the return, reachable only if an exception is thrown.

caller_exception()'s non-error-case code was shorter than the other two, as
it could skip the error check. Sure, there was extra code after the return,
but that only runs if an exception is thrown. Could still cause loads from
disk to slow down a bit, but in exchange for that you get a much more
powerful error handling system and faster normal-case execution once loaded.

> It is a moot point anyway as the original intention was to devise
> something that would work in standard C. Changing a compiler or using
> any compiler-specific facilities is out of scope.

But changes to the OS kernel are fine? :-)

TLS isn't standard C either (unless you're talking about C11?).

How do you invoke a system call without compiler-specific facilities?

>> -- but
>> yes, you'd need some glue code around language transitions unless all the
>> languages involved implement the same exception mechanism (asm could be made
>> to, others maybe with compiler extensions?).  How often are potentially
>> cross-language calls made, compared to component-internal calls?
>
> Not very often, it's true. As a general construct, cross-language
> calls are irrelevant. They may be very important for some of what I am
> doing, though, where some of the code calls assembly routines. And
> they simplify matters by making the interfaces consistent. (If only I
> could do similarly for the calling conventions, eh!)

You can use C linkage with no exceptions on functions that are implemented
in or called from assembly (or other languages), and exceptions on other
things. Excessive consistency for consistency's sake can hurt.

>> >   if (ex = a()) {if (handler(&ex)) return ex;)
>> >   if (ex = b()) {if (handler(&ex)) return ex;}
>>
>> What would go in handler()?  I'd think you'd usually want to make the
>> decision locally about whether/how to handle or propagate and clean up from
>> the error.
>
> The handlers wouldn't need to be the same. They could be the same or
> different depending on need.

I meant you'd most likely want that logic to be right there in the calling
function. Having to put it in a separate function seems awkward.

> True in many cases. I have a piece of code I wrote just the other day
> (using the exception word idea) which needed multiple pieces of
> memory. It naturally fell out as
>
> a = mem_alloc...
> b = mem_alloc...
> if (task_excep) goto excep_handler_mem;
>
> Such a form may not be all that uncommon. Obtain a bunch of things
> then check for an exception before using them.

Yes, and you can do this just as well:

a = mem_alloc...
b = mem_alloc...

if (!a || !b)
goto err_mem;

Yould still have two compare-and-branches (but maybe no load from memory,
and is it really worth nitpicking a cycle here or there compared to dynamic
memory allocation? And later you say these are basically free), but it's
easier to read and maintain than separate source-level checks.

> (I have issues with what to represent as an exception if more than one
> mem_alloc fails, though!)
>
>> > If I wanted to add a sticky state I suppose I could write
>>
>> >   if (ex |= b()) ...
>>
>> > but that puts the OR instruction (and possibly a memory update) into
>> > the main non-exception processing path so would be slower in the
>> > normal processing case.
>>
>> Doing a load from memory followed by a compare and conditional branch will
>> also slow down the normal processing case.  A few ORs is well worth
>> eliminating a bunch of loads.
>
> I don't agree with this at all! The exception word will normally load from
> cache, L1 cache at that. The branch predictor, because a taken
> exception-branch is rare, will predict not taken and the CPU will carry on
> with the next instruction (which it already has read-in and decoded). The
> sequence
>
> cmp [task_excep], 0
> jne handler
>
> should thus virtually disappear in the normal processing case.

Possibly, depends on the details of the CPU you're on (still takes up
icache, though), and what other things are happening. But so will an OR
instruction (especially operating on registers). It will probably run in
parallel with something else.

>> > I'm not ruling it out completely. It does do away with the goto.
>>
>> Don't fear the goto -- it's one of the nicer ways of doing error cleanup in
>> C that I've seen.  The Linux kernel uses it extensively.
>
> Someone mentioned that - maybe you. I'm not sure I'd take the Linux
> kernel as a good model

Linux has its strengths and weaknesses, and is better than it used to be,
but it's not a matter of "trust Linux, they did it right" so much as
"there's a bunch of code over there that does this, go see what you think of
how it works in practice".

-Scott

[James Harris]

On Jan 13, 4:56 am, Scott Wood <sc...@buserror.net> wrote:
> [snipped comp.lang.c, as this is getting rather off-topic for there]

Agreed.

BTW, before I argue against some of the things you have said I should
say that I can tell from what you have written that you have a very
good grasp of the issues. That makes it, for me, a useful and positive
discussion ... even where we disagree ... in fact, especially where we
disagree.

...

> > Where a single address space supports multiple tasks things get a bit
> > more interesting.
>
> > If the multiple tasks are kept on one CPU they cannot interfere
> > because the task switcher changes the exception word on a task switch.
> > This is my preferred model. The address space is kept on one CPU
> > partly to avoid the cost of transferring updated data between CPU
> > caches across the system bus.
>
> That's fine if you have a busy system with more address spaces than CPUs
> runnable at any given time.  It's not making the best use of resources if
> you have only one address space actively running, with multiple tasks, on a
> system with multiple CPUs.
>
> Also consider hyperthreading, where you're sharing a cache anyway.
>
> The ability to take advantage of multiple CPUs in this manner is one of the
> main reasons people use threads^H^H^H^H^H^Hmultiple tasks in an address
> space.  It's not the only reason, but it's a significant benefit.  It's up
> to the application to not toss cache lines around so much that it's worse
> than running on one CPU (or to explicitly ask the OS to keep the tasks on a
> single CPU if it's beneficial to that app).

I'm aware of these objections though I hadn't thought about
hyperthreading specifically. I agree but would make three points:

1. You rightly begin with an "if" clause and your points discuss a
specific emphasis where a single specific app is (or a very small
number of apps are) to be multithreaded in order to work faster. There
are other target uses for an OS which have exactly the opposite
requirements. For example, there are massive cost benefits to be
gained by virtualisation wherein there are many tasks to be run on a
machine at the same time. There is also a middle ground where most PC
OSes sit today in which there may be a key app or two but generally
not a lot happening in the background. In fact, my desktop CPUs
generally sit fairly idle.

2. Many apps can have cooperating components without those components
sharing an entire address space. In fact, I'd suggest this is a good
idea. Only share what must be shared; keep the rest private. The
proposed model doesn't prevent using many CPUs working in different
address spaces on one app but sharing memory where shared memory is
needed.

3. There is a reason for the preferred model but it does not prevent
other models running. Practically, I am not going to code for other
models any time soon as there is more than enough to write already.
But I do want to be sure I don't paint myself into a corner.

> > (In that scenario, although all the tasks in one address space are
> > restricted to running on one CPU at a time, the other CPUs would not
> > need to be idle as they could still be busy executing active tasks in
> > other address spaces.)
>
> But what if they're not busy?

I think I've just answered that, above, but let me know if not.

...

> > 2. Alternatively, let the tasks share most address space but keep the
> > task-local data er, local to a CPU. If it would work this sounds quite
> > attractive.
>
> You can do that.  It's what I meant by keeping separate page tables (you
> could share everything but the toplevel page directory).  If you do it on a
> per-CPU basis, it'll complicate migrating tasks between CPUs.  If you do it
> on a per-thread basis, it'll consume more memory and be less efficient when
> switching between one address space's tasks on the same CPU.  I'd strongly
> suggest just going with a more conventional error handling approach.  But
> what's a hobby OS without a little crazy here and there? :-)
>
> Just be open to revisiting design decisions down the road if you no longer
> feel you're getting enough benefit relative to the cost.

Agreed. I don't want the design to lock-in a particular model but I
have given a lot of thought to designing a model that is best overall
and there are very good reasons for the details. I don't expect people
to see the "why" without seeing the whole picture (and I am aware that
until it is all working I've proven nothing). At any rate, I have
tried to keep in enough flexibility that I can support the intended
model while still allowing other models to work alongside.

...

I've snipped the code section. That needs a separate reply. I will
come back to it.

> > It is a moot point anyway as the original intention was to devise
> > something that would work in standard C. Changing a compiler or using
> > any compiler-specific facilities is out of scope.
>
> But changes to the OS kernel are fine? :-)

:-)

My focus has been on my own requirements, it's true, but I tried to
keep in mind the usefulness for C generally. I think I succeeded,
suggesting something that would be just as fast in a single-threaded
environment as what I plan for my OS, and ways to overcome C's slow
access to TLS where multithreading is in use. Ultimately the solutions
are standard C.

> TLS isn't standard C either (unless you're talking about C11?).

You might be right but does C's errno not have to be thread-local?

> How do you invoke a system call without compiler-specific facilities?

I'm very tempted to go into this here as it's a part of the design I
am particularly excited about but, as with what we have been
discussing, there are subtle elements in it and explaining one part
will require explaining others. If you don't mind, let's not go there
just now.

> >> -- but
> >> yes, you'd need some glue code around language transitions unless all the
> >> languages involved implement the same exception mechanism (asm could be made
> >> to, others maybe with compiler extensions?).  How often are potentially
> >> cross-language calls made, compared to component-internal calls?
>
> > Not very often, it's true. As a general construct, cross-language
> > calls are irrelevant. They may be very important for some of what I am
> > doing, though, where some of the code calls assembly routines. And
> > they simplify matters by making the interfaces consistent. (If only I
> > could do similarly for the calling conventions, eh!)
>
> You can use C linkage with no exceptions on functions that are implemented
> in or called from assembly (or other languages), and exceptions on other
> things.  Excessive consistency for consistency's sake can hurt.

As a maxim it's not bad but I don't accept the emphasis. I like
consistency when it's used for simplicity and efficiency.

...

> > The handlers wouldn't need to be the same. They could be the same or
> > different depending on need.
>
> I meant you'd most likely want that logic to be right there in the calling
> function.  Having to put it in a separate function seems awkward.

OK

> > True in many cases. I have a piece of code I wrote just the other day
> > (using the exception word idea) which needed multiple pieces of
> > memory. It naturally fell out as
>
> >   a = mem_alloc...
> >   b = mem_alloc...
> >   if (task_excep) goto excep_handler_mem;
>
> > Such a form may not be all that uncommon. Obtain a bunch of things
> > then check for an exception before using them.

I had an error in the above concept and code. I realise now that I
would have to check task_excep after each call. Why? Because each call
has to begin with no exceptions already indicated or it could assume
that an exception indication was as a result of a subroutine it had
called. The code therefore should be

a = mem_alloc...

if (task_excep) goto excep_handler_mem;
b = mem_alloc...
if (task_excep) goto excep_handler_mem;

There can still be a common exception handler but the test is required
after each call.

> Yes, and you can do this just as well:
>
>         a = mem_alloc...
>         b = mem_alloc...
>
>         if (!a || !b)
>                 goto err_mem;
>
> Yould still have two compare-and-branches (but maybe no load from memory,
> and is it really worth nitpicking a cycle here or there compared to dynamic
> memory allocation?  And later you say these are basically free), but it's
> easier to read and maintain than separate source-level checks.

As it turns out, I need the two checks for the reasons mentioned
above.

...

> >   cmp [task_excep], 0
> >   jne handler
>
> > should thus virtually disappear in the normal processing case.
>
> Possibly, depends on the details of the CPU you're on (still takes up
> icache, though), and what other things are happening.

On the icache point, if the handler is close by the code is only nine
bytes long. The zero in the instruction can be encoded as one byte and
sign-extended to thirty-two bits with no loss of performance using

cmp dword [task_excep], byte 0 ;7 bytes
jne handler ;2 bytes if handler is within 128
bytes

> But so will an OR
> instruction (especially operating on registers). It will probably run in
> parallel with something else.

Both your OR;JNZ and my CMP;JNE should operate in parallel with other
instructions.

James

[Scott Wood]

On 2012-01-13, James Harris <james.h...@googlemail.com> wrote:
> I'm aware of these objections though I hadn't thought about
> hyperthreading specifically. I agree but would make three points:
>
> 1. You rightly begin with an "if" clause and your points discuss a
> specific emphasis where a single specific app is (or a very small
> number of apps are) to be multithreaded in order to work faster. There
> are other target uses for an OS which have exactly the opposite
> requirements. For example, there are massive cost benefits to be
> gained by virtualisation wherein there are many tasks to be run on a
> machine at the same time. There is also a middle ground where most PC
> OSes sit today in which there may be a key app or two but generally
> not a lot happening in the background. In fact, my desktop CPUs
> generally sit fairly idle.

Yes, I was assuming a general-purpose OS, which includes desktop-like loads.

If you're targeting a specific use, and you think this is an appropriate
simplifying assumption, that's different.

> 2. Many apps can have cooperating components without those components
> sharing an entire address space. In fact, I'd suggest this is a good
> idea. Only share what must be shared; keep the rest private. The
> proposed model doesn't prevent using many CPUs working in different
> address spaces on one app but sharing memory where shared memory is
> needed.

Sometimes that's a good fit for the problem domain, sometimes not. Having
any difference in MMU context at all will slow context switches between a
process's threads when running on the same CPU.

Is the ability to run existing multithreaded software a concern?

>> TLS isn't standard C either (unless you're talking about C11?).
>
> You might be right but does C's errno not have to be thread-local?

It does if you have threads, but threads aren't part of C99. System
programming usually involves compiler extensions (or at least, dependence on
things not guaranteed by the standard).

-Scott

[James Harris]

On Jan 6, 10:31 am, "Rod Pemberton" <do_not_h...@noavailemail.cmm>
wrote:

...

> Is there something wrong with logging whatever is known about the unknown
> exception and continuing execution?

You know, I don't think we are too far apart in opinion on this. I may
be wrong but I think you are focusing on outcome where I have been
talking about the mechanism - in particular the default mechanism.

To take your question from the point of view of outcome, there may be
no problem at all with logging an exception and continuing execution.
In fact, in the majority of cases, that might be the right outcome.

In case someone objects that I am overstating it by saying this is
right in the majority of cases I would say to look at the system as a
whole, not just at one program. Ultimately, even if a constituent part
of a system has a catastrophic failure it is usually possible for some
other part of the system to detect that, fix what it can and restart
the program that failed. Then the system as a whole continues.

Taking take your question from the other point of view, that of the
mechanism: if an exception has occurred the routine has basically
three choices:

1) Ignore it.
2) Deal with it.
3) Pass it on.

To be clear, I don't mean that any of these on its own is an ethos a
routine must follow throughout. A routine can take different choices
at different points in the code depending on the conditions. It can
ignore some exceptions and deal with others, for example.

Ignoring an exception and carrying on can be the right course of
action for some situations, especially if the exception is harmless or
irrelevant.

Dealing with an exception and carrying on can also be the right course
to take in many situations. In this case the routine includes code to
allow it to address what happened and carry on.

Passing on an exception can also be right in certain circumstances. To
make an example, say we have a program which includes many child
routines. We *could* write each one to handle all of its own
exceptions but it may be more appropriate to say that we will put most
of the exception handling in the parent and if certain exceptions
happen in a child fails we handle them in the parent and restart the
child.

In all the three cases, above, the *outcome* is that the system
continues. The only question is the *mechanism* that one uses to
achieve this reliability.

...

> > > However, passively ignoring the exception is probably the best
> > > simple solution for a more 'fault tolerant' approach, if you know
> > > that most exceptions will be something non-serious or ignorable.
>
> > I can't agree. If you knew that *all* exceptions that could occur at a
> > certain point were benign this would be fine but if only *most*
> > exceptions would be ignorable there's always the chance that a
> > malignant exception happens.
>
> What do you intend to do about a "malignant" exception when it happens if
> you don't know what it is or how to code code to handle it, assuming that
> you even recognized that it occurred?

To borrow your phrase: "It depends on your goals." But as a general
solution: as an outcome, I would want to deal with the exception and
carry on. However, that doesn't mean I have to deal with the issue in
the subroutine where it occurred or was first detected.

The natural consequence of program routines calling other routines is
that we can end up with a call chain. Having an exception mechanism
can mean that each routine can handle the exceptions it is set up to
deal with. Say A calls B calls C calls D. D might suffer an exception
that we want to handle in B. Once handled, B can call C which can call
D again, hopefully this time with the original cause of the exception
having been taken away.

> The program has no method to contain
> any complications such an exception could cause, if in fact it does cause
> them.  So, which do you think is more likely: a) it's "benign" b) it's
> "malignant"?  That will decide your solution: a) exit b) ignore.

I would suggest that most exceptions are malignant but in a limited
context. For example, a file-open or a file-read failure may be
malignant in the context of a routine that has been written to process
a file but it might not be malignant in the context of another routine
that calls the first one. The caller may have created the file name in
the first place and be well set up to do something sensible if the
file cannot be opened.

In such a case it makes sense to write exception handling for file
open and file read errors in B and to pass them on from D and from C.

Note that this does not prevent D and C handling some of these
exceptions if we want them to. It just means that, by default, such
exceptions can be handled by B and the system as a whole can carry on.

> > If you ignore the harmful exceptions too the
> > program could do something undefined and, more to
> > the point, do it *silently*.
>
> So?

For a toy app I might not care. For almost anything else (and even for
most toy apps) I would rather be told that something went wrong that I
hadn't anticipated than be told everything was fine when it wasn't. As
an analogy, even if the news is unpleasant I'd rather be told the
truth than lied to!

> If you exit(0), that's done *explosively* ...

You don't need to exit the whole app. I did say that I was not
referring to the Unix exit() call. However, sometimes even that's
better than carrying on regardless.

> Your focus seems to be on only one side of the coin so to speak,
> when the opposite side can have the same theoretical issues.
>
> If you silently ignore, it's possible that data corruption will occur.  If
> you explosively exit, it's possible that data corruption will occur too.

That's true. Neither is desirable. But if you exit at least you get an
indication that something went wrong and the app will stop at that
point with the corruption you mention. That's better than it
continuing to corrupt the rest of the storage!

> The data error when silently ignoring could be due to "corrupted" code
> overwriting the data.  The error when explosively exiting could be due to
> not flushing memory buffers to disk.

Well, I think the OS would flush anything the program had written but
I see your point. Again, if the program 'explosively exists' at least
you know that something went wrong and, as a human, you then have the
opportunity to fix it.

...

> If you've got a low-use program or a program which works correctly 90%+
> percent of the time, you might not ever find or notice an error.  For
> personal C programs, I frequently skip checks for returned NULLs on the
> various string functions.  It can take quite a bit of extra code to make
> them 100% "fail-safe".

True. Of course, if C had an exception mechanism (automatic, not
longjmp-based) C functions could generate exceptions rather than
return invalid values. That could make all of our ad-hoc code short
without compromising safety as we could write code without checking
for many errors that we should check for now. If any occurred they
would be reported automatically and the program would stop.

>  I expect the input data to match what I coded the
> routine to handle.  Occasionally, it doesn't and I have to fix it.  E.g., if
> I'm searching for '\r', I expect the input string to have an '\r', instead
> of not having one and so returning a NULL which is then dereferenced.
> Sometimes it's easier to pad, e.g., with '\r' when searching for it, than it
> is to check for a NULL.

Umm, I see your point but I'm not sure whether it fits with the
concept of exception handling that I have. It may do, depending on the
details. I don't generally see exceptions as part of raw input. One
option you have probably considered is to read the string via a
wrapper that allows and adjusts for minor issues with what it reads.
As a rule, I guess often it's good to sanitise raw input data where we
have no control over what we get passed. We can ask for five lower
case letters but, in all sorts of ways, we might not get them.

...

> > You know the term fail-safe. AIUI the original meaning of that is that
> > is something failed it would fail to a safe condition rather than
> > failing to either an unsafe condition or an indeterminate condition.
>
> If the fail-safe perspective contradicts the fault-tolerance perspective,
> how do you reconcile the two?  Do you preference fail-safe or
> fault-tolerance?  It's your choice as to which way to go.

My preference? Fault-tolerant outcome. Fail-safe default mechanism.

> If I get an NMI in my OS, I know I don't have code in place to fix whatever
> triggered it.  In fact, it took quite a bit of work just to determine what
> could possibly trigger NMIs.  I now know many of the things that will
> supposedly trigger it.  I accumulated a list of 31 or so items.  Most of
> them are non-critical, some are user events, some are unfixable via
> software, etc.  I'm not aware of any standard and current (at least post
> 486...) hardware that uses NMI (yet).  There is bunch of stuff on the list
> such as old PCJr and PS/2 hardware, old DRAM memory failures, (supposedly)
> DMA timeouts, local bus timeouts, watchdog cards, some soundcards, etc.  So,
> I've ignored NMI errors.  If PCI uses it upon timeout or somesuch, I may
> have to change that.  That's clearly not fail-safe as you've described it.
> As you've described it, fail-safe would probably require a complete reboot
> when an NMI occurs to eliminate any potential hardware sources of the
> problem causing futher NMI errors.  Fault-tolerant however can ignore it and
> continue execution, probably without issue in 99% of the situations.  For
> things like double-faults, I still need to review the manuals to see what is
> recoverable and what isn't since I haven't worked on it in a few years.

I understand your comments about NMI and agree with them.

Is it feasible to detect and sensibly handle some of the possible
conditions? Presumably you might be able to at least log the NMI, if
possible, or make the user aware of it in some other way.

A person, being aware that an NMI occurred, if he/she has access to
the machine might then be able to take some sort of remedial action.
Remember I spoke about keeping a "system" running. A person can be
part of a system too so the person fixing the problem is part of that
system.

> > I guess I like a fail-safe system whereas you are an optimist. ;-)
>
> I worked on fault-tolerant systems for a while ...  You couldn't legally
> "exit(0)" the program.  If you did, "you" (the company) were fined for
> breaking the law.  The program had to continue execution - preferably
> correct execution - even if it experienced an "unrecoverable" and "unknown"
> exception or signal or whatever.

Understood and agreed with. Presumably there was some way to indicate
that something unknown or unrecoverable had happened even if only by
turning on or off an LED.

> Fail-safe - as you've described it -
> wasn't an option.

Fail-safe doesn't mean fail-completely! The program does not need to
exit() to be safe.

> We used software that did something similar to STM.

STM?

> I
> think, perhaps, that fault-tolerant concepts corresponded to my own style
> of programming.

Sometimes a *machine* cannot fail to a completely safe state but the
*system* it operates in can add a degree of safety.

Here's a good example of potential tension between two options where
neither is safe: a dialysis machine which does something like pump and
clean blood. Say it detects a slight issue with the blood pressure of
its output. What could be wrong? A number of things. Perhaps the
output tubing could have a small puncture, allowing tiny bubbles of
air to pass with the blood into the patient. Of the two basic choices,
stop pumping or carry on, which is safe? Basically, neither.

However, that's just the machine. The *system* includes humans such as
medical personnel. In this case the machine can stop pumping (not
fully safe but the safer of the two options) and sound an alarm. The
alarm helps to bring into play the rest of the system. A person can
fix the issue and continue with the process.

That dialysis machine's response is rather like a subroutine
experiencing an exception and leaving a higher part of the system to
address the problem that occurred.

James