None of this is set in stone, but take a look and see how everyone feels
about this.

I'll get to IO after, since the two are pretty tightly intertwined, and
changes to this will affect the IO stuff too.

------------>Snip here<---------------
Events, another design sketch

Overview
========

Events are a terrific idea, as evidenced by the vast array of event
handling libraries there are. (There are at least a half dozen on
CPAN) All of which are the same but different, and incompatible. Of
course.

Proper event handling's the only way to manage asynchronous IO, and
since parrot needs event handling it only makes sense to expose it in
general, so we can avoid the problems of having multiple event
handling loops.

Event basics
============

An event is an indication to your program that something has
happened. It may be that a timer has fired, a packet has been
received (or sent) on a socket, a disk read has completed, or that
the user has moved the mouse.

Events are asynchronous--that is they can happen at any time, and
your program has no real control over when they are generated. It
does, however, have control over when they are processed. More on
that later.

Event Classes
=============

There are two main classes of events, which we'll call expected and
unexpected.

An expected event is one that your program is specifically expecting
to get as a result of a request for asynchronous action, and each
expected event has a corresponding event handle. User code can wait
for an expected event to complete, cancel an expected event, and
query the status of an expected event. Expected events may also be
either complete or incomplete. An incomplete event is one whose
requested action hasn't yet happened, while a complete event

Unexpected events, on the other hand, happen for reasons outside of
your program. Signals, GUI events, and exceptions for asynchronous
files (which may not be delivered as signals on many platforms) are
examples of unexpected events. Unexpected events, by their nature, are
always complete, since they aren't generated until an action actually
occurs.

Some event sources may be either expected or unexpected. Network
connections are an example of this--you may have a network filehandle
set either as an expected event source, where your program must
explicitly ask for data, or set as an unexpected event source, where
a monitoring thread captures data as it is received and generates
events for the captured data.

Callbacks and User Data
=======================

Each expected event can have a callback routine and piece of user
data associated with it. When the event request is satisfied (The IO
operation completes or the timer fires, for example) the callback
associated with it is invoked. The user data, as well as the event
data, if there is any, is passed into the callback.

Callback signatures are fixed, and of the form:

   (Preturndata, Icommandstatus) =
          callbacksub(Pevent, Peventdata, Puserdata, Icommand)

The callback is passed in the event PMC, the PMC for the event data,
and the PMC for the user data, if any. (Either or both of these can
be the Null PMC if there is no user or event data) Command is always
0 for callback handling. (Callbacks and IO layer functions are
identical. More detail in the IO section)

The callback must then return the data and a command status. The
status should be 0 for correct completion, a non-zero negative number
for an error, and a non-zero positive number for successful
completion with a change in status for the next handler in the chain,
if there is one.

If a callback function throws an exception the exception itself is
deferred. Expected events defer the exception until the event is
waited on, while unexpected events throw their excecption when their
event functions are done processing. (This does mean that expected
events may have their exceptions effectively ignored if the event is
never waited on)

When there are multiple event handler in a chain, each handler can
alter the data and command passed to the next handler in the
chain. The return data is passed in as the user data to the next
command, and the return status is, if a positive non-zero number,
passed to the next handler in the chain. If the return status is 0,
then the same command is passed into the next handler in the
chain. This allows event handlers to preprocess the event if need be.

Generic Handlers
================

Each event source can have a generic handler associated with it. This
handler is called before the event's specific callback is invoked, if
it has one. Generic handlers have the same function signature as
callback functions do, and behave the same way--if they return 0 the
event is passed to the next handler in the chain, and if they return
non-zero then the event stops passing through and is marked as failed.

Note that the handler table is scoped, however due to the inherently
unpredictable nature of asynchronous events it isn't possible to
guarantee that changes in the handler table due to scope changes will
take place immediately so far as event sources are concerned--that
is, the program may leave a scope and change the handler table yet
have an event generated that uses the old handler table.

Events and Exceptions
=====================

Event handlers are generally discouraged from throwing exceptions,
however it is sometimes necessary. Exceptions are handled differently
by expected and unexpected events.

With an expected event, if an event handler is terminated by an
unhandled exception, that exception is attached to the event itself,
and when user-level code C<wait>s on the event, the exception is
thrown then.

Unexpected events whose handlers throw exceptions are handled somewhat
differently. In that case, the exception is thrown as soon as the
event is drained from the event queue. This may or may not happen
immediately, depending on how the event in question is currently being
handled. If the event is processed in a separate event handling thread
then the event is put in the main thread's event queue and the
exception will be thrown when the main thread drains the queue and
hits the event. If the event is currently being processed in the main
thread (because there is no event-handling thread, or this event
isn't being handled by one) then as soon as the event is drained from
the event queue the exception is thrown.

How events are handled
======================

There are three steps in event handling.

First, an event is triggered. This puts the event into the event
queue, but otherwise does nothing. (This operation is interrupt-safe,
though there issues in obtaining event PMCs at interrupt level in
some cases)

Next, a queue runner pulls an event out of the event queue and calls
its handlers. These are a combination of any event class handlers as
well as an event-specific callback, if there is one. This operation
happens as normal user-level code, though it may be done
significantly after the event itself is put in the queue, and may not
happen in the same thread as the event was requested from. [NB: I'm
not sure what to do about event handling threads and shared data yet]

Finally, for expected events, something can wait on the event to get
the event's data and status. This can be done before the event has
completed, in which case the operation will wait (draining the event
queue of pending events)

Note that the queue runner will automatically wait on any unexpected
event that it completes. This puts the event in the done state and
will throw any exception that the event handlers may have
generated. Expected events, however, will be left alone until
user-level code explicitly waits on it.

Event States
============

Events have five states. They are:

* Quiescent - The event object has been newly allocated but hasn't
              been associated with anything yet and can't be triggered
* Waiting - The operation the event triggers on hasn't yet happened
* Triggered - The operation the event triggers on has happened, but
              the event's handlers haven't been called yet
* Ready - The event has been processed by its handlers but hasn't yet
          been waited on to flush out any exception it might have.
* Done - The event has flushed any pending exceptions and can now
         only be queried for its status and data

Creating Events
===============

Events are just PMCs, and there are no ops specifically to create
them. To create an event, allocate a PMC of the class Event. These
are array/hash combination PMCs and may be accessed as follows:

  0 (CLASS)      - The class of the event
  1 (EVENT_DATA) - The event data associated with this event
  2 (USER_DATA)  - The user data associated with the event
  3 (CALLBACK)   - The callback sub for this event
  4 (EXCEPTION)  - The pending exception for this event, if any
  5 (STATUS)     - The status from the exception handlers
  6 (STATE)      - What state the event is in
  7 (LEVEL)      - The event level this event is at

Event Opcodes
=============

The opcodes in this section are a combination of event requests and
event handling ops. It doesn't include the IO ops--those are separate.

Most of the event request ops have two forms, one of which takes a
callback PMC and user data PMC.

   checkevent

Explicitly check to see if there are any events pending in the event queue and,
if so process one.

   drainqueue

Wait forever, draining the event queue as it fills. This op only
exits if it encounters a QUEUE_DONE event in the queue.

   wait Pevent

Wait for the specified event to complete. If the event has already
completed the op doesn't wait. If there is a pending exception it is
thrown. This opcode will drain the pending event queue.

   getstat Istatus, Pevent

Returns the status of the event, noting whether it has been completed
yet and, if so, what happened.

   getdata Pdata, Pevent
   getdata Sdata, Pevent

Get the data associated with the event, if there is any.  What the data
...

read more »

 One quick point: unless I'm misunderstanding something, there seems to
 be no easy way to wait on multiple events to complete if those events
 can occur in any order. For instance, suppose we have 5 events, P1 - P5
 If we know that the events will occur in some specified order (say
 1-2-3-4-5), then we can just loop over C<wait Px>, specifying the
 apporpriate x for each iteration. However, if the events actually occur
 in the order 4-3-2-5-1, we'll drop events 2-5 while waiting for 1.
 Now, this is fine if 1 _must_ happen before 2 etc., but often this is
 more restrictive than we need to be, since all we really care about is
 ensuring that all 5 events are handled before we do anything else.
 Is there any easy and efficient way to do this with the current
 framework?

 Simon

At 12:20 PM -0400 5/11/04, Simon Glover wrote:

There's no easy way, no. The best you could do is have the code:

    wait E1
    wait E2
    wait E3
    wait E4
    wait E5

which'll work (and everything'll get processed properly, and as soon
as it's available) but is somewhat nasty.

Thinking we might want:

    waitall Parray_of_events
    waitany Parray_of_events

?
--
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
d...@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

  DS> Event Classes
  DS> =============

  DS> There are two main classes of events, which we'll call expected and
  DS> unexpected.

  DS> An expected event is one that your program is specifically expecting
  DS> to get as a result of a request for asynchronous action, and each
  DS> expected event has a corresponding event handle. User code can wait
  DS> for an expected event to complete, cancel an expected event, and
  DS> query the status of an expected event. Expected events may also be
  DS> either complete or incomplete. An incomplete event is one whose
  DS> requested action hasn't yet happened, while a complete event
                                                                  ^^^^^
?????

  DS> Unexpected events, on the other hand, happen for reasons outside of
  DS> your program. Signals, GUI events, and exceptions for asynchronous
  DS> files (which may not be delivered as signals on many platforms) are
  DS> examples of unexpected events. Unexpected events, by their nature, are
  DS> always complete, since they aren't generated until an action actually
  DS> occurs.

  DS> Some event sources may be either expected or unexpected. Network
  DS> connections are an example of this--you may have a network filehandle
  DS> set either as an expected event source, where your program must
  DS> explicitly ask for data, or set as an unexpected event source, where
  DS> a monitoring thread captures data as it is received and generates
  DS> events for the captured data.

i disagree with the two classes. how can you handle an unexpected event
without know it could happen? effectively all events are expected, some
are more expected than others. :)

the only way to get mouse movements (which is really translated to some
i/o or socket event by the windowing system) is to ask for them. same
with signals. if you don't ask to handle a signal, it uses the default
(usually IGNORE or whatever). you need a handler for each possible event
so effectively all events you can handle are expected.

  DS> Callbacks and User Data
  DS> =======================

  DS> Each expected event can have a callback routine and piece of user
  DS> data associated with it. When the event request is satisfied (The IO
  DS> operation completes or the timer fires, for example) the callback
  DS> associated with it is invoked. The user data, as well as the event
  DS> data, if there is any, is passed into the callback.

the event handle(r) itself also is passed to the callback. you say event
data there and maybe you mean event structure/object?

so let me clarify. the callback is passed the event itself, optionally
any data read for the event, and optionally any private data passed into
the event (this is usually the object or user id for this event).

  DS> Callback signatures are fixed, and of the form:

  DS>    (Preturndata, Icommandstatus) =
  DS>           callbacksub(Pevent, Peventdata, Puserdata, Icommand)

  DS> The callback is passed in the event PMC, the PMC for the event data,
  DS> and the PMC for the user data, if any. (Either or both of these can
  DS> be the Null PMC if there is no user or event data) Command is always
  DS> 0 for callback handling. (Callbacks and IO layer functions are
  DS> identical. More detail in the IO section)

how about renaming userdata to iodata since that is what it
is. eventdata could be left alone or actually renamed to userdata since
it is the data passed in to the event call and the user expects it to
come back so the code can identify the event and associate it with an
object/structure.

  DS> If a callback function throws an exception the exception itself is
  DS> deferred. Expected events defer the exception until the event is
  DS> waited on, while unexpected events throw their excecption when their
  DS> event functions are done processing. (This does mean that expected
  DS> events may have their exceptions effectively ignored if the event is
  DS> never waited on)

you need to clarify expected vs unexpected more. i wrote above they
should all be expected. i don't see how this section would change that.

  DS> Events and Exceptions
  DS> =====================

  DS> Event handlers are generally discouraged from throwing exceptions,
  DS> however it is sometimes necessary. Exceptions are handled differently
  DS> by expected and unexpected events.

  DS> With an expected event, if an event handler is terminated by an
  DS> unhandled exception, that exception is attached to the event itself,
  DS> and when user-level code C<wait>s on the event, the exception is
  DS> thrown then.

huh? if you are in an event handler, you have seen the event (and are
presumably in user level code), so why would you wait for it again?

  DS> Unexpected events whose handlers throw exceptions are handled somewhat
  DS> differently. In that case, the exception is thrown as soon as the
  DS> event is drained from the event queue. This may or may not happen
  DS> immediately, depending on how the event in question is currently being
  DS> handled. If the event is processed in a separate event handling thread
  DS> then the event is put in the main thread's event queue and the
  DS> exception will be thrown when the main thread drains the queue and
  DS> hits the event. If the event is currently being processed in the main
  DS> thread (because there is no event-handling thread, or this event
  DS> isn't being handled by one) then as soon as the event is drained from
  DS> the event queue the exception is thrown.

  DS> How events are handled
  DS> ======================

  DS> There are three steps in event handling.

  DS> First, an event is triggered. This puts the event into the event
  DS> queue, but otherwise does nothing. (This operation is interrupt-safe,
  DS> though there issues in obtaining event PMCs at interrupt level in
  DS> some cases)

  DS> Next, a queue runner pulls an event out of the event queue and calls
  DS> its handlers. These are a combination of any event class handlers as
  DS> well as an event-specific callback, if there is one. This operation
  DS> happens as normal user-level code, though it may be done
  DS> significantly after the event itself is put in the queue, and may not
  DS> happen in the same thread as the event was requested from. [NB: I'm
  DS> not sure what to do about event handling threads and shared data yet]

  DS> Finally, for expected events, something can wait on the event to get
  DS> the event's data and status. This can be done before the event has
  DS> completed, in which case the operation will wait (draining the event
  DS> queue of pending events)

  DS> Note that the queue runner will automatically wait on any unexpected
  DS> event that it completes. This puts the event in the done state and
  DS> will throw any exception that the event handlers may have
  DS> generated. Expected events, however, will be left alone until
  DS> user-level code explicitly waits on it.

so you separate expected vs unexpected as really user level handlers vs
parrot internal handlers. why not call them all expected and
internal. only those with user level stuff get a user level callback or
wakeup on a wait. i don't see why the handling is so different.

  DS> Event States
  DS> ============

  DS> Events have five states. They are:

  DS> * Quiescent - The event object has been newly allocated but hasn't
  DS>               been associated with anything yet and can't be triggered

call that 'new' instead. or maybe disabled (but that means the event has
been associated with something already so new is better)?

  DS> * Waiting - The operation the event triggers on hasn't yet
  DS> happened

this will be confusing with the 'wait' operation. i would call it
'watching' (from event.pm) or something similar (maybe 'active' or
'enabled'?)

  DS> * Triggered - The operation the event triggers on has happened, but
  DS>               the event's handlers haven't been called yet
  DS> * Ready - The event has been processed by its handlers but hasn't yet
  DS>           been waited on to flush out any exception it might have.
  DS> * Done - The event has flushed any pending exceptions and can now
  DS>          only be queried for its status and data

you also can use a 'stopped/disabled' state. in many cases you want an active
event but stopped for a while. a good example is a write event on a
an empty socket. if enabled, it will trigger all the time since the
socket is always writeable and that will suck up the cpu. what i do is
make write events disabled by default and you enable then when you have
data to actually write. when the user space write buffer is empty you
disable the event again. this is much cleaner than creating/destroying
events.

so my state list is:

        new
        enabled
        disabled
        triggered
        ready
        done

when an event is done, who/what flushes the event itself from the
system, normal GC?

  DS>    getdata Pdata, Pevent
  DS>    getdata Sdata, Pevent

  DS> Get the data associated with the event, if there is any.  What the
  DS> data is depends on what generated the event. (For filehandle reads
  DS> this will be the data read from the filehandle)

but won't that data already be passed in the callback? another point is
that event i/o data should be passed by reference and treated as
readonly to save copying all over the place.

  DS>    cancel Pevent

  DS> Cancel the event, if possible.

add enable/disable here.

  DS>    settimer Pevent, Nseconds[, Pcallback, Puserdata]
  DS>    settimer Pevent, Iseconds[, Pcallback, Puserdata]

  DS> Set a timer to go off in C<seconds> seconds.

  DS>    setalarm Pevent, Nabs_time[, Pcallback, Puserdata]
  DS>    setalarm Pevent, Iabs_time[, Pcallback, Puserdata]

  DS> Set a timer to go off at the absolute time specified.

  DS>    setinterval Ninterval_seconds[, Pcallback, Puserdata]
  DS>    setinterval Iinterval_seconds[, Pcallback, Puserdata]

  DS> Set an interval
...

read more »

I certainly do!

I'm also curious how to write an interface to an existing event system.
Being able to write it all in PASM is a bonus.

-- c

  SG>  One quick point: unless I'm misunderstanding something, there seems to
  SG>  be no easy way to wait on multiple events to complete if those events
  SG>  can occur in any order. For instance, suppose we have 5 events, P1 - P5
  SG>  If we know that the events will occur in some specified order (say
  SG>  1-2-3-4-5), then we can just loop over C<wait Px>, specifying the
  SG>  apporpriate x for each iteration. However, if the events actually occur
  SG>  in the order 4-3-2-5-1, we'll drop events 2-5 while waiting for 1.
  SG>  Now, this is fine if 1 _must_ happen before 2 etc., but often this is
  SG>  more restrictive than we need to be, since all we really care about is
  SG>  ensuring that all 5 events are handled before we do anything else.
  SG>  Is there any easy and efficient way to do this with the current
  SG>  framework?

you have a callback for each event and it sets a flag (better idea
below). when all the wanted flags are set, you post another event that
all 5 things are done. your main code waits on that event.

a nice way to handle the multiple flag things is with a hash and a
callback. you preset the hash to all the keys you are waiting on. when each
handler is triggered, it deleted its key (which can come from the
private event data stored in the event itself) from the hash. when the
hash is empty, trigger the callback. i call this a scatter (start the
async actions) and gather (collect their done flags) operation.

this is best done at a user level. let parrot handle each event
individually. there is no major benefit to parrot doing this for you IMO
as it is not system level at all.

uri

--
Uri Guttman  ------  u...@stemsystems.com  -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs  ----------------------------  http://jobs.perl.org

At 10:09 AM -0700 5/11/04, chromatic wrote:

Fair enough.

I don't think it can be all-PASM, except maybe (and maybe not...)
with a separate thread for the existing event source. To do it in all
pasm means calling back into parrot from interrupt level, which isn't
really doable, or have a thread just waiting on events from the
alternate event system to post into the parrot event queue.

Which, I suppose, is doable, though there are still those pesky
interrupt-level limitations pontentially in there. We'll need to get
shared-pool threads working better before we can really take a shot
at it.
--
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
d...@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

Another approach may be to expose the PollEvent and WaitEvent functions
to the event system as alternate sources of events.  If I can do that
from PASM, I think I'm okay.

-- c

At 10:33 AM -0700 5/11/04, chromatic wrote:

That'll still need some C. The event system as it stands is all
active--all event sources put events into the system, rather than
having the event system go looking at event sources for events. You'd
either need to queue up regular timer events to go check for new
stuff or have a thread actively polling the source for events and
throwing them into parrot's event queue.
--
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
d...@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

  c> On Tue, 2004-05-11 at 10:24, Dan Sugalski wrote:
  >> >I'm also curious how to write an interface to an existing event system.
  >> >Being able to write it all in PASM is a bonus.
  >>
  >> I don't think it can be all-PASM, except maybe (and maybe not...)
  >> with a separate thread for the existing event source. To do it in all
  >> pasm means calling back into parrot from interrupt level, which isn't
  >> really doable, or have a thread just waiting on events from the
  >> alternate event system to post into the parrot event queue.

  c> Another approach may be to expose the PollEvent and WaitEvent functions
  c> to the event system as alternate sources of events.  If I can do that
  c> from PASM, I think I'm okay.

i don't think exposing them is a good idea as they can be broken by user
stuff. and see my other reply for another solution to waiting on
multiple events without any extra core code. it is very easy and
eliminates the need for a group wait op. one problem with a group wait
is that you can ONLY wait for that group. what if you wanted to know
when group A events are all done and also when group B events are all
done?  and what if you have other single events floating around? rarely
do you just have a single group of events to wait for.

my solution works with any combination of events and groups.

uri

--
Uri Guttman  ------  u...@stemsystems.com  -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs  ----------------------------  http://jobs.perl.org

At 1:10 PM -0400 5/11/04, Uri Guttman wrote:

Just chop that last sentence fragment out. It makes no sense. :)

The terminology there's a bit strained, and I think it's in large
part responsible for most of the rest of the confusion.

They're probably better called Named and Anonymous events, though
that's a bit dodgy in some ways too. Maybe specific and generic
events would be better.

The basic difference is that with Named events you *know* in advance
that the event is coming because you explicitly requested it by
setting a timer or making an IO request. (A specific timer or IO
request, not timer or IO requests in general) This is different from
things like signals as you've not asked for a particular signal to
fire--they fire off whenever something outside your control and
expectation happens.

Because of this, you have the event PMC for a Named event before the
event occurs and thus can wait on it. You *don't* have the event PMC
for an anonymous event, so you can't wait on it, all you can do is
semi-generically react once it's occurred.

No, it isn't.

Nope, I mean event data. It's the data that whatever triggered the
event generated.

For mouse click events it may be the X/Y/Duration of the click, for
read requests it's the data read, and so on.

Yes.

It isn't though. The event data is the IO data here.

You're confused here. The event handler is what throws the exception.
It's the user code that's waited on the event that gets the
exception. (And it's possible, though likely a bad idea, to have an
exception handler make an async request and wait for it to complete)

That can't happen. You can cancel an event, but that's it. You can
stop an event source, but that's different.

Yep.

Yes. Which doesn't do you much good if you're not in the callback... :)

Nope. Events aren't enable-able/disable-able.

Yes.

I didn't see any reason to not do absolute times as well as deltas.
It's no big deal either way.

Ick. No. It's not like there's any less code there, and the intent's
a bit clearer.

Posts an event of the specified type to the event queue of the target
interpreter.

No PMC handling. It means that whatever code's doing it doesn't have
to manage a PMC queue (since if it's in an interrupt handler it can't
allocate one--they need to be preallocated) and set up all the bits
of the event PMC. Of limited utility, but for things like signal
handlers that can't do much anyway and have so little information
available to them it's sufficient.
--
                                         Dan
...

read more »

The thread seems like the way to go, if I can enqueue an event without
writing any C -- just post EventPMC from PASM?

So for SDL, I'd start a separate thread that blocks on SDL_WaitEvent,
creating and posting events when they happen.  My main program would
handle the events as normal Parrot events.  Standard producer consumer
stuff.

Since it's blocking, it won't eat up too many resources -- that's nice.
It'd be nice to have the SDL event thread ignore events I don't care
about though, instead of creating event PMCs I'll just throw away later.

Is this what you have in mind?

-- c

At 11:18 AM -0700 5/11/04, chromatic wrote:

You can always Get Horribly Clever in the event handling thread and
look at what the SDL library's handed you. If it's uninteresting you
can just throw it away rather than creating an event to be discarded.

Yep.
--
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
d...@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

What's interesting in one context is quite dull in another.  This means
shuttling data between threads, which is always the point at which
things start to go wrong*.  At least for me.

-- c

* Since we haven't had a good flamewar in at least twenty minutes,
"Threads are good as long as you keep them far away from each other!"

What's the plan for integrating with system events, then? Mac OS X and
Windows both have robust, irreplacable, system-managed event loops.
parrot's loop can (and should) run in a parallel thread to those, but
certainly can't presume to take over entirely. It simply can't work.

If you're calling this an event system, it ought to mesh in with the
notion of events that every programmer who uses them will have:

 - Mouse events.
 - Keyboard events.
 - Redraw events.
 - Drag-'n'-drop events.
 - Menu command events.
 - etc., etc., etc..

Problems in this domain also include: Focus management, propagation
along the responder chain, enabling/disabling commands, default event
handlers.

You won't get any of these events from an I/O wait on Win32 or on a Mac;
they're not even delivered via Unix I/O. Note that the thread on which
these events must be handled is the thread to which they are delivered
(and not by parrot's event loop): UI APIs are not thread-safe. So the UI
thread needs to be able to enter a parrot callback on the same thread.

Asynchronous I/O completion is surely considered less of an
event-handling problem and more of a thread-synchronization problem.
Also, parrot async I/O completion hopefully won't need to be serialized
through an I/O retirement thread ("event loop," whatever you want to
call it) except when the platform winds up requiring that through sloppy
async APIs.

This is really an asynchronous notification API, where asynchronous
completion is a very important "fires-once" form of asynchronous
notification. It's far, far below what programmers will expect when
hearing the term "event."

.NET's thread pool is a very close match to what you're discussing. A
.NET "WaitHandle" is almost identical to your event source. (Right down
to the waitone/waitany ops you just suggested.)

The intuitive concept of "events" isn't even in the same class of
problem that your document addresses. Probably best to use another term
for this (very cool, very necessary) technology.

--

Gordon Henriksen
IT Manager
ICLUBcentral Inc.
gor...@iclub.com

P.S. - Now that they're in, have you considered using objects and
methods instead of opcodes to define parrot APIs? If parrot technologies
are exposed through objects, you'll save on opcount, and HLLs won't need
to build yet another a shim for every new parrot feature. Dogfood.

As I pointed out in another post, this doesn't work for integrating with
at least two significant "event sources:" Windows and the Mac OS. :) UI
events need to be handled synchronously on the thread to which they were
delivered, since the GUI APIs are not threadsafe. Trying to handle these
events from another thread is, quite simply, a doomed endeavour.

--

Gordon Henriksen
IT Manager
ICLUBcentral Inc.
gor...@iclub.com

  >> i disagree with the two classes. how can you handle an unexpected event
  >> without know it could happen? effectively all events are expected, some
  >> are more expected than others. :)

  DS> The terminology there's a bit strained, and I think it's in large part
  DS> responsible for most of the rest of the confusion.

that seems to be the case. do an edit pass with better names and clearer
definitions.

  DS> They're probably better called Named and Anonymous events, though
  DS> that's a bit dodgy in some ways too. Maybe specific and generic events
  DS> would be better.

  DS> The basic difference is that with Named events you *know* in advance
  DS> that the event is coming because you explicitly requested it by
  DS> setting a timer or making an IO request. (A specific timer or IO
  DS> request, not timer or IO requests in general) This is different from
  DS> things like signals as you've not asked for a particular signal to
  DS> fire--they fire off whenever something outside your control and
  DS> expectation happens.

ok. now signals are not normally 'handled' by any code (c included)
unless you specify something. so all events at that level are always
specified. if parrot needs to handle signals even when no user code
asked to handle them, then i can see a need for this separation. but it
should only be used for internal parrot stuff and all other events
should be specified at the user level. i see no reason for any user code
(PASM and up) to not do a proper handler specification.

  DS> Because of this, you have the event PMC for a Named event before the
  DS> event occurs and thus can wait on it. You *don't* have the event PMC
  DS> for an anonymous event, so you can't wait on it, all you can do is
  DS> semi-generically react once it's occurred.

i think that should read "all parrot can do is react, assuming it is the
one that set the signal handler (at the c/kernel level)".

  DS> Callbacks and User Data
  DS> =======================
  >>
  DS> Each expected event can have a callback routine and piece of user
  DS> data associated with it. When the event request is satisfied (The IO
  DS> operation completes or the timer fires, for example) the callback
  DS> associated with it is invoked. The user data, as well as the event
  DS> data, if there is any, is passed into the callback.
  >>
  >> the event handle(r) itself also is passed to the callback.

  DS> No, it isn't.

i think i meant the event itself. the handler is what is called in the
callback.

  >> you say event
  >> data there and maybe you mean event structure/object?

  DS> Nope, I mean event data. It's the data that whatever triggered the
  DS> event generated.

then those terms need to be cleanly defined. user data is passed into
the event and passed back to the callback. it is meant for the user code
to associate an event with some user state. it can be an
object/structure/id//whatever. the event data is the actual data
returned by the event iself from the event source. it will contain data
read from an file/socket or mouse coords/stuff or signal
information, etc.

  DS> Callback signatures are fixed, and of the form:
  >>
  DS> (Preturndata, Icommandstatus) =
  DS> callbacksub(Pevent, Peventdata, Puserdata, Icommand)
  >>
  DS> The callback is passed in the event PMC, the PMC for the event data,
  DS> and the PMC for the user data, if any. (Either or both of these can
  DS> be the Null PMC if there is no user or event data) Command is always
  DS> 0 for callback handling. (Callbacks and IO layer functions are
  DS> identical. More detail in the IO section)
  >>
  >> how about renaming userdata to iodata since that is what it
  >> is.

  DS> It isn't though. The event data is the IO data here.

ok, i can work with your name choices as long as they are clearly defined.

  >> huh? if you are in an event handler, you have seen the event (and are
  >> presumably in user level code), so why would you wait for it again?

  DS> You're confused here. The event handler is what throws the
  DS> exception. It's the user code that's waited on the event that gets the
  DS> exception. (And it's possible, though likely a bad idea, to have an
  DS> exception handler make an async request and wait for it to complete)

i am confused everywhere! :)

the OP says this:

  DS> With an expected event, if an event handler is terminated by an
  DS> unhandled exception, that exception is attached to the event itself,
  DS> and when user-level code C<wait>s on the event, the exception is
  DS> thrown then.

you have to be in an event handler to terminate it. so you must have
waited for it or be in a callback. so why would you need to wait for it
again after it terminated because of an exception? you are in user code
space now so let the exception system do its thing (like you said). if
it isn't caught, do the normal thing. i don't see why the event needs to
be tagged with the exception. if it wasn't handled in the (already
called) handler environment why would another wait be any better?

my point is that an event handler IS user level code. all the event
system does is monitor the actual events, queue the triggered ones and
dispatch to handlers via callbacks. the handlers and exception handling
is all user space above that.

  >> you also can use a 'stopped/disabled' state. in many cases you want
  >> an active event but stopped for a while.

  DS> That can't happen. You can cancel an event, but that's it. You can
  DS> stop an event source, but that's different.

that is bad IMO. why do you need the entire overhead of
creation/destruction of an event just to stop it temporarily? this
buffered async write scenario is common and very useful. it is so much
easier and faster to just stop/start the existing event instead of
creating/cancelling a new one each time the writer needs to fire up.

this can be done easily for normal i/o events with a flag which makes
the select/poll loop skip it. if you are going to do blocking
reads/writes in individual kernal threads for this, then it will be
harder or impossible. you could still dedicate one thread to handle all
simple i/o event requests and have it run a select/poll loop for them
all. then when it gets a triggered event, it does the same queueing of
it that you want to do now. this way you save on threads, allow for
stop/start and still can use other kernel threads for special event
stuff that won't always work well with event loops (signals for one).

  >> add enable/disable here.

  DS> Nope. Events aren't enable-able/disable-able.

can you explain why not?

  >> i don't think there is a need for all those variants. why would alarm
  >> need any special opcode when it is just a timer with a delay of abs_time
  >> - NOW? let the coder handle that and lose the extra op codes.

  DS> I didn't see any reason to not do absolute times as well as
  DS> deltas. It's no big deal either way.

  >> also the interval can be folded in as a extra arg to the timer (either a
  >> repeat flag or an interval). the initial seconds can be the first delay
  >> and then reused for intervals or the interval value can take over:
  >>
  >> settimer Pevent, Idelay_seconds, Iinterval_seconds[, Pcallback,
  >> Puserdata]
  >>
  >> if interval_seconds is set (>0), use it for a repeating timer. if
  >> delay_seconds is set (>0), it is the first interval.

  DS> Ick. No. It's not like there's any less code there, and the intent's a
  DS> bit clearer.

but a combination of the two still has one nice feature. you can have
the initial delay be X and the interval afterwards be Y. not a deal
breaker and easily done with a wrapper but i like it. :)

  DS> INTVAL Parrot_ping_event(Parrot_Interp Interpreter, INTVAL type);
  >>
  DS> Note that an event of type C<type> has just occurred. Returns either
  DS> 0 on success or 1 on failure. This function may fail if the target
  DS> interpreter is unable to post an event to its event queue. This
  DS> normally happens because there are no event PMCs available to
  DS> allocate for the event.
  >>
  >> clarify. who 'notes'?

  DS> Posts an event of the specified type to the event queue of the target
  DS> interpreter.

  >> maybe call it 'check_event'? how is this different
  >> than post_event?

  DS> No PMC handling. It means that whatever code's doing it doesn't have
  DS> to manage a PMC queue (since if it's in an interrupt handler it can't
  DS> allocate one--they need to be preallocated) and set up all the bits of
  DS> the event PMC. Of limited utility, but for things like signal handlers
  DS> that can't do much anyway and have so little information available to
  DS> them it's sufficient.

ok, so ping is a lightweight post? the spec should say something like
that mixed with your comments.

uri

--
Uri Guttman  ------  u...@stemsystems.com  -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs  ----------------------------  http://jobs.perl.org

  mab> You wrote:
  >> i don't think there is a need for all those variants. why would alarm
  >> need any special opcode when it is just a timer with a delay of abs_time
  >> - NOW? let the coder handle that and lose the extra op codes.

  mab> No, you don't want to do it that way.  Becasue you want to make
  mab> the latency between getting the abs_time, doing the substract and
  mab> actually setting up the time as small, as possible you almost
  mab> have to do this operation as a builtin op.  In fact you can argue
  mab> that you want to lock out async events while doing it as well.

i always assume timers to trigger after a specified delay. accuracy of
delivery (latency) is silly to worry about in perl for granularities of
more than about .05 seconds or so. building a very fine grained accurate
real time system in perl makes little sense to me. so i usually don't
worry about who does the delta calculation and the slight amount of
delay it takes. it will be much smaller than the typical granularity of
the timings anyway.

yes, i can see how an slow async event between the delta calc and the
actual event creation could cause a problem. a way around it is to use a
cronlike thing which has an interval timer and checks for the actual
absolute time and not some precalculated (and possibly way off) delta.

again, i assume event timers are minimum delay devices and i never
assume any sort of decent latency. you get called some time AFTER the
delay is over. so absolute times should be done with a repeat timer set
for the actual granularity you want. and most absolute timers are in 1
minute or more granularites (e.g. cron).

so i would never use the absolute timer in parrot since i couldn't trust
it any more than i can trust a repeated interval timer. but while i can
synchronize interval timers to the real clock, i can't adjust a
miscalculated absolute timer.

uri

--
Uri Guttman  ------  u...@stemsystems.com  -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs  ----------------------------  http://jobs.perl.org

I don't have the time to go through it thorougly, but nethertheless here
are some remarks in no particular order:

1) Before discussing it, the document needs a glossary of all used terms
   There's already enough confusion.
2) I think, the exception related part should just be postponed until
   exceptions are layed out.
3) This "here are some opcodes" part isn't really needed at this stage.
   (not to talk about 12 additional op variants, just to set a timer,
   which is already possible now with *no* additional opcode)
   The ops do and can't in no way cover all possible event's data.
   You started off with "array/hash" combination PMC, which is good,
   then that idea got lost and here's again the opcode explosion.
4) I don't think, that the *internal* event is already a PMC. You can't
   create one in a signal handler (yes the term pre-creation was uttered
   - how many do you pre-create?) PMCs can *only* be created in user
   code. For signals and such, that's at event delivery just before a
   PASM handler get's running.
5) "event level" jumps in at the end in the opcode section. See 1)
6) A lot of Uri's comments apply, e.g. event states ...

Enough for now
leo

I think, *if* we want an event PMC, we can alyways create one, when the
user code indicates that this kind of event should be handled.

  $SIG{CHLD} = sub { 1; };

This could probably create the event PMC, associate the user callback
with it, enable SIGCHLD and be done with it. It's the same as with a
timer event.

*But* the first question is: do we really want PMCs in the inyards of
the event system (except callback sub PMCs)?

There are several issues:
- these PMCs need marking during DOD, which means that we have to mark
  the event queue that might change under us
- event data may cross thread boundaries. This implies that event PMCs
  (at least this kind) have to be shared PMCs (and all their contents)

Finally, if there is something like an "Unnamed Event" that needs
creating a PMC, we can't create the PMC in interrupt code. So the PMC
has to be preallocated. But how many PMCs will you provide? In which
interpreter's arena are these precreated?

For now I think, it's not a good idea to have the internals of an event
as a PMC.

leo

No. Above C<setttimer> definition expands to 8 different functions for
INTVAL arguments only:

  Parrot_settimer_p_i_i
  Parrot_settimer_p_i_i_p_p
  Parrot_settimer_p_i_ic
  Parrot_settimer_p_i_ic_p_p
  Parrot_settimer_p_ic_i
  Parrot_settimer_p_ic_i_p_p
  Parrot_settimer_p_ic_ic
  Parrot_settimer_p_ic_ic_p_p

When you use NULL PMCs for the optional arguments there are still 4.

This is "one" opcode only for one specific event. When we start creating
opcodes for each possible event, we'll have tons of additional opcodes.
This is really not needed.

This is the current and working code to program a timer:

    new P1, .Array
    set P1, 8
    set P1[0], .PARROT_TIMER_NSEC
    set P1[1], 0.2
    set P1[2], .PARROT_TIMER_HANDLER
    find_global P2, "_timer_sub"
    set P1[3], P2
    set P1[4], .PARROT_TIMER_REPEAT
    set P1[5], 2
    set P1[6], .PARROT_TIMER_RUNNING
    set P1[7], 1

    new P0, .Timer, P1

This doesn't use any additional opcodes and is compliant with:
$ perldoc -F docs/pdds/pdd02_vtables.pod
/init_pmc

This scheme is expandable too. The parameters that an event might need
aren't limited and permutations don't hurt.

leo

I've always distinguished these as "solicited" and "unsolicited" events,
assuming I'm understanding you correctly..

Dave.

At 2:59 PM -0400 5/11/04, Gordon Henriksen wrote:

Oh, it's worse than that--GUI commands need to be issued from the
main thread, at least with OS X. (There's no actual requirement as to
which thread handles the actual events as long as you treat the OS
event queue as the thread-unsafe thing it seems to be) Or so the docs
seem to indicate, though they may be a bit conservative.
--
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
d...@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

At 2:47 PM -0400 5/11/04, Gordon Henriksen wrote:

Well, it could work, but it'd take an awful lot of effort and it's
possibly a bad idea. Or... not.

But yes, we may be stuck with two event loops (or a master/slave loop
setup) in some places.

And it does. Or, rather, it can, and it will many places, though not
all of them, unfortunately.

That's not necessarily true, actually, at least not for OS X. (I
can't speak for Win32)

Well... maybe. For many programmers it's dead-on what they're
expecting. If you're writing server or console code, or dealing with
quite a number of the 'cross-platform' GUI APIs, it's exactly what
people expect. For Win32 and OS X programming, it's less so, but in
that case you have to give yourself over to the local event loop.
*Something* has to run the show, and in that case it's pretty clear
that it has to be the OS' event loop.

There is a reasonably simple way around this (And with a bit of
trickery I could do it for OS X) which is to have the OS callbacks do
nothing but post events into Parrot's event queue and leave it to
parrot to then manage the (now local) events and do all the Clever
Things that your average GUI program needs to do. Whether this is
feasible or not is somewhat up in the air, though. (Objective-C makes
this easy. Can't say how it works other places)

I think that very much depends on both the system you're working on
and your concept of events.

I have, but I don't really want to pay the overhead, and since there
are enough shims that'll have to be in place as it is I'm not too
concerned. Once you're past three or four you just end up building a
general-purpose table-driven translator to sort out what's an op,
what's a sub, and what's a method on an object. (Well, OK, you don't
have to, but then things get hellish pretty quickly)

OTOH it's probably about time for me to get over my dislike of
objects and just deal with it. Timers'd be a good place to start, I
suppose.
--
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
d...@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

On May 12, 2004, at 09.12, Dan Sugalski wrote:

That's exactly true, and exactly my point. GUI APIs are definitely not
thread-safe, and for very good reasons. Not only do the system APIs
essentially mandate it, but any depth of thought will make it obvious
that UI events must be handled synchronously. Keeping also in mind that
the main event loop stays on the execution stack, waking up to call
back into event handlers; WaitNextEvent is dead. So funneling all event
delivery through a Big Parrot Queue is unfeasible if your definition of
"event" includes UI events. UI events, with their threading and context
requirements, are really much more an issue of NCI callbacks,
re-entrancy, and wrappers around system types.

That said, the technology you're proposing is still, exactly as it
stands, incredibly useful:

        • Asynchronous completion.
        • Signal handling.
        • Runtime state change notifications, for undoing speculative
optimizations.

It's just not at all suitable for handling GUI events. Which is
perfectly fine: Just either

        1. Don't call it "events" so that people aren't disappointed and
frustrated, or
        2. Figure out some way to fold in GUI events.

—

Gordon Henriksen
IT Manager
ICLUBcentral Inc.
gor...@iclub.com