WaitHandles must be less than or equal to 64 - missing documentation

[Loy]

Calling WaitAll with array of 64 or more causes exception
WaitHandle.WaitAll(whaWaitHandlesArray);

System.NotSupportedException: The number of WaitHandles must be less
than or equal to 64.
at System.Threading.WaitHandle.WaitAll(WaitHandle[] waitHandles,
Int32 millisecondsTimeout, Boolean exitContext)
at System.Threading.WaitHandle.WaitAll(WaitHandle[] waitHandles)

Searching for the error string revealed no KB article or MSDN
documentation.
I did found MSFT answer regarding that on:
http://groups.google.com/group/microsoft.public.sqlserver.notificationsvcs/tree/browse_frm/thread/1e10fc93cf194775/1d8e7ee0e1a3f41c

The API documentation says it might throw NotSupportedException under
the condition: "The number of objects in waitHandles is greater than
the system permits."

My questions are:
1. How can I find the system limitation? what does it depends on?
2. Any kb article that will provide link from the exception message to
the real cause will be helpful for all the developers and will save us
google time

(About Google Vs www.Live.com - Live did not find even a single match
...)

Loy

[Peter Ritchie [C# MVP]]

In .NET 2.0 it's simply hard coded to 64. This is based upon the limit of
object handles that the Win32 function WaitForMultipleObjectsEx can
support--which is documented as being MAXIMUM_WAIT_OBJECTS, or 64.

If you want to wait for more than 64 handles you'll have to break it into
chunks of 64 and create a thread for each <=64 chunk that sets another
WaitHandle that is waited upon in some "main" thread (can't be the main
thread in a WinForms application)--which would give you the ability to wait
on 64 blocks of 64 wait handles. More than that and you'd have to break out
to 3 levels. I think there's an example or two on the web of handling more
than 64 handles...

--
Browse http://connect.microsoft.com/VisualStudio/feedback/ and vote.
http://www.peterRitchie.com/blog/
Microsoft MVP, Visual Developer - Visual C#

"Loy" wrote:

> ....)
>
> Loy
> >

[William Stacey [C# MVP]]

Fortunately, WaitAll is easy to implement with any number of handles. Here
is a simple example.

public static void WaitAll(WaitHandle[] handles)
{
if (handles == null)
throw new ArgumentNullException("handles");
foreach (WaitHandle wh in handles)
{
wh.WaitOne();
}
}

--
William Stacey [C# MVP]

"Loy" <l.oy...@gmail.com> wrote in message
news:1166955143....@f1g2000cwa.googlegroups.com...

[Ben Voigt]

"Loy" <l.oy...@gmail.com> wrote in message
news:1166955143....@f1g2000cwa.googlegroups.com...

> Calling WaitAll with array of 64 or more causes exception
> WaitHandle.WaitAll(whaWaitHandlesArray);
>
> System.NotSupportedException: The number of WaitHandles must be less
> than or equal to 64.
> at System.Threading.WaitHandle.WaitAll(WaitHandle[] waitHandles,
> Int32 millisecondsTimeout, Boolean exitContext)
> at System.Threading.WaitHandle.WaitAll(WaitHandle[] waitHandles)

At this point you should re-architect your application to use fewer wait
handles. What kind of object are you waiting on and why do you need so
many?

[Ben Voigt]

"William Stacey [C# MVP]" <william...@gmail.com> wrote in message
news:ulAX0G9J...@TK2MSFTNGP06.phx.gbl...

> Fortunately, WaitAll is easy to implement with any number of handles.
> Here
> is a simple example.

Which does not replicate the proper semantics and therefore is vulnerable to
deadlock.

[William Stacey [C# MVP]]

How so?

--
William Stacey [C# MVP]

"Ben Voigt" <r...@nospam.nospam> wrote in message
news:Os7SKySK...@TK2MSFTNGP02.phx.gbl...

[Peter Ritchie [C# MVP]]

My calling WaitOne in a loop like that you're allowing the thread to gain
ownership of a WaitHandle without having ownership of all WaitHandles.
WaitAll does not give ownership of all WaitHandes until all WaitHandles have
entered a signaled state.

WaitAll can be given a timeout, if that timeout expires before all
WaitHandles are signaled the thread never gains ownership of any of the
WaitHandles even if some of them have signaled.

BTW, you'd have this same problem if had to split up more thane 64
WaitHandles across multiple threads with multiple calls to WaitAll...

--
Browse http://connect.microsoft.com/VisualStudio/feedback/ and vote.
http://www.peterRitchie.com/blog/
Microsoft MVP, Visual Developer - Visual C#

[William Stacey [C# MVP]]

This can get slightly more involved and depends on the handle type. If the
wait events are ManualResetEvents, then a timeout may not matter to your
code as they stay signaled anyway. AutoResetEvents are more an issue as
some may be reset by the WaitOne() call before the timeout and that could
get messy - but can still be workable depending on needs. Semaphores,
however, do change their state even with the win32 waitformultipleobjects as
it waits for each one in turn (so the net effect would be the same as using
the shown implementation). If one understands the potential issues and
takes care, the pattern can prove useful when waiting on many handles.

--
William Stacey [C# MVP]

"Peter Ritchie [C# MVP]" <PRS...@newsgroups.nospam> wrote in message
news:DA246A3B-4746-47EF...@microsoft.com...

[Loy]

Thank you all for your reply

I implemented the simple workaround with the limitation that it works
only if the handles are not reset again during the wait time.

In my case this is a valid limitation as I wait for objects to close
and the handle is nullified in the objects once it is set.

Being aware of this issue - if there is a correct implementation - why
doesn't the dot net team implement it as part of the framework?

On the documentation side - Need to be clear on how to know your system
limitation
A kb article is the least that I would expect with regard to this
issue.

Am I wrong?

Thanks

Loy

[Peter Ritchie [C# MVP]]

The event will still be signaled; but if the WaitAll times out, that thread
won't get ownership of the handle--in which case another thread would get
ownership. It would appear to the application that one thread was signaled;
but in reality another thread had a chance of getting the signal and didn't.

Looping through a collection of handles using WaitOne means the thread does
get ownership of that event and just ignores it, if all the handles are not
signaled. If some thread is depending on some process occurring because of
the event becoming signaled then you'd result in a deadlock using a series of
WaitOne calls.

For example: one thread starts a WaitAll for an event and a mutex at
time-frame x, another thread simply makes a call to WaitOne for that same
event at time-frame x+n; if the event signals and the mutex doesn't, the
thread making the WaitOne would get ownership of that event signalling,
despite not being the first to start waiting.

...but, if you don't have that situation (you only have on thread waiting)
then it probably doesn't matter.

[William Stacey [C# MVP]]

| The event will still be signaled; but if the WaitAll times out, that
thread
| won't get ownership of the handle--in which case another thread would get
| ownership. It would appear to the application that one thread was
signaled;
| but in reality another thread had a chance of getting the signal and
didn't.
|
| Looping through a collection of handles using WaitOne means the thread
does
| get ownership of that event and just ignores it, if all the handles are
not
| signaled. If some thread is depending on some process occurring because
of
| the event becoming signaled then you'd result in a deadlock using a series
of
| WaitOne calls.

Assuming we have been talking about events, I don't understand as events do
not have owners.

[Peter Ritchie [C# MVP]]

When the system transitions an event from non-signaled to signaled it looks
for a waiting thread waiting for this event to become signaled. It looks for
a waiting thread to give it ownership of the event (which really only makes
sense with an auto-reset event) giving priority to threads that have been
waiting the longest. If a thread is waiting for the event, but cannot be
taken out of a wait state (it has called WaitAll and the other WaitHandle(s)
has not yet signalled) it does not give ownership of the event to that thread
and looks for another. If there is another thread that can be taken out of a
wait state, that thread gets ownership and is therefore signaled. Replacing
WaitAll with a series of WaitOne means the thread now has ownership of that
signal but may be put back into a wait state on another WaitHandle that may
never be signaled. Another thread waiting for the event will never receive
the signal and may result in a deadlock.

I'd have to check but, in the case of a ManualResetEvent and WaitAll, I
would expect the thread that called WaitAll will remain blocked if another
WaitHandle did not signal while another thread waiting on the same
manual-reset came out of a wait state and called the manual-reset event's
Reset method...

--
Browse http://connect.microsoft.com/VisualStudio/feedback/ and vote.
http://www.peterRitchie.com/blog/
Microsoft MVP, Visual Developer - Visual C#

"William Stacey [C# MVP]" wrote:

[William Stacey [C# MVP]]

I guess I am just uneasy with the use of "ownership" here. Which means
something very specific in terms of objects like mutex or monitor which have
explicit thread owners. Events are not owned.

If we do some pseudo code, it may look something like this:

void HandleSignaled()
{
lock(globalKernelWaitLock)
{
foreach(WGroup wg in waiters)
{
if ( wg.All )
{
bool gotAll = true;
foreach(WaitHandle wh in wg.Handles)
if ( ! wh.IsSet ) { gotAll = false; break; }
if ( ! gotAll )
continue;

ResetHandles(wg); // Behavior is handle type specific.
ReleaseWaiter(wg); // Release the thread waiting on WGroup.
return;
}
else { // handle waitany. }
}
}
}

Most of the majic happens inside the global kernel lock. While inside the
lock, no other waithandle can Set (i.e. is blocked). So if we see all the
signals, the kernel does any atomic state updates (based on the handle type)
and releases our waiting thread. After the unlock, any other thread could
Set or Reset, but we have the gaureentee that at a single moment in time,
all handles where in a signaled state. If another thread did a reset of one
of the manual events just after we returned from WaitAll, then that is an
app issue (could be normal or an app bug). If we don't get all handles in
the loop, other threads are free to do Sets and Resets, so it is possible we
could never satisfy our atomic WaitAll (another reason to always use a
timeout). I understand Vista has made some big changes in kernel sync
objects and fairness, etc, it should be interesting to learn more detail on
them. Interesting thread.

[Ben Voigt]

"William Stacey [C# MVP]" <william...@gmail.com> wrote in message

news:uVcWr8UK...@TK2MSFTNGP02.phx.gbl...
> How so?
>

I think Peter has done a fantastic job of explaining why this is so. If you
still aren't convinced there is a significant difference between:

WaitForMultipleObjects({A, B, C, D});
and
WaitForSingleObject(A);WaitForSingle(B);WaitForSingle(C);WaitForSingle(D);

then I will be happy to clarify.

[William Stacey [C# MVP]]

| I think Peter has done a fantastic job of explaining why this is so.

Totally agree.

| still aren't convinced there is a significant difference between:
|
| WaitForMultipleObjects({A, B, C, D});
| and
| WaitForSingleObject(A);WaitForSingle(B);WaitForSingle(C);WaitForSingle(D);
|
| then I will be happy to clarify.

Thanks Ben. Not sure there is any confusion on that. I think we/I
meandered into some different areas. All is good. Happy New Year.