problem with select?
Denis Perelyubskiy
we suspect that we are seeing a problem with select() in
5.3.1, whereby select() returns saying there is data in
socket(), then we do a read. Read blocks. This does not
happen every time, but we saw it once or twice.
Hence, I am wondering if anyone else has seen the same
issue.
thanks,
denis
--
if you want to reply by mail, please tweak my
address in a reasonable way. sorry for the
inconvenience.
David Laight
> Hello,
>
> we suspect that we are seeing a problem with select() in
> 5.3.1, whereby select() returns saying there is data in
> socket(), then we do a read. Read blocks. This does not
> happen every time, but we saw it once or twice.
>
> Hence, I am wondering if anyone else has seen the same
> issue.
I recall adding FIONREAD ioctl requests to a program because
select wasn't behaving itself.
Didn't ever track down what was really happening though :-)
David
Jerry Marcinko
Support for select() in VxWorks is driver specific (i.e. requires driver to
implement select sleep and wakeup primitives), so a bug in the underlying
driver could easily cause this. So the problem may no be VxWorks or the
selLib, but in the driver.
"Denis Perelyubskiy" <denis_newsRE...@hotmail.com> wrote in
message
news:slrnaaskch.ct6.denis...@concorde.cs.ucla.edu...
Hwa Jin Bae
available to read. At that point, you do not know how much data is
available to be read. You should do FIONREAD to make sure that you have
sufficient data for your read request that follows. Otherwise, read can
block.
--
Hwa Jin Bae
mailto:h...@pso.com
http://www.pso.com
Vijay Peshkar
read() blocks if you have not used FIONBIO
in ioctl() call on the descriptor.
Regards,
Vijay
David Laight <da...@spamm.me.l8s.co.uk> wrote in message news:<3CAEEB9D...@spamm.me.l8s.co.uk>...
Pete Flugstad
> Assuming you are selecting for read events, select will return when data is
> available to read. At that point, you do not know how much data is
> available to be read. You should do FIONREAD to make sure that you have
> sufficient data for your read request that follows. Otherwise, read can
> block.
FYI, this is not how normal BSD style sockets work (and
from my experience, vxWorks does not follow this model).
If select returns true for a socket descriptor, you can do read
on that socket, and the data AVAILABLE will be returned to you,
from 1 byte, up to the max you requested. The read should not
block no matter how much you request, but you will only get what
is available.
You do NOT need to do a FIONREAD. All that's going to do is add
an extra system call/ioctl overhead to your program.
From my experience (mostly SNMP agents), vxWorks implements this
model exactly as I've desribed.
As someone else pointed out, if the NIC driver doesn't properly
implment various primatives, then select can indicate data available,
but read will block. This is a bug in the NIC driver. I haven't
seen this personally.
Pete Flugstad
Icon Labs
Hwa Jin Bae
This is how both BSD and VxWorks sockets work. Let's look at it more
closely.
First, it is true that doing a read on a socket can return data of length
smaller than requested length. However, that case, as frequent as it may
be, is not the only case.
For example, it is possible for select to wake up on socket for read side
when there is nothing to read. Not often, but it can happen. When this
happens read will block, unless it is marked FIONBIO.
The device specific select and potential bugs related to device specific
driver is not related to selecting on sockets. The select on sockets is
implemented entired within socket layer and protocol layer code. It is not
coupled with any other devices.
Calling FIONREAD is not always extra overhead either. It can save repeated
calls to read and can simplify code.
Please let me know if any of the above is incorrect.
Joe Chung
something to the effect of:
...
Three independent sets of descriptors are watched. Those
listed in readfds will be watched to see if characters
become available for reading (more precisely, to see if a
read will not block - in particular, a file descriptor is
also ready on end-of-file), those in writefds will be
watched to see if a write will not block, and those in
exceptfds will be watched for exceptions. On exit, the
sets are modified in place to indicate which descriptors
actually changed status.
...
I've also written code that depended on that behavior with no
problems, i.e. in the sequence:
err = select(...);
if (err == -1) {
/* do error or EINTR handling */
} else if (err == 0) {
/* handle timeout situation */
} else {
read(...);
}
The read has never blocked.
-jc
--
(apply 'concat (reverse (list "com"
(char-to-string 46) "yahoo"
(char-to-string 64) "joechung")))
Hwa Jin Bae
never blocking after select returns with read selection. That seems to be
the case right?
However, that is not entirely accurate. The code as you posted *can* block.
For example, select will return with read selection when there is zero byte
to read, if certain uncommon events happen from the network.
Let me give you one example, if your network interface is marked down and
back up again, very quickly (just if_down and if_up, back to back), you will
see that your TCP socket select will come back with read selection. There
won't be anything to read though.
That is not the only case. But it illustrates the point I am trying to
make.
Joe Chung
> I can believe that your code never blocked. It argues strongly for read
> never blocking after select returns with read selection. That seems to be
> the case right?
Right.
> However, that is not entirely accurate. The code as you posted
> *can* block.
>
> For example, select will return with read selection when there
> is zero byte to read, if certain uncommon events happen from
> the network.
Read can and does return 0 bytes (as an indication of EOF).
> Let me give you one example, if your network interface is
> marked down and back up again, very quickly (just if_down and
> if_up, back to back), you will see that your TCP socket select
> will come back with read selection. There won't be anything to
> read though.
>
> That is not the only case. But it illustrates the point I am
> trying to make.
In which case, the implementation of select() is in error or
(more likely) the implementation of the driver in support of
select() is in error.
The whole point of select+read is to avoid having to use
nonblocking I/O. With nonblocking I/O, using select as a
predicate for read becomes almost redundant, except in certain
cases.
Now, I won't argue whether select is indeed implemented correctly
on VxWorks or not. :) Furthermore, if it's your contention that
select+read can block due to incorrect driver implementations,
then I completely agree with you.
If one does decide to use select+nonblocking I/O, note that this
combination seem to have different behaviors (specifically wrt
edge cases) on different systems, in case you ever have to port
your code...
Hwa Jin Bae
Read will not return 0 bytes because it is not EOF. It will block until
bytes become available. If you doubt this, please read BSD Unix code. Just
read soreceive() in uipc_socket. The example I explained last time is a
case where the lower level network stack code tries to inform upper layer
code that certain events of interest has happened and thus select returns to
give the users a chance to do something to find out. Not necessary read().
But something else. Read BSD code to see how the protocols (TCP, UDP, etc.)
are implemented to allow control inputs like this (e.g. tcp_ctlinput). The
intent is to allow select (or the act of polling) to return to give users a
chance.
In those cases, read() will block because the code at socket layer will see
that there is nothing to read in the socket level queue. It will do sbwait.
Then wait until something becomes available (even 1 byte).
The select implement is not incorrect. It is in accordance with the overall
design.
The whole point of select+read is not only to avoid having to use
nonblocking I/O. The other main purpose is to have API to poll over
multiple channels of I/O and other events. It is not designed so that you
can do a read for all occasions where select returns on read selection. It
is designed so that you will do necessary things to find out what to do
after return. One of the necessary things may be read. Other necessary
things can be looking at various things to see what happened.
It is not my contention that select+read can block due to incorrect driver
implementation. That is not my contention at all. In fact, I thought I
mentioned that it is incorrect to link the select on sockets with select on
devices. The select on sockets is implemented within networking code.
Using the same mechansim as the way select is implemented for devices. But
issues related to select on socket has nothing to do with individual device
driver select support. Furthermore, there is no select primitives for
network device drivers. They are for generic I/O drivers. The confusion is
understandable, but there is a fine point here that needs to be considered.
Joe Chung
>
> Read will not return 0 bytes because it is not EOF. It will
> block until bytes become available. If you doubt this, please
> read BSD Unix code. Just read soreceive() in uipc_socket. The
> example I explained last time is a case where the lower level
> network stack code tries to inform upper layer code that
> certain events of interest has happened and thus select returns
> to give the users a chance to do something to find out. Not
> necessary read().
Yes, I agree with all that, and I'm not saying that select + read
will ALWAYS not block. I'm saying:
After select, when you have tested conditions for
timeout/error, then if you test (via FD_ISSET) to see if indeed
the FD that you are interested in is available for reading or
writing, and then you proceed to read or write, then it should
NOT block.
Select should not signal an fd as readable when it is clearly
not. That would be an incorrect implementation of select.
There other are ways that you can block on read after a select
though, for example, if you have multiple threads waiting on the
same FD and both are awakened by select and one of them steals
the input of the other. But that's an easily solvable problem.
> But something else. Read BSD code to see how the protocols
> (TCP, UDP, etc.) are implemented to allow control inputs like
> this (e.g. tcp_ctlinput). The intent is to allow select (or
> the act of polling) to return to give users a chance.
>
> In those cases, read() will block because the code at socket
> layer will see that there is nothing to read in the socket
> level queue. It will do sbwait. Then wait until something
> becomes available (even 1 byte).
Having the lower TCP layers inform the upper layers of some
"interesting" event is a good thing. I don't think I'm talking
about this.
If you return from select, and something in readFDS, writeFDS,
exceptFDS (VxWorks doesn't implement this last one) indicates
"readiness" then operation on it should not block. That
practically defines select().
> The select implement is not incorrect. It is in accordance
> with the overall design.
Then you're saying that select (properly implemented) can
1. Return no error, and
2. Properly set one or more FD bits (testable via FD_ISSET), and
3. Cause you to block when you read/write on the FD associated
with that bit that select indicated is ready for read/write?
[ Rest snipped for brevity sake ]
Hwa Jin Bae
news:rps7knh...@nospam.invalid...
> Yes, I agree with all that, and I'm not saying that select + read
> will ALWAYS not block.
Good. Then we agree on that.
> I'm saying:
> After select, when you have tested conditions for
> timeout/error, then if you test (via FD_ISSET) to see if indeed
> the FD that you are interested in is available for reading or
> writing, and then you proceed to read or write, then it should
> NOT block.
Always? Sigh... then we disagree.
>
> Select should not signal an fd as readable when it is clearly
> not. That would be an incorrect implementation of select.
>
That would be an incorrect implementation according to what? Who?
Please read BSD Unix original implementation of select support for sockets
in the kernel. The same things are done in both BSD and VxWorks. What you
call incorrect is in fact what BSD does. Please go read the relevant
portions of the kernel code, and do some experiments. You will see that
various events going from low half of protocol code to high half of protocol
code can indeed trigger situations where read selection gets satisfied when
there is 0 byte to read and it is not EOF condition. When that happens, as
you can read form soreceive code in the kernel, it will block.
> If you return from select, and something in readFDS, writeFDS,
> exceptFDS (VxWorks doesn't implement this last one) indicates
> "readiness" then operation on it should not block. That
> practically defines select().
>
That seems to be merely your definition. That clearly is not what the BSD
kernel code or VxWorks kernel code does. BSD code is available online,
please read and tell me why you think your assertion here is true. It is
not.
> Then you're saying that select (properly implemented) can
> 1. Return no error, and
> 2. Properly set one or more FD bits (testable via FD_ISSET), and
> 3. Cause you to block when you read/write on the FD associated
> with that bit that select indicated is ready for read/write?
Yes, as incredible as it might be, that is how the code is written. Again,
I urge you to read the code. It is available on gatekeeper
(ftp.gatekeeper.com). Look for BSD directory. Or just go to netbsd.org or
freebsd.org web pages.
I am *not* making this up. :-)
Hwa Jin Bae
> (apply 'concat (reverse (list "com"
> (char-to-string 46) "yahoo"
> (char-to-string 64) "joechung")))
{char*b="u...@cfb.pbz";int
a,i;for(i=0,a=~*b++;i<11;i++,a=~*b++)putchar(~a-1/(~(a|32)/13*2-11)*13);}
Joe Chung
> "Joe Chung" <nos...@nospam.invalid> wrote in message
> > I'm saying:
> > After select, when you have tested conditions for
> > timeout/error, then if you test (via FD_ISSET) to see if indeed
> > the FD that you are interested in is available for reading or
> > writing, and then you proceed to read or write, then it should
> > NOT block.
>
> Always? Sigh... then we disagree.
It's ok. We're just disagreeing. :)
> >
> > Select should not signal an fd as readable when it is clearly
> > not. That would be an incorrect implementation of select.
> >
>
> That would be an incorrect implementation according to what? Who?
According to select's own man pages (in the systems that I've
read them), VxWorks, Linux, Solaris, Hpux:
1. They make no mention of select breaking out under the
condition that you spoke of, and
2. Some of them (Linux and Hpux) makes the stronger assertion
that when select() returns no errors and sets the corresponding
FD bit, then read/write operations on that FD will not block.
> Please read BSD Unix original implementation of select support
> for sockets in the kernel. The same things are done in both
> BSD and VxWorks. What you call incorrect is in fact what BSD
> does. Please go read the relevant portions of the kernel code,
> and do some experiments. You will see that various events
> going from low half of protocol code to high half of protocol
> code can indeed trigger situations where read selection gets
> satisfied when there is 0 byte to read and it is not EOF
> condition. When that happens, as you can read form soreceive
> code in the kernel, it will block.
So what you're saying is that I can't trust the multiply
documented (they wrote the doc) external interface (they wrote
the code), but have to resort to code-reading below the contract
level.
Now, I'm not so naive as to believe that all docs and
implementations match, but both this doc and this API have been
around for a long time, and both are now in such wide use....
> > If you return from select, and something in readFDS,
> > writeFDS, exceptFDS (VxWorks doesn't implement this last one)
> > indicates "readiness" then operation on it should not block.
> > That practically defines select().
> >
>
> That seems to be merely your definition. That clearly is not
> what the BSD kernel code or VxWorks kernel code does. BSD code
> is available online, please read and tell me why you think your
> assertion here is true. It is not.
It's not my definition. It's in the man pages, the relevent bit
of which I quoted in my first reply to this thread here. I wish
life were so convenient that I can just make it up. :)
I only have their man pages to back me up on this. You have the
source code. If the implementation doesn't agree with the
contract, then one or the other needs changing, and it's usually
the implementation, unless this is the first couple of drafts of
the contract.
I haven't seen this behavior in the program with selects that I
do, but maybe that's because I don't toggle the interface down
and up or change its MTU in mid-operation...
But I'll take your word for it, and test it to see if I can make
it happen. If the experiment that I described below is not
correct, I'd appreciate another easy to reproduce example:
So in one of your previous post, you say that I can demonstrate
this behavior in VxWorks by doing the following
1. Open a tcp/ip socket for reading
2. wait on it with select
3. In another task, bring the interface down and then up again
4. At this point the select comes back to the waiting task with
an non-error condition and the FD is set
5. If I read the FD, I would block right?
I can easily test that.
> > Then you're saying that select (properly implemented) can
> > 1. Return no error, and
> > 2. Properly set one or more FD bits (testable via FD_ISSET), and
> > 3. Cause you to block when you read/write on the FD associated
> > with that bit that select indicated is ready for read/write?
>
> Yes, as incredible as it might be, that is how the code is
> written. Again, I urge you to read the code. It is available
> on gatekeeper (ftp.gatekeeper.com). Look for BSD directory.
> Or just go to netbsd.org or freebsd.org web pages.
Thanks for the pointers. I've looked at the BSD (as well as
other) source before, when anomalous situations arose. :)
> I am *not* making this up. :-)
Nor am I accusing you of that. It's just, I'm not making it up,
either.
So I do have another question:
As a good programmer, how can one detect and possibly respond to
this siutation, i.e. what error conditions can I check for that
will tell me this condition has happened?
Joe Chung
Damn it, that expression won't eval on my Emacs...
oh wait.... c-c c-r.. cute :)
-jc
--
Hwa Jin Bae
I think the manual entries on BSD, Solaris, HP/UX, and Linux on select(2)
are all rather terse. None of them explicitly say anything about whether
subsequent read() should ever block or not. You can assume and read
between the lines, but that would be just that.
To me, the original select implementation in BSD Unix is what defines the
original definition of select. And that is what I described. Warts and
all.
Enough is said on the subject, the verification (i.e. reading of available
Unix kernel code and understanding all of it) is left as an excercise to be
done the reader.
Thanks for playing.
Jim Thomas
Pete> Hwa Jin Bae wrote:
>> Assuming you are selecting for read events, select will return when data is
>> available to read. At that point, you do not know how much data is
>> available to be read. You should do FIONREAD to make sure that you have
>> sufficient data for your read request that follows. Otherwise, read can
>> block.
Pete> FYI, this is not how normal BSD style sockets work (and
Pete> from my experience, vxWorks does not follow this model).
Pete> If select returns true for a socket descriptor, you can do read
Pete> on that socket, and the data AVAILABLE will be returned to you,
Pete> from 1 byte, up to the max you requested. The read should not
Pete> block no matter how much you request, but you will only get what
Pete> is available.
Pete> You do NOT need to do a FIONREAD. All that's going to do is add
Pete> an extra system call/ioctl overhead to your program.
Pete> From my experience (mostly SNMP agents), vxWorks implements this
Pete> model exactly as I've desribed.
On a slight topic drift :-)
IME not quite. I'm running a possibly broken 5.4.2 on a PPC (mv2700). I
am trying to use select (the same as my previous comments about select
returning in less than one tick) on a single, input socket channel. I have
the FIONREAD in there so I can see how much is there. My typical case
returns 17 from FIONREAD , but the following read() will only return 1
byte. Has anyone seen that? (I've tried it with 17 and 540 as the number
of bytes to read. Both return 1 byte.)
Mahalo,
Jim
Denis Perelyubskiy
interesting/useful discussion...
* Denis Perelyubskiy <denis_newsRE...@hotmail.com> [04-Fri-02 17:38 -0800]:
