Select and Sockets
eeze...@my-dejanews.com
descriptor with a value >255. This is also confirmed on the WRS web site.
We are planning on using >1000 sockets, and each socket consumes a file
handle. I believe that this could cause applications (such as ftp) that do
use select() to fail.
Does anyone out there use more than 256 sockets/file descriptors and also use
select()? If so, how did you get around this problem?
The only way I can see of overcoming this problem is to avoid using select()
completely.
Thanks,
Ephraim Ezekiel
eeze...@quarrytech.com
-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/ Search, Read, Discuss, or Start Your Own
Ian Willlats
> We recently found out that the VxWorks select() cannot handle a file
> descriptor with a value >255. This is also confirmed on the WRS web site.
>
This is correct. It depends on the value of FD_SETSIZE (256) which is compiled
into several parts of the operating system, most obviously selectLib.c.
> Does anyone out there use more than 256 sockets/file descriptors and also use
> select()? If so, how did you get around this problem?
>
> The only way I can see of overcoming this problem is to avoid using select()
> completely.
Well, that's certainly not a bad idea if you're not committed to select(). The
VxWorks implementation of select() has some problems, not least of which is
that it will tend to fragment memory if you use it with any frequency.
However, there are some alternatives. Here are a couple I've used, on a
project which needed up to 4096 file descriptors:
1) Use special versions of the relevant OS object files, built for us by WRS
with our specified values of FD_SETSIZE.
2) Obtain source for selectLib.c and modify it to work with an unbounded
FD_SETSIZE - in fact, defined for each call by the <width> argument.
I wouldn't recommend option (1) as it is a nuisance (maintenance-wise) for all
concerned. I doubt that WRS would want to do this anyway.
Option (2) works for us but it does require you to buy the selectLib.c source
code. It's a very small amount of work (less than a day) once you get the
source.
Both these options preserve the standard calling interface to select().
Another alternative I considered was to write a special version of select()
with a different API, not based on the fd_set data structure, which was
better-suited to selecting on a small number of high-valued fds.
One thing that has always concerned me about allowing more than 256 file
descriptors is what happens if one of VxWorks' built-in tasks somehow gets an
fd > 255 and tries to use it with select()? This would in principle cause
problems with option (2), because the other parts of VxWorks apart from
selectLib would still have a fixed FD_SETSIZE of 256, so it would still not be
possible for them to select on fd's outside the range [0,255]. (It would not
be a problem with option (1)). However, in practice this problem does not seem
to arise - presumably because any such fds are opened when the system starts up
and therefore tend to be low-numbered.
I hope this ramble is of some use - by all means contact me directly if I can
offer any more help.
Regards,
Ian
---------------------------------------------------------------
Ian Willats e-mail: mailto:i...@rtp.co.uk
Real-Time Products Ltd. direct: +44 (0) 121 234 6637
Chancery House, tel: +44 (0) 121 234 6600
8 Edward Street, Birmingham. fax: +44 (0) 121 234 6611
B1 2RX. England. web: http://www.rtp.co.uk
---------------------------------------------------------------
Leonid Rosenboim
> We recently found out that the VxWorks select() cannot handle a file
> descriptor with a value >255. This is also confirmed on the WRS web
site.
>
Ephraim, it is been known for a long time that select has its limitation
of handling
a large number of file descriptors, and it has some other problems too.
I recommend that you forget about select() altogether, and the file
descriptor number limitation shall not be the only reason for that...
You can in many cases just have a task block on a socket, thereby having
as many tasks on your system as there are active connections. This does
cost you memory, but is positively the most efficient mechanism to
handle socket traffic, and does not employ any proprietary API.
The other alternative is to use an undocumented "socket wakeup hook"
which can activate your own callback when data arrives at a socket, or
when socket buffer becomes available for write(). Unfortunately, I am
tied by non-disclosure agreements with WRS, thus I can not elaborate on
this.
I have however left a considerable amount of information on this issue
on the WRS support knowledge base, so if you call support they can
easily get you the details.
Leonid
ex-WRS-er
Joe User
How would you handle timeouts in the solution(s?) you
proposed? Are you suggesting that each task that owns
a socket still use select? It seemed to me that you were
infering that select should be avoided.
Is there a way of interrupting read() if a timeout occurs,
or were you suggesting another API other than read()
to block on?
Brian Silver.
Hwa-Jin Bae
> or were you suggesting another API other than read()
> to block on?
A timeout capable I/O API would be very useful in an RTOS. Unfortunately
VxWorks does not provide this. You can achieve similar effect by using
seperate watchdog timer. However cancelling I/O in progress is not
always cleanly handled in the OS, so it is not a perfect solution.
--
Hwa-Jin Bae
PSO Systems, Inc.
http://www.pso.com
mailto:h...@pso.com
John Blackwood
on read or writes, since select works with many file descriptors.
Writing a device driver to support select is simple and well documented
in the VxWorks material.
John Blackwood
Hwa-Jin Bae
> VxWorks "select" supports time-outs. This is more useful than time-outs
> on read or writes, since select works with many file descriptors.
> Writing a device driver to support select is simple and well documented
> in the VxWorks material.
Yes, VxWorks does support timeout parameter in select(). Although it
might be useful for some applications, in some cases it is not as useful
as having timeout capabilities built in for read, write, ioctl, send,
recv, etc. For example, VxWorks does support connectWithTimeout() API,
which was implemented by myself while still working at WRS (10 years ago).
One might argue that it is possible to use select() to implement
connectWithTimeout() and one would be correct -- it is indeed implemented
that way. However, it is far from trivial to get it to do the right
thing, and there is no need to impose application programmers to have
to know the details of what is necessary to connect with timeout correctly
via select().
Another issue with select() is that it is not completely compatible with
the Unix equivalent implementation. Even though this is not a requirement,
some applications may fail due to subtle differences. The function call
restarting is one of the areas of deficiency in select and signal
implementations of VxWorks. This is not to fault VxWorks, as its
system call model makes it hard to implement interrupting and restarting
system calls difficult (they're just function calls!).
If select() is indeed implemented universally it would help, but this
is not the case either. And as others have pointed out, sometimes it is
best to dedicate a thread(task) for a given I/O channel. VxWorks is
designed to be used this way. The thread switching overhead is deliberately
minimized that threads can be used liberally this way (according to the
original designer). The select mechanism was added as a competibility
solution (for Unix program porting and programmer convenience), not
as a mechanism for timing out I/O or multi-plexing over I/O channels.
Even with select() functioning properly 100% (which is not the case in
VxWorks version -- for example, exceptions are not handled fully), it is
one thing to use read() on non-blocking file descriptor with select,
it is another thing to have a true asynchronous I/O that can be timed
out properly. The former approach requires putting the I/O channel
in non-blocking mode which has implications in terms of the behavior
of the I/O depending on what type of device or facility you are going over.
The latter approach will be ideally implemented as integral mechanism
in the I/O core system and all I/O will be generically performed so that
each can be restarted or timed out without causing any anomalies.
eeze...@quarrytech.com
support file descriptors whose value is greater than 255. Therefore it
was suggested a separate task be created for reading on each socket.
Since it is the read that is blocking and not the select, then the original
question asked was if there is a way to timeout the read.
- Ephraim
In article <3708FFBD...@adc.com>,
John Blackwood <john_bl...@adc.com> wrote:
> Hwa-Jin Bae wrote:
> >
> > > Is there a way of interrupting read() if a timeout occurs,
> > > or were you suggesting another API other than read()
> > > to block on?
> >
> > A timeout capable I/O API would be very useful in an RTOS. Unfortunately
> > VxWorks does not provide this. You can achieve similar effect by using
> > seperate watchdog timer. However cancelling I/O in progress is not
> > always cleanly handled in the OS, so it is not a perfect solution.
> >
> > --
> > Hwa-Jin Bae
> > PSO Systems, Inc.
> > http://www.pso.com
> > mailto:h...@pso.com
> VxWorks "select" supports time-outs. This is more useful than time-outs
> on read or writes, since select works with many file descriptors.
> Writing a device driver to support select is simple and well documented
> in the VxWorks material.
>
> John Blackwood
Phil Ashby
> How would you handle timeouts in the solution(s?) you
> proposed? Are you suggesting that each task that owns
> a socket still use select? It seemed to me that you were
> infering that select should be avoided.
>
> or were you suggesting another API other than read()
> to block on?
We generally close() the errant socket descriptor from a timeout
mechanism (in this case a timer task), which has so far proved very
successful in causing read() to return an error, whereupon the blocked
task can clean up / terminate gracefully.
--
!--------------------------- Phil "Phlash" Ashby ---------------------!
! BT Wireplay - Lead Programmer and general technical trivia supplier !
! Phone: 01473-644348 WWW: http://www.wireplay.com/ !
! Snail: B81/G41, BT Labs, Martlesham Heath, Ipswich IP5 3RE, England.!
di...@crd.ge.com
I have used this technique in the past with no
observered issues. I had set SO_REUSEADDR and
also the no linger option. When the timer expired,
I was really prepared to loose information in transit.
TIA
standard disclaimers apply
In article <7eaoe7$73j$1...@news.idiom.com>,
Hwa-Jin Bae <ch...@idiom.com> wrote:
> > Is there a way of interrupting read() if a timeout occurs,
> > or were you suggesting another API other than read()
> > to block on?
>
> A timeout capable I/O API would be very useful in an RTOS. Unfortunately
> VxWorks does not provide this. You can achieve similar effect by using
> seperate watchdog timer. However cancelling I/O in progress is not
> always cleanly handled in the OS, so it is not a perfect solution.
>
> --
> Hwa-Jin Bae
> PSO Systems, Inc.
> http://www.pso.com
> mailto:h...@pso.com
>
-----------== Posted via Deja News, The Discussion Network ==----------