Re: [wx-discuss] wxSocket rewrite using Boost.Asio
Stefan Csomor
On 23.05.08 03:06, "William H. Schultz" <whsc...@cedrus.com> wrote:
> I'm about to undertake a major rewrite of my socket code, and I'm
> leaning toward reimplementing wxSocket to use Boost.Asio (vs.
> replacing wxSocket in our code with Boost.Asio). Regardless of
> whether the code goes into the official wxWidgets, I have to do
> something. I just thought I'd open this up for discussion here to see
> what you guys think.
>
> For those of you not familiar, the documentation is here:
> http://asio.sourceforge.net/boost_asio_1_0_0/libs/asio/doc/html/index.html
> As of Boost version 1.35.0, Asio is part of the library.
>
> My biggest concern is that in a ground-up rewrite, I may not be able
> to make it 100% backwards compatible. But considering how much work I
> had to put into getting my code to work with the existing bugs and
> weaknesses in wxSocket, I'm not certain that this would be a big
> deal. Furthermore, this would make wxWidgets dependent on the boost
> libraries, but I'm not certain that's such a bad thing, as boost has a
> lot to offer.
that's 100 % on the line of what I think the future of wx should be,
leverage what's there and proven, and if we really want to make things
better, backwards compatibility shouldn't be the reason for not doing it ..
Best,
Stefan
Vadim Zeitlin
WHS> I'm about to undertake a major rewrite of my socket code, and I'm
WHS> leaning toward reimplementing wxSocket to use Boost.Asio (vs.
WHS> replacing wxSocket in our code with Boost.Asio).
Replacing wxSocket with a 3rd party library is definitely very, very
tempting, I'm sure that ASIO is much better than the existing code. The
problem is whether we're going to be able to preserve the existing API
(even if not 100%, but at least 90%).
However I imagine that not everybody would be happy with requiring Boost
to build wx and, worse, ASIO doesn't seem to support VC6 which we're
apparently not ready to abandon yet. So while I personally would like to do
what you propose provided backwards compatibility could be more or less
preserved, I'm afraid VC6 is going to be the sticking point :-(
Regards,
VZ
William H. Schultz
whether the code goes into the official wxWidgets, I have to do
something. I just thought I'd open this up for discussion here to see
what you guys think.
For those of you not familiar, the documentation is here: http://asio.sourceforge.net/boost_asio_1_0_0/libs/asio/doc/html/index.html
As of Boost version 1.35.0, Asio is part of the library.
My biggest concern is that in a ground-up rewrite, I may not be able
to make it 100% backwards compatible. But considering how much work I
had to put into getting my code to work with the existing bugs and
weaknesses in wxSocket, I'm not certain that this would be a big
deal. Furthermore, this would make wxWidgets dependent on the boost
libraries, but I'm not certain that's such a bad thing, as boost has a
lot to offer.
What do you all think?
-------------------------------
Hank Schultz
Cedrus Corporation
http://www.cedrus.com/
Kevin Ollivier
I'm beginning to think the slogan for wx should be "let's program like
it's 1999!" ;-) The rest of the world is moving on (in fact, VC6
hasn't been available for purchase for half a decade), and if wx
doesn't move on too, users will leave it behind just like they did VC6.
For users who refuse to change and update, there are always the
existing and old versions of wx. And that's how any development
process should work, because the toolkit today should focus on the
needs of the users today (and, more importantly, tomorrow), not focus
on keeping apps written 10 years ago from breaking. If a particular
app developer doesn't want to change and update their code, or can't
afford to over the course of several years, they don't have to change
their wx version - the old one will run as it always did.
+1 to Stefan's ideas.
Regards,
Kevin
>
> Regards,
> VZ
>
> _______________________________________________
> wx-discuss mailing list
> wx-di...@lists.wxwidgets.org
> http://lists.wxwidgets.org/mailman/listinfo/wx-discuss
William H. Schultz
> Hi Vadim and all,
>
> On May 23, 2008, at 5:30 AM, Vadim Zeitlin wrote:
>
>> On Thu, 22 May 2008 18:06:49 -0700 "William H. Schultz" <whsc...@cedrus.com
>> > wrote:
>>
>> WHS> I'm about to undertake a major rewrite of my socket code, and
>> I'm
>> WHS> leaning toward reimplementing wxSocket to use Boost.Asio (vs.
>> WHS> replacing wxSocket in our code with Boost.Asio).
>>
>> Replacing wxSocket with a 3rd party library is definitely very, very
>> tempting, I'm sure that ASIO is much better than the existing code.
>> The
>> problem is whether we're going to be able to preserve the existing
>> API
>> (even if not 100%, but at least 90%).
I'm not convinced that the existing implementation necessarily
preserves 90% of the API. Last I checked, IPv6 wasn't available. Of
course, my memory could be failing me on this detail. A big problem
I've had is that "blocking" operations don't *block*. They don't
return, but they don't block either--they chew up 100% of the CPU.
Anyway, I'm looking through the existing documented API to see if
there's anything that seems like it may not be applicable (or
implementable). I'm still making sense of the Asio API, but I'm only
seeing one sticking point:
http://www.boost.org/doc/libs/1_35_0/doc/html/boost_asio/design/threads.html
http://www.boost.org/doc/libs/1_35_0/doc/html/boost_asio/reference/io_service.html
My take on this is that an implementation using Asio would require one
of two things: 1) call io_service::poll in the application's idle
time, or 2) spawn a separate thread to call io_service::run. I would
prefer the second approach, but I'm aware that this potentially
requires compiling wxWidgets with wxThread support (unless it's
implemented using boost's thread functionality); therefore, the first
approach would probably be the default. I don't feel I'll know what's
truly best until I start trying things, though.
One key piece of functionality that I'm looking to implement that
isn't offered by either library is the ability to spawn a separate
process for multiple connections to a server--the benefit being that a
crash only affects one client. I'd appreciate any input on whether
this is a good or bad idea. I know apache takes this approach (or at
least a similar approach).
>>
>>
>> However I imagine that not everybody would be happy with requiring
>> Boost
See: http://www.boost.org/doc/libs/1_35_0/tools/bcp/bcp.html
>>
>> to build wx and, worse, ASIO doesn't seem to support VC6 which we're
>> apparently not ready to abandon yet. So while I personally would
>> like to do
>> what you propose provided backwards compatibility could be more or
>> less
>> preserved, I'm afraid VC6 is going to be the sticking point :-(
>
> I'm beginning to think the slogan for wx should be "let's program
> like it's 1999!" ;-) The rest of the world is moving on (in fact,
> VC6 hasn't been available for purchase for half a decade), and if wx
> doesn't move on too, users will leave it behind just like they did
> VC6.
>
> For users who refuse to change and update, there are always the
> existing and old versions of wx. And that's how any development
> process should work, because the toolkit today should focus on the
> needs of the users today (and, more importantly, tomorrow), not
> focus on keeping apps written 10 years ago from breaking. If a
> particular app developer doesn't want to change and update their
> code, or can't afford to over the course of several years, they
> don't have to change their wx version - the old one will run as it
> always did.
I agree. On the Mac, wxWidgets has already dropped support for 10.2.x
and older (or is it anything older than 10.3.9?). We're (Cedrus) on
the verge of dropping support even for 10.3.9 (released April of
2005). On Windows, however, wxWidgets still supports Visual C++ 6,
released in 1998. According to the wxWidgets website, VC5 is still
supported ( http://www.wxwidgets.org/about/feature2.htm ), as is
Windows 95. At Cedrus, we recently switched to Visual Studio 2005
because the VS7.1 compiler was failing to compile valid C++ code. The
longer wxWidgets supports archaic platforms with new code, the harder
it's going to be to support new *standards* (see C++0x http://en.wikipedia.org/wiki/C%2B%2B0x
).
There are various areas of our software where we've not used
wxWidgets, as the implementation was too backwards to be useful,
wxMediaCtrl and wxSound being two major examples. On the Mac,
wxMutex, wxBitmap, and wxDC are a few more examples (note that I'm
referring to the wxDC of two years ago here and comparing wxBitmap to
what QuickTime has to offer). We've already left wxWidgets behind
with all of these classes. wxSocket is close to being added to the
list.
>
>
> +1 to Stefan's ideas.
>
> Regards,
>
> Kevin
>
>>
>> Regards,
>> VZ
>>
wxWidgets 3.0 is supposed to be a major upgrade, right? Wouldn't that
make it a good time to drop support for old and broken compilers and
make major leaps forward in ease of programming?
-------------------------------
Hank Schultz
Cedrus Corporation
http://www.cedrus.com/
>> _______________________________________________
Vadim Zeitlin
WHS> I'm not convinced that the existing implementation necessarily
WHS> preserves 90% of the API.
Err, I don't really understand this. Anyhow, just to make clear what I
meant: we need to be able to support all these weird wxSocket flags (BLOCK
&c).
WHS> Last I checked, IPv6 wasn't available.
There is (untested by me personally) IPv6 support now but what does this
have to do with compatibility anyhow?
WHS> A big problem I've had is that "blocking" operations don't block.
WHS> They don't return, but they don't block either--they chew up 100% of
WHS> the CPU.
I'm pretty sure this is due to the incorrect use of wxSocket which was
discussed to death on wx-users in the past. See the last message in the
"bringing wxSocket into shape" thread on wx-dev from Kevin Hock from
2007-12-22 for more explanations.
WHS> >> However I imagine that not everybody would be happy with requiring
WHS> >> Boost
WHS>
WHS> See: http://www.boost.org/doc/libs/1_35_0/tools/bcp/bcp.html
Requiring (relatively big) part of Boost is not very different from
requiring all of it (speaking about the headers, anyhow, but it seems that
ASIO can be used without any other Boost libraries). Anyhow, if we use ASIO
for wxSocket it may make sense to also use other Boost classes.
WHS> wxWidgets 3.0 is supposed to be a major upgrade, right? Wouldn't that
WHS> make it a good time to drop support for old and broken compilers and
WHS> make major leaps forward in ease of programming?
I'm actually ok with dropping support for VC6, you need to convince Julian
and all the others who continue to use it.
But this still leaves the need to implement the baroque wxSocket API on
top of a rather different ASIO one. I'm sure that if you do it and provide
it in a usable form somewhere it will be very useful to a lot of people
whether or not it's included in wx itself. And sooner or later we will drop
VC6, of course, so we would be able to include it then. So I definitely
support the idea of doing it, I just can't promise that it is integrated
into wx 3.0 because it's not [only] my decision.
Regards,
VZ
Kip Warner
> Ah, so _I'm_ the culprit :-) It's out of habit and lack of motivation to
> stop using it, mostly, I'm happy to use a later version of VC++ if
> required. I check out the latest ones periodically anyway, to make sure
> DialogBlocks works against them.
>
> Regards,
>
> Julian
Julian, I'm curious to know, but what does your personal development
environment look like? e.g. IDE, compiler, OS, etc.. Given the nature of
wx, you probably use a variety of platforms, but I'd be interested in
what your "primary" setup looks like.
--
Kip Warner -- Software Engineer
OpenPGP encrypted/signed mail preferred
http://www.thevertigo.com
Julian Smart
> I'm actually ok with dropping support for VC6, you need to convince Julian
> and all the others who continue to use it.
>
Vadim Zeitlin
JS> Vadim Zeitlin wrote:
JS> > I'm actually ok with dropping support for VC6, you need to convince Julian
JS> > and all the others who continue to use it.
JS> >
JS> Ah, so I'm the culprit :-)
Well, you did argue against dropping it in the past -- and so did I as
long as there was no *real* reason to drop it but if this is the price to
pay to get rid of wxSocket code, I'd pay it gladly.
So do we all agree that we're prepared to replace wxSocket with an
ASIO-based implementation if it faithfully preserves the current API even
if it means not supporting non-standard compilers any more (of which I'm
afraid VC6 won't be the only representative, surely OpenWatcom won't be
able to compile it neither and I have my doubts about DMC)?
Regards,
VZ
Julian Smart
> On Fri, 23 May 2008 23:35:34 +0100 Julian Smart <jul...@anthemion.co.uk> wrote:
>
> JS> Vadim Zeitlin wrote:
> JS> > I'm actually ok with dropping support for VC6, you need to convince Julian
> JS> > and all the others who continue to use it.
> JS> >
> JS> Ah, so I'm the culprit :-)
>
> Well, you did argue against dropping it in the past -- and so did I as
> long as there was no *real* reason to drop it but if this is the price to
> pay to get rid of wxSocket code, I'd pay it gladly.
>
> So do we all agree that we're prepared to replace wxSocket with an
> ASIO-based implementation if it faithfully preserves the current API even
> if it means not supporting non-standard compilers any more (of which I'm
> afraid VC6 won't be the only representative, surely OpenWatcom won't be
> able to compile it neither and I have my doubts about DMC)?
>
back, alas. It could be an encouragement for them to improve, of course!
Regards,
Julian
Hisham Abboud
> I'm beginning to think the slogan for wx should be "let's program
> like it's 1999!" ;-) The rest of the world is moving on (in fact,
> VC6 hasn't been available for purchase for half a decade), and if wx
> doesn't move on too, users will leave it behind just like they did
> VC6.
I second Kevin and Stefan's opinion. I've always found wxWidgets'
support for old compilers misguided. It holds back development, and
IMHO, it does so unnecessarily. I think that a better approach is for
the latest wxWidgets release to support the most recent version of a
compiler plus the prior release -- only. The web site can then
indicate which version of wxWidgets is compatible with a particular
compiler, e.g. want to use VC6? Then wxWidgets 2.x is the last
compatible version.
Just my two pesos worth...
Hisham.
_______________
Hisham A. Abboud
Cedrus Corporation, http://www.cedrus.com
William H. Schultz
> On Fri, 23 May 2008 12:00:45 -0700 "William H. Schultz" <whsc...@cedrus.com
> > wrote:
>
> WHS> I'm not convinced that the existing implementation necessarily
> WHS> preserves 90% of the API.
>
> Err, I don't really understand this. Anyhow, just to make clear what I
> meant: we need to be able to support all these weird wxSocket flags
> (BLOCK
> &c).
I just mean that there's documented functionality that is close enough
to useless that it could be argued that it's not available.
>
>
> WHS> Last I checked, IPv6 wasn't available.
>
> There is (untested by me personally) IPv6 support now but what does
> this
> have to do with compatibility anyhow?
According to sckaddr.h, IPv6 isn't implemented. sckaddr.cpp backs
this up. The following function should be enough evidence:
bool wxIPV6address::IsLocalHost() const
{
return (Hostname() == wxT("localhost") || IPAddress() ==
wxT("127.0.0.1"));
}
Therefore, no matter how it's written, it's impossible to break
existing IPv6 code, since it doesn't exist, despite existing in the
header file.
>
>
> WHS> A big problem I've had is that "blocking" operations don't
> block.
> WHS> They don't return, but they don't block either--they chew up
> 100% of
> WHS> the CPU.
>
> I'm pretty sure this is due to the incorrect use of wxSocket which was
> discussed to death on wx-users in the past. See the last message in
> the
> "bringing wxSocket into shape" thread on wx-dev from Kevin Hock from
> 2007-12-22 for more explanations.
I don't recall off the top of my head the implementation that
exhibited this problem, but I had been working on the code so long
that I was well aware of the GUI issues, and I was aware of *why* it
was chewing up 100% of a CPU, even running from a background thread.
Ultimately, I had to poll using wxSocketBase::Peek() to see if data
was available to be read, sleeping the thread manually. All read and
write operations block without giving time to the GUI, since they are
running from a secondary thread.
>
>
> WHS> >> However I imagine that not everybody would be happy with
> requiring
> WHS> >> Boost
> WHS>
> WHS> See: http://www.boost.org/doc/libs/1_35_0/tools/bcp/bcp.html
>
> Requiring (relatively big) part of Boost is not very different from
> requiring all of it (speaking about the headers, anyhow, but it
> seems that
> ASIO can be used without any other Boost libraries). Anyhow, if we
> use ASIO
> for wxSocket it may make sense to also use other Boost classes.
bcp isn't as easy as the documentation makes it out to be... I still
haven't figured out how to just *build* the darn thing.
>
>
> WHS> wxWidgets 3.0 is supposed to be a major upgrade, right?
> Wouldn't that
> WHS> make it a good time to drop support for old and broken
> compilers and
> WHS> make major leaps forward in ease of programming?
>
> I'm actually ok with dropping support for VC6, you need to convince
> Julian
> and all the others who continue to use it.
>
> But this still leaves the need to implement the baroque wxSocket API
> on
> top of a rather different ASIO one. I'm sure that if you do it and
> provide
> it in a usable form somewhere it will be very useful to a lot of
> people
> whether or not it's included in wx itself. And sooner or later we
> will drop
> VC6, of course, so we would be able to include it then. So I
> definitely
> support the idea of doing it, I just can't promise that it is
> integrated
> into wx 3.0 because it's not [only] my decision.
Not a problem. While Asio and wxSocket are very different in
appearance, they are both based on asynchronous functionality; I don't
think it's going to be particularly tricky to make the existing API
work. I don't expect it to be easy, but I believe they are inherently
compatible.
I've started today on figuring out how to organize the Asio classes
within the existing wxSocket classes. Asio is designed so that the
difference between IPv4 and IPv6 is incredibly transparent... so much
so that wxIPaddress will probably no longer be an abstract class.
IPv6 functionality needs to be seamless or the code will be dead
within a couple of years.
>
>
> Regards,
> VZ
Vadim Zeitlin
WHS> I just mean that there's documented functionality that is close enough
WHS> to useless that it could be argued that it's not available.
Maybe but we'd have to look at each case in details. To me personally more
than half of wxSocket code seems broken but if we throw everything I don't
like away there won't be much left. Again, I'm not arguing for keeping
compatibility just because I like to annoy people but because it doesn't
make sense to write a wrapper around ASIO if it's not compatible -- if the
compatibility isn't important, people should just use ASIO directly.
WHS> According to sckaddr.h, IPv6 isn't implemented.
Are you looking at the trunk? Anyhow, this wasn't about compatibility but
just a "FYI" remark.
WHS> I've started today on figuring out how to organize the Asio classes
WHS> within the existing wxSocket classes. Asio is designed so that the
WHS> difference between IPv4 and IPv6 is incredibly transparent... so much
WHS> so that wxIPaddress will probably no longer be an abstract class.
WHS> IPv6 functionality needs to be seamless or the code will be dead
WHS> within a couple of years.
Yes, I think it makes more sense to have a single wxIPaddress class.
Good luck,
VZ
William H. Schultz
On May 23, 2008, at 5:30 AM, Vadim Zeitlin wrote:
> On Thu, 22 May 2008 18:06:49 -0700 "William H. Schultz" <whsc...@cedrus.com
> > wrote:
>
> WHS> I'm about to undertake a major rewrite of my socket code, and I'm
> WHS> leaning toward reimplementing wxSocket to use Boost.Asio (vs.
> WHS> replacing wxSocket in our code with Boost.Asio).
>
> Replacing wxSocket with a 3rd party library is definitely very, very
> tempting, I'm sure that ASIO is much better than the existing code.
> The
> problem is whether we're going to be able to preserve the existing API
> (even if not 100%, but at least 90%).
One thing I forgot to mention that I guess I had assumed was understood:
I am intending to drop GSocket and implement everything in the
wxSocket classes (as appropriate). I guess I'm assuming that anything
that extracts the GSocket from a wxSocket is depending on an
implementation detail and is therefore fair game for broken compiles.
I figure that in case anyone needs GSocket, a compile flag will be
added before the code including the new implementation gets added to
the official wxWidgets distribution.
>
>
> ...
>
> Regards,
> VZ
>
-------------------------------
Hank Schultz
Cedrus Corporation
http://www.cedrus.com/
> _______________________________________________
Vadim Zeitlin
WHS> I am intending to drop GSocket and implement everything in the
WHS> wxSocket classes
Yes, GSocket can rest in peace (actually I'd probably prefer to torture it
to death to repay it a little of what it did to me but well).
Regards,
VZ
William H. Schultz
On May 27, 2008, at 4:14 PM, Vadim Zeitlin wrote:
> On Tue, 27 May 2008 16:06:10 -0700 "William H. Schultz" <whsc...@cedrus.com
> > wrote:
>
> WHS> I just mean that there's documented functionality that is close
> enough
> WHS> to useless that it could be argued that it's not available.
>
> Maybe but we'd have to look at each case in details. To me
> personally more
> than half of wxSocket code seems broken but if we throw everything I
> don't
> like away there won't be much left. Again, I'm not arguing for keeping
> compatibility just because I like to annoy people but because it
> doesn't
> make sense to write a wrapper around ASIO if it's not compatible --
> if the
> compatibility isn't important, people should just use ASIO directly.
>
> WHS> According to sckaddr.h, IPv6 isn't implemented.
>
> Are you looking at the trunk? Anyhow, this wasn't about
> compatibility but
> just a "FYI" remark.
Actually, no. I'm looking at the 2.8 branch. I completely forgot
about that.
>
>
>
> ...
>
> Good luck,
> VZ
Thanks.
William H. Schultz
threaded client/server software does--at least in an initial test. At
the moment, I still haven't tested on anything other than Mac OS X,
and I'm aware that I still have a fair amount of work to do, but I'm
going to get started on a crude first rough draft to post.
Important changes to note:
in sckaddr.h:
The wxSockAddress classes have been refactored as follows:
wxObject
\/ \/ \/ \/ \/
wxSockAddress (pure virtual)
\/ \/ \/ \/ \/
wxIPaddress -- IPv4 *or* IPv6 address or hostname that can be resolved
to either of these.
\/ \/ \/ \/ \/
wxIPendpoint (pure virtual) -- wxIPaddress with a destination port
\/ \/ \/ \/ \/
wxTCPIPendpoint / wxUDPIPendpoint
Also defined is the following typedef:
wxDEPRECATED( typedef wxTCPIPendpoint wxIPV4address );
wxUNIXendpoint derives from wxTCPIPendpoint, but may not be plausible
with Asio.
New event type:
wxResolveEvent: when wxIPendpoint::Resolve() is called, one of these
events are passed to the handler for each possible resolution of the
hostname.
in socket.h:
More enums are available, though not all have been implemented yet.
Notes about classes:
wxSocketBase (pure virtual): A fair amount of this class is
untouched. The pushback buffer is unchanged. However, none of the
actual communication happens at this level, so there are a fair number
more virtual functions.
wxSocket: Most of the TCP/IP communication happens in this class.
Objects of this class can be instantiated, though there is no code
here to set up a connection. wxSocketServer::Accept provides an
object of this type.
wxSocketServer: I don't believe much to note has changed here, other
than that it provides wxSocket objects. I do intend to add the
ability (via flags) to spawn a thread or process for each incoming
connection.
wxSocketClient: This class has the ability to pass wxResolveEvents up
to the event handler during a wxSocketClient::Connect(). The event
handler can then decide to either halt the connection or skip the
specific resolution.
wxDatagramSocket: This is the only UDP/IP socket available. The code
is a bit behind the TCP implementation, since I have nothing that uses
it. This code should be *nearly* identical to that in wxSocket, other
than different function names for sending and receiving data.
wxSocketEvent: This has been expanded to include information on
whether an action was successful and the amount of data transferred
(if appropriate).
wxSocketWaitEvent: Waits can be done asynchronously. That is, you
can now receive an event when a timer expires--instead of using
wxYield to keep the user interface running. This event should be
passed both when a timer expires or when the desired action has
occurred (e.g. data received).
in thread.h:
There are various multi-threading issues that need to be considered
within Asio. To help with one issue, I've added a wxReadWriteMutex
class, along with wxReadMutexLocker and wxWriteMutexLocker. At the
moment, I've only implemented it in pthreads on Mac OS X (since that's
where I'm doing all of my programming--and since pthreads is more
efficient than the old Carbon alternative). I'm aware that Windows
doesn't provide this functionality, so it will need to be implemented
there too. Furthermore, I intend to add the ability to have an inter-
process wxMutex. In pthreads, creating an inter-process mutex is as
simple as passing a flag, but wxMutex will need to be tweaked to allow
for this.
The alternative to upgrading the wxMutex class was to use the
abilities built into boost, but these specific classes require
compiling boost. So far, nothing I've written requires compiling
boost or installing an additional shared library. Technically, this
is because I'm using Boost+Asio instead of Boost.Asio.
I'll get the current state of things into a postable patch tomorrow.
For my own ease of programing, I deleted all the socket code I'm not
using. I'm aware it will need to be reintroduced and enabled/disabled
via macros. I believe I've already done the bakefile and configure
modifications necessary for the read/write and shared mutexes, though.
-------------------------------
Hank Schultz
Cedrus Corporation
http://www.cedrus.com/