RFD: Allowing to use 3rd party libraries bundled with wx from application code
33 views
Skip to first unread message
Vadim Zeitlin
Jun 20, 2021, 8:01:44 AM6/20/21
to wx-dev
Hello,
We currently include several useful 3rd party libraries into wx itself,
but there is no way for the application that also wants to use these
libraries directly to do it without building its own copy of them. This is
somewhat annoying, e.g. almost all my projects end up using zlib, usually
as an indirect dependency, and each of them has to build its own version,
which is relatively straightforward for zlib, but it would still be nice if
it could be avoided.
In fact, for zlib, specifically, we make our own build intentionally
incompatible with the standard version by defining "Z_PREFIX_" as
"wx_zlib_" in order to rename all of its symbols. And this does have the
advantage of isolating wx zlib from anything used by the application, so it
can clearly be useful. But I believe that at least in some situations
letting wx build all these libraries and allowing the applications to use
them would be even more convenient, especially under MSW/Mac where system
libraries are typically not used.
Doing this would require a lot of changes, which is why we must do it
before 3.2 (and even before 3.1.6) if we want to do it at all, and also why
I'd like to check if there is any interest in doing this here because if
I'm the only one who thinks it would be worth it, it's probably not as
useful as I think.
So the idea would be to optionally allow making the built-in libraries
available for the application use, which involves at least making the
3rd party libraries headers and the built libraries themselves available
under the standard names (i.e. libz.a and not libwxzlib-3.1.a), but
probably much more (e.g. I have no idea how would this work with CMake).
What do you think about this idea?
VZ
We currently include several useful 3rd party libraries into wx itself,
but there is no way for the application that also wants to use these
libraries directly to do it without building its own copy of them. This is
somewhat annoying, e.g. almost all my projects end up using zlib, usually
as an indirect dependency, and each of them has to build its own version,
which is relatively straightforward for zlib, but it would still be nice if
it could be avoided.
In fact, for zlib, specifically, we make our own build intentionally
incompatible with the standard version by defining "Z_PREFIX_" as
"wx_zlib_" in order to rename all of its symbols. And this does have the
advantage of isolating wx zlib from anything used by the application, so it
can clearly be useful. But I believe that at least in some situations
letting wx build all these libraries and allowing the applications to use
them would be even more convenient, especially under MSW/Mac where system
libraries are typically not used.
Doing this would require a lot of changes, which is why we must do it
before 3.2 (and even before 3.1.6) if we want to do it at all, and also why
I'd like to check if there is any interest in doing this here because if
I'm the only one who thinks it would be worth it, it's probably not as
useful as I think.
So the idea would be to optionally allow making the built-in libraries
available for the application use, which involves at least making the
3rd party libraries headers and the built libraries themselves available
under the standard names (i.e. libz.a and not libwxzlib-3.1.a), but
probably much more (e.g. I have no idea how would this work with CMake).
What do you think about this idea?
VZ
Eric Jensen
Jun 20, 2021, 8:15:49 AM6/20/21
to Vadim Zeitlin
Hello Vadim,
Sunday, June 20, 2021, 2:01:42 PM, you wrote:
VZ> So the idea would be to optionally allow making the built-in libraries
VZ> available for the application use, which involves at least making the
VZ> 3rd party libraries headers and the built libraries themselves available
VZ> under the standard names (i.e. libz.a and not libwxzlib-3.1.a), but
VZ> probably much more (e.g. I have no idea how would this work with CMake).
VZ> What do you think about this idea?
VZ> VZ
All i can say that i'm using wxWidgets for 15+ years now and i never
had a case where i needed direct access to any of the 3rd party libs.
But of course that's only my personal experience, it may be different
for other users.
Eric
Sunday, June 20, 2021, 2:01:42 PM, you wrote:
VZ> So the idea would be to optionally allow making the built-in libraries
VZ> available for the application use, which involves at least making the
VZ> 3rd party libraries headers and the built libraries themselves available
VZ> under the standard names (i.e. libz.a and not libwxzlib-3.1.a), but
VZ> probably much more (e.g. I have no idea how would this work with CMake).
VZ> What do you think about this idea?
VZ> VZ
All i can say that i'm using wxWidgets for 15+ years now and i never
had a case where i needed direct access to any of the 3rd party libs.
But of course that's only my personal experience, it may be different
for other users.
Eric
Kenneth Porter
Jun 20, 2021, 9:37:59 AM6/20/21
to wx-...@googlegroups.com
--On Sunday, June 20, 2021 3:01 PM +0200 Vadim Zeitlin
I think you need to expand "would be nice" to specific advantages. A few
come to mind, but none seem compelling:
* shorter build time
* smaller memory footprint
* easier servicing of issues in the dependent library
Development systems are fast, and even the smallest systems using wx have
lots of memory compared to the library footprint. The last one is the only
one that would be important to me. If a security bug is found in a
dependent library, then both the dependent library and wx must be patched
or updated, which requires more coordination.
I think you need to expand "would be nice" to specific advantages. A few
come to mind, but none seem compelling:
* shorter build time
* smaller memory footprint
* easier servicing of issues in the dependent library
Development systems are fast, and even the smallest systems using wx have
lots of memory compared to the library footprint. The last one is the only
one that would be important to me. If a security bug is found in a
dependent library, then both the dependent library and wx must be patched
or updated, which requires more coordination.
rohit agarwal
Jun 20, 2021, 9:38:37 AM6/20/21
to wx-...@googlegroups.com
wxWidgets is already too big and too complex.
As a developer, using it, therefore, it works best to boilerplate it out.
The number of files complied to build it are too many for the effective functionality it provides.
Also, the makefile/bakefile/cmake part does appear confusing
and is aniother aspect that I boilerplate out.
Instead of adding even more functionality, I would try and make the code leaner
and focus on making the core functionality easy to access
at the expense of making more complicated scenarios more difficult.
Ideally, a developer should be able to build an app using wxWidgets source files
without having to build a library first.
That is possible.
--
You received this message because you are subscribed to the Google Groups "wx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wx-dev+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wx-dev/808445346.20210620141547%40j-dev.de.
As a developer, using it, therefore, it works best to boilerplate it out.
The number of files complied to build it are too many for the effective functionality it provides.
Also, the makefile/bakefile/cmake part does appear confusing
and is aniother aspect that I boilerplate out.
Instead of adding even more functionality, I would try and make the code leaner
and focus on making the core functionality easy to access
at the expense of making more complicated scenarios more difficult.
Ideally, a developer should be able to build an app using wxWidgets source files
without having to build a library first.
That is possible.
You received this message because you are subscribed to the Google Groups "wx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wx-dev+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wx-dev/808445346.20210620141547%40j-dev.de.
Vadim Zeitlin
Jun 20, 2021, 10:09:50 AM6/20/21
to wx-...@googlegroups.com
On Sun, 20 Jun 2021 19:07:32 +0530 rohit agarwal wrote:
ra> wxWidgets is already too big and too complex.
Sorry, but this is completely irrelevant to this discussion. I'm speaking
about the libraries that are already included in wxWidgets, so this does
nothing to increase (or reduce) the library size or complexity (I also do
plan to add another third party library dependency, but this is a separate
subject too).
Regards,
VZ
ra> wxWidgets is already too big and too complex.
Sorry, but this is completely irrelevant to this discussion. I'm speaking
about the libraries that are already included in wxWidgets, so this does
nothing to increase (or reduce) the library size or complexity (I also do
plan to add another third party library dependency, but this is a separate
subject too).
Regards,
VZ
Vadim Zeitlin
Jun 20, 2021, 10:16:32 AM6/20/21
to wx-...@googlegroups.com
On Sun, 20 Jun 2021 06:36:29 -0700 Kenneth Porter wrote:
KP> --On Sunday, June 20, 2021 3:01 PM +0200 Vadim Zeitlin
KP> <va...@wxwidgets.org> wrote:
KP>
KP> > it would still be nice if it could be avoided.
KP>
KP> I think you need to expand "would be nice" to specific advantages. A few
KP> come to mind, but none seem compelling:
KP>
KP> * shorter build time
KP> * smaller memory footprint
KP> * easier servicing of issues in the dependent library
Yes, sorry, I should have clearly explained it better, thanks for pointing
it out. "Nice" was shorthand for "simpler to use". I.e. if a project
already builds wxWidgets, it's just simpler if you can reuse zlib etc
already built as part of this build, rather than building it separately
because you need to use it directly (or, again, because another library you
use needs to use it directly, which is a more common use case IME).
You're right that the points above are not compelling at all, but having
to keep a different copy of the libraries already used in wx and to build
them is somewhat of a burden. As I already wrote, I end up with 2 copies of
zlib in almost all of my projects, and in at least one of them I had 3
copies of it because another dependency included its own bundled copy of
zlib too, which really doesn't seem ideal, if only because you're almost
surely going to end up using different versions of the same library, sooner
or later, and while this can be harmless, it can also be quite surprising.
Regards,
VZ
KP> --On Sunday, June 20, 2021 3:01 PM +0200 Vadim Zeitlin
KP> <va...@wxwidgets.org> wrote:
KP>
KP> > it would still be nice if it could be avoided.
KP>
KP> I think you need to expand "would be nice" to specific advantages. A few
KP> come to mind, but none seem compelling:
KP>
KP> * shorter build time
KP> * smaller memory footprint
KP> * easier servicing of issues in the dependent library
Yes, sorry, I should have clearly explained it better, thanks for pointing
it out. "Nice" was shorthand for "simpler to use". I.e. if a project
already builds wxWidgets, it's just simpler if you can reuse zlib etc
already built as part of this build, rather than building it separately
because you need to use it directly (or, again, because another library you
use needs to use it directly, which is a more common use case IME).
You're right that the points above are not compelling at all, but having
to keep a different copy of the libraries already used in wx and to build
them is somewhat of a burden. As I already wrote, I end up with 2 copies of
zlib in almost all of my projects, and in at least one of them I had 3
copies of it because another dependency included its own bundled copy of
zlib too, which really doesn't seem ideal, if only because you're almost
surely going to end up using different versions of the same library, sooner
or later, and while this can be harmless, it can also be quite surprising.
Regards,
VZ
Fulvio Senore
Jun 20, 2021, 11:29:10 AM6/20/21
to wx-...@googlegroups.com
Il 20/06/2021 14:01, Vadim Zeitlin ha scritto:
> So the idea would be to optionally allow making the built-in libraries
> available for the application use, which involves at least making the
> 3rd party libraries headers and the built libraries themselves available
> under the standard names (i.e. libz.a and not libwxzlib-3.1.a), but
> probably much more (e.g. I have no idea how would this work with CMake).
>
> What do you think about this idea?
> VZ
>
A few times I had to directly use the jpeg and tiff libraries and that
> So the idea would be to optionally allow making the built-in libraries
> available for the application use, which involves at least making the
> 3rd party libraries headers and the built libraries themselves available
> under the standard names (i.e. libz.a and not libwxzlib-3.1.a), but
> probably much more (e.g. I have no idea how would this work with CMake).
>
> What do you think about this idea?
> VZ
>
needed some sort of hack. Having the ability to avoid those hacks would
be welcome but it is not something that would change my life.
I have a request that maybe can be linked to your proposal. I would like
to be able to statically link a wx program with the jpeg-turbo library
instead of the included one: it yields a substantial speed advantage
that is useful in one of my programs. If this ability can be added I
would definitely welcome it.
Kenneth Porter
Jun 20, 2021, 11:54:04 AM6/20/21
to wx-...@googlegroups.com
On 6/20/2021 7:09 AM, Vadim Zeitlin wrote:
> Sorry, but this is completely irrelevant to this discussion. I'm speaking
> about the libraries that are already included in wxWidgets, so this does
> nothing to increase (or reduce) the library size or complexity (I also do
> plan to add another third party library dependency, but this is a separate
> subject too).
I use wx for portability. I started using it before I learned how to use
> Sorry, but this is completely irrelevant to this discussion. I'm speaking
> about the libraries that are already included in wxWidgets, so this does
> nothing to increase (or reduce) the library size or complexity (I also do
> plan to add another third party library dependency, but this is a separate
> subject too).
Boost, and now I tend to use Boost for non-GUI work and limit use of wx
to just GUI portability. I've used wx for threading and XML but now I'd
go a different route and use Boost for threading and a C++ library
specific to XML. (I understand wxXML won't be improved to address new
features in the underlying library, so if I want to incorporate those,
I'm going to have to "un-wx" that part of my code.) While I'm impressed
that wx does so much, it doesn't have the developer bandwidth that Boost
has, so it makes sense to focus on the GUI part and leave non-GUI
features to other libraries with big teams. Do what you do best.
The image format and compression libraries don't tend to change much
over time so I'm fine using wx wrappers for those. They HAVE seen some
security issues, though, and I wouldn't be surprised to see more in the
future. And there will always be new image formats to deal with.
Compression is definitely useful outside of the GUI so moving that
out-of-tree would be nice.
Vadim Zeitlin
Jun 20, 2021, 11:57:25 AM6/20/21
to wx-...@googlegroups.com
On Sun, 20 Jun 2021 08:52:53 -0700 Kenneth Porter wrote:
KP> On 6/20/2021 7:09 AM, Vadim Zeitlin wrote:
KP> > Sorry, but this is completely irrelevant to this discussion. I'm speaking
KP> > about the libraries that are already included in wxWidgets, so this does
KP> > nothing to increase (or reduce) the library size or complexity (I also do
KP> > plan to add another third party library dependency, but this is a separate
KP> > subject too).
KP>
KP> I use wx for portability. I started using it before I learned how to use
KP> Boost, and now I tend to use Boost for non-GUI work and limit use of wx
KP> to just GUI portability. I've used wx for threading and XML but now I'd
KP> go a different route and use Boost for threading and a C++ library
KP> specific to XML. (I understand wxXML won't be improved to address new
KP> features in the underlying library, so if I want to incorporate those,
KP> I'm going to have to "un-wx" that part of my code.) While I'm impressed
KP> that wx does so much, it doesn't have the developer bandwidth that Boost
KP> has, so it makes sense to focus on the GUI part and leave non-GUI
KP> features to other libraries with big teams. Do what you do best.
FWIW I absolutely agree with this (except that I would use standard
library and not Boost for threading) but, again, this is just not the topic
of this discussion here. We're not discussing adding (or removing)
anything, just repackaging what's already available in a more convenient
form.
Regards,
VZ
KP> On 6/20/2021 7:09 AM, Vadim Zeitlin wrote:
KP> > Sorry, but this is completely irrelevant to this discussion. I'm speaking
KP> > about the libraries that are already included in wxWidgets, so this does
KP> > nothing to increase (or reduce) the library size or complexity (I also do
KP> > plan to add another third party library dependency, but this is a separate
KP> > subject too).
KP>
KP> I use wx for portability. I started using it before I learned how to use
KP> Boost, and now I tend to use Boost for non-GUI work and limit use of wx
KP> to just GUI portability. I've used wx for threading and XML but now I'd
KP> go a different route and use Boost for threading and a C++ library
KP> specific to XML. (I understand wxXML won't be improved to address new
KP> features in the underlying library, so if I want to incorporate those,
KP> I'm going to have to "un-wx" that part of my code.) While I'm impressed
KP> that wx does so much, it doesn't have the developer bandwidth that Boost
KP> has, so it makes sense to focus on the GUI part and leave non-GUI
KP> features to other libraries with big teams. Do what you do best.
FWIW I absolutely agree with this (except that I would use standard
library and not Boost for threading) but, again, this is just not the topic
of this discussion here. We're not discussing adding (or removing)
anything, just repackaging what's already available in a more convenient
form.
Regards,
VZ
Vadim Zeitlin
Jun 20, 2021, 12:03:41 PM6/20/21
to wx-...@googlegroups.com
On Sun, 20 Jun 2021 17:29:06 +0200 Fulvio Senore wrote:
FS> A few times I had to directly use the jpeg and tiff libraries and that
FS> needed some sort of hack. Having the ability to avoid those hacks would
FS> be welcome but it is not something that would change my life.
I'm actually impressed that you've managed to use them, I considered doing
it once or twice but it was too complicated to do for me, so I didn't.
FS> I have a request that maybe can be linked to your proposal. I would like
FS> to be able to statically link a wx program with the jpeg-turbo library
FS> instead of the included one: it yields a substantial speed advantage
FS> that is useful in one of my programs. If this ability can be added I
FS> would definitely welcome it.
This would be possible with another approach of solving the problem of
having multiple copies of the same library, which would consist in allowing
to use a different version of a 3rd party library than the bundled one with
wx. If we supported this, you could build your version of libjpeg and then
just tell wx to use it instead of the one it includes. And, of course, you
could also build your own zlib etc.
I suspect this solution would be preferable, but it risks being even more
difficult to implement, at least under MSW, where this is mostly needed.
As for the special case of jpeg-turbo, I think the simplest would be to
upgrade (or replace, as it's not really an upgrade) the version used by wx
with it. AFAIK it should be compatible with libjpeg that we use, but
faster. And this shouldn't be difficult to do, so if anybody is interested
in doing this, please do (and this can be done completely independently of
the outcome of this discussion).
Regards,
VZ
FS> A few times I had to directly use the jpeg and tiff libraries and that
FS> needed some sort of hack. Having the ability to avoid those hacks would
FS> be welcome but it is not something that would change my life.
I'm actually impressed that you've managed to use them, I considered doing
it once or twice but it was too complicated to do for me, so I didn't.
FS> I have a request that maybe can be linked to your proposal. I would like
FS> to be able to statically link a wx program with the jpeg-turbo library
FS> instead of the included one: it yields a substantial speed advantage
FS> that is useful in one of my programs. If this ability can be added I
FS> would definitely welcome it.
This would be possible with another approach of solving the problem of
having multiple copies of the same library, which would consist in allowing
to use a different version of a 3rd party library than the bundled one with
wx. If we supported this, you could build your version of libjpeg and then
just tell wx to use it instead of the one it includes. And, of course, you
could also build your own zlib etc.
I suspect this solution would be preferable, but it risks being even more
difficult to implement, at least under MSW, where this is mostly needed.
As for the special case of jpeg-turbo, I think the simplest would be to
upgrade (or replace, as it's not really an upgrade) the version used by wx
with it. AFAIK it should be compatible with libjpeg that we use, but
faster. And this shouldn't be difficult to do, so if anybody is interested
in doing this, please do (and this can be done completely independently of
the outcome of this discussion).
Regards,
VZ
Kenneth Porter
Jun 20, 2021, 10:34:09 PM6/20/21
to wx-...@googlegroups.com
On 6/20/2021 8:57 AM, Vadim Zeitlin wrote:
> FWIW I absolutely agree with this (except that I would use standard
> library and not Boost for threading) but, again, this is just not the topic
> of this discussion here. We're not discussing adding (or removing)
> anything, just repackaging what's already available in a more convenient
> form.
At the time I started using Boost, the standard library didn't yet have
> FWIW I absolutely agree with this (except that I would use standard
> library and not Boost for threading) but, again, this is just not the topic
> of this discussion here. We're not discussing adding (or removing)
> anything, just repackaging what's already available in a more convenient
> form.
threading. As pieces of Boost get migrated into the standard ilbrary, I
modify my code to use the standard versions. One has to be careful doing
this, as not all features have the same semantics. But much of it can be
a drop-in.
Reply all
Reply to author
Forward
0 new messages