Re: [sqlite] Could/should Windows build of SQLite use #define WIN32_LEAN_AND_MEAN?
Teg
Isn't that just a build time thing? Include fewer includes during the
compile? I just don't build sqlite often enough to seem to think this
matters. Pretty sure this has no impact on the ultimate size of the
code generated.
Friday, March 23, 2012, 7:51:36 AM, you wrote:
JR> When building using the SQLite amalgamation, I noticed Windows.h
JR> being included without #define WIN32_LEAN_AND_MEAN. This includes
JR> a lot of extraneous "cruft". Any reason not to trim down the windows build this way?
JR> _______________________________________________
JR> sqlite-users mailing list
JR> sqlite...@sqlite.org
JR> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
--
Best regards,
Teg mailto:T...@djii.com
_______________________________________________
sqlite-users mailing list
sqlite...@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Larry Brasfield
> When building using the SQLite amalgamation, I noticed Windows.h being included without #define WIN32_LEAN_AND_MEAN. This includes a lot of extraneous "cruft". Any reason not to trim down the windows build this way?
There is no real reason not to do that. For my build of a customized
shell, this trims amalgamation compile time from 510mS to 400mS after a
series of previous compiles that have the same setting. (This gets
filesystem caching effects to behave as they would in any build where
this time saving would be a practical concern.)
If I was more cantankerous, I would be tempted to say there is no real
reason to complicate a build script for this 110mS time saving. Of
course, if you are building on some ancient machine, you might save more
time.
Cheers,
--
Larry Brasfield
Jeff Robbins
The problem I happened upon was trying to build a Python extension called APSW ( http://code.google.com/p/apsw/ )
This extension includes the SQLite amalgamation, which in turn includes Windows.h.
By not defining WINDOWS_LEAN_AND_MEAN, Windows.h includes RpcNdr.h which (sadly) does a #define small char.
And that #define in turn collides with a Python.h header (accu.h for those still awake!).
It isn't SQLite's fault or problem, but if the WINDOWS_LEAN_AND_MEAN approach works for SQLite, why not use it, if only out of parsimony (Occam's razor etc)? It would thereby reduce potential build conflicts when embedding SQLite into other apps or libraries.
My current workaround is to #define WINDOWS_LEAN_AND_MEAN prior to including the amalgamation, but that required me changing an APSW file (apsw.c). I'm in communication with the author of APSW as well on this, and with the Python team.
A shame to have to deal with this bad behavior from Windows.h...
Larry Brasfield
>
> The problem I happened upon was trying to build a Python extension called APSW ( http://code.google.com/p/apsw/ )
>
> This extension includes the SQLite amalgamation, which in turn includes Windows.h.
>
> By not defining WINDOWS_LEAN_AND_MEAN, Windows.h includes RpcNdr.h which (sadly) does a #define small char.
It is generally recognized that the C (or C++) preprocessor makes this
sort of problem hard to systematically avoid. And it is rude for that
header to flout the widely observed convention that symbols subjected to
#define are in uppercase.
> And that #define in turn collides with a Python.h header (accu.h for those still awake!).
>
> It isn't SQLite's fault or problem, but if the WINDOWS_LEAN_AND_MEAN approach works for SQLite, why not use it, if only out of parsimony (Occam's razor etc)? It would thereby reduce potential build conflicts when embedding SQLite into other apps or libraries.
It's a good practice, I suppose. Of course, where the amalgamation is
treated as a separate translation unit, this issue does not arise. It
is apsw.c's combination of the amalgamation, with its #include
<windows.h>, and another very inclusive header, Python.h, which has set
the stage for this.
Except for its clever hiding of SQLite's C API names (via "#define
SQLITE_API static"), there is no particular reason that apsw.c needed to
expose so many different name sources to each other.
> My current workaround is to #define WINDOWS_LEAN_AND_MEAN prior to including the amalgamation, but that required me changing an APSW file (apsw.c). I'm in communication with the author of APSW as well on this, and with the Python team.
Well, you can inject a -DWINDOWS_LEAN_AND_MEAN into the compiler
invocation for just that source, avoiding the need to modify its text.
> A shame to have to deal with this bad behavior from Windows.h...
Hmmm. It's not clear that the blame belongs solely to Windows.h or its
numerous subsidiary headers. This is a C problem, one which led to the
introduction of namespaces in C++, and one which leads developers to
prefer smaller translation units.
Nevertheless, it would help head off such issues to #include no more
than is necessary, and a "#define WINDOWS_LEAN_AND_MEAN" would get
partway there.
--
Larry Brasfield
Larry Brasfield
> [I am the author of APSW]
Thanks for that, BTW.
> On 24/03/12 14:22, Larry Brasfield wrote:
> > Except for its clever hiding of SQLite's C API names (via "#define
> > SQLITE_API static"),
>
> There is nothing "clever" about it - SQLite exposes various things like
> that so compilation can be controlled. Examples are calling convention
> and visibility.
Please understand that I did not mean "clever" in a denigrative way. I
saw the inclusion of sqlite3.c into apsw.c as a clever way to hide a set
of names. The mechanism is not such a leap.
> > there is no particular reason that apsw.c needed to expose so many
> > different name sources to each other.
>
> Believe it or not, I am not insane. APSW actually supports static
> including of the amalgamation, and static and dynamic linking.
For the record, I believed you were sane shortly after starting to read
this list and have had no cause to suspect otherwise.
> Static amalgamation has two major benefits. The first is performance -
> because everything is a single translation unit the compiler does a lot of
> inlining. The last time I measured the benefit a few years back it was
> around 10% compared to static linking.
Please forgive a bit of skepticism here. Do you mean that the
amalgamation of sqlite3.c into apsw.c provides a 10% speed advantage?
Or is your claim that SQLite itself enjoys this benefit? The former
seems unlikely to me, while the latter is quite believable.
> The second has to do with visibility of symbols. When doing the static
> amalgamation exactly one symbol is visible - the shared library entry
> point for the Python extension.
>
> On some platforms, Mac especially, there will be other copies of SQLite
> loaded into the process. (Mac is especially evil and will override your
> directions and use system SQLite no matter what.) If your shared library
> exposes any SQLite related symbols which could happen with static or
> dynamic linking then it is a crapshot as to how all the pieces of code
> using SQLite interact with the various SQLite's available.
I had no idea that such linkages could reach across DLL boundaries.
Does not an unsatisfied reference refer to a combination of DLL and
entry point? If they can be satisfied by entry point name alone, I
would expect to see some severe naming conventions recommended and
perhaps enforced. (Either that, or a continuing series of strange
incompatibilities.)
> Consequently the single best thing to do is include the SQLite
> amalgamation statically as it is far more robust.
So, the choice is between courting compile-time name collisions and
risking run-time name collisions. With a Hobson's choice like that, the
result will certainly be more robust, once the build succeeds.
> > Well, you can inject a -DWINDOWS_LEAN_AND_MEAN into the compiler
> > invocation for just that source, avoiding the need to modify its text.
>
> Or I'll just modify APSW for the next release. The issue actually only
> happens for some combinations of the Microsoft dev tools and sdks. It
> doesn't happen on the combination I use. (Use is also constrained for
> binary compatibility reasons.)
FYI, I'm having difficulty getting windows.h from the v7.0A Windows SDK
and the VS10 Express edition of the standard C headers to coexist with
the headers in Python 2.7. In particular, I cannot compile apsw.c
whether WINDOWS_LEAN_AND_MEAN is defined or not. (I am not asking for
help with this. I just imagine you could be interested.)
> >> A shame to have to deal with this bad behavior from Windows.h...
>
> Indeed.
As I stated, windows.h violated one of the conventions we use to reduce
the problem of name collision. But it is a problem that exists whenever
huge collections of definitions, in the same namespace, are brought
together. SQLite does a respectable job of mitigating this by
restricting its exposed names via prefixing.
Cheers,
Roger Binns
Hash: SHA1
On 25/03/12 11:28, Larry Brasfield wrote:
> Please forgive a bit of skepticism here. Do you mean that the
> amalgamation of sqlite3.c into apsw.c provides a 10% speed advantage?
Yes. Effectively the point of APSW is to provide a Pythonic API, extract
parameters and then call the SQLite API. When the APSW code and SQLite
code are in the same translation unit then a lot of inlining happens which
is more efficient - it is a lot of fun doing debugging.
> I had no idea that such linkages could reach across DLL boundaries.
If APSW had the SQLite symbols visible then later loading DLLs can end up
using those symbols instead of from some other SQLite such as the system
libraries.
>> Consequently the single best thing to do is include the SQLite
>> amalgamation statically as it is far more robust.
>
> So, the choice is between courting compile-time name collisions and
> risking run-time name collisions.
APSW is a Python extension and hence a shared library. You won't see
collisions at compile time. And you won't see collisions at runtime either.
An example of the inverse happening is on Mac. Setting load paths etc is
completely ignored and the system SQLite always used. This is usually an
older version and will cause grief.
> FYI, I'm having difficulty getting windows.h from the v7.0A Windows SDK
> and the VS10 Express edition of the standard C headers to coexist with
> the headers in Python 2.7.
You need to use Visual Studio 2008 to compile Python extensions on
Windows. Each release of Visual Studio includes a new C runtime library
which is binary incompatible with other releases. Additionally you can
only use one version of the CRT in a process at a time. Most current
versions of Python on Windows are compiled with VS2008 so the extensions
have to be compiled with the same version. These are the hoops I jump
through to setup a Windows compilation system:
http://code.google.com/p/apsw/wiki/Win7build
> In particular, I cannot compile apsw.c whether WINDOWS_LEAN_AND_MEAN is
> defined or not. (I am not asking for help with this. I just imagine
> you could be interested.)
The solution being used to #undef small which is less intrusive. SQLite
keeps getting deeper and deeper hooks in the VFS so I'd expect some point
in the future that the full fat version of Windows.h will be needed.
> As I stated, windows.h violated one of the conventions
It also dates from the early 80s so I don't really fault Microsoft
employees for not having a time machine.
Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iEYEARECAAYFAk9vfH4ACgkQmOOfHg372QS8wACeI/MSXOopVGOzzusymU2x1nXD
I5IAn2KIgHSgUSUepB7lGOsjsubu2lA8
=MGb2
-----END PGP SIGNATURE-----