Where is Berkeley DB on your system? Can you live without DB 1.85?
Skip Montanaro
I'm reworking the code in Python's setup.py and configure scripts that
locates the relevant Berkeley DB libraries and include files. This is
important because on many systems you can install multiple incompatible
versions of this library. Compiling the bsddb module with the include file
from one version and trying to link it with the libary from another version
is an obvious recipe for disaster.
I've figured out such information for a number of systems at my disposal but
I'd like to get as much input as possible about this for other systems so
the next Python release works out of the box for as many people as possible.
Here's what I want you to do. If you have a system on which you use
configure to build Python *and* if your Berkeley DB installation information
doesn't match one of the following (for N in [1,2,3,4] below):
include file location library location
--------------------- ----------------
/usr/include/dbN /usr/lib/lib/dbN
/usr/local/include/dbN /usr/local/lib/lib/dbN
/sw/include/dbN /sw/lib/dbN
/opt/sfw/include /opt/sfw/lib
please reply to this note off-list letting me know where it is on your
system.
Also, please let me know what version(s) of Berkeley DB you have installed
and use. I would *really* like to stop supporting compilation and linkage
with DB 1.85 because of the long known bugs in its hash file implementation.
If you really and truly need a convenient way to enable building with
Berkeley DB 1.85, let me know and I'll figure something out. Otherwise, I
will add the necessary lines to setup.py but comment them out. You'll have
to uncomment them so the bsddb module will build with the now-ancient DB
1.85.
Finally, is compilation with an older version (say, version 2) important to
you in the presence of a newer installed version (version 3 or 4)? Let me
know that as well.
Thx,
--
Skip Montanaro (sk...@pobox.com - http://www.mojam.com/)
Boycott Netflix - they spam - http://www.musi-cal.com/~skip/netflix.html
David LeBlanc
> From: python-l...@python.org
> [mailto:python-l...@python.org]On Behalf Of Skip Montanaro
> Sent: Friday, June 07, 2002 12:52
> To: pytho...@python.org
> Subject: Where is Berkeley DB on your system? Can you live without DB
> 1.85?
>
> 1.85.
>
> Finally, is compilation with an older version (say, version 2)
> important to
> you in the presence of a newer installed version (version 3 or 4)? Let me
> know that as well.
>
> Thx,
>
> --
> Skip Montanaro (sk...@pobox.com - http://www.mojam.com/)
> Boycott Netflix - they spam - http://www.musi-cal.com/~skip/netflix.html
If Sleepycat will allow it, i'm all in favor of updating to the latest
version! However, they want money for commercial use of versions > 1.85
AFAIK.
(FWIW, bsddb lives in python22/dlls on Windows :-). )
David LeBlanc
Seattle, WA USA
Oleg Broytmann
> If Sleepycat will allow it, i'm all in favor of updating to the latest
> version! However, they want money for commercial use of versions > 1.85
> AFAIK.
They don't. You can freely incorporate BSDDB into an opensource project.
http://www.sleepycat.com/faq.html#A22
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ p...@phd.pp.ru
Programmers don't die, they just GOSUB without RETURN.
Christopher Browne
> On Fri, Jun 07, 2002 at 01:14:59PM -0700, David LeBlanc wrote:
>> If Sleepycat will allow it, i'm all in favor of updating to the latest
>> version! However, they want money for commercial use of versions > 1.85
>> AFAIK.
>
> They don't. You can freely incorporate BSDDB into an opensource project.
> http://www.sleepycat.com/faq.html#A22
Ah, but that presumably _wouldn't_ mean that incorporating it into
Python would allow it to be freely incorporated into a
"non-OS-project" that uses Python.
--
(reverse (concatenate 'string "gro.mca@" "enworbbc"))
http://www3.sympatico.ca/cbbrowne/wp.html
Mental health is overrated!!
David LeBlanc
> From: python-l...@python.org
> [mailto:python-l...@python.org]On Behalf Of Oleg Broytmann
> Sent: Saturday, June 08, 2002 4:22
> To: David LeBlanc
> Cc: sk...@pobox.com; pytho...@python.org
> Subject: Re: Where is Berkeley DB on your system? Can you live without
> DB 1.85?
>
>
> > If Sleepycat will allow it, i'm all in favor of updating to the latest
> > version! However, they want money for commercial use of versions > 1.85
> > AFAIK.
>
> They don't. You can freely incorporate BSDDB into an
> opensource project.
> http://www.sleepycat.com/faq.html#A22
>
> --
> Oleg Broytmann http://phd.pp.ru/ p...@phd.pp.ru
> Programmers don't die, they just GOSUB without RETURN.
From the Sleepycat FAQ you cite:
14. Do current Berkeley DB releases have the same license as Berkeley DB
1.85?
Here's the non-commercial license that we're using for current Berkeley DB
releases: http://www.sleepycat.com/license.net. It's different from the one
we used for Berkeley DB 1.85. Generally speaking, the difference is that the
current license for Berkeley DB requires that software using Berkeley DB be
freely redistributable if Berkeley DB is going to be redistributed. If
Berkeley DB is not being redistributed in any way, no commercial license is
ever required. Please see the license for more specific information.
----------------------------------------------------------------------------
----
17. Do I have to license Berkeley DB to create an application for a single
customer?
Yes. Because you are redistributing your application off-site, you must
either make your application freely available in source code form or
purchase a license from Sleepycat Software for use of Berkeley DB in a
proprietary application. Sleepycat generally offers site-based licensing for
small deployments at prices that are lower than we charge for large-scale
redistribution. Please contact us for details.
----------------------------------------------------------------------------
----
18. Do I have to license Berkeley DB to use it in Perl or Python scripts?
No. The Berkeley DB license requires that software that uses Berkeley DB be
freely redistributable. In the case of Perl or Python, that software is Perl
or Python, and not your scripts. Any scripts you write are your property,
including scripts that make use of Berkeley DB. None of the Perl, Python or
Berkeley DB licenses place any restrictions on what you may do with them.
----------------------------------------------------------------------------
----
----------------------------------------------------------------------------
----
AFAIK, this is true for any version of Sleepycat > 1.85, and has been true
for years. Including a later version of Sleepycat BSDDB imposes a
constraint, not on Python itself, which is free, but on someone who would
like to distribute Python along with their _commercial_for_sale_ app (think
Py2Exe for example). That would be a distribution of BSDDB and and so "you
must either make your application freely available in source code form or
purchase a license from Sleepycat Software for use of Berkeley DB in a
proprietary application."
If you want to change Python's market dynamics and require _all_ Python apps
to be free and open source, then you can include any version of Sleepycat
BSDDB you like. I doubt it will have a positive impact on Python's future
prospects. It didn't work for Perl in it's early years and that's why the
Artistic License was change to allow "for fee" distributions that included
Perl.
Dave LeBlanc
Seattle, WA USA
Kragen Sitaker
> AFAIK, this is true for any version of Sleepycat > 1.85, and has been true
> for years. Including a later version of Sleepycat BSDDB imposes a
> constraint, not on Python itself, which is free, but on someone who would
> like to distribute Python along with their _commercial_for_sale_ app (think
> Py2Exe for example).
I don't think anybody has proposed including a modern version of BSD
DB in Python, only dropping support for an old version, which is also
not included in Python.
"commercial for-sale" is misleading. The current license of BSD DB
allows distribution in commercial for-sale apps with no further hassle
unless said commercial for-sale apps are, approximately, free
software, and it forbids inclusion in noncommercial not-for-sale apps
unless said not-for-sale apps are, approximately, free software.
> If you want to change Python's market dynamics and require _all_ Python apps
> to be free and open source, then you can include any version of Sleepycat
> BSDDB you like. I doubt it will have a positive impact on Python's future
> prospects. It didn't work for Perl in it's early years and that's why the
> Artistic License was change to allow "for fee" distributions that included
> Perl.
I thought the AL allowed this from the beginning. Certainly
Sleepycat's license doesn't forbid it.
In any case, your irresponsible, alarmist accusation is a red herring,
as you will no doubt agree once you understand the situation properly.
Dropping support for db 1.85 will not require all Python apps to be
free software or open source; it won't even require all distributions
of Python to be free software or open source. It will only require
distributions of Python with bsddb support to either link to an
external BSD DB library (like the one included with glibc) or be free
software.
If your application doesn't use BSD DB, it won't be affected by this,
even if you ship binary distributions of it, made with e.g. McMillan
Installer. If your application uses BSD DB and you are shipping db
1.85, you are screwing your users, probably without knowing it (unless
you know enough about db 1.85's bugs to work around them.). This
change will allow you to continue to screw your users in the same way,
but you will be required to know that was what you were doing.
At present, there are irresponsible vendors shipping proprietary
distributions of Python including BSD DB 1.85. This course of action
subjects the entire Python community to pressure to drop or deprecate
support for BSD DB, because BSD DB 1.85 is badly broken. Those of us
who use sane versions of BSD DB are hurt by this, because BSD DB is by
far the most featureful, performant, and reliable of the other
dbm-style databases Python supports.
I assume these vendors are being irresponsible because they don't know
they are imperiling their users' data by using this buggy old
software. This change will let them know that the problem exists, and
they can either stop shipping proprietary distributions, stop
supporting BSD DB, or start maintaining BSD DB 1.x.
If it's not clear, I'm in favor of this change. It helps everyone
everywhere.
David LeBlanc
> From: python-l...@python.org
> [mailto:python-l...@python.org]On Behalf Of Kragen Sitaker
> Sent: Monday, June 10, 2002 0:38
> To: pytho...@python.org
> Subject: Re: Where is Berkeley DB on your system? Can you live without
> DB 1.85?
>
>
> > AFAIK, this is true for any version of Sleepycat > 1.85, and
> has been true
> > for years. Including a later version of Sleepycat BSDDB imposes a
> > constraint, not on Python itself, which is free, but on someone
> who would
> > like to distribute Python along with their
> _commercial_for_sale_ app (think
> > Py2Exe for example).
>
> I don't think anybody has proposed including a modern version of BSD
> DB in Python, only dropping support for an old version, which is also
> not included in Python.
Hmmm... I have Python 2.2.1 binary installed from Sourceforge - what is that
bsddb.pyd file dated 4/9/2002 in the dlls directory?
> "commercial for-sale" is misleading. The current license of BSD DB
> allows distribution in commercial for-sale apps with no further hassle
> unless said commercial for-sale apps are, approximately, free
> software, and it forbids inclusion in noncommercial not-for-sale apps
> unless said not-for-sale apps are, approximately, free software.
This makes no sense, so I'll just pass on it. I think you got your facts
backwards.
> > If you want to change Python's market dynamics and require
> _all_ Python apps
> > to be free and open source, then you can include any version of
> Sleepycat
> > BSDDB you like. I doubt it will have a positive impact on
> Python's future
> > prospects. It didn't work for Perl in it's early years and
> that's why the
> > Artistic License was change to allow "for fee" distributions
> that included
> > Perl.
>
> I thought the AL allowed this from the beginning. Certainly
> Sleepycat's license doesn't forbid it.
IIRC, in the early days of Pearl, the "AL" had the same tainting as GPL.
Again, IIRC, some of the Pearl inner circle where the ones who agitated for
the change so they could make some money from all the time they where
pouring into Pearl. Today, the AL is more like the GLL.
Sleepycat's license doesn't prohibit commercial for-fee distribution, it
simply requires that you negotiate a payment scheme with them for the use of
versions after 1.85. They don't charge for redistribution with free open
source apps like Python. However, as I said in my last post (along with
exerpts from Sleepycat's FAQ on the subject), if a Python developer was to
include a Python distribution that included a version of bsddb after 1.85 in
a for-fee commercial program, then they would have to negotiate a payment
scheme with Sleepycat. More to the point, they would have to know that they
would have to do that or risk copyright infringement. (BTW, I was, at one
time, in such discussions with Sleepycat, but the project got dropped. I
found their fees to be extremely reasonable and they where quite flexible -
nice people to work with.)
> In any case, your irresponsible, alarmist accusation is a red herring,
> as you will no doubt agree once you understand the situation properly.
> Dropping support for db 1.85 will not require all Python apps to be
> free software or open source; it won't even require all distributions
> of Python to be free software or open source. It will only require
> distributions of Python with bsddb support to either link to an
> external BSD DB library (like the one included with glibc) or be free
> software.
I don't think I'm the one not understanding the situation properly, nor
could anything I posted be reasonably construed as irresponsible, alarmest
or an accusation! I never said anthing pro or con wrt to dropping bsddb 1.85
from the distribution. I didn't even make any accusations! I was only
referring to versions of bssdb after 1.85 having commercial licensing
provisions.
> If your application doesn't use BSD DB, it won't be affected by this,
> even if you ship binary distributions of it, made with e.g. McMillan
> Installer. If your application uses BSD DB and you are shipping db
> 1.85, you are screwing your users, probably without knowing it (unless
> you know enough about db 1.85's bugs to work around them.). This
> change will allow you to continue to screw your users in the same way,
> but you will be required to know that was what you were doing.
No customers are getting screwed :->
> At present, there are irresponsible vendors shipping proprietary
> distributions of Python including BSD DB 1.85. This course of action
> subjects the entire Python community to pressure to drop or deprecate
> support for BSD DB, because BSD DB 1.85 is badly broken. Those of us
> who use sane versions of BSD DB are hurt by this, because BSD DB is by
> far the most featureful, performant, and reliable of the other
> dbm-style databases Python supports.
Something remarkably like bsddb is getting shipped in python.org python
binaries from sourceforge as far as I can tell (Windows binaries).
> I assume these vendors are being irresponsible because they don't know
> they are imperiling their users' data by using this buggy old
> software. This change will let them know that the problem exists, and
> they can either stop shipping proprietary distributions, stop
> supporting BSD DB, or start maintaining BSD DB 1.x.
<gasp> could Guido be so heartless as to imperil people's data by allowing
the inclusion bsddb in the standard Python distro?!?
> If it's not clear, I'm in favor of this change. It helps everyone
> everywhere.
Whatever gets done, it would be nice to have some useful db shipped in the
standard Python distro. I have heard a rumor that gadfly is such a
candidate - and also that gadfly has bugs of it's own.
BTW, since you're so keen on the failings of the BSDDB that seems to ship
with standard Python, perhaps you'd like to contribute information on how to
use it and what it's pitfalls are and ways to avoid them to the pythondoc?
Hell, you could even help maintain BSD DB 1.x!
Tim Peters
> ...
> Hmmm... I have Python 2.2.1 binary installed from Sourceforge -
> what is that bsddb.pyd file dated 4/9/2002 in the dlls directory?
It's compiled from Sam Rushing's Win32 port of Berkeley's DB library,
version 1.85, from
http://www.nightmare.com/software.html
under the "bsd db" link (*not* the "bsddb" link).
It's a bug pit, through no fault of Sam's. The only reason we ship it is
that we've always shipped it, and nobody has volunteered the work to package
something better.
> ...
> <gasp> could Guido be so heartless as to imperil people's data by
> allowing the inclusion bsddb in the standard Python distro?!?
Yes <wink>. It wasn't known to be badly damaged at the time it was first
shipped, and now it's supplied for backward compatibility. A better
alternative would be most welcome, but PythonLabs has no bandwidth to work
on it.
Skip Montanaro
Tim> It wasn't known to be badly damaged at the time it was first
Tim> shipped, and now it's supplied for backward compatibility. A
Tim> better alternative would be most welcome, but PythonLabs has no
Tim> bandwidth to work on it.
Perhaps a goodhearted member of the community could be persuaded to build a
.pyd file using the 1.86 patch. It changes the file format, but fixes the
hash implementation bugs. Check here (bottom of page):
http://www.sleepycat.com/update/index.html
I'm not sure (and don't really care) about any license implications.
Kragen Sitaker
> > I don't think anybody has proposed including a modern version of BSD
> > DB in Python, only dropping support for an old version, which is also
> > not included in Python.
>
> Hmmm... I have Python 2.2.1 binary installed from Sourceforge - what is that
> bsddb.pyd file dated 4/9/2002 in the dlls directory?
That itself is probably the Python module that interfaces to whatever
version of Berkeley DB you have, not Berkeley DB itself (although it
could be statically linked in there), but Tim Peters has asserted that
a binary version of db 1.85 is included, regardless.
So I was wrong. Certainly Berkeley DB is not included in the Python
source tarball.
> > "commercial for-sale" is misleading. The current license of BSD DB
> > allows distribution in commercial for-sale apps with no further hassle
> > unless said commercial for-sale apps are, approximately, free
> > software, and it forbids inclusion in noncommercial not-for-sale apps
> > unless said not-for-sale apps are, approximately, free software.
>
> This makes no sense, so I'll just pass on it. I think you got your facts
> backwards.
It makes no sense because I got the sense of my first clause
backwards; thanks for the correction. It should read as follows:
It allows distribution in commercial for-sale apps with no further
hassle unless they are *not* free software.
> > > prospects. It didn't work for Perl in it's early years and
> > that's why the
> > > Artistic License was change to allow "for fee" distributions
> > that included
> > > Perl.
> >
> > I thought the AL allowed this from the beginning. Certainly
> > Sleepycat's license doesn't forbid it.
>
> IIRC, in the early days of Pearl, the "AL" had the same tainting as GPL.
> Again, IIRC, some of the Pearl inner circle where the ones who agitated for
> the change so they could make some money from all the time they where
> pouring into Pearl. Today, the AL is more like the GLL.
My experience with Perl doesn't go back to the days before its public
release when it was spelled with an "a", so I'll take your word for it.
> Sleepycat's license doesn't prohibit commercial for-fee distribution, it
> simply requires that you negotiate a payment scheme with them for the use of
> versions after 1.85. They don't charge for redistribution with free open
> source apps like Python.
This is self-contradictory. The second statement is accurate; the
first statement is not. They don't require that you negotiate a
payment scheme with them *unless your commercial for-fee distribution
is a distribution of proprietary software*. For example, Red Hat
doesn't have to negotiate a license to Berkeley DB to include it in their
commercial for-free Linux distribution.
> However, as I said in my last post (along with exerpts from
> Sleepycat's FAQ on the subject), if a Python developer was to
> include a Python distribution that included a version of bsddb after
> 1.85 in a for-fee commercial program, then they would have to
> negotiate a payment scheme with Sleepycat.
Again, *only* if it was a *proprietary* for-fee commercial program.
The same issue would come up if they were distributing a proprietary
not-for-fee noncommercial program.
> More to the point, they would have to know that they would have to
> do that or risk copyright infringement.
Yes, they would have to understand the licensing terms of the software
they were redistributing.
> > In any case, your irresponsible, alarmist accusation is a red herring,
> > as you will no doubt agree once you understand the situation properly.
> > Dropping support for db 1.85 will not require all Python apps to be
> > free software or open source; it won't even require all distributions
> > of Python to be free software or open source. It will only require
> > distributions of Python with bsddb support to either link to an
> > external BSD DB library (like the one included with glibc) or be free
> > software.
>
> I don't think I'm the one not understanding the situation properly, nor
> could anything I posted be reasonably construed as irresponsible, alarmest
> or an accusation!
You said:
> If you want to change Python's market dynamics and require _all_ Python apps
> to be free and open source, then you can include any version of Sleepycat
> BSDDB you like.
That was what I meant by "your irresponsible, alarmist accusation". It
isn't true, and it sounds scary and accusatory to me. Only Python
apps that use Berkeley DB would be affected, and then only if they're
distributed in binary form, and then only if they're running on
operating systems that don't ship with Berkeley DB.
> > If it's not clear, I'm in favor of this change. It helps everyone
> > everywhere.
>
> Whatever gets done, it would be nice to have some useful db shipped in the
> standard Python distro. I have heard a rumor that gadfly is such a
> candidate - and also that gadfly has bugs of it's own.
I think dumbdbm and gdbm are more comparable to Berkeley DB than Gadfly is;
either or both of them could be included in a standard Python distro
with no licensing problems, even for proprietary software.
> BTW, since you're so keen on the failings of the BSDDB that seems to ship
> with standard Python, perhaps you'd like to contribute information on how to
> use it and what it's pitfalls are and ways to avoid them to the pythondoc?
I don't know what they are because I use sane versions of Berkeley DB.
I seem to recall that hash-style db 1.85 files will fail after a lot
of inserts, and Sleepycat has patches on their page that fixes a
couple of core dumps and one case of lost updates in the BTree code,
but ISTR that there are other problems, too.
> Hell, you could even help maintain BSD DB 1.x!
I think it's more cost-effective to just ship 4.0 in binary
distributions of Python and require users to switch. If someone finds
4.0 (or 2.x or 3.x) unacceptable because they want to incorporate
Berkeley DB functionality into proprietary software, I would be
willing to work on 1.x for them at my regular contract rate, but
increasing the profits of proprietary software companies who are too
cheap to buy a license from Sleepycat isn't really my idea of a fun
way to spend my spare time. Given that, it would probably be more
cost-effective for such a company to just pay Sleepycat for a 4.0
license.
Tim Peters
[on the bsddb.pyd installed by the PythonLabs Windows installer]
> That itself is probably the Python module that interfaces to whatever
> version of Berkeley DB you have, not Berkeley DB itself (although it
> could be statically linked in there),
Sam Rushing's Windows port of Berkeley DB 1.85 is statically linked in
there. Read PCbuild/readme.txt in the source distribution; for the
convenience of compiler-less Windows users, we build and ship several
precompiled 3rd-party components in the Windows distro.
[on what the pitfalls of 1.85 are]
> I don't know what they are because I use sane versions of Berkeley DB.
See "2. Are there known problems with the 1.85 and 1.86 versions?" at
http://www.sleepycat.com/historic.html
1.85-is-hopeless-ly y'rs - tim