I can confirm from my recent experience of writing a DB backend that
some spec is needed. There is now a de-facto public interface
(".connection", "cursor()", "_commit()", "_rollback()") but nobody knows
what's going to change.
On Thu, 2008-03-20 at 06:20 -0700, jeffself wrote:
> My employer currently uses DB2 for its database. Antonio Cangiano
> recently mentioned that IBM has written a DB2 driver for Python. This
> driver uses the SQLAlchemy ORM. I asked him if IBM had any plans to
> write a driver that would work with Django and here is his response:
>
> "The API team (which currently maintains all the dynamic language APIs
> for DB2) considers the Django adapter a priority.
Such a priority that they've never brought this request to us in the
past. Guess that's why the word "high" wasn't used in that sentence. :-)
> Due to IBM's strict
> rules in terms of open source participation though, the developers
> can't simply look at the code of the other adapters (not even the
> dummy one) and adapt it for DB2. Therefore, in their case, the lack of
> a public spec, that outlines method by method the requirements for any
> new Django ORM adapters, is the biggest obstacle. If such a document
> was freely published on the web (without restrictions),
Except for copyright and normal open source licensing requirements,
presumably.
> the API team
> could come up with a Django adapter for CPython and for Jython in a
> short amount of time."
>
> Are there any plans to implement a public spec?
Well, there already is a spec. It's written in Python, which is mostly
Dutch pseudo-code rather than colloquial English, but it's fairly well
commented and the method names are sensible. Interestingly that's
enough, for example, for IBM's kernel developers, who regularly read
other parts of the kernel when constructing their APIs and driver
contributions.
Still, there's no accounting for some people's wish for paperwork. The
best way to continue would be for you to write up something based on the
current backends, send it here and we can review it. It's probably
something that would fit nicely alongside DevModelCreation in the wiki
and would be nice to have just for documenting how things work. We could
tidy up the DeveloperDocs page and promote it to "official" in some
sense. There'll probably be a bit of back-and-forth needed after the
first version to work out which features really are to be relied upon,
but certainly stuff in django/db/backends/__init__.py is going to be
pretty much an accurate representation of the state of play at the
moment. In fact, that file shows precisely how much duplicated effort
this "specification" requirement will require: it's got all the methods
you need, a honking great docstring on almost all of them -- adding
docstrings to the remaining ones will be easy and satisfy, say, Ivan's
requirements I would thinking -- and is laid out in a fashion that a
competent Python programmer can scan and implement really fast.
Regards,
Malcolm
--
The early bird may get the worm, but the second mouse gets the cheese.
http://www.pointy-stick.com/blog/
You do realize that any document so written and released would be (a)
released under the same BSD license as the rest of Django's source and
would (b) contain any supposed tainted IP that IBM's so terrified of.
Keep in mind that the BSD license is the most permissive software
license to stand up to legal tests; IBM is perfect within their rights
as licensees to read, modify, and distribute -- commercially, even --
Django's source. I encourage them to do so.
I *highly* doubt that IBM engineers are not allowed to look at Open
Source -- in fact, I know IBM engineers who contribute to Open Source
daily. Perhaps they do it while wearing blinders So IBM is insinuating
here that Django's IP provenance is tainted. This is FUD, pure and
simple.
I'll bet at least someone from IBM reads this list. If that's the
case, wanna let me know -- publicly or privately -- exactly what the
problem with Django's IP is?
Jacob
No, I just wanted to add:
Of course we'd like the database backend API to be documented; it's on
my list of "internals" documents I want to start working on, and it's
fairly high up. I'm not, however, going to write it just because of
some vague legal threats of a has-been computer company.
Jacob
See also Zen of Python #14:
"Although that way may not be obvious at first unless you're Dutch."
Perhaps IBM needs more Dutchmen on its dev team?
Also, none of these concerns were raised six months ago when Antonio
Cangiano announced[1] that IBM would be contributing DB2 adapters for
Python and Python ORMs; Antonio, if you're on this list could you
chime in on this issue?
[1] http://antoniocangiano.com/2007/09/04/python-django-and-db2-status-update/
--
"Bureaucrat Conrad, you are technically correct -- the best kind of correct."
Not directly, no. Think it though, though. If Django's IP is clean,
then there's nothing wrong with looking, using, adopting, modifying,
or distributing Django. That's the point of the BSD API. If, however,
there's a problem with Django's IP, then looking at the tainted IP
could be construed as copyright infringement. Thus, if IBM really has
this policy, it must be because someone there distrusts Django's IP.
I'd encourage that someone to contact me directly instead of spreading
insinuations.
> Basically, it sounds like IBM is actually covering their own ass by
> not allowing their developers to look at existing code for fear of
> another lawsuit from someone like SCO.
Who won that lawsuit, again?
> I think thats why they "need" a public spec.
Again, when I write that spec it will be released under the same BSD
license as Django itself, and will contain the same IP as Django does.
If Django's IP is tainted, one level of indirection won't protect IBM.
> Remember, IBM employs hundreds of developers that work
> on open source projects including many who work on the Apache HTTP
> Server.
I'm curious how they do this without reading anyone else's code.
Jacob
I think it's coming up now because IBM has customers who want to use
Django and who are dropping DB2 in favor of an Open Source database.
IBM wants in on the hot action [*] but wants us to do the heavy
lifting. Yay.
Jacob
Thanks for jumping on here to help us figure this stuff out. I'd love
to see a DB2 adaptor make its way into Django or be developed as a
third-party project.
On Thu, Mar 20, 2008 at 11:42 AM, Alex P <apit...@ca.ibm.com> wrote:
> 1. IBM does indeed have an incredible large number of developers
> involved in the Open Source Software (OSS) development and one can
> easily find them everywhere from apache.org to Linux kernel group. The
> only difference between those fully OSS engaged developers and those
> in the Python, Ruby, PHP, Perl support for IBM Data Servers
> development is the scope of their work: the first are working solely
> on OSS development, while the later might be involved in proprietary
> developments, too. The idea is that no OSS code fragment gets spilled
> "contaminating" proprietary IBM developments. So, it's solely an IBM's
> IP pedigree preservation that could be at risk if some isolated
> developer, without even intention (!) codes some proprietary component
> in a similar way as something seen maybe years ago in an OSS
> component. I hope this clears up the concerns about any perceived
> insinuation; it's not the case.
I think I must be dense here 'cause I've read this paragraph about
three times and don't understand what you're saying. So is the idea
that you've got essentially a firewall between those contributing to
Open Source and those merely using it? If so, is the problem that
nobody in the former group wants to work on Django/DB2?
Or is it deeper, and is Antonio correct that no IBM developer is
allowed to look at open source?
Either way, how would a spec -- released under the *same license* --
be any different in terms of IP than the code? If I copy a DB backend
into a .txt file and call it a spec can we move forward? If I rewrite
it in psudocode then is it OK?
> 2. What Python (and Ruby, PHP, etc.) developers in IBM are really
> looking for is some level of development doc/guide that would get them
> started (being able to start running compliance test suite).
See, this is a different problem then "we can't help Django because of
IP problems". Yes, we need to document the internals better; that's
never been in doubt. What I'm cheesed about is mixing that issue up
with all this IP business.
Which is the real problem?
> And yes, all these developers
> are indeed in a very awkward, blind position that no OSS developer
> would envy, and they usually ask: "why don't you check the existing
> code, mate?"... well, yes one would think that to be easiest thing to
> do, but that would expose one to the risk of later "contaminating"
> some proprietary code with some OSS bits, and that is certainly
> unacceptable, right?
Once again, Django's license explicitly allows this kind of
derivation. The BSD license is, in general, the most reuse-friendly
license you'll find. That is, you *have the copyright owner's
permission* to read the code. Do you think we're not qualified to give
that permssion because of IP provenance?
Also, please give my condolences to those poor developers. That's a
ridiculous position for IBM to put them in. Mention to them that there
are dozens of jobs coding Python and Django at companies without such
silly restrictions and that I'd be happy to help them make that move.
> In the end, to provide a concrete example of how members of the
> community have helped the "blind" IBM developers implement SQLAlchemy
> adapter: they started by just listing the subset of API that need to
> be implemented (pointing to the signatures and API contract), and also
> mentioning default APIs that likely needed to be overridden.
Right now, for better or for worse, the spec is the existing code. IIf
you're concerned about IP, you're fucked. No spec we release will have
different IP provenance, so I still don't see how a spec would help.
If, howeer, you just want better documentation, then it's message
heard loud and clear. Let's all shut up about IP and write some
documentation :)
Jacob
As Jacob has pointed out, any documentation you receive will be under
the BSD license. If your developers aren't allowed to look at
BSD-licensed content for fear of getting contaminated with its evil
permissiveness, too bad for you, and honestly I wonder how IBM gets
anything done -- don't you kind of have to have a TCP/IP stack at some
point?
> 2. What Python (and Ruby, PHP, etc.) developers in IBM are really
> looking for is some level of development doc/guide that would get them
> started (being able to start running compliance test suite).
There's a dummy backend in django/db/backends which simply raises
errors on all the parts of the API. Of course, you say you're not
allowed to look at that.
> And yes, all these developers
> are indeed in a very awkward, blind position that no OSS developer
> would envy, and they usually ask: "why don't you check the existing
> code, mate?"... well, yes one would think that to be easiest thing to
> do, but that would expose one to the risk of later "contaminating"
> some proprietary code with some OSS bits, and that is certainly
> unacceptable, right?
Awkward and blind? Not really. Try "impossible".
> Hopefully this is clearing up some of the concerns on this issue in
> this community, and I also hope our Dutch gets better with time :-)
Again: any docs you'll get will be BSD licensed, because the docs will
be part of Django and Django is BSD licensed. If you can't look at
anything that's BSD licensed, that's your problem and your product --
DB2 -- will get left behind.
And, again, why didn't Antonio raise these issues six months ago when
he announced IBM would be working on these adapters?
Question of course is, once they write the adapter with the
psuedo-code, how will they test it ? Wont they they accidentally see
the code in the debugger, or when they add print statements all over
the codebase ?
Regards
Rajeev J Sebastian
Thanks for the clarification -- I understand now. Or rather, I don't
really understand why they continue to operate this way, but whatever,
not my fight.
> To address James's post, I think they're not allowed to look at BSD licensed
> source code. BSD licensed documents are fine. See: Giant blanket statement.
IANAL, so I'm asking this sincerely: is there a difference between
code and English such that IP provenance isn't an issue in a spec but
is in code?
Jacob
Any such document provided by Django would be BSD licensed. You have
indicated that you are not able to look at any material which is BSD
licensed, for fear of contamination.
Unless you actually *are* allowed to look at BSD licensed material in
some cases, in which case something doesn't add up here.
Um, have you *seen* our documentation yet? It's chock-full of example
code. I'm not a good enough technical writer to write about code
without writing code.
If this is really your policy, your developers are violating it
whenever they open the Django documentation, read the book I wrote, or
participate on this mailing list or in IRC. All contain code taken
directly from the Open Source projects.
When I write the db backend documentation -- which, by the way, I just
started; thanks for the push! -- I'm going to use code to explain
things better explained in code. If IBM seriously can't deal with
that, it's not our problem; it's yours.
Jacob
A better summary: Employers. You can always find another one ;)
Maybe I can distill this: the "expected vendor API signature and
contract" will be part of BSD-licensed documentation which will
include BSD-licensed code. There will be no public domain
documentation. Unless they can look at BSD-licensed documentation
(which will include BSD-licensed code), Django cannot help them.
Period.
::tries not to headdesk::
Can you point me at *any* documentation of Open Source that conforms
to this? I can't think of a single OSS project that somehow manages to
document their internals without using any code examples. Python's
documentation contains source, as does Linux's, PostgreSQL's, MySQL's,
and all the other ones I've ever seen.
I *still* can't believe that IBM seriously expects to operate in this
way. If this really is IBM's corporate policy w/r/t Open Source y'all
are in a world of hurt.
Jacob
If we're your vendor, when do we get a check for our work? ;)
The thing is, though, that there *is* a minimal API documented:
django.db.backends.dummy. If this were Java, you could take that as
laying out the interface to be implemented. The confusion came from
the apparent refusal to deal with this because it's in the form of
open-source code.
-- Ned Batchelder, http://nedbatchelder.com
For more information, the SQLite backend is nicely implemented, and
fills in quite a bit if/when needed. And, amusingly, SQLite itself has
a policy of only implementing techniques that are demonstrably older
than the term of patent, so it's a great little DB to study for people
who have IP concerns ;)