Django 1.0 and multithreading issues.
Graham Dumpleton
of multithreading issues.
I note that all the links to tickets referenced in:
http://code.djangoproject.com/wiki/DjangoSpecifications/Core/Threading
seem to be crossed out, implying they are closed.
Does this mean we are closer to being confident in saying Django may
be multithread safe or are there other issues that still need to be
addressed?
Graham
Jeremy Dunck
<Graham.D...@gmail.com> wrote:
>
> Can someone give an update as to where Django 1.0 may stand in respect
> of multithreading issues.
>
> I note that all the links to tickets referenced in:
>
> http://code.djangoproject.com/wiki/DjangoSpecifications/Core/Threading
That's an interesting page I wasn't aware of.
Who is mrts? :)
Malcolm Tredinnick
Keep in mind that that page isn't neither a "specification" nor "core".
It's a scratch pad of ideas by a few people (hard to tell how many
because it's been edited anonymously at times). The naming is
unfortunate and something we'll probably change in the near future.
Definitely a useful page in some aspects, but to be treated
appropriately as with anything in the wiki: anyone can, and has edited
it, and it's difficult to view the history of what's gone on. At some
point we'll have to rename it because people have been confused by the
name.
To answer your actual question, we should be pretty much threadsafe
across the board. All tickets reported of that nature have been
addressed. There are some places where we do intentional sharing, but
they are just that: intentional. Other places should be filed as bugs,
because they most likely are. To the best of my knowledge (and I do have
some about the code), using multiple threads of execution for the same
virtual host / location config blog (i.e. one settings file) should
work.
There are still things going on like os.environ being updated in the
mod_python handler and Django having the general issue that it's going
to be much harder to run multiple issues in one process space (than
compared with your standard example of Trac) because people can and do
import pretty much anything and they all populate the same memory space.
That's why we don't currently have the multiple settings files for one
virtual host type of setup: it's going to take a lot of documentation
and care to make sure people have enough information to not blow their
own feet off by importing contradictory sets of things. As a low level
framework that is really just a small portion of any advanced usage, but
which does do dynamic loading, that's a project for later (still a
project, though, at least for me).
Regards,
Malcolm
mrts
The page was created and updated by me and I'm quite certain the
anonymous updates are also mainly by me. So I am entirely to blame for
the confusing naming and for any incorrect information. But I do hope
that the information is mostly correct and relevant, it is just
somewhat incomplete -- that's the main reason I never got to
announcing it here. As Malcolm pointed out, it is somewhat sketchy and
should be amended. But it is intended to be shared eventually, that's
why it's in Django wiki. Regarding Malcolm's concerns on whether it
somehow "looks" official due to naming -- as I've already iterated
many times previously, I never intended that and I accept whatever the
core devs decide in regard of renaming.
I really think something in the lines of that page -- sans the
automatic review results -- should be part of the original docs (docs/
internals/threading.txt perhaps, of course post-1.0), so that people
who contribute can be aware how threading is done in Django.
As for thread safety itself, the automatic review helped me track down
and patch two bugs -- #6950 and #7676. There was one spot in django/
db/models/fields/related.py I didn't analyze completely, quoting
myself "append() in add_lazy_relation() can add duplicated values,
which may or may not confuse pop() in do_pending_lookups()". Otherwise
all looked good when I reviewed the code. As you can see from the
page, the review was automatic and thus inherently incomplete -- I
only grepped for globals and reviewed the code around them. Class
attributes are reviewed to a lesser extent as tracking them is more
complicated.
And regarding thread-safety in practice: I have used Django with
mod_wsgi and threaded Apache for over a year with no ill effects (with
thorough testing as we did run an important project on threaded
Django). mod_wsgi is such a breath of fresh air after the heavy weight
of mod_python, everything runs lighter and faster and daemon mode +
user separation + touch the wsgi wrapper for reloading Python code has
made my life *considerably* easier -- so IMHO we should switch to
recommending mod_wsgi in docs instead of mod_python as the default
deployment option after 1.0.
P.S. I still think community design (wiki) is as important as are code
contributions (tickets and SVN), see http://mpt.net.nz/archive/2008/08/01/free-software-usability
for some points. Discussions here are a nice starting point, but they
tend to *diffuse* instead of *converging*. Wikis are good for
converging ideas and it has not to be a high-ritual process that most
coders loathe :). Currently the core spec process is either mostly
hidden from community -- devs meet in-person during conferences etc,
or is already done in practice this way -- both qs-rf and fs-rf had
extensive spec/doc pages. Why not make it semi-official with a PEP-
like process after 1.0?
Random collection of things that need specs for post-1.0:
* app handling -- there's lot to gain, think of e.g. extending
setuptools
* Identity Map (#17)
* multi-db support
* object history
On Aug 29, 3:32 am, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote: