SoC merge plans?
Jacob Kaplan-Moss
I'd like to start thinking about merging the SoC branches. As far as I
can tell we've got solid, worth-merging work out of all six of 'em --
WOOHOO! -- so let's cracking getting this stuff to trunk so people can
play with it.
I'd like to ask each mentor/student pair to give me a bit of information:
* A quick summary (more for the list than for me) of what new things
are on the branch, and give some instructions to people wanting to
review your code.
* A detailed explanation of *any* backwards-incompatible changes
merging your branch would make.
* Are all the technical aspects of your house in order? Is what's on
the branch at about the same level as we'd require from a patch (tests
passing, APIs complete, documentation included)? If not, what's left
to be done in this department?
* Is it going to be better to merge bits off the branch piecemeal, or
should we plan to merge the whole dang thing?
* Are there any other branches you're worried about comparability
with? Or any other sort of sequencing concerns we should take into
account?
I'm going to be reviewing the branches looking for these sorts of
things, and I'm going to try to work up a merge plan over the next
week or so.
Jacob
Alex Gaynor
On Sat, Sep 12, 2009 at 6:40 AM, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:
>
> Hi folks --
>
> I'd like to start thinking about merging the SoC branches. As far as I
> can tell we've got solid, worth-merging work out of all six of 'em --
> WOOHOO! -- so let's cracking getting this stuff to trunk so people can
> play with it.
>
> I'd like to ask each mentor/student pair to give me a bit of information:
>
> * A quick summary (more for the list than for me) of what new things
> are on the branch, and give some instructions to people wanting to
> review your code.
>
> * A detailed explanation of *any* backwards-incompatible changes
> merging your branch would make.
>
> * Are all the technical aspects of your house in order? Is what's on
> the branch at about the same level as we'd require from a patch (tests
> passing, APIs complete, documentation included)? If not, what's left
> to be done in this department?
>
to work on, Russ and I will be working on this stuff today hopefully.
> * Is it going to be better to merge bits off the branch piecemeal, or
> should we plan to merge the whole dang thing?
>
one that refactors many to many relations. The latter should be
merged before the former.
> * Are there any other branches you're worried about comparability
> with? Or any other sort of sequencing concerns we should take into
> account?
>
> I'm going to be reviewing the branches looking for these sorts of
> things, and I'm going to try to work up a merge plan over the next
> week or so.
>
> Jacob
>
> >
>
--
"I disapprove of what you say, but I will defend to the death your
right to say it." -- Voltaire
"The people's good is the highest law." -- Cicero
"Code can always be simpler than you think, but never as simple as you
want" -- Me
Honza Král
see inline answers for info about model-validation
On Sat, Sep 12, 2009 at 6:40 AM, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:
>
> Hi folks --
>
> I'd like to start thinking about merging the SoC branches. As far as I
> can tell we've got solid, worth-merging work out of all six of 'em --
> WOOHOO! -- so let's cracking getting this stuff to trunk so people can
> play with it.
>
> I'd like to ask each mentor/student pair to give me a bit of information:
>
> * A quick summary (more for the list than for me) of what new things
> are on the branch, and give some instructions to people wanting to
> review your code.
I added hooks on Model class to enable validation on Models. I changed
the way ModelForms work by having it call the actual Model's clean
method.
> * A detailed explanation of *any* backwards-incompatible changes
> merging your branch would make.
Everything should be pretty backwards compatible, I moved some stuff
around (old import paths may be brought back if needed).
The only thing that behaves differently is ModelForm and it's creation
in two ways:
a) ModelField.formfield() now returns just the most rudimentary
FormField to avoid doing complex validations twice. For example
EmailField.formfield() no longer returns EmailField but returns just a
CharField and let's the validation be run on the ModelField.
b) manipulating cleaned_data in .clean method of a ModelForm will no
longer influence the resulting model instance, instead you have to
modify the .instance attribute.
> * Are all the technical aspects of your house in order? Is what's on
> the branch at about the same level as we'd require from a patch (tests
> passing, APIs complete, documentation included)? If not, what's left
> to be done in this department?
The API is not 100% there, I don't believe we can make it 100% without
more people using the code. Also the documentation if not in ideal
order, I would appreciate any help there.
> * Is it going to be better to merge bits off the branch piecemeal, or
> should we plan to merge the whole dang thing?
Merging the whole dang thing makes much more sense
> * Are there any other branches you're worried about comparability
> with? Or any other sort of sequencing concerns we should take into
> account?
I am not aware of any potential conflicts.
> I'm going to be reviewing the branches looking for these sorts of
> things, and I'm going to try to work up a merge plan over the next
> week or so.
I will be out of contact for two or three weeks now traveling around,
I will be fully available to work on it after that.
Honza
Russell Keith-Magee
>
> Hey!
>
> On Sat, Sep 12, 2009 at 6:40 AM, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:
>>
>> Hi folks --
>>
>> I'd like to start thinking about merging the SoC branches. As far as I
>> can tell we've got solid, worth-merging work out of all six of 'em --
>> WOOHOO! -- so let's cracking getting this stuff to trunk so people can
>> play with it.
>>
>> I'd like to ask each mentor/student pair to give me a bit of information:
>>
>> * A quick summary (more for the list than for me) of what new things
>> are on the branch, and give some instructions to people wanting to
>> review your code.
>>
>
> Multiple database support is in my branch :)
To elaborate on Alex's extensive instructions...
The multi-db branch operates exactly the same as the old code for all
current use cases.
However, if you want to have access to multiple database instances at
runtime, you can:
1) Define a DATABASES dictionary. This replaces the DATABASE_NAME etc
settings, giving a collection of database configurations, keyed by
name. Using old-style DATABASE_* settings will raise a warning.
2) Issue a query directly on a specific database:
Author.objects.using('master').filter(name='fred') => Get all the
authors named fred on the master database
3) Default models to a specific database. This is done by adding
"using = 'master'" to the Meta definition for the model. e.g., if
Author had such a definition, then Author.objects.filter(name='fred')
would be issued on the Master database.
Full docs are available on the branch.
>> * A detailed explanation of *any* backwards-incompatible changes
>> merging your branch would make.
>>
>
> If you touch anything internal to Query, it may break.
There are also changes to some of the functions on Field
(get_db_prep_* etc), but these are handled with a warning on first
use. If you have any custom fields, you will need to update these
methods to accept a 'connection' argument.
>> * Are all the technical aspects of your house in order? Is what's on
>> the branch at about the same level as we'd require from a patch (tests
>> passing, APIs complete, documentation included)? If not, what's left
>> to be done in this department?
>>
>
> We have one last DDN issue to solve and some smaller technical stuff
> to work on, Russ and I will be working on this stuff today hopefully.
Although the branch isn't ready for merge yet, the code that is
present is all up to spec. I think the sprints have produced workable
solutions for the outstanding problems; fixes implementing the
solutions we discussed should be forthcoming shortly.
There are still one or two small issues regarding usage, but I suspect
that these just require more documentation for common use cases.
>> * Is it going to be better to merge bits off the branch piecemeal, or
>> should we plan to merge the whole dang thing?
>>
>
> I technically have 2 branches. One that does multiple database, and
> one that refactors many to many relations. The latter should be
> merged before the former.
I would also add that both of these branches require some work. An
idle discussion during the sprints pointed out a possible complication
with the m2m refactor that may require some attention. The multi-db
branch is still missing a couple of features, and requires updates as
soon as the m2m refactor lands.
As an optimistic estimate, I'd guess we're maybe a month or two out
from a final review, depending on how busy Alex and myself are.
>> * Are there any other branches you're worried about comparability
>> with? Or any other sort of sequencing concerns we should take into
>> account?
The only branch that gives me pause is the model validation branch. I
haven't dug into that branch in detail, but it's possible that some of
the model level checks could intersect with some of the multi-db
changes. There is also a 'validate' method that the multi-db code has
added - we need to make sure that this API doesn't clash, either
functionally or conceptually, with any APIs introduced by the model
validation code.
Russ %-)
Zain Memon
* A quick summary (more for the list than for me) of what new things
are on the branch, and give some instructions to people wanting to
review your code.
My code adds a few new things:
1) models.OrderField and ModelAdmin.order_field for reordering inlines
2) admin.SelectorInline for the new inline type
3) Javascript in the existing inline templates
* A detailed explanation of *any* backwards-incompatible changes
merging your branch would make.
I can't think of any backwards-incompatible changes. It was pretty easy to avoid breaking old things because I was mostly adding stuff.
* Are all the technical aspects of your house in order? Is what's on
the branch at about the same level as we'd require from a patch (tests
passing, APIs complete, documentation included)? If not, what's left
to be done in this department?
I've got tests and documentation scattered around, but they aren't yet at the level I'd like them to be (also, the documentation is on my blog and not actually checked in, or y'know, useful). I need to spend a couple weeks gathering those things together.
* Is it going to be better to merge bits off the branch piecemeal, or
should we plan to merge the whole dang thing?
It's probably less work to merge the whole thing, but we could ostensibly merge it piecemeal in 4 or 5 chunks.
* Are there any other branches you're worried about comparability
with? Or any other sort of sequencing concerns we should take into
account?
I'm sure Alex is planning nefarious deeds to back out JQuery if we merge my branch before his, but that's about it.
I'm going to be reviewing the branches looking for these sorts of
things, and I'm going to try to work up a merge plan over the next
week or so.
Jacob
Sweet. I'm around django-dev and on IM if you need me.
Zain
Simon Willison
wrote:
> present is all up to spec. I think the sprints have produced workable
> solutions for the outstanding problems; fixes implementing the
> solutions we discussed should be forthcoming shortly.
trunk /now/ and fixing the remaining outstanding issues there? It
might make it easier to get the other SoC branches compatible with
this one (which sounds to me like it makes the most far-reaching
changes) and it would encourage the wider community to help check that
everything really is backwards compatible.
<cowboy>
Russell Keith-Magee
Howdy Pardner... :-)
My hesitation here is that there one of the outstanding changes is a
fairly big refactor of the way the SQL backend handles queries, and I
want to make sure it's right before we inflict it on the world. In
particular, I need to make sure that the needs of Oracle and
contrib.GIS aren't broken by the refactor. I don't want to commit the
patch only to have to roll large parts of it back in the near future.
To keep your inner cowboy happy - I'm not actually anticipating that
much of an overlap problem with the other GSoC branches. Although the
changes are fairly extensive, they are mostly with internals, and not
in areas that the other GSoC branches should be touching, such as
database backends, details of connection handling, use of connections
by management commands.
The m2m refactoring patch (which is a prerequisite for the multi-db
work) could be committed now without any major problems.
Yours,
Russ (The lone cowboy of the West) %-)
Waldemar Kornewald
wrote:
> fairly big refactor of the way the SQL backend handles queries, and I
> want to make sure it's right before we inflict it on the world. In
> particular, I need to make sure that the needs of Oracle and
> contrib.GIS aren't broken by the refactor. I don't want to commit the
> patch only to have to roll large parts of it back in the near future.
query_class() function or whether to override the whole query module?
Any other issues?
> The m2m refactoring patch (which is a prerequisite for the multi-db
> work) could be committed now without any major problems.
the non-relational branch. Thanks!
Bye,
Waldemar Kornewald
Russell Keith-Magee
<wkorn...@gmail.com> wrote:
>
> On Sep 17, 1:48 pm, Russell Keith-Magee <freakboy3...@gmail.com>
> wrote:
>> My hesitation here is that there one of the outstanding changes is a
>> fairly big refactor of the way the SQL backend handles queries, and I
>> want to make sure it's right before we inflict it on the world. In
>> particular, I need to make sure that the needs of Oracle and
>> contrib.GIS aren't broken by the refactor. I don't want to commit the
>> patch only to have to roll large parts of it back in the near future.
>
> What exactly is still unclear? Are you not sure whether to have a
> query_class() function or whether to override the whole query module?
> Any other issues?
If you dig into the code around supporting Oracle, GIS and sql.Query
you will discover an interesting nest of vipers regarding the handling
of the sql.Query subclasses. Untangling that nest is not a trivial
operation, but it is something we need to do in order to make MultiDB
work. While Alex and I have a plan, we have not yet demonstrated that
the plan will work in practice. I don't want to commit anything to
trunk until I know that it will work.
Based on the discussion Alex and I had at the sprints, query_class()
will still exist, but it will play a slightly different role -
essentially, it will be more important for non-SQL backends than for
Oracle support.
I hope to have a chance to look at this over the next couple of weeks,
but this will be largely controlled by the amount of free time I have
and the level of activity on django-dev. Once we've got a solution
working, I'll post something on django-dev to let people know what is
going on.
>> The m2m refactoring patch (which is a prerequisite for the multi-db
>> work) could be committed now without any major problems.
>
> Then could you please commit the m2m patch? This would help us with
> the non-relational branch. Thanks!
The preliminary work is available on Alex Gaynors github account:
http://github.com/alex/django/tree/m2m-refactor
It will be in trunk as soon as I'm sure it's ready and I've got
another core developer to give the all clear.
Yours,
Russ Magee %-)
Waldemar Kornewald
wrote:
> will still exist, but it will play a slightly different role -
> essentially, it will be more important for non-SQL backends than for
> Oracle support.
it?
For example, I'm not sure if query_class() is such a good solution for
non-SQL backends. While it would work, query_class() makes things
unnecessarily complicated because you have to keep track of and cache
all your generated subclasses and the Query base class. Also, you'd
have to pass class names around (e.g., query_class('UpdateQuery'))
because non-SQL backends will have to use their own subquery base
class (e.g., nonrelational.subqueries instead of sql.subqueries), so
this makes query_class() not a great solution.
Wouldn't it be nicer to just return a complete module (query_module())
which contains all overridden classes? This way you won't have to
cache anything because the module is automatically cached by Python
and you don't need to check for subquery class names in the function
because you return all classes at once via the module. By default, the
module would just be django.db.models.sql.
Bye,
Waldemar Kornewald
Russell Keith-Magee
<wkorn...@gmail.com> wrote:
>
> On Sep 20, 1:54 pm, Russell Keith-Magee <freakboy3...@gmail.com>
> wrote:
>> Based on the discussion Alex and I had at the sprints, query_class()
>> will still exist, but it will play a slightly different role -
>> essentially, it will be more important for non-SQL backends than for
>> Oracle support.
>
> Could you please explain what you have in mind, so I can comment on
> it?
While I appreciate the enthusiasm, I need to repeat what I said
originally - this is a fairly big refactor of the way the SQL backend
handles queries, and I want to make sure it's right before we inflict
it on the world. This includes abstract discussions of what might be.
Give me a chance to flesh out the ideas that I have and make sure that
they work for the problems we already have (Oracle and GIS). Once I'm
sure we have got those issues under control, we can tackle the new
problems like supporting non-SQL backends.
Yours,
Russ Magee %-)