GSOC2012 Proposal for 'Finishing off the App Refactor'
Nauho Zen
---------------
Django currently assumes that an application will only ever be loaded
once, and that
the name of that application will be determined solely by the package
name holding
the models.py. A gread idea has been proposed as a GSOC idea in
2010[1] and continued
in 2011[2], which will improve the old app loading mechism a lot, e.g.
deploy several
instances of the same application, deploy two applications with the
same name, support
convenient interface for internationalizing application names, etc.
However, the [2]
branch is not well prepared for being merged to the master branch, and
work been made in
2010 and 2011 towards these feathers has made some big changes to
current codebase, see[3].
So it's very necessary to check the changes and make sure whether
these work can be merged
and everything runs smoothly or not after merged.
Introduction
-----------------
Because the purpose is to check if everything runs ok after merged, so
we
should know what kind of features current app loading mechanism
supports and what kind of
improvements 'future' advanced work has made.
1 current app loading features
1) app can be reused in multiple projects
2) reused app can be found by Django in INSTALLED_APPS of settings.py,
which is written in
dotted path
3) each string in INSTALLED_APPS should be a full Python path to a
Python package that
contains a Django application, as created by django-admin.py startapp
4) app names must be unique
2 'future' app loading features
1) backward compatibility: 'future' app loading mechanism should
support current app
loading features well
2) can deploy several instaces of the same application
3) can deploy two applications with the same name,(not have the
requirements of unique
app name)
4) convenient interface for internationalizing application names
5) good support to rename an application with a name that isn't
helpful from a UI
perspective
While,how to check these above, I think the best tool we can use is
unit-test. First,
we should check if all the funtions(get_apps, get_app,get_models,
etc.) still work after
these changes, thus the test cases towards these funtions should be
run again to see if all
is ok. Second, good use cases should be establised to check if it
still supports current
features well; we can borrow these use cases from previous work I
think. Third, also
construct good use cases be used to check if will-be-added features
work well after merged,
we can reuse the tests new app-loading has offered.
But to the unittest point of view, we should pay more attention on the
files which has been modified.
Specifically, except for the unittest files, in [3], 6 files are
added, 20 files are modifiled.
Thus one part important work is to check if file changes does make bad
or unexpected
consequenses or not. Towards unittest files, 21 files are added, 1
file is removed, 6 files
are modified, thus we also should check after these changes, whether
tests still cover old
app loading unit test or not, what's more, cover new features has been
implemented in 2010 and 2011
well or not.
Detailed Plan:
Week 1: Read the source code corresponding with app loading of current
Django thoroughly,
to know best what current app loading does and how it does
Week 2: Read the source code of new app loading code, to check whether
it implement the
new features interested developers proposed before [4]
Week 3: Try to merge and check if old test cases can all be run
successfully, and make
some necessary improvement work
Week 4-5: Establish good use cases to check if old app loading
features are all not been
destroyed, meanwhile do necessary modification work
Week 6: Check all new test cases can be run sucessfully or not, and
make some necessary
work to help pass all tests
Week 7-8: Construct use cases to check how many new features have
already been implemented,
and if there are some interesting features that should be added, then
I can do neccessary
coding work
Week 9: make up current not well-done coding and tests work to make
app loading run perfectly
Week 10: Begin to create patches and create/write the documention
Week 11-12: Investigate the Django tickets host and contact with
corresponding and interested
developers to know if there are some necessary work or changes should
be added , and if all
is ok, try to begin to submit the patches to Django
About me:
I am a student from China, and have about 3 years Python programming
skill and uses Django
for 2 years. I love this kind of activity Google has offered and am
very interested in
communicating with open source guys all over the world. Hope I can
make some contributions
to Django through this wonderful activity.
Email: zenn...@gmail.com
IRC: zennauho
Links:
[1] https://code.djangoproject.com/wiki/SummerOfCode2010#Apploading
[2]https://github.com/jezdez/django/commits/app-loading
[3]https://github.com/jezdez/django/compare/master...app-loading
[4]https://code.djangoproject.com/wiki/InstalledAppsRevision
-------------------------------------------------------
I will appreciate that anyone can make some suggestiones about this
proposal. We have all
the same purpose to make Django better and better.
Best wishes to everyone!
nauho
Nauho Zen
ptone
On Monday, March 26, 2012 7:57:02 PM UTC-7, Nauho Zen wrote:
I have reformated my proposal content on this group, any suggestion?
Introduction
-----------------
Because the purpose is to check if everything runs ok after merged, so
we should know what kind of features current app loading mechanism
supports and what kind of improvements 'future' advanced work has
made.
1 current app loading features
1) app can be reused in multiple projects
2) reused app can be found by Django in INSTALLED_APPS of settings.py,
which is written in dotted path
3) each string in INSTALLED_APPS should be a full Python path to a
Python package that contains a Django application, as created by
django-admin.py startapp
4) app names must be unique
2 'future' app loading features
1) backward compatibility: 'future' app loading mechanism should
support current app loading features well
2) can deploy several instaces of the same application
3) can deploy two applications with the same name,(not have the
requirements of unique app name)
4) convenient interface for internationalizing application names
5) good support to rename an application with a name that isn't
helpful from a UI
perspective
Nauho Zen
Thank your for taking the time to consider this project, hopefully you will get some other feedback than just mine - I'm not sure that many projects are worthy of *two* GSOC projects myself.
And this idea has been listed on the Django GSOC-Idea page, which has a little high priority, so I think it may
be a good summer project for student to complete it to a good status. Take a look at:
https://code.djangoproject.com/wiki/SummerOfCode2012
I believe that the baton was somewhat passed to tswicegood at pycon, see:
I will take a good look at it.
1 current app loading features
1) app can be reused in multiple projectsnot sure how this is related to app_loading, it is the case currently
thus can allow same app to be used in different projects.
2) reused app can be found by Django in INSTALLED_APPS of settings.py,
which is written in dotted path
3) each string in INSTALLED_APPS should be a full Python path to a
Python package that contains a Django application, as created by
django-admin.py startappWhile I think this should still be true, currently there is the option for an iterable for defining an app, there was an "APP_CLASSES" setting in the branch transiently, but is now gone. I think it would be better to keep INSTALLED_APPS flat and introduce a new setting for configured apps - which could then add their flat representation to INSTALLED_APPS
settings.INSTALLED_APPS = ('django.contrib.auth',
App('django.contrib.admin', name='SuperAdmin'),
App('myapp', name='Wow', db_prefix='pref'))
But I am still not sure I should consider this stuff in this summer proposal or project, because it has been implemented in a
raw way through the work in 2010 and 2011 I think, what I do is to check if it can well do this and other good things as expected. However, if it does not perform well, I still need to enhance the code.
4) app names must be uniqueThis is, as far as I can tell from my review, and unresolved limitation in the current branch - that django.contrib.auth and mypackage.auth collide. Seems a duplicate of your #3 below.
2 'future' app loading features
1) backward compatibility: 'future' app loading mechanism should
support current app loading features wellThis backwards compatibility is present currently as a set of module level functions at the old location in the db module
Sounds good.
2) can deploy several instaces of the same applicationIf you see the discussion here:It was determined that this feature was too problematic to tackle as part of app_loading
3) can deploy two applications with the same name,(not have the
requirements of unique app name)There remains some room for improvement and clarification about what an app_name is vs app_label vs fully qualified name
4) convenient interface for internationalizing application names5) good support to rename an application with a name that isn't
helpful from a UI
perspectiveSome of the above needs clarification.I do think app_loading needs some sort of new push - but I'm dubious that a GSOC would be the way to go.
work can be continued after summer.
For your detailed plan, I would complete your items for week 1 and 2 before writing a final proposal, not as part of your proposal.
OK, I will spend more time on the week1and week2 work before summer coding begins.
At last, thank you very much to you for these good suggestions, and I think it will help me a lot to make a good
proposal and maybe-good project in this summer.
Best regards,
nauho
Travis Swicegood
I believe that the baton was somewhat passed to tswicegood at pycon, see:
I will take a good look at it.
This is, as far as I can tell from my review, and unresolved limitation in the current branch - that django.contrib.auth and mypackage.auth collide. Seems a duplicate of your #3 below.OK, there is still not a workaround way?
2) can deploy several instaces of the same applicationIf you see the discussion here:It was determined that this feature was too problematic to tackle as part of app_loadingA good start point to check if previous work has solved this problem or not ?
Nauho Zen
> makes it into GSoC or not) is make sure that we keep the scope reigned in on
> this merge. From my point-of-view (and hopefully Jannis will correct me if
> I'm off) these are the things this branch should do:
>
> * Introduce the concept of an App object to represent each
> * Keep the concept of an `app_label` as just the last part of an app's
> module name (example: django.contrib.auth has an app_label of auth).
> * Be entirely backwards compatible
>
> The majority of the above work is already done. Right now the two biggest
> hurdles to getting the code merged in are:
>
> * Getting the main test suite running again. The tests break spectacularly
> with the new code. Some of the failures are tests using internal APIs that
> changed, other are valid public APIs that haven't been ported yet. Ideally
> this merge should be able to take place without a single test having to be
> changed.
> * Buy-in from all of the core-devs that this is a needed and/or good change.
>
accepted to finish off the "App Refactor" work.
> All of this said, I'd love for this to be a GSoC project, but I agree
> Preston that yet-another-GSoC slot spent on this might not be the best use
> of the project's GSoC slots.
>
Well, as a candidate student, I still cann't judge the whole workload
should be paid
in this project and don't know whether if it make the best use of GSoC
slots. However, if because it has been taken as a project in previous
GSOC which make
a similar idea (finish off old work) not that significant to be a 2012
project, then
how about taking it as one part of 2012 work which I may charge if
accepted, and at the same time importing more interesting ideas that
can help my proposal make the best use of our slots?
Travis and Preston, what's your opinion?
> There is not a way to work around this. The app-loading branch provides a
> path forward, but does not take it. I don't think this pull request should
> fix this problem.
OK. I will consider that.
> Dealing with application conflicts requires a broader design decision on
> what constitutes an application's "name": it's app_label
> ("django.contrib.auth".split(".")[-1]), or it's full name
> ("django.contrib.auth"). The biggest issue is backwards compatibility -- we
> have to keep app_label around for awhile, so we need to deal with both. We
> can't immediately switch all table names around to refer to something other
> than the app_label.
OK.
> This is a problem and I recommend punting on it (see above).
OK.
Best regards,
nauho
Nauho Zen
> All of this said, I'd love for this to be a GSoC project, but I agree
> Preston that yet-another-GSoC slot spent on this might not be the best use
> of the project's GSoC slots.
>
How about combing the "Test framework cleanup" idea on the SummerOfCode2012 wiki page with
"Finishing off the App Refactor" to one proposal, which can make better use of our GSOC slots?
If you agree, I will refactor my proposal.
Best regards,
Travis Swicegood
Nauho Zen
> On Thursday, March 29, 2012 at 10:50 AM, Nauho Zen wrote:
> I think we should probably stick with one problem domain per-application
> just to keep things simple.
OK. Then I will enhance my proposal focusing on app-loading stuff and
submit it to google-melange. Hope the possible will-being summer
project will help Django a lot, even it may occupy one GSOC slot.
Best regards,
nauho
Nauho Zen
Proposal For “Finishing off the App Refactor”
Abstract
---------------
Django currently assumes that an application will only ever be loaded once, and that the name of that application will be determined solely by the package name holding the models.py. A gread idea has been proposed as a GSOC idea in 2010[1] and continued in 2011[2], which will improve the old app loading mechism a lot, e.g. deploy several instances of the same application, deploy two applications with the same name, supportconvenient interface for internationalizing application names, etc. However, the [2]branch is not well prepared for being merged to the master branch, and work been made in 2010 and 2011 towards these feathers has made some big changes to current codebase, see[3]. So it's very necessary to check the changes and make sure whether these work can be merged and everything runs smoothly or not after merged.
Introduction
-----------------
Because the purpose is to check if everything runs ok after merged, so we should know what kind of features current app loading mechanism supports and what kind of improvements 'future' advanced work has made.
1 current app loading features
1> reused app can be found by Django in INSTALLED_APPS of settings.py, which is written in dotted path
2> each string in INSTALLED_APPS should be a full Python path to a Python package that contains a Django application, as created by django-admin.py startapp
3> app names must be unique
2 'future' app loading features
1> backward compatibility: 'future' app loading mechanism should support current app loading features well
2> can deploy several instaces of the same application
3> can deploy two applications with the same name,(not have the requirements of unique app name)
4> convenient interface for internationalizing application names
5> good support to rename an application with a name that isn't helpful from a UI perspective
While,how to check these above, I think the best tool we can use is unit-test. First, we should check if all the funtions(get_apps, get_app,get_models, etc.) still work after these changes, thus the test cases towards these funtions should be run again to see if all is ok. Second, good use cases should be establised to check if it still supports current features well; we can borrow these use cases from previous work I think. Third, also construct good use cases be used to check if will-be-added features work well after merged, we can reuse the tests new app-loading has offered.
But to the unittest point of view, we should pay more attention on the files which has been modified. Specifically, except for the unittest files, in [3], 6 files are added, 20 files are modifiled. Thus one part important work is to check if file changes does make bad or unexpected consequenses or not. Towards unittest files, 21 files are added, 1 file is removed, 6 files are modified, thus we also should check after these changes, whether tests still cover old app loading unit test or not, what's more, cover new features has been implemented in 2010 and 2011 well or not.
The best place to start is the pull request: https://github.com/django/django/pull/129
This branch should do the following things in three aspects:
a) Introduce the concept of an App object to represent each
b) Keep the concept of an `app_label` as just the last part of an app's module name (example: django.contrib.auth has an app_label of auth).
c) Be entirely backwards compatible
The majority of the above work is already done. Two biggest hurdles should be considered to get the code merged in are:
a) Getting the main test suite running again. The tests break spectacularly with the new code. Some of the failures are tests using internal APIs that changed, other are valid public APIs that haven't been ported yet. Ideally this merge should be able to take place without a single test having to be changed.
b) Buy-in from all of the core-devs that this is a needed and/or good change.
According to the above considerations, the detailed plan is as follows:
Detailed Plan:
------------------------
Week 1: Try to merge and check if old test cases can all be run successfully, and make some necessary improvement work
Week 2-3: Establish good use cases to check if old app loading features are all not been destroyed, meanwhile do necessary modification work
Week 4: Check all new test cases can be run sucessfully or not, and make some necessary work to help pass all tests
Week 5-6: Construct use cases to check how many new features have already been implemented, and if there are some interesting features that should be added, then I can do necessary coding work
Week 7: Check if App object perform well or not and do necessary improvement work
Week 8: Check if the concept of ‘app_label’ perform well not (vs. ‘app_name’) and do necessary improvement work
Week 9: Make sure if the branch is entirely backwards compatible or not
Week 10 : Buy-in from all of the core-devs if needed
Week 11: Begin to create patches and create/write the documention
Week 12-Week 13: Investigate the Django tickets host and contact with corresponding and interested developers to know if there are some necessary work or changes should be added , and if all is ok, try to begin to submit the patches to Django
About me:
------------------------
I am an undergraduate student from China, and have about 3 years Python programming skill and uses Django for 2 years. I love this kind of activity Google has offered and am very interested in communicating with open source guys all over the world. Hope I can make some contributions to Django through this wonderful activity.
Email: zenn...@gmail.com
IRC: zennauho
Links:
------------------