Cleaning up manage.py and import paths
Carl Meyer
Hash: SHA1
Hi all,
In the spirit of making Django behave better as a Python library (c.f.
Glyph's keynote at djangocon.us), I'd like to finally tackle removing
the sys.path hacks in django.core.management.setup_environ. I'll give
the full detailed rundown here on the current behavior and how I propose
to change it. Fortunately, the fix isn't that complicated, and I think
it's a no-brainer. For the impatient, you can skip straight to my
proposed patch [2].
The tl;dr summary is that I think with some small changes to manage.py
and the default project layout, we can fix some very common bugs and
deployment problems, dramatically reduce the extent to which manage.py
is "unknown magic" for newcomers to Django, and take a significant step
towards being "just a Python library."
This issue is tracked in ticket #15372 [1], though there's a lot more
detailed info in this message than on that ticket.
The problem
===========
Right now "django-admin.py startproject mysite" generates this:
mysite/
__init__.py
manage.py
settings.py
urls.py
This layout is the root of the problem, because it commingles manage.py
(a command-line script entry point that should not be imported) with
importable modules inside a package.
When Python runs a command-line script it puts the script's directory
onto sys.path (even if you run "python /full/path/to/mysite/manage.py"
from somewhere else entirely). So with this layout, via manage.py,
"import settings" and "import urls" will both work, and if we add any
other modules or packages to this directory, they are also importable as
top-level modules/packages. This is a standard feature of Python.
But "mysite" is a package, and Django wants us to be able to import
"mysite.settings" and "mysite.urls" (in fact, the latter is the default
ROOT_URLCONF setting). In order to make that possible, setup_environ
does some magic: it finds the parent directory of "mysite", temporarily
adds it to sys.path, imports mysite, and then reverts sys.path. Now we
can either "import settings" or "import mysite.settings", and both will
work. Same is true for any other modules or packages (e.g. apps) we add
under mysite/. How nice!
Not really. This bit of clever causes a boat-load of problems:
1. Python's importer doesn't know that the same module imported under
two different names is the same module, so it imports it twice. Which
means module-level code can run twice, which causes really confusing bugs.
2. People write code that imports things inconsistently (sometimes with
the project prefix, sometimes without it), and then when they go to
deploy (without manage.py or setup_environ), their imports break and
they don't understand why. This comes up frequently in #django. In order
to fix it they either have to make all their imports consistent, or add
both the inner and outer directories to sys.path/PYTHONPATH in their
deployment setup. Inconsistent imports should fail fast, not when you
move to production.
2a. Even worse, our tutorial now encourages people to make their imports
inconsistent! ROOT_URLCONF is "mysite.urls" but we tell people to use
just "polls" rather than "mysite.polls" - so any tutorial-based project
now requires the double import paths in order to even run.
3. Since it's not obvious what Django is doing, people don't always
realize that the name of the outer directory matters, they think it's
just a container (which is what it ought to be). If they commit the
startproject layout at the top of a VCS repo, then check it out under a
different name, imports break. (If they check it out under a name that
isn't a valid Python module name, setup_environ breaks entirely).
4. The temporary addition to sys.path can cause unrelated packages
adjacent to the project to be imported, if the import of the project
module triggers an import of a name that happens to exist adjacent. This
is extremely difficult to track down; since Django restores sys.path, it
appears the adjacent package is somehow imported without even being on
sys.path. This can catch very experienced Python devs; I once spent an
hour on IRC working with Ned Batchelder to track this down in his project.
5. Understanding Python import paths is hard enough for beginners. The
Django tutorial could help them on the road to understanding, with no
loss in usability; pulling a trick like this behind the scenes helps
them stay confused instead.
The solution
============
The key is to fix the "startproject" layout:
mysite/
manage.py
mysite/
__init__.py
settings.py
urls.py
The outer "mysite/" is now just a container, not a Python package. It
can be renamed without consequence. The inner "mysite/" is the Python
package, and contains settings and urls. This means they are no longer
importable as top-level modules; they must always be "mysite.settings"
and "mysite.urls", which is better than having them pollute the
top-level namespace. Python's built-in sys.path handling for scripts
means that sys.path will be correct using manage.py as the entry-point
(or the wsgi.py entry point that may be added shortly for #16360), and
people not using either of those entry points only need to add one
sys.path entry rather than two.
With that simple change, setup_environ and execute_manager (in
django.core.management) both become superfluous, and manage.py can be
made much simpler. Since it no longer needs to muck with sys.path, its
only remaining function is to set a default value for
DJANGO_SETTINGS_MODULE, if it isn't already set. This also gets rid of
the double import of the settings module that manage.py currently
causes, which Graham Dumpleton discusses here [3]. And it's now obvious
from inspection what exactly manage.py does, which certainly wasn't true
before.
All win so far. But what about...
Backwards compatibility
- -----------------------
For new projects, there's no backward-compat issue. For existing
projects that upgrade to Django 1.4, we leave setup_environ and
execute_manager in place (so the old-style manage.py will continue to
work as it always did), but we start them on a deprecation path.
When the project decides to make the update (possibly when they read the
1.4 release notes, probably when they start getting loud
DeprecationWarning in 1.5), here's what they need to do:
1. Replace the contents of manage.py with the new version (which is five
lines long and can be included in its entirety in the release notes).
2. Make their imports consistent. This means that anything imported with
the project-module prefix needs to live inside the project module, and
anything imported as a top-level module (that isn't a
separately-installed dependency) needs to live outside the project
module next to manage.py. And, of course, a given module needs to
consistently be imported one way or the other.
I think this is a reasonable amount of work for a deprecation upgrade,
and a good trade-off for the benefits. Really, step 2 is cleanup of
broken code where manage.py is currently hiding its brokenness.
Other issues
============
What to do about startapp?
- --------------------------
Currently, startapp always creates the new app inside the "project
directory", currently defined as "the directory where the settings
module lives" (and found via a combination of baling wire and ugly
hackery). If we want to maintain this behavior of startapp, we need to
change the tutorial back to using "mysite.polls" in imports rather than
"polls". I have a version of my proposed patch that does this [4].
My preference, however, and what I've implemented in my preferred patch
[2], is to change startapp so that it always creates the app in the
current directory, rather than jumping through hoops to find the
"project directory". This makes startapp useful for starting reusable
apps as well as project-tied apps, and allows us to keep "polls" in the
tutorial rather than "mysite.polls" (because if the tutorial tells the
user to run "python manage.py startapp polls", the polls app will be
created next to manage.py, which means it's imported as a top-level
package).
Removing the restrictions on startproject
- -----------------------------------------
My patch also removes the current restriction that "startproject" cannot
be run if there is a DJANGO_SETTINGS_MODULE environment variable. I
don't think there's any good reason for this restriction, and it can be
really irritating and confusing. (This also means you can now run
startproject via manage.py, which doesn't make a ton of sense, but also
doesn't actually hurt anything).
What's the impact on adding a standard WSGI entrypoint?
- -------------------------------------------------------
#16360 [5] has a patch to add a "wsgi.py" file to the default project,
which will serve as a standard WSGI entrypoint that can be used both by
runserver and by production WSGI servers. I love the idea and the patch
- - except that its approach to easing the deployment path is to extend
the current sys.path-hacking of setup_environ beyond just manage.py and
into every WSGI deployment.
But if we first fix the sys.path-hacking as I propose, the wsgi.py file
will go next to manage.py in the default project layout. Since WSGI
servers chdir() to the directory of the wsgi file you point them at, no
sys.path additions (whether manually or magically in Django) will be
needed in order to deploy a Django project under a production WSGI
server. Again, win.
Next steps
==========
Unless there are significant objections, I'd like to commit this in the
next few days and get it into 1.4. I'll also modify the patch on #16360
to take advantage of the new layout; hopefully that can get into 1.4 as
well.
Let me know if you see something I've overlooked!
Thanks,
Carl
[1] https://code.djangoproject.com/ticket/15372
[2] https://github.com/django/django/pull/62
[3] http://blog.dscpl.com.au/2010/03/improved-wsgi-script-for-use-with.html
[4] https://github.com/carljm/django/compare/master...t15372-app-in-project
[5] https://code.djangoproject.com/ticket/16360
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6TbDIACgkQ8W4rlRKtE2dhCQCgzoXxt245wB7UVkaDeeK0U7NO
2hsAn2Wx90fdbJte2lPfW4onyoIqbyf0
=MUge
-----END PGP SIGNATURE-----
Dan Poirier
> In the spirit of making Django behave better as a Python library (c.f.
> Glyph's keynote at djangocon.us), I'd like to finally tackle removing
> the sys.path hacks in django.core.management.setup_environ.
Yes! I can't wait. I cannot count the hours I wasted in confusion
caused by these inconsistencies.
Luke Plant
> Unless there are significant objections, I'd like to commit this in the
> next few days and get it into 1.4. I'll also modify the patch on #16360
> to take advantage of the new layout; hopefully that can get into 1.4 as
> well.
+1. Spent hours cleaning up a project with inconsistent imports
recently. And everything you suggested is how I'd like to see it done.
Luke
--
"My middle name is 'Organised'! My first name is 'Poorly'."
Luke Plant || http://lukeplant.me.uk/
TiNo
On 10/10/11 23:05, Carl Meyer wrote:
> Unless there are significant objections, I'd like to commit this in the
> next few days and get it into 1.4. I'll also modify the patch on #16360
> to take advantage of the new layout; hopefully that can get into 1.4 as
> well.
Aymeric Augustin
> Unless there are significant objections, I'd like to commit this in the
> next few days and get it into 1.4. I'll also modify the patch on #16360
> to take advantage of the new layout; hopefully that can get into 1.4 as
> well.
>
> Let me know if you see something I've overlooked!
Carl,
+1 from me too.
The new structure highlights that a Django project is:
- an entrypoint for scripts manage.py — and soon another one for WSGI servers,
- a settings module and a root URLconf, who happen to live in the same
package by default,
- some applications, templates, static files, etc.
I was about to ask a question about the recommended project layout but
I eventually found the answer here:
https://github.com/carljm/django/commit/4ba999060d8c030dd43ef4a8ec2844850f0310b3#L2R380
This information will still be interesting and useful long after the
1.4 release. Shouldn't we move it somewhere in the documentation and
link to that page from the release notes?
Also, following the Zen of Python, shouldn't we recommend the
decoupled style (ie. put apps in the outer "mysite" directory)?
Finally, if I understand correctly, the only reason why the default
settings and URLconf are in a package ("mysite.settings" and
"mysite.urls") is to avoid name conflicts. In particular, it makes it
possible to run several Django projects in parallel with the same
PYTHONPATH. It'd be worth mentioning this in the documentation, so
that newcomers understand there's nothing magic with the name of the
inner "mysite" directory.
I believe all this could go either in the tutorial or in a new page
dedicated to project layout — the latter would allow us to give more
details on the project layout options.
Thanks for your work on this long-standing issue!
--
Aymeric.
Hanne Moa
> On 10/10/11 23:05, Carl Meyer wrote:
>
>> Unless there are significant objections, I'd like to commit this in the
>> next few days and get it into 1.4. I'll also modify the patch on #16360
>> to take advantage of the new layout; hopefully that can get into 1.4 as
>> well.
I already do it this way except the manage.py is vanilla.
project-package/
project.wsgi
requirements.txt
setup.py
.git
bin/ # virtualenv
...
project/
__init__.py
manage.py
settings.py
urls.py
...
HM
Carl Meyer
Hash: SHA1
Hi TiNo,
On 10/11/2011 06:10 AM, TiNo wrote:
> Definitely +1, as this has bitten me far too often. It would be nice
> however to also depend less on the ('hacky', as Glyph also noted)
> DJANGO_SETTINGS_MODULE environment variable. We shouldn't need an env
> variable to run a python website, and it certaintly shouldn't screw up
> settings loading when set accidently.
I'd certainly like to get rid of DJANGO_SETTINGS_MODULE in the long run,
but I don't see how we can do so without also getting rid of
process-global settings, and that's a much bigger project and patch, far
out of scope for what I'm attempting here.
As long as we have process-global settings, we need an explicit way for
users to specify which settings module they want to use, that is
possible to provide across a variety of different ways of invoking a
process (including e.g. having it implicitly invoked by mod_wsgi, in
which case some other options, like command-line parameters, are not
available). Having multiple settings modules available and picking one
for a given invocation is a feature, and an important one for many
people. If you have a proposal for a better way to do that than
DJANGO_SETTINGS_MODULE, that can be implemented in a
backwards-compatible way, we can certainly discuss it - but it doesn't
need to hold up this patch.
The new proposed setup_settings
> function accepts a path, but listens to the env variable first, and if
> that is not set it it wil still try to load 'settings'. Why not listen
> to the path that is provided so I can pass 'mysite.settings.production'
> for example? And if it's not provided, listen to DJANGO_SETTINGS_MODULE
> for backwards compatibility.
The "setup_settings" function is not a part of my proposed patch for
#15372 (cleaning up sys.path handling), it's part of the current patch
for #16360 (wsgi entrypoint), and it's precisely that part of the latter
patch which I expect to modify once #15372 is fixed.
Once I'm done with that patch, I don't think the setup_settings function
will be needed anymore (as it's basically a wrapper around
setup_environ, which will be deprecated). Instead, I expect wsgi.py will
simply contain a line very similar to what is currently in manage.py,
"os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.settings')".
This is much simpler and more explicit than setup_settings.
This still gives DJANGO_SETTINGS_MODULE priority, and just sets a
fallback value - I don't think we can change that in the default
manage.py or wsgi.py, without breaking the ability for people to
override using DJANGO_SETTINGS_MODULE if they want, which is the
established pattern. However, the advantage of it being explicit is that
you can easily change the behavior if you want. If you want the
environment variable ignored, you can change that line to
"os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'" in your
project, and then Django will always use "mysite.settings", regardless
of the environment variable's value.
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6UYhIACgkQ8W4rlRKtE2d9xACglQHuQsuqk2DCUzWJVILTuv56
C84AoLRFUE4Z4hCwQxbGQa1VfUN5tdPh
=Y4O5
-----END PGP SIGNATURE-----
Carl Meyer
Hash: SHA1
On 10/11/2011 09:34 AM, Carl Meyer wrote:
> setup_environ, which will be deprecated). Instead, I expect wsgi.py will
> simply contain a line very similar to what is currently in manage.py,
> "os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.settings')".
I should say rather, "what is in the new manage.py in my #15372 patch",
not "what is currently in manage.py".
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6UYr4ACgkQ8W4rlRKtE2ffDQCgubhrYgsp5u6pDmTfOnV1aVxg
KqUAn31gJXw8GxbvYOBASy6lqYpIsk42
=XPTp
-----END PGP SIGNATURE-----
Carl Meyer
Hash: SHA1
Hi Hanne,
On 10/11/2011 06:31 AM, Hanne Moa wrote:
> I already do it this way except the manage.py is vanilla.
>
> project-package/
> project.wsgi
> requirements.txt
> setup.py
> .git
> bin/ # virtualenv
> ...
> project/
> __init__.py
> manage.py
> settings.py
> urls.py
> ...
I think many people are doing something similar to this - just taking
the current startproject layout and wrapping it in an outer container
directory. This solves problems 3 and (partly) 4, out of the five I
listed. It means renaming the outermost directory no longer causes
issues (#3), and it means that although you could have something
adjacent to "project/" accidentally imported without seeming to be on
sys.path, at least it'd be reliably reproduced because its a part of
your repo, not outside it (thus partly #4).
It doesn't do anything to solve 1, 2, or 5, because you still have the
sys.path hack in setup_environ and you still have the doubled import
paths, where you can import both "project.settings" and just "settings".
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6UY+AACgkQ8W4rlRKtE2f9bACfT5GLcOtz0RGWtMHhZYD0j2Zi
zJwAn1z8qfxw3m6JXEPtwbQL+6JPnSzr
=lnk9
-----END PGP SIGNATURE-----
Carl Meyer
Hash: SHA1
Hi Aymeric,
On 10/11/2011 06:12 AM, Aymeric Augustin wrote:
> The new structure highlights that a Django project is:
> - an entrypoint for scripts manage.py � and soon another one for WSGI servers,
> - a settings module and a root URLconf, who happen to live in the same
> package by default,
> - some applications, templates, static files, etc.
>
> I was about to ask a question about the recommended project layout but
> I eventually found the answer here:
> https://github.com/carljm/django/commit/4ba999060d8c030dd43ef4a8ec2844850f0310b3#L2R380
>
> This information will still be interesting and useful long after the
> 1.4 release. Shouldn't we move it somewhere in the documentation and
> link to that page from the release notes?
Yes, I think adding a bit more detailed information about project layout
and Python path issues to the documentation makes sense.
> Also, following the Zen of Python, shouldn't we recommend the
> decoupled style (ie. put apps in the outer "mysite" directory)?
With my proposed patch we already implicitly recommend this by modeling
it in the tutorial, but we don't explain it particularly.
Personally I don't think it's as simple as a universal recommendation.
In practice I find that some apps are potentially destined for reuse,
and with those I prefer to start out with a top-level namespace. A few
(often those containing core content models) are inextricably tied to
the particular site/project, and for those I prefer to namespace them
inside the project namespace.
I'm not sure if that level of discussion belongs in the documentation or
not. I can try and see how it reads.
> Finally, if I understand correctly, the only reason why the default
> settings and URLconf are in a package ("mysite.settings" and
> "mysite.urls") is to avoid name conflicts. In particular, it makes it
> possible to run several Django projects in parallel with the same
> PYTHONPATH. It'd be worth mentioning this in the documentation, so
> that newcomers understand there's nothing magic with the name of the
> inner "mysite" directory.
Yeah - that's tricky to describe in the docs, since you still can't run
multiple Django projects in parallel *in a single process* (until we get
rid of settings as a process-global), but it does allow you to run
multiple projects in different processes using the same PYTHONPATH
setup. I can try to discuss this as well and see how it looks.
> I believe all this could go either in the tutorial or in a new page
> dedicated to project layout � the latter would allow us to give more
> details on the project layout options.
I think we should add a dedicated "Python path and project layout" page,
and link to it from the tutorial and elsewhere. This gets way too
complex to load onto beginners right away in the tutorial.
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6UetEACgkQ8W4rlRKtE2eTxgCfYbzpvkYU5BIY87U4QU5Ucf/A
dWUAmQFtfgluu0Ubf/RXaK9X+f3DD3sd
=4v8O
-----END PGP SIGNATURE-----
Markus Gattol
I think we should add a dedicated "Python path and project layout" page,
and link to it from the tutorial and elsewhere. This gets way too
complex to load onto beginners right away in the tutorial.
example.com <---- the virtualenv, the outer container, result of mkvirtualenv example.com- manage.py- mysite/ <---- result of django-admin.py startproject mysite from within example.com (after pip install django)
Carl Meyer
Hash: SHA1
Hi Markus,
On 10/11/2011 11:50 AM, Markus Gattol wrote:
>
> I think we should add a dedicated "Python path and project layout" page,
> and link to it from the tutorial and elsewhere. This gets way too
> complex to load onto beginners right away in the tutorial.
>
> IIRC then this page has been proposed before and ultimately not been
> done because the rationale people presented was that, well, it
> depends...
Yes, I think there's a lot of "it depends" involved. I'm envisioning a
page that basically explains the default layout, why it works, why it's
done that way, and briefly discusses what directions you can go with it.
Nothing very extensive or highly prescriptive.
After talking to Aymeric more on IRC, we agreed that this added docs
page doesn't need to block the patch. The current documentation in the
patch is adequate, this page would be an additional improvement. Which
is good, because I'm not sure I have time to write that doc page right
now :-) So I'm planning to commit the patch as-is and file a ticket for
more docs with a link to this discussion.
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEUEARECAAYFAk6Ung0ACgkQ8W4rlRKtE2fb7gCVF3SD9aCilRUk6ieCN4Qs8wjW
2ACff+HtJSl1IE8iPNwPNVxMIl20D28=
=PCjx
-----END PGP SIGNATURE-----
Karen Tracey
After talking to Aymeric more on IRC, we agreed that this added docs
page doesn't need to block the patch. The current documentation in the
patch is adequate, this page would be an additional improvement. Which
is good, because I'm not sure I have time to write that doc page right
now :-) So I'm planning to commit the patch as-is and file a ticket for
more docs with a link to this discussion.
One thing that I think should be added before commit is a note in the tutorial along the lines of "what if startproject didn't actually do what we are saying it did?" and explaining that the structure has recently changed and if you didn't get the structure described here than you are likely using an older version of the code than the documentation you are reading and you should get the two in sync.
This was done back in the day of changing maxlength to max_length, and that note stayed in the tutorial a laughably long time after the change was no longer "recent", but it likely prevented lots of questions and duplicate bug reports about the tutorial being wrong.
Karen
Idan Gazit
holdenweb
standing problem, and moves things in a very positive direction.
S
Russell Keith-Magee
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi all,
>
> In the spirit of making Django behave better as a Python library (c.f.
> Glyph's keynote at djangocon.us), I'd like to finally tackle removing
> the sys.path hacks in django.core.management.setup_environ. I'll give
> the full detailed rundown here on the current behavior and how I propose
> to change it. Fortunately, the fix isn't that complicated, and I think
> it's a no-brainer. For the impatient, you can skip straight to my
> proposed patch [2].
>
> The tl;dr summary is that I think with some small changes to manage.py
> and the default project layout, we can fix some very common bugs and
> deployment problems, dramatically reduce the extent to which manage.py
> is "unknown magic" for newcomers to Django, and take a significant step
> towards being "just a Python library."
Pile on another +1 from me too. This looks like an extremely elegant
solution to something that has been a wart for a long time.
My only feedback on the patch is a point of clarification in the
tutorial. Rather than creating a mysite directory with a mysite
project directory in it, and then having to refer to the "inner/outer
directory" or "the directory with manage.py in it", it strikes me that
it might be cleaner to name the outer directory something generic
(like "django_tutorial"). This reinforces that the outer directory
name doesn't matter, and that startproject is only creating the inner
directory.
Other than that, it looks great to me.
Russ %-)
Graham Dumpleton
On Oct 11, 9:05 am, Carl Meyer <c...@oddbird.net> wrote:
> What's the impact on adding a standard WSGI entrypoint?
> - -------------------------------------------------------
>
> #16360 [5] has a patch to add a "wsgi.py" file to the default project,
> which will serve as a standard WSGI entrypoint that can be used both by
> runserver and by production WSGI servers. I love the idea and the patch
> - - except that its approach to easing the deployment path is to extend
> the current sys.path-hacking of setup_environ beyond just manage.py and
> into every WSGI deployment.
>
> But if we first fix the sys.path-hacking as I propose, the wsgi.py file
> will go next to manage.py in the default project layout. Since WSGI
> servers chdir() to the directory of the wsgi file you point them at, no
> sys.path additions (whether manually or magically in Django) will be
> needed in order to deploy a Django project under a production WSGI
> server. Again, win.
where the WSGI file is located so that will not work for Apache/
mod_wsgi. A sys.path modification would still be need.
Graham
Mateusz Harasymczuk
example.com
- contrib
- docs
- public
- src
- manage.py
- myapp
- __init__.py
- settings.py
- urls.py
- virtualenv
Jeremy Dunck
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 2. People write code that imports things inconsistently (sometimes with
> the project prefix, sometimes without it), and then when they go to
> deploy (without manage.py or setup_environ), their imports break and
> they don't understand why. This comes up frequently in #django. In order
> to fix it they either have to make all their imports consistent, or add
> both the inner and outer directories to sys.path/PYTHONPATH in their
> deployment setup. Inconsistent imports should fail fast, not when you
> move to production.
If anyone is interested in this specific problem, I've written a
script to highlight places your sys.path allows aliasing of modules:
https://gist.github.com/857091
Alec Taylor
Backwards compatibility is easily solved, include an upgrade.py file
which is called if project rootfolder has settings.py. upgrade.py
would fix the directory structure
> --
> You received this message because you are subscribed to the Google Groups "Django developers" group.
> To post to this group, send email to django-d...@googlegroups.com.
> To unsubscribe from this group, send email to django-develop...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
>
>
Carl Meyer
Hash: SHA1
Hi Alec,
On 10/12/2011 06:38 AM, Alec Taylor wrote:
> Backwards compatibility is easily solved, include an upgrade.py file
> which is called if project rootfolder has settings.py. upgrade.py
> would fix the directory structure
Thanks for the suggestion! I don't think it's quite that simple, as we
can't assume that every upgrading project has maintained exactly the
default startproject layout - I think quite a significant percentage
don't. And we also don't know which style of import the project is using
(with or without project prefix) for various things, and inconsistent
imports would be tricky to fix automatically. We need to provide
migration help for all projects that currently work, not just for the
previous default layout.
I think we're better off carefully documenting how the location of
things in the new setup determines how they can be imported, and letting
projects make the necessary changes themselves, rather than trying to
provide an upgrade script that isn't likely to have a real high success
rate.
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6VjcUACgkQ8W4rlRKtE2fRqACeOAL+Etvd2n2IqyV/aUVPQ4EY
09wAoJqbQ6NcCH0QD46zE8q/pRCUE1Ad
=D0va
-----END PGP SIGNATURE-----
Brett H
Post 1.4 you actually might want to think a bit about integrating a
distutils2 setup.cfg into the manage.py folder and the startproject
command.
Since distutils2 will be the standard in python3.3 for packaging and
backported to python 2.5 it makes sense to leverage it for Django.
At the very least the setup.cfg could provide the
DJANGO_SETTINGS_MODULE env variable for installation scripts as a
custom meta-data variable that an installation script could do
something with.
It will probably be good practice to setup a minimal setup.cfg file
anyway with the project name.
My thoughts are that the whole Django project structure existed as a
way to get started quickly, and people replicated it on servers
because python packaging sucked.
Hopefully distutils2 will make it suck a little less and make it
feasible to make Django play nicely with it.
Brett H
Carl Meyer
Hash: SHA1
Hi Karen,
On 10/11/2011 06:21 PM, Karen Tracey wrote:
> One thing that I think should be added before commit is a note in the
> tutorial along the lines of "what if startproject didn't actually do
> what we are saying it did?" and explaining that the structure has
> recently changed and if you didn't get the structure described here than
> you are likely using an older version of the code than the documentation
> you are reading and you should get the two in sync.
>
> This was done back in the day of changing maxlength to max_length, and
> that note stayed in the tutorial a laughably long time after the change
> was no longer "recent", but it likely prevented lots of questions and
> duplicate bug reports about the tutorial being wrong.
Interesting, I didn't realize we had precedent for doing that
explicitly. I've added just such a note to the patch, thanks for the
suggestion.
Perhaps we should have done this for the "from django.conf.urls import
url, patterns" change, too :-)
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6Vk5wACgkQ8W4rlRKtE2c8qACfZwIXGuXrm8gxrnV/aPCqXJlr
UY8AoKt21Fh167sZfiZdy5O82iL07z5N
=POwS
-----END PGP SIGNATURE-----
Carl Meyer
Hash: SHA1
Hi Russ,
On 10/12/2011 05:14 AM, Russell Keith-Magee wrote:
> My only feedback on the patch is a point of clarification in the
> tutorial. Rather than creating a mysite directory with a mysite
> project directory in it, and then having to refer to the "inner/outer
> directory" or "the directory with manage.py in it", it strikes me that
> it might be cleaner to name the outer directory something generic
> (like "django_tutorial"). This reinforces that the outer directory
> name doesn't matter, and that startproject is only creating the inner
> directory.
So the repeated directory name does make the tutorial wording slightly
clumsy (though I don't think it's much of a problem otherwise). The
issue is that startproject _does_ in fact create both the outer and
inner directory; and I think it must do so, else its creating a
manage.py in the current directory, and we have to explicitly tell
people "first create an empty directory, then run startproject inside of
it" - I don't think that's a good plan.
I toyed with ideas like allowing startproject to take two arguments
instead of one, or allowing its argument to be an invalid Python module
name, using that for the outer directory, and then doing an automatic
slugify-like conversion to a valid Python module name for the inner
directory. In the end I decided it was simplest to just live with the
repeated directory name.
I suppose the one other thing we could do is explicitly have the user
rename the outer directory in the tutorial? This doesn't really seem
worth it, though...
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6VlOAACgkQ8W4rlRKtE2fWywCfZ/Lzj2F4AGUn6l4xo4rB1YTg
bkgAniAMyoP/S1zUUILEUKBeyVGgyhaZ
=m891
-----END PGP SIGNATURE-----
Carl Meyer
Hash: SHA1
Hi Graham,
On 10/12/2011 05:16 AM, Graham Dumpleton wrote:
> Apache/mod_wsgi does not change the current working directory to be
> where the WSGI file is located so that will not work for Apache/
> mod_wsgi. A sys.path modification would still be need.
Thanks for the correction, sorry to misinform. I'll make sure the
Apache/mod_wsgi docs are correct on this point when I update the patch
for #16360. Adding one directory to sys.path is still better than adding
two!
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6VlUEACgkQ8W4rlRKtE2flQwCgneEzv5tiKGPajdewDzUE4CpR
WFsAn394n7mUkP9xlOwkmslyufKKDdmM
=ja+f
-----END PGP SIGNATURE-----
Carl Meyer
Hash: SHA1
Hi Mateusz,
Thanks for the suggestion. I think this is out of scope for this patch.
I'm not opposed to further discussion of startproject enhancements and
additions, though I think we're going to have a hard time reaching
consensus on exactly what should be added and how, which is a good
argument for keeping it minimal. In any case, my goal at this point is
to make the minimal change necessary to fix the path issues.
The good news is that if you like this layout, it's trivial to create it
for your project using the new startproject, since you just create your
outer directories and move the result of startproject to src/.
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6VliYACgkQ8W4rlRKtE2d8/gCgzLVm8Ua/657FT0lz3L/HZLnN
YSAAn3TmCJS0Gm83wtvzJB73w+paUyfh
=84Ok
-----END PGP SIGNATURE-----
Carl Meyer
Hash: SHA1
Hi Brett,
On 10/12/2011 07:11 AM, Brett H wrote:
> Post 1.4 you actually might want to think a bit about integrating a
> distutils2 setup.cfg into the manage.py folder and the startproject
> command.
Oh, I certainly have thought about it :-) A number of people, myself
included, have played around with making projects installable, which is
an even more thorough solution to making sure its on sys.path properly.
Without that, we're still depending on either using an entrypoint script
in the correct directory, having the correct directory be the CWD, or
using PYTHONPATH or equivalent.
Currently, the main difficulty with installable projects is that it's
hard to do properly, because projects typically rely on a large number
of templates and static assets, and today in distutils/setuptools it's
quite painful to get all those listed correctly in your setup.py so
they'll be installed. I think a lot of people doing "installable
projects" just punt on that and install --editable and/or reference
static assets and templates from the original checkout rather than an
installed location. This is a bit janky, since normally when a Python
package is installed you expect it to come with everything it needs, and
not still be relying on stuff from the installation source.
With distutils2 it should be much easier to do it right, and I'm
certainly interested in the idea of adding a setup.cfg to the
startproject layout, and possibly as an option to startapp as well. I
think we should wait to make this move until distutils2 has seen some
adoption and the initial warts are hammered out, though - so hopefully
some time late next year? Even that might be optimistic.
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6VmHEACgkQ8W4rlRKtE2f/BACgg/VLBmQEghsgTAOHIRPf4it8
YXkAoNVNYSFPf2O6wd7yoHFS9uTpRbcg
=MqHe
-----END PGP SIGNATURE-----
Russell Keith-Magee
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi Russ,
>
> On 10/12/2011 05:14 AM, Russell Keith-Magee wrote:
>> My only feedback on the patch is a point of clarification in the
>> tutorial. Rather than creating a mysite directory with a mysite
>> project directory in it, and then having to refer to the "inner/outer
>> directory" or "the directory with manage.py in it", it strikes me that
>> it might be cleaner to name the outer directory something generic
>> (like "django_tutorial"). This reinforces that the outer directory
>> name doesn't matter, and that startproject is only creating the inner
>> directory.
>
> So the repeated directory name does make the tutorial wording slightly
> clumsy (though I don't think it's much of a problem otherwise). The
> issue is that startproject _does_ in fact create both the outer and
> inner directory; and I think it must do so, else its creating a
> manage.py in the current directory, and we have to explicitly tell
> people "first create an empty directory, then run startproject inside of
> it" - I don't think that's a good plan.
I'm not convinced it's a bad idea. From an pedagogical perspective,
it's easy to explain -- Make a directory to contain all the bits for
your project, move into the directory, then bootstrap your project.
My only hesitation about this is that startproject wouldn't be
namespacing it's own output -- this is akin to the problem of zip/tar
files dumping their contents into the current working directory.
However, we're only dropping 1 file -- manage.py -- and if overwriting
manage.py is a concern, we could put protections on startproject so
that it only runs on an empty directory (unless you specify a --force
option or similar).
> I toyed with ideas like allowing startproject to take two arguments
> instead of one, or allowing its argument to be an invalid Python module
> name, using that for the outer directory, and then doing an automatic
> slugify-like conversion to a valid Python module name for the inner
> directory. In the end I decided it was simplest to just live with the
> repeated directory name.
Agreed that extra arguments aren't needed. Duplicating the directory
name is weird, but it's a one time thing, and can be easily renamed.
> I suppose the one other thing we could do is explicitly have the user
> rename the outer directory in the tutorial? This doesn't really seem
> worth it, though...
Yeah - this doesn't sound worth it. The only upside I can see is that
it would really drive home the fact that it doesn't matter what the
top level directory is called.
However, it's something that we'd be teaching users to do for no
reason other than to not confuse them -- but by putting it in the
tutorial, it becomes "best practice". In the long run, it's probably
better to avoid doing things like this.
Russ %-)
Carl Meyer
Hash: SHA1
On 10/12/2011 07:53 AM, Russell Keith-Magee wrote:
> I'm not convinced it's a bad idea. From an pedagogical perspective,
> it's easy to explain -- Make a directory to contain all the bits for
> your project, move into the directory, then bootstrap your project.
It's not so easy to explain why startproject can't just create the
directory to contain all the bits it creates in the first place, instead
of making me do it manually :-)
I really do think dumping non-namespaced bits in the current directory
is a serious usability problem for a utility like startproject, even if
we tried to implement some kind of "manage.py overwriting protection".
It's also a much more significant departure from how you use
startproject currently.
And if the only justification is "well, it makes it very slightly easier
for the tutorial to explain which directory you should be in to do
certain things" - I just don't see that as a good tradeoff, at all. I
think the current tutorial wording in the patch is plenty clear, if
slightly awkward.
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6VnvwACgkQ8W4rlRKtE2ehUwCeKyy0lSw9GCmTSczn46pXkD6F
HVkAmwZNV7bsuWJOpxQ+E0ChoCegroDc
=bOma
-----END PGP SIGNATURE-----
Luke Plant
>
>
> On 10/12/2011 07:53 AM, Russell Keith-Magee wrote:
>> I'm not convinced it's a bad idea. From an pedagogical perspective,
>> it's easy to explain -- Make a directory to contain all the bits for
>> your project, move into the directory, then bootstrap your project.
>
> It's not so easy to explain why startproject can't just create the
> directory to contain all the bits it creates in the first place, instead
> of making me do it manually :-)
I think there could be at least one good reason - if the developer has
already created a directory in which to store all the stuff, which would
actually be my natural instinct especially when it comes to a new
project with VCS:
$ mkdir foo
$ cd foo
$ hg init
# OK, now what am I going to put in it? Oh, a Django project.
In fact, it might be good idea to encourage use of VCS by mentioning it.
If I remember SVN correctly, you would actually need to think about it
before 'mkdir foo' - you 'svnadmin create', then checkout the empty repo
to start working. For either of these workflows, you probably wouldn't
want startproject creating your main directory. I actually like the idea
that Django is not supposed to be managing your entire project - rather,
your project uses Django.
Luke
--
"Oh, look. I appear to be lying at the bottom of a very deep, dark
hole. That seems a familiar concept. What does it remind me of? Ah,
I remember. Life." (Marvin the paranoid android)
Luke Plant || http://lukeplant.me.uk/
Donald Stufft
Carl Meyer
Hash: SHA1
Hi Luke,
On 10/12/2011 08:59 AM, Luke Plant wrote:
>> It's not so easy to explain why startproject can't just create the
>> directory to contain all the bits it creates in the first place, instead
>> of making me do it manually :-)
>
> I think there could be at least one good reason - if the developer has
> already created a directory in which to store all the stuff, which would
> actually be my natural instinct especially when it comes to a new
> project with VCS:
>
> $ mkdir foo
> $ cd foo
> $ hg init
>
> # OK, now what am I going to put in it? Oh, a Django project.
In a real project, I think a layout more similar to what Mateusz
suggested is better - there should actually be a third containing
directory outside anything startproject creates, in which you put all
the non-Python stuff (including templates). This is what I generally do
- - I consider it preferable to keep non-Python stuff out of directories
that are on sys.path. In which case that outer directory would be the
one you initially create yourself.
This (and VCS considerations) are a topic for that "project layout" docs
page, which can be added separately. I don't think we need to add more
stuff to the first page of the tutorial - and even if some people want
to, it doesn't need to happen in this patch.
> In fact, it might be good idea to encourage use of VCS by mentioning it.
> If I remember SVN correctly, you would actually need to think about it
> before 'mkdir foo' - you 'svnadmin create', then checkout the empty repo
> to start working. For either of these workflows, you probably wouldn't
> want startproject creating your main directory. I actually like the idea
> that Django is not supposed to be managing your entire project - rather,
> your project uses Django.
Right. I think the stuff that startproject currently creates in my patch
is just the "Django stuff", not your entire project - and that that
"Django stuff" should be self-contained in a directory, and startproject
should create it that way. The other non-Django/non-Python bits of your
project should go outside that.
I'm pretty strongly opposed to any change that involves startproject
dumping files directly into the current directory, not inside a
container that you provide the name for. That's just bad behavior, IMO.
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6VrvMACgkQ8W4rlRKtE2ebTACfS8KNH+Zj7U5P9EYD+S6sfXag
jtMAoMyu5/sA4sTIzqcRgeuw5fmc6HzL
=08bN
-----END PGP SIGNATURE-----
Alec Taylor
constant (i.e. the settings.py file)
Markus Gattol
Carl Meyer
Hash: SHA1
Hi Markus,
On 10/12/2011 10:30 AM, Markus Gattol wrote:
> django-admin.py startproject would create site_root and project_root
> put as this patch intends, one could always easily rename site_root to
> whatever he wants. example.com is entirely decoupled anyway and can be
> named anything.
>
> Maybe it is not bad having project_root as a fix name at this location
> in the filesystem tree as it makes it easy to reference things -- not
> just for people but also scripts and such. I am generally against to
> much rules/conventions but in this case I think having some fixed name
> at a fixed location would make things easier (even if it is just for
> new people to grasp things).
Since "project_root" is the actual importable Python package name, I
don't think it's a good idea to have it be a fixed name; it should be an
appropriately-chosen package name for each individual project.
I guess we could consider always using the same name for the outer
directory, but I think it's better to reuse the package name for that
directory. It means you can run startproject twice from within the same
directory without overwriting things, and I think it just makes
intuitive sense that the output of startproject is a directory with the
same name you just gave it.
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6VxIwACgkQ8W4rlRKtE2f9ggCeOgOpcbsGHw4h/zuUs9o5J8eV
xA0AnisROi6OlHIRG2UR9MkWw7fWXDRs
=f6cP
-----END PGP SIGNATURE-----
ptone
On Oct 12, 7:59 am, Luke Plant <L.Plant...@cantab.net> wrote:
> $ mkdir foo
> $ cd foo
> $ hg init
>
> # OK, now what am I going to put in it? Oh, a Django project.
>
> In fact, it might be good idea to encourage use of VCS by mentioning it.
> If I remember SVN correctly, you would actually need to think about it
> before 'mkdir foo' - you 'svnadmin create', then checkout the empty repo
> to start working. For either of these workflows, you probably wouldn't
> want startproject creating your main directory. I actually like the idea
> that Django is not supposed to be managing your entire project - rather,
> your project uses Django.
>
> Luke
that allows startproject/startapp to write into an existing folder.
That way, if you want to turn an existing folder into a project you
can, otherwise the default outer directory will be created.
https://code.djangoproject.com/ticket/17042
https://github.com/django/django/pull/65
-Preston
Chris Beaven
Carl Meyer
Hash: SHA1
Hi Chris,
On 10/12/2011 05:28 PM, Chris Beaven wrote:
> Great job on getting the ball rolling on this, Carl!
>
> +1 on the whole idea
Thanks!
> Similar to what Russ and Luke are saying, I'd also prefer it if
> startproject dropped manage.py and the project Python package in the
> current working directory (with overwrite checks first).
I remain -1 on CWD pollution by default; I haven't yet seen any
arguments in favor of it that I find even slightly convincing.
As we just discussed on IRC, I'd be happier to have an optional second
argument to startproject, if this is really an issue. If two arguments
were given, the first would be the containing directory name (which
could also be "."), and the second the package name. (Could also do it
in the other order, but I think putting the containing directory first
feels more natural, as it's "first" in the hierarchy).
I'm also fine with ptone's patch to allow using an existing directory,
which maybe addresses some of the same cases.
I think either one of these should be done as a separate follow-on
patch, though, as I think they are mostly-orthogonal new features for
startproject, not closely related to the purpose of this patch.
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6WLCAACgkQ8W4rlRKtE2fK7wCgqnSaF8lHx0/bKcSIb4ElYEJ4
hdsAn27cKx/j+m0prV9Kz8MptfQR2Dj+
=GUEr
-----END PGP SIGNATURE-----
Brett H
mkproject command.
I have a much longer reasoning why startproject should not get into
creating the outer folder which is effectively the distribution
folder, and the domain of distribution packaging tools, so I'll follow
this post up.
Brett H
outer folder 'site_root' with startproject is a bad idea. Making
startproject install manage.py in the current working directory is the
best transitional solution until distutils2 becomes stable.
Even though distutils2 is not ready for primetime, by python
convention the outer folder IS the source distribution folder. Django
can have many sites under one project, so conceptually site_roots are
just distributions.
Having been stung with setup.py or any .py file being in the outer
directory I believe ultimately manage.py should be deprecated post 1.4
since ultimately all it does is save 3 keystrokes, and deal with path
issues that could go away when replaced with a setup.cfg file and
appropriate meta-data to locate the settings.py.
Ultimately distutils2 was accepted into Python 3.3 so it will be the
standard for installation, and so the python way for creating
distributions will be `pysetup.py create [distribution]`[http://
docs.python.org/dev/install/pysetup.html]
Since distutils2 is meant to be extensible pretty much like Django
management commands, you should probably be able to have a Django
installed pysetup command so that `pysetup.py create-django
[distribution] --project=[different project name]` extends the
pysetup.py create command and also creates the project in the same
step.
Of course as I said - distutils2 is a way off being stable and
backported, and the documentation is not good enough, despite making
it into python 3.3 alpha, so if people wanted an interim solution then
having an additional django command such as `django-admin.py
createdist [distribution] --project=[alt project name]` that created a
minimal distribution folder with setup.cfg (the most minimal required
to package), manage.py, and project structure in one command would I
think be one way to do it.
Having a separate createdist command would I think also allow creating
default folders for templates, static, and media as part of that
distribution, since I think ultimately pysetup.py should be able to
install that django project distribution on a server, and play nicely
with other installation tools. Logically some of the meta information
about project layout required for installation might conceivably move
to setup.cfg while retaining settings.py for actual non file-location
settings, but baby steps are good.
Since people love inventing their own weird and wonderful ways to
organise things, startproject should I think remain a much more
minimalist command that just creates the minimum to get going, and
doesn't preconceive whether or where someone wants to have a
site_root, templates or a kitchen_sink folder (except for the new
manage.py).
I think it's important that Django focus on not creating a new wheel
when a lot of people are already working on fixing the existing
packaging & installation wheel in other forums. As I said previously,
the historical reasons for how django deals with python paths and
finding package directories is there because the python methods
sucked, but really django projects are no more special than any other
python distribution with metadata, packages, data, docs, scripts, and
sources.
Brett
Luke Plant
>> Similar to what Russ and Luke are saying, I'd also prefer it if
>> startproject dropped manage.py and the project Python package in the
>> current working directory (with overwrite checks first).
>
> I remain -1 on CWD pollution by default; I haven't yet seen any
> arguments in favor of it that I find even slightly convincing.
Just to say that personally I'm not that bothered either way, I'm happy
to let you paint this bike shed.
Luke
--
O'REILLY'S LAW OF THE KITCHEN
Cleanliness is next to impossible.
Luke Plant || http://lukeplant.me.uk/
Carl Meyer
Hash: SHA1
Hi Brett,
On 10/12/2011 06:31 PM, Brett H wrote:
> Having reworked project structures so many times I think creating the
> outer folder 'site_root' with startproject is a bad idea. Making
> startproject install manage.py in the current working directory is the
> best transitional solution until distutils2 becomes stable.
I do understand the desire to have startproject play nicely with other
tools (like distutils2, or VCS') that may want to create an outer
directory themselves. And I also understand that not everyone cares to
keep the sys.path-added directory isolated from other things, so many
people will want manage.py directly in the outermost containing directory.
My concern is that the default behavior of startproject should do
something that is experimentation-friendly and beginner-friendly, and I
don't think dumping multiple things into the CWD is either of those. I
think that you should be able to just try out startproject and get
something sensible and workable right off the bat, without having to
first understand distutils2 or why you should use a VCS, or read the
docs carefully to find out that you really should create an outer
directory yourself first manually before running it.
So I think it makes sense to (as a separate feature) make startproject
flexible enough to work with pre-existing directories, for more advanced
users who want that flexibility, either via the two-argument extension
or ptone's patch. But I still think the best default behavior is to
contain all our output under a single provided name, not dump into the CWD.
> Of course as I said - distutils2 is a way off being stable and
> backported, and the documentation is not good enough, despite making
> it into python 3.3 alpha, so if people wanted an interim solution then
> having an additional django command such as `django-admin.py
> createdist [distribution] --project=[alt project name]` that created a
> minimal distribution folder with setup.cfg (the most minimal required
> to package), manage.py, and project structure in one command would I
> think be one way to do it.
As I said before, I think that some type of Django integration with
distutils2 is an interesting idea and worth exploring, but I think it
belongs in third-party extensions for quite some yet before it should be
considered in core.
> I think it's important that Django focus on not creating a new wheel
> when a lot of people are already working on fixing the existing
> packaging & installation wheel in other forums. As I said previously,
> the historical reasons for how django deals with python paths and
> finding package directories is there because the python methods
> sucked, but really django projects are no more special than any other
> python distribution with metadata, packages, data, docs, scripts, and
> sources.
Hear hear! Couldn't agree more with all of this. The only part I don't
understand is if you're drawing an equivalence between "startproject
containing its output by default" and "Django creating a new wheel" -
that seems like a bit of a stretch :-)
Thanks for your thoughts!
Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6WOQoACgkQ8W4rlRKtE2dR1wCfesxGqxvN1TOZIjJa16TO+5TY
eBsAoJTOfucTz41agwkeDxMFCRAezqlb
=fG0n
-----END PGP SIGNATURE-----
Carl Meyer
Hash: SHA1
I just committed the patch as r16964 [1]. Yay!
Follow-up to other issues discussed in this thread:
* #17042 [2] is tracking enhancements to startproject to allow
specifying the target container directory (including CWD), and allowing
it to write into an existing directory. ptone already has good progress
on a patch there.
* I just filed #17044 [3], to track the idea of creating a "project
layout and python path" page in the docs, where some of the issues
discussed in this thread can be explained more fully.
Thanks everyone for the feedback and discussion!
Carl
[1] https://code.djangoproject.com/changeset/16964
[2] https://code.djangoproject.com/ticket/17042
[3] https://code.djangoproject.com/ticket/17044
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6WhnwACgkQ8W4rlRKtE2eU5wCgvEd8YI4sXyZO8QJPyh6cStaV
1ecAn1pt7BySbBpUnE9saZkEzm+xOf55
=bIuA
-----END PGP SIGNATURE-----