Django is not a serious framework, really

[Jason Ma]

Hi,
I download and tried to use the Django 1.4 yesterday. I am a dummy
and I just follow the official document, but When I just start a
project.
I found that it is what I see from my computer:

jason@jason-pc:~/workspace/hunqing$ tree .
.
├── hunqing
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── settings.py
│   ├── settings.pyc
│   ├── urls.py
│   ├── urls.pyc
│   ├── wsgi.py
│   └── wsgi.pyc
├── __init__.py
├── manage.py
├── settings.py
└── urls.py

but what doc say?
mysite/
manage.py
mysite/
__init__.py
settings.py
urls.py
wsgi.py

If you're a beginner, what are you going to say, yes, F! Why I created
more files? I heavily doubted that whether the writers have tested
that carefully. Ok, forget that, We'll see and continue.
In the later chapter, we created two classes in the models.py in
polls, I do all the steps same as the doc except that one columns
name, mine is questions whereas the doc is question, so I want to test
the power of the syncdb, I modified the model.py and I just do the
python manage.py sql polls, that's ok, it is correct name this time.
So I just run it to change it in database using python manage.py
syncdb, it works too. But go to the db and see, the table is not
changed at all. I want to say F again now. That's what doc say:

The syncdb command runs the SQL from sqlall on your database for all
apps in INSTALLED_APPS that don't already exist in your database. This
creates all the tables, initial data and indexes for any apps you've
added to your project since the last time you ran syncdb. syncdb can
be called as often as you like, and it will only ever create the
tables that don't exist.

That's gr8, If you just create the tables that don't exist, why do you
syncdb successfully? One basic rule of database is consistence, if you
can't created the tables you want, why don't get alert? I am not a
good programmer though, I do know if you can't do something, just say
it. How can I know the error without any prompt?

There are many people saying the Django is well-documented, do you
still think it is true?

--
Best wishes,

Jason Ma
HP Enterprise Services

[Jan Schotsmans]

Hi Jason,

pyc are compiled py files ... it means you most likely ran the code before doing the "ls".

As for the structure of the directories, I'm fairly certain that its MySite first, then MyApp, not 2x mysite.

Not going to speak on the database syncs, other then that they work fine for me.

I'd also suggest you don't use titles like you do, don't post user problems in the developers list (this is for core django development discussions, not users having problems).

If your comment with file layouts was regarding the ".pyc" files, might I suggest you first read a bit of the basics on Python. They aren't really essential, but if pyc files confuse and agitate you as much as your message title suggest, it might alleviate quite a bit of stress.

r/Jan

2012/4/11 Jason Ma <rose...@gmail.com>

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Russell Keith-Magee]

First rule: If you're following a tutorial and want to follow along, you need to actually follow the instructions as given. The tutorial asks you to type:

django-admin.py startproject mysite

From the looks of it, you didn't type that. You typed:

django-admin.py startproject hunqing

Furthermore, you've evidently run some of the code before you looked at the directory structure. .pyc files are the Python runtime's precompiled byte code output. If you look at the directory structure at the point the tutorial asks you to, you shouldn't see any .pyc files.

If you're using an IDE, it's possible the IDE might have compiled these files for you. Regardless, the existence of .pyc files shouldn't be a surprise to anyone that has used Python before. Django's installation guides tells you that you're going to need to install Python -- that should be enough of a hint that you're probably going to need to know a little bit about Python in order to use Django. Django's tutorial can't -- nor should it -- teach you everything there is to know about Python. At some point, we have to assume that you're going to learn the language that Django uses.

> If you're a beginner, what are you going to say, yes, F! Why I created
> more files? I heavily doubted that whether the writers have tested
> that carefully. Ok, forget that, We'll see and continue.

We've checked the tutorial quite carefully. To be doubly sure, I've just worked through the start of the tutorial myself to make sure it matches what is described -- and it does. If you follow the instructions as written, you should get the output as described. If you don't follow the instructions as written, then its anyone's guess what you'll get.

> In the later chapter, we created two classes in the models.py in
> polls, I do all the steps same as the doc except that one columns
> name, mine is questions whereas the doc is question, so I want to test
> the power of the syncdb, I modified the model.py and I just do the
> python manage.py sql polls, that's ok, it is correct name this time.
> So I just run it to change it in database using python manage.py
> syncdb, it works too. But go to the db and see, the table is not
> changed at all. I want to say F again now. That's what doc say:
>
> The syncdb command runs the SQL from sqlall on your database for all
> apps in INSTALLED_APPS that don't already exist in your database. This
> creates all the tables, initial data and indexes for any apps you've
> added to your project since the last time you ran syncdb. syncdb can
> be called as often as you like, and it will only ever create the
> tables that don't exist.
>
> That's gr8, If you just create the tables that don't exist, why do you
> syncdb successfully? One basic rule of database is consistence, if you
> can't created the tables you want, why don't get alert? I am not a
> good programmer though, I do know if you can't do something, just say
> it. How can I know the error without any prompt?

But it *does* give you a prompt.

When you run syncdb, the output tells you exactly what has, and what has not, been created.

So, if a table for myapp.MyModel has been created, in the output of syncdb you'll see a message that looks something like:

Creating table myapp_mymodel

If you then go and modify MyModel, and then run syncdb again, you won't see this message. That means that the table hasn't been created as a result of your syncdb call. If you run syncdb, and you *don't* see a "Creating table" message that you were expecting, then you should probably go looking to see why.

> There are many people saying the Django is well-documented, do you
> still think it is true?

I may be biased, but I certainly think so.

If you print Django's documentation, it runs to over 900 pages. That's not 900 pages of auto generated JavaDoc style APIs, either -- it's 900 pages of hand-crafted prose. There aren't too many open source frameworks (or frameworks of any stripe, for that matter) that can claim that.

As for the question in your subject -- Is Django a "Serious framework"?

Well, Instagram just got sold for $1 billion, and it's a Django site. AMD, Canonical, Discovery, Disqus, HP, IBM, Intel, Lexis-Nexis, the Library of Congress, Mozilla, NASA, National Geographic, the New York Times, Orbitz, PBS, Pinterest, Rdio, VMWare, Walt Disney, the Washington Post, and many, many more all use Django in some capacity. Sounds like a serious framework to me.

Is Django perfect? Certainly not. Is the documentation perfect? Certainly not. But is it a solid, scalable framework that has some of the most comprehensive documentation you'll find on an open source project? Yes.

We're always open to suggestions -- so if you can come up with any constructive suggestions on how we could improve the tutorial, the documentation, or the framework itself, feel free to make those suggestions.

Yours,
Russ Magee %-)

[Jason Ma]

Hi Jan,
I don't mean the .pyc files in the folder, If it was that,
everything will be fine. Please read what I typed carefully. And
please test the case I just mentioned, I tested again and still the
same situation. It maybe the feature, but hope it will get more
user-friendly because we are all human being right? We will get more
fun if we get more helpful document. Forget it if it bother you.

Regards,
Jason

[Aymeric Augustin]

Hi Jason,

Le 11 avril 2012 14:10, Jason Ma <rose...@gmail.com> a écrit :
> I heavily doubted that whether the writers have tested that carefully.

As one of the many people who replayed the tutorial from A to Z,
checked every little detail, updated screenshots, etc. before the
release of 1.4, I feel your feedback is rather unfair.

> There are many people saying the Django is well-documented, do you
> still think  it is true?

Django's documentation assumes that the reader:
- has some familiarity with Python (e.g. knows what a __init__.py or a
*.pyc file is)
- is an autodidact and is able to investigate by himself when (s)he
deviates from the recommended path and encounters an unexpected
behavior (e.g. syncdb doesn't perform migrations).

Honestly, if this doesn't match your expectations at all, then Django
might not be the right framework for you.

I still believe our documentation compares favorably to most
open-source software entirely developed by volunteers in their free
time.

Best regards,

--
Aymeric.

[Juan Pablo Martínez]

gooby pls :)

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

--
juanpex

[Babatunde Akinyanmi]

I wonder why you guys are replying OP.

--
Sent from my mobile device

[Daniel Sokolowski]

Would it be hard for django to check during syncdb and complain that a
schema migration is required for an app? I vaguely recall being stumped
myself after changing a model, running syndb and getting my first database
integrity error. I believe even a NOTE in the tutorial clarifying that
schema migration is not automatic yet would be sufficient.

First impression of new comers to django are rather important I believe.

django-admin.py startproject mysite

django-admin.py startproject hunqing

Creating table myapp_mymodel

Yours,
Russ Magee %-)

[Jan Schotsmans]

Hi, Jason,

I suggest you go to the django-users list for help:  http://groups.google.com/group/django-users 

This list is only for django core development (aka, the framework itself), not for people using Django.

And, it's no bother, but you came on rather strong, with wording that real world information, directly contradicts, not only is Django used for a variety of sites, it is also used by some very high profile sites and corporations.

Besides that, the documentation is worked on almost as hard as Django's core, very little auto generated content and it is checked, for each release, by a multitude of people.

If you found an actual error in said documentation, it would be best to give detailed information and start a constructive conversation about the issue's, so that it can be resolved.

Starting with a premise that mostly condenses down to "Django/Django Documentation is a 3de rate POS", isn't very constructive and will only serve to tick of the developers that put their free time into creating this application.

But as someone else replied already and I just did again, when following the documentation point by point, the correct layouts and behavior, are obtained as expressed in the documentation.

If you repost your problem to the appropriate list (django-users), I would also suggest you give a bit more information.

Things like:

Python version.

OS you are using. (and I think in your case the language of the OS might be useful).

Exact commands you used to.

Etc.

The more information the better.

[ionic drive]

I wonder how someone can write such a provocative message as the thread-starter.

Django - is great - thanks for all!

[bhuztez]

The document clearly states that "You'll see a message for each
database table it creates".

I guess Jason Ma had a hard time reading the document because it is
written in English. Native Chinese speakers who are not quite familiar
with English will feel desperate when they had to read serveral pages
of document in English. Just imagine how desperate will you be if you
have to read pages of document in Chinese when you just learned Ni
Hao.

Django is getting more and more popular in China recently. More and
more people there are asking questions like how to do this or that in
Django, most of the time, it is just because it is too hard for them
to understand the document on their own. I propose that Django has a
Chinese translation of its document. Sure, there is a huge amount of
work. If core team decides to work on this, I would like to help.


Disclaimer: I do not know Jason Ma, I guess he is a Native Chinese
speaker because I googled his email found his email appears on some
Chinese website.


On 4月11日, 下午9时29分, Aymeric Augustin
<aymeric.augus...@polytechnique.org> wrote:
> Hi Jason,

[Jan Schotsmans]

I saw the Chinese date-stamp in his email and deducted the same, hence why I said it might be wise to add the language of his installation, so people are aware (but to django-users).

Don't really know what serveral pages are, but I do understand language barrier problems :p (just kidding, hugz typo's)

As for a Chinese documentation project, I think that might be something for the Chinese users, who are fluent in English, to do.

I don't think many of the devs are versed in Mandarin or Cantonese, (other then the little Cantonese they might have picked up from watching movies out of Hong Kong XD).

Personally, as I keep telling people in my lil country, even if it bothers you, try doing stuff in other languages, knowing them will only benefit you in the long run.

I personally hate dubbed TV shows, especially children programs for over the age of 8. If they leave them in the native language, with subtitles, the kids will pick up the basics of the languages very easily and the more contact they have at a young age, with other languages, the more easily they'll learn new languages throughout their life.

But, for older users, I can certainly understand it to be a potential source of frustrations, even more so, if you are professionally tasked to research the subject and don't have the time to get well versed in English.

r/Jan

2012/4/11 bhuztez <bhu...@gmail.com>

[Erik Stein]

The other day I went back to one of my first django projects. It's running on production since 2007/2008 flawlessly serving a rather complex scientific database for about 100000 visits/month, still using django 1.1. The reason was that the current maintainer needed some help with the project and django.

I was extremely happy to see how easy it was to get back to an understanding of the code and to show the current maintainer what he missed. He is not a programmer, but only got some variables wrong and missed to write a view function.

The problem and another problem, too, were solved within an hour.

Django has it's limits, but it is one of the most serious frameworks out there. Thanks to everyone who contributes to it and especially to those who took the best of python and designed an API and project structure which prove to be well-arranged til today.

-- erik

Am 11.04.2012 um 14:54 schrieb Russell Keith-Magee:

> On Wednesday, 11 April 2012 at 8:10 PM, Jason Ma wrote:

[snip]

Erik Stein
Programmierung, Grafik
Oranienstr. 32 10999 Berlin, Germany
fon +49 30 69201880 fax +49 30 692018809
email er...@classlibrary.net

[Russell Keith-Magee]

On Wednesday, 11 April 2012 at 11:10 PM, bhuztez wrote:

> The document clearly states that "You'll see a message for each
> database table it creates".
>
> I guess Jason Ma had a hard time reading the document because it is
> written in English. Native Chinese speakers who are not quite familiar
> with English will feel desperate when they had to read serveral pages
> of document in English. Just imagine how desperate will you be if you
> have to read pages of document in Chinese when you just learned Ni
> Hao.

I studied Mandarin in high school, so I know *exactly* how desperate you get :-)

However, my point stands -- if he "just typed what was in the tutorial", then he *won't* get the result he described. He's doing *something* else. I have great sympathy and patience for anyone working through a language barrier, but I don't have sympathy for someone who doesn't follow the instructions, and then blames us because our instructions are wrong.

> Django is getting more and more popular in China recently. More and
> more people there are asking questions like how to do this or that in
> Django, most of the time, it is just because it is too hard for them
> to understand the document on their own. I propose that Django has a
> Chinese translation of its document. Sure, there is a huge amount of
> work. If core team decides to work on this, I would like to help.

If someone wants to take on the task of writing and maintaining a translating of the tutorial (or the whole documentation base), I'm sure we can find a way to host it. We've always been very proud of our internationalization infrastructure, and I see no reason why this shouldn't be extended to our documentation. At one point, I believe there was a French translation of the docs; however, I don't know if that is still being maintained.

Ideally, we'd have some sort of toolset to help with this sort of translation (just like we have Transifex for the in-app strings) -- but failing that, simple text with a clear warning that it possibly lags behind the English translation will probably suffice.

Yours,
Russ Magee %-)

[Cal Leeming [Simplicity Media Ltd]]

Yo Jason, I'm really sorry for you, and imma let you finish, but Django is one of the best frameworks of all time.

(I knew I'd get a chance to use that meme one day!)

On Wed, Apr 11, 2012 at 7:37 PM, Erik Stein <co...@classlibrary.net> wrote:

-- erik

Am 11.04.2012 um 14:54 schrieb Russell Keith-Magee:

> On Wednesday, 11 April 2012 at 8:10 PM, Jason Ma wrote:

[snip]

Erik Stein
Programmierung, Grafik
       Oranienstr. 32   10999 Berlin, Germany
       fon +49 30 69201880   fax +49 30 692018809
       email er...@classlibrary.net

[Carl Meyer]

Hi Jason,

Others have commented on the pyc files, but I don't think that's really
the point here. The point is that there is an extra __init__.py,
settings.py, and urls.py in the outer directory next to manage.py that
should not be there.

This is definitely a bug, and it's one that I've already seen reported
several times, but it is not a bug in Django. With a clean installation
of Django 1.4, the tutorial steps work as advertised. This bug occurs if
somehow your installation of Django 1.4 is "layered" on top of an
installation of Django 1.3, without the 1.3 installation ever having
been removed. I'm not sure how this happens, but my best guess is that
it could happen if you are using a very old version of pip (like, pip
0.3 era, before pip gained uninstall support). You're most likely to be
using an old pip if you are using a Linux distribution's packaged pip;
those tend to be quite outdated.

If you are interested in helping to solve this issue (as opposed to, for
instance, trollish hyperbole), it would be very helpful to know what
operating system and version you are using, and how you installed Django
1.4 (i.e the exact commands you used).

Thanks!

Carl

[Alex Ogier]

Maybe it would be worth experimenting with various combinations of django 1.x django-admin.py executables with django 1.4 libraries? Maybe if django-admin.py is a symlink into a 1.3 tree but django 1.4 is on the search path this stuff could crop up?

Best,
Alex Ogier

[Carl Meyer]

Hi Alex,

On 04/12/2012 01:07 PM, Alex Ogier wrote:
> Maybe it would be worth experimenting with various combinations of
> django 1.x django-admin.py executables with django 1.4 libraries? Maybe
> if django-admin.py is a symlink into a 1.3 tree but django 1.4 is on the
> search path this stuff could crop up?

Sorry, I wasn't clear - I already know what the immediate cause of the
problem is, and it's easy to reproduce manually.

"startproject" takes the actual files in django/conf/project_template/
as the template for the new project. If an installation of Django 1.4 is
installed over top of an installation of Django 1.3 (and I mean
literally on top of, in the same filesystem tree), then the project
template files that were moved to a subdirectory in 1.4 (__init__.py,
urls.py, settings.py) are found in both locations. So even if it's
purely the 1.4 startproject code running, it'll install these extra
files, because it finds them there in the project template.

The open question is just how this situation occurs in the first place.
In other words, which particular buggy installer or installation
technique is causing an overlaid installation like that, so we can warn
people away from it and better advise them how to fix it.

Carl

[Aymeric Augustin]

On 12 avr. 2012, at 21:16, Carl Meyer wrote:

> The open question is just how this situation occurs in the first place.
> In other words, which particular buggy installer or installation
> technique is causing an overlaid installation like that, so we can warn
> people away from it and better advise them how to fix it.

This problem occurs at least when you run "setup.py install" before and after the commit that introduced the new project layout.

Some people who had the habit of running "setup.py install" from a git clone to keep up-to-date with the development version reported the problem.
(Just to be 100% clear — this technique doesn't work because it doesn't remove .py or .pyc files that are removed from Django.)
As a result, I added a warning to the docs in r17636.

Most likely, installing 1.4 with "setup.py install", as explained in our docs [1], has the same result when 1.3 was previously installed in the same fashion.

I suggest we add a warning to this section of the docs: "if a previous version of Django is installed, go delete it manually in your site-packages directory" (unless there's a better method to remove a Python package?)

Best regards,

--
Aymeric.

[1] https://docs.djangoproject.com/en/dev/topics/install/#installing-an-official-release-manually

[Alex Ogier]

That seems like too much to ask. "setup.py install" should Just Work(tm), and if django requires that a certain directory be clean of any .py files in order to function properly then "setup.py install" should make that happen. The note might still be valuable, because we should inform people that untar-ing a version on top of another can cause problems like this, but breaking "setup.py install" shouldn't be necessary.

 

If none of these are acceptable, then we can always use a different directory (django/conf/project_template_new), so that conflicts don't happen. It's a bit hacky but it would work.

 

Best,

Alex Ogier

[Carl Meyer]

On 04/12/2012 03:16 PM, Aymeric Augustin wrote:
> This problem occurs at least when you run "setup.py install" before
> and after the commit that introduced the new project layout.
>
> Some people who had the habit of running "setup.py install" from a
> git clone to keep up-to-date with the development version reported
> the problem. (Just to be 100% clear — this technique doesn't work
> because it doesn't remove .py or .pyc files that are removed from
> Django.) As a result, I added a warning to the docs in r17636.

Ah, this makes sense. Thanks!

> Most likely, installing 1.4 with "setup.py install", as explained in
> our docs [1], has the same result when 1.3 was previously installed
> in the same fashion.

Yes, it would.

> I suggest we add a warning to this section of the docs: "if a
> previous version of Django is installed, go delete it manually in
> your site-packages directory" (unless there's a better method to
> remove a Python package?)

The "better way" would normally be "pip uninstall". Unfortunately, since
Django's setup.py uses pure distutils (not setuptools), installing
Django with "python setup.py install" does not record any metadata along
with the installation, making an automated uninstall impossible. So the
only workable technique is to manually remove the "django" directory
from site-packages.

I agree that we should add a documentation warning about this. I think
the current warning is not quite in the right place, as it follows the
section on installing via a pth file or symbolic link. "setup.py
install" should not be used in that case, but not for reasons of the
above bug, rather simply because it's not needed. So (IMO) it's
confusing to combine the warnings.

I've filed #18115 to track this.

Carl

[Carl Meyer]

On 04/12/2012 03:27 PM, Alex Ogier wrote:
> That seems like too much to ask. "setup.py install" should Just
> Work(tm), and if django requires that a certain directory be clean of
> any .py files in order to function properly then "setup.py install"
> should make that happen. The note might still be valuable, because we
> should inform people that untar-ing a version on top of another can
> cause problems like this, but breaking "setup.py install" shouldn't be
> necessary.

Who is breaking or proposing to break "setup.py install"? Distutils
never designed "setup.py install" to support repeated installations into
the same location without removing the old version first. You could
argue that's an issue with distutils, but it's certainly not in scope
for Django to fix. In general, an installed Python package should be
able to assume that there aren't random additional .py files scattered
about in its source tree that aren't part of the source tree for that
version.

The correct solution is to warn people away from using installation
techniques in ways they were not intended to be used, and that don't
work correctly. Repeated use of "setup.py install" without removing the
previously-installed version is inherently broken; if we work around one
specific case where it breaks, there will be others in the future (there
probably already are).

> If none of these are acceptable, then we can always use a different
> directory (django/conf/project_template_new), so that conflicts don't
> happen. It's a bit hacky but it would work.

No thanks.

Carl

[Alex Ogier]

On Apr 12, 2012 6:16 PM, "Carl Meyer" <ca...@oddbird.net> wrote:
>
> The correct solution is to warn people away from using installation
> techniques in ways they were not intended to be used, and that don't
> work correctly. Repeated use of "setup.py install" without removing the
> previously-installed version is inherently broken; if we work around one
> specific case where it breaks, there will be others in the future (there
> probably already are).
>

There are ways to support cleaning directories as part of the 'install' command, for example 'distutils.dir_util.remove_tree'. Adding that for our specific directory that needs to be clean should work, yes?

Best,
Alex Ogier

[Carl Meyer]

Hi Alex,

On 04/12/2012 04:23 PM, Alex Ogier wrote:
> On Apr 12, 2012 6:16 PM, "Carl Meyer" <ca...@oddbird.net

The entire django/ subtree should be clean before a new install of
Django, not just the new project template. Startproject is not unique,
just more visibly problematic. Off the top of my head, a stray file in
the built-in management commands directory would also cause an issue -
and a stray .py file anywhere would cause imports to continue to work
(and bring in outdated code) when they ought to fail for that version of
Django.

I'm firmly -1 on any workaround that specifically targets the project
template; it's just encouraging people to continue to use an
installation method that will cause other breakages, perhaps subtler ones.

It's possible that Django's setup.py could manually remove the entire
django/ directory from the target site-packages, but I don't think that
would be a good idea either; it'd be non-standard behavior that would
break the expectations of anyone who's used other setup.py files.

I still think the right solution is to encourage (via the documentation)
installation practices that work reliably, not to try to apply piecemeal
workarounds to specific breakages caused by installation practices that
don't work reliably (and still won't after the piecemeal workaround).

Carl

[Luciano Pacheco]

On Fri, Apr 13, 2012 at 8:30 AM, Carl Meyer <ca...@oddbird.net> wrote:

Hi Alex,

On 04/12/2012 04:23 PM, Alex Ogier wrote:
> On Apr 12, 2012 6:16 PM, "Carl Meyer" <ca...@oddbird.net

> <mailto:ca...@oddbird.net>> wrote:

[...] 

I still think the right solution is to encourage (via the documentation)
installation practices that work reliably, not to try to apply piecemeal
workarounds to specific breakages caused by installation practices that
don't work reliably (and still won't after the piecemeal workaround).

Please add a note how to detect Django version and path:

>>> import django

>>> django.VERSION

(1, 3, 0, 'final', 0)

>>> django.__file__

'/home/lucianopacheco/src/tmp_py/local/lib/python2.7/site-packages/django/__init__.pyc'

So, it will be much more clear to people not familiar with python installation structure to understand what "site-packages" means.

[],

--
Luciano Pacheco
blog.lucmult.com.br

[Ben Finney]

Alex Ogier <alex....@gmail.com> writes:

> That seems like too much to ask. "setup.py install" should Just
> Work(tm),

In the absence of a proper package management system (as implemented in
operating systems that solved this problem decades ago), you can't
expect it to Just Work. Parallel installation of multiple versions,
detection of previous versions, upgrade and uninstallation, dependency
management – these are problems solved by a package manager, and
‘setup.py’ doesn't play well with them.

Python's ‘setup.py’ can't be expected to do the job of a package
manager, especially not in parallel to the OS which often has its own
package manager.

--
\ “For fast acting relief, try slowing down.” —Jane Wagner, via |
`\ Lily Tomlin |
_o__) |
Ben Finney

[Alex Ogier]

On Thu, Apr 12, 2012 at 11:56 PM, Ben Finney <ben+p...@benfinney.id.au> wrote:
>
> Alex Ogier <alex....@gmail.com> writes:
>
> > That seems like too much to ask. "setup.py install" should Just
> > Work(tm),
>
> In the absence of a proper package management system (as implemented in
> operating systems that solved this problem decades ago), you can't
> expect it to Just Work. Parallel installation of multiple versions,
> detection of previous versions, upgrade and uninstallation, dependency
> management – these are problems solved by a package manager, and
> ‘setup.py’ doesn't play well with them.
>
> Python's ‘setup.py’ can't be expected to do the job of a package
> manager, especially not in parallel to the OS which often has its own
> package manager.

The problem is that not everyone uses package managers. A reasonable
way to track django trunk for example is to periodically pull and run
"setup.py install" which is in most cases approximately idempotent. I
have seen setup.py's that use remove_tree as part of a "clean" command
to allow someone to run "setup.py clean && setup.py install" to obtain
a pristine distribution idempotently, which I think is a good idea.
The alternative is to have everyone remember to "rm -rf" their
site-packages django every time they run setup.py install which is a
bit unsavory in my opinion.

If someone has managed to get extra files in their site-packages,
because at any point they followed a tutorial on how to build from
source, then their django installation is basically caput until they
manually "rm -rf" a deep library path. One option is to document this
and explain what to do, but this really isn't very discoverable
because telling people that when django is broken, running "sudo rm
-rf" in their python site-packages might fix it is rather akin to
someone calling tech support and them asking, "Well, did you turn off
your computer and turn it on again?" It might fix the problem, but
it's invasive and time-consuming and it shouldn't be necessary.

The "real" solution to this problem is to stop treating the existence
of files as implicit indication of their inclusion in Django. That
would mean listing somewhere the files from
django/conf/project_template/ that should be included, which isn't
very DRY, but is the only 100% solution I think.

I was bit by this sort of thing a few weeks ago. I had just removed
the simplejson/__init__.py{c,} module and related code and installed a
shim at simplejson.py, all in a feature branch in git. Imagine my
surprise, upon switching to the feature branch several days later, I
found that django was using the old version. The reason was that when
I switched back to the master branch, the simplejson/__init__.pyc was
recompiled and upon checkout out the feature branch git happily
ignored all the .pyc files in my tree. Python saw them and assumed I
had a compiled version of the module and continued using them. So,
that should give you some idea of the perils of not cleaning your
output directories (or in this case, input directory).

My recommendation is to make "setup.py clean" do everything possible
to ensure idempotent installation across any version, document that,
and call it a day. Yes, Django can't make up for people who circumvent
their package manager, but we can make it a lot easier to fix than
sending newbies off into their system libraries armed with superuser
permissions and "rm -rf".

Best,
Alex Ogier

[Florian Apolloner]

On Friday, April 13, 2012 6:49:32 AM UTC+2, Alex Ogier wrote:

I have seen setup.py's that use remove_tree as part of a "clean" command
to allow someone to run "setup.py clean && setup.py install" to obtain
a pristine distribution idempotently, which I think is a good idea.

No, they should work on fixing distutils instead of creating solutions which probably could break even worse.
 

The alternative is to have everyone remember to "rm -rf" their
site-packages django every time they run setup.py install which is a
bit unsavory in my opinion.

Or just tell them to use either pip even for development installs or just set their PYTHONPATH.
 

If someone has managed to get extra files in their site-packages,
because at any point they followed a tutorial on how to build from
source, then their django installation is basically caput until they
manually "rm -rf" a deep library path. One option is to document this
and explain what to do

You made me lol, that approach is documented in the install guide: https://docs.djangoproject.com/en/dev/intro/install/#remove-any-old-versions-of-django -- If people would actually read the docs this issue wouldn't exist. FWIW the docs also mention to symlink a dev checkout and don't tell you to run setup.py
 

That would mean listing somewhere the files from
django/conf/project_template/ that should be included, which isn't
very DRY, but is the only 100% solution I think.

Given that the documentation shows how to do it properly I don't see any point. Especially since this problem isn't related to the project_template alone -- that's just where it's most visible.

 

So, that should give you some idea of the perils of not cleaning your
output directories (or in this case, input directory).

We are aware of those, and fwiw: If you use git and switch branches it's up to you to know how python works and how git clean works, or do you want to suggest that django should rm al pyc files on startup?!
 

My recommendation is to make "setup.py clean" do everything possible
to ensure idempotent installation across any version, document that,
and call it a day.

What's wrong with the current documented approach? (Aside from the fact that people don't read it, but then again they won't read the setup.py clean either).

Regards,
Florian

[Aymeric Augustin]

On 12 avr. 2012, at 23:16, Aymeric Augustin wrote:

> Some people who had the habit of running "setup.py install" from a git clone to keep up-to-date with the development version reported the problem.
> (Just to be 100% clear — this technique doesn't work because it doesn't remove .py or .pyc files that are removed from Django.)

On 13 avr. 2012, at 06:49, Alex Ogier wrote:

> A reasonable
> way to track django trunk for example is to periodically pull and run
> "setup.py install" which is in most cases approximately idempotent.

No, this isn't a reasonable alternative, and installing software isn't an approximative operation.

Here's another example: this installation technique wouldn't reflect the changes in r17842 [1] properly — two files were removed. #18069 [2] was filed as a result.
The link between the problem and its cause was particularly tenuous in this case.

[1] https://code.djangoproject.com/changeset/17842
[2] https://code.djangoproject.com/ticket/18069

So a documentation fix might not be sufficient to eradicate the problem. Could we add this in a pre-install hook in setup.py?

try:
import django
except ImportError:
pass
else:
print "It appears that Django %s is already installed." % django.get_version()
print "If you want to upgrade Django, please remove the existing installation first."
sys.exit(1)

To support pip install --upgrade, this code should be executed right before the new version is installed. I don't know very well what's possible with distutils.

Best regards,

--
Aymeric.

[Luciano Pacheco]

On Fri, Apr 13, 2012 at 5:15 PM, Aymeric Augustin <aymeric....@polytechnique.org> wrote:

On 12 avr. 2012, at 23:16, Aymeric Augustin wrote:

[...] 

So a documentation fix might not be sufficient to eradicate the problem. Could we add this in a pre-install hook in setup.py?

try:
   import django
except ImportError:
   pass
else:
  print "It appears that Django %s is already installed." % django.get_version()
  print "If you want to upgrade Django, please remove the existing installation first."
  sys.exit(1)

This "import django" will work even when django is not installed, because usually "python setup.py " is ran from checkout of django, that contains the valid folder (python package) named "django". So, this "import django" will import relative to current directory and will work.

[],

--
Luciano Pacheco
blog.lucmult.com.br

[Alex Ogier]

On Apr 13, 2012 3:30 AM, "Luciano Pacheco" <luc...@gmail.com> wrote:
>
> This "import django" will work even when django is not installed, because usually "python setup.py " is ran from checkout of django, that contains the valid folder (python package) named "django". So, this "import django" will import relative to current directory and will work.
>

And in fact, this behavior is relied upon. Django's setup.py imports the relative django to get the version number.

Best,
Alex Ogier

[Carl Meyer]

Hi Alex,

On 04/12/2012 10:49 PM, Alex Ogier wrote:
> The problem is that not everyone uses package managers. A reasonable
> way to track django trunk for example is to periodically pull and run
> "setup.py install" which is in most cases approximately idempotent. I
> have seen setup.py's that use remove_tree as part of a "clean" command
> to allow someone to run "setup.py clean && setup.py install" to obtain
> a pristine distribution idempotently, which I think is a good idea.
> The alternative is to have everyone remember to "rm -rf" their
> site-packages django every time they run setup.py install which is a
> bit unsavory in my opinion.

I do agree that asking newer users to go digging in site-packages with
"rm -rf" is sub-optimal. I think the first-line answer is to discourage
newer users (and anyone, really) from installing Django using "python
setup.py install", so they'll never have to. As a second-line answer,
I'm open to the idea of "python setup.py clean" as an alternative to
manual rm -rf in site-packages, if you can provide a reliable
implementation of it as a patch attached to #18115 (and no other core
developer vetoes the idea; James Bennett in particular has traditionally
been the guardian of Django's setup.py).

> The "real" solution to this problem is to stop treating the existence
> of files as implicit indication of their inclusion in Django.

This is requesting a flat-out impossibility, since "existence of files
implies their inclusion" is simply *how Python module imports work*. If
a .py file exists somewhere under django/ in a directory with an
__init__.py, it can be imported as part of Django.

> would mean listing somewhere the files from
> django/conf/project_template/ that should be included, which isn't
> very DRY, but is the only 100% solution I think.

I think this is now the fourth or fifth time it's been pointed out that
the problem is not limited to startproject (and you yourself
demonstrated that with your own simplejson example!), so I'm a bit
mystified how you can still refer to this as a "100% solution."

> My recommendation is to make "setup.py clean" do everything possible
> to ensure idempotent installation across any version, document that,
> and call it a day. Yes, Django can't make up for people who circumvent
> their package manager, but we can make it a lot easier to fix than
> sending newbies off into their system libraries armed with superuser
> permissions and "rm -rf".

Agreed.

Carl

[Daniel Sokolowski]

People won’t always read all the docs – it’s a fact – so sooner or later some other new comer will experience this issue complain, gave up and worse even blog his/hers negative experience. We do want the newbie experience to be as painless as possible which means popularity and growth of the framework - and ultimately continuation of our paying jobs.

 

This thread was started by a newbie to the framework, putting aside the confrontational tone it stated some valid concerns from a beginners perspective – a very important perspective.

 

I’ve been using django for a long time and had no idea about this ‘distutils’ caveat; I do not understand or know how django installs setup process works but am up for some kind of a warning/error directly to the console.

 

Thank you for reading my opinion.

Daniel

From: Florian Apolloner

Sent: Friday, April 13, 2012 2:15 AM

To: django-d...@googlegroups.com

Subject: Re: extra files in startproject

 

--
You received this message because you are subscribed to the Google Groups "Django developers" group.

To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/SbdWA7plRx4J.

[Carl Meyer]

On 04/13/2012 08:30 AM, Daniel Sokolowski wrote:
> People won’t always read all the docs – it’s a fact – so sooner or later
> some other new comer will experience this issue complain, gave up and
> worse even blog his/hers negative experience. We do want the newbie
> experience to be as painless as possible which means popularity and
> growth of the framework - and ultimately continuation of our paying jobs.
>
> This thread was started by a newbie to the framework, putting aside the
> confrontational tone it stated some valid concerns from a beginners
> perspective – a very important perspective.
>
> I’ve been using django for a long time and had no idea about this
> ‘distutils’ caveat; I do not understand or know how django installs
> setup process works but am up for some kind of a warning/error directly
> to the console.

Thanks Daniel. I'm experimenting right now with a patch to setup.py that
would print a loud warning to console if it detects an existing django/
directory in the target site-packages.

Carl

[Daniel Sokolowski]

You sir are Epic!

[Jan Schotsmans]

Tbh, you can not go cater to this type of new users, that start badmouthing the project right out the gate, with the first hint of trouble, either becoming insulting, out right, on the developer list, or doing as you put in your example, posting bad things on their blogs.

If Jason had sent a mail to the developers list stating, "this is the problem I'm having, this is my installation, any idea's?", then we wouldn't have had 10 people defend Django before one came up with "this may be a longshot to be confirmed with further system information, but this seems to be a problem that isn't related to Django itself, but to an unclean installation environment with full or partial, duplicate Django installations."

I did respond to him to post his installation base and details about the system he is using, and so have others, but he hasn't responded since, even though there has been quite a decent discussion going about the source of the problem and a possible solution to protect installations against it happening, even though it isn't really Django's fault that package management systems cause the problem, nor is Django in the power to fix "ALL" package management systems that can cause this to happen and most certainly isn't able to protect against this behavior entirely, in case a manual/custom installation is done.

Yes, you can make it as easy as as humanly possible for new users, but no, you shouldn't have to go out of your way to cater to users that act like this.

2012/4/13 Daniel Sokolowski <daniel.s...@klinsight.com>

[Carl Meyer]

On 04/13/2012 08:33 AM, Carl Meyer wrote:
> Thanks Daniel. I'm experimenting right now with a patch to setup.py that
> would print a loud warning to console if it detects an existing django/
> directory in the target site-packages.

I've filed a pull request (https://github.com/django/django/pull/136)
with this setup.py modification and some documentation edits (and also
linked it from #18115, of course). Review and comment welcome!

Carl

[Daniel Sokolowski]

Hi Carl,

Looks fine to me, and I think throwing the warning at the end is indeed a
good idea.

-----Original Message-----
From: Carl Meyer