Should the docs suggest namespace packages?

224 views
Skip to first unread message

René Fleschenberg

unread,
May 20, 2020, 6:44:51 AM5/20/20
to django-d...@googlegroups.com
Hi,

https://github.com/django/django/pull/12939

My opinion on this is not particularly strong, but it seems odd to me
that we tell users to create namespace packages, without any
explanation. What do you think?

--
René Fleschenberg

Carlton Gibson

unread,
May 20, 2020, 6:57:59 AM5/20/20
to Django developers (Contributions to Django itself)
As far as I can see, standard packages are what users want in almost every case. So +1.

(Q:What is the use-case for namespace packages?)
> --
> You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a604b2d5-23a5-67da-b8c4-f50e1d434fb7%40fleschenberg.net.

James Bennett

unread,
May 20, 2020, 7:04:14 AM5/20/20
to django-d...@googlegroups.com
The use case for namespace packages is the ability to fragment
portions of a single package across multiple disparate locations on
the filesystem. And, potentially, to install or selectively
enable/disable access to only certain sub-portions of the package by
choosing which parts are present or by adding/removing their locations
in the import path.

Way way back in the day template tag libraries used to do a hack that
simulated this, where Django would see a "templatetags" directory in
your app and magically make it be importable from under
"django.templatetags" instead. That was one of the last pieces of the
original "magic" to be removed.

There's no reason to create implicit namespace packages for management
commands unless we want to move back to that sort of magical "lives
over *there* but imports as if it's actually over *here*" behavior,
and I'm pretty sure we don't want to do that.

Adam Johnson

unread,
May 20, 2020, 8:13:03 AM5/20/20
to django-d...@googlegroups.com
+1 for always showing standard packages in the docs.

We could also add a recommendation against namespace packages inside Django apps, for all the reasons that James lists. I'm not sure the best place for that though.

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.


--
Adam

René Fleschenberg

unread,
May 20, 2020, 8:58:59 AM5/20/20
to django-d...@googlegroups.com
On Github, Tim pointed out that with
https://github.com/django/django/commit/ccc25bfe4f0964a00df3af6f91c2d9e20159a0c2,
various __init__.py files in Django's own code were removed.

I tend to say it would be good to revert that change. Or we could update
the docs only, since they serve much more directly as an example for
user code.

--
René Fleschenberg

Aymeric Augustin

unread,
May 20, 2020, 2:55:02 PM5/20/20
to django-d...@googlegroups.com
Hello,

Commit ccc25bf refers to ticket #23919 in the commit message. In that ticket, I argued that the __init__.py files should be kept: https://code.djangoproject.com/ticket/23919#comment:102. No one brought a counter argument.

It's weird that the __init__.py files were removed anyway without further discussion (that I can find with my browser's search on that page, at least).

It's fairly minor, but I think that ccc25bf should be reverted.

Best regards,

-- 
Aymeric.



--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

Adam Johnson

unread,
May 21, 2020, 7:52:49 AM5/21/20
to django-d...@googlegroups.com
+1 for reverting ccc25bf .



--
Adam

Tim Graham

unread,
Jul 19, 2020, 4:25:19 PM7/19/20
to Django developers (Contributions to Django itself)
On the topic of namespace packages, I noticed this line in the 3.1 release notes:  "Migrations are now loaded also from directories without __init__.py files." https://code.djangoproject.com/ticket/30300

The ticket's rationale seems to be "We've just finished migrating our codebase to all-python 3, and nuked all our __init__py files. We are now upgrading to Django 2.1, and noticing the various places where they're still de-facto required by Django. This particular case seemed unnecessary and easy to fix."

Another commenter remarked, "We had a similar issue with test discovery. One of our developers read an article that __init__.py files are not required on Python3 and started removing them. Everything seemingly worked but some tests were not discovered and did not run in CI (which made it difficult to spot). I think Django should not required them if technically possible or at least it should be made clear in the documentation (sorry if I overlooked it)."

Supporting namespace packages without a real use case seems contrary to the consensus in this thread (which I see as not promoting implicit namespace packages). Based on that consensus, my inclination wouldn't be to try to make Django work with as few __init__.py files as possible. What do you think?


Aymeric Augustin

unread,
Jul 19, 2020, 4:27:59 PM7/19/20
to django-d...@googlegroups.com
I was still planning to revert ccc25bf. It's just a matter of finding time.

-- 
Aymeric.


On 19 Jul 2020, at 22:25, Tim Graham <timog...@gmail.com> wrote:

On the topic of namespace packages, I noticed this line in the 3.1 release notes:  "Migrations are now loaded also from directories without __init__.py files." https://code.djangoproject.com/ticket/30300

The ticket's rationale seems to be "We've just finished migrating our codebase to all-python 3, and nuked all our __init__py files. We are now upgrading to Django 2.1, and noticing the various places where they're still de-facto required by Django. This particular case seemed unnecessary and easy to fix."

Another commenter remarked, "We had a similar issue with test discovery. One of our developers read an article that __init__.py files are not required on Python3 and started removing them. Everything seemingly worked but some tests were not discovered and did not run in CI (which made it difficult to spot). I think Django should not required them if technically possible or at least it should be made clear in the documentation (sorry if I overlooked it)."

Supporting namespace packages without a real use case seems contrary to the consensus in this thread (which I see as not promoting implicit namespace packages). Based on that consensus, my inclination wouldn't be to try to make Django work with as few __init__.py files as possible. What do you think?


On Thursday, May 21, 2020 at 7:52:49 AM UTC-4 Adam Johnson wrote:
+1 for reverting ccc25bf .

On Wed, 20 May 2020 at 19:54, Aymeric Augustin <aymeric....@polytechnique.org> wrote:
Hello,

Commit ccc25bf refers to ticket #23919 in the commit message. In that ticket, I argued that the __init__.py files should be kept: https://code.djangoproject.com/ticket/23919#comment:102. No one brought a counter argument.

It's weird that the __init__.py files were removed anyway without further discussion (that I can find with my browser's search on that page, at least).

It's fairly minor, but I think thatccc25bf should be reverted.

Tim Graham

unread,
Jul 19, 2020, 4:35:36 PM7/19/20
to Django developers (Contributions to Django itself)
(It was already reverted in 578b3046e3b4adc43655bb5dd955054b7bf19f89. Sorry if raising a related issue on this thread was confusing.)

René Fleschenberg

unread,
Jul 19, 2020, 7:14:41 PM7/19/20
to django-d...@googlegroups.com
Hi,

On 7/19/20 10:25 PM, Tim Graham wrote:
> Another commenter remarked, "We had a similar issue with test discovery.
> One of our developers read an article that __init__.py files are not
> required on Python3 and started removing them. Everything seemingly
> worked but some tests were not discovered and did not run in CI (which
> made it difficult to spot). I think Django should not required them if
> technically possible or at least it should be made clear in the
> documentation (sorry if I overlooked it)."
>
> Supporting namespace packages without a real use case seems contrary to
> the consensus in this thread (which I see as not promoting implicit
> namespace packages). Based on that consensus, my inclination wouldn't be
> to try to make Django work with as few __init__.py files as possible.
> What do you think?

I agree. The situation of "One of our developers read an article that
__init__.py files are not required on Python3" is exactly the one we
should act against, as far as we can. This is a common misunderstanding
that we should try to clear up, not promote.

The person you quoted gives an example of why this is not purely
academical. Maybe it's debatable whether Python should behave
differently, but that's not something that Django can fix.

I am in favour of failing early and loudly here. IMHO Django should
ignore migration packages without an __init__.py file, maybe with a
warning, or maybe even with an error.

--
René Fleschenberg

Florian Apolloner

unread,
Jul 20, 2020, 3:29:03 AM7/20/20
to Django developers (Contributions to Django itself)
There are also various issues in python itself like https://bugs.python.org/issue23882 -- so I strongly recommend against namespace packages unless you actually need them.
Reply all
Reply to author
Forward
0 new messages