Re: [Django] #14087: django.core.management.get_commands only sees commands in the last package of a namespace package

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------------+------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: reopened
Milestone: | Component: Core (Other)
Version: 1.2 | Severity: Normal
Resolution: | Keywords:
Triage Stage: Accepted | Has patch: 1
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 1 | Easy pickings: 0
UI/UX: 0 |
------------------------------------+------------------------------
Changes (by bhuztez):

* ui_ux: => 0
* easy: => 0

Comment:

I tried to improve, but could not get it work with zip-archived eggs.

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:11>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------------+------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: reopened
Milestone: | Component: Core (Other)
Version: 1.2 | Severity: Normal
Resolution: | Keywords:
Triage Stage: Accepted | Has patch: 1
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 1 | Easy pickings: 0
UI/UX: 0 |
------------------------------------+------------------------------
Changes (by bhuztez):

* cc: bhuztez (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:12>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package

------------------------------+------------------------------------

Reporter: KyleMac | Owner: nobody
Type: Bug | Status: reopened

Component: Core (Other) | Version: 1.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 1

Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0

------------------------------+------------------------------------
Changes (by bhuztez):

* needs_docs: 0 => 1

Comment:

with patch zip_egg_fixed.2.diff, django will be able to find all commands
in a package, if importer for that package implements interfaces explained
below and is not in sys.meta_path.

* '''importer.iter_modules(prefix=!'')'''
importer.iter_modules is not defined in PEP 302, however
pkgutil.ImpImporter does implement this method. Django would use this
interface to list all modules in the commands package.

* '''loader.get_filename(fullname)'''
loader.get_filename is defined in PEP 302. But this patch use it to
figure out search path of packages.

{{{
filename = loader.get_filename(fullname)
path_item = os.path.dirname(filename)
}}}

Just assume os.path.dirname would work out the search path.

* '''loader.is_package(fullname)'''
This is the same as loader.is_package defined in PEP 302.

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:13>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: reopened
Component: Core (Other) | Version: 1.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Comment (by bhuztez):

update patch for revision 17517

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:14>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: reopened
Component: Core (Other) | Version: 1.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Changes (by bhuztez):

* needs_docs: 1 => 0

Comment:

namespace_package_pth.2.diff will not look up importer in
sys.path_importer_cache, since django do not support PEP 302 importer.

should find commands in modules loaded by PEP 302 importer be in another
ticket?

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:15>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: reopened
Component: Core (Other) | Version: 1.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Comment (by carljm):

The zip_egg_fixed patch should also fix #12206.

I can't seem to find a separate bug for the management-commands-in-eggs
issue, though I thought there was one. Yes, there should be a separate bug
for that (or more broadly, PEP 302 importers). It's confusing to have two
patches under development on a single ticket.

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:16>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: reopened
Component: Core (Other) | Version: 1.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Comment (by bhuztez):

> I thought there was one.

They are #8280 and #13587.

> It's confusing to have two patches under development on a single ticket.

the namespace_package_pth patch is cleaned up to address the problem of
finding command in namespace packages only, so that it could be applied
without further investigation. However, in practice, namespaced packages
are often being zip archived. So, we need a general solution for PEP 302
importers.

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:17>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: reopened
Component: Core (Other) | Version: 1.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Comment (by bhuztez):

update namespace_package_pth and zip_egg_fixed for revision 17777.

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:18>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: reopened

Component: Core (Other) | Version: SVN

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Changes (by bhuztez):

* needs_better_patch: 1 => 0
* version: 1.2 => SVN

Comment:

I found #596 is a better place for patch zip_egg_fixed. I will move it
there.

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:19>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: reopened
Component: Core (Other) | Version: SVN
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Changes (by alex@…):

* cc: alex@… (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:20>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: reopened
Component: Core (Other) | Version: SVN
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Comment (by bhuztez):

the zip_egg_fixed patch is now maintained under #8280.

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:21>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: reopened

Component: Core (Other) | Version: master

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Comment (by bhuztez):

my pull request for the namespace_package_pth patch:
https://github.com/django/django/pull/178

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:22>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: reopened
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Comment (by Natim):

My patch for Python 1.4.1 : https://code.djangoproject.com/ticket/19048

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:23>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody

Type: Bug | Status: new

Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Comment (by DavidReynolds):

bhuztez' pull request certainly addresses the problem I am seeing, any way
we can move this forward?

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:25>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: new
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Comment (by kmtracey):

Alternate fix proposed in #20344.

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:26>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: new
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Changes (by alexkoshelev):

* cc: daevaorn@… (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:27>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: new
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Comment (by timo):

https://github.com/django/django/pull/866

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:28>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: new
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Changes (by timo):

* needs_better_patch: 0 => 1

Comment:

From apollo13 on the pull request:
{{{
As I noted on the previous PR (#866) I'd like to see:

* Support for PEP 420
* Check other locations in Django for the same issues (app template
loaders etc come to mind)
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:29>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody
Type: Bug | Status: new
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Changes (by drakkan):

* cc: drakkan (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:30>

[Django]

#14087: django.core.management.get_commands only sees commands in the last package
of a namespace package
------------------------------+------------------------------------
Reporter: KyleMac | Owner: nobody

Type: Bug | Status: closed
Component: Core (Other) | Version: dev
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------

Changes (by Jacob Walls):

* has_patch: 1 => 0

* needs_better_patch: 1 => 0

* resolution: => fixed
* status: new => closed

Comment:

As far as I can tell, there isn't anything to fix anymore: PEP 420
distributed namespace packages are identified just fine. (We don't use
`imp` anymore. #21630 did that, see
fe53bc524b9c7cbc814868acde94226d0ba2bc84.)

A tree of namespaced packages without `__init__.py` anywhere:
{{{
myApps
├── app1
│   ├── management
│   │   └── commands
│   │   ├── command1.py
│   │   └── command2.py
│   └── templatetags
│   └── app1.py
└── app2
├── management
│   └── commands
│   ├── command3.py
│   └── command4.py
└── templatetags
└── app2.py

9 directories, 6 files
}}}

Add `app1` and `app2` to `INSTALLED_APPS`. Then, launch Django with
`myApps` on your path:

{{{
PYTHONPATH='myApps' ./manage.py
}}}
{{{
[app1]
command1
command2

[app2]
command3
command4

...
}}}
{{{
PYTHONPATH='myApps' python3 manage.py shell
}}}
{{{
In [1]: from django.template.backends.django import
get_template_tag_modules

In [2]: list(get_template_tag_modules())
Out[2]:
[
...
('app1', 'app1.templatetags.app1'),
('app2', 'app2.templatetags.app2')
]
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/14087#comment:31>