Pre-DEP: community support of unmaintained versions of Django
483 views
Skip to first unread message
James Bennett
Mar 19, 2015, 6:59:59 PM3/19/15
to django-d...@googlegroups.com
There's been some recent discussion in a django-users thread[1] about
the impending end-of-life of Django 1.6, which in turn will mean
end-of-life for Django supported on Python 2.6. In addition, the
end-of-life of the current LTS release, Django 1.4, will mean
end-of-life for Django supported on Python 2.5. The following is a
draft of a proposal based on ideas raised in that thread; it should be
considered pre-DEP status, to be elevated to a DEP submission if there
is sufficient support.
Abstract
--------
The support process for older versions of Django is changed, as
follows:
* Official support for older versions of Django from the Django core
team will continue to end at the scheduled times (i.e., upon release
of a new minor version for non-LTS releases, at end of advertised
support lifetime for LTS releases).
* Unofficial support, by persons willing to volunteer their time, will
be permitted through a secondary code repository existing solely to
provide this support.
* As the official end-of-life of each Django version approaches, the
Django core team will solicit proposals and volunteers to provide
unofficial support for that version past its official end-of-life,
and select maintainer(s) from amongst those volunteers. This support
will be known as "community maintenance".
Rationale
---------
For the past five years, Django has been in the process of deprecating
support for older 2.x release of Python, as part of a plan to move
Django toward, first, compatibility with both Python 2 and Python 3
(completed as of Django 1.6), and ultimately with only Python 3 (in a
future, to-be-determined, release).
The timeline for this was as follows:
* May 2010: Django 1.2 instituted a minimum Python version of 2.4.
* March 2011: Django 1.3 was the final release to support Python 2.4.
* March 2012: Django 1.4 (LTS) raised the minimum Python version to
2.5, and was the final release to support Python 2.5.
* February 2013: Django 1.5 raised the minimum Python version to 2.6.
* November 2013: Django 1.6 was the final release to support Python
2.6.
* September 2014: Django 1.7 raised the minimum Python version to 2.7.
* Late Q1/early Q2 2015: Django 1.4 and Django 1.6 will reach
end-of-life (the former on a scheduled date, the latter upon the
release of Django 1.8).
While this process was necessary to achieve at least a minimum of
Python 2.6 (the lowest version enabling compatibility with Python 2
and Python 3 in a single codebase) and ultimately a minimum of Python
2.7 (the only Python 2.x release still receiving upstream security
support), and while moving Django in the direction of Python 3 is an
absolute necessity for the future of the framework, this has had an
effect on some users of Django who are effectively locked in to now-
or soon-to-be-unsupported versions of Python by their choice of
operating system and associated support contract.
In particular, the impending end-of-life of Django 1.6, the final
release which supported Python 2.6, is poised to cut off support for
users of products like Red Hat Enterprise Linux, who may be locked to
Python 2.6 and thus unable to use a supported version of Django past
1.6 end-of-life (these users continue to receive security support for
Python itself from the operating-system vendor, who provides such
support on their behalf even after official upstream support has
ended; no such support appears to be provided, at this time, for
Django).
Although these users are not a majority of Django's users, they are
still a significant group, and simply ending their ability to use
Django is likely to be an unacceptable option. But the Django core
team cannot and will not extend the official support lifetimes of
older Django releases; the additional maintenance burden is beyond
that which can be borne, and would impede the future development of
Django.
This is not the first time in Django's history that a significant
minority of users have been affected by backwards-incompatible
releases. Prior to Django 1.0, the large incompatibility of the
"magic-removal" branch, merged shortly before the release of Django
0.95, left some users effectively stranded with large already-existing
codebases that could not be quickly migrated.
During that time period, the Django core team allowed, on an
unofficial volunteer basis, continuing bugfix and security support to
be provided by interested parties for an extended period of time past
the official end-of-life of Django 0.91. This support was conducted
solely through commits to Django's (at the time Subversion)
repository, and never through issuing additional releases in an
end-of-life'd release series.
This experiment in unofficial support was successful overall; many
high-visibility installations of Django were able to migrate to
post-"magic-removal" versions of Django at the best pace they could
manage, while still being assured of necessary support for security
and crashing/data-loss issues on their existing installations as they
did so.
With the similar situation caused by the impending end-of-life of
Django 1.6, it seems appropriate to once again extend the option of
unofficial, community-based extended support as a measure to assist
such installations of Django during the time they will require to
migrate to a more recent Python or Django version.
Details
-------
Starting with Django 1.6, each time a minor (i.e., X.Y) release series
of Django is nearing its end-of-life, a three-phase process will
begin.
In the first phase, the Django core team will post publicly that a
release series of Django is near end-of-life, and will issue a call
for discussion on the possibility of extended community support for
the soon-to-EOL release series. If, in the opinion of the core team,
the resulting discussion produces convincing arguments for the
necessity of such support, the process will move to the second phase.
In the second phase, which only occurs following a decision by the
core team that extended community support is neeed, there will be a
public solicitation for volunteers to provide such support. Any
interested person may volunteer. Members of the Django core team are
as eligible as anyone else, but will need to take care to
differentiate this role from their work on officially-supported
releases.
The core team will then choose from among the volunteers at least one,
but preferably two or more, people -- herafter known as "community
maintainers" -- who will be given commit access to a repository for
community maintenance purposes. Community maintainers will be advised
prior to acceptance that this indicates a commitment on their part to
be vigilant and active in maintaining the relevant branch of Django,
and if any community maintainer declines or later fails to honor that
commitment (either by notifying the core team of an inability, or in
the core team's determination), the core team will solicit additional
volunteers. If no suitable persons can be found to act as community
maintainers, end-of-life will occur as normal, and no extended
community support will exist. Persons who later wish to remedy this by
becomimg community maintainers will be advised to contact the
django-developers list to initiate that process.
In the third phase, the community maintainers will, on their own
initiative, provide ongoing support, in the form of fixing security
and crashing/data-loss issues, in the relevant branch of Django in the
repository to which they are granted access. They will *not* become
members of the core team simply by virtue of their status as community
maintainers, and will *not* be granted access to any additional
repositories as a result of their status.
However, community maintainers will, at the discretion of the core
team, be included in pre-disclosure discussion of security issues, and
provided with sufficient details to backport security patches into, or
develop security patches for, their branches. Maintaining such
communications with community maintainers shall be the responsibiltiy
of the current Django Fellow or Django release manager, as
appropriate.
Concerns for discussion
-----------------------
There are several potential areas in this proposal which will require
discussion and ironing out of details prior to full
implementation. They are as follows.
Differentiating unofficial and official support
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Community maintainers will need to work in a repository other than the
primary Django development repository[2]. This is both to avoid
confusion between official and unofficial support, and because it is
not possible, in git, to give commit access to only a specific branch
or portion of a repository, and community maintainers will not (as
noted above) become committers on Django as a result of being
community maintainers.
However, the community maintenance repository should still be
somewhere under the Django organzation on GitHub, both for visibility
and for ease of administration. The exact location and name of the
repository, and its access-rights policies, will need to be decided
prior to commencing use by community maintainers.
Ensuring active work by community maintainers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Although the Django core team will not be providing direct support for
community-maintained versions, some level of oversight will be needed
to ensure community maintainers are keeping up with their
commitments. Primarily this will consist of ensuring that community
maintainers have security patches ready to commit, and applied to
their branches, at the same time as affected supported versions in the
primary Django repository, but it may be useful to develop a stronger
metric for ensuring community maintenance lives up to its goals.
Coordinating security disclosure
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The Django core team has historically treated knowledge of security
issues with the utmost discretion, keeping details restricted to as
few people and entities as possible until the moment of public
disclosure.
Community maintainers will necessarily need to be aware of potential
security issues in the versions of Django they maintain, and may also
need access to security-reporting functionality for issues present in
their versions of Django but not in later, officially-supported
versions.
The core team will need to develop a plan for keeping community
maintainers appropriately in the loop while maintaining the currently
small, trusted circle of knowledge of security issues.
Warning users about unsupported Python versions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The primary use case for community maintenance is to allow extended
unofficial support for Django on Python versions which
officially-supported releases of Django no longer run on, particularly
Python 2.6. But Python 2.6 itself is also unsupported officially by
the Python core team; it is and has been past end-of-life for quite
some time.
As such, the only safe way to use Python 2.6 -- which does have known
security issues discovered past its end-of-life -- is when the Python
version itself is also receiving third-party support. Typically only a
handful of large enterprise operating-system vendors are willing to
provide such support, so only users who are customers of such vendors
will be able to safely use community-maintained versions of Django.
Large and loud warnings will be necessary on community-maintained
versions of Django, indicating to potential users that they should
*not* stay on a community-maintained version unless they are also
using an officially-supported version of Python, or are customers of
an operating-system vendor which backports security patches into its
supplied version of Python.
References
----------
[1] https://groups.google.com/forum/#!topic/django-users/B74QJaqA0b4
[2] https://github.com/django/django/
the impending end-of-life of Django 1.6, which in turn will mean
end-of-life for Django supported on Python 2.6. In addition, the
end-of-life of the current LTS release, Django 1.4, will mean
end-of-life for Django supported on Python 2.5. The following is a
draft of a proposal based on ideas raised in that thread; it should be
considered pre-DEP status, to be elevated to a DEP submission if there
is sufficient support.
Abstract
--------
The support process for older versions of Django is changed, as
follows:
* Official support for older versions of Django from the Django core
team will continue to end at the scheduled times (i.e., upon release
of a new minor version for non-LTS releases, at end of advertised
support lifetime for LTS releases).
* Unofficial support, by persons willing to volunteer their time, will
be permitted through a secondary code repository existing solely to
provide this support.
* As the official end-of-life of each Django version approaches, the
Django core team will solicit proposals and volunteers to provide
unofficial support for that version past its official end-of-life,
and select maintainer(s) from amongst those volunteers. This support
will be known as "community maintenance".
Rationale
---------
For the past five years, Django has been in the process of deprecating
support for older 2.x release of Python, as part of a plan to move
Django toward, first, compatibility with both Python 2 and Python 3
(completed as of Django 1.6), and ultimately with only Python 3 (in a
future, to-be-determined, release).
The timeline for this was as follows:
* May 2010: Django 1.2 instituted a minimum Python version of 2.4.
* March 2011: Django 1.3 was the final release to support Python 2.4.
* March 2012: Django 1.4 (LTS) raised the minimum Python version to
2.5, and was the final release to support Python 2.5.
* February 2013: Django 1.5 raised the minimum Python version to 2.6.
* November 2013: Django 1.6 was the final release to support Python
2.6.
* September 2014: Django 1.7 raised the minimum Python version to 2.7.
* Late Q1/early Q2 2015: Django 1.4 and Django 1.6 will reach
end-of-life (the former on a scheduled date, the latter upon the
release of Django 1.8).
While this process was necessary to achieve at least a minimum of
Python 2.6 (the lowest version enabling compatibility with Python 2
and Python 3 in a single codebase) and ultimately a minimum of Python
2.7 (the only Python 2.x release still receiving upstream security
support), and while moving Django in the direction of Python 3 is an
absolute necessity for the future of the framework, this has had an
effect on some users of Django who are effectively locked in to now-
or soon-to-be-unsupported versions of Python by their choice of
operating system and associated support contract.
In particular, the impending end-of-life of Django 1.6, the final
release which supported Python 2.6, is poised to cut off support for
users of products like Red Hat Enterprise Linux, who may be locked to
Python 2.6 and thus unable to use a supported version of Django past
1.6 end-of-life (these users continue to receive security support for
Python itself from the operating-system vendor, who provides such
support on their behalf even after official upstream support has
ended; no such support appears to be provided, at this time, for
Django).
Although these users are not a majority of Django's users, they are
still a significant group, and simply ending their ability to use
Django is likely to be an unacceptable option. But the Django core
team cannot and will not extend the official support lifetimes of
older Django releases; the additional maintenance burden is beyond
that which can be borne, and would impede the future development of
Django.
This is not the first time in Django's history that a significant
minority of users have been affected by backwards-incompatible
releases. Prior to Django 1.0, the large incompatibility of the
"magic-removal" branch, merged shortly before the release of Django
0.95, left some users effectively stranded with large already-existing
codebases that could not be quickly migrated.
During that time period, the Django core team allowed, on an
unofficial volunteer basis, continuing bugfix and security support to
be provided by interested parties for an extended period of time past
the official end-of-life of Django 0.91. This support was conducted
solely through commits to Django's (at the time Subversion)
repository, and never through issuing additional releases in an
end-of-life'd release series.
This experiment in unofficial support was successful overall; many
high-visibility installations of Django were able to migrate to
post-"magic-removal" versions of Django at the best pace they could
manage, while still being assured of necessary support for security
and crashing/data-loss issues on their existing installations as they
did so.
With the similar situation caused by the impending end-of-life of
Django 1.6, it seems appropriate to once again extend the option of
unofficial, community-based extended support as a measure to assist
such installations of Django during the time they will require to
migrate to a more recent Python or Django version.
Details
-------
Starting with Django 1.6, each time a minor (i.e., X.Y) release series
of Django is nearing its end-of-life, a three-phase process will
begin.
In the first phase, the Django core team will post publicly that a
release series of Django is near end-of-life, and will issue a call
for discussion on the possibility of extended community support for
the soon-to-EOL release series. If, in the opinion of the core team,
the resulting discussion produces convincing arguments for the
necessity of such support, the process will move to the second phase.
In the second phase, which only occurs following a decision by the
core team that extended community support is neeed, there will be a
public solicitation for volunteers to provide such support. Any
interested person may volunteer. Members of the Django core team are
as eligible as anyone else, but will need to take care to
differentiate this role from their work on officially-supported
releases.
The core team will then choose from among the volunteers at least one,
but preferably two or more, people -- herafter known as "community
maintainers" -- who will be given commit access to a repository for
community maintenance purposes. Community maintainers will be advised
prior to acceptance that this indicates a commitment on their part to
be vigilant and active in maintaining the relevant branch of Django,
and if any community maintainer declines or later fails to honor that
commitment (either by notifying the core team of an inability, or in
the core team's determination), the core team will solicit additional
volunteers. If no suitable persons can be found to act as community
maintainers, end-of-life will occur as normal, and no extended
community support will exist. Persons who later wish to remedy this by
becomimg community maintainers will be advised to contact the
django-developers list to initiate that process.
In the third phase, the community maintainers will, on their own
initiative, provide ongoing support, in the form of fixing security
and crashing/data-loss issues, in the relevant branch of Django in the
repository to which they are granted access. They will *not* become
members of the core team simply by virtue of their status as community
maintainers, and will *not* be granted access to any additional
repositories as a result of their status.
However, community maintainers will, at the discretion of the core
team, be included in pre-disclosure discussion of security issues, and
provided with sufficient details to backport security patches into, or
develop security patches for, their branches. Maintaining such
communications with community maintainers shall be the responsibiltiy
of the current Django Fellow or Django release manager, as
appropriate.
Concerns for discussion
-----------------------
There are several potential areas in this proposal which will require
discussion and ironing out of details prior to full
implementation. They are as follows.
Differentiating unofficial and official support
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Community maintainers will need to work in a repository other than the
primary Django development repository[2]. This is both to avoid
confusion between official and unofficial support, and because it is
not possible, in git, to give commit access to only a specific branch
or portion of a repository, and community maintainers will not (as
noted above) become committers on Django as a result of being
community maintainers.
However, the community maintenance repository should still be
somewhere under the Django organzation on GitHub, both for visibility
and for ease of administration. The exact location and name of the
repository, and its access-rights policies, will need to be decided
prior to commencing use by community maintainers.
Ensuring active work by community maintainers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Although the Django core team will not be providing direct support for
community-maintained versions, some level of oversight will be needed
to ensure community maintainers are keeping up with their
commitments. Primarily this will consist of ensuring that community
maintainers have security patches ready to commit, and applied to
their branches, at the same time as affected supported versions in the
primary Django repository, but it may be useful to develop a stronger
metric for ensuring community maintenance lives up to its goals.
Coordinating security disclosure
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The Django core team has historically treated knowledge of security
issues with the utmost discretion, keeping details restricted to as
few people and entities as possible until the moment of public
disclosure.
Community maintainers will necessarily need to be aware of potential
security issues in the versions of Django they maintain, and may also
need access to security-reporting functionality for issues present in
their versions of Django but not in later, officially-supported
versions.
The core team will need to develop a plan for keeping community
maintainers appropriately in the loop while maintaining the currently
small, trusted circle of knowledge of security issues.
Warning users about unsupported Python versions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The primary use case for community maintenance is to allow extended
unofficial support for Django on Python versions which
officially-supported releases of Django no longer run on, particularly
Python 2.6. But Python 2.6 itself is also unsupported officially by
the Python core team; it is and has been past end-of-life for quite
some time.
As such, the only safe way to use Python 2.6 -- which does have known
security issues discovered past its end-of-life -- is when the Python
version itself is also receiving third-party support. Typically only a
handful of large enterprise operating-system vendors are willing to
provide such support, so only users who are customers of such vendors
will be able to safely use community-maintained versions of Django.
Large and loud warnings will be necessary on community-maintained
versions of Django, indicating to potential users that they should
*not* stay on a community-maintained version unless they are also
using an officially-supported version of Python, or are customers of
an operating-system vendor which backports security patches into its
supplied version of Python.
References
----------
[1] https://groups.google.com/forum/#!topic/django-users/B74QJaqA0b4
[2] https://github.com/django/django/
Collin Anderson
Mar 20, 2015, 10:26:36 AM3/20/15
to django-d...@googlegroups.com
Hi All,
Two thoughts:
- This doesn't _need_ to happen through Django officially, but doing it this way has the advantage of receiving pre-notify and the having some of Django's blessing gives more confidence to anyone using it.
- I'm not saying anyone wants this, but I could also see the possibility of volunteers wanting to continue 1.4's support on 2.6 and 2.7 only (dropping 2.5 support).
If it helps, I have the historical Python support in table form:
Collin
Carl Meyer
Mar 20, 2015, 1:02:40 PM3/20/15
to django-d...@googlegroups.com
Hi James,
Thanks for taking the time to write this up carefully, research the
history, etc. I think some form of extended community-based support
could work, but I have some concerns about your specific proposal;
mostly, that it places too much of the responsibility with the core team.
My feeling is that we don't really need a DEP or this much process
specified at this point; it'd be plenty for now to simply express
guarded openness to the idea of extended community maintenance for 1.6
and/or 1.4 and see if anyone actually steps up to do the work.
I'm also curious how this sounds to Christian and Stephen, who asked for
the extended support on django-users. Since they're the only ones who
have asked, if a proposal along these lines won't work for their cases,
then there's not much point in going forward with it.
Comments interspersed:
On 03/19/2015 04:59 PM, James Bennett wrote:
> There's been some recent discussion in a django-users thread[1] about
> the impending end-of-life of Django 1.6, which in turn will mean
> end-of-life for Django supported on Python 2.6. In addition, the
> end-of-life of the current LTS release, Django 1.4, will mean
> end-of-life for Django supported on Python 2.5. The following is a
> draft of a proposal based on ideas raised in that thread; it should be
> considered pre-DEP status, to be elevated to a DEP submission if there
> is sufficient support.
>
>
> Abstract
> --------
>
> The support process for older versions of Django is changed, as
> follows:
>
> * Official support for older versions of Django from the Django core
> team will continue to end at the scheduled times (i.e., upon release
> of a new minor version for non-LTS releases, at end of advertised
> support lifetime for LTS releases).
>
> * Unofficial support, by persons willing to volunteer their time, will
> be permitted through a secondary code repository existing solely to
> provide this support.
"Permitted" sounds odd to me, since regardless of this DEP I don't think
the core team is in a position to forbid anyone else from continuing to
patch older Django releases if they want to. (I suppose if we really
wanted to we could prevent them from using the Django trademark to
describe what they are doing, but IANAL.)
I'm also not convinced we should provide a special repo under the
django/ org for this. That doesn't bring any technical benefits, just
social ones (perception of "officialness"). And I'm not clear that
perception of officialness is a good thing - after all, this support is
unofficial. So I would lean towards saying that this concept should be
well-proven in a third-party repo (created by whoever is doing the work)
before we even consider a django/ repo for it.
> * As the official end-of-life of each Django version approaches, the
> Django core team will solicit proposals and volunteers to provide
> unofficial support for that version past its official end-of-life,
> and select maintainer(s) from amongst those volunteers. This support
> will be known as "community maintenance".
This is my strongest objection to the proposal. This whole scheme will
only work if the community maintainer is highly motivated, and there's a
strong need for the extended community maintenance for a specific
release. One indicator of both is that the initiative comes unsolicited
from the community. I think it makes sense to clarify our general
openness to such initiative, but I'm opposed to adding an additional
responsibility to the core team's plate to "solicit proposals and
volunteers" for community maintenance of every release that hits EOL.
I think that there should be no such initiative taken by the core team
for every release; instead this DEP (if we even need a DEP at all)
should simply state in general terms that the core team is open to this,
should anyone wish to do it.
> In the second phase, which only occurs following a decision by the
> core team that extended community support is neeed, there will be a
> public solicitation for volunteers to provide such support. Any
> interested person may volunteer. Members of the Django core team are
> as eligible as anyone else, but will need to take care to
> differentiate this role from their work on officially-supported
> releases.
>
> The core team will then choose from among the volunteers at least one,
> but preferably two or more, people -- herafter known as "community
> maintainers" -- who will be given commit access to a repository for
> community maintenance purposes. Community maintainers will be advised
> prior to acceptance that this indicates a commitment on their part to
> be vigilant and active in maintaining the relevant branch of Django,
> and if any community maintainer declines or later fails to honor that
> commitment (either by notifying the core team of an inability, or in
> the core team's determination), the core team will solicit additional
> volunteers. If no suitable persons can be found to act as community
> maintainers, end-of-life will occur as normal, and no extended
> community support will exist. Persons who later wish to remedy this by
> becomimg community maintainers will be advised to contact the
> django-developers list to initiate that process.
Again, I think these two paragraphs put much more responsibility in the
hands of the core team than I would like. I would favor a much shorter
DEP which basically just affirms our openness to something which is
already entirely possible (somebody continuing to patch a fork of an EOL
Django release), adds an openness to granting security pre-notification
to such community maintainers, and doesn't otherwise add any new core
team responsibilities or mention any grants of access to any special repos.
I think "posting on django-developers and django-users" is sufficient
visibility, and there should not be a special community maintenance repo
under github.com/django.
> Ensuring active work by community maintainers
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> Although the Django core team will not be providing direct support for
> community-maintained versions, some level of oversight will be needed
> to ensure community maintainers are keeping up with their
> commitments. Primarily this will consist of ensuring that community
> maintainers have security patches ready to commit, and applied to
> their branches, at the same time as affected supported versions in the
> primary Django repository, but it may be useful to develop a stronger
> metric for ensuring community maintenance lives up to its goals.
Again, I would not place this responsibility on the core team. If
community maintainers don't actually do the community maintenance, so it
goes; apparently it wasn't that badly needed after all. The core team
should explicitly _not_ take on the responsibility of "ensuring that
I agree that if any mention is made in the Django docs or on the Django
website of the community-support, it should come with such warnings
about unsupported Python versions.
>
> References
> ----------
>
> [1] https://groups.google.com/forum/#!topic/django-users/B74QJaqA0b4
> [2] https://github.com/django/django/
>
> --
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-develop...@googlegroups.com
> <mailto:django-develop...@googlegroups.com>.
> To post to this group, send email to django-d...@googlegroups.com
> <mailto:django-d...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAL13Cg_LQLQXfmvONJKSArsU96oQ_jzgurNQrxHxCLTOf7-BSg%40mail.gmail.com
> <https://groups.google.com/d/msgid/django-developers/CAL13Cg_LQLQXfmvONJKSArsU96oQ_jzgurNQrxHxCLTOf7-BSg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.
Thanks for taking the time to write this up carefully, research the
history, etc. I think some form of extended community-based support
could work, but I have some concerns about your specific proposal;
mostly, that it places too much of the responsibility with the core team.
My feeling is that we don't really need a DEP or this much process
specified at this point; it'd be plenty for now to simply express
guarded openness to the idea of extended community maintenance for 1.6
and/or 1.4 and see if anyone actually steps up to do the work.
I'm also curious how this sounds to Christian and Stephen, who asked for
the extended support on django-users. Since they're the only ones who
have asked, if a proposal along these lines won't work for their cases,
then there's not much point in going forward with it.
Comments interspersed:
On 03/19/2015 04:59 PM, James Bennett wrote:
> There's been some recent discussion in a django-users thread[1] about
> the impending end-of-life of Django 1.6, which in turn will mean
> end-of-life for Django supported on Python 2.6. In addition, the
> end-of-life of the current LTS release, Django 1.4, will mean
> end-of-life for Django supported on Python 2.5. The following is a
> draft of a proposal based on ideas raised in that thread; it should be
> considered pre-DEP status, to be elevated to a DEP submission if there
> is sufficient support.
>
>
> Abstract
> --------
>
> The support process for older versions of Django is changed, as
> follows:
>
> * Official support for older versions of Django from the Django core
> team will continue to end at the scheduled times (i.e., upon release
> of a new minor version for non-LTS releases, at end of advertised
> support lifetime for LTS releases).
>
> * Unofficial support, by persons willing to volunteer their time, will
> be permitted through a secondary code repository existing solely to
> provide this support.
the core team is in a position to forbid anyone else from continuing to
patch older Django releases if they want to. (I suppose if we really
wanted to we could prevent them from using the Django trademark to
describe what they are doing, but IANAL.)
I'm also not convinced we should provide a special repo under the
django/ org for this. That doesn't bring any technical benefits, just
social ones (perception of "officialness"). And I'm not clear that
perception of officialness is a good thing - after all, this support is
unofficial. So I would lean towards saying that this concept should be
well-proven in a third-party repo (created by whoever is doing the work)
before we even consider a django/ repo for it.
> * As the official end-of-life of each Django version approaches, the
> Django core team will solicit proposals and volunteers to provide
> unofficial support for that version past its official end-of-life,
> and select maintainer(s) from amongst those volunteers. This support
> will be known as "community maintenance".
only work if the community maintainer is highly motivated, and there's a
strong need for the extended community maintenance for a specific
release. One indicator of both is that the initiative comes unsolicited
from the community. I think it makes sense to clarify our general
openness to such initiative, but I'm opposed to adding an additional
responsibility to the core team's plate to "solicit proposals and
volunteers" for community maintenance of every release that hits EOL.
for every release; instead this DEP (if we even need a DEP at all)
should simply state in general terms that the core team is open to this,
should anyone wish to do it.
> In the second phase, which only occurs following a decision by the
> core team that extended community support is neeed, there will be a
> public solicitation for volunteers to provide such support. Any
> interested person may volunteer. Members of the Django core team are
> as eligible as anyone else, but will need to take care to
> differentiate this role from their work on officially-supported
> releases.
>
> The core team will then choose from among the volunteers at least one,
> but preferably two or more, people -- herafter known as "community
> maintainers" -- who will be given commit access to a repository for
> community maintenance purposes. Community maintainers will be advised
> prior to acceptance that this indicates a commitment on their part to
> be vigilant and active in maintaining the relevant branch of Django,
> and if any community maintainer declines or later fails to honor that
> commitment (either by notifying the core team of an inability, or in
> the core team's determination), the core team will solicit additional
> volunteers. If no suitable persons can be found to act as community
> maintainers, end-of-life will occur as normal, and no extended
> community support will exist. Persons who later wish to remedy this by
> becomimg community maintainers will be advised to contact the
> django-developers list to initiate that process.
hands of the core team than I would like. I would favor a much shorter
DEP which basically just affirms our openness to something which is
already entirely possible (somebody continuing to patch a fork of an EOL
Django release), adds an openness to granting security pre-notification
to such community maintainers, and doesn't otherwise add any new core
team responsibilities or mention any grants of access to any special repos.
visibility, and there should not be a special community maintenance repo
under github.com/django.
> Ensuring active work by community maintainers
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> Although the Django core team will not be providing direct support for
> community-maintained versions, some level of oversight will be needed
> to ensure community maintainers are keeping up with their
> commitments. Primarily this will consist of ensuring that community
> maintainers have security patches ready to commit, and applied to
> their branches, at the same time as affected supported versions in the
> primary Django repository, but it may be useful to develop a stronger
> metric for ensuring community maintenance lives up to its goals.
community maintainers don't actually do the community maintenance, so it
goes; apparently it wasn't that badly needed after all. The core team
should explicitly _not_ take on the responsibility of "ensuring that
community maintainers have security patches ready to commit, and applied
to their branches, at the same time as affected supported versions in
the primary Django repository."
to their branches, at the same time as affected supported versions in
website of the community-support, it should come with such warnings
about unsupported Python versions.
>
> References
> ----------
>
> [1] https://groups.google.com/forum/#!topic/django-users/B74QJaqA0b4
> [2] https://github.com/django/django/
>
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-develop...@googlegroups.com
> <mailto:django-develop...@googlegroups.com>.
> To post to this group, send email to django-d...@googlegroups.com
> <mailto:django-d...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAL13Cg_LQLQXfmvONJKSArsU96oQ_jzgurNQrxHxCLTOf7-BSg%40mail.gmail.com
> <https://groups.google.com/d/msgid/django-developers/CAL13Cg_LQLQXfmvONJKSArsU96oQ_jzgurNQrxHxCLTOf7-BSg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.
Aymeric Augustin
Mar 20, 2015, 4:53:43 PM3/20/15
to django-d...@googlegroups.com
Hi James,
When I read your proposal I thought -- "this makes sense, but if maintainers
are making this commitment, they should be part of the Django team".
Only after reading Carl's comment did I realize that they shouldn't be part of
the team in order to keep the "unofficial" aspect.
This makes me think that you've drawn the line too close to official extended
support.
Otherwise I'm +0 on the general concept, mostly because I'm lucky enough to
care about Python 3.4 only ;-)
--
Aymeric.
Christian Hammond
Mar 26, 2015, 7:11:12 PM3/26/15
to django-d...@googlegroups.com
Hi Carl, James,
Sorry for the late reply on this. I've been unavailable the past week.
I know you guys are still sorting out how you want to run this, but I wanted to let you know that, given our current dependence on Python 2.6, I'm willing to do what's needed to maintain security backports for either an official or unofficial fork.
I'd love to know some further thoughts on the logistics for this. Things like how versioning would work, what you'd need from me, what kind of timeframes we'd have to work with, any legalese needed to be notified of security issues, etc.
Thanks!
Christian
> <mailto:django-developers+unsub...@googlegroups.com>.
Tim Graham
Mar 27, 2015, 10:06:17 AM3/27/15
to django-d...@googlegroups.com
Instructions for requesting to be added to the security prenotification list are here: https://docs.djangoproject.com/en/dev/internals/security/#requesting-notifications
John Paulett
Mar 27, 2015, 11:47:11 AM3/27/15
to django-d...@googlegroups.com
James, thanks for putting this together.
Christian, I am in a similar position, supporting 2.6 for the next 6-12 months. I had planned to keep a personal fork of Django 1.6, backporting security patches as needed, but I would be happy to contribute to a common effort in this regard.
As mentioned by others, keeping the effort unofficial may be ideal. A planed end-of-life of this secondary support may also be helpful to still convince people to migrate forward.
For a personal fork this wasn't going to be needed, but I wonder how package distribution should occur. I doubt that publishing unofficial distributable under Django's PyPI project is advisable.
John
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4d1572ce-820d-4032-bfaa-8fc0a48d1862%40googlegroups.com.
Carl Meyer
Mar 27, 2015, 12:25:20 PM3/27/15
to django-d...@googlegroups.com
Hi John and Christian,
On 03/27/2015 09:46 AM, John Paulett wrote:
> James, thanks for putting this together.
>
> Christian, I am in a similar position, supporting 2.6 for the next 6-12
> months. I had planned to keep a personal fork of Django 1.6,
> backporting security patches as needed, but I would be happy to
> contribute to a common effort in this regard.
>
> As mentioned by others, keeping the effort unofficial may be ideal. A
> planed end-of-life of this secondary support may also be helpful to
> still convince people to migrate forward.
>
> For a personal fork this wasn't going to be needed, but I wonder how
> package distribution should occur. I doubt that publishing unofficial
> distributable under Django's PyPI project is advisable.
I agree.
I'll outline my idea of how this could work; consider it an alternative
proposal (or proposed revision of) James' DEP. (I don't think a DEP is
needed for this, but if other core team members feel that it is, I am
willing to work this up into DEP format).
- The community support work happens in a third-party GitHub fork. If
you and Christian want to work together on it for 1.6, one of you can
create a repo and grant the other commit access. (This is of course
something that you have every right to do regardless of what the core
team thinks of it.)
- One of you applies for security pre-notifications (under the
"distributor of Django" category -- essentially you are applying to be
equivalent to e.g. a Debian or Fedora packager of Django, who
could/might provide the same kind of extended support you are planning to.)
- I think from the core team's perspective, the ideal situation would be
that these releases not be on PyPI at all, but that your users would
download them directly from the "releases" section of your github repo,
or perhaps pip-install them from a PyPI-compatible index that you
publish somewhere on static file hosting. If this is acceptable, it
would allow you to use PEP 440 "local version identifiers" [1] in your
versions, which I think is the most semantically correct for this
situation. So your versions would look something like
"1.6.11+extended-1" (choice of actual local-version label up to you).
If not-on-PyPI is a total dealbreaker for your users, it might be an
option to release under a different PyPI project name, such as
"Django-16-Community-Support" or similar, but this is less than ideal.
For one, it would require the actual package to have a name other than
"Django", and it would require the use of something like "1.6.11.post1"
instead of the local version identifier, since PyPI does not accept
local version identifiers.
- A preamble should be added to the README of the project on GitHub to
indicate that this is unofficial community-based extended support for
Django 1.6, intended only for users of distributions who are providing
extended security support for Python 2.6, and should only be used under
those circumstances. It should clearly warn that Python 2.6 is EOL and
should not be used unless your distro is continuing to support it. It
should also clearly indicate that the Django core team is not providing
this extended support, and point to the correct GitHub repo issues
tracker for raising any issues with it. The same information should be
provided in public announcements about the extended support, web pages
about it, etc.
How does this sound to you, Christian and John?
How does it sound to other members of the core team? Any objections to
any of what I proposed?
Thanks,
Carl
[1] https://www.python.org/dev/peps/pep-0440/#local-version-identifiers
On 03/27/2015 09:46 AM, John Paulett wrote:
> James, thanks for putting this together.
>
> Christian, I am in a similar position, supporting 2.6 for the next 6-12
> months. I had planned to keep a personal fork of Django 1.6,
> backporting security patches as needed, but I would be happy to
> contribute to a common effort in this regard.
>
> As mentioned by others, keeping the effort unofficial may be ideal. A
> planed end-of-life of this secondary support may also be helpful to
> still convince people to migrate forward.
>
> For a personal fork this wasn't going to be needed, but I wonder how
> package distribution should occur. I doubt that publishing unofficial
> distributable under Django's PyPI project is advisable.
I'll outline my idea of how this could work; consider it an alternative
proposal (or proposed revision of) James' DEP. (I don't think a DEP is
needed for this, but if other core team members feel that it is, I am
willing to work this up into DEP format).
- The community support work happens in a third-party GitHub fork. If
you and Christian want to work together on it for 1.6, one of you can
create a repo and grant the other commit access. (This is of course
something that you have every right to do regardless of what the core
team thinks of it.)
- One of you applies for security pre-notifications (under the
"distributor of Django" category -- essentially you are applying to be
equivalent to e.g. a Debian or Fedora packager of Django, who
could/might provide the same kind of extended support you are planning to.)
- I think from the core team's perspective, the ideal situation would be
that these releases not be on PyPI at all, but that your users would
download them directly from the "releases" section of your github repo,
or perhaps pip-install them from a PyPI-compatible index that you
publish somewhere on static file hosting. If this is acceptable, it
would allow you to use PEP 440 "local version identifiers" [1] in your
versions, which I think is the most semantically correct for this
situation. So your versions would look something like
"1.6.11+extended-1" (choice of actual local-version label up to you).
If not-on-PyPI is a total dealbreaker for your users, it might be an
option to release under a different PyPI project name, such as
"Django-16-Community-Support" or similar, but this is less than ideal.
For one, it would require the actual package to have a name other than
"Django", and it would require the use of something like "1.6.11.post1"
instead of the local version identifier, since PyPI does not accept
local version identifiers.
- A preamble should be added to the README of the project on GitHub to
indicate that this is unofficial community-based extended support for
Django 1.6, intended only for users of distributions who are providing
extended security support for Python 2.6, and should only be used under
those circumstances. It should clearly warn that Python 2.6 is EOL and
should not be used unless your distro is continuing to support it. It
should also clearly indicate that the Django core team is not providing
this extended support, and point to the correct GitHub repo issues
tracker for raising any issues with it. The same information should be
provided in public announcements about the extended support, web pages
about it, etc.
How does this sound to you, Christian and John?
How does it sound to other members of the core team? Any objections to
any of what I proposed?
Thanks,
Carl
[1] https://www.python.org/dev/peps/pep-0440/#local-version-identifiers
Carl Meyer
Mar 27, 2015, 12:28:09 PM3/27/15
to django-d...@googlegroups.com
Hi Christian,
On 03/26/2015 05:11 PM, Christian Hammond wrote:
> I know you guys are still sorting out how you want to run this, but I
> wanted to let you know that, given our current dependence on Python 2.6,
> I'm willing to do what's needed to maintain security backports for
> either an official or unofficial fork.
>
> I'd love to know some further thoughts on the logistics for this. Things
> like how versioning would work, what you'd need from me, what kind of
> timeframes we'd have to work with, any legalese needed to be notified of
> security issues, etc.
Tim pointed you to the details for requesting security pre-notification.
We pre-notify a week in advance, so that's the timeframe you'd have
available to get a patch (or patches) prepped to apply to your fork.
Obviously we'd expect that you wouldn't push any patches to a public
fork until the embargo ends with our release announcement; you could
manage that by simply staging patches on a local machine, or by
coordinating patches during the embargo in a private GitHub repo.
Carl
On 03/26/2015 05:11 PM, Christian Hammond wrote:
> I know you guys are still sorting out how you want to run this, but I
> wanted to let you know that, given our current dependence on Python 2.6,
> I'm willing to do what's needed to maintain security backports for
> either an official or unofficial fork.
>
> I'd love to know some further thoughts on the logistics for this. Things
> like how versioning would work, what you'd need from me, what kind of
> timeframes we'd have to work with, any legalese needed to be notified of
> security issues, etc.
We pre-notify a week in advance, so that's the timeframe you'd have
available to get a patch (or patches) prepped to apply to your fork.
Obviously we'd expect that you wouldn't push any patches to a public
fork until the embargo ends with our release announcement; you could
manage that by simply staging patches on a local machine, or by
coordinating patches during the embargo in a private GitHub repo.
Carl
Tim Graham
Mar 27, 2015, 12:36:14 PM3/27/15
to django-d...@googlegroups.com
Sounds good to me.
Marc Tamlyn
Mar 27, 2015, 6:43:50 PM3/27/15
to django-d...@googlegroups.com
Sounds broadly good to me. I'd be happy for it to be pushed to pypi under django-legacy or similar name.
Marc
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/3eaf7467-90bc-41d6-b903-6b3198bd38d2%40googlegroups.com.
John Paulett
Apr 1, 2015, 4:36:17 PM4/1/15
to django-d...@googlegroups.com
Carl, your proposal sounds good to me. I would be happy to contribute to a DEP, if formalization of the process is necessary.
Christian, let's coordinate to set up a public/private repo pair with the appropriate warnings and have one of us apply for the pre-release notifications (if you haven't already).
Personally, I want to ensure we stay true to the core team's wishes. So as we proceed, please reach out if there is anything we could be doing better, particularly clarification that this is an unofficial community effort and encouraging the transition to the official release.
John
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMwjO1EizLpYtKMjrshPLn%3DspjuyjtnW04LwEH23v0NyJFckSg%40mail.gmail.com.
Christian Hammond
Aug 18, 2015, 10:01:58 PM8/18/15
to Django developers (Contributions to Django itself)
Hi Carl,
That means consumers of the builds either need to specify an explicit version (preventing other packages or administrators from choosing a higher version when a new build is released), or need to depend on the official 1.6.11 and rely on administrators to keep up-to-date with the latest releases and manually install them.
I know it's been a while since we discussed this, but today's security release is the first one that's really affecting our product and we've finally got things in shape to be able to start distributing unofficial Django security releases (we've also just been swamped since our discussion earlier this year).
I wanted to dig into the versioning scheme just a bit. We tried going with local version identifiers, but it turns out these are fairly useless. When specifying a dependency with a local version identifier, your only option is direct equality (==1.6.11+extended.1). You can't use any form of range (>=1.6.11+extended.4), as these get interpreted as >=1.6.11. This is the case with both easy_install and pip, and is documented in the PEP.
That means consumers of the builds either need to specify an explicit version (preventing other packages or administrators from choosing a higher version when a new build is released), or need to depend on the official 1.6.11 and rely on administrators to keep up-to-date with the latest releases and manually install them.
Not to mention that this all falls apart if using setuptools < 8, which doesn't support these identifiers.
Given that, what would your feelings be on allowing for 1.6.11.x releases? This way we wouldn't take up any possible future versioning slots (unlikely as they may be), while being compatible with all versions of setuptools, and compatible with specifying version ranges. This would still appear solely on our own downloads page and announcements, with an appropriate note on them being unofficial builds.
Also as a status update, we've started a fork and applied for the pre-notification list. I've backported all current security fixes to a branch, ensured the test suite passes with flying colors, and have added a README detailing everything you've requested. This is all up at https://github.com/beanbaginc/django. Let me know if there's anything you'd like changed.
Thanks,
Christian
Carl Meyer
Aug 19, 2015, 11:25:58 AM8/19/15
to django-d...@googlegroups.com
Hi Christian,
On 08/18/2015 08:01 PM, Christian Hammond wrote:
> I know it's been a while since we discussed this, but today's security
> release is the first one that's really affecting our product and we've
> finally got things in shape to be able to start distributing unofficial
> Django security releases (we've also just been swamped since our
> discussion earlier this year).
>
> I wanted to dig into the versioning scheme just a bit. We tried going
> with local version identifiers, but it turns out these are fairly
> useless...
[snip]
a bit more strongly, but after re-reading PEP 440 I don't think it
leaves us with any good options. I considered post-releases (e.g.
1.6.11.post1), but "The use of post-releases to publish maintenance
releases containing actual bug fixes is strongly discouraged." So given
the lack of good options, I'm OK with 1.6.11.x. Anyone else on the core
team have a problem with that?
> Also as a status update, we've started a fork and applied for the
> pre-notification list. I've backported all current security fixes to a
> branch, ensured the test suite passes with flying colors, and have added
> a README detailing everything you've requested. This is all up at
> https://github.com/beanbaginc/django. Let me know if there's anything
> you'd like changed.
The README looks great, thanks!
Carl
On 08/18/2015 08:01 PM, Christian Hammond wrote:
> I know it's been a while since we discussed this, but today's security
> release is the first one that's really affecting our product and we've
> finally got things in shape to be able to start distributing unofficial
> Django security releases (we've also just been swamped since our
> discussion earlier this year).
>
> I wanted to dig into the versioning scheme just a bit. We tried going
> with local version identifiers, but it turns out these are fairly
[snip]
> Given that, what would your feelings be on allowing for 1.6.11.x
> releases? This way we wouldn't take up any possible future versioning
> slots (unlikely as they may be), while being compatible with all
> versions of setuptools, and compatible with specifying version ranges.
> This would still appear solely on our own downloads page and
> announcements, with an appropriate note on them being unofficial builds.
In my ideal world, the version number would help convey unofficial-ness
> releases? This way we wouldn't take up any possible future versioning
> slots (unlikely as they may be), while being compatible with all
> versions of setuptools, and compatible with specifying version ranges.
> This would still appear solely on our own downloads page and
> announcements, with an appropriate note on them being unofficial builds.
a bit more strongly, but after re-reading PEP 440 I don't think it
leaves us with any good options. I considered post-releases (e.g.
1.6.11.post1), but "The use of post-releases to publish maintenance
releases containing actual bug fixes is strongly discouraged." So given
the lack of good options, I'm OK with 1.6.11.x. Anyone else on the core
team have a problem with that?
> Also as a status update, we've started a fork and applied for the
> pre-notification list. I've backported all current security fixes to a
> branch, ensured the test suite passes with flying colors, and have added
> a README detailing everything you've requested. This is all up at
> https://github.com/beanbaginc/django. Let me know if there's anything
> you'd like changed.
Carl
Donald Stufft
Aug 19, 2015, 11:28:13 AM8/19/15
to Carl Meyer, django-d...@googlegroups.com
>
> > Also as a status update, we've started a fork and applied for the
> > pre-notification list. I've backported all current security fixes to a
> > branch, ensured the test suite passes with flying colors, and have added
> > a README detailing everything you've requested. This is all up at
> > https://github.com/beanbaginc/django. Let me know if there's anything
> > you'd like changed.
>
> The README looks great, thanks!
>
> Carl
>
> --
> You received this message because you are subscribed to the Google Groups "Django developers
> (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/55D49FEE.80503%40oddbird.net.
> You received this message because you are subscribed to the Google Groups "Django developers
> (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Carl Meyer
Aug 19, 2015, 11:31:50 AM8/19/15
to Donald Stufft, django-d...@googlegroups.com
On 08/19/2015 09:28 AM, Donald Stufft wrote:
> On August 19, 2015 at 11:25:55 AM, Carl Meyer (ca...@oddbird.net) wrote:
>> In my ideal world, the version number would help convey unofficial-ness
>> a bit more strongly, but after re-reading PEP 440 I don't think it
>> leaves us with any good options. I considered post-releases (e.g.
>> 1.6.11.post1), but "The use of post-releases to publish maintenance
>> releases containing actual bug fixes is strongly discouraged." So given
>> the lack of good options, I'm OK with 1.6.11.x. Anyone else on the core
>> team have a problem with that?
>
>
> 1.6.11.x should work fine, though I’m confused why not just issue 1.6.12+?
Because that looks exactly like the version number an official Django
> On August 19, 2015 at 11:25:55 AM, Carl Meyer (ca...@oddbird.net) wrote:
>> In my ideal world, the version number would help convey unofficial-ness
>> a bit more strongly, but after re-reading PEP 440 I don't think it
>> leaves us with any good options. I considered post-releases (e.g.
>> 1.6.11.post1), but "The use of post-releases to publish maintenance
>> releases containing actual bug fixes is strongly discouraged." So given
>> the lack of good options, I'm OK with 1.6.11.x. Anyone else on the core
>> team have a problem with that?
>
>
> 1.6.11.x should work fine, though I’m confused why not just issue 1.6.12+?
release would use, and the idea is to be as clear as possible that these
are not official Django releases. For some users (who don't read READMEs
etc) the version number is one of the few things we can be pretty sure
they'll see.
Carl
Marc Tamlyn
Aug 19, 2015, 11:35:52 AM8/19/15
to django-d...@googlegroups.com, Donald Stufft
Very happy with 1.6.11.x
Carl
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/55D4A159.5050605%40oddbird.net.
Donald Stufft
Aug 19, 2015, 11:38:40 AM8/19/15
to Carl Meyer, django-d...@googlegroups.com
Carl Meyer
Aug 19, 2015, 11:44:34 AM8/19/15
to Donald Stufft, django-d...@googlegroups.com
On 08/19/2015 09:38 AM, Donald Stufft wrote:
> Do we have any evidence that users will notice a fourth digit and
> will make the mental association that they are not official releases?
> I wouldn’t if I hadn’t read this thread. I mean I don’t actually care
> if they use 1.6.11.x or 1.6.12+, I just don’t think it’s really
> buying anything extra. If you want the version number to contain
> something that says “unofficial” just tack an unofficial local
> version number on the end of whichever solution is picked (so
> 1.6.11.5+unofficial or 1.6.12+unofficial) or whatever.
That's a fair point, and IMO an even better suggestion (tacking on
> Do we have any evidence that users will notice a fourth digit and
> will make the mental association that they are not official releases?
> I wouldn’t if I hadn’t read this thread. I mean I don’t actually care
> if they use 1.6.11.x or 1.6.12+, I just don’t think it’s really
> buying anything extra. If you want the version number to contain
> something that says “unofficial” just tack an unofficial local
> version number on the end of whichever solution is picked (so
> 1.6.11.5+unofficial or 1.6.12+unofficial) or whatever.
+something for clarity even though it's not technically needed). If we
do that I don't care much between 1.6.11.x vs 1.6.12+, though I guess I
still slightly lean towards the former, as it doesn't constrain the
Django project's options in the (vanishingly unlikely) scenario that
we'd ever want to release something new in the 1.6.x series.
Carl
Carl Meyer
Aug 19, 2015, 12:09:00 PM8/19/15
to django-d...@googlegroups.com
Hi Christian,
On 08/18/2015 08:01 PM, Christian Hammond wrote:
[snip]
On 08/18/2015 08:01 PM, Christian Hammond wrote:
> Also as a status update, we've started a fork and applied for the
> pre-notification list. I've backported all current security fixes to a
> branch, ensured the test suite passes with flying colors, and have added
> a README detailing everything you've requested. This is all up at
> https://github.com/beanbaginc/django. Let me know if there's anything
> you'd like changed.
One thing was pointed out to me just now -- the downloads page _really_
> pre-notification list. I've backported all current security fixes to a
> branch, ensured the test suite passes with flying colors, and have added
> a README detailing everything you've requested. This is all up at
> https://github.com/beanbaginc/django. Let me know if there's anything
> you'd like changed.
should use TLS. Any chance you could make that happen?
Carl
Christian Hammond
Aug 19, 2015, 5:42:18 PM8/19/15
to django-d...@googlegroups.com
Hey everyone,
Thanks for the quick feedback on this :)
Regarding the version number, from what I've read, there's still a reported compatibility issue if we use the +localidentifer part of a version number with older versions of setuptools. I need to do some testing on this. If I don't hit any issues, I'm very happy to put an identifier on there. I would like to keep it at 1.6.11.x, rather than 1.6.12+, so that it's very clear we're based on upstream Django 1.6.11.
We're looking into an approach that spins up a couple VMs that essentially wrap S3 with an HTTPS endpoint, but it increases our costs a bit, and we're just not ready to do that yet. I want HTTPS on this pretty badly, though, so we might try that approach when we can get to it.
As for the downloads page, we're hosting everything on S3, as opposed to a dedicated host. This is so that our deliverables will be replicated and still be up if either Amazon or we have some outage. We *can* get HTTPS on this, using CloudFront, but that uses SNI, meaning only the latest Python 2.7.x releases (and 3.x) would be able to download files from there.
We're looking into an approach that spins up a couple VMs that essentially wrap S3 with an HTTPS endpoint, but it increases our costs a bit, and we're just not ready to do that yet. I want HTTPS on this pretty badly, though, so we might try that approach when we can get to it.
To attempt to compensate for this, though, we sign all the builds and include checksums in both the links and in dedicated .sha256sum files. Not the best solution, but it's something at least :/
Christian
--
Christian Hammond - chi...@chipx86.com
Review Board - https://www.reviewboard.org
Christian Hammond - chi...@chipx86.com
Review Board - https://www.reviewboard.org
Beanbag, Inc. - https://www.beanbaginc.com
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/69fOquu8v-U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/55D4AA04.3090403%40oddbird.net.
Markus Holtermann
Aug 19, 2015, 7:43:33 PM8/19/15
to django-d...@googlegroups.com
Hey Christian,
On Wed, Aug 19, 2015 at 02:41:49PM -0700, Christian Hammond wrote:
>Regarding the version number, from what I've read, there's still a reported
>compatibility issue if we use the +localidentifer part of a version number
>with older versions of setuptools. I need to do some testing on this. If I
>don't hit any issues, I'm very happy to put an identifier on there. I would
>like to keep it at 1.6.11.x, rather than 1.6.12+, so that it's very clear
>we're based on upstream Django 1.6.11.
Thanks, that sounds good to me. 1.6.11.x+unofficial if possible,
otherwise 1.6.11.x
>I want HTTPS on this pretty badly, though, so we might try that
>approach when we can get to it.
Good :)
>To attempt to compensate for this, though, we sign all the builds and
>include checksums in both the links and in dedicated .sha256sum files. Not
>the best solution, but it's something at least :/
Checksums alone won't provide integrity checks if I can play MITM. Only
(e.g. pgp) signatures will help. But then make sure the key ID is
published on an HTTPS page ;)
/Markus
--
On Wed, Aug 19, 2015 at 02:41:49PM -0700, Christian Hammond wrote:
>Regarding the version number, from what I've read, there's still a reported
>compatibility issue if we use the +localidentifer part of a version number
>with older versions of setuptools. I need to do some testing on this. If I
>don't hit any issues, I'm very happy to put an identifier on there. I would
>like to keep it at 1.6.11.x, rather than 1.6.12+, so that it's very clear
>we're based on upstream Django 1.6.11.
otherwise 1.6.11.x
>I want HTTPS on this pretty badly, though, so we might try that
>approach when we can get to it.
>To attempt to compensate for this, though, we sign all the builds and
>include checksums in both the links and in dedicated .sha256sum files. Not
>the best solution, but it's something at least :/
(e.g. pgp) signatures will help. But then make sure the key ID is
published on an HTTPS page ;)
/Markus
>You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
>To post to this group, send email to django-d...@googlegroups.com.
>Visit this group at http://groups.google.com/group/django-developers.
>To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHfcSbz%3DN9%2BFduAcSODtTVLEVmbD42Nu_CsiJHAQiXxtU70kCg%40mail.gmail.com.
>Visit this group at http://groups.google.com/group/django-developers.
--
Christian Hammond
Aug 24, 2015, 5:53:15 PM8/24/15
to django-d...@googlegroups.com
On Wed, Aug 19, 2015 at 4:43 PM, Markus Holtermann <in...@markusholtermann.eu> wrote:
Hey Christian,
On Wed, Aug 19, 2015 at 02:41:49PM -0700, Christian Hammond wrote:
Regarding the version number, from what I've read, there's still a reported
compatibility issue if we use the +localidentifer part of a version number
with older versions of setuptools. I need to do some testing on this. If I
don't hit any issues, I'm very happy to put an identifier on there. I would
like to keep it at 1.6.11.x, rather than 1.6.12+, so that it's very clear
we're based on upstream Django 1.6.11.
Thanks, that sounds good to me. 1.6.11.x+unofficial if possible,
otherwise 1.6.11.x
Looks like it's going to have to be 1.6.11.x.
However, I've hit a pretty major stumbling block here.
pip's removed support for dependency_links, requiring that all package dependencies be made available on PyPI. This means there's no way we can host a build of this ourselves, have a project depend on it, and build the project as a package. It simply will ignore our builds and fail with a dependency error (assuming a version range of >=1.6.11.1,<1.7). (If installing from source, it looks like requirements.txt can still specify links, but that doesn't help when building/distributing wheels/eggs.)
I tried a hacky workaround of creating a sdist package that invoked setuptools to install using dependency_links, and depended on that package instead. Older versions of pip prior to 7.0 would execute our setup.py and in turn install our build of Django. However, since pip now converts sdists to wheels prior to installing, it just skips over dependency_links, resulting in the original problem.
The only other option I can think of is to actually upload the 1.6.11.x build to PyPI under a different name ("beanbag-django16", for instance), but keep the top-level module as "django". This will inevitably cause conflicts with the officially installed Django package, though. Not a good option.
So at this point, it seems that while we can maintain security backports of Django 1.6.11, we can't realistically depend on them or trigger upgrading to them. Short of the official Django package on PyPI either hosting our backports or registering URLs to them, there might not be a clean way to do this. I'm pretty sure is not the route you guys want to go for this, though.
Any ideas on alternative ways to tackle this? I'm officially stuck.
Christian
Carl Meyer
Aug 24, 2015, 6:37:49 PM8/24/15
to django-d...@googlegroups.com
Hi Christian,
On 08/24/2015 03:52 PM, Christian Hammond wrote:
> pip's removed support for dependency_links, requiring that all package
> dependencies be made available on PyPI. This means there's no way we can
> host a build of this ourselves, have a project depend on it, and build
> the project as a package. It simply will ignore our builds and fail with
> a dependency error (assuming a version range of >=1.6.11.1,<1.7). (If
> installing from source, it looks like requirements.txt can still specify
> links, but that doesn't help when building/distributing wheels/eggs.)
Yes, I'm afraid you're trying to solve an additional problem I didn't
realize you had, because I didn't realize ReviewBoard was distributed as
wheels/eggs.
In general, my feeling has been that Python packaging is not well suited
to distributing deployable web apps, and they are better distributed as
source (with requirements.txt) plus installation instructions. I think
pip's developers feel similarly, and are aiming for a clear distinction
between setup.py "abstract" dependencies and requirements.txt "concrete"
ones. (See e.g. https://caremad.io/2013/07/setup-vs-requirement/,
authored by Donald Stufft, the primary pip maintainer.)
So the official line is that if you need to get _that_ specific about
dependency versions (and especially if you're needing to source those
versions off-PyPI), you should be distributing a requirements.txt file,
not a wheel or an egg.
Obviously there's nothing stopping you from doing that -- just have your
ReviewBoard setup.py depend on ``Django>=1.6,<1.7``, and then strongly
recommend to your clients that they have a ``requirements.txt`` file
with a ``--find-links`` line referencing your Django 1.6 unofficial
support downloads page in order to get the latest security-patched
versions. But that doesn't allow you to automatically enforce that all
the consumers of your wheels/eggs will actually do this.
[snip non-functional workaround]
> So at this point, it seems that while we can maintain security backports
> of Django 1.6.11, we can't realistically depend on them or trigger
> upgrading to them. Short of the official Django package on PyPI either
> hosting our backports or registering URLs to them, there might not be a
> clean way to do this. I'm pretty sure is not the route you guys want to
> go for this, though.
I don't really think that's an option. At that point they'd have to be
officially supported, because there's no longer any opt-in required to
get them.
> Any ideas on alternative ways to tackle this? I'm officially stuck.
I'm afraid I don't have any solution to offer, other than embracing the
"abstract vs concrete" dependencies distinction, and accepting the fact
that users of your wheels/eggs are choosing to be more "on their own"
when it comes to their chosen dependency versions.
Carl
On 08/24/2015 03:52 PM, Christian Hammond wrote:
> pip's removed support for dependency_links, requiring that all package
> dependencies be made available on PyPI. This means there's no way we can
> host a build of this ourselves, have a project depend on it, and build
> the project as a package. It simply will ignore our builds and fail with
> a dependency error (assuming a version range of >=1.6.11.1,<1.7). (If
> installing from source, it looks like requirements.txt can still specify
> links, but that doesn't help when building/distributing wheels/eggs.)
realize you had, because I didn't realize ReviewBoard was distributed as
wheels/eggs.
In general, my feeling has been that Python packaging is not well suited
to distributing deployable web apps, and they are better distributed as
source (with requirements.txt) plus installation instructions. I think
pip's developers feel similarly, and are aiming for a clear distinction
between setup.py "abstract" dependencies and requirements.txt "concrete"
ones. (See e.g. https://caremad.io/2013/07/setup-vs-requirement/,
authored by Donald Stufft, the primary pip maintainer.)
So the official line is that if you need to get _that_ specific about
dependency versions (and especially if you're needing to source those
versions off-PyPI), you should be distributing a requirements.txt file,
not a wheel or an egg.
Obviously there's nothing stopping you from doing that -- just have your
ReviewBoard setup.py depend on ``Django>=1.6,<1.7``, and then strongly
recommend to your clients that they have a ``requirements.txt`` file
with a ``--find-links`` line referencing your Django 1.6 unofficial
support downloads page in order to get the latest security-patched
versions. But that doesn't allow you to automatically enforce that all
the consumers of your wheels/eggs will actually do this.
[snip non-functional workaround]
> The only other option I can think of is to actually upload the 1.6.11.x
> build to PyPI under a different name ("beanbag-django16", for instance),
> but keep the top-level module as "django". This will inevitably cause
> conflicts with the officially installed Django package, though. Not a
> good option.
Nope, not great.
> build to PyPI under a different name ("beanbag-django16", for instance),
> but keep the top-level module as "django". This will inevitably cause
> conflicts with the officially installed Django package, though. Not a
> good option.
> So at this point, it seems that while we can maintain security backports
> of Django 1.6.11, we can't realistically depend on them or trigger
> upgrading to them. Short of the official Django package on PyPI either
> hosting our backports or registering URLs to them, there might not be a
> clean way to do this. I'm pretty sure is not the route you guys want to
> go for this, though.
officially supported, because there's no longer any opt-in required to
get them.
> Any ideas on alternative ways to tackle this? I'm officially stuck.
"abstract vs concrete" dependencies distinction, and accepting the fact
that users of your wheels/eggs are choosing to be more "on their own"
when it comes to their chosen dependency versions.
Carl
Ryan Hiebert
Aug 25, 2015, 12:32:44 AM8/25/15
to django-d...@googlegroups.com
> On Aug 24, 2015, at 5:37 PM, Carl Meyer <ca...@oddbird.net> wrote:
>
>> Any ideas on alternative ways to tackle this? I'm officially stuck.
>
> I'm afraid I don't have any solution to offer, other than embracing the
> "abstract vs concrete" dependencies distinction, and accepting the fact
> that users of your wheels/eggs are choosing to be more "on their own"
> when it comes to their chosen dependency versions.
Christian Hammond
Aug 25, 2015, 12:52:50 AM8/25/15
to django-d...@googlegroups.com
That sort of setup works fine when it's a standalone install on a server that someone experienced with the Python world is maintaining, or a developer working against a set of modules. It's not really an option for a full-blown application meant to be easily installed by people used to being able to run 'apt-get install <app>' or 'yum install <app>'
As a more complex webapp, our packaging requires the compilation/minification of .less/.js files, which means any user using the sdist would need a suitable build environment. That raises a huge barrier to installation for a significant chunk of our userbase. Our goal is to keep installation simple for everyone, treating it much like any typical application, without requiring a whole bunch of pre-setup. Aside from this particular issue with the backported Django builds, we've been able to do this for years.
It's basically the difference between being able to download/install a setup.exe for OpenOffice as opposed to needing a build environment and source to install it.
It's starting to sound like the best bet for us is to move away from pip/easy_install for end-user consumption of the software and onto more of a dedicated installer that can manage dependencies more explicitly, maybe taking care of the building/consumption of a requirements.txt for them.
Thanks for your thoughts on this, everyone :)
Christian
--
Christian Hammond - chi...@chipx86.com
Review Board - https://www.reviewboard.org
Christian Hammond - chi...@chipx86.com
Review Board - https://www.reviewboard.org
Beanbag, Inc. - https://www.beanbaginc.com
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/69fOquu8v-U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/486DA079-7F97-4597-A7FB-CD179DFB54DC%40ryanhiebert.com.
Reply all
Reply to author
Forward
0 new messages