Please don't kill the ecosystem

[Pkl]

Hello,

I once was once lured to an ideal of long-term stability and retrocompatibility, by nice docs like this one :
https://docs.djangoproject.com/en/dev/misc/api-stability/

But for some years, stuffs have actually been getting worse and worse, with each django release bringing its little crowd of nightmares.

In random order, I stumbled upon:
- removal of django.conf.urls.defaults
- removal of markup contrib lib (adios built-in RST support)
- removal of request.raw_post_data (thus breaking about all existing webservice libs)
- lots of changes in mandatory settings, like allowed_hosts or DB conf format,
- future removal of the very basic "mimetype" argument from django objects (how many apps impacted ? quite MANY I guess)
- future removal of request.REQUEST (aka "we're not all consenting adults, let's remove all tools that people could misuse")

For example, I had great fun installing django + djangocms + zinnia + cms_plugin_zinnia ; the probability of having no version mismatch between all these components being more or less zero.

And the future looks grimmer again, having a look at https://docs.djangoproject.com/en/dev/internals/deprecation/.

I don't get this relentlessness at breaking, for the sake of purity, all stuffs that are not actively maintained, yet could be perfectly working.

What is easier, in a deeply dynamic language like python, than keeping retrocompatibility ? Why not keep "raw_post_data" as an undocumented alias for "body" ? Why not keep "mimetype" as a silent alias for "content_type" ? What harm did the "markup" support in django, since no one expects such extensions to be 100% secure anyway (no software on earth is) ? If code "purity" is wanted, then why not have all these monkey-patched from a builtin, but removable, "django.compatibility" util ? Why not wait for 2.0, to remove all compatibility aliases in one single shot ?

The one killing features of django is its app ecosystem - being able to plug blogs, authentication systems, webservice generators, and all other kinds of applications, into this common framework, and benefit from the common auto-admin. And since all these apps don't evolve at the same rythm, retrocompatibility is IMHO a NECESSITY for django, not a LUXURY. Or isn't it ?

Please don't kill the ecosystem, please respect the promises of api-stability that were once made.....

regards,
Pkl

[Collin Anderson]

I agree that there have been a lot of backwards-incompatible changes over the last few years clean up the API. We've really been in "get stuff done" mode.

I do however think it's a not good idea to break all compatibility in one single shot. Python 3 did this and 6 years later it's still preventing many of us from even starting to use it. Spreading out the backwards incompatibilities over many releases is a very good idea.

[Tim Graham]

Pkl, if you're interested, I'd encourage you to be more involved in the development process including this mailing list when changes are proposed. I'm afraid raising a whole bunch of old issues that have already been discussed and decided isn't that helpful.

Our deprecation policy (as you probably know) is to maintain backwards compatibility (where at all possible) for two releases. Keeping these shims indefinitely, however, would add a lot of complexity and hinder our ability to evolve the framework. I don't believe we've diverged from our API stability policy regarding the deprecation process.

As you may know, we have added an experimental "Long Term Support" release (currently 1.4) for projects that don't wish to update to the newest versions of Django as quickly as we put them out.

We've also decided that "2.0" will simply be the release following 1.9. It won't be a big backwards incompatible release with sweeping changes.

If you have other suggestions to address your concerns let us know. Out of respect for the many volunteers that contribute their time to Django, please consider toning down the grim tone. :-)

Tim

[Curtis Maloney]

Mostly for clarity and historical purpose, I'd like to expand on the points you listed.

Do not take this in any way as a dismissal of the issues you've raised.

On 2 September 2014 06:45, Pkl <chambon...@gmail.com> wrote:

Hello,

I once was once lured to an ideal of long-term stability and retrocompatibility, by nice docs like this one :
https://docs.djangoproject.com/en/dev/misc/api-stability/

But for some years, stuffs have actually been getting worse and worse, with each django release bringing its little crowd of nightmares.

In random order, I stumbled upon:
- removal of django.conf.urls.defaults

This was moved into django.conf.urls ...
 

- removal of markup contrib lib (adios built-in RST support)

From memory, this was due to a security issue with the Markdown library used.  I, too , was hit by this, and ended up vendoring the tag library into my project.
 

- removal of request.raw_post_data (thus breaking about all existing webservice libs)

It was renamed to request.body, not removed.  This is because it represents the request body, no matter the HTTP method used.
 

- lots of changes in mandatory settings, like allowed_hosts or DB conf format,

ALOWED_HOSTS was added as a security measure.

The only significant DB config format I can recall was adding multi-DB support.  Was there another?

- future removal of the very basic "mimetype" argument from django objects (how many apps impacted ? quite MANY I guess)

IIRC this was renamed to content-type, since "mime type" is, in fact, for email.
 

- future removal of request.REQUEST (aka "we're not all consenting adults, let's remove all tools that people could misuse")

I remember the discussion for this.  Primarily it was because it did cause security issues, as well as being unclear as to which was looked up first- GET or POST.

As someone who has been using Django since 0.91... I've seen  a lot of change over the years, some more drastic than others.

I don't get this relentlessness at breaking, for the sake of purity, all stuffs that are not actively maintained, yet could be perfectly working.

 

Whilst it could be argued that, for instance, request.body and requset.REQUEST were removed for "purity", it can also be said they were removed for clarity (to avoid confusion) and safety (to avoid real security issues).

As for breaking unmaintained 3rd party apps... sorry, that's how the world works.  If you want that project to keep up to date with Django, either incentivise the devlopers to do so, or contribute to them yourself.  Such is open source.

What is easier, in a deeply dynamic language like python, than keeping retrocompatibility ?

I do like this idea... it should be possible to monkey-path in at least some of these ideas.

Open a ticket, CC me, and I'll happily work with you on a PR.

If nothing else, we can make it a 3rd party app.
 

Why not keep "raw_post_data" as an undocumented alias for "body" ? Why not keep "mimetype" as a silent alias for "content_type" ?

mimetype _was_ an alias for content_type for, IIRC, much longer than the usual 2-release deprecation cycle.

 

What harm did the "markup" support in django, since no one expects such extensions to be 100% secure anyway (no software on earth is) ? If code "purity" is wanted, then why not have all these monkey-patched from a builtin, but removable, "django.compatibility" util ? Why not wait for 2.0, to remove all compatibility aliases in one single shot ?

The current trend with contrib apps is to remove them if they'd benefit from a more frequent release cycle - one not tied to that of core.  Howerver, I was a little surprised to see django.contrib.markup wasn't moved to a django-maintained 3rd party app like some others were.  Perhaps open a ticket for that?
 

The one killing features of django is its app ecosystem - being able to plug blogs, authentication systems, webservice generators, and all other kinds of applications, into this common framework, and benefit from the common auto-admin. And since all these apps don't evolve at the same rythm, retrocompatibility is IMHO a NECESSITY for django, not a LUXURY. Or isn't it ?

 

Please don't kill the ecosystem, please respect the promises of api-stability that were once made.....

These promises have been kept.  There is a LOT of code that could be dropped or rewritten immediately if it weren't for the 2-relese deprecation cycle.

One of my PRs was rejected specifically because I hadn't implemented a deprecation path.  It would likely speed up template rendering, which I think everyone agrees is desirable.

--

Curtis

[Marc Tamlyn]

Apologies if this does not directly address your points, but I recently took over a Django site still running 1.1. I brought it fully up to 1.6 in under 2 hours work, including rearranging the project structure, updating settings, URL tags, import paths, switching to pip from buildout etc. Considering the version of Django this was written for is five years old I consider this an impressive achievement in backwards compatibility.

On another note, given Django's slow release cycle, deprecated features are visible if you run with warnings for between 18 months and 2 years before they stop working. I don't think it is unreasonable to expect a small amount of maintenance within these time frames.

As the Django ecosystem is aging, it is sadly inevitable that some unmaintained projects will stop working. The only way Django could avoid this is complete stagnation which would kill Django much more than the small maintenance burden it imposes.

Django remains one of the most stable web frameworks out there, but it must also continue to evolve or risk being IE6.

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAG_XiSDBkbqrk0XFhdTFjMGsZmggE4RP31WjXQBew5wadm%2BOUg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Florian Apolloner]

Hi Pkl,

On Monday, September 1, 2014 10:45:30 PM UTC+2, Pkl wrote:

In random order, I stumbled upon:
- removal of django.conf.urls.defaults
- removal of markup contrib lib (adios built-in RST support)
- removal of request.raw_post_data (thus breaking about all existing webservice libs)
- lots of changes in mandatory settings, like allowed_hosts or DB conf format,
- future removal of the very basic "mimetype" argument from django objects (how many apps impacted ? quite MANY I guess)
- future removal of request.REQUEST (aka "we're not all consenting adults, let's remove all tools that people could misuse")

At least three of those are security related, if you read the api stability docs you will see that we will not consider api stability if this affects a security issue. If that creates an issue for you, then by all means Django is not for you. As for the other changes, I'd say they are debatable, but caused serious confusion for users (I am helping out on IRC quite a bit, so I was really glad to see them changed). At the speed we release Django, this gives you roughly two years to change your code, I think this is reasonable enough to ask for.

For example, I had great fun installing django + djangocms + zinnia + cms_plugin_zinnia ; the probability of having no version mismatch between all these components being more or less zero.

Oh, how so? They say to support Django 1.6 -- I doubt any of the above stated issues caused breakage there.
 

I don't get this relentlessness at breaking, for the sake of purity, all stuffs that are not actively maintained, yet could be perfectly working.

If we would break for purity, we'd be breaking a lot more…
 

What is easier, in a deeply dynamic language like python, than keeping retrocompatibility ? Why not keep "raw_post_data" as an undocumented alias for "body" ? Why not keep "mimetype" as a silent alias for "content_type" ?

For what? By the time we remove the feature from Django all apps should work with new Django versions. The older Django versions we are dropping support for by not keeping raw_post_data are no longer maintained and have __known__ and sometimes __easily__ exploitable security issues.

What harm did the "markup" support in django, since no one expects such extensions to be 100% secure anyway (no software on earth is) ?

No software on earth is 100% secure, but supporting something which has quite a few security implications and having to release security releases just for this component quite often isn't going to help. In the end users expect us to deliver secure code for known security issues in the third party libraries, this is not something we want to do.

 

If code "purity" is wanted, then why not have all these monkey-patched from a builtin, but removable, "django.compatibility" util ? Why not wait for 2.0, to remove all compatibility aliases in one single shot ?

You feel up to this? This should work perfectly fine as 3rd party app, just don't expect us to maintain compatibility for releases which aren't even under security maintenance anymore.

The one killing features of django is its app ecosystem - being able to plug blogs, authentication systems, webservice generators, and all other kinds of applications, into this common framework, and benefit from the common auto-admin. And since all these apps don't evolve at the same rythm, retrocompatibility is IMHO a NECESSITY for django, not a LUXURY. Or isn't it ?

Dunno, I usually never had problems. That said I only use relatively maintained apps, cause I want to be able to get bugs fixed "soonish".

Cheers,
Florian

[Aymeric Augustin]

Hello,

While I generally agree with what other core devs have said regarding the examples you raised, I still find your feedback interesting because it reminds us that we iterate too quickly for at least some parts of the community.

It's unavoidable that some people will find the pace of Django's evolution too fast and others will find it too slow. We're trying to ease the pain of the former group by documenting thoroughly the changes in each release.

I'm not sure I understand completely where your frustration comes from. The tone of your email seems disproportionate with the effort needed to replace all instances of "django.conf.urls.defaults" with "django.conf.urls". That takes about ten minutes, if you count the time to locate the information in the release notes and make the commit. Is it a case of death by a thousand cuts? Perhaps you could explain how you proceed when you need to upgrade Django in your projects? Are the release notes somehow lacking?

Overall, I don't think we're going to change our pace — especially when it comes to security-related issues — but we can try to improve our communication around deprecations and removals.

-- 

Aymeric.

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/6abfd1ee-0056-41c1-b079-f89f2f463cf5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Aymeric.

[Yo-Yo Ma]

With as many new frameworks as there are out there, with the gains in popularity seen in Go and Node, my thinking is, move quickly, break (some very small) things, or die slowly. As was said, if your favorite lib doesn't work with 1.6 or 1.7, either use a prior version, or spend some time contributing the 10 minutes to 2 hours that most of these changes would take at most. You'll feel good about it. You'll feel less frustrated, and you'll understand, thereafter, what it means to be an active part of the community. I too used to be a rail bird. It gets you "exactly nowhere", in the words of Russell K.M., and it makes those who contribute feel unappreciated.

[Tom Evans]

On Mon, Sep 1, 2014 at 9:45 PM, Pkl <chambon...@gmail.com> wrote:
>
> Hello,
>
> I once was once lured to an ideal of long-term stability and
> retrocompatibility, by nice docs like this one :
> https://docs.djangoproject.com/en/dev/misc/api-stability/
>
> But for some years, stuffs have actually been getting worse and worse, with
> each django release bringing its little crowd of nightmares.
>

Our solution is to not chase releases. Django 1.4 is marked as "Long
Term Stable", all our projects use Django 1.4 and no more; if/when a
newer stable release is proposed we will consider moving our projects
to that. We do then have some problems with 3rd party apps that do not
support 1.4..

There was a thread a few weeks ago that discussed (perhaps
tangentially) what the next LTS release branch will be, where it was
brought up that whatever release it is, the API will actually have to
be stable(!), probably ruling out 1.7. So, LTS will probably be 1.4
for some time yet.

On Tue, Sep 2, 2014 at 1:48 PM, Aymeric Augustin
<aymeric....@polytechnique.org> wrote:
> I'm not sure I understand completely where your frustration comes from. The
> tone of your email seems disproportionate with the effort needed to replace
> all instances of "django.conf.urls.defaults" with "django.conf.urls". That
> takes about ten minutes, if you count the time to locate the information in
> the release notes and make the commit.

It's not really, though is it? You encounter the problem because
you've upgraded django version; this will not be the only thing that
is broken, and so fixing this one thing will not enable you to run
your test suites.. it can take several iterations before you know that
it is fixed.

If you track django development and have used django for several
years, then these changes are relatively easy. If you are new to using
django, it is not so obvious and easy; as an example, we had an old
site running on django 1.3 that we wanted to move to 1.4 so that we
are using the same version of django throughout; this story was scored
at 8 points, it took a junior developer much longer than 8 points and
wasn't finished in a single sprint - and 1.3->1.4 was *easy*! We
prefer to spend our points on our own features (and senior developers
are expensive :)


Cheers

Tom

[Aymeric Augustin]

2014-09-02 15:33 GMT+02:00 Tom Evans <teva...@googlemail.com>:

this story was scored
at 8 points, it took a junior developer much longer than 8 points and
wasn't finished in a single sprint - and 1.3->1.4 was *easy*

I don't know how much a point or a sprint is worth in this context :-/

I know that upgrading is very painful if you do it without following a strict

process. Shotgun debugging is awful in this context. Here's the process I

would recommend.

1. Upgrade your dependencies. If possible (depending on how many you have)

check that they support the target Django version. Otherwise, you'll notice at

step 3.

2. Read through the release notes. At each bullet point, do a quick check for

affected code with "Find all in project" and fix it. It doesn't matter if you

miss something at this stage. Provided you read every item, you'll make the

connection if you hit the problem. This is tedious, but if you do it another

way, you'll waste more time.

3. Run your test suite under -Wall or even -Werror until it's clean. Fix stuff

you've missed until it passes.

Don't forget that, if your project didn't run cleanly under the previous

Django version, you need to start from the previous release notes and clear

deprecation warnings from your own code.

Did your junior developer follow a similar process? If he did, what took them

so long?

What I want to understand is how much of an advantage I have when it comes to

upgrading Django. What do I know that isn't written down and makes it so hard

for others?

--
Aymeric.

[shmengie]

Pkl,

You shouldn't have to upgrade all your old sites to the latest version of Django, unless you want maintain current methods.

This is why we like virtual environments, much.

The core team has made a nice set of security releases available via pip. 
Django 1.4.14, Django 1.5.9, Django 1.6.6, and Django 1.7

# See package version in your environment
$\> pip freeze

# This command should get fetch security updates w/out breaking eco-system.
$\> pip install Django --upgrade

# Worst case scenario you'll have to un-install current version and Install a specific version.
$\> pip uninstall Django
$\> pip install Django==1.4.14 

More work to upgrade a bunch of virtuaenv's, but not as much as keeping all sites current, don't break your eco-system, unless there's a reason.

It's wise to choose a balance best practices that suit your eco-system.

I'd thank the team for issuing security releases for old versions.

[Tom Evans]

On Tue, Sep 2, 2014 at 2:53 PM, Aymeric Augustin
<aymeric....@polytechnique.org> wrote:
> 2014-09-02 15:33 GMT+02:00 Tom Evans <teva...@googlemail.com>:
>
>> this story was scored
>> at 8 points, it took a junior developer much longer than 8 points and
>> wasn't finished in a single sprint - and 1.3->1.4 was *easy*
>
>
> I don't know how much a point or a sprint is worth in this context :-/

Sorry, context is important! "A sprint" ~ 9 work days, points vary
wildly from sprint to sprint unfortunately - 12 is about a sprints
worth of work.

> <snip>

Good steps.

>
> Did your junior developer follow a similar process? If he did, what took
> them
> so long?
>
> What I want to understand is how much of an advantage I have when it comes
> to
> upgrading Django. What do I know that isn't written down and makes it so
> hard
> for others?

I think it was more distraction by topics he had not come across. We
set him off by saying "look at the release notes, go thru each change
in turn, see if we are affected and what we need to fix it".

The problem then was that he was new. He hadn't had to deal too much
with TZ support, so adding TZ support to our project meant a day of
learning about how TZ work in UNIX.

Similarly, changing the project layout is a good thing, but then he
spent a day learning about package layouts - not a bad thing by any
means, but in the 1.3 -> 1.4 release there are 75 similarly complex
enhancements/deprecations/incompatibilities - it just takes some time
to go through if you have not come across that topic before, eg,
clickjacking.

Cheers

Tom

[Aymeric Augustin]

2014-09-03 10:40 GMT+02:00 Tom Evans <teva...@googlemail.com>:

I think it was more distraction by topics he had not come across. We
set him off by saying "look at the release notes, go thru each change
in turn, see if we are affected and what we need to fix it".

Thanks Tom for the details. I understand better. Upgrading Django exposes you

to components or features you didn't even know existed. The time I spent

triaging Trac tickets gives me a massive advantage there :-)

The problem then was that he was new. He hadn't had to deal too much
with TZ support, so adding TZ support to our project meant a day of
learning about how TZ work in UNIX.

You went for USE_TZ = True. That required a lost of work to adjust your entire

codebase. I'm considering that a new feature for your project. If you wanted

to upgrade as fast as possible, you could have kept the legacy behavior with

USE_TZ = False. That was the default for projects upgraded from 1.3.

Similarly, changing the project layout is a good thing, but then he
spent a day learning about package layouts - not a bad thing by any
means, but in the 1.3 -> 1.4 release there are 75 similarly complex
enhancements/deprecations/incompatibilities - it just takes some time
to go through if you have not come across that topic before, eg,
clickjacking.

Right, this was another complicated change you couldn't skip.

1.5 and 1.6 must have been easier than 1.4. 1.7 may be more difficult because

of migrations and app-loading. If anyone has improvements to suggest for the

documentation, please do!

--
Aymeric.

[Anssi Kääriäinen]

On Wed, 2014-09-03 at 14:46 +0200, Aymeric Augustin wrote:


> 1.5 and 1.6 must have been easier than 1.4. 1.7 may be more difficult
> because
> of migrations and app-loading. If anyone has improvements to suggest
> for the
> documentation, please do!

We could perhaps have better documentation of how to handle deprecations
in 3rd party libraries. Namely, how to remove usage of deprecated
features in a way that works silently both before and after the addition
of the deprecation.

For example, lets consider the get_query_set() -> get_queryset() naming
change done in 1.6. If 3rd party library writers change the method name
to get_queryset() for 1.6, then their code won't work in 1.5. If they
don't do the change, then all users trying to run with -Wall will get a
bunch of deprecation warnings they can't do much about. This will
effectively ruin the "run with -Wall" advice.

The usual way to solve these kinds of problems in code that needs to
work in multiple Django versions is to write code which is conditional
on Django's version. It looks a bit ugly to define methods
conditionally, but in practice this works extremely well.

So, when adding a new deprecation to Django, we should document how 3rd
party library writers can write code that works *silently* both before
and after the feature deprecation is added. If there is no easy way to
achieve that, then we should reconsider the deprecation.

- Anssi

[Pkl]

Hello,

Hello,

My apologies for sounding rude, and for raising these issues so late.

Yes, upgrading a project itself is a matter of a pair of evenings, the problem is with third-party libraries, and more again, plugins of third-party libraries (especially CMS-related).
I've had to bundle a dozen of these with my app, because they didn't quite follow the pace of Django evolutions. Now of course, as you say, I might as well stabilize for some time on a LTS version, but as was said too, the next steps would also be much harder then. Or I could forget about modular apps, and go towards monolitic third-party apps instead...

I felt that crucial variables like "request.REQUEST" or "raw_post_data" could stay aliased for much more than 2 minor versions (but undocumented and with warnings, of course), for teh sake of retrocompatibility.
However if it's acknowledged, in the core team, that code maintainability and security would be hindered by keeping these artefacts, I can't argue that.

Concerning the "rules of open source", I've yet to find a satisfying way to apply them regarding these "micro breaks". Imagine that project "myvideoplugin" is unmaintained (not handling PR) : unable to patch the original Repo, I'd have to fork it ; and unable to push results to the original pypi name, I'd have to create a separate myvideoplugin-pkl package on Pypi... that's quite some hassle for (often) a 10-lines patch.

Thanks Curtis for the proposal, I was thinking about a django-retrocompat package indeed, however I realize that doing a decent job of retrocompatibility would be a much harder job than initially expected, since breaking changes are rarely a matter of simple aliases (https://docs.djangoproject.com/en/1.7/releases/1.7/#backwards-incompatible-changes-in-1-7) ; django-retrocompat wouldn't stand the expectations of users. But a minimal package dealing only with "renamed fields" would be doable, I'll have a look at it.

regards,
Pkl

[Collin Anderson]

Considering we're trying to come out with more frequent releases, I wonder if it would sometimes make sense to sometimes have a 3-release deprecation for some features.

[Robert Grant]

As a general point, and this is coming from an "enterprise" background with software that costs horrible amounts to purchase and maintain every year, I'm astonished at how good the quality of the Django migration notes are, and how many little details are there with "oh and if you do this then the combination won't work, here's what you do now". Combine that with virtualenv and security releases for older versions and I don't see how on earth the devs could do better on that front!

You'd be paying some IBM consultants for 3 months to get that level of expertise upgrading some of their products. Or 6 months for SAP.

[Luke Plant]

On 03/09/14 14:21, Anssi Kääriäinen wrote:

> For example, lets consider the get_query_set() -> get_queryset() naming
> change done in 1.6. If 3rd party library writers change the method name
> to get_queryset() for 1.6, then their code won't work in 1.5. If they
> don't do the change, then all users trying to run with -Wall will get a
> bunch of deprecation warnings they can't do much about. This will
> effectively ruin the "run with -Wall" advice.
>
> The usual way to solve these kinds of problems in code that needs to
> work in multiple Django versions is to write code which is conditional
> on Django's version. It looks a bit ugly to define methods
> conditionally, but in practice this works extremely well.
>
> So, when adding a new deprecation to Django, we should document how 3rd
> party library writers can write code that works *silently* both before
> and after the feature deprecation is added. If there is no easy way to
> achieve that, then we should reconsider the deprecation.

Regarding get_queryset, it's possible to do it without warnings I think,
and the RenameMethodsBase class helps, but it's not entirely
straightforward.

I've added a paragraph to the docs that documents how to do it correctly
for the get_queryset renamings, and I agree that we should make this
kind of thing a standard practice. It might bloat the release notes a
bit, but I think that in general people appreciate the detail.

Luke


--
"Cross country skiing is great if you live in a small country."
(Steven Wright)

Luke Plant || http://lukeplant.me.uk/

[Luke Plant]

Hi Pkl,

Sorry for slow reply, but I thought this was worth it:

On 03/09/14 20:45, Pkl wrote:

> Concerning the "rules of open source", I've yet to find a satisfying way
> to apply them regarding these "micro breaks". Imagine that project
> "myvideoplugin" is unmaintained (not handling PR) : unable to patch the
> original Repo, I'd have to fork it ; and unable to push results to the
> original pypi name, I'd have to create a separate myvideoplugin-pkl
> package on Pypi... that's quite some hassle for (often) a 10-lines patch.

There is no need to create a new pypi package and release - you can
install straight from github like this:

pip install git+git://github.com/foo/bar.git@abcde12345

You can put this straight into your requirements.txt as well. I have
dozens of instances of this in my requirements.txt files.

The trick then is to get your changes merged upstream. That is not
always very easy - you have to be willing to genuinely contribute
upstream i.e. contributions that make the maintainer's life easier, not
harder. That means writing unit tests, or bootstrapping use of tox so
that the test suite can be easily run against multiple versions. It
might even mean starting a test suite because upstream didn't have one -
in which case you also should reassess the process by which you choose
dependencies...

However, I've generally found that with effort and *genuinely helpful*
upstream contributions, my PRs are accepted, and I can go back to using
a PyPI release after some time. When I can't, it's because the upstream
dependency wasn't solid, and it kind of serves me right for not being
more careful and adding something to my project that wasn't reliable. In
some cases I might have to take over being maintainer of the project -
which is a burden, but part of the price you pay for being a part of the
open source world and getting tons of amazing quality software for free.

I do agree that we should be more careful about finding and documenting
ways of doing deprecations so that libraries can support both old and
new versions without generating deprecation warnings. I've just added
one fix to 1.6 release notes on this front.

Regards,