Testing pre-release Django

180 views
Skip to first unread message

Claude Paroz

unread,
May 20, 2016, 9:32:53 AM5/20/16
to Django developers (Contributions to Django itself)
Hi,

I have the general feeling that too few people are testing the new Django major releases before the .0 release. The result being that many regressions are often reported after the release, while those could have been detected at alpha/beta/rc stages.

I found myself in the situation where I really wanted to test my own projects with a fresh new Django, but couldn't because some dependency was not updated and was crashing the project.

I wonder if it would help taking the most popular third-party dependencies and help those projects testing with pre-release Django. As an example, I added a Wiki page, https://code.djangoproject.com/wiki/Version1.10ThirdPartySupport (to be completed) as a mean to track Django 1.10 support progress in most popular apps. The first step would be to at least add Django 1.10 (or even master) as allowed_failures in Travis setups, so as 1.10 support is plainly visible.

Thoughts?

Claude

Tim Graham

unread,
May 20, 2016, 1:50:06 PM5/20/16
to Django developers (Contributions to Django itself)
I'm completely supportive of this effort. In past release cycles, I've done testing with djangoproject.com and sent some PRs to its dependencies. The blocker to upgrading is always waiting for each project to make a release with the fixes.

We could provide some guidance and suggest that projects should work on adding support for the next release of Django during the alpha period and try to make a next-Django-version compatible release by the beta release date, but of course we can't control how people spend their time.

I hoped that enforcing a feature freeze at alpha rather than beta (starting with the 1.8 release cycle) would make packages more eager to do prerelease testing since there should be less changes between then and the final release. I haven't seen evidence that this has helped though.

I'd be curious to hear from any third-party package maintainers how they feel about it.

Ed Morley

unread,
May 20, 2016, 7:31:23 PM5/20/16
to Django developers (Contributions to Django itself)
Another idea might be to encourage more packages to test on Travis against Django master (with that sub-job marked as allowed to fail, so it doesn't fail the whole run) - so any incompatibilities become apparent earlier.

eg:
https://github.com/evansd/whitenoise/commit/c1a9f04cc90a7e48e536c651d9624660cee44073

James Pic

unread,
May 21, 2016, 12:25:05 AM5/21/16
to django-d...@googlegroups.com

Please test your projects against django master too.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/7129d115-482e-4564-bce4-45322da2a0f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hugo Osvaldo Barrera

unread,
May 21, 2016, 4:05:39 AM5/21/16
to django-d...@googlegroups.com
When contributing to apps (dependencies), I've noticed developers tend to expect a stable django release before merging in support and making a new stable release, rather than doing so before the new django release.
This happened when I send patched to a few project with dj1.9 support.
 
If the django project provided some clear guidelines in the lines of "we recommend that you make your releases during the feature freeze period", with the explanations given in this thread,  it's be easier to convince those app developers to actually accept those patches and create those releases earlier, and start pushing the ecosystem in that direction.
 
The same applies to testing vs master.
 
Compatible [dependency] apps mean it's easier to test earlier too.
 
Just me two cents,
 
Cheers,
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
 
--
Hugo Osvaldo Barrera
 

Tom Christie

unread,
May 25, 2016, 6:37:35 AM5/25/16
to Django developers (Contributions to Django itself)
> I'd be curious to hear from any third-party package maintainers how they feel about it.

I've not found testing against master to be particularly helpful, as it's not apparent if an issue is substantial or not, without putting in work.

The alpha releases *are* tremendously helpful for having a pre-release to start working against,
although I don't know what a good process is for feeding back between third party packages that are updating, and the Django team.

For what it's worth, I'm seeing a surprising number of test failures for REST framework against 1.10a, although I suspect that may will be simple enough to resolve.

Another thing that might help is some guidance on ensuring that travis builds are properly displaying pending deprecation warnings - that'd go some way towards helping package maintainers get ready for a new release before it lands.

James Pic

unread,
May 25, 2016, 6:57:51 AM5/25/16
to django-d...@googlegroups.com
I've found testing your own projects on django master to be
tremendously useful. Then, I don't have any surprise when I test on
django alpha, everything passes and I have nothing to do. Not to
mention the tremendous amount of things I learn on the way, at a
slower, more regular pace. Compare this to before, when I was waiting
too long to test against the new/next version of django, had to face a
ton of different failures at the last minute, had to go and couchsurf
at another contributor's house to peer-hack at night after our
dayjobs. BUT that's how I made a best friend for life and that's
priceless hehe even though I wouldn't hold this as a technical
argument for not testing against master !

I guess each person will have a different opinion. Nowadays, I add
djangomaster to my test matrix on all projects, in "allowed failures",
and find out it's broken every once in a while, and just have 1
problem to fix at the time. It works very well for our community, so,
definitely worth trying and then deciding if that's how you want to
work or not.

Ola Sitarska

unread,
May 25, 2016, 7:38:53 AM5/25/16
to django-d...@googlegroups.com
The way we do it for Djangae right now is creating a `support-110` branch whenever we've got some capacity to start working on it, or we wanna assess the amount of work required. Here is a current status for our Django 1.10 support.

However, we're aiming to move into making nightly builds on Travis that would include running tests on all supported Django versions + django master, once a day. It doesn't seem trivial though (seems like we will need to run our own version of https://nightli.es/ to run this build on different branch than master). Hopefully this will allow us to catch big breaking changes earlier, even before alpha is out. 

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
Reply all
Reply to author
Forward
0 new messages