[Django] #19891: Travis CI support
Django
--------------------------------------+---------------------------------
Reporter: marko@… | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Testing framework | Version: master
Severity: Normal | Keywords: Travis, CI, testing
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+---------------------------------
= Use cases =
== Standardizing testing ==
Contributors test their own changes against all different databases and
supported python versions and testing is standardized.
This means they do not have to setup configure and maintain multiple
database setups, install multiple python versions etc.
== Pull requests get tested automatically ==
This gives core-devs a first impression on the quality of a patch.
= History =
@apollo13 has made some initial attempts at supporting Travis CI, but
stopped working on this
because the tests were taking too long and were causing timeouts. New work
being done is based
on his initial attempts.
= Development =
Work on this topic is currently being done at
https://github.com/dokterbob/django
= Discussion =
Not everyone is too fond of travis-ci. Sometimes people tend to pollute
the commit history with commits just to
trigger Travis CI.
The work being done is not to replace Jenkins CI. It is only intended to
improve and standardize testing of Django by
none-core developers.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: nobody
Type: | Status: new
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage:
Keywords: Travis, CI, testing | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by jacob):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
I don't think having Travis test all versions is such a good idea. Right
now the build generates 12 different builds for each pull request which'll
take approximately forever to run. Asking people to wait on pull requests
while a dozen different builds run is... not so good.
Instead, how about using Travis to augment what we already have in
Jenkins? Just have it run maybe two builds -- Python 2 and Python 3? That
was contributors can get a quick feedback on a pull request, but don't
have to wait forever.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:1>
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage:
Keywords: Travis, CI, testing | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by dokterbob):
There's a pull request now, running the tests:
https://github.com/django/django/pull/805
The tests take 25-45 minutes to run, which is not 'forever' AFAIK. Travis
runs 3-5 builds in parallel which actually makes it fairly fast. However,
somehow, Travis adds up the time for the individual builds to come up with
a rather unrealistic build time of multiple hours.
As the core dev's have a Jenkins CI anyways, this shouldn't matter too
much for them. However, many contributors would benefit tremendously by
being able to test their code throughly even before firing pull requests -
which is the main point of this patch. And lastly, it *might* happen that
Travis finds issues which (somehow) slip through the mazes of the Jenkins
CI, as was the case while we set up the testing system. (Ref:
https://github.com/django/django/commit/8c1cc4b3b04f639ae40bc806380bc4a9dd568eb0)
However, having it run on 4 Python versions might indeed be too much.
Which two Python releases do you suggest we use?
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:2>
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage:
Keywords: Travis, CI, testing | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by jacob):
Hm, 45 minutes isn't bad; Travis (or our tests) must have gotten faster
since last I checked. So I'll drop my objection.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:3>
Django
--------------------------------------+------------------------------------
Reporter: marko@… | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Testing framework | Version: master
Keywords: Travis, CI, testing | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by jacob):
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:4>
Django
-------------------------------------+-------------------------------------
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Keywords: Travis, CI, testing | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by dokterbob):
* owner: nobody => dokterbob
* status: new => assigned
* has_patch: 0 => 1
* cc: mathijs@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: dokterbob
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Travis, CI, testing | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by dokterbob):
* easy: 0 => 1
Comment:
Any news on this one?
Will it me merged? If not; what's blocking it?
Any further modifications needed?
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: dokterbob
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Travis, CI, testing | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 1 |
-------------------------------------+-------------------------------------
Comment (by apollo13):
I am waiting for feedback on
https://groups.google.com/forum/?fromgroups=#!topic/django-
developers/RobUVK0y0Mg
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:7>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: dokterbob
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Travis, CI, testing | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 1 |
-------------------------------------+-------------------------------------
* cc: apollo13 (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:8>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: dokterbob
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Travis, CI, testing | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 1 |
-------------------------------------+-------------------------------------
Comment (by aaugustin):
I simply hope that I won't receive too many emails indicating failures of
Travis' infrastructure.
During the sprint I received an email informing me that I had broken the
build. The email pointed to a docs commit. No comment!
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:9>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: dokterbob
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Travis, CI, testing | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 1 |
-------------------------------------+-------------------------------------
Comment (by dokterbob):
@aaugustin Could you forward the mail to me? I'm *very* curious as to why
you'd receive mail for tests in a repo which isn't yours. In any case such
emails should be disabled until Travis' stability is sufficiently proven -
this should mainly be a tool for use by contributors *before* the code
gets to you guys.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:10>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: dokterbob
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Travis, CI, testing | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 1 |
-------------------------------------+-------------------------------------
Comment (by aaugustin):
I just attached the email (I scrubbed the headers first).
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:11>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: dokterbob
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Travis, CI, testing | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 1 |
-------------------------------------+-------------------------------------
Comment (by dokterbob):
@aaugustin Actually, the changeset seems to contain a whole bunch of
changes. I remember that during the sprint for a while the master branch
was broken which might have caused the e-mail.
But really, we don't want to bother you guys with tons of emails for
broken/fixed pull requests. Hence, I propose adding to the `.travis.yml`:
{{{
notifications:
email: false
}}}
However, having notifications on IRC might be nice. I'm not an avid user
myself so I'm open to any suggestions. Refer to: http://about.travis-
ci.org/docs/user/notifications/#IRC-notification
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:12>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: dokterbob
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Travis, CI, testing | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 1 |
-------------------------------------+-------------------------------------
Comment (by apollo13):
> However, having notifications on IRC might be nice. I'm not an avid
user myself so I'm open to any suggestions. Refer to: http://about
.travis-ci.org/docs/user/notifications/#IRC-notification
Absolutely not, we don't want notifications for pull requests etc.
Also the email issue has to be figured out, travis is an absolute nogo if
people start receiving mails while Django doesn't even have a travis.yml
file (Independent of what the notification setting is).
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:13>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: dokterbob
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Travis, CI, testing | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 1 |
-------------------------------------+-------------------------------------
Comment (by dokterbob):
@apollo13 I was surprised about that as well. Perhaps you could ask around
on #travis what's happening here?
As the default behaviour of Travis is to be verbose, I suppose switching
email notifications off will do the trick for now.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:14>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: dokterbob
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Travis, CI, testing | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 1 |
-------------------------------------+-------------------------------------
Comment (by apollo13):
Ugh, I retried my separate config approach and I am getting crazy errors
all along: https://api.travis-ci.org/jobs/5486568/log.txt?deansi=true
(https://travis-ci.org/apollo13/django/builds/5486564). Also fwiw: loading
the last job with a failed log completely freezes Firefox. I am once again
wondering how it worked for you :þ
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:15>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: dokterbob
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Travis, CI, testing | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 1 |
-------------------------------------+-------------------------------------
Comment (by camilonova):
+1 on this, will be great to have it, seems more easy to read the config
file from dokterbob and adding the configuration for not sending emails
should work fine as well http://about.travis-
ci.org/docs/user/notifications/#Email-notifications
What else is needed to have this merged? i'm glad to help if needed
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:16>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: dokterbob
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Travis, CI, testing | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 1 |
-------------------------------------+-------------------------------------
Comment (by apollo13):
@camilonova: As discussed on the mailing list we are going with the
"harder" approach, that is fetching travis config from a 3rd server and
keeping the travis file as small as possible…
Regarding help, well forking my fork ;) and figuring out why it's broken
would help, I am currently out of ideas.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:17>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: dokterbob
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Travis, CI, testing | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 1 |
-------------------------------------+-------------------------------------
Comment (by anonymous):
TLDR; Single commit everytime Travis tests break should be no problem, but
if it is will work on Travis repo.
@apollo13 The talk just now at DjangoCon made me remember this issue and
read the related thread on the mailinglist. Have had no time to actually
discuss on the dev' list but will present some arguments here.
Personally I don't think it's elegant having a seperate repository just to
keep a couple of commits out of the public timeline. Most Travis issues
can be easily fixed in a fork/pull request as the exact same tests are run
under the exact same circumstances there - part of the point of having
Travis there in the first place.
Having a seperate repo with just a config file potentially makes the tests
more fragile as a change to the Travis config could cause tests to fail
for no clearly apparent reason.
If I could talk you guys into having Travis support in the main Django
repo I will to the following:
1. Pick up old work; rebase my branch to current master.
2. Disable e-mail and IRC notifications for core dev's piece of mind.
3. Run the tests again a couple of times to see if anything breaks.
4. Create a single cute litte patch so there'll no 'fixing Travis' in the
public timeline for now.
Anyways, if the core devs do decide to go with a split-repo approach to
Travis I'm willing to work on it during DjangoCon EU and will either
finish it or leave it 'open' by the beginning of the sprints (will take
train on friday). What I need to do this is:
1. (Temporary) access (directly or indirectly, through pull req's) to a
`travis-support` GH repo.
2. A quick overview of what happened on this since the latest sprint from
apollo13.
3. Clear instructions on what the new approach will entail, what the goals
are and what the conditions are under which it will be committed.
Having Travis for Django still seems like a really good idea which could
significantly increase the (quality of) contributions. In any case, I'm
walking around on DjangoCon until friday afternoon. If you have questions
or want to work on me with this one, poke me. If you are a core dev and
want to discuss, poke me. If you're not a core dev, poke a core dev to get
this in (preferably before the sprints). Then, during the sprints, we'll
see quick enough whether Travis will prove to be a curse or a blessing but
I'm counting on the latter to be the case.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:18>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: dokterbob
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Travis, CI, testing | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 1 |
-------------------------------------+-------------------------------------
Comment (by dokterbob):
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:19>
Django
--------------------------------------+------------------------------------
Reporter: marko@… | Owner:
Component: Testing framework | Version: master
Keywords: Travis, CI, testing | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
--------------------------------------+------------------------------------
Changes (by anonymous):
* status: assigned => new
* owner: dokterbob =>
Comment:
I'm leaving this for others to work on during the DjangoCon EU sprints as
it is not going to happen today.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:20>
Django
--------------------------------------+------------------------------------
Reporter: marko@… | Owner:
Component: Testing framework | Version: master
Keywords: Travis, CI, testing | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
--------------------------------------+------------------------------------
Comment (by gcc):
@dokterbob I think from the discussion above that we **don't** want Travis
to build master or any other branch of django, only pull requests? Core
devs have their own Jenkins setups to run tests for them.
So polluting the timeline is not a problem, because it should never be
necessary to make a commit on any branch of django/django to trigger a
Travis build because these branches are not built by Travis. Is that
correct?
I had trouble getting started on closing this ticket so I've made a TODO
list of the [wiki:ContinuousIntegration outstanding issues]. I hope this
is useful.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:21>
Django
--------------------------------------+------------------------------------
Type: Cleanup/optimization | Status: assigned
Component: Testing framework | Version: master
Keywords: Travis, CI, testing | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
--------------------------------------+------------------------------------
* owner: => gcc
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:22>
Django
--------------------------------------+------------------------------------
Reporter: marko@… | Owner: gcc
Type: Cleanup/optimization | Status: assigned
Keywords: Travis, CI, testing | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
--------------------------------------+------------------------------------
Comment (by gcc):
[https://code.djangoproject.com/wiki/ContinuousIntegration#PostgresCrashing
Postgres on Travis crashes] when we run all the tests due to running out
of disk space. It seems that this might happen when
[http://www.postgresql.org/message-id/4EBE12B4...@2ndQuadrant.com
disk writes are slow], which I can imagine happening in a virtual machine.
Can any Postgres expert help debug and fix this?
The current state of my work is
[https://github.com/aptivate/django/tree/travisci in
github/aptivate/django].
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:23>
Django
--------------------------------------+------------------------------------
Reporter: marko@… | Owner: gcc
Type: Cleanup/optimization | Status: assigned
Keywords: Travis, CI, testing | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
--------------------------------------+------------------------------------
Comment (by gcc):
[https://github.com/django/django/pull/1223 Pull request].
This branch adds the necessary support for Travis into Django, as
discussed on [ticket #19891](https://code.djangoproject.com/ticket/19891).
It runs all Django tests on the following environments:
* SQLite, no GIS
* MySQL, with and without GIS
* PostgreSQL, with and without GIS
It also builds the docs. It does not currently run Selenium tests (because
it cannot run `ChromeDriver`, even though it's executable and in the
PATH).
Due to some quirks in Travis, it will try to build branches even if they
have no `.travis.yml` file. It will try to build them as Ruby code, which
will fail. Therefore this file must be created with some minimal contents
in all regularly-changing branches of Django before enabling Travis,
otherwise people will get email notifications when those branches are
modified, triggering a Travis build which will fail. I suggest using a
copy of `.travis.yml` from this branch.
I have also seen the [https://travis-ci.org/aptivate/django/jobs/7518794
build time out] when running PostgreSQL tests (over 50 minutes). This may
well happen again. We might need to split the PostgreSQL tests into two
separate runs.
Many thanks to @dokterbob and @apollo13 who did all the hard work that
made this possible.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:24>
Django
--------------------------------------+------------------------------------
Reporter: marko@… | Owner: gcc
Type: Cleanup/optimization | Status: assigned
Keywords: Travis, CI, testing | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
--------------------------------------+------------------------------------
Comment (by apollo13):
Hi, there are a few issues with the PR:
* As I said before I don't wan't the config stuff to be in the repo
* The postgres fixes are crazy, please get in touch with the travis guys
(I don't think writes are slow they moved to ramfs to my knowledge)
* "emails: false" is not enough, this doesn't prevent someone from re
enabling it in his PR and then spamming everyone else…
* Splitting PG tests is not an option.
* Selenium used to work, why can't it run chromedriver all of a sudden?
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:25>
Django
--------------------------------------+------------------------------------
Reporter: marko@… | Owner: gcc
Type: Cleanup/optimization | Status: assigned
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: Travis, CI, testing | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
--------------------------------------+------------------------------------
Comment (by gcc):
Hi @apollo13,
Please could you tell me why you don't want config stuff in the repo?
Loading them from an outside web server:
* is dangerous (security risk to Travis and hence our reputation),
* is unreliable (might go away suddenly with no warning),
* makes it impossible to test new configs if you don't have write access
to the repo where they're stored, and
* eliminates them from version control, or at least separates them into a
different repo or VCS for no reason that I can see.
I have already been in touch with Travis guys about the
[https://github.com/travis-ci/travis-ci/issues/1125 Postgres issues] as I
could not find any other way to stop the tests failing due to running out
of disk space. What's crazy about them?
Will look into the email issue, but I'm not sure we can do anything except
creating a new account with a /dev/null email address and making it a
collaborator on the django repo. Will look into Selenium as well.
Why is splitting the PG tests not an option?
Cheers, Chris.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:26>
Django
--------------------------------------+------------------------------------
Reporter: marko@… | Owner: gcc
Type: Cleanup/optimization | Status: assigned
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: Travis, CI, testing | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
--------------------------------------+------------------------------------
Comment (by apollo13):
Replying to [comment:26 gcc]:
> Please could you tell me why you don't want config stuff in the repo?
As I already said on the mailinglist: I don't want to have commits in
Django's timeline saying: "And another fix for travis, oh and another one
for travis again". It's bad enough that they require me to add a
.travis.yml, CI shouldn't be bound to a repo like this.
> * is dangerous (security risk to Travis and hence our reputation),
Why would that be a security risk?
> * is unreliable (might go away suddenly with no warning),
More unreliable than Travis? Also once we deploy this we will either fetch
from dp.com or similar, so it surely won't just go away.
> * makes it impossible to test new configs if you don't have write access
to the repo where they're stored, and
No, you just have to change the environment variable in your branch and
point to your server.
> * eliminates them from version control, or at least separates them into
a different repo or VCS for no reason that I can see.
The main idea for this split was to be able to version them in a separate
repository; after all CI-config depends on the one running the CI and
shouldn't be in the main repo.
> What's crazy about them?
Cause it's completely fragile, if travis changes from /var/ramfs to
something else your builds start failing.
> Will look into the email issue, but I'm not sure we can do anything
except creating a new account with a /dev/null email address and making it
a collaborator on the django repo.
I doubt that will help, Travis (at least that's how I understood it) looks
at the email addy of the commits that got pushed to the repo and then bugs
people accordingly.
> Why is splitting the PG tests not an option?
Cause it introduces more complexity, this should be a last resort.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:27>
Django
--------------------------------------+------------------------------------
Reporter: marko@… | Owner: gcc
Type: Cleanup/optimization | Status: assigned
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: Travis, CI, testing | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
--------------------------------------+------------------------------------
Changes (by gcc):
* easy: 1 => 0
Comment:
Hi @apollo13,
Status update:
* I finally managed to persuade the Travis guys to [https://github.com
/travis-ci/travis-ci/issues/1125#issuecomment-19539914 increase the size
of the ramdisk] used for the databases. The tests now [https://travis-
ci.org/aptivate/django/jobs/8274059 pass] without reconfiguring
PostgreSQL.
* I also moved the Travis configuration scripts and files to a Git repo so
that I don't have to host them somewhere (except in Github) to run tests
in Travis. I hope that's an acceptable alternative to HTTP hosting.
* I can't do anything about the [https://github.com/travis-ci/travis-
ci/issues/280 email notification issue], it's a limitation of Travis that
[https://github.com/travis-ci/travis-ci/issues/1094 will hopefully be
fixed one day]. If this is a blocker to merging then Travis support will
not be merged any time soon, and there's nothing I can do about it.
* I added support for UNLOGGED tables that should speed up running tests
in Postgres for all users. That could be considered a separate
branch/ticket if necessary.
* Selenium tests are still broken. I want to see if I can overcome the
other issues first before investing time in fixing them.
You can see the latest work
[https://github.com/aptivate/django/commits/ticket_19891 here]. Please let
me know if these things are acceptable. If so then I'll keep working on
fixing selenium tests.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:28>
Django
--------------------------------------+------------------------------------
Reporter: marko@… | Owner: gcc
Type: Cleanup/optimization | Status: assigned
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: Travis, CI, testing | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
Comment (by timo):
Talking to apollo13 on IRC, it seems like the email notification issue is
indeed a blocker to merging this. We discussed the possibility of adding
[http://buddylindsey.com/jenkins-and-github-pull-requests/ pull request
support] to our Jenkins setup. We need to add additional executors to
Jenkins and make it more stable before we can proceed on that.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:29>
Django
-------------------------------------+-------------------------------------
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timo):
* stage: Accepted => Someday/Maybe
Comment:
If Travis fixes email notifications, we can move this back to "Accepted"
and continue work.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:30>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: gcc
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage:
Keywords: Travis, CI, testing | Someday/Maybe
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by dokterbob):
There is an issue for Travis suggesting to move notification settings away
from the `.travis.yml` file and into the database. This would probably fix
the notification problems for forks/pull requests.
https://github.com/travis-ci/travis-ci/issues/1094
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:31>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: gcc
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage:
Keywords: Travis, CI, testing | Someday/Maybe
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by bouke):
There's a policy change landing next week (27 nov) worth looking into:
> The new policy will be as follows:
> 1. If the commit occurs on the default branch, then the owners of the
repository will be notified.
> 2. If the commit occurs on a non-default branch, the author and the
committer of the commit who are also owners of the repository will be
notified.
http://about.travis-ci.org/blog/2013-11-19-upcoming-email-notification-
policy-change/
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:32>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: gcc
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage:
Keywords: Travis, CI, testing | Someday/Maybe
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by mjtamlyn):
My instinct is that this change, combined with a `notifications: false` in
the config should be sufficient to prevent spam.
Apollo's other comments still stand though, and in particular if the
selenium tests continue to fail then we have a blocking issue.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:33>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: gcc
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Testing framework | Resolution:
Severity: Normal | Triage Stage:
Keywords: Travis, CI, testing | Someday/Maybe
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by gcc):
Also please confirm that the other changes I made in my branch are
acceptable. If so then I'll look at fixing Selenium. Otherwise I give up.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:34>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: gcc
Cleanup/optimization | Version: master
Component: Testing framework | Resolution: wontfix
Severity: Normal | Triage Stage:
Keywords: Travis, CI, testing | Someday/Maybe
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => wontfix
Comment:
We now have a scalable Jenkins setup with pull request integration at
djangoci.com.
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:35>
Django
-------------------------------------+-------------------------------------
Reporter: marko@… | Owner: gcc
Type: | Status: closed
Cleanup/optimization |
Keywords: Travis, CI, testing | Triage Stage:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by tobiasmcnulty):
I created a "lightweight" Travis/tox setup for my own purposes so I can
check pull requests before submitting them to Django proper. The goal is
not to replace traviceci.com but to provide a lightweight sanity check and
help avoid clogging up Travis when making frequent commits.
Anyone who stumbles across this ticket is welcome to copy what I did to
your own GitHub/Travis account or submit a PR to my fork to see if it
builds. '''I will not, of course, review your PR or merge it into this
fork.''' There are no guarantees that it'll be completely up to date
(create a fork for yourself if needed).
The repo is here, on the "travis" branch:
https://github.com/tobiasmcnulty/django
--
Ticket URL: <https://code.djangoproject.com/ticket/19891#comment:36>