On 31 Oct 2019, at 09:49, Florian Apolloner <f.apo...@gmail.com> wrote:
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/07a4594d-d2db-4ae2-a121-1d1bda3e649e%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/FE9D758A-F4EB-47A3-9964-FCDA36663387%40tomforb.es.
1. How do we support Oracle?
2. Can we (or do we want to?) unify it somehow with docker-box?
3. Will the standard github actions concurrency handle the number of builds we put through it in a reasonable time?
On Thursday, October 31, 2019 at 11:08:49 AM UTC+1, Tom Forbes wrote:1. How do we support Oracle?For now we don't
Is there benefit enough in GitHub Actions (over Jenkins) to justify a
move from an open-source based solution?
Is there benefit enough in GitHub Actions (over Jenkins) to justify a
move from an open-source based solution?
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/e509afbc-21ea-4b3b-9e23-18718594fa49%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/d34795bc-7490-43b0-bdcb-8a3023939c9b%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMzw02X%2BqLKO6hKKh6bT0JzVBUubeO4AaW4o3YQAXnCpxcJvYw%40mail.gmail.com.
The usage limits are documented here: https://help.github.com/en/github/automating-your-workflow-with-github-actions/about-github-actions#usage-limits. I’m not sure how they apply to organisations (maybe just “free”?), but I think Github would allow us an increased limit.
What do we think?
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/e509afbc-21ea-4b3b-9e23-18718594fa49%40googlegroups.com.
On 7 Nov 2019, at 01:10, Matemática A3K <matemat...@gmail.com> wrote:
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CA%2BFDnh%2BTMwfRNTWQiK_gtFjJ-A4wTLsSofv_DV3hPgqcmU7D_g%40mail.gmail.com.
Maybe this discussion is slightly off topic, and at the risk of derailing things I’d like to put out my view on this.There is more to it than just “using Github’s computing power”, just as there is more to using AWS than “using Amazon’s computing power”. That’s only a small part of it: it’s also the ecosystem, the integrations, the maintenance cost, the *ease of use*, the support, etc.Django’s “core business” is Django. Time and resources that are spent not focusing on improving that should be reduced. Keeping Jenkins alive and stable seems like a waste to me. Something similar could be said about flake8 vs black.Jenkins is actually a really interesting and unique piece of technology, far more interesting than most people realize. But who cares because we don’t need interesting. We need to run “pip install flake8 && flake8” on every commit and show the results in Github. That’s a solved problem thanks to Travis, CircleCI, actions or even Gitlab-CI with 0 effort from us.So IMO it shouldn’t matter at all if Jenkins is open or closed source. Time is, as always, our fiat and the expenditure of it is what counts. To that end I would honestly avoid self hosting runners, there are some easier alternatives we can try first to support Oracle.Anyway, I will make the flake8 merge request tomorrow and we can see where we go from there.
On 7 Nov 2019, at 01:10, Matemática A3K <matema...@gmail.com> wrote:
On Wed, Nov 6, 2019 at 3:51 AM Carlton Gibson <carlto...@gmail.com> wrote:Hey Shai.
On Wednesday, 6 November 2019 08:43:21 UTC+1, Shai Berger wrote:
Is there benefit enough in GitHub Actions (over Jenkins) to justify a
move from an open-source based solution?I don't think we have to move away entirely but it would be good to bring in something else... (or at least try it...)These are the top on my mind reasons for wanting to:* Mariusz spends Quite a lot of time™ maintaining Jenkins and all the job definitions etc. This doesn't go away with another builder but if we can move to declarative config file in the repo, then that could become shared work. (Jenkins has this these days no...? But we don't...)* I'd really like to try GitHub actions Windows builds. Maybe we could get Jenkins to behave better but currently we have a failure on every force push.* Maybe we can stage runs: i.e. do the lint, and some basic builds first. Do one Python against each DB before running then all. And so on, to save some trees. (Again maybe we might be able to do this with Jenkins but it seems more likely to actually happen if we give GitHub Actions a trial.)* I think we're running up against capacity for the sponsored space, so builds slow down. If we can spread the load we should get faster CI.Kind Regards,CarltonI also share Shai's concerns. Thinking a bit about it, it is about depending on other's resources for the workflow. Nothing in Django changes, as all the tests are the same and can be ran with tox - and all the source code is the same - the difference would be using Github's computing power instead of Jenkin's for leveraging resources (i.e. manpower) which are not infinite.It would be ideal - to me - to have the entire workflow depending on FOSS solutions. I guess this was one of the reasons that Jenkins was chosen instead, say, Travis, but if the health of Django improves with this, the overall impact to the the community will be better than staying being users of a project which we don't contribute (at least to my knowledge): almost nothing changes to Jenkins - well, it starts loosing users over Github, I think the same may happen with Gitlab - and Django gets an improvement because is already dependent on Github.It would be also an alarm to Jenkins in the way that it needs to catch up with others, if Github Actions end up providing the same functionality with a lower setup and maintainability effort, the might migrate there.When you use others' servers like in SaaS, there is no way but trusting the other to "behave as expected", even using the AGPL. As long as it a conscious decision, seems good to me :)
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/e509afbc-21ea-4b3b-9e23-18718594fa49%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-d...@googlegroups.com.
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/xDCmZPLTOQQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/60D608A7-2902-42A2-A2A7-E4EA8395DA07%40tomforb.es.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/e509afbc-21ea-4b3b-9e23-18718594fa49%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CA%2BFDnh%2BTMwfRNTWQiK_gtFjJ-A4wTLsSofv_DV3hPgqcmU7D_g%40mail.gmail.com.
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/xDCmZPLTOQQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-developers+unsubscribe@googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a499aadf-1cd7-4000-b7a1-8ef4bca39364%40googlegroups.com.
This is fantastic! Thank you Johannes! Regarding sqlite MacOS and Windows tests, this is historical AFAIK (we don’t have the resources). With this we should test MacOS and Windows as part of the Sqlite matrix (i.e all python versions).A few notes:1. We need to change the test runner output to be verbose when running inside actions, as GitHub actions uses line buffered output.
2. We can’t run Oracle in actions for a number of reasons. We could consider exposing the test database instance to Github actions via a proxy or a whitelisted IP address, and we could implement the current “Buildbot, test on oracle” command in GitHub actions pretty trivially.
3. We should have separate stages for Chrome and Firefox, but Firefox should be set to allow failures for now. We can use the selenium hub feature of the test runner in combination with the “services” actions option to make this a bit simpler.
4. Regarding the databases, I think the more correct way would be to use a “service” rather than apt-installing it. See here for example: https://github.com/actions/example-services/blob/master/.github/workflows/postgres-service.yml#L20-L30. This would enable us to test the matrix of database versions, which is as important as python versions IMO.
5. Developers hate duplication, but I think a readable CI definition that includes duplication is better than a de-duplicated one that’s a mess of YAML anchors and special cases (see my attempts in docker-box!). I might even suggest a separate actions file per test-group (linting/each database/selenium).
However for now I propose this as a course of action:1. We add Flake8, isort and docs to the main Github repository (basically this: https://github.com/codingjoe/django/blob/30962/flake8/.github/workflows/ci.yml#L1-L31). I think this is good to go as-is and is low risk. We leave this running in parallel with Jenkins for a period of time to evaluate it, and after that we remove those Jenkins jobs entirely when we are happy.
2. We add SQLite tests one at a time: first Ubuntu, then Windows and finally MacOS. We can experiment with these in separate repositories if needed until we are happy with the configuration. After this we should become comfortable with developing CI changes as merge requests, and should not need separate repositories. We can also use the Sqlite definitions as a template for future databases. We also need to add specific test settings files to the repository, even for SQlite which needs memcached configured (see django-docker-box for examples).
3. We add a full-matrix MariaDB build. This is our newest supported database and there have been a few failures on master on specific untested versions.
4. We then add each database individually in a similar fashion, and think about Oracle right at the end.
If this plan sounds reasonable then let's create some smaller tickets. IMO the exact specifics of how the database tests will look is still fuzzy, but for the linting it’s perfectly clear. However if anyone is not happy with actions in the main repository then let's keep discussing.
Tom
On 7 Nov 2019, at 01:10, Matemática A3K <matema...@gmail.com> wrote:
On Wed, Nov 6, 2019 at 3:51 AM Carlton Gibson <carlto...@gmail.com> wrote:Hey Shai.
On Wednesday, 6 November 2019 08:43:21 UTC+1, Shai Berger wrote:
Is there benefit enough in GitHub Actions (over Jenkins) to justify a
move from an open-source based solution?I don't think we have to move away entirely but it would be good to bring in something else... (or at least try it...)These are the top on my mind reasons for wanting to:* Mariusz spends Quite a lot of time™ maintaining Jenkins and all the job definitions etc. This doesn't go away with another builder but if we can move to declarative config file in the repo, then that could become shared work. (Jenkins has this these days no...? But we don't...)* I'd really like to try GitHub actions Windows builds. Maybe we could get Jenkins to behave better but currently we have a failure on every force push.* Maybe we can stage runs: i.e. do the lint, and some basic builds first. Do one Python against each DB before running then all. And so on, to save some trees. (Again maybe we might be able to do this with Jenkins but it seems more likely to actually happen if we give GitHub Actions a trial.)* I think we're running up against capacity for the sponsored space, so builds slow down. If we can spread the load we should get faster CI.Kind Regards,CarltonI also share Shai's concerns. Thinking a bit about it, it is about depending on other's resources for the workflow. Nothing in Django changes, as all the tests are the same and can be ran with tox - and all the source code is the same - the difference would be using Github's computing power instead of Jenkin's for leveraging resources (i.e. manpower) which are not infinite.It would be ideal - to me - to have the entire workflow depending on FOSS solutions. I guess this was one of the reasons that Jenkins was chosen instead, say, Travis, but if the health of Django improves with this, the overall impact to the the community will be better than staying being users of a project which we don't contribute (at least to my knowledge): almost nothing changes to Jenkins - well, it starts loosing users over Github, I think the same may happen with Gitlab - and Django gets an improvement because is already dependent on Github.It would be also an alarm to Jenkins in the way that it needs to catch up with others, if Github Actions end up providing the same functionality with a lower setup and maintainability effort, the might migrate there.When you use others' servers like in SaaS, there is no way but trusting the other to "behave as expected", even using the AGPL. As long as it a conscious decision, seems good to me :)
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/e509afbc-21ea-4b3b-9e23-18718594fa49%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CA%2BFDnh%2BTMwfRNTWQiK_gtFjJ-A4wTLsSofv_DV3hPgqcmU7D_g%40mail.gmail.com.
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/xDCmZPLTOQQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-developers+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/60D608A7-2902-42A2-A2A7-E4EA8395DA07%40tomforb.es.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsub...@googlegroups.com.
Hey Tom,
Good to hear for you and very good points. I'll leave you a couple of inline comments:
On Friday, November 8, 2019 at 1:04:50 AM UTC+9, Tom Forbes wrote:This is fantastic! Thank you Johannes! Regarding sqlite MacOS and Windows tests, this is historical AFAIK (we don’t have the resources). With this we should test MacOS and Windows as part of the Sqlite matrix (i.e all python versions).A few notes:1. We need to change the test runner output to be verbose when running inside actions, as GitHub actions uses line buffered output.Yes, 100%2. We can’t run Oracle in actions for a number of reasons. We could consider exposing the test database instance to Github actions via a proxy or a whitelisted IP address, and we could implement the current “Buildbot, test on oracle” command in GitHub actions pretty trivially.You could use the official docker image, but it's going to be painfully slow.
3. We should have separat stages for Chrome and Firefox, but Firefox should be set to allow failures for now. We can use the selenium hub feature of the test runner in combination with the “services” actions option to make this a bit simpler.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/e509afbc-21ea-4b3b-9e23-18718594fa49%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CA%2BFDnh%2BTMwfRNTWQiK_gtFjJ-A4wTLsSofv_DV3hPgqcmU7D_g%40mail.gmail.com.
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/xDCmZPLTOQQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/60D608A7-2902-42A2-A2A7-E4EA8395DA07%40tomforb.es.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a499aadf-1cd7-4000-b7a1-8ef4bca39364%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a3e81cca-e658-4b73-80ba-48fd92ef697c%40googlegroups.com.
Hey Tom,Good to hear for you and very good points. I'll leave you a couple of inline comments:
On Friday, November 8, 2019 at 1:04:50 AM UTC+9, Tom Forbes wrote:This is fantastic! Thank you Johannes! Regarding sqlite MacOS and Windows tests, this is historical AFAIK (we don’t have the resources). With this we should test MacOS and Windows as part of the Sqlite matrix (i.e all python versions).A few notes:1. We need to change the test runner output to be verbose when running inside actions, as GitHub actions uses line buffered output.Yes, 100%2. We can’t run Oracle in actions for a number of reasons. We could consider exposing the test database instance to Github actions via a proxy or a whitelisted IP address, and we could implement the current “Buildbot, test on oracle” command in GitHub actions pretty trivially.You could use the official docker image, but it's going to be painfully slow.3. We should have separate stages for Chrome and Firefox, but Firefox should be set to allow failures for now. We can use the selenium hub feature of the test runner in combination with the “services” actions option to make this a bit simpler.We currently use borwserhub on our CI system, if I am not mistaken. So using the headless version, was just a PoC. We should connect it to browserhub too I guess.
4. Regarding the databases, I think the more correct way would be to use a “service” rather than apt-installing it. See here for example: https://github.com/actions/example-services/blob/master/.github/workflows/postgres-service.yml#L20-L30. This would enable us to test the matrix of database versions, which is as important as python versions IMO.I am not sure if this is really the best move. I decided against "services" on other projects, mainly because of performance reasons. My current understanding is, that the containers are executed inside the virtual machine. Besides that you are stuck with stock images. In the case of Postgres, you will not get any extensions, like PostGIS.Then again, we should try and verify if any of my points are valid.
5. Developers hate duplication, but I think a readable CI definition that includes duplication is better than a de-duplicated one that’s a mess of YAML anchors and special cases (see my attempts in docker-box!). I might even suggest a separate actions file per test-group (linting/each database/selenium).Hm... I think I undestand where you want to go with this, but will having separate files really avoid duplication? Anyhow, a file has a meaning. It is a workflow. You can have multiple jobs in a workflow. Jobs can depend on one anohter, but only if they belong to the same workflow. So you can't build stages across workflows. Therefore, I would stick with a single file approach and wait until GitHub fully supports YAML.It's not that much duplication and steps cloud be bundled using another strategy, but I won't go for that just now.
However for now I propose this as a course of action:1. We add Flake8, isort and docs to the main Github repository (basically this: https://github.com/codingjoe/django/blob/30962/flake8/.github/workflows/ci.yml#L1-L31). I think this is good to go as-is and is low risk. We leave this running in parallel with Jenkins for a period of time to evaluate it, and after that we remove those Jenkins jobs entirely when we are happy.Me likey!2. We add SQLite tests one at a time: first Ubuntu, then Windows and finally MacOS. We can experiment with these in separate repositories if needed until we are happy with the configuration. After this we should become comfortable with developing CI changes as merge requests, and should not need separate repositories.
We can also use the Sqlite definitions as a template for future databases. We also need to add specific test settings files to the repository, even for SQlite which needs memcached configured (see django-docker-box for examples).MacOS fails right now. On my local machine too, we should really test aginst it. But it will require fixing some tests.3. We add a full-matrix MariaDB build. This is our newest supported database and there have been a few failures on master on specific untested versions.Yes, please I need a helping hand here. I'm a Postgres kind a guy.4. We then add each database individually in a similar fashion, and think about Oracle right at the end.Poor Oracle, but I absolutely agree.If this plan sounds reasonable then let's create some smaller tickets. IMO the exact specifics of how the database tests will look is still fuzzy, but for the linting it’s perfectly clear. However if anyone is not happy with actions in the main repository then let's keep discussing.It does, but let's wait just a bit longer. I'd like others to evaluate the new insides. I'm sure they have some good idea too.
Tom
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/e509afbc-21ea-4b3b-9e23-18718594fa49%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CA%2BFDnh%2BTMwfRNTWQiK_gtFjJ-A4wTLsSofv_DV3hPgqcmU7D_g%40mail.gmail.com.
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/xDCmZPLTOQQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/60D608A7-2902-42A2-A2A7-E4EA8395DA07%40tomforb.es.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a499aadf-1cd7-4000-b7a1-8ef4bca39364%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a3e81cca-e658-4b73-80ba-48fd92ef697c%40googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/e509afbc-21ea-4b3b-9e23-18718594fa49%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CA%2BFDnh%2BTMwfRNTWQiK_gtFjJ-A4wTLsSofv_DV3hPgqcmU7D_g%40mail.gmail.com.
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/xDCmZPLTOQQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-developers+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/60D608A7-2902-42A2-A2A7-E4EA8395DA07%40tomforb.es.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a499aadf-1cd7-4000-b7a1-8ef4bca39364%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-d...@googlegroups.com.
On Thu, Nov 7, 2019 at 11:31 AM Johannes Hoppe <in...@johanneshoppe.com> wrote:Hey Tom,Good to hear for you and very good points. I'll leave you a couple of inline comments:
On Friday, November 8, 2019 at 1:04:50 AM UTC+9, Tom Forbes wrote:This is fantastic! Thank you Johannes! Regarding sqlite MacOS and Windows tests, this is historical AFAIK (we don’t have the resources). With this we should test MacOS and Windows as part of the Sqlite matrix (i.e all python versions).A few notes:1. We need to change the test runner output to be verbose when running inside actions, as GitHub actions uses line buffered output.Yes, 100%2. We can’t run Oracle in actions for a number of reasons. We could consider exposing the test database instance to Github actions via a proxy or a whitelisted IP address, and we could implement the current “Buildbot, test on oracle” command in GitHub actions pretty trivially.You could use the official docker image, but it's going to be painfully slow.3. We should have separate stages for Chrome and Firefox, but Firefox should be set to allow failures for now. We can use the selenium hub feature of the test runner in combination with the “services” actions option to make this a bit simpler.We currently use borwserhub on our CI system, if I am not mistaken. So using the headless version, was just a PoC. We should connect it to browserhub too I guess.Browserhub == Browserstack == Selenium hub? If so, there should be the 3, with Chrome and Firefox the main focus, and if some device fails in the Selenium hub it wouldn't be a merge-blocker.
4. Regarding the databases, I think the more correct way would be to use a “service” rather than apt-installing it. See here for example: https://github.com/actions/example-services/blob/master/.github/workflows/postgres-service.yml#L20-L30. This would enable us to test the matrix of database versions, which is as important as python versions IMO.I am not sure if this is really the best move. I decided against "services" on other projects, mainly because of performance reasons. My current understanding is, that the containers are executed inside the virtual machine. Besides that you are stuck with stock images. In the case of Postgres, you will not get any extensions, like PostGIS.Then again, we should try and verify if any of my points are valid.According to https://help.github.com/en/actions/automating-your-workflow-with-github-actions/virtual-environments-for-github-hosted-runners, you can run Docker containers besides the stock images
5. Developers hate duplication, but I think a readable CI definition that includes duplication is better than a de-duplicated one that’s a mess of YAML anchors and special cases (see my attempts in docker-box!). I might even suggest a separate actions file per test-group (linting/each database/selenium).Hm... I think I undestand where you want to go with this, but will having separate files really avoid duplication? Anyhow, a file has a meaning. It is a workflow. You can have multiple jobs in a workflow. Jobs can depend on one anohter, but only if they belong to the same workflow. So you can't build stages across workflows. Therefore, I would stick with a single file approach and wait until GitHub fully supports YAML.It's not that much duplication and steps cloud be bundled using another strategy, but I won't go for that just now.Seems reasonable to me also, conditional and stages functionality is more important in this case.
However for now I propose this as a course of action:1. We add Flake8, isort and docs to the main Github repository (basically this: https://github.com/codingjoe/django/blob/30962/flake8/.github/workflows/ci.yml#L1-L31). I think this is good to go as-is and is low risk. We leave this running in parallel with Jenkins for a period of time to evaluate it, and after that we remove those Jenkins jobs entirely when we are happy.Me likey!2. We add SQLite tests one at a time: first Ubuntu, then Windows and finally MacOS. We can experiment with these in separate repositories if needed until we are happy with the configuration. After this we should become comfortable with developing CI changes as merge requests, and should not need separate repositories.We can also use the Sqlite definitions as a template for future databases. We also need to add specific test settings files to the repository, even for SQlite which needs memcached configured (see django-docker-box for examples).MacOS fails right now. On my local machine too, we should really test aginst it. But it will require fixing some tests.3. We add a full-matrix MariaDB build. This is our newest supported database and there have been a few failures on master on specific untested versions.Yes, please I need a helping hand here. I'm a Postgres kind a guy.4. We then add each database individually in a similar fashion, and think about Oracle right at the end.Poor Oracle, but I absolutely agree.If this plan sounds reasonable then let's create some smaller tickets. IMO the exact specifics of how the database tests will look is still fuzzy, but for the linting it’s perfectly clear. However if anyone is not happy with actions in the main repository then let's keep discussing.It does, but let's wait just a bit longer. I'd like others to evaluate the new insides. I'm sure they have some good idea too.The plan sounds right to me, taking away the linting from Jenkins would alleviate a bit the overload on it and not hit the capacity limit.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CA%2BFDnhJ7xchX2%3DcEQVx4XCUTv_KcxMJrpyO_%2BQwhH3TrCt7vwQ%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAPcC2onP%2BKt0U0fr5w-U5ZnPq74PioDFD6xFnkzquAV%3DsyhoZA%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/fdcbfc00-977c-4587-a823-8e806cddf280%40Spark.
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/xDCmZPLTOQQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/17BD40A2-6A1B-4B44-8F49-11F2520F9898%40tomforb.es.
I would suggest to open a PR to GitHub now and to get the Fellows involved. Ask them how to best integrate a trial without disturbing their workflow. What do you guys think?
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/xDCmZPLTOQQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/6692d50f-b02d-4824-aa12-be22a20c3f0f%40googlegroups.com.