Make Development More Accessible

[John Gooding]

I'd like to propose moving Django issues to github and make a real decision on it here in this thread. If there has been a recent discussion on this I apologize, but searching for issue tracking / github links to about every thread ever posted here.

I believe this would lower the barrier to entry and to help promote community involvement. People are already there, people already use it, and we already do pull requests there. Now I could be wrong here, but I also feel that it would improve and promote discussion about changes and feature additions to Django, because right now they are pretty hidden away in the current system. 

I'd also like to see the inclusion of a "discussion" label or similar for issues. I think many of the conversations here on this forum would be much better off as github issues. I see a lot of great stuff, and it's not clear at all what the status is, has it moved forward, been officially denied? etc. If they are github issues they will have definitive resolutions, whatever it may be, and links to relevant code, PR's etc if needed.

I think there is a huge amount to gain by consolidating the ticket system and many of the discussions on this forum into github's issue tracker. I don't see any reason why it wouldn't be wroth the effort, and we only have much to gain as a community from it. But that's just my 2 cents. I'd love to hear what others think, for or against it.

John

[Aymeric Augustin]

Hello John,

This was discussed before, when we moved from self-hosted svn to GitHub-hosted git, but I'm not sure there are public archives of all discussions.

As far as I remember, the main points to tackle are:

1. Does GitHub allow "anonymous triage" i.e. labelling, closing, and reopening issues by non-committers? I think there was a recent announcement in this area. I didn't check the details. Previously, bot-powered workarounds were suggested, but they wouldn't provide a good user experience. You want discoverable buttons, not a cheat sheet of magic comments.

2. Does the GitHub UI scale to thousands of issues? In theory, any classification system can be reproduced with namespaced labels e.g. "component:ORM", "status:ready-for-checkin", etc. In practice, it's unlikely to be as convenient as what currently exists on Trac.

Perhaps it's just me, but I always found GitHub issues hard to use when I had more than on page of issues. Indeed, at that point, I need a labelling system to filter issues. Then I need to keep all the rules of that system in my head instead of having the UI guide me — and prevent me from infringing the system...

3. How do we migrate issues history from Trac to GitHub? Preserving comment authorship doesn't seem obvious, especially for authors who don't have the same username on Trac and GitHub or authors who don't have a GitHub account.

Initially an effort was made to sync usernames of core devs between Trac and GitHub to prevent security problems but that's a small subset of contributors.

4. Are we still able to export everything from GitHub and move on to the next thing? Perhaps there's an obvious answer. I didn't look. Usually Django takes a pragmatic position: we won't reject GitHub outright because it isn't open source. However, we wouldn't want to lock ourselves into a platform we don't control.

Who would have bet, three years ago, that GitHub would be the property of Microsoft today? What if Microsoft sells it to Oracle in three years? It's nice to keep our options open :-)

We put the code there because we were confident that we could pull the git history. Then everyone started using pull requests, which was likely a good thing, but wasn't really planned or thought through, and I don't think we can export PR comments meaningfully. GitHub did some good vendor lock in there.

5. How do we preserve links to SVN commits? Currently, they're redirected on https://code.djangoproject.com/ with this nginx rule:

    rewrite ^/changeset/(\d+)/?$ https://www.djangoproject.com/svntogit/$1/ permanent;

and then redirected again by this application:

    https://github.com/django/djangoproject.com/tree/master/svntogit

It would be nice to preserve these links in issues copied from Trac to GitHub, which probably means pre-processing comments to rewrite links.

There may be more, but that's what comes to mind!

A process DEP is the way to go to propose this change.

Best regards,

-- 

Aymeric.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/da5ca4b1-fb84-4ed4-b2cb-324b8bea9c42%40googlegroups.com.

[John Gooding]

To put it short, the barrier to entry is far too high and difficult for newcomers and even long time users of Django. 

I agree with others sentiment that there isn't anything that trac can do which github issues cannot, especially for the overwhelming majority of tickets. As a long time user of Django myself, I agree with Andrew Godwins statement on Django loosing many contributors over the years and being in largely maintenance mode. 

I believe this has contributed significantly to the current culture that things are fine as a third party application and that most big new features or additions are simply too difficult, or not worth the effort etc. There is very much a culture of "it can't be, or wont be done", and is quite pessimistic. That's simply one point of my assessment about the current culture, and isn't meant to be derogatory towards the current contributors and the fine work they are doing.

I agree that something like django-async, big, bold, new features are exactly what Django needs to get new people on board and actively participating in the project. The barrier to that should be as low as possible, and the entire development process should be as consolidated, clear, and accessible as possible. That's why I think moving the majority of this forum and trac to github issues is the right move.

For a couple of recent-ish background posts that discuss much of this:

https://groups.google.com/d/topic/django-developers/aiyE__qSHBY/discussion

https://groups.google.com/d/msg/django-developers/5CVsR9FSqmg/qKD3QCrLCAAJ

[John Gooding]

Hi Aymeric,

You bring up a lot of good points. There will undoubtedly be challenges and huge amount of work in moving to a new system, or implementing any big sweeping changes, however, I truly honestly believe that it would be worth it in the long run, and the payoff would far outweigh the cost.

As far as Microsoft owning github, etc I think it is almost moot. Any process will have some amount of vendor lock in, whether github, atlassian (jira & bitbucket parent company), or even gitlab. I think what is important is to pick one system as a community that we are happy with. Any one of those three could do what is ultimately needed, which is a centralized and consistent development platform.

-- 

Aymeric.

To unsubscribe from this group and stop receiving emails from it, send an email to django-d...@googlegroups.com.

[Josh Smeaton]

Mariatta has put together a some PEPs for migrating CPython issues over to GitHub.

https://www.python.org/dev/peps/pep-0581/ proposing the migration.

https://www.python.org/dev/peps/pep-0588/ migration plan.

Django and Cpython are not the same, so there'll be substantial differences. But it's worth familiarising oneself with prior art.

For what it's worth I'd strongly support such a move just for the increase in engagement.

Carlton, Mariusz, how would you gauge the level of triage activity in Trac from non-core members? High/Medium/Low?

https://github.blog/changelog/2019-05-23-triage-and-maintain-roles-beta/ describes the new triage and maintain roles, but they're still to be granted to trusted individuals (which would be an excellent gateway into full core membership if that is the direction Django is going to continue in).

[Carlton Gibson]

The more I use Trac, the more I appreciate its power. I'm normally all for Progress™ but I'm not sure GitHub's UI is up to it. 

(Being able to find the old discussion is super handy: it's not that often that an idea has not come up before at this stage.) 

I'd be interested to see what a prototype export looks like in a test GitHub repo. Maybe it's possible... (Note this is in bold.)

Maybe more people would participate, but I'm not sure... Do we just suspect that? I worry we go to a load of effort for no real gain. 

Current input is quite good I'd say. Claude and Simon are regulars. There's a good number more who make frequent appearances. 

I think there's more people commenting than we suspect. (Anyone trying the export would be able to do numbers I'd guess...)

If the new Triaging role on GH would allow "Request a review..." I think it would be super handy. (But currently that's restricted to the more powerful roles.) (I'm nagging GH about as best I can but if anyone knows anyone...)

Happy to comment more if people want, but those are the highlights. 

C.

[Andrew Godwin]

We actually discussed this a little at the PyCon AU sprints and the consensus was that GitHub issues would be great if only they were a bit more featureful.

The problems I feel are specifically an issue:

- Ticket states; this is not easily replicated with labels, while components etc. are mapped slightly more easily

- Assignment of tickets to people outside the organisation - while bots allow triage, I believe taking ticket ownership requires that you are either a member of the organization or have commented on the ticket, which makes the process there a little longer.

- Ticket history, as mentioned (we'd need to preserve ticket numbers and redirect links from the current domain).

Exporting from GitHub seems very possible - exporting pull requests and issues can both be done via the API, though this does, like the bot, need extra work.

This all seems eminently doable, but the question I'd really want everyone to answer is if it's worth the porting effort. I suspect the answer is yes, but this does need a process DEP and some discussion, and maybe also looking at what cpython are doing and comparing and contrasting.

Andrew

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/b9b9713b-6ca8-467d-b0ab-818ae1a7ee68%40googlegroups.com.

[John Gooding]

Hi All,

I want to thank you all for your time and input. I'll start doing some research into how cpython and others are managing this. I will draw up a few options and present them so we can better work out the possibilities and details before submitting a process DEP. I don't want to submit one if there is not a clear path forward, or a general feeling that it could work and be worth it.

As far as will we actually get more engagement, or would it all be for no gain? I think the only place there is to go is up. There's 43k stars on github and 18k forks. I'll try to get some more concrete numbers about current engagement level by looking at the data. It might be worth it to then compare this projects of a similar nature / size, though I'm not sure how much validity that would have, being apples and oranges.

~John

On Wednesday, August 7, 2019 at 1:47:58 PM UTC-7, Andrew Godwin wrote:

We actually discussed this a little at the PyCon AU sprints and the consensus was that GitHub issues would be great if only they were a bit more featureful.

The problems I feel are specifically an issue:

- Ticket states; this is not easily replicated with labels, while components etc. are mapped slightly more easily

- Assignment of tickets to people outside the organisation - while bots allow triage, I believe taking ticket ownership requires that you are either a member of the organization or have commented on the ticket, which makes the process there a little longer.

- Ticket history, as mentioned (we'd need to preserve ticket numbers and redirect links from the current domain).

Exporting from GitHub seems very possible - exporting pull requests and issues can both be done via the API, though this does, like the bot, need extra work.

This all seems eminently doable, but the question I'd really want everyone to answer is if it's worth the porting effort. I suspect the answer is yes, but this does need a process DEP and some discussion, and maybe also looking at what cpython are doing and comparing and contrasting.

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/b9b9713b-6ca8-467d-b0ab-818ae1a7ee68%40googlegroups.com.

[Carlton Gibson]

Just on this point: 

> I agree with Andrew Godwins statement on Django loosing many contributors over the years and being in largely maintenance mode. 

First, I'm not sure Andrew actually said this. Rather I think he reported is a point raised. However...

I hear this kind of thing said. It ties-in with the "Django is boring" trope, when that's not meant as a compliment. 

I think it couldn't be further from the truth. 

Yes, these ain't the wild west days of yore. (Granted)

But Django has successfully transitioned (with large thanks to the effort Tim Graham put in as Fellow over that time) from young to mature and it still growing. 

Even if you take out your favourite headline feature — don't say it :) — v3.0 is going to be an awesome release. 2.2 was too. And 2.1. And it's continuing forward. 

I wish we'd drop negative self-talk. It lowers morale and isn't even correct. Django is in great shape, and only getting stronger. 

Could we do with more contributors? Yes. So let's do what we can to make that better. 

I think of the effort to move from Trac. Maybe it's worth it... But I think of the same effort spent improving djangoproject.com, and the How to contribute guides (I think Tobias said the Contributing docs are X-thousands of words—all great but not necessarily that approachable), and so on, and I wonder if it's the best ROI? (That's a genuinely open question: DRF uses GitHub issues I don't see it leading to any more engagement there...) 

Go Django Go! 🙂

[Tim Graham]

Although I'm not engaged too much with Django development now, a big drawback of moving to GitHub issues for me would be that I could no longer do a Google search like "something something site:code.djangoproject.com". I could pretty much always find the ticket I was looking for that way. Maybe GitHub issue search would be just as good but I find Google results (with a snippet from the page) more useful.

Another migration consideration is that Trac tickets are formatted using a different syntax than GitHub issues.

A migration feels like weeks of work and it doesn't strike me as a "heck yea!" improvement. There would probably be benefits and drawbacks. As Carlton said, I think it's time that could be more usefully allocated.

I wonder how putting issues on GitHub would increase engagement? Is it that Trac isn't discoverable from GitHub? What if GitHub let us redirect https://github.com/django/django/issues (with the "Issues" tab) to code.djangoproject.com?

[William Vincent]

I think the way Rails does it, aka with well-done newcomers guide (https://edgeguides.rubyonrails.org/contributing_to_ruby_on_rails.html) is worth looking at, as Carlton notes. A bit more streamlined than the current Django How To Contribute Guides.

Incidentally, Carlton and I will be having a chat with DHH of Ruby on Rails fame in September for the DjangoChat podcast so if there are any questions we should pose to him on managing the Rails community, do pass along to us. Certainly I'll ask him about issue tracking!

[1337 Shadow Hacker]

Given the number of Open Pull request, does Django craves more contribution quantity, or quality ? Not the same focus