[GSoC 2014 Proposal] Improving Error and Fixing up related issues.

[anubhav joshi]

Hi all,
       My previous thread: https://groups.google.com/forum/#!topic/django-developers/PDY8EEYaHao , where I was told by Tim Graham to make all analysis and discussion on tickets itself and then present here.

I did what I was told, I discussed issues and strategies on Trac and IRC both. My proposal is available as a gist : https://gist.github.com/coder9042/c505dfba9514a6e52fdc .

Please have a look and help me in improving it.
If there are any questions, I will be happy to answer.

Regards,
Anubhav Joshi
IIT Patna

[Josh Smeaton]

Just a brief comment. Perhaps it'd be useful to link to the appropriate trac tickets in your proposal.

Regards,

Josh

[anubhav joshi]

Please refer to the thread : https://groups.google.com/forum/#!topic/django-developers/PDY8EEYaHao for tickets...

Actually, all of a sudden there has been some problem in my laptop since this morning and I can't open my hard disk currently. So, I request you all to please view the proposal and leave your reviews and comments. I am posting this using a friend's computer, I will be back as soon as I fix up my laptop. Probably not more than a day.

[Josh Smeaton]

Yes, people can look elsewhere for the tickets, but it'd probably have more *impact* if the tickets were referenced directly in the proposal. That way, someone can dive into the deeper discussion for a particular bug (by reading the comments on the ticket). The idea is that you don't want your readers to have to work harder. Make it as easy as possible for your reader to find the information they may want.

Good luck.

Regards,

[anubhav joshi]

OK, will do.

Anubhav Joshi

[anubhav joshi]

Just realized that the proposal name is : Improving error reporting and fixing up related issues.
Missed 'reporting' in the thread topic.

Anubhav Joshi

[Tim Graham]

Hi Anubhav,

It'll be easier to review if you can include ticket numbers with each item in the 2.X sections. e.g. "Unpickling Models and QuerySet from incompatible version. (#XXXXX)" Ideally, the ticket number would be linked to Trac. Then I would repeat these summaries & ticket numbers in each 3.X bullet before your analysis. I would also combine the timeline details in section 4 with the analysis in section 3. As it is now, it's difficult to review because you have to cross reference the two sections. You could include a summary in section 4 of just ticket numbers and estimated weeks, but the text like "Approach 1:... would be better in section 3 I think.

[anubhav joshi]

Done the required.
https://gist.github.com/coder9042/c505dfba9514a6e52fdc

[anubhav joshi]

I do keep updating gist upon getting feedback/reviews on IRC as well.
Please keep a track.
https://gist.github.com/coder9042/c505dfba9514a6e52fdc

[Russell Keith-Magee]

Hi Anubhav,

I've taken a look at your proposal, and I'm of two minds.

On the one hand, you've identified a bunch of reasonably uncontroversial tickets - issues where there is a clear problem, a solution that seems obvious, and in most cases indications of support from the core team. Your proposal gives a clear indication of what is on your "hit list", and because the work is already nicely granular, your timeline is expressed in units that make it believable. So - from this perspective, you've got a good proposal for a block of work that can be performed in a 12 week window.

The problem I have is that other than "squashing bugs", there isn't really a cohesive theme in these tickets. They don't even stick to a single area of the code - you're spread over db. http, core, forms and template. 

There's two reasons why this matters. The first is theoretical, and the second is practical.

Firstly, the theoretical concern: The GSoC is an opportunity to look at something big, not just squash bugs. We've got an opportunity for someone to spend 12 weeks sinking their teeth into something complex. While there's certainly nothing wrong with 12 weeks of fixing little problems, it doesn't really strike me as something that meets the broader aspirations of a program like GSoC. We've been given a great opportunity here; it seems a waste to spend it on lots of little things, however, annoying those little things may be.

The practical concern is this: Tackling lots of little tickets is going to be a much higher workload on a potential mentor. Consider - you've listed 17 tickets (from a quick count); each of them describes a different problem. In the worst possible case, each of these tickets may require a detailed discussion on django-developers to discuss the implications of a potential fix. At the very least, there's going to be a lot of disparate review work that your mentor will need to do. This changes the mentor's workload from "weekly guidance and a few small code reviews, and one big code review at the end" to something that requires an almost constant stream of little code reviews. Without these code reviews, your project will essentially stall. Speaking personally, I'd have to consider very carefully before I took on this kind of workload.

I appreciate that we've possibly led you down this path by pointing you at the ticket database and saying "find tickets about errors". What we haven't communicated well is that ideally, there should be a common theme. 

As a point of comparison, here's a commit from about 4 years ago. 

https://github.com/django/django/commit/c4c27d8a04c9125cfbc5c3611557d8e5d3845b0d

This wasn't done as a GSoC project, but it was done by someone who was a student at the time; and it's a good representation of the sort of outcome that I'd like to see. 

This single commit closed 10 tickets. But it wasn't a matter of fixing 10 little things; Ben identified a systemic set of problem in the syndication framework, and set about fixing the bigger problem. With one commit, it was possible to address all 10 issues - and probably a dozen others that hadn't been reported - because the revised syndication framework was structured so as to avoid these issues.

Don't get me wrong - bugs are bad, and it would be nice to have less of them. Many of the bugs you've found relate to the ways that errors manifest as a result of slightly abnormal usage. And your proposal, as written, is solid. If we can find someone willing to mentor it, and we get enough slots, it stands a fair chance of being selected. 

However, if you're looking for my advice on how to make it stronger and improve your chances -- I'd go looking to see if you can either recast it so that the "common thread" is more obvious (I'm not sure if this is even possible), or take another look at the list of open tickets and try and pick a set that has a slightly more cohesive focus. 

As one last piece of guidance -- the reason this idea was on the wiki was because there were (at least historically), a bunch of areas where Django would raise errors, but the errors that Django raised were not helpful. This was particularly true during template rendering, and to a lesser extent when import errors occurred during model loading. The idea driving the project was to improve this "grand idea" - making it easier to identify the actual source of the problem. Looking at tickets was suggested as a course of action because when big picture "it's bad" issues exist in a framework, they often get reported in a myriad different ways, each reflecting a subtlety in the bigger problem. The idea wasn't strictly to get you to dig up a bunch of tickets to fix; it was to use the tickets as a roadmap to identifying the "bigger" problem.

 

That said - if someone steps up and volunteers to take on the workload of shepherding lots of smaller, mostly unrelated tickets, I think this project would probably represent a solid contribution to Django. I just know that personally, I'm not going to be able to put my hand up on this one.

Yours,

Russ Magee %-)

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/c1d75c1f-f6a7-4d9e-b633-ea467a80aa9c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[anubhav joshi]

On Wednesday, March 12, 2014 5:57:23 PM UTC+5:30, Russell Keith-Magee wrote:

Hi Anubhav,

I've taken a look at your proposal, and I'm of two minds.

On the one hand, you've identified a bunch of reasonably uncontroversial tickets - issues where there is a clear problem, a solution that seems obvious, and in most cases indications of support from the core team. Your proposal gives a clear indication of what is on your "hit list", and because the work is already nicely granular, your timeline is expressed in units that make it believable. So - from this perspective, you've got a good proposal for a block of work that can be performed in a 12 week window.

The problem I have is that other than "squashing bugs", there isn't really a cohesive theme in these tickets. They don't even stick to a single area of the code - you're spread over db. http, core, forms and template. 

There's two reasons why this matters. The first is theoretical, and the second is practical.

Firstly, the theoretical concern: The GSoC is an opportunity to look at something big, not just squash bugs. We've got an opportunity for someone to spend 12 weeks sinking their teeth into something complex. While there's certainly nothing wrong with 12 weeks of fixing little problems, it doesn't really strike me as something that meets the broader aspirations of a program like GSoC. We've been given a great opportunity here; it seems a waste to spend it on lots of little things, however, annoying those little things may be.

The practical concern is this: Tackling lots of little tickets is going to be a much higher workload on a potential mentor. Consider - you've listed 17 tickets (from a quick count); each of them describes a different problem. In the worst possible case, each of these tickets may require a detailed discussion on django-developers to discuss the implications of a potential fix. At the very least, there's going to be a lot of disparate review work that your mentor will need to do. This changes the mentor's workload from "weekly guidance and a few small code reviews, and one big code review at the end" to something that requires an almost constant stream of little code reviews. Without these code reviews, your project will essentially stall. Speaking personally, I'd have to consider very carefully before I took on this kind of workload.

Well I have already sought out much of what needs to be done by talking to several people on IRC channel. I have very good idea of exactly what needs to be done in which. I have discussed issue with loic, akaariai, timgraham, etc on IRC and taken a note of the way they suggested to handle that particular problem. So there is not going to be any such concern.
 

I appreciate that we've possibly led you down this path by pointing you at the ticket database and saying "find tickets about errors". What we haven't communicated well is that ideally, there should be a common theme. 

As a point of comparison, here's a commit from about 4 years ago. 

https://github.com/django/django/commit/c4c27d8a04c9125cfbc5c3611557d8e5d3845b0d

This wasn't done as a GSoC project, but it was done by someone who was a student at the time; and it's a good representation of the sort of outcome that I'd like to see. 

This single commit closed 10 tickets. But it wasn't a matter of fixing 10 little things; Ben identified a systemic set of problem in the syndication framework, and set about fixing the bigger problem. With one commit, it was possible to address all 10 issues - and probably a dozen others that hadn't been reported - because the revised syndication framework was structured so as to avoid these issues.

Don't get me wrong - bugs are bad, and it would be nice to have less of them. Many of the bugs you've found relate to the ways that errors manifest as a result of slightly abnormal usage. And your proposal, as written, is solid. If we can find someone willing to mentor it, and we get enough slots, it stands a fair chance of being selected. 

I took different parts of django under consideration because errors and theie reporting is not just confined to a single part or module. If we need to fiz something we need to do that completely. And as I said earlier I have been constantly making a good note of how to deal with such problems and I assure that their will not that amount of workload on the mentor as it seems from the first look of the proposal.
 

However, if you're looking for my advice on how to make it stronger and improve your chances -- I'd go looking to see if you can either recast it so that the "common thread" is more obvious (I'm not sure if this is even possible), or take another look at the list of open tickets and try and pick a set that has a slightly more cohesive focus. 

As one last piece of guidance -- the reason this idea was on the wiki was because there were (at least historically), a bunch of areas where Django would raise errors, but the errors that Django raised were not helpful. This was particularly true during template rendering, and to a lesser extent when import errors occurred during model loading. The idea driving the project was to improve this "grand idea" - making it easier to identify the actual source of the problem. Looking at tickets was suggested as a course of action because when big picture "it's bad" issues exist in a framework, they often get reported in a myriad different ways, each reflecting a subtlety in the bigger problem. The idea wasn't strictly to get you to dig up a bunch of tickets to fix; it was to use the tickets as a roadmap to identifying the "bigger" problem.

 

All these areas that are mentioned in my proposal are related to errors being reported because at places where they should not be, or being raised because of some problem in the code, which is related to 'improve error reporting'. Further I extended the idea to fixing related issues.
As you would have seen there are many things that are proposed to be taken up like 'refactoring handlers', 'uri_to_iri', etc. which are important and they in their help in raising wrong types of error.
 

[anubhav joshi]

Having replied to the potential questions above, I would like to summarize here once again:

1.) I have a very clear idea of what area of code I will be dealing with.
2.) I have been in conversation with community members on IRC, and have taken their say in fixing the issues mentioned in the proposal.
3.) I assure that the mentor will have no such problem of overloading himself with a lot of work.
Even if the issues consider several parts, I have good knowledge of those things and clear idea of what I have to do.
4.) The reason I chose for taking different modules of django under considerations is that if we want to improve upon something why not do it entirely instead of some parts.
5.) The idea is still targeted at making error reporting proper because all the issues taken into considerations do deal with bad errors coming out misleading the user.

I hope my answers satisfy the questions.
Waiting to hear more from you.

[anubhav joshi]

Well another thing I think I realize now is that I think is there seems to be problem of absence of common theme.
To this I would say that if one would look closely at the issues covered up in the proposal are those which do report of bad error reporting not just simple bugs. The issues do report of misleading messages popping up, whatever the reason may be. I did keep in mind this thing while doing my background research of issues on Trac or IRC.

Regards,
Anubhav Joshi

[Russell Keith-Magee]

On Wed, Mar 12, 2014 at 9:58 PM, anubhav joshi <anubh...@gmail.com> wrote:

Well another thing I think I realize now is that I think is there seems to be problem of absence of common theme.

To this I would say that if one would look closely at the issues covered up in the proposal are those which do report of bad error reporting not just simple bugs. The issues do report of misleading messages popping up, whatever the reason may be. I did keep in mind this thing while doing my background research of issues on Trac or IRC.\

I understand that there is a theme in that sense, but that's not really what I was referring to. In Ben's case with the syndication tickets, you could argue that there was a common theme - syndication didn't work. However, the fix wasn't lots of little unrelated changes; the fix was to address a root cause. When Ben's patch was committed, we closed 10 bugs - and 10 others that existed, but hadn't been reported yet. 

The sort of bugs you've reported here are definitely all bugs - but the fix to one ticket doesn't relate to the fix for another. The approach you take to fix bug 1 won't impact on the approach for bug 2. And there almost certainly won't be any "related" bugs cleaned up along the way. There's no "root cause", beyond "humans make mistakes", or "we've been lazy in our error checking". You're not going to be going through the entire codebase looking for ways that we're systematically doing one small thing badly.

Again - I'm not saying that these bugs aren't worth fixing - they are. But speaking personally, I like to see a bit more "big picture" in a GSoC application.

Yours,

Russ Magee %-)

[anubhav joshi]

I read on google melange:
"mentoring organizations may request further proposal detail from the student applicant."

My tests are about to start so I want to know if and when should I be expecting this.

Regards,
Anubhav

[Tim Graham]

As far as I know, we have no plans to request further details from any of our applications.

[anubhav joshi]

Ok. Thanks

Anubhav