Disclaimer: I'm only just joining this community so I absolutely want
those of you with more experience developing/fixing bugs in Hudson and
elsewhere to chime in on this conversation. What I want to do is try
to come to some consensus on how Hudson bugs are to be identified,
disambiguated, prioritized, and ultimately, scheduled (more or less)
for a release. This is part of what I hope will be a larger overall
effort to bring the Technical Debt I've observed since diving into the
source code into check.
As of right now there are 1853 open bugs in Hudson, of which the
average age is just under a year at 359 days. Some users, myself
included, have been getting discouraged with cycle time on bugs,
particularly when the Continuous Integration server is supposed to be
the one thing you can most rely on. When teams make their CI system
the heart and soul of their day-to-day development, testing,
configuration management, and release processes, they need to able to
rely on it. If any tool can be categorized as mission-critical, it's
the CI system.
Here is what I am proposing:
1. Users submit bug reports at issues.hudson-ci.org like they always
have.
2. At some frequency (daily?, weekly?, as they pop up?), developers go
out and perform the following procedure:
a. Pick the oldest non-triaged bug.
b. Check whether or not the bug is a duplicate with another bug.
This will likely require aid from the more seasoned Hudson
contributors.
i. If a duplicate exists but it has been resolved, I would
recommend not reopening it, but just linking the two issues together
so the person who goes to fix the current incantation of the bug can
use it as a resource if necessary. Continue to the next step.
ii. If a duplicate exists, but is still open, link the two issues
for future reference, Close the new one as Duplicate, and redirect all
users to the older one. Go back to 2.a.
iii. If no duplicate exists, continue.
c. Is the bug reproducible in the most recent version of Hudson? I
don't think it's worth verifying the bug in their reported version
since they'll ultimately have to upgrade to get the bug fix anyway, so
we might as well work towards the shortest path.
i. If the bug is not reproducible in the latest version, close as
Cannot Reproduce and inform the user(s) that it appears to have been
fixed in version X, and go back to 2.a.
ii. If the bug appears to still exist in the current version, note
which version it was verified in and continue.
d. Gather as much information as possible from users. What version
of the system is this bug occurring in? What environment are they
using? Did this error start occurring after an upgrade? What version
did they upgrade to/from? Are they able to work around the issue or
has it stopped their usage of Hudson in its tracks? Ask any questions
that might help isolate the commit that introduced the bug, and help
other contributors make the best possible prioritization decisions.
e. Set the priority of the issue based on some criteria. I need
some suggestions from the community about what would be the best
criteria. I would actually suggest trying to get an open source
license of GreenHopper from Atlassian. Their ranking functionality
within GreenHopper works very well for this purpose since you rank
items relative to each other, not just with priorities.
f. Mark the bug as triaged and if there are still non-triaged issues
go back to 2.a.
3. Contributors grab the highest (relative) priority bug to fix and
get to it. Votes should also contribute to deciding which bug to fix
since that information is real-time and earlier triage information can
become more and more out of date.
I'm extremely interested in getting some feedback about these
procedures. I'm sure I missed some opportunity in this process or
overlooked something, so please let me know what you think. I hope
that by unifying these procedures across the community we can work
towards reducing that "average age" stat to below a month or two. I'm
not saying it will be easy, but I'm certain this community can do it.
I think this is a very nice proposal. Some comments:
In my experience most of the very old bugs fall into one of these scenarios:
- happen only under very rare circumstances. Developer cannot reproduce it easily - are not reproducible because reporter didn't provide enough details (and often doesn't respond to questions about details) - fix would break some other functionality and/or would mean serious re-engineering of some parts of Hudson - are related to a plugin which is not (really) maintained any more
And then there are of course the 'normal' bugs, but I think they are the minority.
This is just my very subjective feeling why there are so many old bugs.
What we can do easily IMHO:
- search actively for duplicates and close them
- ask reporters about more details - if bug is old and reporter doesn't respond in a timely manner, then close it as 'not reproducible'
- And what I really like to see, is a prominent message for reporters of new bugs to provide as much details as possible ALREADY WHEN they enter the bug initially. So much time is wasted by asking for details and waiting for the answer. Your list in 2d is very good. I'd like to add console output and logfile to it.
> Disclaimer: I'm only just joining this community so I absolutely want > those of you with more experience developing/fixing bugs in Hudson and > elsewhere to chime in on this conversation. What I want to do is try > to come to some consensus on how Hudson bugs are to be identified, > disambiguated, prioritized, and ultimately, scheduled (more or less) > for a release. This is part of what I hope will be a larger overall > effort to bring the Technical Debt I've observed since diving into the > source code into check.
> As of right now there are 1853 open bugs in Hudson, of which the > average age is just under a year at 359 days. Some users, myself > included, have been getting discouraged with cycle time on bugs, > particularly when the Continuous Integration server is supposed to be > the one thing you can most rely on. When teams make their CI system > the heart and soul of their day-to-day development, testing, > configuration management, and release processes, they need to able to > rely on it. If any tool can be categorized as mission-critical, it's > the CI system.
> Here is what I am proposing:
> 1. Users submit bug reports at issues.hudson-ci.org like they always > have. > 2. At some frequency (daily?, weekly?, as they pop up?), developers go > out and perform the following procedure: > a. Pick the oldest non-triaged bug. > b. Check whether or not the bug is a duplicate with another bug. > This will likely require aid from the more seasoned Hudson > contributors. > i. If a duplicate exists but it has been resolved, I would > recommend not reopening it, but just linking the two issues together > so the person who goes to fix the current incantation of the bug can > use it as a resource if necessary. Continue to the next step. > ii. If a duplicate exists, but is still open, link the two issues > for future reference, Close the new one as Duplicate, and redirect all > users to the older one. Go back to 2.a. > iii. If no duplicate exists, continue. > c. Is the bug reproducible in the most recent version of Hudson? I > don't think it's worth verifying the bug in their reported version > since they'll ultimately have to upgrade to get the bug fix anyway, so > we might as well work towards the shortest path. > i. If the bug is not reproducible in the latest version, close as > Cannot Reproduce and inform the user(s) that it appears to have been > fixed in version X, and go back to 2.a. > ii. If the bug appears to still exist in the current version, note > which version it was verified in and continue. > d. Gather as much information as possible from users. What version > of the system is this bug occurring in? What environment are they > using? Did this error start occurring after an upgrade? What version > did they upgrade to/from? Are they able to work around the issue or > has it stopped their usage of Hudson in its tracks? Ask any questions > that might help isolate the commit that introduced the bug, and help > other contributors make the best possible prioritization decisions. > e. Set the priority of the issue based on some criteria. I need > some suggestions from the community about what would be the best > criteria. I would actually suggest trying to get an open source > license of GreenHopper from Atlassian. Their ranking functionality > within GreenHopper works very well for this purpose since you rank > items relative to each other, not just with priorities. > f. Mark the bug as triaged and if there are still non-triaged issues > go back to 2.a. > 3. Contributors grab the highest (relative) priority bug to fix and > get to it. Votes should also contribute to deciding which bug to fix > since that information is real-time and earlier triage information can > become more and more out of date.
> I'm extremely interested in getting some feedback about these > procedures. I'm sure I missed some opportunity in this process or > overlooked something, so please let me know what you think. I hope > that by unifying these procedures across the community we can work > towards reducing that "average age" stat to below a month or two. I'm > not saying it will be easy, but I'm certain this community can do it.
I am also really in favor of introducing a backlog with priorities. For the moment, we just have no ideas when things will be fixed (or not). The Greenhoper plugin is easy to use in Jira.
On Sun, Jan 30, 2011 at 8:25 PM, Christoph Kutzinski <ku...@gmx.de> wrote: > I think this is a very nice proposal. > Some comments:
> In my experience most of the very old bugs fall into one of these > scenarios:
> - happen only under very rare circumstances. Developer cannot reproduce it > easily > - are not reproducible because reporter didn't provide enough details (and > often doesn't respond to questions about details) > - fix would break some other functionality and/or would mean serious > re-engineering of some parts of Hudson > - are related to a plugin which is not (really) maintained any more
> And then there are of course the 'normal' bugs, but I think they are the > minority.
> This is just my very subjective feeling why there are so many old bugs.
> What we can do easily IMHO:
> - search actively for duplicates and close them
> - ask reporters about more details > - if bug is old and reporter doesn't respond in a timely manner, then > close it as 'not reproducible'
> - And what I really like to see, is a prominent message for reporters of > new bugs to provide as much details as possible ALREADY WHEN they enter the > bug initially. > So much time is wasted by asking for details and waiting for the answer. > Your list in 2d is very good. I'd like to add console output and logfile to > it.
> Am 30.01.2011 18:13, schrieb mlb5000:
> Hi All,
>> Disclaimer: I'm only just joining this community so I absolutely want >> those of you with more experience developing/fixing bugs in Hudson and >> elsewhere to chime in on this conversation. What I want to do is try >> to come to some consensus on how Hudson bugs are to be identified, >> disambiguated, prioritized, and ultimately, scheduled (more or less) >> for a release. This is part of what I hope will be a larger overall >> effort to bring the Technical Debt I've observed since diving into the >> source code into check.
>> As of right now there are 1853 open bugs in Hudson, of which the >> average age is just under a year at 359 days. Some users, myself >> included, have been getting discouraged with cycle time on bugs, >> particularly when the Continuous Integration server is supposed to be >> the one thing you can most rely on. When teams make their CI system >> the heart and soul of their day-to-day development, testing, >> configuration management, and release processes, they need to able to >> rely on it. If any tool can be categorized as mission-critical, it's >> the CI system.
>> Here is what I am proposing:
>> 1. Users submit bug reports at issues.hudson-ci.org like they always >> have. >> 2. At some frequency (daily?, weekly?, as they pop up?), developers go >> out and perform the following procedure: >> a. Pick the oldest non-triaged bug. >> b. Check whether or not the bug is a duplicate with another bug. >> This will likely require aid from the more seasoned Hudson >> contributors. >> i. If a duplicate exists but it has been resolved, I would >> recommend not reopening it, but just linking the two issues together >> so the person who goes to fix the current incantation of the bug can >> use it as a resource if necessary. Continue to the next step. >> ii. If a duplicate exists, but is still open, link the two issues >> for future reference, Close the new one as Duplicate, and redirect all >> users to the older one. Go back to 2.a. >> iii. If no duplicate exists, continue. >> c. Is the bug reproducible in the most recent version of Hudson? I >> don't think it's worth verifying the bug in their reported version >> since they'll ultimately have to upgrade to get the bug fix anyway, so >> we might as well work towards the shortest path. >> i. If the bug is not reproducible in the latest version, close as >> Cannot Reproduce and inform the user(s) that it appears to have been >> fixed in version X, and go back to 2.a. >> ii. If the bug appears to still exist in the current version, note >> which version it was verified in and continue. >> d. Gather as much information as possible from users. What version >> of the system is this bug occurring in? What environment are they >> using? Did this error start occurring after an upgrade? What version >> did they upgrade to/from? Are they able to work around the issue or >> has it stopped their usage of Hudson in its tracks? Ask any questions >> that might help isolate the commit that introduced the bug, and help >> other contributors make the best possible prioritization decisions. >> e. Set the priority of the issue based on some criteria. I need >> some suggestions from the community about what would be the best >> criteria. I would actually suggest trying to get an open source >> license of GreenHopper from Atlassian. Their ranking functionality >> within GreenHopper works very well for this purpose since you rank >> items relative to each other, not just with priorities. >> f. Mark the bug as triaged and if there are still non-triaged issues >> go back to 2.a. >> 3. Contributors grab the highest (relative) priority bug to fix and >> get to it. Votes should also contribute to deciding which bug to fix >> since that information is real-time and earlier triage information can >> become more and more out of date.
>> I'm extremely interested in getting some feedback about these >> procedures. I'm sure I missed some opportunity in this process or >> overlooked something, so please let me know what you think. I hope >> that by unifying these procedures across the community we can work >> towards reducing that "average age" stat to below a month or two. I'm >> not saying it will be easy, but I'm certain this community can do it.
For that we should try to improve our Jira usage and especially using versions (affect vs fix for). To have this working we need to split the big jira into small pieces (one per plugin + 1 for core).
vtintillier.private...@gmail.com> wrote: > I am also really in favor of introducing a backlog with priorities. For the > moment, we just have no ideas when things will be fixed (or not). The > Greenhoper plugin is easy to use in Jira.
> Vivien
> On Sun, Jan 30, 2011 at 8:25 PM, Christoph Kutzinski <ku...@gmx.de> wrote:
>> I think this is a very nice proposal. >> Some comments:
>> In my experience most of the very old bugs fall into one of these >> scenarios:
>> - happen only under very rare circumstances. Developer cannot reproduce it >> easily >> - are not reproducible because reporter didn't provide enough details (and >> often doesn't respond to questions about details) >> - fix would break some other functionality and/or would mean serious >> re-engineering of some parts of Hudson >> - are related to a plugin which is not (really) maintained any more
>> And then there are of course the 'normal' bugs, but I think they are the >> minority.
>> This is just my very subjective feeling why there are so many old bugs.
>> What we can do easily IMHO:
>> - search actively for duplicates and close them
>> - ask reporters about more details >> - if bug is old and reporter doesn't respond in a timely manner, then >> close it as 'not reproducible'
>> - And what I really like to see, is a prominent message for reporters of >> new bugs to provide as much details as possible ALREADY WHEN they enter the >> bug initially. >> So much time is wasted by asking for details and waiting for the answer. >> Your list in 2d is very good. I'd like to add console output and logfile >> to it.
>> Am 30.01.2011 18:13, schrieb mlb5000:
>> Hi All,
>>> Disclaimer: I'm only just joining this community so I absolutely want >>> those of you with more experience developing/fixing bugs in Hudson and >>> elsewhere to chime in on this conversation. What I want to do is try >>> to come to some consensus on how Hudson bugs are to be identified, >>> disambiguated, prioritized, and ultimately, scheduled (more or less) >>> for a release. This is part of what I hope will be a larger overall >>> effort to bring the Technical Debt I've observed since diving into the >>> source code into check.
>>> As of right now there are 1853 open bugs in Hudson, of which the >>> average age is just under a year at 359 days. Some users, myself >>> included, have been getting discouraged with cycle time on bugs, >>> particularly when the Continuous Integration server is supposed to be >>> the one thing you can most rely on. When teams make their CI system >>> the heart and soul of their day-to-day development, testing, >>> configuration management, and release processes, they need to able to >>> rely on it. If any tool can be categorized as mission-critical, it's >>> the CI system.
>>> Here is what I am proposing:
>>> 1. Users submit bug reports at issues.hudson-ci.org like they always >>> have. >>> 2. At some frequency (daily?, weekly?, as they pop up?), developers go >>> out and perform the following procedure: >>> a. Pick the oldest non-triaged bug. >>> b. Check whether or not the bug is a duplicate with another bug. >>> This will likely require aid from the more seasoned Hudson >>> contributors. >>> i. If a duplicate exists but it has been resolved, I would >>> recommend not reopening it, but just linking the two issues together >>> so the person who goes to fix the current incantation of the bug can >>> use it as a resource if necessary. Continue to the next step. >>> ii. If a duplicate exists, but is still open, link the two issues >>> for future reference, Close the new one as Duplicate, and redirect all >>> users to the older one. Go back to 2.a. >>> iii. If no duplicate exists, continue. >>> c. Is the bug reproducible in the most recent version of Hudson? I >>> don't think it's worth verifying the bug in their reported version >>> since they'll ultimately have to upgrade to get the bug fix anyway, so >>> we might as well work towards the shortest path. >>> i. If the bug is not reproducible in the latest version, close as >>> Cannot Reproduce and inform the user(s) that it appears to have been >>> fixed in version X, and go back to 2.a. >>> ii. If the bug appears to still exist in the current version, note >>> which version it was verified in and continue. >>> d. Gather as much information as possible from users. What version >>> of the system is this bug occurring in? What environment are they >>> using? Did this error start occurring after an upgrade? What version >>> did they upgrade to/from? Are they able to work around the issue or >>> has it stopped their usage of Hudson in its tracks? Ask any questions >>> that might help isolate the commit that introduced the bug, and help >>> other contributors make the best possible prioritization decisions. >>> e. Set the priority of the issue based on some criteria. I need >>> some suggestions from the community about what would be the best >>> criteria. I would actually suggest trying to get an open source >>> license of GreenHopper from Atlassian. Their ranking functionality >>> within GreenHopper works very well for this purpose since you rank >>> items relative to each other, not just with priorities. >>> f. Mark the bug as triaged and if there are still non-triaged issues >>> go back to 2.a. >>> 3. Contributors grab the highest (relative) priority bug to fix and >>> get to it. Votes should also contribute to deciding which bug to fix >>> since that information is real-time and earlier triage information can >>> become more and more out of date.
>>> I'm extremely interested in getting some feedback about these >>> procedures. I'm sure I missed some opportunity in this process or >>> overlooked something, so please let me know what you think. I hope >>> that by unifying these procedures across the community we can work >>> towards reducing that "average age" stat to below a month or two. I'm >>> not saying it will be easy, but I'm certain this community can do it.
@Arnaud, I completely agree that each Hudson plugin should receive its
own "Project" in the Hudson Jira instance. It's weird that they're
split into components, but maybe there's a reason for it?
On Jan 30, 3:49 pm, Arnaud Héritier <arnaud.herit...@exoplatform.com>
wrote:
> For that we should try to improve our Jira usage and especially using
> versions (affect vs fix for).
> To have this working we need to split the big jira into small pieces (one
> per plugin + 1 for core).
> Cheers
> Arnaud
> On Sun, Jan 30, 2011 at 8:56 PM, vtintillier <
> vtintillier.private...@gmail.com> wrote:
> > I am also really in favor of introducing a backlog with priorities. For the
> > moment, we just have no ideas when things will be fixed (or not). The
> > Greenhoper plugin is easy to use in Jira.
> > Vivien
> > On Sun, Jan 30, 2011 at 8:25 PM, Christoph Kutzinski <ku...@gmx.de> wrote:
> >> I think this is a very nice proposal.
> >> Some comments:
> >> In my experience most of the very old bugs fall into one of these
> >> scenarios:
> >> - happen only under very rare circumstances. Developer cannot reproduce it
> >> easily
> >> - are not reproducible because reporter didn't provide enough details (and
> >> often doesn't respond to questions about details)
> >> - fix would break some other functionality and/or would mean serious
> >> re-engineering of some parts of Hudson
> >> - are related to a plugin which is not (really) maintained any more
> >> And then there are of course the 'normal' bugs, but I think they are the
> >> minority.
> >> This is just my very subjective feeling why there are so many old bugs.
> >> What we can do easily IMHO:
> >> - search actively for duplicates and close them
> >> - ask reporters about more details
> >> - if bug is old and reporter doesn't respond in a timely manner, then
> >> close it as 'not reproducible'
> >> - And what I really like to see, is a prominent message for reporters of
> >> new bugs to provide as much details as possible ALREADY WHEN they enter the
> >> bug initially.
> >> So much time is wasted by asking for details and waiting for the answer.
> >> Your list in 2d is very good. I'd like to add console output and logfile
> >> to it.
> >> Am 30.01.2011 18:13, schrieb mlb5000:
> >> Hi All,
> >>> Disclaimer: I'm only just joining this community so I absolutely want
> >>> those of you with more experience developing/fixing bugs in Hudson and
> >>> elsewhere to chime in on this conversation. What I want to do is try
> >>> to come to some consensus on how Hudson bugs are to be identified,
> >>> disambiguated, prioritized, and ultimately, scheduled (more or less)
> >>> for a release. This is part of what I hope will be a larger overall
> >>> effort to bring the Technical Debt I've observed since diving into the
> >>> source code into check.
> >>> As of right now there are 1853 open bugs in Hudson, of which the
> >>> average age is just under a year at 359 days. Some users, myself
> >>> included, have been getting discouraged with cycle time on bugs,
> >>> particularly when the Continuous Integration server is supposed to be
> >>> the one thing you can most rely on. When teams make their CI system
> >>> the heart and soul of their day-to-day development, testing,
> >>> configuration management, and release processes, they need to able to
> >>> rely on it. If any tool can be categorized as mission-critical, it's
> >>> the CI system.
> >>> Here is what I am proposing:
> >>> 1. Users submit bug reports at issues.hudson-ci.org like they always
> >>> have.
> >>> 2. At some frequency (daily?, weekly?, as they pop up?), developers go
> >>> out and perform the following procedure:
> >>> a. Pick the oldest non-triaged bug.
> >>> b. Check whether or not the bug is a duplicate with another bug.
> >>> This will likely require aid from the more seasoned Hudson
> >>> contributors.
> >>> i. If a duplicate exists but it has been resolved, I would
> >>> recommend not reopening it, but just linking the two issues together
> >>> so the person who goes to fix the current incantation of the bug can
> >>> use it as a resource if necessary. Continue to the next step.
> >>> ii. If a duplicate exists, but is still open, link the two issues
> >>> for future reference, Close the new one as Duplicate, and redirect all
> >>> users to the older one. Go back to 2.a.
> >>> iii. If no duplicate exists, continue.
> >>> c. Is the bug reproducible in the most recent version of Hudson? I
> >>> don't think it's worth verifying the bug in their reported version
> >>> since they'll ultimately have to upgrade to get the bug fix anyway, so
> >>> we might as well work towards the shortest path.
> >>> i. If the bug is not reproducible in the latest version, close as
> >>> Cannot Reproduce and inform the user(s) that it appears to have been
> >>> fixed in version X, and go back to 2.a.
> >>> ii. If the bug appears to still exist in the current version, note
> >>> which version it was verified in and continue.
> >>> d. Gather as much information as possible from users. What version
> >>> of the system is this bug occurring in? What environment are they
> >>> using? Did this error start occurring after an upgrade? What version
> >>> did they upgrade to/from? Are they able to work around the issue or
> >>> has it stopped their usage of Hudson in its tracks? Ask any questions
> >>> that might help isolate the commit that introduced the bug, and help
> >>> other contributors make the best possible prioritization decisions.
> >>> e. Set the priority of the issue based on some criteria. I need
> >>> some suggestions from the community about what would be the best
> >>> criteria. I would actually suggest trying to get an open source
> >>> license of GreenHopper from Atlassian. Their ranking functionality
> >>> within GreenHopper works very well for this purpose since you rank
> >>> items relative to each other, not just with priorities.
> >>> f. Mark the bug as triaged and if there are still non-triaged issues
> >>> go back to 2.a.
> >>> 3. Contributors grab the highest (relative) priority bug to fix and
> >>> get to it. Votes should also contribute to deciding which bug to fix
> >>> since that information is real-time and earlier triage information can
> >>> become more and more out of date.
> >>> I'm extremely interested in getting some feedback about these
> >>> procedures. I'm sure I missed some opportunity in this process or
> >>> overlooked something, so please let me know what you think. I hope
> >>> that by unifying these procedures across the community we can work
> >>> towards reducing that "average age" stat to below a month or two. I'm
> >>> not saying it will be easy, but I'm certain this community can do it.
On Mon, Jan 31, 2011 at 2:32 AM, mlb5000 <baker.mat...@gmail.com> wrote: > @Arnaud, I completely agree that each Hudson plugin should receive its > own "Project" in the Hudson Jira instance. It's weird that they're > split into components, but maybe there's a reason for it?
> On Jan 30, 3:49 pm, Arnaud Héritier <arnaud.herit...@exoplatform.com> > wrote: > > For that we should try to improve our Jira usage and especially using > > versions (affect vs fix for). > > To have this working we need to split the big jira into small pieces (one > > per plugin + 1 for core).
> > Cheers
> > Arnaud
> > On Sun, Jan 30, 2011 at 8:56 PM, vtintillier <
> > vtintillier.private...@gmail.com> wrote: > > > I am also really in favor of introducing a backlog with priorities. For > the > > > moment, we just have no ideas when things will be fixed (or not). The > > > Greenhoper plugin is easy to use in Jira.
> > > Vivien
> > > On Sun, Jan 30, 2011 at 8:25 PM, Christoph Kutzinski <ku...@gmx.de> > wrote:
> > >> I think this is a very nice proposal. > > >> Some comments:
> > >> In my experience most of the very old bugs fall into one of these > > >> scenarios:
> > >> - happen only under very rare circumstances. Developer cannot > reproduce it > > >> easily > > >> - are not reproducible because reporter didn't provide enough details > (and > > >> often doesn't respond to questions about details) > > >> - fix would break some other functionality and/or would mean serious > > >> re-engineering of some parts of Hudson > > >> - are related to a plugin which is not (really) maintained any more
> > >> And then there are of course the 'normal' bugs, but I think they are > the > > >> minority.
> > >> This is just my very subjective feeling why there are so many old > bugs.
> > >> What we can do easily IMHO:
> > >> - search actively for duplicates and close them
> > >> - ask reporters about more details > > >> - if bug is old and reporter doesn't respond in a timely manner, then > > >> close it as 'not reproducible'
> > >> - And what I really like to see, is a prominent message for reporters > of > > >> new bugs to provide as much details as possible ALREADY WHEN they > enter the > > >> bug initially. > > >> So much time is wasted by asking for details and waiting for the > answer. > > >> Your list in 2d is very good. I'd like to add console output and > logfile > > >> to it.
> > >> Am 30.01.2011 18:13, schrieb mlb5000:
> > >> Hi All,
> > >>> Disclaimer: I'm only just joining this community so I absolutely want > > >>> those of you with more experience developing/fixing bugs in Hudson > and > > >>> elsewhere to chime in on this conversation. What I want to do is try > > >>> to come to some consensus on how Hudson bugs are to be identified, > > >>> disambiguated, prioritized, and ultimately, scheduled (more or less) > > >>> for a release. This is part of what I hope will be a larger overall > > >>> effort to bring the Technical Debt I've observed since diving into > the > > >>> source code into check.
> > >>> As of right now there are 1853 open bugs in Hudson, of which the > > >>> average age is just under a year at 359 days. Some users, myself > > >>> included, have been getting discouraged with cycle time on bugs, > > >>> particularly when the Continuous Integration server is supposed to be > > >>> the one thing you can most rely on. When teams make their CI system > > >>> the heart and soul of their day-to-day development, testing, > > >>> configuration management, and release processes, they need to able to > > >>> rely on it. If any tool can be categorized as mission-critical, it's > > >>> the CI system.
> > >>> Here is what I am proposing:
> > >>> 1. Users submit bug reports at issues.hudson-ci.org like they always > > >>> have. > > >>> 2. At some frequency (daily?, weekly?, as they pop up?), developers > go > > >>> out and perform the following procedure: > > >>> a. Pick the oldest non-triaged bug. > > >>> b. Check whether or not the bug is a duplicate with another bug. > > >>> This will likely require aid from the more seasoned Hudson > > >>> contributors. > > >>> i. If a duplicate exists but it has been resolved, I would > > >>> recommend not reopening it, but just linking the two issues together > > >>> so the person who goes to fix the current incantation of the bug can > > >>> use it as a resource if necessary. Continue to the next step. > > >>> ii. If a duplicate exists, but is still open, link the two issues > > >>> for future reference, Close the new one as Duplicate, and redirect > all > > >>> users to the older one. Go back to 2.a. > > >>> iii. If no duplicate exists, continue. > > >>> c. Is the bug reproducible in the most recent version of Hudson? I > > >>> don't think it's worth verifying the bug in their reported version > > >>> since they'll ultimately have to upgrade to get the bug fix anyway, > so > > >>> we might as well work towards the shortest path. > > >>> i. If the bug is not reproducible in the latest version, close as > > >>> Cannot Reproduce and inform the user(s) that it appears to have been > > >>> fixed in version X, and go back to 2.a. > > >>> ii. If the bug appears to still exist in the current version, > note > > >>> which version it was verified in and continue. > > >>> d. Gather as much information as possible from users. What version > > >>> of the system is this bug occurring in? What environment are they > > >>> using? Did this error start occurring after an upgrade? What > version > > >>> did they upgrade to/from? Are they able to work around the issue or > > >>> has it stopped their usage of Hudson in its tracks? Ask any > questions > > >>> that might help isolate the commit that introduced the bug, and help > > >>> other contributors make the best possible prioritization decisions. > > >>> e. Set the priority of the issue based on some criteria. I need > > >>> some suggestions from the community about what would be the best > > >>> criteria. I would actually suggest trying to get an open source > > >>> license of GreenHopper from Atlassian. Their ranking functionality > > >>> within GreenHopper works very well for this purpose since you rank > > >>> items relative to each other, not just with priorities. > > >>> f. Mark the bug as triaged and if there are still non-triaged > issues > > >>> go back to 2.a. > > >>> 3. Contributors grab the highest (relative) priority bug to fix and > > >>> get to it. Votes should also contribute to deciding which bug to fix > > >>> since that information is real-time and earlier triage information > can > > >>> become more and more out of date.
> > >>> I'm extremely interested in getting some feedback about these > > >>> procedures. I'm sure I missed some opportunity in this process or > > >>> overlooked something, so please let me know what you think. I hope > > >>> that by unifying these procedures across the community we can work > > >>> towards reducing that "average age" stat to below a month or two. > I'm > > >>> not saying it will be easy, but I'm certain this community can do it.
@Christoph - You bring up a good point that many old bugs may need to
be handled in a more aggressive fashion. Maybe we could pick a cutoff
date and then write a comment on any of the issues that were created
before then. In it we could notify the interested users that if they
do not respond and says that the bug still affects them (and that
they're running a reasonable version of Hudson), the issue will be
closed. We wouldn't be starting from scratch (I think that would be
ill-advised), but at least we could weed out bugs affecting
uninterested users, or bugs that the reporters no longer feel the
effects of.
Thoughts?
On Jan 30, 2:25 pm, Christoph Kutzinski <ku...@gmx.de> wrote:
> I think this is a very nice proposal.
> Some comments:
> In my experience most of the very old bugs fall into one of these scenarios:
> - happen only under very rare circumstances. Developer cannot reproduce
> it easily
> - are not reproducible because reporter didn't provide enough details
> (and often doesn't respond to questions about details)
> - fix would break some other functionality and/or would mean serious
> re-engineering of some parts of Hudson
> - are related to a plugin which is not (really) maintained any more
> And then there are of course the 'normal' bugs, but I think they are the
> minority.
> This is just my very subjective feeling why there are so many old bugs.
> What we can do easily IMHO:
> - search actively for duplicates and close them
> - ask reporters about more details
> - if bug is old and reporter doesn't respond in a timely manner, then
> close it as 'not reproducible'
> - And what I really like to see, is a prominent message for reporters of
> new bugs to provide as much details as possible ALREADY WHEN they enter
> the bug initially.
> So much time is wasted by asking for details and waiting for the answer.
> Your list in 2d is very good. I'd like to add console output and logfile
> to it.
> Am 30.01.2011 18:13, schrieb mlb5000:
> > Hi All,
> > Disclaimer: I'm only just joining this community so I absolutely want
> > those of you with more experience developing/fixing bugs in Hudson and
> > elsewhere to chime in on this conversation. What I want to do is try
> > to come to some consensus on how Hudson bugs are to be identified,
> > disambiguated, prioritized, and ultimately, scheduled (more or less)
> > for a release. This is part of what I hope will be a larger overall
> > effort to bring the Technical Debt I've observed since diving into the
> > source code into check.
> > As of right now there are 1853 open bugs in Hudson, of which the
> > average age is just under a year at 359 days. Some users, myself
> > included, have been getting discouraged with cycle time on bugs,
> > particularly when the Continuous Integration server is supposed to be
> > the one thing you can most rely on. When teams make their CI system
> > the heart and soul of their day-to-day development, testing,
> > configuration management, and release processes, they need to able to
> > rely on it. If any tool can be categorized as mission-critical, it's
> > the CI system.
> > Here is what I am proposing:
> > 1. Users submit bug reports at issues.hudson-ci.org like they always
> > have.
> > 2. At some frequency (daily?, weekly?, as they pop up?), developers go
> > out and perform the following procedure:
> > a. Pick the oldest non-triaged bug.
> > b. Check whether or not the bug is a duplicate with another bug.
> > This will likely require aid from the more seasoned Hudson
> > contributors.
> > i. If a duplicate exists but it has been resolved, I would
> > recommend not reopening it, but just linking the two issues together
> > so the person who goes to fix the current incantation of the bug can
> > use it as a resource if necessary. Continue to the next step.
> > ii. If a duplicate exists, but is still open, link the two issues
> > for future reference, Close the new one as Duplicate, and redirect all
> > users to the older one. Go back to 2.a.
> > iii. If no duplicate exists, continue.
> > c. Is the bug reproducible in the most recent version of Hudson? I
> > don't think it's worth verifying the bug in their reported version
> > since they'll ultimately have to upgrade to get the bug fix anyway, so
> > we might as well work towards the shortest path.
> > i. If the bug is not reproducible in the latest version, close as
> > Cannot Reproduce and inform the user(s) that it appears to have been
> > fixed in version X, and go back to 2.a.
> > ii. If the bug appears to still exist in the current version, note
> > which version it was verified in and continue.
> > d. Gather as much information as possible from users. What version
> > of the system is this bug occurring in? What environment are they
> > using? Did this error start occurring after an upgrade? What version
> > did they upgrade to/from? Are they able to work around the issue or
> > has it stopped their usage of Hudson in its tracks? Ask any questions
> > that might help isolate the commit that introduced the bug, and help
> > other contributors make the best possible prioritization decisions.
> > e. Set the priority of the issue based on some criteria. I need
> > some suggestions from the community about what would be the best
> > criteria. I would actually suggest trying to get an open source
> > license of GreenHopper from Atlassian. Their ranking functionality
> > within GreenHopper works very well for this purpose since you rank
> > items relative to each other, not just with priorities.
> > f. Mark the bug as triaged and if there are still non-triaged issues
> > go back to 2.a.
> > 3. Contributors grab the highest (relative) priority bug to fix and
> > get to it. Votes should also contribute to deciding which bug to fix
> > since that information is real-time and earlier triage information can
> > become more and more out of date.
> > I'm extremely interested in getting some feedback about these
> > procedures. I'm sure I missed some opportunity in this process or
> > overlooked something, so please let me know what you think. I hope
> > that by unifying these procedures across the community we can work
> > towards reducing that "average age" stat to below a month or two. I'm
> > not saying it will be easy, but I'm certain this community can do it.
@Arnaud - I suppose, but being a Jira administrator in a multi-project
environment myself, I don't think it's that bad :). Either way, if
they went this direction each plugin would need a named maintainer to
be the "Project Lead" on that project.
On Jan 30, 8:36 pm, Arnaud Héritier <arnaud.herit...@exoplatform.com>
wrote:
> It is historical
> less things to administrate and manage ...
> It is easier to automate the add of a component than creating a new project.
> On Mon, Jan 31, 2011 at 2:32 AM, mlb5000 <baker.mat...@gmail.com> wrote:
> > @Arnaud, I completely agree that each Hudson plugin should receive its
> > own "Project" in the Hudson Jira instance. It's weird that they're
> > split into components, but maybe there's a reason for it?
> > On Jan 30, 3:49 pm, Arnaud Héritier <arnaud.herit...@exoplatform.com>
> > wrote:
> > > For that we should try to improve our Jira usage and especially using
> > > versions (affect vs fix for).
> > > To have this working we need to split the big jira into small pieces (one
> > > per plugin + 1 for core).
> > > Cheers
> > > Arnaud
> > > On Sun, Jan 30, 2011 at 8:56 PM, vtintillier <
> > > vtintillier.private...@gmail.com> wrote:
> > > > I am also really in favor of introducing a backlog with priorities. For
> > the
> > > > moment, we just have no ideas when things will be fixed (or not). The
> > > > Greenhoper plugin is easy to use in Jira.
> > > > Vivien
> > > > On Sun, Jan 30, 2011 at 8:25 PM, Christoph Kutzinski <ku...@gmx.de>
> > wrote:
> > > >> I think this is a very nice proposal.
> > > >> Some comments:
> > > >> In my experience most of the very old bugs fall into one of these
> > > >> scenarios:
> > > >> - happen only under very rare circumstances. Developer cannot
> > reproduce it
> > > >> easily
> > > >> - are not reproducible because reporter didn't provide enough details
> > (and
> > > >> often doesn't respond to questions about details)
> > > >> - fix would break some other functionality and/or would mean serious
> > > >> re-engineering of some parts of Hudson
> > > >> - are related to a plugin which is not (really) maintained any more
> > > >> And then there are of course the 'normal' bugs, but I think they are
> > the
> > > >> minority.
> > > >> This is just my very subjective feeling why there are so many old
> > bugs.
> > > >> What we can do easily IMHO:
> > > >> - search actively for duplicates and close them
> > > >> - ask reporters about more details
> > > >> - if bug is old and reporter doesn't respond in a timely manner, then
> > > >> close it as 'not reproducible'
> > > >> - And what I really like to see, is a prominent message for reporters
> > of
> > > >> new bugs to provide as much details as possible ALREADY WHEN they
> > enter the
> > > >> bug initially.
> > > >> So much time is wasted by asking for details and waiting for the
> > answer.
> > > >> Your list in 2d is very good. I'd like to add console output and
> > logfile
> > > >> to it.
> > > >> Am 30.01.2011 18:13, schrieb mlb5000:
> > > >> Hi All,
> > > >>> Disclaimer: I'm only just joining this community so I absolutely want
> > > >>> those of you with more experience developing/fixing bugs in Hudson
> > and
> > > >>> elsewhere to chime in on this conversation. What I want to do is try
> > > >>> to come to some consensus on how Hudson bugs are to be identified,
> > > >>> disambiguated, prioritized, and ultimately, scheduled (more or less)
> > > >>> for a release. This is part of what I hope will be a larger overall
> > > >>> effort to bring the Technical Debt I've observed since diving into
> > the
> > > >>> source code into check.
> > > >>> As of right now there are 1853 open bugs in Hudson, of which the
> > > >>> average age is just under a year at 359 days. Some users, myself
> > > >>> included, have been getting discouraged with cycle time on bugs,
> > > >>> particularly when the Continuous Integration server is supposed to be
> > > >>> the one thing you can most rely on. When teams make their CI system
> > > >>> the heart and soul of their day-to-day development, testing,
> > > >>> configuration management, and release processes, they need to able to
> > > >>> rely on it. If any tool can be categorized as mission-critical, it's
> > > >>> the CI system.
> > > >>> Here is what I am proposing:
> > > >>> 1. Users submit bug reports at issues.hudson-ci.org like they always
> > > >>> have.
> > > >>> 2. At some frequency (daily?, weekly?, as they pop up?), developers
> > go
> > > >>> out and perform the following procedure:
> > > >>> a. Pick the oldest non-triaged bug.
> > > >>> b. Check whether or not the bug is a duplicate with another bug.
> > > >>> This will likely require aid from the more seasoned Hudson
> > > >>> contributors.
> > > >>> i. If a duplicate exists but it has been resolved, I would
> > > >>> recommend not reopening it, but just linking the two issues together
> > > >>> so the person who goes to fix the current incantation of the bug can
> > > >>> use it as a resource if necessary. Continue to the next step.
> > > >>> ii. If a duplicate exists, but is still open, link the two issues
> > > >>> for future reference, Close the new one as Duplicate, and redirect
> > all
> > > >>> users to the older one. Go back to 2.a.
> > > >>> iii. If no duplicate exists, continue.
> > > >>> c. Is the bug reproducible in the most recent version of Hudson? I
> > > >>> don't think it's worth verifying the bug in their reported version
> > > >>> since they'll ultimately have to upgrade to get the bug fix anyway,
> > so
> > > >>> we might as well work towards the shortest path.
> > > >>> i. If the bug is not reproducible in the latest version, close as
> > > >>> Cannot Reproduce and inform the user(s) that it appears to have been
> > > >>> fixed in version X, and go back to 2.a.
> > > >>> ii. If the bug appears to still exist in the current version,
> > note
> > > >>> which version it was verified in and continue.
> > > >>> d. Gather as much information as possible from users. What version
> > > >>> of the system is this bug occurring in? What environment are they
> > > >>> using? Did this error start occurring after an upgrade? What
> > version
> > > >>> did they upgrade to/from? Are they able to work around the issue or
> > > >>> has it stopped their usage of Hudson in its tracks? Ask any
> > questions
> > > >>> that might help isolate the commit that introduced the bug, and help
> > > >>> other contributors make the best possible prioritization decisions.
> > > >>> e. Set the priority of the issue based on some criteria. I need
> > > >>> some suggestions from the community about what would be the best
> > > >>> criteria. I would actually suggest trying to get an open source
> > > >>> license of GreenHopper from Atlassian. Their ranking functionality
> > > >>> within GreenHopper works very well for this purpose since you rank
> > > >>> items relative to each other, not just with priorities.
> > > >>> f. Mark the bug as triaged and if there are still non-triaged
> > issues
> > > >>> go back to 2.a.
> > > >>> 3. Contributors grab the highest (relative) priority bug to fix and
> > > >>> get to it. Votes should also contribute to deciding which bug to fix
> > > >>> since that information is real-time and earlier triage information
> > can
> > > >>> become more and more out of date.
> > > >>> I'm extremely interested in getting some feedback about these
> > > >>> procedures. I'm sure I missed some opportunity in this process or
> > > >>> overlooked something, so please let me know what you think. I hope
> > > >>> that by unifying these procedures across the community we can work
> > > >>> towards reducing that "average age" stat to below a month or two.
> > I'm
> > > >>> not saying it will be easy, but I'm certain this community can do it.
On Mon, Jan 31, 2011 at 2:40 AM, mlb5000 <baker.mat...@gmail.com> wrote: > @Arnaud - I suppose, but being a Jira administrator in a multi-project > environment myself, I don't think it's that bad :). Either way, if > they went this direction each plugin would need a named maintainer to > be the "Project Lead" on that project.
> On Jan 30, 8:36 pm, Arnaud Héritier <arnaud.herit...@exoplatform.com> > wrote: > > It is historical > > less things to administrate and manage ... > > It is easier to automate the add of a component than creating a new > project.
> > On Mon, Jan 31, 2011 at 2:32 AM, mlb5000 <baker.mat...@gmail.com> wrote: > > > @Arnaud, I completely agree that each Hudson plugin should receive its > > > own "Project" in the Hudson Jira instance. It's weird that they're > > > split into components, but maybe there's a reason for it?
> > > On Jan 30, 3:49 pm, Arnaud Héritier <arnaud.herit...@exoplatform.com> > > > wrote: > > > > For that we should try to improve our Jira usage and especially using > > > > versions (affect vs fix for). > > > > To have this working we need to split the big jira into small pieces > (one > > > > per plugin + 1 for core).
> > > > Cheers
> > > > Arnaud
> > > > On Sun, Jan 30, 2011 at 8:56 PM, vtintillier <
> > > > vtintillier.private...@gmail.com> wrote: > > > > > I am also really in favor of introducing a backlog with priorities. > For > > > the > > > > > moment, we just have no ideas when things will be fixed (or not). > The > > > > > Greenhoper plugin is easy to use in Jira.
> > > > > Vivien
> > > > > On Sun, Jan 30, 2011 at 8:25 PM, Christoph Kutzinski <ku...@gmx.de
> > > wrote:
> > > > >> I think this is a very nice proposal. > > > > >> Some comments:
> > > > >> In my experience most of the very old bugs fall into one of these > > > > >> scenarios:
> > > > >> - happen only under very rare circumstances. Developer cannot > > > reproduce it > > > > >> easily > > > > >> - are not reproducible because reporter didn't provide enough > details > > > (and > > > > >> often doesn't respond to questions about details) > > > > >> - fix would break some other functionality and/or would mean > serious > > > > >> re-engineering of some parts of Hudson > > > > >> - are related to a plugin which is not (really) maintained any > more
> > > > >> And then there are of course the 'normal' bugs, but I think they > are > > > the > > > > >> minority.
> > > > >> This is just my very subjective feeling why there are so many old > > > bugs.
> > > > >> What we can do easily IMHO:
> > > > >> - search actively for duplicates and close them
> > > > >> - ask reporters about more details > > > > >> - if bug is old and reporter doesn't respond in a timely manner, > then > > > > >> close it as 'not reproducible'
> > > > >> - And what I really like to see, is a prominent message for > reporters > > > of > > > > >> new bugs to provide as much details as possible ALREADY WHEN they > > > enter the > > > > >> bug initially. > > > > >> So much time is wasted by asking for details and waiting for the > > > answer. > > > > >> Your list in 2d is very good. I'd like to add console output and > > > logfile > > > > >> to it.
> > > > >> Am 30.01.2011 18:13, schrieb mlb5000:
> > > > >> Hi All,
> > > > >>> Disclaimer: I'm only just joining this community so I absolutely > want > > > > >>> those of you with more experience developing/fixing bugs in > Hudson > > > and > > > > >>> elsewhere to chime in on this conversation. What I want to do is > try > > > > >>> to come to some consensus on how Hudson bugs are to be > identified, > > > > >>> disambiguated, prioritized, and ultimately, scheduled (more or > less) > > > > >>> for a release. This is part of what I hope will be a larger > overall > > > > >>> effort to bring the Technical Debt I've observed since diving > into > > > the > > > > >>> source code into check.
> > > > >>> As of right now there are 1853 open bugs in Hudson, of which the > > > > >>> average age is just under a year at 359 days. Some users, myself > > > > >>> included, have been getting discouraged with cycle time on bugs, > > > > >>> particularly when the Continuous Integration server is supposed > to be > > > > >>> the one thing you can most rely on. When teams make their CI > system > > > > >>> the heart and soul of their day-to-day development, testing, > > > > >>> configuration management, and release processes, they need to > able to > > > > >>> rely on it. If any tool can be categorized as mission-critical, > it's > > > > >>> the CI system.
> > > > >>> Here is what I am proposing:
> > > > >>> 1. Users submit bug reports at issues.hudson-ci.org like they > always > > > > >>> have. > > > > >>> 2. At some frequency (daily?, weekly?, as they pop up?), > developers > > > go > > > > >>> out and perform the following procedure: > > > > >>> a. Pick the oldest non-triaged bug. > > > > >>> b. Check whether or not the bug is a duplicate with another > bug. > > > > >>> This will likely require aid from the more seasoned Hudson > > > > >>> contributors. > > > > >>> i. If a duplicate exists but it has been resolved, I would > > > > >>> recommend not reopening it, but just linking the two issues > together > > > > >>> so the person who goes to fix the current incantation of the bug > can > > > > >>> use it as a resource if necessary. Continue to the next step. > > > > >>> ii. If a duplicate exists, but is still open, link the two > issues > > > > >>> for future reference, Close the new one as Duplicate, and > redirect > > > all > > > > >>> users to the older one. Go back to 2.a. > > > > >>> iii. If no duplicate exists, continue. > > > > >>> c. Is the bug reproducible in the most recent version of > Hudson? I > > > > >>> don't think it's worth verifying the bug in their reported > version > > > > >>> since they'll ultimately have to upgrade to get the bug fix > anyway, > > > so > > > > >>> we might as well work towards the shortest path. > > > > >>> i. If the bug is not reproducible in the latest version, > close as > > > > >>> Cannot Reproduce and inform the user(s) that it appears to have > been > > > > >>> fixed in version X, and go back to 2.a. > > > > >>> ii. If the bug appears to still exist in the current version, > > > note > > > > >>> which version it was verified in and continue. > > > > >>> d. Gather as much information as possible from users. What > version > > > > >>> of the system is this bug occurring in? What environment are > they > > > > >>> using? Did this error start occurring after an upgrade? What > > > version > > > > >>> did they upgrade to/from? Are they able to work around the issue > or > > > > >>> has it stopped their usage of Hudson in its tracks? Ask any > > > questions > > > > >>> that might help isolate the commit that introduced the bug, and > help > > > > >>> other contributors make the best possible prioritization > decisions. > > > > >>> e. Set the priority of the issue based on some criteria. I > need > > > > >>> some suggestions from the community about what would be the best > > > > >>> criteria. I would actually suggest trying to get an open source > > > > >>> license of GreenHopper from Atlassian. Their ranking > functionality > > > > >>> within GreenHopper works very well for this purpose since you > rank > > > > >>> items relative to each other, not just with priorities. > > > > >>> f. Mark the bug as triaged and if there are still non-triaged > > > issues > > > > >>> go back to 2.a. > > > > >>> 3. Contributors grab the highest (relative) priority bug to fix > and > > > > >>> get to it. Votes should also contribute to deciding which bug to > fix > > > > >>> since that information is real-time and earlier triage > information > > > can > > > > >>> become more and more out of date.
> > > > >>> I'm extremely interested in getting some feedback about these > > > > >>> procedures. I'm sure I missed some opportunity in this process > or > > > > >>> overlooked something, so please let me know what you think. I > hope > > > > >>> that by unifying these procedures across the community we can > work > > > > >>> towards reducing that "average age" stat to below a month or two. > > > I'm > > > > >>> not saying it will be easy, but I'm certain this community can do > it.
Things haven't completely calmed down since the rename yet, but I'd
like to get started closing out some of these old issues if at all
possible. I noticed there was a "!CLOSED && STALE" query already on
issues.jenkins-ci.org and I was wondering how we want to handle Bug
reports that fall into this category. An issue is considered STALE if
it hasn't been updated within the last year. I'd say that's a pretty
good criteria for just calling it and closing the issue without
discretion, but I don't personally have any insight into what some of
those older bug reports might be, and the last thing I want is to
upset the users.
At any rate, does anyone have any (further) suggestions for how to
handle these older bugs reports without having to go through them one
by one? Just to provide some perspective, 494 (http://issues.jenkins- ci.org/secure/IssueNavigator.jspa?mode=hide&requestId=10243) of the
1862 currently unresolved bug reports (http://issues.jenkins-ci.org/ secure/IssueNavigator.jspa?mode=hide&requestId=10244) are considered
stale.
Also, would someone who has administrative privileges in the Jenkins
Jira instance be able to add a Boolean field to the Bug issue type for
marking whether or not a bug report has been Triaged, or does anyone
have another way to keep track of whether a bug report has been
vetted?
On Jan 30, 12:13 pm, mlb5000 <baker.mat...@gmail.com> wrote:
> Disclaimer: I'm only just joining this community so I absolutely want
> those of you with more experience developing/fixing bugs in Hudson and
> elsewhere to chime in on this conversation. What I want to do is try
> to come to some consensus on how Hudson bugs are to be identified,
> disambiguated, prioritized, and ultimately, scheduled (more or less)
> for a release. This is part of what I hope will be a larger overall
> effort to bring the Technical Debt I've observed since diving into the
> source code into check.
> As of right now there are 1853 open bugs in Hudson, of which the
> average age is just under a year at 359 days. Some users, myself
> included, have been getting discouraged with cycle time on bugs,
> particularly when the Continuous Integration server is supposed to be
> the one thing you can most rely on. When teams make their CI system
> the heart and soul of their day-to-day development, testing,
> configuration management, and release processes, they need to able to
> rely on it. If any tool can be categorized as mission-critical, it's
> the CI system.
> Here is what I am proposing:
> 1. Users submit bug reports at issues.hudson-ci.org like they always
> have.
> 2. At some frequency (daily?, weekly?, as they pop up?), developers go
> out and perform the following procedure:
> a. Pick the oldest non-triaged bug.
> b. Check whether or not the bug is a duplicate with another bug.
> This will likely require aid from the more seasoned Hudson
> contributors.
> i. If a duplicate exists but it has been resolved, I would
> recommend not reopening it, but just linking the two issues together
> so the person who goes to fix the current incantation of the bug can
> use it as a resource if necessary. Continue to the next step.
> ii. If a duplicate exists, but is still open, link the two issues
> for future reference, Close the new one as Duplicate, and redirect all
> users to the older one. Go back to 2.a.
> iii. If no duplicate exists, continue.
> c. Is the bug reproducible in the most recent version of Hudson? I
> don't think it's worth verifying the bug in their reported version
> since they'll ultimately have to upgrade to get the bug fix anyway, so
> we might as well work towards the shortest path.
> i. If the bug is not reproducible in the latest version, close as
> Cannot Reproduce and inform the user(s) that it appears to have been
> fixed in version X, and go back to 2.a.
> ii. If the bug appears to still exist in the current version, note
> which version it was verified in and continue.
> d. Gather as much information as possible from users. What version
> of the system is this bug occurring in? What environment are they
> using? Did this error start occurring after an upgrade? What version
> did they upgrade to/from? Are they able to work around the issue or
> has it stopped their usage of Hudson in its tracks? Ask any questions
> that might help isolate the commit that introduced the bug, and help
> other contributors make the best possible prioritization decisions.
> e. Set the priority of the issue based on some criteria. I need
> some suggestions from the community about what would be the best
> criteria. I would actually suggest trying to get an open source
> license of GreenHopper from Atlassian. Their ranking functionality
> within GreenHopper works very well for this purpose since you rank
> items relative to each other, not just with priorities.
> f. Mark the bug as triaged and if there are still non-triaged issues
> go back to 2.a.
> 3. Contributors grab the highest (relative) priority bug to fix and
> get to it. Votes should also contribute to deciding which bug to fix
> since that information is real-time and earlier triage information can
> become more and more out of date.
> I'm extremely interested in getting some feedback about these
> procedures. I'm sure I missed some opportunity in this process or
> overlooked something, so please let me know what you think. I hope
> that by unifying these procedures across the community we can work
> towards reducing that "average age" stat to below a month or two. I'm
> not saying it will be easy, but I'm certain this community can do it.
I fully agree with you that obsolete bugs should be closed, in order
for most users to know what are really the current issues and in order
for most developers to know what has to be fixed.
But I think that the 494 stale bugs can't be closed like that.
For example, a large number of issues were written by Kohsuke for
things to check or for things to fix (for example issue 5036).
A bug having a low priority for one year probably doesn't mean that it
doesn't need to be fixed.
I will try to do that myself (instead of sleeping) and as Christoph
said:
- search actively for duplicates and close them [we may also read the
changelogs of releases]
- ask reporters about more details [if needed]
- if bug is old and reporter doesn't respond in a timely manner,
then close it as 'not reproducible'
Matt: Because you are already a user, please choose some issues to be
tested and write your test results in the issues : it will save most
of the time needed by the developers to know if the stale issues can
be closed ! (and it may help you later to submit patchs)
bye,
Emeric
On 7 fév, 00:29, mlb5000 <baker.mat...@gmail.com> wrote:
> Things haven't completely calmed down since the rename yet, but I'd
> like to get started closing out some of these old issues if at all
> possible. I noticed there was a "!CLOSED && STALE" query already on
> issues.jenkins-ci.org and I was wondering how we want to handle Bug
> reports that fall into this category. An issue is considered STALE if
> it hasn't been updated within the last year. I'd say that's a pretty
> good criteria for just calling it and closing the issue without
> discretion, but I don't personally have any insight into what some of
> those older bug reports might be, and the last thing I want is to
> upset the users.
> At any rate, does anyone have any (further) suggestions for how to
> handle these older bugs reports without having to go through them one
> by one? Just to provide some perspective, 494 (http://issues.jenkins- > ci.org/secure/IssueNavigator.jspa?mode=hide&requestId=10243) of the
> 1862 currently unresolved bug reports (http://issues.jenkins-ci.org/ > secure/IssueNavigator.jspa?mode=hide&requestId=10244) are considered
> stale.
> Also, would someone who has administrative privileges in the Jenkins
> Jira instance be able to add a Boolean field to the Bug issue type for
> marking whether or not a bug report has been Triaged, or does anyone
> have another way to keep track of whether a bug report has been
> vetted?
> On Jan 30, 12:13 pm, mlb5000 <baker.mat...@gmail.com> wrote:
> > Hi All,
> > Disclaimer: I'm only just joining this community so I absolutely want
> > those of you with more experience developing/fixing bugs in Hudson and
> > elsewhere to chime in on this conversation. What I want to do is try
> > to come to some consensus on how Hudson bugs are to be identified,
> > disambiguated, prioritized, and ultimately, scheduled (more or less)
> > for a release. This is part of what I hope will be a larger overall
> > effort to bring the Technical Debt I've observed since diving into the
> > source code into check.
> > As of right now there are 1853 open bugs in Hudson, of which the
> > average age is just under a year at 359 days. Some users, myself
> > included, have been getting discouraged with cycle time on bugs,
> > particularly when the Continuous Integration server is supposed to be
> > the one thing you can most rely on. When teams make their CI system
> > the heart and soul of their day-to-day development, testing,
> > configuration management, and release processes, they need to able to
> > rely on it. If any tool can be categorized as mission-critical, it's
> > the CI system.
> > Here is what I am proposing:
> > 1. Users submit bug reports at issues.hudson-ci.org like they always
> > have.
> > 2. At some frequency (daily?, weekly?, as they pop up?), developers go
> > out and perform the following procedure:
> > a. Pick the oldest non-triaged bug.
> > b. Check whether or not the bug is a duplicate with another bug.
> > This will likely require aid from the more seasoned Hudson
> > contributors.
> > i. If a duplicate exists but it has been resolved, I would
> > recommend not reopening it, but just linking the two issues together
> > so the person who goes to fix the current incantation of the bug can
> > use it as a resource if necessary. Continue to the next step.
> > ii. If a duplicate exists, but is still open, link the two issues
> > for future reference, Close the new one as Duplicate, and redirect all
> > users to the older one. Go back to 2.a.
> > iii. If no duplicate exists, continue.
> > c. Is the bug reproducible in the most recent version of Hudson? I
> > don't think it's worth verifying the bug in their reported version
> > since they'll ultimately have to upgrade to get the bug fix anyway, so
> > we might as well work towards the shortest path.
> > i. If the bug is not reproducible in the latest version, close as
> > Cannot Reproduce and inform the user(s) that it appears to have been
> > fixed in version X, and go back to 2.a.
> > ii. If the bug appears to still exist in the current version, note
> > which version it was verified in and continue.
> > d. Gather as much information as possible from users. What version
> > of the system is this bug occurring in? What environment are they
> > using? Did this error start occurring after an upgrade? What version
> > did they upgrade to/from? Are they able to work around the issue or
> > has it stopped their usage of Hudson in its tracks? Ask any questions
> > that might help isolate the commit that introduced the bug, and help
> > other contributors make the best possible prioritization decisions.
> > e. Set the priority of the issue based on some criteria. I need
> > some suggestions from the community about what would be the best
> > criteria. I would actually suggest trying to get an open source
> > license of GreenHopper from Atlassian. Their ranking functionality
> > within GreenHopper works very well for this purpose since you rank
> > items relative to each other, not just with priorities.
> > f. Mark the bug as triaged and if there are still non-triaged issues
> > go back to 2.a.
> > 3. Contributors grab the highest (relative) priority bug to fix and
> > get to it. Votes should also contribute to deciding which bug to fix
> > since that information is real-time and earlier triage information can
> > become more and more out of date.
> > I'm extremely interested in getting some feedback about these
> > procedures. I'm sure I missed some opportunity in this process or
> > overlooked something, so please let me know what you think. I hope
> > that by unifying these procedures across the community we can work
> > towards reducing that "average age" stat to below a month or two. I'm
> > not saying it will be easy, but I'm certain this community can do it.
On Mon, Feb 7, 2011 at 2:27 AM, evernat <ever...@free.fr> wrote: > Hi Matt,
> I fully agree with you that obsolete bugs should be closed, in order > for most users to know what are really the current issues and in order > for most developers to know what has to be fixed. > But I think that the 494 stale bugs can't be closed like that. > For example, a large number of issues were written by Kohsuke for > things to check or for things to fix (for example issue 5036). > A bug having a low priority for one year probably doesn't mean that it > doesn't need to be fixed.
> I currently suggest that some, if not all, developers read each one of > the stale bugs in
> I will try to do that myself (instead of sleeping) and as Christoph > said: > - search actively for duplicates and close them [we may also read the > changelogs of releases]
We could ask to have a free version of suggestimate to find duplicates. It helps also users to not open issues which are already existing. http://www.suggestimate.com/?r=view
> - ask reporters about more details [if needed] > - if bug is old and reporter doesn't respond in a timely manner, > then close it as 'not reproducible'
> Matt: Because you are already a user, please choose some issues to be > tested and write your test results in the issues : it will save most > of the time needed by the developers to know if the stale issues can > be closed ! (and it may help you later to submit patchs)
> bye, > Emeric
> On 7 fév, 00:29, mlb5000 <baker.mat...@gmail.com> wrote: > > Things haven't completely calmed down since the rename yet, but I'd > > like to get started closing out some of these old issues if at all > > possible. I noticed there was a "!CLOSED && STALE" query already on > > issues.jenkins-ci.org and I was wondering how we want to handle Bug > > reports that fall into this category. An issue is considered STALE if > > it hasn't been updated within the last year. I'd say that's a pretty > > good criteria for just calling it and closing the issue without > > discretion, but I don't personally have any insight into what some of > > those older bug reports might be, and the last thing I want is to > > upset the users.
> > At any rate, does anyone have any (further) suggestions for how to > > handle these older bugs reports without having to go through them one > > by one? Just to provide some perspective, 494 (http://issues.jenkins- > > ci.org/secure/IssueNavigator.jspa?mode=hide&requestId=10243) of the > > 1862 currently unresolved bug reports (http://issues.jenkins-ci.org/ > > secure/IssueNavigator.jspa?mode=hide&requestId=10244) are considered > > stale.
> > Also, would someone who has administrative privileges in the Jenkins > > Jira instance be able to add a Boolean field to the Bug issue type for > > marking whether or not a bug report has been Triaged, or does anyone > > have another way to keep track of whether a bug report has been > > vetted?
> > On Jan 30, 12:13 pm, mlb5000 <baker.mat...@gmail.com> wrote:
> > > Hi All,
> > > Disclaimer: I'm only just joining this community so I absolutely want > > > those of you with more experience developing/fixing bugs in Hudson and > > > elsewhere to chime in on this conversation. What I want to do is try > > > to come to some consensus on how Hudson bugs are to be identified, > > > disambiguated, prioritized, and ultimately, scheduled (more or less) > > > for a release. This is part of what I hope will be a larger overall > > > effort to bring the Technical Debt I've observed since diving into the > > > source code into check.
> > > As of right now there are 1853 open bugs in Hudson, of which the > > > average age is just under a year at 359 days. Some users, myself > > > included, have been getting discouraged with cycle time on bugs, > > > particularly when the Continuous Integration server is supposed to be > > > the one thing you can most rely on. When teams make their CI system > > > the heart and soul of their day-to-day development, testing, > > > configuration management, and release processes, they need to able to > > > rely on it. If any tool can be categorized as mission-critical, it's > > > the CI system.
> > > Here is what I am proposing:
> > > 1. Users submit bug reports at issues.hudson-ci.org like they always > > > have. > > > 2. At some frequency (daily?, weekly?, as they pop up?), developers go > > > out and perform the following procedure: > > > a. Pick the oldest non-triaged bug. > > > b. Check whether or not the bug is a duplicate with another bug. > > > This will likely require aid from the more seasoned Hudson > > > contributors. > > > i. If a duplicate exists but it has been resolved, I would > > > recommend not reopening it, but just linking the two issues together > > > so the person who goes to fix the current incantation of the bug can > > > use it as a resource if necessary. Continue to the next step. > > > ii. If a duplicate exists, but is still open, link the two issues > > > for future reference, Close the new one as Duplicate, and redirect all > > > users to the older one. Go back to 2.a. > > > iii. If no duplicate exists, continue. > > > c. Is the bug reproducible in the most recent version of Hudson? I > > > don't think it's worth verifying the bug in their reported version > > > since they'll ultimately have to upgrade to get the bug fix anyway, so > > > we might as well work towards the shortest path. > > > i. If the bug is not reproducible in the latest version, close as > > > Cannot Reproduce and inform the user(s) that it appears to have been > > > fixed in version X, and go back to 2.a. > > > ii. If the bug appears to still exist in the current version, note > > > which version it was verified in and continue. > > > d. Gather as much information as possible from users. What version > > > of the system is this bug occurring in? What environment are they > > > using? Did this error start occurring after an upgrade? What version > > > did they upgrade to/from? Are they able to work around the issue or > > > has it stopped their usage of Hudson in its tracks? Ask any questions > > > that might help isolate the commit that introduced the bug, and help > > > other contributors make the best possible prioritization decisions. > > > e. Set the priority of the issue based on some criteria. I need > > > some suggestions from the community about what would be the best > > > criteria. I would actually suggest trying to get an open source > > > license of GreenHopper from Atlassian. Their ranking functionality > > > within GreenHopper works very well for this purpose since you rank > > > items relative to each other, not just with priorities. > > > f. Mark the bug as triaged and if there are still non-triaged issues > > > go back to 2.a. > > > 3. Contributors grab the highest (relative) priority bug to fix and > > > get to it. Votes should also contribute to deciding which bug to fix > > > since that information is real-time and earlier triage information can > > > become more and more out of date.
> > > I'm extremely interested in getting some feedback about these > > > procedures. I'm sure I missed some opportunity in this process or > > > overlooked something, so please let me know what you think. I hope > > > that by unifying these procedures across the community we can work > > > towards reducing that "average age" stat to below a month or two. I'm > > > not saying it will be easy, but I'm certain this community can do it.
> On Mon, Feb 7, 2011 at 2:27 AM, evernat <ever...@free.fr> wrote:
>> Hi Matt,
>> I fully agree with you that obsolete bugs should be closed, in order >> for most users to know what are really the current issues and in order >> for most developers to know what has to be fixed. >> But I think that the 494 stale bugs can't be closed like that. >> For example, a large number of issues were written by Kohsuke for >> things to check or for things to fix (for example issue 5036). >> A bug having a low priority for one year probably doesn't mean that it >> doesn't need to be fixed.
>> I currently suggest that some, if not all, developers read each one of >> the stale bugs in
>> I will try to do that myself (instead of sleeping) and as Christoph >> said: >> - search actively for duplicates and close them [we may also read the >> changelogs of releases]
> We could ask to have a free version of suggestimate to find duplicates. > It helps also users to not open issues which are already existing. > http://www.suggestimate.com/?r=view
> WDYT ?
>> - ask reporters about more details [if needed] >> - if bug is old and reporter doesn't respond in a timely manner, >> then close it as 'not reproducible'
>> Matt: Because you are already a user, please choose some issues to be >> tested and write your test results in the issues : it will save most >> of the time needed by the developers to know if the stale issues can >> be closed ! (and it may help you later to submit patchs)
>> bye, >> Emeric
>> On 7 fév, 00:29, mlb5000 <baker.mat...@gmail.com> wrote: >> > Things haven't completely calmed down since the rename yet, but I'd >> > like to get started closing out some of these old issues if at all >> > possible. I noticed there was a "!CLOSED && STALE" query already on >> > issues.jenkins-ci.org and I was wondering how we want to handle Bug >> > reports that fall into this category. An issue is considered STALE if >> > it hasn't been updated within the last year. I'd say that's a pretty >> > good criteria for just calling it and closing the issue without >> > discretion, but I don't personally have any insight into what some of >> > those older bug reports might be, and the last thing I want is to >> > upset the users.
>> > At any rate, does anyone have any (further) suggestions for how to >> > handle these older bugs reports without having to go through them one >> > by one? Just to provide some perspective, 494 (http://issues.jenkins- >> > ci.org/secure/IssueNavigator.jspa?mode=hide&requestId=10243) of the >> > 1862 currently unresolved bug reports (http://issues.jenkins-ci.org/ >> > secure/IssueNavigator.jspa?mode=hide&requestId=10244) are considered >> > stale.
>> > Also, would someone who has administrative privileges in the Jenkins >> > Jira instance be able to add a Boolean field to the Bug issue type for >> > marking whether or not a bug report has been Triaged, or does anyone >> > have another way to keep track of whether a bug report has been >> > vetted?
>> > On Jan 30, 12:13 pm, mlb5000 <baker.mat...@gmail.com> wrote:
>> > > Hi All,
>> > > Disclaimer: I'm only just joining this community so I absolutely want >> > > those of you with more experience developing/fixing bugs in Hudson and >> > > elsewhere to chime in on this conversation. What I want to do is try >> > > to come to some consensus on how Hudson bugs are to be identified, >> > > disambiguated, prioritized, and ultimately, scheduled (more or less) >> > > for a release. This is part of what I hope will be a larger overall >> > > effort to bring the Technical Debt I've observed since diving into the >> > > source code into check.
>> > > As of right now there are 1853 open bugs in Hudson, of which the >> > > average age is just under a year at 359 days. Some users, myself >> > > included, have been getting discouraged with cycle time on bugs, >> > > particularly when the Continuous Integration server is supposed to be >> > > the one thing you can most rely on. When teams make their CI system >> > > the heart and soul of their day-to-day development, testing, >> > > configuration management, and release processes, they need to able to >> > > rely on it. If any tool can be categorized as mission-critical, it's >> > > the CI system.
>> > > Here is what I am proposing:
>> > > 1. Users submit bug reports at issues.hudson-ci.org like they always >> > > have. >> > > 2. At some frequency (daily?, weekly?, as they pop up?), developers go >> > > out and perform the following procedure: >> > > a. Pick the oldest non-triaged bug. >> > > b. Check whether or not the bug is a duplicate with another bug. >> > > This will likely require aid from the more seasoned Hudson >> > > contributors. >> > > i. If a duplicate exists but it has been resolved, I would >> > > recommend not reopening it, but just linking the two issues together >> > > so the person who goes to fix the current incantation of the bug can >> > > use it as a resource if necessary. Continue to the next step. >> > > ii. If a duplicate exists, but is still open, link the two issues >> > > for future reference, Close the new one as Duplicate, and redirect all >> > > users to the older one. Go back to 2.a. >> > > iii. If no duplicate exists, continue. >> > > c. Is the bug reproducible in the most recent version of Hudson? I >> > > don't think it's worth verifying the bug in their reported version >> > > since they'll ultimately have to upgrade to get the bug fix anyway, so >> > > we might as well work towards the shortest path. >> > > i. If the bug is not reproducible in the latest version, close as >> > > Cannot Reproduce and inform the user(s) that it appears to have been >> > > fixed in version X, and go back to 2.a. >> > > ii. If the bug appears to still exist in the current version, note >> > > which version it was verified in and continue. >> > > d. Gather as much information as possible from users. What version >> > > of the system is this bug occurring in? What environment are they >> > > using? Did this error start occurring after an upgrade? What version >> > > did they upgrade to/from? Are they able to work around the issue or >> > > has it stopped their usage of Hudson in its tracks? Ask any questions >> > > that might help isolate the commit that introduced the bug, and help >> > > other contributors make the best possible prioritization decisions. >> > > e. Set the priority of the issue based on some criteria. I need >> > > some suggestions from the community about what would be the best >> > > criteria. I would actually suggest trying to get an open source >> > > license of GreenHopper from Atlassian. Their ranking functionality >> > > within GreenHopper works very well for this purpose since you rank >> > > items relative to each other, not just with priorities. >> > > f. Mark the bug as triaged and if there are still non-triaged issues >> > > go back to 2.a. >> > > 3. Contributors grab the highest (relative) priority bug to fix and >> > > get to it. Votes should also contribute to deciding which bug to fix >> > > since that information is real-time and earlier triage information can >> > > become more and more out of date.
>> > > I'm extremely interested in getting some feedback about these >> > > procedures. I'm sure I missed some opportunity in this process or >> > > overlooked something, so please let me know what you think. I hope >> > > that by unifying these procedures across the community we can work >> > > towards reducing that "average age" stat to below a month or two. I'm >> > > not saying it will be easy, but I'm certain this community can do it.
On Mon, Feb 7, 2011 at 9:17 AM, Richard Bywater <rbywa...@gmail.com> wrote: > I'd say it couldn't hurt though probably depends on how good the matches > are...
>> On Mon, Feb 7, 2011 at 2:27 AM, evernat <ever...@free.fr> wrote:
>>> Hi Matt,
>>> I fully agree with you that obsolete bugs should be closed, in order >>> for most users to know what are really the current issues and in order >>> for most developers to know what has to be fixed. >>> But I think that the 494 stale bugs can't be closed like that. >>> For example, a large number of issues were written by Kohsuke for >>> things to check or for things to fix (for example issue 5036). >>> A bug having a low priority for one year probably doesn't mean that it >>> doesn't need to be fixed.
>>> I currently suggest that some, if not all, developers read each one of >>> the stale bugs in
>>> I will try to do that myself (instead of sleeping) and as Christoph >>> said: >>> - search actively for duplicates and close them [we may also read the >>> changelogs of releases]
>> We could ask to have a free version of suggestimate to find duplicates. >> It helps also users to not open issues which are already existing. >> http://www.suggestimate.com/?r=view
>> WDYT ?
>>> - ask reporters about more details [if needed] >>> - if bug is old and reporter doesn't respond in a timely manner, >>> then close it as 'not reproducible'
>>> Matt: Because you are already a user, please choose some issues to be >>> tested and write your test results in the issues : it will save most >>> of the time needed by the developers to know if the stale issues can >>> be closed ! (and it may help you later to submit patchs)
>>> bye, >>> Emeric
>>> On 7 fév, 00:29, mlb5000 <baker.mat...@gmail.com> wrote: >>> > Things haven't completely calmed down since the rename yet, but I'd >>> > like to get started closing out some of these old issues if at all >>> > possible. I noticed there was a "!CLOSED && STALE" query already on >>> > issues.jenkins-ci.org and I was wondering how we want to handle Bug >>> > reports that fall into this category. An issue is considered STALE if >>> > it hasn't been updated within the last year. I'd say that's a pretty >>> > good criteria for just calling it and closing the issue without >>> > discretion, but I don't personally have any insight into what some of >>> > those older bug reports might be, and the last thing I want is to >>> > upset the users.
>>> > At any rate, does anyone have any (further) suggestions for how to >>> > handle these older bugs reports without having to go through them one >>> > by one? Just to provide some perspective, 494 (http://issues.jenkins- >>> > ci.org/secure/IssueNavigator.jspa?mode=hide&requestId=10243) of the >>> > 1862 currently unresolved bug reports (http://issues.jenkins-ci.org/ >>> > secure/IssueNavigator.jspa?mode=hide&requestId=10244) are considered >>> > stale.
>>> > Also, would someone who has administrative privileges in the Jenkins >>> > Jira instance be able to add a Boolean field to the Bug issue type for >>> > marking whether or not a bug report has been Triaged, or does anyone >>> > have another way to keep track of whether a bug report has been >>> > vetted?
>>> > On Jan 30, 12:13 pm, mlb5000 <baker.mat...@gmail.com> wrote:
>>> > > Hi All,
>>> > > Disclaimer: I'm only just joining this community so I absolutely want >>> > > those of you with more experience developing/fixing bugs in Hudson >>> and >>> > > elsewhere to chime in on this conversation. What I want to do is try >>> > > to come to some consensus on how Hudson bugs are to be identified, >>> > > disambiguated, prioritized, and ultimately, scheduled (more or less) >>> > > for a release. This is part of what I hope will be a larger overall >>> > > effort to bring the Technical Debt I've observed since diving into >>> the >>> > > source code into check.
>>> > > As of right now there are 1853 open bugs in Hudson, of which the >>> > > average age is just under a year at 359 days. Some users, myself >>> > > included, have been getting discouraged with cycle time on bugs, >>> > > particularly when the Continuous Integration server is supposed to be >>> > > the one thing you can most rely on. When teams make their CI system >>> > > the heart and soul of their day-to-day development, testing, >>> > > configuration management, and release processes, they need to able to >>> > > rely on it. If any tool can be categorized as mission-critical, it's >>> > > the CI system.
>>> > > Here is what I am proposing:
>>> > > 1. Users submit bug reports at issues.hudson-ci.org like they always >>> > > have. >>> > > 2. At some frequency (daily?, weekly?, as they pop up?), developers >>> go >>> > > out and perform the following procedure: >>> > > a. Pick the oldest non-triaged bug. >>> > > b. Check whether or not the bug is a duplicate with another bug. >>> > > This will likely require aid from the more seasoned Hudson >>> > > contributors. >>> > > i. If a duplicate exists but it has been resolved, I would >>> > > recommend not reopening it, but just linking the two issues together >>> > > so the person who goes to fix the current incantation of the bug can >>> > > use it as a resource if necessary. Continue to the next step. >>> > > ii. If a duplicate exists, but is still open, link the two issues >>> > > for future reference, Close the new one as Duplicate, and redirect >>> all >>> > > users to the older one. Go back to 2.a. >>> > > iii. If no duplicate exists, continue. >>> > > c. Is the bug reproducible in the most recent version of Hudson? I >>> > > don't think it's worth verifying the bug in their reported version >>> > > since they'll ultimately have to upgrade to get the bug fix anyway, >>> so >>> > > we might as well work towards the shortest path. >>> > > i. If the bug is not reproducible in the latest version, close as >>> > > Cannot Reproduce and inform the user(s) that it appears to have been >>> > > fixed in version X, and go back to 2.a. >>> > > ii. If the bug appears to still exist in the current version, >>> note >>> > > which version it was verified in and continue. >>> > > d. Gather as much information as possible from users. What version >>> > > of the system is this bug occurring in? What environment are they >>> > > using? Did this error start occurring after an upgrade? What >>> version >>> > > did they upgrade to/from? Are they able to work around the issue or >>> > > has it stopped their usage of Hudson in its tracks? Ask any >>> questions >>> > > that might help isolate the commit that introduced the bug, and help >>> > > other contributors make the best possible prioritization decisions. >>> > > e. Set the priority of the issue based on some criteria. I need >>> > > some suggestions from the community about what would be the best >>> > > criteria. I would actually suggest trying to get an open source >>> > > license of GreenHopper from Atlassian. Their ranking functionality >>> > > within GreenHopper works very well for this purpose since you rank >>> > > items relative to each other, not just with priorities. >>> > > f. Mark the bug as triaged and if there are still non-triaged >>> issues >>> > > go back to 2.a. >>> > > 3. Contributors grab the highest (relative) priority bug to fix and >>> > > get to it. Votes should also contribute to deciding which bug to fix >>> > > since that information is real-time and earlier triage information >>> can >>> > > become more and more out of date.
>>> > > I'm extremely interested in getting some feedback about these >>> > > procedures. I'm sure I missed some opportunity in this process or >>> > > overlooked something, so please let me know what you think. I hope >>> > > that by unifying these procedures across the community we can work >>> > > towards reducing that "average age" stat to below a month or two. >>> I'm >>> > > not saying it will be easy, but I'm certain this community can do it.
> Things haven't completely calmed down since the rename yet, but I'd > like to get started closing out some of these old issues if at all > possible. I noticed there was a "!CLOSED&& STALE" query already on > issues.jenkins-ci.org and I was wondering how we want to handle Bug > reports that fall into this category. An issue is considered STALE if > it hasn't been updated within the last year. I'd say that's a pretty > good criteria for just calling it and closing the issue without > discretion, but I don't personally have any insight into what some of > those older bug reports might be, and the last thing I want is to > upset the users.
I think in a first step we should categorize issues before we are updating or closing them.
It makes sense to handle issues from core (and core components) differently then for plug-ins. E.g., I have several issues in my plug-ins that are quite old, but I did not yet find the time to fix them. I would like to have these issues still open, since these issues reflect real bugs (or feature requests).
We also should handle issues differently that have no assignee yet. There are 1958 unresolved issues without assignee (from 3251). I think for issues with assignee the assignee should look at his issues in order to suggest if the issue could be closed or not.
> At any rate, does anyone have any (further) suggestions for how to > handle these older bugs reports without having to go through them one > by one? Just to provide some perspective, 494 (http://issues.jenkins- > ci.org/secure/IssueNavigator.jspa?mode=hide&requestId=10243) of the > 1862 currently unresolved bug reports (http://issues.jenkins-ci.org/ > secure/IssueNavigator.jspa?mode=hide&requestId=10244) are considered > stale.
I think we need to look at an issue before we can close it...
> Also, would someone who has administrative privileges in the Jenkins > Jira instance be able to add a Boolean field to the Bug issue type for > marking whether or not a bug report has been Triaged, or does anyone > have another way to keep track of whether a bug report has been > vetted?
What do you mean with issues that are triaged? Accepted as a real bug or feature? Or already planned for a release?
> On Jan 30, 12:13 pm, mlb5000<baker.mat...@gmail.com> wrote: >> Hi All,
>> Disclaimer: I'm only just joining this community so I absolutely want >> those of you with more experience developing/fixing bugs in Hudson and >> elsewhere to chime in on this conversation. What I want to do is try >> to come to some consensus on how Hudson bugs are to be identified, >> disambiguated, prioritized, and ultimately, scheduled (more or less) >> for a release. This is part of what I hope will be a larger overall >> effort to bring the Technical Debt I've observed since diving into the >> source code into check.
>> As of right now there are 1853 open bugs in Hudson, of which the >> average age is just under a year at 359 days. Some users, myself >> included, have been getting discouraged with cycle time on bugs, >> particularly when the Continuous Integration server is supposed to be >> the one thing you can most rely on. When teams make their CI system >> the heart and soul of their day-to-day development, testing, >> configuration management, and release processes, they need to able to >> rely on it. If any tool can be categorized as mission-critical, it's >> the CI system.
>> Here is what I am proposing:
>> 1. Users submit bug reports at issues.hudson-ci.org like they always >> have. >> 2. At some frequency (daily?, weekly?, as they pop up?), developers go >> out and perform the following procedure: >> a. Pick the oldest non-triaged bug. >> b. Check whether or not the bug is a duplicate with another bug. >> This will likely require aid from the more seasoned Hudson >> contributors. >> i. If a duplicate exists but it has been resolved, I would >> recommend not reopening it, but just linking the two issues together >> so the person who goes to fix the current incantation of the bug can >> use it as a resource if necessary. Continue to the next step. >> ii. If a duplicate exists, but is still open, link the two issues >> for future reference, Close the new one as Duplicate, and redirect all >> users to the older one. Go back to 2.a. >> iii. If no duplicate exists, continue. >> c. Is the bug reproducible in the most recent version of Hudson? I >> don't think it's worth verifying the bug in their reported version >> since they'll ultimately have to upgrade to get the bug fix anyway, so >> we might as well work towards the shortest path. >> i. If the bug is not reproducible in the latest version, close as >> Cannot Reproduce and inform the user(s) that it appears to have been >> fixed in version X, and go back to 2.a. >> ii. If the bug appears to still exist in the current version, note >> which version it was verified in and continue. >> d. Gather as much information as possible from users. What version >> of the system is this bug occurring in? What environment are they >> using? Did this error start occurring after an upgrade? What version >> did they upgrade to/from? Are they able to work around the issue or >> has it stopped their usage of Hudson in its tracks? Ask any questions >> that might help isolate the commit that introduced the bug, and help >> other contributors make the best possible prioritization decisions. >> e. Set the priority of the issue based on some criteria. I need >> some suggestions from the community about what would be the best >> criteria. I would actually suggest trying to get an open source >> license of GreenHopper from Atlassian. Their ranking functionality >> within GreenHopper works very well for this purpose since you rank >> items relative to each other, not just with priorities. >> f. Mark the bug as triaged and if there are still non-triaged issues >> go back to 2.a. >> 3. Contributors grab the highest (relative) priority bug to fix and >> get to it. Votes should also contribute to deciding which bug to fix >> since that information is real-time and earlier triage information can >> become more and more out of date.
>> I'm extremely interested in getting some feedback about these >> procedures. I'm sure I missed some opportunity in this process or >> overlooked something, so please let me know what you think. I hope >> that by unifying these procedures across the community we can work >> towards reducing that "average age" stat to below a month or two. I'm >> not saying it will be easy, but I'm certain this community can do it.
> - ask reporters about more details [if needed] > - if bug is old and reporter doesn't respond in a timely manner, > then close it as 'not reproducible'
I have done a fair amount of this over the past couple years (not recently though).. asking if issues still exist on recent versions, and closing if no response in a few weeks.
@evernat - See, I knew there were people here who knew a lot more than
me about the state of the Jenkins bug backlog :). I never suggested
you should lose sleep over it though, lol. Were you actually able to
make any progress? I can definitely carve out some time to verify
whether or not stale bugs still exist.
@arnaud I've never used that tool, but if you think it might help make
a dent in discovering duplicate bug reports it can't hurt, especially
if it doesn't cost anything. It might be difficult to decide who the
Open Source license gets assigned to until the Jenkins Governance
group gets the project under a legal umbrella.
QUESTION: Does anyone know who the custodian on Open Source software
licenses for Jenkins is?
@ulli Just to be clear, I'm only focusing on bug reports right now,
not the other issue types like Feature or Improvement. By your same
query there are 709 unresolved bug reports that have no one assigned
to them. It seems like this list would be a good place to start. As
far as the Jira request, what I'm looking for is a way that multiple
people can tackle this problem at once without stepping on each
other's toes and duplicating work. Perhaps simply posting a comment
saying "Trying to reproduce" while you work on it, and "Bug Reproduced
in version X" or "Bug not reproducible in version X" will suffice?
That should at least be enough to keep people from trying to do the
same thing, right? Making sure a bug is assigned to the right
"component" is also a good idea. That way we the developers can
prioritize the work that needs to be done, particularly if a bug is
associated with a plugin like you describe.
QUESTION: Who should have the authority to mark a bug as Resolved and
then Closed in Jira? I don't want to feel like I'm encroaching by
going in there and changing the state of a bunch of open issues
without getting some direction first. I personally believe that bugs
marked as Resolved should have a second step where someone else
verifies that a bug no longer exists, but how should we handle it if a
bug is not reproducible at all in the current version? Should we
Close as "Cannot Reproduce" right away, or make a note that it is not
reproducible on some version and then have someone else do the same
check just to be safe? I'm sure it will also have a lot to do with
who reported that it was not reproducible. I'm sure the community
would be much less likely to trust me than Kohsuke.
@Alan, would you suggest that we take some of the more stale issues
and post a comment asking the reporter(s) if they're still having the
problem in the latest version? It seems like the general idea I'm
getting from everyone is that we should make an effort to confirm a
bug's existence in these cases as well.
On Feb 6, 8:27 pm, evernat <ever...@free.fr> wrote:
> I fully agree with you that obsolete bugs should be closed, in order
> for most users to know what are really the current issues and in order
> for most developers to know what has to be fixed.
> But I think that the 494 stale bugs can't be closed like that.
> For example, a large number of issues were written by Kohsuke for
> things to check or for things to fix (for example issue 5036).
> A bug having a low priority for one year probably doesn't mean that it
> doesn't need to be fixed.
> I will try to do that myself (instead of sleeping) and as Christoph
> said:
> - search actively for duplicates and close them [we may also read the
> changelogs of releases]
> - ask reporters about more details [if needed]
> - if bug is old and reporter doesn't respond in a timely manner,
> then close it as 'not reproducible'
> Matt: Because you are already a user, please choose some issues to be
> tested and write your test results in the issues : it will save most
> of the time needed by the developers to know if the stale issues can
> be closed ! (and it may help you later to submit patchs)
> bye,
> Emeric
> On 7 fév, 00:29, mlb5000 <baker.mat...@gmail.com> wrote:
> > Things haven't completely calmed down since the rename yet, but I'd
> > like to get started closing out some of these old issues if at all
> > possible. I noticed there was a "!CLOSED && STALE" query already on
> > issues.jenkins-ci.org and I was wondering how we want to handle Bug
> > reports that fall into this category. An issue is considered STALE if
> > it hasn't been updated within the last year. I'd say that's a pretty
> > good criteria for just calling it and closing the issue without
> > discretion, but I don't personally have any insight into what some of
> > those older bug reports might be, and the last thing I want is to
> > upset the users.
> > At any rate, does anyone have any (further) suggestions for how to
> > handle these older bugs reports without having to go through them one
> > by one? Just to provide some perspective, 494 (http://issues.jenkins- > > ci.org/secure/IssueNavigator.jspa?mode=hide&requestId=10243) of the
> > 1862 currently unresolved bug reports (http://issues.jenkins-ci.org/ > > secure/IssueNavigator.jspa?mode=hide&requestId=10244) are considered
> > stale.
> > Also, would someone who has administrative privileges in the Jenkins
> > Jira instance be able to add a Boolean field to the Bug issue type for
> > marking whether or not a bug report has been Triaged, or does anyone
> > have another way to keep track of whether a bug report has been
> > vetted?
> > On Jan 30, 12:13 pm, mlb5000 <baker.mat...@gmail.com> wrote:
> > > Hi All,
> > > Disclaimer: I'm only just joining this community so I absolutely want
> > > those of you with more experience developing/fixing bugs in Hudson and
> > > elsewhere to chime in on this conversation. What I want to do is try
> > > to come to some consensus on how Hudson bugs are to be identified,
> > > disambiguated, prioritized, and ultimately, scheduled (more or less)
> > > for a release. This is part of what I hope will be a larger overall
> > > effort to bring the Technical Debt I've observed since diving into the
> > > source code into check.
> > > As of right now there are 1853 open bugs in Hudson, of which the
> > > average age is just under a year at 359 days. Some users, myself
> > > included, have been getting discouraged with cycle time on bugs,
> > > particularly when the Continuous Integration server is supposed to be
> > > the one thing you can most rely on. When teams make their CI system
> > > the heart and soul of their day-to-day development, testing,
> > > configuration management, and release processes, they need to able to
> > > rely on it. If any tool can be categorized as mission-critical, it's
> > > the CI system.
> > > Here is what I am proposing:
> > > 1. Users submit bug reports at issues.hudson-ci.org like they always
> > > have.
> > > 2. At some frequency (daily?, weekly?, as they pop up?), developers go
> > > out and perform the following procedure:
> > > a. Pick the oldest non-triaged bug.
> > > b. Check whether or not the bug is a duplicate with another bug.
> > > This will likely require aid from the more seasoned Hudson
> > > contributors.
> > > i. If a duplicate exists but it has been resolved, I would
> > > recommend not reopening it, but just linking the two issues together
> > > so the person who goes to fix the current incantation of the bug can
> > > use it as a resource if necessary. Continue to the next step.
> > > ii. If a duplicate exists, but is still open, link the two issues
> > > for future reference, Close the new one as Duplicate, and redirect all
> > > users to the older one. Go back to 2.a.
> > > iii. If no duplicate exists, continue.
> > > c. Is the bug reproducible in the most recent version of Hudson? I
> > > don't think it's worth verifying the bug in their reported version
> > > since they'll ultimately have to upgrade to get the bug fix anyway, so
> > > we might as well work towards the shortest path.
> > > i. If the bug is not reproducible in the latest version, close as
> > > Cannot Reproduce and inform the user(s) that it appears to have been
> > > fixed in version X, and go back to 2.a.
> > > ii. If the bug appears to still exist in the current version, note
> > > which version it was verified in and continue.
> > > d. Gather as much information as possible from users. What version
> > > of the system is this bug occurring in? What environment are they
> > > using? Did this error start occurring after an upgrade? What version
> > > did they upgrade to/from? Are they able to work around the issue or
> > > has it stopped their usage of Hudson in its tracks? Ask any questions
> > > that might help isolate the commit that introduced the bug, and help
> > > other contributors make the best possible prioritization decisions.
> > > e. Set the priority of the issue based on some criteria. I need
> > > some suggestions from the community about what would be the best
> > > criteria. I would actually suggest trying to get an open source
> > > license of GreenHopper from Atlassian. Their ranking functionality
> > > within GreenHopper works very well for this purpose since you rank
> > > items relative to each other, not just with priorities.
> > > f. Mark the bug as triaged and if there are still non-triaged issues
> > > go back to 2.a.
> > > 3. Contributors grab the highest (relative) priority bug to fix and
> > > get to it. Votes should also contribute to deciding which bug to fix
> > > since that information is real-time and earlier
> QUESTION: Who should have the authority to mark a bug as Resolved and > then Closed in Jira? I don't want to feel like I'm encroaching by > going in there and changing the state of a bunch of open issues > without getting some direction first. I personally believe that bugs > marked as Resolved should have a second step where someone else > verifies that a bug no longer exists,
I think to date we've interpreted "Resolved" as closed.. a few people come back and mark things verified, but not many. I never query Resolved issues and wonder if they need to be verified anyway..
> but how should we handle it if a > bug is not reproducible at all in the current version? Should we > Close as "Cannot Reproduce" right away, or make a note that it is not > reproducible on some version and then have someone else do the same > check just to be safe? I'm sure it will also have a lot to do with > who reported that it was not reproducible. I'm sure the community > would be much less likely to trust me than Kohsuke.
As with stale bugs I would add a comment here and give the reporter a chance to give more detail.. there are a lot of environment specific issues out there.
> @Alan, would you suggest that we take some of the more stale issues > and post a comment asking the reporter(s) if they're still having the > problem in the latest version? It seems like the general idea I'm > getting from everyone is that we should make an effort to confirm a > bug's existence in these cases as well.
It's the method I've used in the past, to avoid complaints as you said.. but it does require looking at all the issues, no large scale pruning..
On Mon, Feb 7, 2011 at 7:57 PM, Alan Harder <mindl...@dev.java.net> wrote:
> As with stale bugs I would add a comment here and give the reporter a chance > to give more detail.. there are a lot of environment specific issues out > there.
I'm the one that put that !CLOSED && STALE query out there, so I am personally interested in how this unfolds.. Anyhow, it would be very nice if we could somehow tag at least the ones which are in Resolved as 'last-chance' or something similar. If that were possible, I am sure we can bulk update add a comment and tag to the effect that the issue is a cold case and to update the record with a fresh confirmation of the problem. If no update is received in 5 weeks, we then bulk close any !CLOSED && hasTag(last-chance) or some such.
There are so many issues like this that it doesn't make sense to do it manually. There are something like 4300 issues in Resolved state, there are 4100 !CLOSED && STALE, of which 1050 are !CLOSED && !RESOLVED && STALE. For this final batch, if they are in plugins then we should probably just leave them; if they are in core, then they should also receive last-chance warnings on the presumption that there have been at least 52 releases since the bug could conceivably have been open and we want a fresh confirmation of the problem to proceed.
-Jesse
-- There are 10 types of people in this world, those that can read binary and those that can not.
> @ulli Just to be clear, I'm only focusing on bug reports right now, > not the other issue types like Feature or Improvement. By your same > query there are 709 unresolved bug reports that have no one assigned > to them. It seems like this list would be a good place to start. As > far as the Jira request, what I'm looking for is a way that multiple > people can tackle this problem at once without stepping on each > other's toes and duplicating work. Perhaps simply posting a comment > saying "Trying to reproduce" while you work on it, and "Bug Reproduced > in version X" or "Bug not reproducible in version X" will suffice? > That should at least be enough to keep people from trying to do the > same thing, right? Making sure a bug is assigned to the right > "component" is also a good idea. That way we the developers can > prioritize the work that needs to be done, particularly if a bug is > associated with a plugin like you describe.
Yes, that would be an option.
What we can also do is a more formal issue submission and QA process (that I used successfully in several projects): use an additional status (e.g., Submitted) that will be assigned to all created issues. All those submitted issues need to be verified by the assignee and could be then moved to the state Open. And for those issues that don't get an assignee: here we need some volunteers who are evaluating these issues e.g. on a daily or weekly basis (trying to reproduce or write a comment to request additional information from the reporter). All of our old open and unassigned issues could be moved back to the Submitted state.
On Tue, Feb 8, 2011 at 12:35 AM, mlb5000 <baker.mat...@gmail.com> wrote: > @evernat - See, I knew there were people here who knew a lot more than > me about the state of the Jenkins bug backlog :). I never suggested > you should lose sleep over it though, lol. Were you actually able to > make any progress? I can definitely carve out some time to verify > whether or not stale bugs still exist.
> @arnaud I've never used that tool, but if you think it might help make > a dent in discovering duplicate bug reports it can't hurt, especially > if it doesn't cost anything. It might be difficult to decide who the > Open Source license gets assigned to until the Jenkins Governance > group gets the project under a legal umbrella.
They weren't difficult to give mine when I asked it. I don't think it will be a problem to not have a real foundation to receive it.
> QUESTION: Does anyone know who the custodian on Open Source software > licenses for Jenkins is?
> @ulli Just to be clear, I'm only focusing on bug reports right now, > not the other issue types like Feature or Improvement. By your same > query there are 709 unresolved bug reports that have no one assigned > to them. It seems like this list would be a good place to start. As > far as the Jira request, what I'm looking for is a way that multiple > people can tackle this problem at once without stepping on each > other's toes and duplicating work. Perhaps simply posting a comment > saying "Trying to reproduce" while you work on it, and "Bug Reproduced > in version X" or "Bug not reproducible in version X" will suffice? > That should at least be enough to keep people from trying to do the > same thing, right? Making sure a bug is assigned to the right > "component" is also a good idea. That way we the developers can > prioritize the work that needs to be done, particularly if a bug is > associated with a plugin like you describe.
> QUESTION: Who should have the authority to mark a bug as Resolved and > then Closed in Jira? I don't want to feel like I'm encroaching by > going in there and changing the state of a bunch of open issues > without getting some direction first. I personally believe that bugs > marked as Resolved should have a second step where someone else > verifies that a bug no longer exists, but how should we handle it if a > bug is not reproducible at all in the current version? Should we > Close as "Cannot Reproduce" right away, or make a note that it is not > reproducible on some version and then have someone else do the same > check just to be safe? I'm sure it will also have a lot to do with > who reported that it was not reproducible. I'm sure the community > would be much less likely to trust me than Kohsuke.
> @Alan, would you suggest that we take some of the more stale issues > and post a comment asking the reporter(s) if they're still having the > problem in the latest version? It seems like the general idea I'm > getting from everyone is that we should make an effort to confirm a > bug's existence in these cases as well.
> On Feb 6, 8:27 pm, evernat <ever...@free.fr> wrote: >> Hi Matt,
>> I fully agree with you that obsolete bugs should be closed, in order >> for most users to know what are really the current issues and in order >> for most developers to know what has to be fixed. >> But I think that the 494 stale bugs can't be closed like that. >> For example, a large number of issues were written by Kohsuke for >> things to check or for things to fix (for example issue 5036). >> A bug having a low priority for one year probably doesn't mean that it >> doesn't need to be fixed.
>> I will try to do that myself (instead of sleeping) and as Christoph >> said: >> - search actively for duplicates and close them [we may also read the >> changelogs of releases] >> - ask reporters about more details [if needed] >> - if bug is old and reporter doesn't respond in a timely manner, >> then close it as 'not reproducible'
>> Matt: Because you are already a user, please choose some issues to be >> tested and write your test results in the issues : it will save most >> of the time needed by the developers to know if the stale issues can >> be closed ! (and it may help you later to submit patchs)
>> bye, >> Emeric
>> On 7 fév, 00:29, mlb5000 <baker.mat...@gmail.com> wrote:
>> > Things haven't completely calmed down since the rename yet, but I'd >> > like to get started closing out some of these old issues if at all >> > possible. I noticed there was a "!CLOSED && STALE" query already on >> > issues.jenkins-ci.org and I was wondering how we want to handle Bug >> > reports that fall into this category. An issue is considered STALE if >> > it hasn't been updated within the last year. I'd say that's a pretty >> > good criteria for just calling it and closing the issue without >> > discretion, but I don't personally have any insight into what some of >> > those older bug reports might be, and the last thing I want is to >> > upset the users.
>> > At any rate, does anyone have any (further) suggestions for how to >> > handle these older bugs reports without having to go through them one >> > by one? Just to provide some perspective, 494 (http://issues.jenkins- >> > ci.org/secure/IssueNavigator.jspa?mode=hide&requestId=10243) of the >> > 1862 currently unresolved bug reports (http://issues.jenkins-ci.org/ >> > secure/IssueNavigator.jspa?mode=hide&requestId=10244) are considered >> > stale.
>> > Also, would someone who has administrative privileges in the Jenkins >> > Jira instance be able to add a Boolean field to the Bug issue type for >> > marking whether or not a bug report has been Triaged, or does anyone >> > have another way to keep track of whether a bug report has been >> > vetted?
>> > On Jan 30, 12:13 pm, mlb5000 <baker.mat...@gmail.com> wrote:
>> > > Hi All,
>> > > Disclaimer: I'm only just joining this community so I absolutely want >> > > those of you with more experience developing/fixing bugs in Hudson and >> > > elsewhere to chime in on this conversation. What I want to do is try >> > > to come to some consensus on how Hudson bugs are to be identified, >> > > disambiguated, prioritized, and ultimately, scheduled (more or less) >> > > for a release. This is part of what I hope will be a larger overall >> > > effort to bring the Technical Debt I've observed since diving into the >> > > source code into check.
>> > > As of right now there are 1853 open bugs in Hudson, of which the >> > > average age is just under a year at 359 days. Some users, myself >> > > included, have been getting discouraged with cycle time on bugs, >> > > particularly when the Continuous Integration server is supposed to be >> > > the one thing you can most rely on. When teams make their CI system >> > > the heart and soul of their day-to-day development, testing, >> > > configuration management, and release processes, they need to able to >> > > rely on it. If any tool can be categorized as mission-critical, it's >> > > the CI system.
>> > > Here is what I am proposing:
>> > > 1. Users submit bug reports at issues.hudson-ci.org like they always >> > > have. >> > > 2. At some frequency (daily?, weekly?, as they pop up?), developers go >> > > out and perform the following procedure: >> > > a. Pick the oldest non-triaged bug. >> > > b. Check whether or not the bug is a duplicate with another bug. >> > > This will likely require aid from the more seasoned Hudson >> > > contributors. >> > > i. If a duplicate exists but it has been resolved, I would >> > > recommend not reopening it, but just linking the two issues together >> > > so the person who goes to fix the current incantation of the bug can >> > > use it as a resource if necessary. Continue to the next step. >> > > ii. If a duplicate exists, but is still open, link the two issues >> > > for future reference, Close the new one as Duplicate, and redirect all >> > > users to the older one. Go back to 2.a. >> > > iii. If no duplicate exists, continue. >> > > c. Is the bug reproducible in the most recent version of Hudson? I >> > > don't think it's worth verifying the bug in their reported version >> > > since they'll ultimately have to upgrade to get the bug fix anyway, so >> > > we might as well work towards the shortest path. >> > > i. If the bug is not reproducible in the latest version, close as >> > > Cannot Reproduce and inform the user(s) that it appears to have been >> > > fixed in version X, and go back to 2.a. >> > > ii. If the bug appears to still exist in the current version, note >> > > which version it was verified in and continue. >> > > d. Gather as much information as possible from users. What version >> > > of the system is this bug occurring in? What environment are they >> > > using? Did this error start occurring after an upgrade? What version >> > > did they upgrade to/from? Are they able to work around the issue or >> > > has it stopped their usage of Hudson in its tracks? Ask any questions >> > > that might help isolate the commit that introduced the bug, and help >> > > other contributors make the best possible prioritization decisions. >> > > e. Set the priority of the issue based on some criteria. I need >> > > some suggestions from the community about what would be the best >> > > criteria. I would actually suggest trying to get an open source >> > > license of GreenHopper from Atlassian. Their ranking functionality >> > > within GreenHopper works very well for this purpose since you rank >> > > items relative to each other, not just with priorities. >> > > f. Mark the bug as triaged and if
I really like the idea of the comment "Reproduced in version X" or "unable to reproduce on Y platform" because it gives much more details than just the version tag or the issue status.
Is there any way for a user to quickly describe his environment? I was thinking of a simple link "get installation details" reporting info like: - OS version for the master and each node - JDK used - Version of Jenkins and all plugins - job type (and eventually plugins used)
This could to reproduce the issues related to plugins or specific platform.
Cheers, Francois
Le 8 févr. 2011 à 17:45, Ullrich Hafner <ullrich.haf...@gmail.com> a écrit :
> On 02/08/2011 12:35 AM, mlb5000 wrote: >> @ulli Just to be clear, I'm only focusing on bug reports right now, >> not the other issue types like Feature or Improvement. By your same >> query there are 709 unresolved bug reports that have no one assigned >> to them. It seems like this list would be a good place to start. As >> far as the Jira request, what I'm looking for is a way that multiple >> people can tackle this problem at once without stepping on each >> other's toes and duplicating work. Perhaps simply posting a comment >> saying "Trying to reproduce" while you work on it, and "Bug >> Reproduced >> in version X" or "Bug not reproducible in version X" will suffice? >> That should at least be enough to keep people from trying to do the >> same thing, right? Making sure a bug is assigned to the right >> "component" is also a good idea. That way we the developers can >> prioritize the work that needs to be done, particularly if a bug is >> associated with a plugin like you describe.
> Yes, that would be an option.
> What we can also do is a more formal issue submission and QA process > (that I used successfully in several projects): use an additional > status (e.g., Submitted) that will be assigned to all created > issues. All those submitted issues need to be verified by the > assignee and could be then moved to the state Open. And for those > issues that don't get an assignee: here we need some volunteers who > are evaluating these issues e.g. on a daily or weekly basis (trying > to reproduce or write a comment to request additional information > from the reporter). All of our old open and unassigned issues could > be moved back to the Submitted state.
> I really like the idea of the comment "Reproduced in version X" or > "unable to reproduce on Y platform" because it gives much more details > than just the version tag or the issue status.
> Is there any way for a user to quickly describe his environment? > I was thinking of a simple link "get installation details" reporting > info like: > - OS version for the master and each node > - JDK used > - Version of Jenkins and all plugins > - job type (and eventually plugins used)
> This could to reproduce the issues related to plugins or specific > platform.
> Cheers, > Francois
I think this is not possible out of the box.
However, one can write a simple plug-in (or core feature) that does add a create issue link to a Jenkins installation, that will open a pre-filled Jira create issue screen when clicked. Maybe we could use such a feature to replace the current log message 'Please report this bug at the mailing list...'.
On Tue, Feb 8, 2011 at 13:16, Ullrich Hafner <ullrich.haf...@gmail.com> wrote: > On 02/08/2011 01:06 PM, Francois Cottet wrote:
>> Hi all,
>> I really like the idea of the comment "Reproduced in version X" or "unable >> to reproduce on Y platform" because it gives much more details than just the >> version tag or the issue status.
>> Is there any way for a user to quickly describe his environment? >> I was thinking of a simple link "get installation details" reporting info >> like: >> - OS version for the master and each node >> - JDK used >> - Version of Jenkins and all plugins >> - job type (and eventually plugins used)
>> This could to reproduce the issues related to plugins or specific >> platform.
>> Cheers, >> Francois
> I think this is not possible out of the box.
> However, one can write a simple plug-in (or core feature) that does add a > create issue link to a Jenkins installation, that will open a pre-filled > Jira create issue screen when clicked. Maybe we could use such a feature to > replace the current log message 'Please report this bug at the mailing > list...'.
I would love to see something like that. Unfortunately Jira doesnt seem to have any such feature in the webui today, but there are several ways to do it in code, [1], [2] and of course there is a CLI that is written in java that can create issues [3].
Such a plugin should * Require that the user to login, so they can later be notified with changes and perhaps close it when the issue is fixed. * The user should be able to verify that it doesnt send any confidential data to the jira. * Maybe a hash of an exception could be added to the created issues, so user wont create issue for the same error. Instead they can be allowed to vote and/or watch an issue. * Perhaps it could even be possible to group an exception on a component/plugin dependent on the classes involved. But that may be stretching it :)
Is there any such extension today that will allow any plugin to listen on exception crashes or other problems?