Simplifying JIRA via a custom workflow

[Chas Emerick]

"""

In the past, I've suggested using a custom workflow in JIRA to make it easier for everyone involved to follow and understand the contribution process.  That's always languished, but the cumulative pain I was seeing prompted me to speak up again about it at the /dev meetup at the Conj on Saturday.

I've created a custom workflow that attempts to formalize my understanding of the contribution process (based on this for the most part), and attached it to a dummy project in JIRA.  The result is what I think is what would be a less confusing experience for everyone involved, with useful actions ('submit for screening', 'reject patch', 'change approved', etc) being elevated to workflow transitions rather than being represented by multiple edits involving custom fields in addition to default workflow transitions.

"""

This turned into far too long of an email for the dev list, so I posted it on the wiki:

http://dev.clojure.org/display/design/Clojure+workflow+%28experimental%29

Give the dummy project a run and see how the workflow strikes you.

Cheers,

- Chas

[Paul Stadig]

Chas,
Thanks for doing this. One of the links from the wiki page (the one going to the graphical representation of the Jira workflow?) says I don't have permissions to see it.

I think this would help make things clearer as to where tickets are and where they need to go.


Paul

--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To post to this group, send email to cloju...@googlegroups.com.
To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.

[Paul Stadig]

Sorry. Kind of a half formed thought there. I read the wiki page, and it looks good to me. I looked at the history for the one issue, and it looks like it has the state transitions I would expect.

I wasn't able to get to the graphical view of the workflow, though, to get a sense of the overall flow.

Would it be a lot of work to get this setup in an existing project with lot's of tickets?


Paul

[Chas Emerick]

Yeah, the workflow page is behind the admin firewall.  No getting around that I'm afraid, though I've added a screenshot of the current state of the draft workflow to give you a sense of things:

http://dev.clojure.org/display/design/Clojure+workflow+%28experimental%29

The impact question is a good one.  I haven't eliminated or changed any of the existing JIRA workflow, so existing states should carry over without any impact.  Certainly, it would be a good idea to clone the CLJ project and change its workflow over to see if anything clearly bad happens.  Beyond hard failures, the biggest problem will be that there's no way to get the state implied by the custom Patch and Approval fields translated into updated ticket states, short of mucking with the JIRA API.  Maybe Stu H. has some bits left over from the assembla conversion that could be repurposed with a minimum of pain?

- Chas

[Aaron Bedra]

Chas, do you need admin access on JIRA? 

[Chas Emerick]

No, I have admin on JIRA (not on Confluence); otherwise, I wouldn't have been able to make the workflow to begin with. There's just no way to make it possible for non-admins to see the rendering of the workflow diagram since it's on the other side of the admin DMZ.

- Chas

[Aaron Bedra]

Ah ok. I just wanted to make sure you have what you need.

[David Nolen]

On Mon, Nov 14, 2011 at 12:51 PM, Chas Emerick <ceme...@snowtide.com> wrote:

Thanks for taking this on. Seems logical to me. Is there anything special that needs be done to adopt this workflow in the contrib JIRA projects?

David 

[Chas Emerick]

Not particularly, no.  The biggest unknown is whether Rich and the screeners will find this a net positive.

It seems like the workflow should be as similar as possible between contrib projects and CLJ — having just one for all Clojure projects would be great — but the stickiest piece of that is going to be permissions.  As I pointed out on the wiki page, there are no fine-grained permissions around ticket actions now, so I'm not sure it's worth bothering wandering into the weeds to lock things down hard to fully implement the idealized workflow depicted @ http://dev.clojure.org/display/design/JIRA+workflow   Worst case scenario is that an admin reopens a closed issue, or Rich indicates that he didn't actually approve a patch, etc.  The latter *could* result in stuff being committed that wasn't actually reviewed and approved (e.g. if a committer is churning through approved tickets) — but again, that can happen today.

- Chas

[Sean Corfield]

On Tue, Nov 15, 2011 at 9:09 AM, Chas Emerick <ceme...@snowtide.com> wrote:
> It seems like the workflow should be as similar as possible between contrib
> projects and CLJ — having just one for all Clojure projects would be great —
> but the stickiest piece of that is going to be permissions.

But contrib projects aren't bound by screening / approval in the same
way core/CLJ is... The only thing restricted is the 1.0.0 release and
that's not controlled by JIRA anyway.
--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)

[Stuart Halloway]

Hi Chas,

Thanks for taking this on. Having poked around briefly, here's what I
see so far.

* We use the "vetted" status to indicate that a second (or nth) person
has reviewed the ticket and agrees that it is a valid issue. This is
less important than the others, but as long as we are at it...

* The frustration I have with JIRA is that you cannot easily (1) allow
a state to be reachable from almost anywhere, or (2) have an ubermode
where you can ignore the transition rules. Two examples where this is
likely to come up: (1) go directly to accepted from anywhere, and (2)
go directly to screened from anywhere. When I looked at this before
the only solution I could see was actually defining a zillion
transitions.

* I prefer the word "declined" to "rejected". To me, the former
connotes no value judgment beyond "not what the team wants to do
here", and the latter seems negative. (I will admit it is the word
most people use.)

I am certainly willing to switch to something like if the transitions
are all available (or skippable), and if somebody can do the
conversion.

Stu

[Chas Emerick]

Vetted would just be a state transitioned to from Open, and leading to In Progress / Ready for Screening in most cases. I didn't add it yet, but doing so will essentially only require re-routing (nearly) all transitions that lead back to Open to go to Vetted instead. I suspect there's no case where an issue that was vetted and gotten through to being worked on or screened would get yanked back to an un-vetted, Open state.

> * The frustration I have with JIRA is that you cannot easily (1) allow
> a state to be reachable from almost anywhere, or (2) have an ubermode
> where you can ignore the transition rules. Two examples where this is
> likely to come up: (1) go directly to accepted from anywhere, and (2)
> go directly to screened from anywhere. When I looked at this before
> the only solution I could see was actually defining a zillion
> transitions.

Your experience must be from some prior version of JIRA: what's installed on dev.clojure.org doesn't have limitation (1) at the very least. I changed the Approve and Patch Screened transitions to be "global", meaning that they are available from any other state. This bulks up the workflow transition options quite a bit for project owners and admins, but patch submitters shouldn't see them at all and thus shouldn't have much of an impact on their experience.

To make this correct, we should have a "screeners" group — not all of whom are admins AFAIK.

> * I prefer the word "declined" to "rejected". To me, the former
> connotes no value judgment beyond "not what the team wants to do
> here", and the latter seems negative. (I will admit it is the word
> most people use.)

Changed "rejected" -> "declined".

> I am certainly willing to switch to something like if the transitions
> are all available (or skippable), and if somebody can do the
> conversion.

If Accepted and Screened are always available to the appropriate individuals, then do you really ever want to be able to go from any state M to N? There is an actual workflow; poking holes in it selectively for project owners/screeners/"lieutenants" makes sense, but I don't see the use case of suspending transition rules in general.

What "conversion" means is unclear. Insofar as we aren't (AFAICT) going to eliminate any states from the default workflow, a project's issues will switch over without much problem. I had suggested using a script before, but actually just a couple of easy mass-change edits should get the job done equally well.

What are the specific issues/concerns re: conversion?

- Chas

[Stuart Halloway]

* We use the "vetted" status to indicate that a second (or nth) person

has reviewed the ticket and agrees that it is a valid issue. This is

less important than the others, but as long as we are at it...

Vetted would just be a state transitioned to from Open, and leading to In Progress / Ready for Screening in most cases.  I didn't add it yet, but doing so will essentially only require re-routing (nearly) all transitions that lead back to Open to go to Vetted instead. I suspect there's no case where an issue that was vetted and gotten through to being worked on or screened would get yanked back to an un-vetted, Open state.

Cool.

* The frustration I have with JIRA is that you cannot easily (1) allow

a state to be reachable from almost anywhere, or (2) have an ubermode

where you can ignore the transition rules. Two examples where this is

likely to come up: (1) go directly to accepted from anywhere, and (2)

go directly  to screened from anywhere. When I looked at this before

the only solution I could see was actually defining a zillion

transitions.

Your experience must be from some prior version of JIRA: what's installed on dev.clojure.org doesn't have limitation (1) at the very least.  I changed the Approve and Patch Screened transitions to be "global", meaning that they are available from any other state.  This bulks up the workflow transition options quite a bit for project owners and admins, but patch submitters shouldn't see them at all and thus shouldn't have much of an impact on their experience.

To make this correct, we should have a "screeners" group — not all of whom are admins AFAIK.

Cool.

* I prefer the word "declined" to "rejected". To me, the former

connotes no value judgment beyond "not what the team wants to do

here", and the latter seems negative. (I will admit it is the word

most people use.)

Changed "rejected" -> "declined".

I am certainly willing to switch to something like if the transitions

are all available (or skippable), and if somebody can do the

conversion.

If Accepted and Screened are always available to the appropriate individuals, then do you really ever want to be able to go from any state M to N?  There is an actual workflow; poking holes in it selectively for project owners/screeners/"lieutenants" makes sense, but I don't see the use case of suspending transition rules in general.

Ok.

What "conversion" means is unclear.  Insofar as we aren't (AFAICT) going to eliminate any states from the default workflow, a project's issues will switch over without much problem.  I had suggested using a script before, but actually just a couple of easy mass-change edits should get the job done equally well.

What are the specific issues/concerns re: conversion?

None, just hadn't thought it through.

Stu

Stuart Halloway
Clojure/core
http://clojure.com

[Chas Emerick]

On Nov 16, 2011, at 11:27 AM, Stuart Halloway wrote:

>>> * We use the "vetted" status to indicate that a second (or nth) person
>>> has reviewed the ticket and agrees that it is a valid issue. This is
>>> less important than the others, but as long as we are at it...
>>
>> Vetted would just be a state transitioned to from Open, and leading to In Progress / Ready for Screening in most cases. I didn't add it yet, but doing so will essentially only require re-routing (nearly) all transitions that lead back to Open to go to Vetted instead. I suspect there's no case where an issue that was vetted and gotten through to being worked on or screened would get yanked back to an un-vetted, Open state.
>
> Cool.
>

>> What "conversion" means is unclear. Insofar as we aren't (AFAICT) going to eliminate any states from the default workflow, a project's issues will switch over without much problem. I had suggested using a script before, but actually just a couple of easy mass-change edits should get the job done equally well.
>>
>> What are the specific issues/concerns re: conversion?
>
> None, just hadn't thought it through.

The concurrent thread ("defrecord change suggestion") with the (understandable) confusion around vetting and general workflow prompts me to poke at this.

The quoted bits above are the two open issues I'm aware of with the draft JIRA workflow I put together (making Vetted a state with proper transitions to and from, and any open issues re: conversion of projects to use the workflow). Are there any others?

If I can get an authoritative "OK" from /core that a (suitable) JIRA workflow would be picked up ("vetting" the effort, as it were — perhaps there should be a clojure/dev JIRA project for such meta usages?), then I'll make the necessary workflow changes and put together a transition proposal.

- Chas

[Chas Emerick]

On Nov 15, 2011, at 1:40 PM, Sean Corfield wrote:

> On Tue, Nov 15, 2011 at 9:09 AM, Chas Emerick <ceme...@snowtide.com> wrote:
>> It seems like the workflow should be as similar as possible between contrib
>> projects and CLJ — having just one for all Clojure projects would be great —
>> but the stickiest piece of that is going to be permissions.
>
> But contrib projects aren't bound by screening / approval in the same
> way core/CLJ is... The only thing restricted is the 1.0.0 release and
> that's not controlled by JIRA anyway.

I can imagine some contrib projects getting to a scale of size/complexity where a screening and approval process is desirable, though that's perhaps more up to the preferences of the lead than anything else.

In any case, given the global transitions, the screen/approval states are entirely opt-in on a per-ticket basis (just click button A instead of button B). Whether those states are there all for a given project can be a point of discussion and/or decision on the part of a project lead.

- Chas

[Sean Corfield]

On Mon, Nov 21, 2011 at 1:13 PM, Chas Emerick <ceme...@snowtide.com> wrote:
> I can imagine some contrib projects getting to a scale of size/complexity where a screening and approval process is desirable, though that's perhaps more up to the preferences of the lead than anything else.
>
> In any case, given the global transitions, the screen/approval states are entirely opt-in on a per-ticket basis (just click button A instead of button B).  Whether those states are there all for a given project can be a point of discussion and/or decision on the part of a project lead.

Cool. I'm satisfied with that. Thank you!

[Chouser]

Chas,

Awesome work. It's so much easier for a patch submitter to do the
right thing, and having done so it's so much easier for me to find
what needs to be screened. Those are the two very key things from my
perspective. If it's indeed true that there are patches sitting out
there whose authors are feeling ignored but that the screeners aren't
finding, and if it's also true that Rich has been keeping up with
making approval decisions on screened patches, then the impact this
change could have on the sense of the devs could be huge.

Thanks for working on this, and I hope you quickly get the approval
you need to make this happen.

--Chouser

[Stuart Halloway]

Go for it.

Stu