Whew

[Patrick Wilson-Welsh]

Hi Assessing-agility,

  I look forward to this discussion. For every ounce of passion I might have around "certification," I have metric tons of passion around learning. Thanks, Josh, for kicking this off. 

--Patrick

-------

patrick welsh

248 565 6130

twitter: patrickwelsh

blog: patrickwilsonwelsh.com

[Joshua Kerievsky]

On Wed, Sep 16, 2009 at 12:23 PM, Patrick Wilson-Welsh <patrickwi...@gmail.com> wrote:

  I look forward to this discussion. For every ounce of passion I might have around "certification," I have metric tons of passion around learning. Thanks, Josh, for kicking this off. 

Thanks for joining Patrick.

I'm presently working on a talk about Agile and Art and it's got me looking at the work of many master painters.  Their work is masterful and therefore, long-lasting.  I'd like to do things on this planet that will be long-lasting and masterful.  

Helping folks, or rather, inspiring them, to become ever-more-Agile seems like a worthy and challenging goal.  

I mentioned World of Warcraft in the first post to this list.  I wonder if a community of us could create a similarly addictive world for Agile learning?

best

jk

[scott.duncan]

So this group really isn't about "assessing" agility, then as the
title suggests,
but more about discussing how to develop an online form of experience
for
people to improve their knowledge in and skill at an Agile approach to
software
development?

On Sep 16, 3:33 pm, Joshua Kerievsky <jos...@industriallogic.com>
wrote:

> On Wed, Sep 16, 2009 at 12:23 PM, Patrick Wilson-Welsh <
>

[Joshua Kerievsky]

On Wed, Sep 16, 2009 at 5:36 PM, scott.duncan <scott....@gmail.com> wrote:

So this group really isn't about "assessing" agility, then as the
title suggests,
but more about discussing how to develop an online form of experience
for
people to improve their knowledge in and skill at an Agile approach to
software
development?

Well, I wouldn't infer that Scott. 

I simply started one thread, mostly because this World of Warcraft stuff intrigues me and seems to provide a model for assessing abilities and gaining mastery. 

If you've got a specific question or topic to discuss on "assessing" agility, please start one.   For example, we could discuss how to assess TDD knowledge -- I've been doing a lot of work in that area and would be happy to discuss it.  

best

jk

[Chris Simmons]

I would think that "online" is important, for all the normal benefits (existing tools, ubiquity, ease of deployment), but I don't necessarily think that 'entertaining' or 'fun' (things that could be inferred from the WoW references) should have priority. We should optimize for usefulness.

(I say this as someone who greatly enjoys learning and assessment but doesn't relly classify them as fun)

-chris

---

From: Joshua Kerievsky
Date: Wed, 16 Sep 2009 17:43:57 -0700
To: <assessin...@googlegroups.com>
Subject: Re: Whew

[Patrick Wilson-Welsh]

Hello Joshua,

I applaud the goal; I want learning to be addictive, engaging, and fun. But I wonder if we might experiment with something less expensive as a medium than fully-immersive 3D virtual worlds?

I find fun test-driving, storytest-driving, and refactoring exercises engaging enough to pursue them with very little supportive media. I have been imagining something like a ning network with downloadable "Grand Kata" that are essentially non-trivial learning exercises of increasing challenge and difficulty. 

I think that the art of engaging learning, as many Industrial Logic systems illustrate, is in how much engagement, fun, and autodidactic addiction can be created with the least scaffolding. :)

I am also interested in making these systems open source. I don't think the entire agile programming movement would have succeeded without, for example, Eclipse and jUnit both being open source (and Java would have died for sure). 

--Patrick

[Joshua Kerievsky]

On Thu, Sep 17, 2009 at 5:06 AM, Patrick Wilson-Welsh <patrickwi...@gmail.com> wrote:

I applaud the goal; I want learning to be addictive, engaging, and fun. But I wonder if we might experiment with something less expensive as a medium than fully-immersive 3D virtual worlds?

Somehow I think we need to spend some time figuring out an agreeable way to assess agility prior to getting into implementation questions. 

The WoW ranking is interesting and I wonder how it could be applied to agile.  What would a rank of 1 be?   What would be necessary to get to rank 2?  Would achieving a rank 1 involve the same exercise for all people?   What about coding languages?   And what about non-coding "quests"?  

I find fun test-driving, storytest-driving, and refactoring exercises engaging enough to pursue them with very little supportive media. I have been imagining something like a ning network with downloadable "Grand Kata" that are essentially non-trivial learning exercises of increasing challenge and difficulty. 

I like the metaphor of Learning is a Journey.  One could go from easy, individual exercises to full-scale Agile projects in the pursuit of a higher and higher rating, and perhaps the medium could present this as a journey.    

I think that the art of engaging learning, as many Industrial Logic systems illustrate, is in how much engagement, fun, and autodidactic addiction can be created with the least scaffolding. :)  

Agreed.  

I am also interested in making these systems open source. I don't think the entire agile programming movement would have succeeded without, for example, Eclipse and jUnit both being open source (and Java would have died for sure). 

Open source makes a lot of sense.  

More soon...

best

jk

[Patrick Wilson-Welsh]

Hello Joshua,

Learning as a Journey resonates strongly with me. Adventure, comeraderie, unexpected setbacks and ordeals, digging down deep for solutions to hard problems, experiential learning, joy of arriving at a hilltop with a commanding view. 

This is what led me last year to the metaphor of "Thru-Hikes" for these non-trivial exercises I am beginning to compile. Another term might be "Grand Kata." (For a description of a thru-hike, see 

http://en.wikipedia.org/wiki/Thru-hiking )

My original notion was that each hike could be successfully more challenging (and, in Nayan's system, each that was successfully complete might win someone more XP points). 

We could encourage participants to blog about their journeys through our paths, hikes, ordeals, Grand Kata, exercises. The game could spread virally across the various social media we all enjoy. 

As to assessment:  I wish for the simplest, least gamable pass/fail assessment system that could possibly work. Basically, either you survived the hike, or "The Bear Got You" and you have to repeat it. We could establish, for a given exercise, a combination of objective static analysis metrics, and subjective critieria assessed via pairing time with an Assessor (who checks for cheating, plagiarism, the extent to which lessons have been internalized, etc). 

Man do I want Google Wave for these conversations. 

Cheers, 

--Patrick

[scott.duncan]

I guess my overall question is, how is assessment (other than self-
assessment) different from certification in
terms of the net judgment made about the results? Regardless of the
format/implementation, if some automated
system or a set of people recognizes someone else as having achieved
some "level" of ability, skill, knowledge,
etc. that's a kind of certification.

Now, developing some useful criteria that individuals/orgs could use
to self-assess and then guide them toward
improvement would interest me. Again, the delivery format could be
important for acceptance and engagement,
but I think the beef should precede the sizzle.

[Ola Ellnestam]

To me a certification comes from a single instance whereas an assessment
is a combination of evaluations.

I.e certification is more based in an authority.

Does that make any sense? Do you share my view?

//Ola

--
---------------------------------------------------------
Ola Ellnestam
Agical AB
Västerlånggatan 79, 2 tr
111 29 Stockholm, SWEDEN

Mobile: +46-708-754000
E-mail: ola.el...@agical.se
Blog: http://ellnestam.wordpress.com
Twitter: ellnestam

[Ralph Jocham]

+1

Good point.

Ralph

[Chris Simmons]

Perhaps a site like StackOverflow is a good example of experience-based community learning. Indeed, there are already categories including 'agile', 'lean', 'kanban', etc. The Badges in stackoverflow represent a sort of skill / levelling system. That said, I wouldn't treat someone with a gold badge in the project-management tag as PMI certified.

-chris

---

From: Ralph Jocham
Date: Fri, 18 Sep 2009 15:35:50 +0200
To: <assessin...@googlegroups.com>
Subject: [Assessing-Agility] Re: Whew

[Charlie Poole]

Hi Ola,

> To me a certification comes from a single instance whereas an
> assessment is a combination of evaluations.
>
> I.e certification is more based in an authority.
>
> Does that make any sense? Do you share my view?

I think an assessment is some measure of knowledge, skill or
ability. It can have one or many dimensions.

A certification combines assessment with a guarantee that some
trusted authority has made it.

Unfortunately, when we say that someone is a "Certified X"
it seems to imply more than that which was assessed in the
first place - at least in English. I think that's in part
because it seems to mean that the person can do everything
we would expect an X to do, without defining it.

Of course that's the problem with any short label applied
to a complex construction.

Charlie

[Nayan Hajratwala]

Would you treat them differently if you could see under the "gold badge" a list of all the "quests" s/he completed in order to attain that badge?

[Chris Simmons]

You can already, to a certain extent. In fact, if the types of questions this person has successfully answered are complex and varied enough, I would treat this as a very positive point towards then (if I were looking to hire, for example).

-chris

---

From: Nayan Hajratwala
Date: Fri, 18 Sep 2009 11:22:58 -0400

[Jeremy Kriegel]

for my money, agile is about continuous improvement along a spectrum, relevant to your context. For those reasons, I struggle with anyone who says one team is 'more agile' than another. Sure, one team may be further towards valuing working code over extensive documentation, but the context of another team may require more on the doc front. As long as teams are able to honestly look at their own performance and challenge their assumptions about what circumstances can and cannot be changed, then it would seem  that they are being agile. 

Perhaps exercises that facilitated that kind of unique analysis would be useful, but any kind of universal system would seem to put us right back in the certification camp.

-jer

www.methodsansmadness.com

"Be well, do good work & keep in touch."
    - Garrison Keillor

[scott.duncan]

Note that I said "Other than self-assessment" since, if someone other
than yourself is doing
the assessing, they are declaring that what they believe about your
status, quality, skill, etc.
is true. This is the same thing as a certification in terms of the
fact that a second party is
doing the assessing/claiming/certifying.

Now, when it comes to a public claim of what has been assessed/
certified, then the latter
term normally would get attached to it. In the early days of the CMM,
for example, the
assessments done were not, in the eyes of SEI, a "certification" by
them because they
did not personally conduct the assessment. But they were done by
people trained and,
in effect, "certified," by the SEI to be Lead Assessors to conduct the
assessments. So
that seemed to the industry/clients as a form of "certification." SEI
made the point, and
may continue to do so, that the SEI was certifying nothing; however,
those assessed
very often felt they had been by the Lead Assessors, at least.
Indeed, if nobody was making
a claim that they were "certifying" the accuracy of the assessments,
they would have no
meaning. The word "certification" might not be used but the act was
one of one person/body
claiming that they had confirmed that an organization had met certain
criteria.

Again, if any assessment would be used for other than purely internal/
personal purposes,
someone is going to want to know who says that the person/group
assessed met some
sort of requirements/criteria. Perhaps that is "certification" with a
small 'c' rather than a
large "C" which might imply a more public (even profit-making)
effort. But the activity is
the same for both:

some criteria are established and some way to show those criteria are
met is defined;
some one/group works to mee those crtiteria;
some other one/group claims the first one/group has met the criteria.

I believe self-assessment for improvement purposes is a highly
worthwhile goal which
avoids much of the difficulty when results are intended for any public
purpose. Once
results "go public" there are other issues to be concerned with if the
results of the
assessment are to have meaning to any third party.

On another point, regarding a one-time vs series of events, either a
certification or
an assessment can involve both:

1) There are certifications where you apply, take a test, pass, and
are certified (or
even less than that).
2) There are certifications where you must demonstrate an accredidated
educational
achievement, pass a test (or tests), and be observed in practice
before you get
certified.
3) There are assesments where someone comes in for some period of
time, checks
against some criteria, and acknowledges meeting the criteria after
that one event.
4) There are assessments which expect/require periodic reconfirmation
of criteria or
confirmation that some criteria missed on one assessment are met
after some
period of time elapses.

> E-mail: ola.ellnes...@agical.se

[Scott Dunn]

Two thoughts on certification (not assessment) -

One, I like Chris Simmons point. Like the Rails community with "Authority", how that is assessed is clear, wide and public - http://www.workingwithrails.com/browse/people/authority

Also, as Scott said, true, honest accredited certifications would require review or observation by an approved person or body. When I was credentialed as a teacher, a body of work and accomplishments was submitted, including three or four observations. Even before that, I had to pass a basic test.

IMO, if you want to make this approachable, the first level needs to be straightforward and attainable goal for apprentices. Even the PMI has created the CAPM as a step towards PMP. Microsoft has the MCP and MCAD as steps toward the MCSD, while their Certified Architect reviews something like a board review.

The SA may take knocks for the low bar it has set, but their progression path is clear. If we are truly iterative, a simple test of a basic, common body of agile knowledge is an improvement, and have a second level that, for now, simply requires review by a small panel. That may sound like a large effort, but team interviews of a candidate on a conference call aren't much different and are after the same goal - "Is this person qualified at sufficient level."

Thanks,
Scott

Scott Dunn, Certified Scrum Practitioner, PMP
http://scottdunn.blogspot.com
Software Development and Human Capital - Leadership, Agile and Strengths

[scott.duncan]

I agree with your position on assessment and it being done best along
some scale
intended to support continuous improvement. And I, too, feel using
such assessment
for comparison with others can bring along with it baggage that can be
undesirable.

To do any such comparison, though, the results of assessment must be
"public" in
some way and it would be hard to avoid results becoming public in some
fashion if
the assessment approach were successful. That is, if people found it
valuable for
themselves (or saw it useful applied by others), then there would be a
desire to use
the results to see who is being more successful. The words "more" and
"most" imply
comparison.

Of course, "more" can be used against oneself, i.e., that you are more
<something>
than you used to me. You could even decide you are the most
<something> you
believe you can be (at a given time).

Now I am sure some folks see all this discussion (and talk of
certification) as nit-picking
distraction from real improvement focus. However, as soon as you
start to formalize
any kind of judgment (which assessment and certification involves),
you begin to move
toward questions which it is sometimes better (and easier, I believe)
to answer sooner
than later. But, almost no matter what you do (the SEI is a case in
point), formalizing
the judgment will lead some people to use it to make decisions about
some one/group
without firsthand experience with the judgment process.

This is part of "crossing the chasm" in any field, I believe. Once
something gets out
of the domain of some small(er) community, you cannot easily control
how it will be used
by others -- other than to take legal action to prevent it from being
used in a way you do
not endorse/intend. That legal action itself may then be seen as an
attempt to "corner
the market" or "exclude others" or whatever.

On Sep 18, 12:55 pm, Jeremy Kriegel <jeremy.krie...@gmail.com> wrote:
> for my money, agile is about continuous improvement along a spectrum,
> relevant to your context. For those reasons, I struggle with anyone who says
> one team is 'more agile' than another. Sure, one team may be further towards
> valuing working code over extensive documentation, but the context of
> another team may require more on the doc front. As long as teams are able to
> honestly look at their own performance and challenge their assumptions about
> what circumstances can and cannot be changed, then it would seem  that they
> are being agile.
> Perhaps exercises that facilitated that kind of unique analysis would be
> useful, but any kind of universal system would seem to put us right back in
> the certification camp.
>
> -jerwww.methodsansmadness.com
>
> "Be well, do good work & keep in touch."
>     - Garrison Keillor
>

> On Fri, Sep 18, 2009 at 11:38 AM, Chris Simmons <simmons.ch...@gmail.com>wrote:
>
>
>
> > You can already, to a certain extent. In fact, if the types of questions
> > this person has successfully answered are complex and varied enough, I would
> > treat this as a very positive point towards then (if I were looking to hire,
> > for example).
>
> > -chris
>

> > ------------------------------
> > *From*: Nayan Hajratwala
> > *Date*: Fri, 18 Sep 2009 11:22:58 -0400
>
> > *To*: <assessin...@googlegroups.com>
> > *Subject*: [Assessing-Agility] Re: Whew

>
> > Would you treat them differently if you could see under the "gold badge" a
> > list of all the "quests" s/he completed in order to attain that badge?
>
> > On Sep 18, 2009, at 11:12 AM, Chris Simmons wrote:
>
> > Perhaps a site like StackOverflow is a good example of experience-based
> > community learning. Indeed, there are already categories including 'agile',
> > 'lean', 'kanban', etc. The Badges in stackoverflow represent a sort of skill
> > / levelling system. That said, I wouldn't treat someone with a gold badge in
> > the project-management tag as PMI certified.
>
> > -chris

> > ------------------------------
> > *From*: Ralph Jocham
> > *Date*: Fri, 18 Sep 2009 15:35:50 +0200
> > *To*: <assessin...@googlegroups.com>
> > *Subject*: [Assessing-Agility] Re: Whew
>
> > +1
> > Good point.
>
> > Ralph

> >> E-mail: ola.ellnes...@agical.se
> >> Blog:http://ellnestam.wordpress.com

> >> Twitter: ellnestam- Hide quoted text -
>
> - Show quoted text -