Introducing RMM, the Rails Maturity Model
Obie Fernandez
certification and I advocated an organization to set guidelines for
what the characteristics and measurements that set apart a successful
and disciplined Rails shop apart from the unwashed masses. Rather than
trying to certify individuals (too easy to game!) I think it would be
useful to have a certification process for organizations that involved
an actual extensive interview process and audit of code and practices,
with associated scorecards and registration in some sort of official
directory. Yes, you would have to pay handsomely to get this
certification, and the result would not necessarily be what you
expect. The details for this is all very rough and frankly I don't
know the value of it, just an idea that's been in the back of my mind
for a long time so I'm putting it out there for debate.
The name and concept is somewhat influenced by CMM, which I suppose is
anathema to most Agilistas. I'm the first to admit that I don't know
much about CMM other than it provides a scoring system for
organizations.
In a Rails context, you would establish a scorecard that categorized
your shop on a scale of 0 to 3 (or whatever, just thinking out loud
here)
RMM0 "aka Cowboy level"
- No formal development process
- No test coverage
- No standardized business practices
- Static analysis failures
RMM1 and RMM2 would have to be something in-between. RMM1 is
considered negative. RMM2 is considered positive.
RMM3 "aka Master level" (Purposely exclusive territory here, I can
imagine that only a handful of shops in the world could achieve this
level!)
- Agile software development practices
- 100% test coverage WITH THE APPROPRIATE TYPES OF TESTS (For
instance, at Hashrocket lately we are doing much less unit testing at
the MVC level because automated acceptance and integration testing
with Cucumber is so powerful and effective.
- 100% pair-programming (muahaha)
- Formal and standardized business practices
- Institutionalized continuous learning and process improvement
- Positive customer testimonials
- Successful deployment of Rails application(s) with substantial scaling demands
Incidentally, I'm bringing this up for discussion, but I might be
interested in a joint-venture along these lines with an out of work
senior Rails developer that feels like taking the idea and running
with it. (Rick?) In fact I can envision the idea expanding to the
point where being a RMM auditor could be a profitable little part-time
gig for Rails freelancers with the right personality type and
enthusiasm.
Obie Fernandez
CEO & Founder | Hashrocket
904.435.1671 office
404.934.9201 mobile
Hashrocket, Inc.
320 N 1st Street
Suite 712
Jacksonville Beach, FL 32250
Will Sargent
> imagine that only a handful of shops in the world could achieve this
> level!)
> - Agile software development practices
> - 100% test coverage WITH THE APPROPRIATE TYPES OF TESTS (For
> instance, at Hashrocket lately we are doing much less unit testing at
> the MVC level because automated acceptance and integration testing
> with Cucumber is so powerful and effective.
Who decides what's appropriate? According to what metrics? Is this
testing all possible branches? And if you're doing integration
testing, how do you cover all possible datasets? This just seems way
too hard to quantify and too easy to game.
> - 100% pair-programming (muahaha)
Again, this is way too easy to game; you could hire a seasoned expert
and a "net-loss" programmer and stick them together, and you'd have
"pair programming." And if this means that XP is the One True Agile
Process, that cuts Crystal Clear, Scrum, DSDM and others completely
out of the loop, and that can't be right...
> - Formal and standardized business practices
> - Institutionalized continuous learning and process improvement
> - Positive customer testimonials
Again, may have little or nothing to do with the quality of the work
done. There have been plenty of technical snowjobs done that the
client didn't discover until well after.
Honestly, I think the only thing that counts is the positive
testimonials of your peers; by which I mean that the next engineer
coming in after you've left, taking over your code, should be happy
with it and can stand up and defend it as if it were his own.
There have been places I've come into where I've seen good quality
code holding the system together, and I didn't have to ask who had
written it; it was obvious just from looking at it that these guys
were from a pro consultant shop.
Ironically, what really made it obvious to me was not that they'd used
DSLs, metaprogramming or self-modifying code; it was that they clearly
knew when it was appropriate to use those tools and when not to. In
some cases they specifically went with a lower tech solution because
it was more maintainable, even though it wasn't quite as cool as what
they wishfully described in the comments. They'd clearly considered
their audience, and were writing with the next guy in mind.
> - Successful deployment of Rails application(s) with substantial scaling demands
See, this is so nebulous, and so hard. Scaling is not something you
can do in the context of a six month project, and the nature of a
software application is to have bottlenecks and features that pop up
in the oddest places in the strangest ways. You can quote the CAP
theorum and wave memcached around, but I don't see how you can say
"substantial scaling" without quantifying it further.
The reason that CMM is so reviled amongst programmers is because the
system is inherently corrupt. The CMM evaluators are paid by the
people who are being evaluated, and they're the ones who make an
industry out of giving people high marks for having a process. Having
a high CMM process doesn't mean you actually produce software more
efficiently (although there's a Construx presentation that says
differently, I believe they're taking the evaluator's reports and
repackaging it) -- it just means that you have a well defined process,
and (at the highest level) can predict the number of bugs you get out
of that process.
Will.
Obie Fernandez
>
>> RMM3 "aka Master level" (Purposely exclusive territory here, I can
>> imagine that only a handful of shops in the world could achieve this
>> level!)
>> - Agile software development practices
>> - 100% test coverage WITH THE APPROPRIATE TYPES OF TESTS (For
>> instance, at Hashrocket lately we are doing much less unit testing at
>> the MVC level because automated acceptance and integration testing
>> with Cucumber is so powerful and effective.
>
> Who decides what's appropriate? According to what metrics? Is this
> testing all possible branches? And if you're doing integration
> testing, how do you cover all possible datasets? This just seems way
> too hard to quantify and too easy to game.
The auditor, based on guidelines established by a group like this one?
Believe me I realize the challenges in making something like this work
in practice. The auditors would have to have "know it when I see it"
capabilities for judging code quality.
>> - 100% pair-programming (muahaha)
>
> Again, this is way too easy to game; you could hire a seasoned expert
> and a "net-loss" programmer and stick them together, and you'd have
I run one of the only shops that I know of that does proper and
effective pair-programming ALWAYS, even onsite with clients, etc.
Hence the evil laugh... the only other shop that I know for a fact
operates similarly is Pivotal. I admit that it might not be vital it
is to attach this criterion to RMM -- on the other hand, good Agile
practices are indispensible nowadays.
> "pair programming." And if this means that XP is the One True Agile
> Process, that cuts Crystal Clear, Scrum, DSDM and others completely
> out of the loop, and that can't be right...
Flawed reasoning. Pair-programming is included in XP but is really its
own thing apart from the particular Agile implementation you choose.
Besides, XP is the only Agile practice that I'm aware of which
"manages down" to the developer level and dictates how they operate.
Scrum and others manage up in terms of releases and stakeholders.
Right? We pick and choose whatever works from XP, Scrum, etc..
>> - Formal and standardized business practices
>> - Institutionalized continuous learning and process improvement
>> - Positive customer testimonials
>
> Again, may have little or nothing to do with the quality of the work
> done. There have been plenty of technical snowjobs done that the
> client didn't discover until well after.
Dunno about that. You might be able to snow your clients, but you
shouldn't be able to snow the auditor.
>
> Honestly, I think the only thing that counts is the positive
> testimonials of your peers; by which I mean that the next engineer
> coming in after you've left, taking over your code, should be happy
> with it and can stand up and defend it as if it were his own.
Agreed in principle. In practice, the client devs taking over are
usually not as good as we are and don't have a basis to judge, nor do
I see a good system for tallying that data. Actually, we've known
client devs to fuck up our code almost immediately. One of the worst
feelings in this business... :-/
> There have been places I've come into where I've seen good quality
> code holding the system together, and I didn't have to ask who had
> written it; it was obvious just from looking at it that these guys
> were from a pro consultant shop.
That's what I would want the auditor to do.
>
> Ironically, what really made it obvious to me was not that they'd used
> DSLs, metaprogramming or self-modifying code; it was that they clearly
> knew when it was appropriate to use those tools and when not to. In
> some cases they specifically went with a lower tech solution because
> it was more maintainable, even though it wasn't quite as cool as what
> they wishfully described in the comments. They'd clearly considered
> their audience, and were writing with the next guy in mind.
True.
>> - Successful deployment of Rails application(s) with substantial scaling demands
>
> See, this is so nebulous, and so hard. Scaling is not something you
> can do in the context of a six month project, and the nature of a
> software application is to have bottlenecks and features that pop up
> in the oddest places in the strangest ways. You can quote the CAP
> theorum and wave memcached around, but I don't see how you can say
> "substantial scaling" without quantifying it further.
I agree. Needs a lot more thought, but in theory there must be some
way of establishing whether a shop has delivered real, scalable
solutions rather than just toy apps.
> The reason that CMM is so reviled amongst programmers is because the
> system is inherently corrupt. The CMM evaluators are paid by the
> snip...
Totally. What if you had to pay a non-refundable fee to RMM upfront in
order to get evaluated. We're not such a huge "industry" that we
couldn't use that money to compensate a neutral and properly-qualified
auditor -- one that wouldn't be subject to coercion or bribing.
Rick Bradley
<obiefe...@gmail.com> wrote:
>> The reason that CMM is so reviled amongst programmers is because the
>> system is inherently corrupt. The CMM evaluators are paid by the
>> snip...
>
> Totally. What if you had to pay a non-refundable fee to RMM upfront in
> order to get evaluated. We're not such a huge "industry" that we
> couldn't use that money to compensate a neutral and properly-qualified
> auditor -- one that wouldn't be subject to coercion or bribing.
We've discussed the potential of a somewhat objective audit before.
One of the things I'm personally still unclear on is whether doing
these audits is something that works well in a distributed (aka
"meatcloud") fashion, or whether it works better in a centralized
fashion. Does the emergent intelligence of crowds (the same emergent
intelligence that drives large-scale open source to success) sift out
the identify of the high quality software teams, or can such high
quality only really be discerned by other high-quality software teams?
If the latter is the case there's a clear bootstrapping issue.
Regardless, because it may nonetheless be the case that the builders
of good software can only be recognized by rare quality practitioners,
there's the unfortunate ballast of experience: every certification or
large-scale auditing process in software that I can recall has, for
all practical purposes, failed due to either widespread skepticism,
outright corruption, or inherent bias.
It's been my limited experience in auditing software and working with
(and after) other developers, that there are a number of objective
metrics one can put into play. Any of them can be "gamed", as has
been hinted earlier in this discussions. In aggregate, however, it is
very difficult to game the majority of them.
Furthermore, In reality, where we happen to live, there seems to
actually be a power law at work: the vast majority of software and
teams plying our trade are so bad that they fail nearly every metric
we could reasonably apply. Only the vanishing minority of products
and practitioners pass even a single gameable metric. Yet we
ultimately have little means of discerning for ourselves, for our
clients (or for differentiating for our future clients) the difference
between one or the other.
To be able to deploy a standard of comparison (whether distributed or
centrally administered and computed) that forces products or teams to
either score in the vast gutter of mediocrity, or work diligently to
at least game performance on a number of axes (when such gaming is,
realistically, often more difficult than just working to better
oneself) to separate from the pack -- would imply a vast improvement
in the current and long-standing state of affairs.
In my limited experience the difference is usually striking and
obvious: "this is trash" vs. "wow, this is pretty good". In the
Rails community the latter is really only elicited by the products of
about 10-20 shops (those shops Obie was trying to catalog in one of
his solicitations this evening). It would be great if we could get
that number up to 50, 100, or even 1000. I'd like to believe that
putting a workable set of metrics out there would eventially bias
progress in that direction.
Best,
Rick
Rick Bradley
>> system is inherently corrupt. The CMM evaluators are paid by the
>> snip...
>
> Totally. What if you had to pay a non-refundable fee to RMM upfront in
> order to get evaluated. We're not such a huge "industry" that we
> couldn't use that money to compensate a neutral and properly-qualified
> auditor -- one that wouldn't be subject to coercion or bribing.
We've discussed the potential of a somewhat objective audit before.
Obie Fernandez
> intelligence that drives large-scale open source to success) sift out
> the identify of the high quality software teams, or can such high
> quality only really be discerned by other high-quality software teams?
It will not work on a distributed basis because of issues related to
privacy and confidentiality. I simply can't publish 90% of the work we
do in any significant way because my clients would freak out. On the
other hand, getting an auditor in here with a signed NDA for a day or
two to review our code and processes seems doable.
>
> If the latter is the case there's a clear bootstrapping issue.
Why? Because it would take an objective auditor to step up to the
plate? Because he would need to be vetted by some initial group, which
itself would have nobody to audit them? I don't think that challenge
is unsurmountable.
> Regardless, because it may nonetheless be the case that the builders
> of good software can only be recognized by rare quality practitioners,
> there's the unfortunate ballast of experience: every certification or
> large-scale auditing process in software that I can recall has, for
> all practical purposes, failed due to either widespread skepticism,
> outright corruption, or inherent bias.
I wouldn't consider that a reason not to try.
> It's been my limited experience in auditing software and working with
> (and after) other developers, that there are a number of objective
> metrics one can put into play. Any of them can be "gamed", as has
> been hinted earlier in this discussions. In aggregate, however, it is
> very difficult to game the majority of them.
That's right.
> Furthermore, In reality, where we happen to live, there seems to
> actually be a power law at work: the vast majority of software and
> teams plying our trade are so bad that they fail nearly every metric
> we could reasonably apply. Only the vanishing minority of products
I suspect that's everywhere, not just where you live. We are asked to
rescue atrocities on a regular basis.
> and practitioners pass even a single gameable metric. Yet we
> ultimately have little means of discerning for ourselves, for our
> clients (or for differentiating for our future clients) the difference
> between one or the other.
Clearly published standards would go a long way towards starting to
establish criteria for judgment. It will be a challenge to write the
criteria descriptions in such a way that it educated interested
parties about what they should be seeking. If it succeeded at that,
the project would be an educational resource and net positive for the
overall community. Wouldn't it?
> To be able to deploy a standard of comparison (whether distributed or
> centrally administered and computed) that forces products or teams to
> either score in the vast gutter of mediocrity, or work diligently to
> at least game performance on a number of axes (when such gaming is,
> realistically, often more difficult than just working to better
> oneself) to separate from the pack -- would imply a vast improvement
> in the current and long-standing state of affairs.
Right! I fully expect that if such a system was put into place, that
only a handful of companies would qualify for the top-level RMM
certification. Maybe a slightly larger would qualify for the RMM level
right below top-level, and feel that it is a selling point. To the
extent that the rest of our "industry" admired the people at the top,
they might work towards qualifying. Idealistic, yes...
> In my limited experience the difference is usually striking and
> obvious: "this is trash" vs. "wow, this is pretty good". In the
> Rails community the latter is really only elicited by the products of
> about 10-20 shops (those shops Obie was trying to catalog in one of
> his solicitations this evening). It would be great if we could get
> that number up to 50, 100, or even 1000. I'd like to believe that
> putting a workable set of metrics out there would eventially bias
> progress in that direction.
Yes, yes, yes! Amen, brother! Thanks for clearly expressing what I was
suggesting!
Cheers,
Obie
Rick Bradley
<obiefe...@gmail.com> wrote:
>> In my limited experience the difference is usually striking and
>> obvious: "this is trash" vs. "wow, this is pretty good". In the
>> Rails community the latter is really only elicited by the products of
>> about 10-20 shops (those shops Obie was trying to catalog in one of
>> his solicitations this evening). It would be great if we could get
>> that number up to 50, 100, or even 1000. I'd like to believe that
>> putting a workable set of metrics out there would eventially bias
>> progress in that direction.
>
> Yes, yes, yes! Amen, brother! Thanks for clearly expressing what I was
> suggesting!
:-)
My suggestion, then, would be that the process be two-tiered. The
first tier is a suite of basically automated metrics that most
software in the field will probably fail spectacularly: test
coverage, cyclomatic complexity, lack of revision control :-),
revision control code churn, class/module/function coupling, etc.
There's a maintained, publicly available package of tools along with a
set of accompanying standards documents. Anyone can pull the suite
tools, run them over their software, and derive an aggregate metric
that's gameable, but tends to indicate that at least something is
probably going right if the scores are high. The tools needed to do
this are in large part already available and open source.
The second tier would be an NDA-agreeable, rigorous professional audit
of code and/or practices, conducted by expert(s) in developing and
assessing software and practices. I'm presuming here that we dodge
the bootstrap process by saying (a) those who care probably can
identify the proper auditors, and (b) if we make the audit process
open enough then a bad bootstrapping could be supplanted by better
experts following the same recipe.
The gateway to the second tier is a reasonable score on the first
tier, as well as presumably some sort of cost barrier -- to both
defray costs and to squelch overwhelming demand to use the auditing
resource. In theory one could get a limited per-application expert
audit or a more comprehensive team-and-process audit.
The net result, presuming the two tiers are evolved over time to try
to promote and identify the best techniques and methodologies, would
be to provide a way for developers to tune their processes cheaply, to
really hone their craft (more expensively), and to be able to
differentiate quality from hackery.
Best,
Rick
tmor...@gmail.com
I strongly believe that Rails is being held back by a shortage of
skilled developers.
One of the most common questions our sales people hear is "Do you know
any good developers?" :-)
One small nitpick: I don't think it's fair to exclude the skilled
single developers out of the highest level certification because they
cannot pair program. I think pairing works, but good code is good code
at the end of the day. A strong organization with multiple developers
working on a project is a benefit, to be sure, but I don't see it as
part of a skills/deeds based certification.
I'd say the same thing about Positive customer testimonials. After
all, a dev shop with a solid balance sheet is a benefit to many
customers, but are we going to require open books before
certification? :-)
--
-- Tom Mornini
-- Founder and CTO
-- Engine Yard, Inc.
Dr Nic
foobar" and "cap deploy".
My gut feeling reading through Obie's "Hashrocket Licensing
Certification" post is that Expert Rails == Expert Agile. No ifs/buts/
maybes. Or perhaps Expert Rails requires Expert Agile. I disagree. As
the owner of a Rails consultancy, Mocra, where we wholeheartedly sell
agile to clients, I nonetheless rebuke the notion that pair
programming is a requirement for an Expert Rails Developer or even an
Expert Rails Consultancy.
The aim of being an "Expert Developer" is to get the job done upfront
and long term. An Expert Rails Developer would know how to use Rails
to do this. An Expert Agile Developer would use Agile processes to do
this. My guess is that Scrum (tm) and other agile communities have
their certifications and two-day mastery courses/certifications all
bedded down.
In my opinion, any Rails/Ruby/JavaScript/Unix/TDD/BDD super combo
certification should be independent of development process. I don't
think any "Learning Rails" or "Expert Rails" books have chapters on
how to pair program, and at best their references to testing is a
summary of the test APIs. I don't think DHH pair programs.
How many people write unit tests for their JavaScript? 5-10% maybe. I
believe that its more important that an Expert Rails developer is an
Expert JavaScript developer than that they subscribe to any
development process/philosophy. Most Rails developers still treat
their JavaScript like a toy. Or like icing on a cake: you slap it on
the top and hope it hasn't melted in the morning.
I do pair program. Some times. And other times I don't. I always TDD
as long as I can figure out how the f@#$ to do it for the given
rubygem/cocoa GUI/rails webapp/javascript badge that I'm working on.
How many people write integration tests for RubyGems or unit tests for
JavaScript? Both are more critical skills for an Expert Rails
developer than the ability to work with a 2nd person 8 hrs a day x 5
days a week.
Penultimately, if there is to be the notion of a "consultancy"
certification: if you have 5 developers when you spend the 1 mth
attempting to get certified, and over the next 12 mths you lose 1 and
hire 10 others, what then? Does the staff who left carry their
certification with them? Do the new 10 staff automatically get the
consultancy's rating?
Finally, how does an "official" "We're an Awesome Rails consultancy"
badge on their website change the pure dynamics of capitalism? Good
companies will get referrals, get more business and stay around. Bad
companies will just get maintenance jobs on old PHP sites. If we have
a "We're so Awesome" official institution then we're going to take our
eyes off the ball - writing awesome code - and get back to dark days
of Rails in the beginning when every 3rd blog post was defending
Rails. Except now we'll be attacking or defending the "We're so
Awesome" brand: people who have the official mark will wish to defend
their expensive certification, and people who don't care about it will
spend their time rubbishing the very concept. Our community will look
silly. More importantly we won't be Awesome anymore.
Cheers
Dr NIc
On Feb 13, 4:45 pm, "tmorn...@engineyard.com" <tmorn...@gmail.com>
wrote:
Bryan Liles
Robert Dempsey
Also, going on the subject of testing, as Bryan well knows, that's a controversial topic in of itself. Do we test views? Would a certification impose that on us? What level of unit testing is acceptable? Salesforce goes for 70%+ test coverage (with 100% passing tests). They're huge, have lot's of money, and changed their industry. Should that then be the metric, even though they aren't doing Rails?
No client has ever come to us and asked about certification, insurance, etc. They look at what we've done, talk to our clients, and have candid conversations with me.
Having said that, large enterprises might care about certifications, but I have yet to have any of our huge clients ask for one.
Sincerely,
Robert Dempsey, CEO
Atlantic Dominion Solutions
http://adsdevshop.com
http://twitter.com/rdempsey
Phone: 321-274-4684
Robert Dempsey
> Yes, you would have to pay handsomely to get this
> certification, and the result would not necessarily be what you
> expect.
limit competition in the Rails space
Obie - can you clarify what you mean by this?
- Robert Dempsey
Colin Bartlett
> No client has ever come to us and asked about certification,
> insurance, etc. They look at what we've done, talk to our clients,
> and have candid conversations with me.
Agreed.
Clients do ask about insurance in contracts and whatnot. But no one's
ever demanded certification. To be fair, none exists for Rails, so
there's nothing to ask for. But even when we were an all-Microsoft
shop, we were never Microsoft Certified. But no one cared. We did
great work and that's what mattered.
I say, let the markets decide.
Colin
Colin A. Bartlett
Kinetic Web Solutions
http://blog.kineticweb.com
Obie Fernandez
Sorry, that was only meant to say that you can't *buy* a top-level RMM
grading. That's all.
Obie Fernandez
http://en.wikipedia.org/wiki/Capability_Immaturity_Model
smeade
> shop, we were never Microsoft Certified. But no one cared. We did
> great work and that's what mattered.
>
> I say, let the markets decide.
>
> Colin
because of the marketing power. The Microsoft brand is one of the
most powerful in the world, beating out Coke and GE in some cases.
So, companies go through the hoops and pay the big bucks to become
associated with the brand. It often does little to ensure success of
any given project.
Rails, on the other hand, explicitly and purposefully does not have
such dominant brand awareness in end-users or clients. In
http://www.loudthinking.com/posts/6-why-theres-no-rails-inc David
says, "The growth of the Rails ecosystem has been staggering. There
are so many shops out there offering Rails consulting and training. I
believe part of that proliferation is due to the fact that there's no
core-group monopoly that can dominate the market." You could easily
drop in "Rails certification monopoly" in place of core-group monopoly
and have a similar growth impacting effect.
RMM could be a good idea for shops that want to improve. Continuous
improvement is a good thing and RMM could provide a framework for
that. Yet the certification piece would be completely and 100% a
marketing tool granted by the Rails-shop certification monopoly. In
that light, it starts to sound like Microsoft Gold Partner
certification without the power of the brand.
_ Scott
Chris W
where certifications are almost required in order to get new business
(ISO 9000, CMMI, etc). It is very costly to get certificated and to
keep the certification. There is a whole industry around doing CMMI
based training and assessment. While our site is CMMI level 5, I know
of projects that don't follow CMMI level 1 principles.
Obie, I would read this paper about CMMI and Agile to get an idea of
how the very process oriented companies look at Agile. It would be
interesting to see where Hashrocket might rank within the CMMI scale
(at least a 3 and maybe a 5 in my mind)
http://www.sei.cmu.edu/news-at-sei/whats-new/cmmiagile.html
Maybe a certification is overkill but a website where companies can
register and document the level of process they implement allowing for
customers to provide feedback so they could get ranked. If that kind
of site took off, it would give prospective customers a way to compare
apples to apples.
Just my 2 cents...
Chris
On Feb 13, 9:38 am, Obie Fernandez <obiefernan...@gmail.com> wrote:
bcardarella
you'll find is that people/shops will only be working towards that
model. The strength in the Ruby community has been innovation and the
freedom to try new things and ideas. If clients begin to believe that
the only good shops are those that are RMM compliant you will quickly
see a death to this spirit of innovation.
- Brian Cardarella
On Feb 13, 10:28 am, Chris W <chris.d.willi...@gmail.com> wrote:
> I am a -1 on the wholecertificationidea. I work for a large company
> (ISO 9000, CMMI, etc). It is very costly to get certificated and to
> of projects that don't follow CMMI level 1 principles.
>
> Obie, I would read this paper about CMMI and Agile to get an idea of
> how the very process oriented companies look at Agile. It would be
> interesting to see where Hashrocket might rank within the CMMI scale
> (at least a 3 and maybe a 5 in my mind)http://www.sei.cmu.edu/news-at-sei/whats-new/cmmiagile.html
>
Obie Fernandez
would entail from the concept of official certification, since
certification implies a bunch of bad stuff and politics.
smeade
> But even when we were an all-Microsoft
> shop, we were never Microsoft Certified. But no one cared. We did
> great work and that's what mattered.
can sort that out.
In "Why there's no Rails Inc" http://www.loudthinking.com/posts/6-why-theres-no-rails-inc,
"The growth of the Rails ecosystem has been staggering. There are so
many shops out there offering Rails consulting and training. I believe
part of that proliferation is due to the fact that there's no core-
like you'd like to substitute 'core-group monopoly' with 'Rails-
certification monopoly'.
In any industry, certification is all about and only about marketing.
Firms go through the hoops and pay big bucks to be a Microsoft Gold
partner (which requires certification) because the Microsoft brand is
so strong - stronger than Coke and GE in many cases. RMM could be a
good idea for firms that want a framework for continuos improvement,
certainly. Yet the RMM certification piece seems like Microsoft Gold
without the power of a brand.
Nick Plante
..nap
bcardarella
still implied and the end will still be the same.
I think a better idea would be to follow the model of the medical
community. Have some sort of apprenticeship process for individual
developers. Let the shops worry about the skills of those that are
being hired and let the results build a reputation that speak to the
clients.
This RMM idea feels too much like a 4-star rating process that
companies can brag about.
- Brian Cardarella
On Feb 13, 10:49 am, Obie Fernandez <obiefernan...@gmail.com> wrote:
> Let's disconnect the concept of a Rails Maturity Model and what that
>
> On Fri, Feb 13, 2009 at 10:28 AM, Chris W <chris.d.willi...@gmail.com> wrote:
>
> > I am a -1 on the wholecertificationidea. I work for a large company
> > (ISO 9000, CMMI, etc). It is very costly to get certificated and to
> > of projects that don't follow CMMI level 1 principles.
>
> > Obie, I would read this paper about CMMI and Agile to get an idea of
> > how the very process oriented companies look at Agile. It would be
> > interesting to see where Hashrocket might rank within the CMMI scale
> > (at least a 3 and maybe a 5 in my mind)
> >http://www.sei.cmu.edu/news-at-sei/whats-new/cmmiagile.html
>
Mike Gunderloy
that you can disconnect any sort of maturity model from "bad stuff and
politics" seems rather questionable.
Mike
steve....@gmail.com
individual certifications, as well as certifications of
organizations. As a longtime Microsoft developer I obtained a number
of certifications (MCSD, MCSDBA, MCAD) and while they helped me to get
some more cash out of my employer they were pretty useless. I can't
tell you how many people I interviewed that held all sorts of
certifications but couldn't complete the most basic coding
assignment. At an organizational level my employer at the time was
also a Microsoft Certified Partner. The only reason for applying and
obtaining that level of certification was to get more business. Being
certified didn't make us anymore qualified to build great software
then a guy working out of his basement it just meant we had the money
to buy into the program and send our consultants to get their certs,
and he didn't.
These past experiences make it difficult for me to value any sort of
certification. I see certifications as a way for folks to make money
by giving those who wish to be certified a false sense of
accomplishment and belonging. I don't see a way for it to happen that
doesn't involve some seriously negative ramifications for all those
involved. Sure, it might make someone some money, and sure it might
make a big corporate client feel all warm and fuzzy by working with a
"Rails Certified Master", but at the end of the day, it won't make
anyone more likely to succeed at building and delivering great
software. Building great software is too hard to be compiled down to
a checklist, and I, for one, am 100% against the idea of putting
something together that implies otherwise.
Cheers,
Steve
Jonathan Weiss
> This is a terrible idea. Anytime you create a certification model what
> you'll find is that people/shops will only be working towards that
> model. The strength in the Ruby community has been innovation and the
> freedom to try new things and ideas. If clients begin to believe that
> the only good shops are those that are RMM compliant you will quickly
> see a death to this spirit of innovation.
+1
Jonathan
--
Jonathan Weiss
http://blog.innerewut.de
http://twitter.com/jweiss
Mike Mangino
On Feb 13, 2009, at 10:50 AM, smeade wrote:
>
> On Feb 13, 7:26 am, Colin Bartlett <colinbartl...@gmail.com> wrote:
>> But even when we were an all-Microsoft
>> shop, we were never Microsoft Certified. But no one cared. We did
>> great work and that's what mattered.
>
> I agree with Colin. Great results is what matters and a free market
> can sort that out.
It's a nice thought, but there isn't a well functioning market here.
Transaction costs are non-trivial, the products aren't commodities and
the information asymmetry is huge.
This situation isn't unique to software. I had a horrible time trying
to find an accountant and an attorney because I don't know enough
about either field to judge the worthiness of my potential providers.
Our clients are in the same position. After several mis-starts, I
ended up finding good people via recommendations from people I trust.
That's the same way we get most of our business. It allows us to go
into an engagement with a little trust. Of course, there is a self
selection problem. Most people don't give out references of clients
that weren't happy. (We do, but I think we're in the minority there)
I mention other fields because this problem is universal and isn't
well solved in any field. I got my first attorney from a trade
organization similar to the one proposed here. He wasted a ton of my
money and never delivered the goods. The trade organization is only
helpful if the people who run it our trustworthy. That leads us back
to the same information asymmetry.
I wish I had a brilliant solution to this problem. I would just
recommend that before an organization is created, you look at what
problem you're trying to solve and think closely about whether or not
the organization solves the problem.
Mike
--
Mike Mangino
http://www.elevatedrails.com
Roland Moriz
> bcardarella wrote:
>> This is a terrible idea. Anytime you create a certification model what
>> you'll find is that people/shops will only be working towards that
>> model. The strength in the Ruby community has been innovation and the
>> freedom to try new things and ideas. If clients begin to believe that
>> the only good shops are those that are RMM compliant you will quickly
>> see a death to this spirit of innovation.
>
> +1
>
+1
Roland
--
Moriz GmbH
Hedwig-Dransfeld-Allee 14
80637 München
Tel: 089/78795079 (AB)
Vertretungsberechtigter
Geschäftsführer:
Roland Moriz
Registergericht: Amtsgericht München
Registernummer: HRB 174 294
USt-ID: DE260422784
Phlip
>> +1
> +1
The ultimate argument against certification is decertification.
Do you really want to give so much power to some Ubergeek that they can
_de_certify you, without appeal? For the certification to gain industry
acceptance, it must have teeth. And if clients discover that the best works of
art are hung in the "Salon de Refuses" (Gallery of Rejects), then they will
start shopping there, instead!
Robby Russell
As I mentioned in the other thread, the concept is interesting. The
responses have been very interesting as well. However, with any
process/change, it's important for us to assess the cost that would
result.
Certification (or whatever you want to call it) would costs teams time
and money. This then gets passed on to their clients, which raises
their costs. Now, you could argue that clients who work with shitty
development teams will incur a big cost as well, but we don't have
enough real data to suggest that a certification process would save
clients money in the long run.
Given the situations we face with the economy, we need to be looking
at ways to remove unnecessary costs (time+money) from our processes
and my initial response to the concept.. is that this would have the
opposite result. As others have said, there is no *right* way
(globally) for designing and developing web applications with Ruby on
Rails. I suspect most of our teams work in very different ways and
that's up for debate on whether that is a good thing or not.
What is the problem that this idea is trying to solve? I don't think
that's been clearly established and would be more interested in
learning more about that before we jump to possible solutions.
Robby
--
Robby Russell
Chief Evangelist, Partner
PLANET ARGON, LLC
design // development // hosting w/Ruby on Rails
http://planetargon.com/
http://robbyonrails.com/
http://twitter.com/planetargon
aim: planetargon
+1 503 445 2457
+1 877 55 ARGON [toll free]
+1 815 642 4068 [fax]
Jon Dahl
> Let's disconnect the concept of a Rails Maturity Model and what that
> would entail from the concept of official certification, since
> certification implies a bunch of bad stuff and politics.
>
is a stamp of approval given by a paid auditor, it's going to be an
official certification.
What about disconnecting the RMM idea from certification altogether by
making it a set of practices and not an audited status? In other
words, teams can aspire to a certain level by adopting a set of
practices. This doesn't "guarantee" quality, but then again, neither
does an audit by an official RMM certifier. Instead, they make a claim
about their practices ("We follow X, Y, and Z"), and their customers,
peers, and the team members themselves hold the team accountable.
Removing money from the mix is probably the best way to avoid the
political problems. And for those so inclined, there is probably more
money to be had in training than in certification itself.
Will Sargent
>
> This is a terrible idea. Anytime you create a certification model what
> you'll find is that people/shops will only be working towards that
> model. The strength in the Ruby community has been innovation and the
> freedom to try new things and ideas. If clients begin to believe that
> the only good shops are those that are RMM compliant you will quickly
> see a death to this spirit of innovation.
There's a great book, _Measuring and Measuring Performance in
Organizations_ which makes this point exactly; if you grade people to
a metric, they will optimize for the metric rather than for the
intangible quality that the metric is intended to measure.
Still, according to Tom Gilb, "anything you need to quantify can be
measured in some way that is superior to not measuring it at all."
Will.
Robby Russell
* http://macournoyer.com/rmm/
Interesting to see the commentary on Twitter.
* http://search.twitter.com/search?q=rmm
Robby
Obie Fernandez
>
> An interesting take on the idea.
>
> * http://macournoyer.com/rmm/
>
> Interesting to see the commentary on Twitter.
>
> * http://search.twitter.com/search?q=rmm
>
Hysterical.
David
Maybe a better place to start would be with a (complete-ish) list of
what would be expected of each level. And discussions on how to move
from Level 0 to Level 1, etc.
That way, instead of being exclusive and rejecting some those that
didn't meet some criteria, we would be inclusive and help the entire
ruby cummunity. A nice side effect would be making the "average"
rubyist more proficient.
Who know, maybe book could come out of it...
- David
On Thu, Feb 12, 2009 at 10:45 PM, Obie Fernandez
<obiefe...@gmail.com> wrote:
>
> A couple of Railsconfs ago, Courtenay and I did indeed discuss
> certification and I advocated an organization to set guidelines for
> what the characteristics and measurements that set apart a successful
> and disciplined Rails shop apart from the unwashed masses. Rather than
> trying to certify individuals (too easy to game!) I think it would be
> useful to have a certification process for organizations that involved
> an actual extensive interview process and audit of code and practices,
> with associated scorecards and registration in some sort of official
Matt Aimonetti
think a lot of people raised valid concerns, but I would like to add
to that and mention people/companies outside of the US. (I hear they
*do* exist). Even if, let say, we could agree about a good way of
defining different level of certifications, how would you cover the
developing countries?
I see the certification programs as yet another way to discriminate.
However, I'm not against the idea of partnership which I feel is much
less discriminating.
- Matt
Jeremy McAnally
market and gain respect in that sector will inevitably dominate it,
leading to massive revenues with little incentive to improve the
metrics used and little incentive to avoid it becoming "corrupt."
Seeing any sort of praise for the MS certifications is pretty funny,
given that you can just buy the MS books, memorize them cover to
cover, pay MS the exam fees, to work with MS products for MS clients.
It's just another money maker for them, and they care very little for
how qualified people are.
--Jeremy
http://jeremymcanally.com/
http://entp.com/
http://omgbloglol.com
My books:
http://manning.com/mcanally/
http://humblelittlerubybook.com/ (FREE!)
