Introducing RMM, the Rails Maturity Model

196 views
Skip to first unread message

Obie Fernandez

unread,
Feb 12, 2009, 10:45:09 PM2/12/09
to rails-b...@googlegroups.com
A couple of Railsconfs ago, Courtenay and I did indeed discuss
certification and I advocated an organization to set guidelines for
what the characteristics and measurements that set apart a successful
and disciplined Rails shop apart from the unwashed masses. Rather than
trying to certify individuals (too easy to game!) I think it would be
useful to have a certification process for organizations that involved
an actual extensive interview process and audit of code and practices,
with associated scorecards and registration in some sort of official
directory. Yes, you would have to pay handsomely to get this
certification, and the result would not necessarily be what you
expect. The details for this is all very rough and frankly I don't
know the value of it, just an idea that's been in the back of my mind
for a long time so I'm putting it out there for debate.

The name and concept is somewhat influenced by CMM, which I suppose is
anathema to most Agilistas. I'm the first to admit that I don't know
much about CMM other than it provides a scoring system for
organizations.

In a Rails context, you would establish a scorecard that categorized
your shop on a scale of 0 to 3 (or whatever, just thinking out loud
here)

RMM0 "aka Cowboy level"
- No formal development process
- No test coverage
- No standardized business practices
- Static analysis failures

RMM1 and RMM2 would have to be something in-between. RMM1 is
considered negative. RMM2 is considered positive.

RMM3 "aka Master level" (Purposely exclusive territory here, I can
imagine that only a handful of shops in the world could achieve this
level!)
- Agile software development practices
- 100% test coverage WITH THE APPROPRIATE TYPES OF TESTS (For
instance, at Hashrocket lately we are doing much less unit testing at
the MVC level because automated acceptance and integration testing
with Cucumber is so powerful and effective.
- 100% pair-programming (muahaha)
- Formal and standardized business practices
- Institutionalized continuous learning and process improvement
- Positive customer testimonials
- Successful deployment of Rails application(s) with substantial scaling demands

Incidentally, I'm bringing this up for discussion, but I might be
interested in a joint-venture along these lines with an out of work
senior Rails developer that feels like taking the idea and running
with it. (Rick?) In fact I can envision the idea expanding to the
point where being a RMM auditor could be a profitable little part-time
gig for Rails freelancers with the right personality type and
enthusiasm.

Obie Fernandez
CEO & Founder | Hashrocket
904.435.1671 office
404.934.9201 mobile

Hashrocket, Inc.
320 N 1st Street
Suite 712
Jacksonville Beach, FL 32250

http://hashrocket.com
http://obiefernandez.com

Will Sargent

unread,
Feb 12, 2009, 11:48:20 PM2/12/09
to rails-b...@googlegroups.com
> RMM3 "aka Master level" (Purposely exclusive territory here, I can
> imagine that only a handful of shops in the world could achieve this
> level!)
> - Agile software development practices
> - 100% test coverage WITH THE APPROPRIATE TYPES OF TESTS (For
> instance, at Hashrocket lately we are doing much less unit testing at
> the MVC level because automated acceptance and integration testing
> with Cucumber is so powerful and effective.

Who decides what's appropriate? According to what metrics? Is this
testing all possible branches? And if you're doing integration
testing, how do you cover all possible datasets? This just seems way
too hard to quantify and too easy to game.

> - 100% pair-programming (muahaha)

Again, this is way too easy to game; you could hire a seasoned expert
and a "net-loss" programmer and stick them together, and you'd have
"pair programming." And if this means that XP is the One True Agile
Process, that cuts Crystal Clear, Scrum, DSDM and others completely
out of the loop, and that can't be right...

> - Formal and standardized business practices
> - Institutionalized continuous learning and process improvement
> - Positive customer testimonials

Again, may have little or nothing to do with the quality of the work
done. There have been plenty of technical snowjobs done that the
client didn't discover until well after.

Honestly, I think the only thing that counts is the positive
testimonials of your peers; by which I mean that the next engineer
coming in after you've left, taking over your code, should be happy
with it and can stand up and defend it as if it were his own.

There have been places I've come into where I've seen good quality
code holding the system together, and I didn't have to ask who had
written it; it was obvious just from looking at it that these guys
were from a pro consultant shop.

Ironically, what really made it obvious to me was not that they'd used
DSLs, metaprogramming or self-modifying code; it was that they clearly
knew when it was appropriate to use those tools and when not to. In
some cases they specifically went with a lower tech solution because
it was more maintainable, even though it wasn't quite as cool as what
they wishfully described in the comments. They'd clearly considered
their audience, and were writing with the next guy in mind.

> - Successful deployment of Rails application(s) with substantial scaling demands

See, this is so nebulous, and so hard. Scaling is not something you
can do in the context of a six month project, and the nature of a
software application is to have bottlenecks and features that pop up
in the oddest places in the strangest ways. You can quote the CAP
theorum and wave memcached around, but I don't see how you can say
"substantial scaling" without quantifying it further.

The reason that CMM is so reviled amongst programmers is because the
system is inherently corrupt. The CMM evaluators are paid by the
people who are being evaluated, and they're the ones who make an
industry out of giving people high marks for having a process. Having
a high CMM process doesn't mean you actually produce software more
efficiently (although there's a Construx presentation that says
differently, I believe they're taking the evaluator's reports and
repackaging it) -- it just means that you have a well defined process,
and (at the highest level) can predict the number of bugs you get out
of that process.

Will.

Obie Fernandez

unread,
Feb 13, 2009, 12:08:32 AM2/13/09
to rails-b...@googlegroups.com
On Thu, Feb 12, 2009 at 11:48 PM, Will Sargent <will.s...@gmail.com> wrote:
>
>> RMM3 "aka Master level" (Purposely exclusive territory here, I can
>> imagine that only a handful of shops in the world could achieve this
>> level!)
>> - Agile software development practices
>> - 100% test coverage WITH THE APPROPRIATE TYPES OF TESTS (For
>> instance, at Hashrocket lately we are doing much less unit testing at
>> the MVC level because automated acceptance and integration testing
>> with Cucumber is so powerful and effective.
>
> Who decides what's appropriate? According to what metrics? Is this
> testing all possible branches? And if you're doing integration
> testing, how do you cover all possible datasets? This just seems way
> too hard to quantify and too easy to game.

The auditor, based on guidelines established by a group like this one?
Believe me I realize the challenges in making something like this work
in practice. The auditors would have to have "know it when I see it"
capabilities for judging code quality.

>> - 100% pair-programming (muahaha)
>
> Again, this is way too easy to game; you could hire a seasoned expert
> and a "net-loss" programmer and stick them together, and you'd have

I run one of the only shops that I know of that does proper and
effective pair-programming ALWAYS, even onsite with clients, etc.
Hence the evil laugh... the only other shop that I know for a fact
operates similarly is Pivotal. I admit that it might not be vital it
is to attach this criterion to RMM -- on the other hand, good Agile
practices are indispensible nowadays.

> "pair programming." And if this means that XP is the One True Agile
> Process, that cuts Crystal Clear, Scrum, DSDM and others completely
> out of the loop, and that can't be right...

Flawed reasoning. Pair-programming is included in XP but is really its
own thing apart from the particular Agile implementation you choose.
Besides, XP is the only Agile practice that I'm aware of which
"manages down" to the developer level and dictates how they operate.
Scrum and others manage up in terms of releases and stakeholders.
Right? We pick and choose whatever works from XP, Scrum, etc..

>> - Formal and standardized business practices
>> - Institutionalized continuous learning and process improvement
>> - Positive customer testimonials
>
> Again, may have little or nothing to do with the quality of the work
> done. There have been plenty of technical snowjobs done that the
> client didn't discover until well after.

Dunno about that. You might be able to snow your clients, but you
shouldn't be able to snow the auditor.


>
> Honestly, I think the only thing that counts is the positive
> testimonials of your peers; by which I mean that the next engineer
> coming in after you've left, taking over your code, should be happy
> with it and can stand up and defend it as if it were his own.

Agreed in principle. In practice, the client devs taking over are
usually not as good as we are and don't have a basis to judge, nor do
I see a good system for tallying that data. Actually, we've known
client devs to fuck up our code almost immediately. One of the worst
feelings in this business... :-/

> There have been places I've come into where I've seen good quality
> code holding the system together, and I didn't have to ask who had
> written it; it was obvious just from looking at it that these guys
> were from a pro consultant shop.

That's what I would want the auditor to do.


>
> Ironically, what really made it obvious to me was not that they'd used
> DSLs, metaprogramming or self-modifying code; it was that they clearly
> knew when it was appropriate to use those tools and when not to. In
> some cases they specifically went with a lower tech solution because
> it was more maintainable, even though it wasn't quite as cool as what
> they wishfully described in the comments. They'd clearly considered
> their audience, and were writing with the next guy in mind.

True.

>> - Successful deployment of Rails application(s) with substantial scaling demands
>
> See, this is so nebulous, and so hard. Scaling is not something you
> can do in the context of a six month project, and the nature of a
> software application is to have bottlenecks and features that pop up
> in the oddest places in the strangest ways. You can quote the CAP
> theorum and wave memcached around, but I don't see how you can say
> "substantial scaling" without quantifying it further.

I agree. Needs a lot more thought, but in theory there must be some
way of establishing whether a shop has delivered real, scalable
solutions rather than just toy apps.

> The reason that CMM is so reviled amongst programmers is because the
> system is inherently corrupt. The CMM evaluators are paid by the

> snip...

Totally. What if you had to pay a non-refundable fee to RMM upfront in
order to get evaluated. We're not such a huge "industry" that we
couldn't use that money to compensate a neutral and properly-qualified
auditor -- one that wouldn't be subject to coercion or bribing.

Rick Bradley

unread,
Feb 13, 2009, 12:28:18 AM2/13/09
to rails-b...@googlegroups.com
On Thu, Feb 12, 2009 at 11:08 PM, Obie Fernandez
<obiefe...@gmail.com> wrote:
>> The reason that CMM is so reviled amongst programmers is because the
>> system is inherently corrupt. The CMM evaluators are paid by the
>> snip...
>
> Totally. What if you had to pay a non-refundable fee to RMM upfront in
> order to get evaluated. We're not such a huge "industry" that we
> couldn't use that money to compensate a neutral and properly-qualified
> auditor -- one that wouldn't be subject to coercion or bribing.

We've discussed the potential of a somewhat objective audit before.
One of the things I'm personally still unclear on is whether doing
these audits is something that works well in a distributed (aka
"meatcloud") fashion, or whether it works better in a centralized
fashion. Does the emergent intelligence of crowds (the same emergent
intelligence that drives large-scale open source to success) sift out
the identify of the high quality software teams, or can such high
quality only really be discerned by other high-quality software teams?

If the latter is the case there's a clear bootstrapping issue.

Regardless, because it may nonetheless be the case that the builders
of good software can only be recognized by rare quality practitioners,
there's the unfortunate ballast of experience: every certification or
large-scale auditing process in software that I can recall has, for
all practical purposes, failed due to either widespread skepticism,
outright corruption, or inherent bias.

It's been my limited experience in auditing software and working with
(and after) other developers, that there are a number of objective
metrics one can put into play. Any of them can be "gamed", as has
been hinted earlier in this discussions. In aggregate, however, it is
very difficult to game the majority of them.

Furthermore, In reality, where we happen to live, there seems to
actually be a power law at work: the vast majority of software and
teams plying our trade are so bad that they fail nearly every metric
we could reasonably apply. Only the vanishing minority of products
and practitioners pass even a single gameable metric. Yet we
ultimately have little means of discerning for ourselves, for our
clients (or for differentiating for our future clients) the difference
between one or the other.

To be able to deploy a standard of comparison (whether distributed or
centrally administered and computed) that forces products or teams to
either score in the vast gutter of mediocrity, or work diligently to
at least game performance on a number of axes (when such gaming is,
realistically, often more difficult than just working to better
oneself) to separate from the pack -- would imply a vast improvement
in the current and long-standing state of affairs.

In my limited experience the difference is usually striking and
obvious: "this is trash" vs. "wow, this is pretty good". In the
Rails community the latter is really only elicited by the products of
about 10-20 shops (those shops Obie was trying to catalog in one of
his solicitations this evening). It would be great if we could get
that number up to 50, 100, or even 1000. I'd like to believe that
putting a workable set of metrics out there would eventially bias
progress in that direction.

Best,
Rick

Rick Bradley

unread,
Feb 13, 2009, 12:28:18 AM2/13/09
to rails-b...@googlegroups.com
On Thu, Feb 12, 2009 at 11:08 PM, Obie Fernandez
<obiefe...@gmail.com> wrote:
>> The reason that CMM is so reviled amongst programmers is because the
>> system is inherently corrupt. The CMM evaluators are paid by the
>> snip...
>
> Totally. What if you had to pay a non-refundable fee to RMM upfront in
> order to get evaluated. We're not such a huge "industry" that we
> couldn't use that money to compensate a neutral and properly-qualified
> auditor -- one that wouldn't be subject to coercion or bribing.

We've discussed the potential of a somewhat objective audit before.

Obie Fernandez

unread,
Feb 13, 2009, 12:43:21 AM2/13/09
to rails-b...@googlegroups.com
> fashion. Does the emergent intelligence of crowds (the same emergent
> intelligence that drives large-scale open source to success) sift out
> the identify of the high quality software teams, or can such high
> quality only really be discerned by other high-quality software teams?

It will not work on a distributed basis because of issues related to
privacy and confidentiality. I simply can't publish 90% of the work we
do in any significant way because my clients would freak out. On the
other hand, getting an auditor in here with a signed NDA for a day or
two to review our code and processes seems doable.


>
> If the latter is the case there's a clear bootstrapping issue.

Why? Because it would take an objective auditor to step up to the
plate? Because he would need to be vetted by some initial group, which
itself would have nobody to audit them? I don't think that challenge
is unsurmountable.

> Regardless, because it may nonetheless be the case that the builders
> of good software can only be recognized by rare quality practitioners,
> there's the unfortunate ballast of experience: every certification or
> large-scale auditing process in software that I can recall has, for
> all practical purposes, failed due to either widespread skepticism,
> outright corruption, or inherent bias.

I wouldn't consider that a reason not to try.

> It's been my limited experience in auditing software and working with
> (and after) other developers, that there are a number of objective
> metrics one can put into play. Any of them can be "gamed", as has
> been hinted earlier in this discussions. In aggregate, however, it is
> very difficult to game the majority of them.

That's right.

> Furthermore, In reality, where we happen to live, there seems to
> actually be a power law at work: the vast majority of software and
> teams plying our trade are so bad that they fail nearly every metric
> we could reasonably apply. Only the vanishing minority of products

I suspect that's everywhere, not just where you live. We are asked to
rescue atrocities on a regular basis.

> and practitioners pass even a single gameable metric. Yet we
> ultimately have little means of discerning for ourselves, for our
> clients (or for differentiating for our future clients) the difference
> between one or the other.

Clearly published standards would go a long way towards starting to
establish criteria for judgment. It will be a challenge to write the
criteria descriptions in such a way that it educated interested
parties about what they should be seeking. If it succeeded at that,
the project would be an educational resource and net positive for the
overall community. Wouldn't it?

> To be able to deploy a standard of comparison (whether distributed or
> centrally administered and computed) that forces products or teams to
> either score in the vast gutter of mediocrity, or work diligently to
> at least game performance on a number of axes (when such gaming is,
> realistically, often more difficult than just working to better
> oneself) to separate from the pack -- would imply a vast improvement
> in the current and long-standing state of affairs.

Right! I fully expect that if such a system was put into place, that
only a handful of companies would qualify for the top-level RMM
certification. Maybe a slightly larger would qualify for the RMM level
right below top-level, and feel that it is a selling point. To the
extent that the rest of our "industry" admired the people at the top,
they might work towards qualifying. Idealistic, yes...

> In my limited experience the difference is usually striking and
> obvious: "this is trash" vs. "wow, this is pretty good". In the
> Rails community the latter is really only elicited by the products of
> about 10-20 shops (those shops Obie was trying to catalog in one of
> his solicitations this evening). It would be great if we could get
> that number up to 50, 100, or even 1000. I'd like to believe that
> putting a workable set of metrics out there would eventially bias
> progress in that direction.

Yes, yes, yes! Amen, brother! Thanks for clearly expressing what I was
suggesting!

Cheers,
Obie

Rick Bradley

unread,
Feb 13, 2009, 1:09:02 AM2/13/09
to rails-b...@googlegroups.com
On Thu, Feb 12, 2009 at 11:43 PM, Obie Fernandez
<obiefe...@gmail.com> wrote:
>> In my limited experience the difference is usually striking and
>> obvious: "this is trash" vs. "wow, this is pretty good". In the
>> Rails community the latter is really only elicited by the products of
>> about 10-20 shops (those shops Obie was trying to catalog in one of
>> his solicitations this evening). It would be great if we could get
>> that number up to 50, 100, or even 1000. I'd like to believe that
>> putting a workable set of metrics out there would eventially bias
>> progress in that direction.
>
> Yes, yes, yes! Amen, brother! Thanks for clearly expressing what I was
> suggesting!

:-)

My suggestion, then, would be that the process be two-tiered. The
first tier is a suite of basically automated metrics that most
software in the field will probably fail spectacularly: test
coverage, cyclomatic complexity, lack of revision control :-),
revision control code churn, class/module/function coupling, etc.
There's a maintained, publicly available package of tools along with a
set of accompanying standards documents. Anyone can pull the suite
tools, run them over their software, and derive an aggregate metric
that's gameable, but tends to indicate that at least something is
probably going right if the scores are high. The tools needed to do
this are in large part already available and open source.

The second tier would be an NDA-agreeable, rigorous professional audit
of code and/or practices, conducted by expert(s) in developing and
assessing software and practices. I'm presuming here that we dodge
the bootstrap process by saying (a) those who care probably can
identify the proper auditors, and (b) if we make the audit process
open enough then a bad bootstrapping could be supplanted by better
experts following the same recipe.

The gateway to the second tier is a reasonable score on the first
tier, as well as presumably some sort of cost barrier -- to both
defray costs and to squelch overwhelming demand to use the auditing
resource. In theory one could get a limited per-application expert
audit or a more comprehensive team-and-process audit.

The net result, presuming the two tiers are evolved over time to try
to promote and identify the best techniques and methodologies, would
be to provide a way for developers to tune their processes cheaply, to
really hone their craft (more expensively), and to be able to
differentiate quality from hackery.

Best,
Rick

tmor...@gmail.com

unread,
Feb 13, 2009, 1:45:58 AM2/13/09
to Ruby on Rails meets the business world
We've been discussing this sort of thing internally for a few months.

I strongly believe that Rails is being held back by a shortage of
skilled developers.

One of the most common questions our sales people hear is "Do you know
any good developers?" :-)

One small nitpick: I don't think it's fair to exclude the skilled
single developers out of the highest level certification because they
cannot pair program. I think pairing works, but good code is good code
at the end of the day. A strong organization with multiple developers
working on a project is a benefit, to be sure, but I don't see it as
part of a skills/deeds based certification.

I'd say the same thing about Positive customer testimonials. After
all, a dev shop with a solid balance sheet is a benefit to many
customers, but are we going to require open books before
certification? :-)

--
-- Tom Mornini
-- Founder and CTO
-- Engine Yard, Inc.

Dr Nic

unread,
Feb 13, 2009, 4:26:54 AM2/13/09
to Ruby on Rails meets the business world
I don't think it requires two people at one table to type "rails
foobar" and "cap deploy".

My gut feeling reading through Obie's "Hashrocket Licensing
Certification" post is that Expert Rails == Expert Agile. No ifs/buts/
maybes. Or perhaps Expert Rails requires Expert Agile. I disagree. As
the owner of a Rails consultancy, Mocra, where we wholeheartedly sell
agile to clients, I nonetheless rebuke the notion that pair
programming is a requirement for an Expert Rails Developer or even an
Expert Rails Consultancy.

The aim of being an "Expert Developer" is to get the job done upfront
and long term. An Expert Rails Developer would know how to use Rails
to do this. An Expert Agile Developer would use Agile processes to do
this. My guess is that Scrum (tm) and other agile communities have
their certifications and two-day mastery courses/certifications all
bedded down.

In my opinion, any Rails/Ruby/JavaScript/Unix/TDD/BDD super combo
certification should be independent of development process. I don't
think any "Learning Rails" or "Expert Rails" books have chapters on
how to pair program, and at best their references to testing is a
summary of the test APIs. I don't think DHH pair programs.

How many people write unit tests for their JavaScript? 5-10% maybe. I
believe that its more important that an Expert Rails developer is an
Expert JavaScript developer than that they subscribe to any
development process/philosophy. Most Rails developers still treat
their JavaScript like a toy. Or like icing on a cake: you slap it on
the top and hope it hasn't melted in the morning.

I do pair program. Some times. And other times I don't. I always TDD
as long as I can figure out how the f@#$ to do it for the given
rubygem/cocoa GUI/rails webapp/javascript badge that I'm working on.
How many people write integration tests for RubyGems or unit tests for
JavaScript? Both are more critical skills for an Expert Rails
developer than the ability to work with a 2nd person 8 hrs a day x 5
days a week.

Penultimately, if there is to be the notion of a "consultancy"
certification: if you have 5 developers when you spend the 1 mth
attempting to get certified, and over the next 12 mths you lose 1 and
hire 10 others, what then? Does the staff who left carry their
certification with them? Do the new 10 staff automatically get the
consultancy's rating?

Finally, how does an "official" "We're an Awesome Rails consultancy"
badge on their website change the pure dynamics of capitalism? Good
companies will get referrals, get more business and stay around. Bad
companies will just get maintenance jobs on old PHP sites. If we have
a "We're so Awesome" official institution then we're going to take our
eyes off the ball - writing awesome code - and get back to dark days
of Rails in the beginning when every 3rd blog post was defending
Rails. Except now we'll be attacking or defending the "We're so
Awesome" brand: people who have the official mark will wish to defend
their expensive certification, and people who don't care about it will
spend their time rubbishing the very concept. Our community will look
silly. More importantly we won't be Awesome anymore.

Cheers
Dr NIc

On Feb 13, 4:45 pm, "tmorn...@engineyard.com" <tmorn...@gmail.com>
wrote:

Bryan Liles

unread,
Feb 13, 2009, 7:53:10 AM2/13/09
to rails-b...@googlegroups.com
I think this whole process is too subjective to try to wrap rules around it.  The successful business are the ones who keep clients and attract new business.  The ones who aren't will either be called out by their peers or not attract new business.  

The problem with formalization is the assertion that there is a golden path to software development.  Unfortunately, this isn't true because practices will have to vary depending on the client industry and economic conditions. 

From my experience, your clients don't really care how you operate.  They care that there is an appearance that they are getting decent quality software at a decent price in a decent timeframe.  Coming up with lists with guidelines only benefits those who actually understand what all that jargon means.  

Robert Dempsey

unread,
Feb 13, 2009, 8:56:01 AM2/13/09
to rails-b...@googlegroups.com
+1 to both Dr. Nic and Bryan. It appears once more that larger shops will gain favoritism and be the ones determining these guidelines. Process (how you do what you do) and performance (providing a result) should be separated. Just because a shop isn't agile (or only has one person) does not mean that they do not produce high-quality software.

Also, going on the subject of testing, as Bryan well knows, that's a controversial topic in of itself. Do we test views? Would a certification impose that on us? What level of unit testing is acceptable? Salesforce goes for 70%+ test coverage (with 100% passing tests). They're huge, have lot's of money, and changed their industry. Should that then be the metric, even though they aren't doing Rails?

No client has ever come to us and asked about certification, insurance, etc. They look at what we've done, talk to our clients, and have candid conversations with me.

Having said that, large enterprises might care about certifications, but I have yet to have any of our huge clients ask for one.

Sincerely,  

Robert Dempsey, CEO
Atlantic Dominion Solutions

http://adsdevshop.com
http://twitter.com/rdempsey
Phone: 321-274-4684

Robert Dempsey

unread,
Feb 13, 2009, 9:15:25 AM2/13/09
to Ruby on Rails meets the business world
On Feb 12, 10:45 pm, Obie Fernandez <obiefernan...@gmail.com> wrote:
> Yes, you would have to pay handsomely to get this
> certification, and the result would not necessarily be what you
> expect.

This statement is quite worrisome, and sounds like it's trying to
limit competition in the Rails space

Obie - can you clarify what you mean by this?

- Robert Dempsey

Colin Bartlett

unread,
Feb 13, 2009, 9:26:02 AM2/13/09
to rails-b...@googlegroups.com
On Feb 13, 2009, at 8:56 AM, Robert Dempsey wrote:

> No client has ever come to us and asked about certification,
> insurance, etc. They look at what we've done, talk to our clients,
> and have candid conversations with me.


Agreed.

Clients do ask about insurance in contracts and whatnot. But no one's
ever demanded certification. To be fair, none exists for Rails, so
there's nothing to ask for. But even when we were an all-Microsoft
shop, we were never Microsoft Certified. But no one cared. We did
great work and that's what mattered.

I say, let the markets decide.

Colin

Colin A. Bartlett
Kinetic Web Solutions
http://blog.kineticweb.com

Obie Fernandez

unread,
Feb 13, 2009, 9:38:44 AM2/13/09
to rails-b...@googlegroups.com

Sorry, that was only meant to say that you can't *buy* a top-level RMM
grading. That's all.

Obie Fernandez

unread,
Feb 13, 2009, 10:22:14 AM2/13/09
to rails-b...@googlegroups.com
Maybe we need a Rails Immaturity Model instead! :)
http://en.wikipedia.org/wiki/Capability_Immaturity_Model

smeade

unread,
Feb 13, 2009, 10:03:16 AM2/13/09
to Ruby on Rails meets the business world
> But even when we were an all-Microsoft  
> shop, we were never Microsoft Certified. But no one cared. We did  
> great work and that's what mattered.
>
> I say, let the markets decide.
>
> Colin

I agree with Colin. Microsoft Certification is valuable to some firms
because of the marketing power. The Microsoft brand is one of the
most powerful in the world, beating out Coke and GE in some cases.
So, companies go through the hoops and pay the big bucks to become
associated with the brand. It often does little to ensure success of
any given project.

Rails, on the other hand, explicitly and purposefully does not have
such dominant brand awareness in end-users or clients. In
http://www.loudthinking.com/posts/6-why-theres-no-rails-inc David
says, "The growth of the Rails ecosystem has been staggering. There
are so many shops out there offering Rails consulting and training. I
believe part of that proliferation is due to the fact that there's no
core-group monopoly that can dominate the market." You could easily
drop in "Rails certification monopoly" in place of core-group monopoly
and have a similar growth impacting effect.

RMM could be a good idea for shops that want to improve. Continuous
improvement is a good thing and RMM could provide a framework for
that. Yet the certification piece would be completely and 100% a
marketing tool granted by the Rails-shop certification monopoly. In
that light, it starts to sound like Microsoft Gold Partner
certification without the power of the brand.

_ Scott

Chris W

unread,
Feb 13, 2009, 10:28:34 AM2/13/09
to Ruby on Rails meets the business world
I am a -1 on the whole certification idea. I work for a large company
where certifications are almost required in order to get new business
(ISO 9000, CMMI, etc). It is very costly to get certificated and to
keep the certification. There is a whole industry around doing CMMI
based training and assessment. While our site is CMMI level 5, I know
of projects that don't follow CMMI level 1 principles.

Obie, I would read this paper about CMMI and Agile to get an idea of
how the very process oriented companies look at Agile. It would be
interesting to see where Hashrocket might rank within the CMMI scale
(at least a 3 and maybe a 5 in my mind)
http://www.sei.cmu.edu/news-at-sei/whats-new/cmmiagile.html

Maybe a certification is overkill but a website where companies can
register and document the level of process they implement allowing for
customers to provide feedback so they could get ranked. If that kind
of site took off, it would give prospective customers a way to compare
apples to apples.

Just my 2 cents...
Chris

On Feb 13, 9:38 am, Obie Fernandez <obiefernan...@gmail.com> wrote:

bcardarella

unread,
Feb 13, 2009, 10:42:00 AM2/13/09
to Ruby on Rails meets the business world
This is a terrible idea. Anytime you create a certification model what
you'll find is that people/shops will only be working towards that
model. The strength in the Ruby community has been innovation and the
freedom to try new things and ideas. If clients begin to believe that
the only good shops are those that are RMM compliant you will quickly
see a death to this spirit of innovation.

- Brian Cardarella

On Feb 13, 10:28 am, Chris W <chris.d.willi...@gmail.com> wrote:
> I am a -1 on the wholecertificationidea.  I work for a large company
> where certifications are almost required in order to get new business
> (ISO 9000, CMMI, etc).  It is very costly to get certificated and to
> keep thecertification.  There is a whole industry around doing CMMI
> based training and assessment.  While our site is CMMI level 5, I know
> of projects that don't follow CMMI level 1 principles.
>
> Obie, I would read this paper about CMMI and Agile to get an idea of
> how the very process oriented companies look at Agile.  It would be
> interesting to see where Hashrocket might rank within the CMMI scale
> (at least a 3 and maybe a 5 in my mind)http://www.sei.cmu.edu/news-at-sei/whats-new/cmmiagile.html
>
> Maybe acertificationis overkill but a website where companies can

Obie Fernandez

unread,
Feb 13, 2009, 10:49:20 AM2/13/09
to rails-b...@googlegroups.com
Let's disconnect the concept of a Rails Maturity Model and what that
would entail from the concept of official certification, since
certification implies a bunch of bad stuff and politics.

smeade

unread,
Feb 13, 2009, 10:50:44 AM2/13/09
to Ruby on Rails meets the business world
On Feb 13, 7:26 am, Colin Bartlett <colinbartl...@gmail.com> wrote:
> But even when we were an all-Microsoft  
> shop, we were never Microsoft Certified. But no one cared. We did  
> great work and that's what mattered.

I agree with Colin. Great results is what matters and a free market
can sort that out.

In "Why there's no Rails Inc" http://www.loudthinking.com/posts/6-why-theres-no-rails-inc,
David said:
"The growth of the Rails ecosystem has been staggering. There are so
many shops out there offering Rails consulting and training. I believe
part of that proliferation is due to the fact that there's no core-
group monopoly that can dominate the market." This proposal sounds
like you'd like to substitute 'core-group monopoly' with 'Rails-
certification monopoly'.

In any industry, certification is all about and only about marketing.
Firms go through the hoops and pay big bucks to be a Microsoft Gold
partner (which requires certification) because the Microsoft brand is
so strong - stronger than Coke and GE in many cases. RMM could be a
good idea for firms that want a framework for continuos improvement,
certainly. Yet the RMM certification piece seems like Microsoft Gold
without the power of a brand.

Nick Plante

unread,
Feb 13, 2009, 10:55:48 AM2/13/09
to rails-b...@googlegroups.com
Well summarized. My feelings exactly.

..nap

bcardarella

unread,
Feb 13, 2009, 10:54:03 AM2/13/09
to Ruby on Rails meets the business world
You can try to disconnect the word "certification" but the idea is
still implied and the end will still be the same.

I think a better idea would be to follow the model of the medical
community. Have some sort of apprenticeship process for individual
developers. Let the shops worry about the skills of those that are
being hired and let the results build a reputation that speak to the
clients.

This RMM idea feels too much like a 4-star rating process that
companies can brag about.

- Brian Cardarella

On Feb 13, 10:49 am, Obie Fernandez <obiefernan...@gmail.com> wrote:
> Let's disconnect the concept of a Rails Maturity Model and what that
> would entail from the concept of officialcertification, sincecertificationimplies a bunch of bad stuff and politics.
>
> On Fri, Feb 13, 2009 at 10:28 AM, Chris W <chris.d.willi...@gmail.com> wrote:
>
> > I am a -1 on the wholecertificationidea.  I work for a large company
> > where certifications are almost required in order to get new business
> > (ISO 9000, CMMI, etc).  It is very costly to get certificated and to
> > keep thecertification.  There is a whole industry around doing CMMI
> > based training and assessment.  While our site is CMMI level 5, I know
> > of projects that don't follow CMMI level 1 principles.
>
> > Obie, I would read this paper about CMMI and Agile to get an idea of
> > how the very process oriented companies look at Agile.  It would be
> > interesting to see where Hashrocket might rank within the CMMI scale
> > (at least a 3 and maybe a 5 in my mind)
> >http://www.sei.cmu.edu/news-at-sei/whats-new/cmmiagile.html
>
> > Maybe acertificationis overkill but a website where companies can

Mike Gunderloy

unread,
Feb 13, 2009, 11:01:08 AM2/13/09
to rails-b...@googlegroups.com
With the example of CMM/CMMI out in the wild as an example, the notion
that you can disconnect any sort of maturity model from "bad stuff and
politics" seems rather questionable.

Mike

steve....@gmail.com

unread,
Feb 13, 2009, 11:04:33 AM2/13/09
to Ruby on Rails meets the business world
In a previous life I had some pretty negative experiences with both
individual certifications, as well as certifications of
organizations. As a longtime Microsoft developer I obtained a number
of certifications (MCSD, MCSDBA, MCAD) and while they helped me to get
some more cash out of my employer they were pretty useless. I can't
tell you how many people I interviewed that held all sorts of
certifications but couldn't complete the most basic coding
assignment. At an organizational level my employer at the time was
also a Microsoft Certified Partner. The only reason for applying and
obtaining that level of certification was to get more business. Being
certified didn't make us anymore qualified to build great software
then a guy working out of his basement it just meant we had the money
to buy into the program and send our consultants to get their certs,
and he didn't.

These past experiences make it difficult for me to value any sort of
certification. I see certifications as a way for folks to make money
by giving those who wish to be certified a false sense of
accomplishment and belonging. I don't see a way for it to happen that
doesn't involve some seriously negative ramifications for all those
involved. Sure, it might make someone some money, and sure it might
make a big corporate client feel all warm and fuzzy by working with a
"Rails Certified Master", but at the end of the day, it won't make
anyone more likely to succeed at building and delivering great
software. Building great software is too hard to be compiled down to
a checklist, and I, for one, am 100% against the idea of putting
something together that implies otherwise.

Cheers,
Steve

Jonathan Weiss

unread,
Feb 13, 2009, 11:18:12 AM2/13/09
to rails-b...@googlegroups.com
bcardarella wrote:
> This is a terrible idea. Anytime you create a certification model what
> you'll find is that people/shops will only be working towards that
> model. The strength in the Ruby community has been innovation and the
> freedom to try new things and ideas. If clients begin to believe that
> the only good shops are those that are RMM compliant you will quickly
> see a death to this spirit of innovation.

+1

Jonathan

--
Jonathan Weiss
http://blog.innerewut.de
http://twitter.com/jweiss

Mike Mangino

unread,
Feb 13, 2009, 11:34:08 AM2/13/09
to rails-b...@googlegroups.com

On Feb 13, 2009, at 10:50 AM, smeade wrote:

>
> On Feb 13, 7:26 am, Colin Bartlett <colinbartl...@gmail.com> wrote:
>> But even when we were an all-Microsoft
>> shop, we were never Microsoft Certified. But no one cared. We did
>> great work and that's what mattered.
>
> I agree with Colin. Great results is what matters and a free market
> can sort that out.


It's a nice thought, but there isn't a well functioning market here.
Transaction costs are non-trivial, the products aren't commodities and
the information asymmetry is huge.

This situation isn't unique to software. I had a horrible time trying
to find an accountant and an attorney because I don't know enough
about either field to judge the worthiness of my potential providers.
Our clients are in the same position. After several mis-starts, I
ended up finding good people via recommendations from people I trust.
That's the same way we get most of our business. It allows us to go
into an engagement with a little trust. Of course, there is a self
selection problem. Most people don't give out references of clients
that weren't happy. (We do, but I think we're in the minority there)

I mention other fields because this problem is universal and isn't
well solved in any field. I got my first attorney from a trade
organization similar to the one proposed here. He wasted a ton of my
money and never delivered the goods. The trade organization is only
helpful if the people who run it our trustworthy. That leads us back
to the same information asymmetry.

I wish I had a brilliant solution to this problem. I would just
recommend that before an organization is created, you look at what
problem you're trying to solve and think closely about whether or not
the organization solves the problem.

Mike

--
Mike Mangino
http://www.elevatedrails.com

Roland Moriz

unread,
Feb 13, 2009, 11:52:44 AM2/13/09
to rails-b...@googlegroups.com
Jonathan Weiss schrieb:

> bcardarella wrote:
>> This is a terrible idea. Anytime you create a certification model what
>> you'll find is that people/shops will only be working towards that
>> model. The strength in the Ruby community has been innovation and the
>> freedom to try new things and ideas. If clients begin to believe that
>> the only good shops are those that are RMM compliant you will quickly
>> see a death to this spirit of innovation.
>
> +1
>

+1

Roland


--
Moriz GmbH
Hedwig-Dransfeld-Allee 14
80637 München

Tel: 089/78795079 (AB)

Vertretungsberechtigter
Geschäftsführer:
Roland Moriz

Registergericht: Amtsgericht München
Registernummer: HRB 174 294
USt-ID: DE260422784


Phlip

unread,
Feb 13, 2009, 12:07:19 PM2/13/09
to rails-b...@googlegroups.com
>>> see a death to this spirit of innovation.
>> +1
> +1

The ultimate argument against certification is decertification.

Do you really want to give so much power to some Ubergeek that they can
_de_certify you, without appeal? For the certification to gain industry
acceptance, it must have teeth. And if clients discover that the best works of
art are hung in the "Salon de Refuses" (Gallery of Rejects), then they will
start shopping there, instead!

Robby Russell

unread,
Feb 13, 2009, 12:29:14 PM2/13/09
to rails-b...@googlegroups.com

As I mentioned in the other thread, the concept is interesting. The
responses have been very interesting as well. However, with any
process/change, it's important for us to assess the cost that would
result.

Certification (or whatever you want to call it) would costs teams time
and money. This then gets passed on to their clients, which raises
their costs. Now, you could argue that clients who work with shitty
development teams will incur a big cost as well, but we don't have
enough real data to suggest that a certification process would save
clients money in the long run.

Given the situations we face with the economy, we need to be looking
at ways to remove unnecessary costs (time+money) from our processes
and my initial response to the concept.. is that this would have the
opposite result. As others have said, there is no *right* way
(globally) for designing and developing web applications with Ruby on
Rails. I suspect most of our teams work in very different ways and
that's up for debate on whether that is a good thing or not.

What is the problem that this idea is trying to solve? I don't think
that's been clearly established and would be more interested in
learning more about that before we jump to possible solutions.

Robby

--
Robby Russell
Chief Evangelist, Partner

PLANET ARGON, LLC
design // development // hosting w/Ruby on Rails

http://planetargon.com/
http://robbyonrails.com/
http://twitter.com/planetargon
aim: planetargon

+1 503 445 2457
+1 877 55 ARGON [toll free]
+1 815 642 4068 [fax]

Jon Dahl

unread,
Feb 13, 2009, 12:30:02 PM2/13/09
to Ruby on Rails meets the business world
On Feb 13, 9:49 am, Obie Fernandez <obiefernan...@gmail.com> wrote:
> Let's disconnect the concept of a Rails Maturity Model and what that
> would entail from the concept of official certification, since
> certification implies a bunch of bad stuff and politics.
>

How do you envision disconnecting these things? As long as a RMM level
is a stamp of approval given by a paid auditor, it's going to be an
official certification.

What about disconnecting the RMM idea from certification altogether by
making it a set of practices and not an audited status? In other
words, teams can aspire to a certain level by adopting a set of
practices. This doesn't "guarantee" quality, but then again, neither
does an audit by an official RMM certifier. Instead, they make a claim
about their practices ("We follow X, Y, and Z"), and their customers,
peers, and the team members themselves hold the team accountable.

Removing money from the mix is probably the best way to avoid the
political problems. And for those so inclined, there is probably more
money to be had in training than in certification itself.

Will Sargent

unread,
Feb 13, 2009, 12:40:27 PM2/13/09
to rails-b...@googlegroups.com
On Fri, Feb 13, 2009 at 7:42 AM, bcardarella <bcard...@gmail.com> wrote:
>
> This is a terrible idea. Anytime you create a certification model what
> you'll find is that people/shops will only be working towards that
> model. The strength in the Ruby community has been innovation and the
> freedom to try new things and ideas. If clients begin to believe that
> the only good shops are those that are RMM compliant you will quickly
> see a death to this spirit of innovation.

There's a great book, _Measuring and Measuring Performance in
Organizations_ which makes this point exactly; if you grade people to
a metric, they will optimize for the metric rather than for the
intangible quality that the metric is intended to measure.

Still, according to Tom Gilb, "anything you need to quantify can be
measured in some way that is superior to not measuring it at all."

Will.

Robby Russell

unread,
Feb 13, 2009, 12:43:52 PM2/13/09
to rails-b...@googlegroups.com
An interesting take on the idea.

* http://macournoyer.com/rmm/

Interesting to see the commentary on Twitter.

* http://search.twitter.com/search?q=rmm

Robby

Obie Fernandez

unread,
Feb 13, 2009, 1:02:36 PM2/13/09
to rails-b...@googlegroups.com
On Fri, Feb 13, 2009 at 12:43 PM, Robby Russell <ro...@planetargon.com> wrote:
>
> An interesting take on the idea.
>
> * http://macournoyer.com/rmm/
>
> Interesting to see the commentary on Twitter.
>
> * http://search.twitter.com/search?q=rmm
>

Hysterical.

David

unread,
Feb 13, 2009, 1:04:26 PM2/13/09
to rails-b...@googlegroups.com
There seems to be a lot of resistance to the idea, for various reasons.

Maybe a better place to start would be with a (complete-ish) list of
what would be expected of each level. And discussions on how to move
from Level 0 to Level 1, etc.

That way, instead of being exclusive and rejecting some those that
didn't meet some criteria, we would be inclusive and help the entire
ruby cummunity. A nice side effect would be making the "average"
rubyist more proficient.

Who know, maybe book could come out of it...



- David




On Thu, Feb 12, 2009 at 10:45 PM, Obie Fernandez
<obiefe...@gmail.com> wrote:
>
> A couple of Railsconfs ago, Courtenay and I did indeed discuss
> certification and I advocated an organization to set guidelines for
> what the characteristics and measurements that set apart a successful
> and disciplined Rails shop apart from the unwashed masses. Rather than
> trying to certify individuals (too easy to game!) I think it would be
> useful to have a certification process for organizations that involved
> an actual extensive interview process and audit of code and practices,
> with associated scorecards and registration in some sort of official
> directory. Yes, you would have to pay handsomely to get this
> certification, and the result would not necessarily be what you

Matt Aimonetti

unread,
Feb 13, 2009, 1:08:06 PM2/13/09
to Ruby on Rails meets the business world
I'm personally very much against the whole certification system. I
think a lot of people raised valid concerns, but I would like to add
to that and mention people/companies outside of the US. (I hear they
*do* exist). Even if, let say, we could agree about a good way of
defining different level of certifications, how would you cover the
developing countries?

I see the certification programs as yet another way to discriminate.
However, I'm not against the idea of partnership which I feel is much
less discriminating.

- Matt

Jeremy McAnally

unread,
Feb 13, 2009, 4:52:37 PM2/13/09
to rails-b...@googlegroups.com
Well, not only that, but it's also just a big racket. The first to
market and gain respect in that sector will inevitably dominate it,
leading to massive revenues with little incentive to improve the
metrics used and little incentive to avoid it becoming "corrupt."

Seeing any sort of praise for the MS certifications is pretty funny,
given that you can just buy the MS books, memorize them cover to
cover, pay MS the exam fees, to work with MS products for MS clients.
It's just another money maker for them, and they care very little for
how qualified people are.

--Jeremy
--
http://jeremymcanally.com/
http://entp.com/
http://omgbloglol.com

My books:
http://manning.com/mcanally/
http://humblelittlerubybook.com/ (FREE!)

Josh Ferguson

unread,
Feb 13, 2009, 4:58:16 PM2/13/09
to Ruby on Rails meets the business world
I think this is backwards, why don't we start with what problem we're
trying to solve first.

1) What is the current problem?
2) Who is bearing the cost of the current problem?
3) What is that cost?
4) How is the problem currently being solved?
5) What are the benefits of the proposed solution?
6) What is the overall cost to existing and new entrants into the
space?

Lets just ask these questions first. This thread opened up with a
solution in search of a problem

Josh Ferguson

On a personal note it's very clear what the intention here is, to
separate successful rails shops from the "unwashed masses". Lets be
honest here, we were all unwashed masses 3-4 years ago. Most of those
who are passionate about their craft, who drove the adoption of rails
far and wide by sacrificed night and weekend hours, are here because
they wanted to escape the world where this kind of thing was the
status quo.

On an economic note solutions like this always masquerade as "lowering
risk for clients". It attempts to do this of course at the cost of
"raising risks for development shops". The end effect is to raise the
price floor of development services, shrinking the gap between high
priced and medium price consultants, while at the same time increasing
the lag time between entering the market and being able to get *real*
contracts. It causes inefficiency in the market at the expense of the
medium ranged players or really anyone who want to play by their own
rules.

On a developer note attempting to define any company in terms of their
process is flawed. The output of a company is not their process, it is
their products. The only grading that can realistically occur is that
of the final product. We have a mechanism for grading companies by
their products already, it's called the market. Products that are
successful make revenue or are used passionately. The addition of a
further grading scale controlled by anyone *other* than the people
buying the products and services if a flawed concept in itself.

That is all.


On Feb 13, 10:04 am, David <david.lanoue...@gmail.com> wrote:
> There seems to be a lot of resistance to the idea, for various reasons.
>
> Maybe a better place to start would be with a (complete-ish) list of
> what would be expected of each level.  And discussions on how to move
> from Level 0 to Level 1, etc.
>
> That way, instead of being exclusive and rejecting some those that
> didn't meet some criteria, we would be inclusive and help the entire
> ruby cummunity.  A nice side effect would be making the "average"
> rubyist more proficient.
>
> Who know, maybe book could come out of it...
>
> - David
>
> On Thu, Feb 12, 2009 at 10:45 PM, Obie Fernandez
>

Josh Ferguson

unread,
Feb 13, 2009, 4:58:16 PM2/13/09
to Ruby on Rails meets the business world
> There seems to be a lot of resistance to the idea, for various reasons.
>
> Maybe a better place to start would be with a (complete-ish) list of
> what would be expected of each level.  And discussions on how to move
> from Level 0 to Level 1, etc.
>
> That way, instead of being exclusive and rejecting some those that
> didn't meet some criteria, we would be inclusive and help the entire
> ruby cummunity.  A nice side effect would be making the "average"
> rubyist more proficient.
>
> Who know, maybe book could come out of it...
>
> - David
>
> On Thu, Feb 12, 2009 at 10:45 PM, Obie Fernandez
>

Joseph

unread,
Feb 13, 2009, 2:43:56 PM2/13/09
to Ruby on Rails meets the business world
On Feb 13, 11:08 am, Matt Aimonetti <mattAimone...@gmail.com> wrote:
> I'm personally very much against the whole certification system. I
> think a lot of people raised valid concerns, but I would like to add
> to that and mention people/companies outside of the US. (I hear they
> *do* exist).  Even if, let say, we could agree about a good way of
> defining different level of certifications, how would you cover the
> developing countries?
>
> I see the certification programs as yet another way to discriminate.
> However, I'm not against the idea of partnership which I feel is much
> less discriminating.
>
> - Matt

+1

This is the worst idea I've seen in a long time, and with everything
the US Congress has been doing, that's saying something.


Joseph

Phlip

unread,
Feb 13, 2009, 5:05:55 PM2/13/09
to rails-b...@googlegroups.com
Jeremy McAnally wrote:

> Seeing any sort of praise for the MS certifications is pretty funny,
> given that you can just buy the MS books, memorize them cover to
> cover, pay MS the exam fees, to work with MS products for MS clients.
> It's just another money maker for them, and they care very little for
> how qualified people are.

You can also work for an MS certified software vendor, with the right number of
real MS system administrators and software engineers, and watch in horror as it
leaves a very large crater in the ground, too!

--
Phlip

Robert Fischer

unread,
Feb 13, 2009, 5:12:20 PM2/13/09
to rails-b...@googlegroups.com
+1 Josh.

For someone -- even a bunch of someones -- to "grade" another business's process is an inherently
arrogant position, born more of hubris than of any kind of real business statement. You may think
that you know The Right Way to run a development business, but so did all the "enterprisey"
certifications and standards that the Rails community -- at least once upon a time -- thumbed their
nose at. It's a really sad day when Rubyists start wanting to vet each other on their "standardized
business practices".

The only real judgment that can be passed on a business is whether or not they deliver to the
satisfaction of their clients. To equate "successful" or "reliable" or "dependable" (in short:
"good") with some standards-based concept of "maturity" is the same stupid idea that's failed time
and time before. There will be businesses out there which are successful and satisfy clients
despite not using whatever standards you feel they should have, and there will be business out there
that will do everything on your personal "Good Idea" list and still produce piles of nonfunctional,
indecipherable code and leave destroyed clients in their wake. Some of the worst code I've ever
seen had 100% code coverage and still didn't work.

~~ Robert.
--
~~ Robert Fischer.
Grails Training http://GroovyMag.com/training
Smokejumper Consulting http://SmokejumperIT.com
Enfranchised Mind Blog http://EnfranchisedMind.com/blog

Check out my book, "Grails Persistence with GORM and GSQL"!
http://www.smokejumperit.com/redirect.html

Benjamin Curtis

unread,
Feb 13, 2009, 5:13:26 PM2/13/09
to rails-b...@googlegroups.com
Well said.

--
Benjamin Curtis
http://railskits.com/ - Ready-made Rails code
http://catchthebest.com/ - Team-powered recruiting
http://www.bencurtis.com/ - Personal blog

Wes Gamble

unread,
Feb 13, 2009, 5:20:46 PM2/13/09
to rails-b...@googlegroups.com
Josh Ferguson wrote:
> On an economic note solutions like this always masquerade as "lowering
> risk for clients". It attempts to do this of course at the cost of
> "raising risks for development shops". The end effect is to raise the
> price floor of development services, shrinking the gap between high
> priced and medium price consultants, while at the same time increasing
> the lag time between entering the market and being able to get *real*
> contracts. It causes inefficiency in the market at the expense of the
> medium ranged players or really anyone who want to play by their own
> rules.
>
> On a developer note attempting to define any company in terms of their
> process is flawed. The output of a company is not their process, it is
> their products. The only grading that can realistically occur is that
> of the final product. We have a mechanism for grading companies by
> their products already, it's called the market. Products that are
> successful make revenue or are used passionately. The addition of a
> further grading scale controlled by anyone *other* than the people
> buying the products and services if a flawed concept in itself.
>
+1 for this post.

One question to ask about the motivation behind a "certification body"
of some sort is whether the intent is to
establish a minimum standard of operational efficiency (so that clients
who hire based on it can be sure they aren't
dealing with complete idiots), or whether to establish some objective
measurement of quality (good/better/best/awesome).

It gets me thinking about whether this body would be more of an OSHA
(minimum safety standards), an FDA (reasonable
standards for drugs), or <fill-in-your-government-agency-here>.

I think I see some of the positive intent behind the idea, and I
probably agree with some of those intentions, but given that
no one worth their salt seems to take existing certifications very
seriously, I have a hard time thinking that "we" (the Rails community)
will somehow come up with a certification system that really means
something. Best case, it will establish some very baseline, minimum
standard
that cannot be used to distinguish between more than complete crap and
not-crap. I suspect that he people who came up with CMM, and CNEs and
MCSEs and what-have-you's probably attempted to inject lots of "meaning"
into their certifications, and may have mostly failed.

Also, if the body that provides the certifications is private, there
will be some level of corruption involved in the giving out of
certifications, I am pretty certain.

Wes Gamble

Brian

unread,
Feb 13, 2009, 5:26:48 PM2/13/09
to Ruby on Rails meets the business world
I came from the Java Enterprise world before I entered Rails a year
ago. I left the Java Enterprise world because of the overcomplexity
that became developing software, the red tape of the enterprise, and
the terrible hiring habits of most enterprise companies, including the
dependency on certifications.

From my 15 yrs in Java, it was a safe bet that if someone was hired
with a certification they were a walking encyclopedia and couldn't
code worth a darn. They knew it, and had opted for the certification
in order to get hired. But once the coding situation fell outside
their memorized guidelines they were lost.

On the flip side, the great developers and problem solvers could be
determined by talking with them and seeing their recommendations and
past work, and rarely did they have certifications, because they
weren't the "memorize a textbook" people. They were the "get the job
done, in time, with quality" people.

The problem with certification, is that once its in place, clients now
ignore everything else and only look at that, thinking the industry
has ensured the certification as the ultimate stamp of approval. It
eliminates word of mouth and making your customers or employers happy
with your work, and puts stress on the wrong areas, defocusing from
past success.

Brian Burridge
Agile Nomads
www.agilenomads.com
www.rubyrailsreview.com

On Feb 13, 5:13 pm, Benjamin Curtis <benjamin.cur...@gmail.com> wrote:
> Well said.
> --
> Benjamin Curtishttp://railskits.com/- Ready-made Rails codehttp://catchthebest.com/- Team-powered recruitinghttp://www.bencurtis.com/- Personal blog

mdeiters

unread,
Feb 13, 2009, 6:05:38 PM2/13/09
to Ruby on Rails meets the business world
+ 1

Perhaps it would be a good idea to discuss where we can see more
easily track the consensus, i created a uservoice forum at:
http://rmm.uservoice.com/

On Feb 13, 9:42 am, bcardarella <bcardare...@gmail.com> wrote:
> This is a terrible idea. Anytime you create a certification model what
> you'll find is that people/shops will only be working towards that
> model. The strength in the Ruby community has been innovation and the
> freedom to try new things and ideas. If clients begin to believe that
> the only good shops are those that are RMM compliant you will quickly
> see a death to this spirit of innovation.
>
> - Brian Cardarella
>
> On Feb 13, 10:28 am, Chris W <chris.d.willi...@gmail.com> wrote:
>
> > I am a -1 on the wholecertificationidea.  I work for a large company
> > where certifications are almost required in order to get new business
> > (ISO 9000, CMMI, etc).  It is very costly to get certificated and to
> > keep thecertification.  There is a whole industry around doing CMMI
> > based training and assessment.  While our site is CMMI level 5, I know
> > of projects that don't follow CMMI level 1 principles.
>
> > Obie, I would read this paper about CMMI and Agile to get an idea of
> > how the very process oriented companies look at Agile.  It would be
> > interesting to see where Hashrocket might rank within the CMMI scale
> > (at least a 3 and maybe a 5 in my mind)http://www.sei.cmu.edu/news-at-sei/whats-new/cmmiagile.html
>
> > Maybe acertificationis overkill but a website where companies can
> > register and document the level of process they implement allowing for
> > customers to provide feedback so they could get ranked.  If that kind
> > of site took off, it would give prospective customers a way to compare
> > apples to apples.
>
> > Just my 2 cents...
> > Chris
>
> > On Feb 13, 9:38 am, Obie Fernandez <obiefernan...@gmail.com> wrote:
>
> > > On Fri, Feb 13, 2009 at 9:15 AM, Robert Dempsey <robertonra...@gmail.com> wrote:
>
> > > > On Feb 12, 10:45 pm, Obie Fernandez <obiefernan...@gmail.com> wrote:
> > > >> Yes, you would have to pay handsomely to get this
> > > >>certification, and the result would not necessarily be what you
> > > >> expect.
>

Jaime Bellmyer

unread,
Feb 13, 2009, 6:45:09 PM2/13/09
to Ruby on Rails meets the business world
To me, this sounds like excluding - let's weed out the "bad"
developers. We're doing the clients a favor, right? What I haven't
seen in this concept is the element of *improving* developers.

Microsoft certification produces a lot of "programmers" who have no
business writing code (along with great devs, certainly), but that
late night radio ad promised them a bigger income, and a better life,
dammit, so it's off to the nearest 6 month program. I'm not a big
fan. But at least that process is based on education: here's what a
MS certified developer should know, and here's the test that
(supposedly) proves that knowledge. They publish books, they offer
classes - yeah they make money at it, but I applaud education being
part of the equation.

They provide the standard, means of attaining that standard (reading/
training), and process of assuring that standard has been met
(testing). RMM, in its current incarnation, is missing that crucial
middle step.

That's really what we want, right? Not just to exclude the unfit, but
to help them *become* fit if they so choose. Not just identify good
developers, but help *create* them. You know what I'd love? A list,
however long it needs to be, of well-defined tasks any pro rails dev
should be able to do, in the allotted time. Something I could
practice on my own, get better at, then do a proxied test. Or, maybe
I skip the test because what I really wanted was to *become* a better
developer, branded certification or not.

This is an unfleshed idea - more thinking out loud. I just thought
I'd offer my perspective :)

Jeremy McAnally

unread,
Feb 13, 2009, 11:29:16 PM2/13/09
to rails-b...@googlegroups.com
I really think if we do anything, it needs to be more like the AIGA
than the "industry associations" we typically think about.

They're all about enrichment, best practices, funding new projects,
getting things done, and promoting their members where possible. The
good people will float to the top in their own right with the help of
other members and the org itself and the bad people will sort of drift
away.

Sounds like Ruby Central on crack to me.

--Jeremy

tmor...@gmail.com

unread,
Feb 14, 2009, 4:42:00 AM2/14/09
to Ruby on Rails meets the business world
I wanted to follow up on my post for clarification. I've seem some
incredibly
vehement comments flashing about the net regarding this thread, it's
clearly
struck a nerve!

It's clear that much of the Rails developer community isn't on board
with this
concept. I hear that loud and clear, and I understand the concerns.
Please
don't make me out to be some fiend for agreeing that there may be room
for
this sort of thing. :-)

I apologize to anyone who thought I was supporting something exclusive
and Java or Microsoft like. And, I apologize to Obie for even needing
to
say that, because I don't believe for a moment that he was suggesting
something exclusive and Java or Microsoft like in the first place. :-(

My largest interest was about growing the developer community to bring
in more developers to increase capacity in the Rails ecosystem itself,
and I think that's 100% clear in my supportive response to Obie's
idea.

Our customers at Engine Yard are screaming for more developer
capacity,
and in fact, TWO previous customers abandoned Rails (one for PHP and
the other for Java) simply because they could not find enough skilled
and
savvy Rails developers to move their projects forward.

What I was supporting was, perhaps, more education than certification:
a
commonly agreed upon set of practices could because a curriculum for
the large influx of new Rails Developers to follow so that us old
timers
in the space don't have to read code with multi-hundred line
controllers, for
instance. :-)

--
-- Tom Mornini
-- Founder and CTO
-- Engine Yard, Inc.

On Feb 12, 10:45 pm, "tmorn...@engineyard.com" <tmorn...@gmail.com>
wrote:

> We've been discussing this sort of thing internally for a few months.
>
> I strongly believe that Rails is being held back by a shortage of
> skilled developers.
>
> One of the most common questions our sales people hear is "Do you know
> any good developers?" :-)
>
> One small nitpick: I don't think it's fair to exclude the skilled
> single developers out of the highest level certification because they
> cannot pair program. I think pairing works, but good code is good code
> at the end of the day. A strong organization with multiple developers
> working on a project is a benefit, to be sure, but I don't see it as
> part of a skills/deeds based certification.
>
> I'd say the same thing about Positive customer testimonials. After
> all, a dev shop with a solid balance sheet is a benefit to many
> customers, but are we going to require open books before
> certification? :-)
>
> --
> -- Tom Mornini
> -- Founder and CTO
> -- Engine Yard, Inc.

Raul Murciano

unread,
Feb 14, 2009, 7:02:08 AM2/14/09
to rails-b...@googlegroups.com
2009/2/14 Jeremy McAnally <jeremym...@gmail.com>:

>
> I really think if we do anything, it needs to be more like the AIGA
> than the "industry associations" we typically think about.
>
> They're all about enrichment, best practices, funding new projects,
> getting things done, and promoting their members where possible. The
> good people will float to the top in their own right with the help of
> other members and the org itself and the bad people will sort of drift
> away.

* About "Ruby/Rails Certification":

I completely agree with Jeremy: imho this is the more constructive
path we could take because it will make us move as a community which
promotes activity, diversity and innovation, and not like a certified
elite group, a bunch of wannabes and a big list of talented developers
disappointed with the separatist way that their community is taking.

I also agree with DrNic and Bryan Liles: I've seen different ways to
develop high quality software. Even in the relatively small Rails
community we have great developers who don't pair all the time and
others that don't entirely buy the "always TDD" approach. So yeah,
your teams are doing great stuff, I'm sure, but please don't call your
practices "The Rails Way" ;) because that is a way that is working for
*your teams*, with *your projects* and *at this precise moment*. I'm
pretty sure that, even with your well proven and high quality
development processes, you are all experimenting with new techniques.
And I you don't I would consider it a bad smell ;) If you find a great
new technique, should we all upgrade our certificates?

The ruby people is well known by its innovation and creativity, not
only applied to their libraries and frameworks but also to their
approaches and techniques. Even the opinionated Rails code is changing
to be more flexible, so why would we like to restrict ourselves to a
strict group of practices assuming that it is the One Way to Go?


* About "Agile Certification":

Some experienced people have critizised the certification concept. I
think that all certifications - even the ones which started as a
motivation to improve and spread best practices by standarization -
become the same on the long term: a business for certifiers, a
memory-exercise-to-win-a-badge for candidates and a wrong selection
criteria.
Take a look at one of your certification materials, any of them. It's
obsolete now, isn't it?

In my humble opinion, agileness is not only a group of techniques but,
also and more importantly, a compromise with a state of mind which
implies activity, reaction, evolution and adaptability. And to me,
certification looks like the opposite.


Best regards,

--
Raul Murciano - Freelance Web Developer
http://raul.murciano.net

Robert Fischer

unread,
Feb 14, 2009, 11:34:47 AM2/14/09
to rails-b...@googlegroups.com
tmor...@engineyard.com wrote:
> What I was supporting was, perhaps, more education than certification:
> a
> commonly agreed upon set of practices could because a curriculum for
> the large influx of new Rails Developers to follow so that us old
> timers
> in the space don't have to read code with multi-hundred line
> controllers, for
> instance. :-)
>

To that extent, an apprenticeship program really your only solid bet. You need to test-infect these
developers, along with getting them hooked on a bunch of other best practices to the point where
they can't conceive of living without them. But, to do that, they need to be in a position where
they're inundated with those practices and the social culture is appropriately corrective of bad
practices.

An apprenticeship style of system is inevitably going to be a drag on existing, successful programs
-- at least in terms of productivity, if not in strict terms of income. It's an investment in the
community. So if you're interested in investing in the community that way, jump on in. The Rails
community will be ecstatic.

Note that these issues aren't unique to Rails. The entire development world is riddled with poor
developers. As Obie and I were talking about on Twitter, what's going on here is that Rails is
hitting the mainstream, so knowing Rails is no longer to validate a developer's skill. Without that
informal check, the tendency is to want to define a formal check (a "maturity model").
Unfortunately, none of those really work.

BTW, it would seem a lot less enterprisey if it wasn't presented as a collection of standards which
teams have to check off. And it was doubly harsh when presented as "I know the right way to write
software, and so if you're not doing it this way, you're not as good as me, and I'd like to
formalize just how not as good as me you are". (See Obie's explanation of his "mwahahaha" for
evidence of how that's working.)

Robert Fischer

unread,
Feb 14, 2009, 11:41:29 AM2/14/09
to rails-b...@googlegroups.com
Raul Murciano wrote:
> Some experienced people have critizised the certification concept. I
> think that all certifications - even the ones which started as a
> motivation to improve and spread best practices by standarization -
> become the same on the long term: a business for certifiers, a
> memory-exercise-to-win-a-badge for candidates and a wrong selection
> criteria.
> Take a look at one of your certification materials, any of them. It's
> obsolete now, isn't it?
>

My co-blogger, Brian Hurt, pointed out something while we were talking about this yesterday.
Certifications are about *getting* a job, whereas what's important is how you do *in the job*.
Certifications are a way for people who don't have experience in software to prove they know
software to other people who don't know software.

For job-seekers who have experience, they don't need certifications: their experience is more
meaningful than any certification. For those position-fillers who know software, they can ask to
look at code, do a technical interview, or a bit of pair programming.

As far as I'm concerned -- as someone who knows software -- open source can play the role of
certification for me. Show me an interesting/useful project you started, or point me to bugs that
you submitted matches for. *Now* you've got my attention, because it shows you 1) are engaged in
the wider development community, and 2) code well enough to not be afraid of putting your code out
there. And I can take a look at what you did and see both the kind of code you write and the way
you carry yourself in a semi-professional context.

Obie Fernandez

unread,
Feb 14, 2009, 11:46:01 AM2/14/09
to rails-b...@googlegroups.com
I agree with Robert. My original message was purposely provocative and
somewhat tongue-in-cheek because it was meant for a small audience of
this mailing list and I wanted to spur vigorous discussion. Once the
meme got out into the wild it's taken on a life of its own. I'll try
to put my thoughts about it into a blog post soon.

Sent from my iPhone

On Feb 14, 2009, at 11:34, Robert Fischer <robert....@smokejumperit.com

Robert Bazinet

unread,
Feb 14, 2009, 12:18:33 PM2/14/09
to rails-b...@googlegroups.com
I agree with Robert here but even certifications for getting a job are a bit off as well.  I have interviewed and hired many people over the years and I noticed a pattern which was easily recognizable...people who often had little or no experience would make up for that with certifications.   I remember interviewing an individual one time that had scored a perfect score on the Microsoft certifications but he could not answer any questions about software development process or other simple software engineering principles.  The same day I interviewed someone who had no certifications and did not finish college but was really bright and had held a position with a company which allowed him to learn a lot on the job.  Guess which one I hired?

I think certifications are such a false tool to measure someones knowledge and abilities.  I have seen many more times than not that the person with certs knew so much less than those with practical knowledge.

-Rob
--
Rob Bazinet
http://stillriversoftware.com
http://accidentaltechnologist.com
InfoQ Ruby Community Editor - http://www.infoq.com

Robert Fischer

unread,
Feb 14, 2009, 12:28:47 PM2/14/09
to rails-b...@googlegroups.com
Yeah: I think Brian's assessment was assuming a best-case scenario for certification. In practice,
Rob's experience with certifications is pretty much the same as mine.

~~ Robert.

Robert Bazinet wrote:
> I agree with Robert here but even certifications for getting a job are a
> bit off as well. I have interviewed and hired many people over the
> years and I noticed a pattern which was easily recognizable...people who
> often had little or no experience would make up for that with
> certifications. I remember interviewing an individual one time that
> had scored a perfect score on the Microsoft certifications but he could
> not answer any questions about software development process or other
> simple software engineering principles. The same day I interviewed
> someone who had no certifications and did not finish college but was
> really bright and had held a position with a company which allowed him
> to learn a lot on the job. Guess which one I hired?
>
> I think certifications are such a false tool to measure someones
> knowledge and abilities. I have seen many more times than not that the
> person with certs knew so much less than those with practical knowledge.
>
> -Rob
>
> On Sat, Feb 14, 2009 at 11:41 AM, Robert Fischer
> <robert....@smokejumperit.com
--

Obie Fernandez

unread,
Feb 14, 2009, 12:29:45 PM2/14/09
to rails-b...@googlegroups.com
+1

The worst case I remember was circa 2002 I interviewed a guy who must
have been in his late 30s or early 40s with a long resume and all
three Sun Java certifications. To my astonishment, he could not tell
me the purpose of a Java interface.

I don't think I'm ever going to vigorously defend the idea of
individual certifications. That's a concept that Courtenay thinks he
might have a good angle on, but I don't see it.

Brian

unread,
Feb 14, 2009, 12:01:56 AM2/14/09
to Ruby on Rails meets the business world
Oops! Sorry, I mixed up two thoughts together there: my 15 yrs of
corporate IT with my 9 yrs of Java...and none of it was as fun as my
last year in Rails. But let's keep it fun, work hard to improve our
skills, and produce great software, teach others and bring them into
the group, without enforcing certifications or our own ways of doing
things on them.

Brian Burridge
Agile Nomads
www.agilenomads.com
www.rubyrailsreview.com

> > Benjamin Curtishttp://railskits.com/-Ready-made Rails codehttp://catchthebest.com/-Team-powered recruitinghttp://www.bencurtis.com/-Personal blog

CogWiseWill

unread,
Feb 13, 2009, 10:38:49 PM2/13/09