Introducing RMM, the Rails Maturity Model
Obie Fernandez
certification and I advocated an organization to set guidelines for
what the characteristics and measurements that set apart a successful
and disciplined Rails shop apart from the unwashed masses. Rather than
trying to certify individuals (too easy to game!) I think it would be
useful to have a certification process for organizations that involved
an actual extensive interview process and audit of code and practices,
with associated scorecards and registration in some sort of official
directory. Yes, you would have to pay handsomely to get this
certification, and the result would not necessarily be what you
expect. The details for this is all very rough and frankly I don't
know the value of it, just an idea that's been in the back of my mind
for a long time so I'm putting it out there for debate.
The name and concept is somewhat influenced by CMM, which I suppose is
anathema to most Agilistas. I'm the first to admit that I don't know
much about CMM other than it provides a scoring system for
organizations.
In a Rails context, you would establish a scorecard that categorized
your shop on a scale of 0 to 3 (or whatever, just thinking out loud
here)
RMM0 "aka Cowboy level"
- No formal development process
- No test coverage
- No standardized business practices
- Static analysis failures
RMM1 and RMM2 would have to be something in-between. RMM1 is
considered negative. RMM2 is considered positive.
RMM3 "aka Master level" (Purposely exclusive territory here, I can
imagine that only a handful of shops in the world could achieve this
level!)
- Agile software development practices
- 100% test coverage WITH THE APPROPRIATE TYPES OF TESTS (For
instance, at Hashrocket lately we are doing much less unit testing at
the MVC level because automated acceptance and integration testing
with Cucumber is so powerful and effective.
- 100% pair-programming (muahaha)
- Formal and standardized business practices
- Institutionalized continuous learning and process improvement
- Positive customer testimonials
- Successful deployment of Rails application(s) with substantial scaling demands
Incidentally, I'm bringing this up for discussion, but I might be
interested in a joint-venture along these lines with an out of work
senior Rails developer that feels like taking the idea and running
with it. (Rick?) In fact I can envision the idea expanding to the
point where being a RMM auditor could be a profitable little part-time
gig for Rails freelancers with the right personality type and
enthusiasm.
Obie Fernandez
CEO & Founder | Hashrocket
904.435.1671 office
404.934.9201 mobile
Hashrocket, Inc.
320 N 1st Street
Suite 712
Jacksonville Beach, FL 32250
Will Sargent
> imagine that only a handful of shops in the world could achieve this
> level!)
> - Agile software development practices
> - 100% test coverage WITH THE APPROPRIATE TYPES OF TESTS (For
> instance, at Hashrocket lately we are doing much less unit testing at
> the MVC level because automated acceptance and integration testing
> with Cucumber is so powerful and effective.
Who decides what's appropriate? According to what metrics? Is this
testing all possible branches? And if you're doing integration
testing, how do you cover all possible datasets? This just seems way
too hard to quantify and too easy to game.
> - 100% pair-programming (muahaha)
Again, this is way too easy to game; you could hire a seasoned expert
and a "net-loss" programmer and stick them together, and you'd have
"pair programming." And if this means that XP is the One True Agile
Process, that cuts Crystal Clear, Scrum, DSDM and others completely
out of the loop, and that can't be right...
> - Formal and standardized business practices
> - Institutionalized continuous learning and process improvement
> - Positive customer testimonials
Again, may have little or nothing to do with the quality of the work
done. There have been plenty of technical snowjobs done that the
client didn't discover until well after.
Honestly, I think the only thing that counts is the positive
testimonials of your peers; by which I mean that the next engineer
coming in after you've left, taking over your code, should be happy
with it and can stand up and defend it as if it were his own.
There have been places I've come into where I've seen good quality
code holding the system together, and I didn't have to ask who had
written it; it was obvious just from looking at it that these guys
were from a pro consultant shop.
Ironically, what really made it obvious to me was not that they'd used
DSLs, metaprogramming or self-modifying code; it was that they clearly
knew when it was appropriate to use those tools and when not to. In
some cases they specifically went with a lower tech solution because
it was more maintainable, even though it wasn't quite as cool as what
they wishfully described in the comments. They'd clearly considered
their audience, and were writing with the next guy in mind.
> - Successful deployment of Rails application(s) with substantial scaling demands
See, this is so nebulous, and so hard. Scaling is not something you
can do in the context of a six month project, and the nature of a
software application is to have bottlenecks and features that pop up
in the oddest places in the strangest ways. You can quote the CAP
theorum and wave memcached around, but I don't see how you can say
"substantial scaling" without quantifying it further.
The reason that CMM is so reviled amongst programmers is because the
system is inherently corrupt. The CMM evaluators are paid by the
people who are being evaluated, and they're the ones who make an
industry out of giving people high marks for having a process. Having
a high CMM process doesn't mean you actually produce software more
efficiently (although there's a Construx presentation that says
differently, I believe they're taking the evaluator's reports and
repackaging it) -- it just means that you have a well defined process,
and (at the highest level) can predict the number of bugs you get out
of that process.
Will.
Obie Fernandez
>
>> RMM3 "aka Master level" (Purposely exclusive territory here, I can
>> imagine that only a handful of shops in the world could achieve this
>> level!)
>> - Agile software development practices
>> - 100% test coverage WITH THE APPROPRIATE TYPES OF TESTS (For
>> instance, at Hashrocket lately we are doing much less unit testing at
>> the MVC level because automated acceptance and integration testing
>> with Cucumber is so powerful and effective.
>
> Who decides what's appropriate? According to what metrics? Is this
> testing all possible branches? And if you're doing integration
> testing, how do you cover all possible datasets? This just seems way
> too hard to quantify and too easy to game.
The auditor, based on guidelines established by a group like this one?
Believe me I realize the challenges in making something like this work
in practice. The auditors would have to have "know it when I see it"
capabilities for judging code quality.
>> - 100% pair-programming (muahaha)
>
> Again, this is way too easy to game; you could hire a seasoned expert
> and a "net-loss" programmer and stick them together, and you'd have
I run one of the only shops that I know of that does proper and
effective pair-programming ALWAYS, even onsite with clients, etc.
Hence the evil laugh... the only other shop that I know for a fact
operates similarly is Pivotal. I admit that it might not be vital it
is to attach this criterion to RMM -- on the other hand, good Agile
practices are indispensible nowadays.
> "pair programming." And if this means that XP is the One True Agile
> Process, that cuts Crystal Clear, Scrum, DSDM and others completely
> out of the loop, and that can't be right...
Flawed reasoning. Pair-programming is included in XP but is really its
own thing apart from the particular Agile implementation you choose.
Besides, XP is the only Agile practice that I'm aware of which
"manages down" to the developer level and dictates how they operate.
Scrum and others manage up in terms of releases and stakeholders.
Right? We pick and choose whatever works from XP, Scrum, etc..
>> - Formal and standardized business practices
>> - Institutionalized continuous learning and process improvement
>> - Positive customer testimonials
>
> Again, may have little or nothing to do with the quality of the work
> done. There have been plenty of technical snowjobs done that the
> client didn't discover until well after.
Dunno about that. You might be able to snow your clients, but you
shouldn't be able to snow the auditor.
>
> Honestly, I think the only thing that counts is the positive
> testimonials of your peers; by which I mean that the next engineer
> coming in after you've left, taking over your code, should be happy
> with it and can stand up and defend it as if it were his own.
Agreed in principle. In practice, the client devs taking over are
usually not as good as we are and don't have a basis to judge, nor do
I see a good system for tallying that data. Actually, we've known
client devs to fuck up our code almost immediately. One of the worst
feelings in this business... :-/
> There have been places I've come into where I've seen good quality
> code holding the system together, and I didn't have to ask who had
> written it; it was obvious just from looking at it that these guys
> were from a pro consultant shop.
That's what I would want the auditor to do.
>
> Ironically, what really made it obvious to me was not that they'd used
> DSLs, metaprogramming or self-modifying code; it was that they clearly
> knew when it was appropriate to use those tools and when not to. In
> some cases they specifically went with a lower tech solution because
> it was more maintainable, even though it wasn't quite as cool as what
> they wishfully described in the comments. They'd clearly considered
> their audience, and were writing with the next guy in mind.
True.
>> - Successful deployment of Rails application(s) with substantial scaling demands
>
> See, this is so nebulous, and so hard. Scaling is not something you
> can do in the context of a six month project, and the nature of a
> software application is to have bottlenecks and features that pop up
> in the oddest places in the strangest ways. You can quote the CAP
> theorum and wave memcached around, but I don't see how you can say
> "substantial scaling" without quantifying it further.
I agree. Needs a lot more thought, but in theory there must be some
way of establishing whether a shop has delivered real, scalable
solutions rather than just toy apps.
> The reason that CMM is so reviled amongst programmers is because the
> system is inherently corrupt. The CMM evaluators are paid by the
> snip...
Totally. What if you had to pay a non-refundable fee to RMM upfront in
order to get evaluated. We're not such a huge "industry" that we
couldn't use that money to compensate a neutral and properly-qualified
auditor -- one that wouldn't be subject to coercion or bribing.
Rick Bradley
<obiefe...@gmail.com> wrote:
>> The reason that CMM is so reviled amongst programmers is because the
>> system is inherently corrupt. The CMM evaluators are paid by the
>> snip...
>
> Totally. What if you had to pay a non-refundable fee to RMM upfront in
> order to get evaluated. We're not such a huge "industry" that we
> couldn't use that money to compensate a neutral and properly-qualified
> auditor -- one that wouldn't be subject to coercion or bribing.
We've discussed the potential of a somewhat objective audit before.
One of the things I'm personally still unclear on is whether doing
these audits is something that works well in a distributed (aka
"meatcloud") fashion, or whether it works better in a centralized
fashion. Does the emergent intelligence of crowds (the same emergent
intelligence that drives large-scale open source to success) sift out
the identify of the high quality software teams, or can such high
quality only really be discerned by other high-quality software teams?
If the latter is the case there's a clear bootstrapping issue.
Regardless, because it may nonetheless be the case that the builders
of good software can only be recognized by rare quality practitioners,
there's the unfortunate ballast of experience: every certification or
large-scale auditing process in software that I can recall has, for
all practical purposes, failed due to either widespread skepticism,
outright corruption, or inherent bias.
It's been my limited experience in auditing software and working with
(and after) other developers, that there are a number of objective
metrics one can put into play. Any of them can be "gamed", as has
been hinted earlier in this discussions. In aggregate, however, it is
very difficult to game the majority of them.
Furthermore, In reality, where we happen to live, there seems to
actually be a power law at work: the vast majority of software and
teams plying our trade are so bad that they fail nearly every metric
we could reasonably apply. Only the vanishing minority of products
and practitioners pass even a single gameable metric. Yet we
ultimately have little means of discerning for ourselves, for our
clients (or for differentiating for our future clients) the difference
between one or the other.
To be able to deploy a standard of comparison (whether distributed or
centrally administered and computed) that forces products or teams to
either score in the vast gutter of mediocrity, or work diligently to
at least game performance on a number of axes (when such gaming is,
realistically, often more difficult than just working to better
oneself) to separate from the pack -- would imply a vast improvement
in the current and long-standing state of affairs.
In my limited experience the difference is usually striking and
obvious: "this is trash" vs. "wow, this is pretty good". In the
Rails community the latter is really only elicited by the products of
about 10-20 shops (those shops Obie was trying to catalog in one of
his solicitations this evening). It would be great if we could get
that number up to 50, 100, or even 1000. I'd like to believe that
putting a workable set of metrics out there would eventially bias
progress in that direction.
Best,
Rick
Rick Bradley
>> system is inherently corrupt. The CMM evaluators are paid by the
>> snip...
>
> Totally. What if you had to pay a non-refundable fee to RMM upfront in
> order to get evaluated. We're not such a huge "industry" that we
> couldn't use that money to compensate a neutral and properly-qualified
> auditor -- one that wouldn't be subject to coercion or bribing.
We've discussed the potential of a somewhat objective audit before.
Obie Fernandez
> intelligence that drives large-scale open source to success) sift out
> the identify of the high quality software teams, or can such high
> quality only really be discerned by other high-quality software teams?
It will not work on a distributed basis because of issues related to
privacy and confidentiality. I simply can't publish 90% of the work we
do in any significant way because my clients would freak out. On the
other hand, getting an auditor in here with a signed NDA for a day or
two to review our code and processes seems doable.
>
> If the latter is the case there's a clear bootstrapping issue.
Why? Because it would take an objective auditor to step up to the
plate? Because he would need to be vetted by some initial group, which
itself would have nobody to audit them? I don't think that challenge
is unsurmountable.
> Regardless, because it may nonetheless be the case that the builders
> of good software can only be recognized by rare quality practitioners,
> there's the unfortunate ballast of experience: every certification or
> large-scale auditing process in software that I can recall has, for
> all practical purposes, failed due to either widespread skepticism,
> outright corruption, or inherent bias.
I wouldn't consider that a reason not to try.
> It's been my limited experience in auditing software and working with
> (and after) other developers, that there are a number of objective
> metrics one can put into play. Any of them can be "gamed", as has
> been hinted earlier in this discussions. In aggregate, however, it is
> very difficult to game the majority of them.
That's right.
> Furthermore, In reality, where we happen to live, there seems to
> actually be a power law at work: the vast majority of software and
> teams plying our trade are so bad that they fail nearly every metric
> we could reasonably apply. Only the vanishing minority of products
I suspect that's everywhere, not just where you live. We are asked to
rescue atrocities on a regular basis.
> and practitioners pass even a single gameable metric. Yet we
> ultimately have little means of discerning for ourselves, for our
> clients (or for differentiating for our future clients) the difference
> between one or the other.
Clearly published standards would go a long way towards starting to
establish criteria for judgment. It will be a challenge to write the
criteria descriptions in such a way that it educated interested
parties about what they should be seeking. If it succeeded at that,
the project would be an educational resource and net positive for the
overall community. Wouldn't it?
> To be able to deploy a standard of comparison (whether distributed or
> centrally administered and computed) that forces products or teams to
> either score in the vast gutter of mediocrity, or work diligently to
> at least game performance on a number of axes (when such gaming is,
> realistically, often more difficult than just working to better
> oneself) to separate from the pack -- would imply a vast improvement
> in the current and long-standing state of affairs.
Right! I fully expect that if such a system was put into place, that
only a handful of companies would qualify for the top-level RMM
certification. Maybe a slightly larger would qualify for the RMM level
right below top-level, and feel that it is a selling point. To the
extent that the rest of our "industry" admired the people at the top,
they might work towards qualifying. Idealistic, yes...
> In my limited experience the difference is usually striking and
> obvious: "this is trash" vs. "wow, this is pretty good". In the
> Rails community the latter is really only elicited by the products of
> about 10-20 shops (those shops Obie was trying to catalog in one of
> his solicitations this evening). It would be great if we could get
> that number up to 50, 100, or even 1000. I'd like to believe that
> putting a workable set of metrics out there would eventially bias
> progress in that direction.
Yes, yes, yes! Amen, brother! Thanks for clearly expressing what I was
suggesting!
Cheers,
Obie
Rick Bradley
<obiefe...@gmail.com> wrote:
>> In my limited experience the difference is usually striking and
>> obvious: "this is trash" vs. "wow, this is pretty good". In the
>> Rails community the latter is really only elicited by the products of
>> about 10-20 shops (those shops Obie was trying to catalog in one of
>> his solicitations this evening). It would be great if we could get
>> that number up to 50, 100, or even 1000. I'd like to believe that
>> putting a workable set of metrics out there would eventially bias
>> progress in that direction.
>
> Yes, yes, yes! Amen, brother! Thanks for clearly expressing what I was
> suggesting!
:-)
My suggestion, then, would be that the process be two-tiered. The
first tier is a suite of basically automated metrics that most
software in the field will probably fail spectacularly: test
coverage, cyclomatic complexity, lack of revision control :-),
revision control code churn, class/module/function coupling, etc.
There's a maintained, publicly available package of tools along with a
set of accompanying standards documents. Anyone can pull the suite
tools, run them over their software, and derive an aggregate metric
that's gameable, but tends to indicate that at least something is
probably going right if the scores are high. The tools needed to do
this are in large part already available and open source.
The second tier would be an NDA-agreeable, rigorous professional audit
of code and/or practices, conducted by expert(s) in developing and
assessing software and practices. I'm presuming here that we dodge
the bootstrap process by saying (a) those who care probably can
identify the proper auditors, and (b) if we make the audit process
open enough then a bad bootstrapping could be supplanted by better
experts following the same recipe.
The gateway to the second tier is a reasonable score on the first
tier, as well as presumably some sort of cost barrier -- to both
defray costs and to squelch overwhelming demand to use the auditing
resource. In theory one could get a limited per-application expert
audit or a more comprehensive team-and-process audit.
The net result, presuming the two tiers are evolved over time to try
to promote and identify the best techniques and methodologies, would
be to provide a way for developers to tune their processes cheaply, to
really hone their craft (more expensively), and to be able to
differentiate quality from hackery.
Best,
Rick
tmor...@gmail.com
I strongly believe that Rails is being held back by a shortage of
skilled developers.
One of the most common questions our sales people hear is "Do you know
any good developers?" :-)
One small nitpick: I don't think it's fair to exclude the skilled
single developers out of the highest level certification because they
cannot pair program. I think pairing works, but good code is good code
at the end of the day. A strong organization with multiple developers
working on a project is a benefit, to be sure, but I don't see it as
part of a skills/deeds based certification.
I'd say the same thing about Positive customer testimonials. After
all, a dev shop with a solid balance sheet is a benefit to many
customers, but are we going to require open books before
certification? :-)
--
-- Tom Mornini
-- Founder and CTO
-- Engine Yard, Inc.
Dr Nic
foobar" and "cap deploy".
My gut feeling reading through Obie's "Hashrocket Licensing
Certification" post is that Expert Rails == Expert Agile. No ifs/buts/
maybes. Or perhaps Expert Rails requires Expert Agile. I disagree. As
the owner of a Rails consultancy, Mocra, where we wholeheartedly sell
agile to clients, I nonetheless rebuke the notion that pair
programming is a requirement for an Expert Rails Developer or even an
Expert Rails Consultancy.
The aim of being an "Expert Developer" is to get the job done upfront
and long term. An Expert Rails Developer would know how to use Rails
to do this. An Expert Agile Developer would use Agile processes to do
this. My guess is that Scrum (tm) and other agile communities have
their certifications and two-day mastery courses/certifications all
bedded down.
In my opinion, any Rails/Ruby/JavaScript/Unix/TDD/BDD super combo
certification should be independent of development process. I don't
think any "Learning Rails" or "Expert Rails" books have chapters on
how to pair program, and at best their references to testing is a
summary of the test APIs. I don't think DHH pair programs.
How many people write unit tests for their JavaScript? 5-10% maybe. I
believe that its more important that an Expert Rails developer is an
Expert JavaScript developer than that they subscribe to any
development process/philosophy. Most Rails developers still treat
their JavaScript like a toy. Or like icing on a cake: you slap it on
the top and hope it hasn't melted in the morning.
I do pair program. Some times. And other times I don't. I always TDD
as long as I can figure out how the f@#$ to do it for the given
rubygem/cocoa GUI/rails webapp/javascript badge that I'm working on.
How many people write integration tests for RubyGems or unit tests for
JavaScript? Both are more critical skills for an Expert Rails
developer than the ability to work with a 2nd person 8 hrs a day x 5
days a week.
Penultimately, if there is to be the notion of a "consultancy"
certification: if you have 5 developers when you spend the 1 mth
attempting to get certified, and over the next 12 mths you lose 1 and
hire 10 others, what then? Does the staff who left carry their
certification with them? Do the new 10 staff automatically get the
consultancy's rating?
Finally, how does an "official" "We're an Awesome Rails consultancy"
badge on their website change the pure dynamics of capitalism? Good
companies will get referrals, get more business and stay around. Bad
companies will just get maintenance jobs on old PHP sites. If we have
a "We're so Awesome" official institution then we're going to take our
eyes off the ball - writing awesome code - and get back to dark days
of Rails in the beginning when every 3rd blog post was defending
Rails. Except now we'll be attacking or defending the "We're so
Awesome" brand: people who have the official mark will wish to defend
their expensive certification, and people who don't care about it will
spend their time rubbishing the very concept. Our community will look
silly. More importantly we won't be Awesome anymore.
Cheers
Dr NIc
On Feb 13, 4:45 pm, "tmorn...@engineyard.com" <tmorn...@gmail.com>
wrote:
Bryan Liles
Robert Dempsey
Also, going on the subject of testing, as Bryan well knows, that's a controversial topic in of itself. Do we test views? Would a certification impose that on us? What level of unit testing is acceptable? Salesforce goes for 70%+ test coverage (with 100% passing tests). They're huge, have lot's of money, and changed their industry. Should that then be the metric, even though they aren't doing Rails?
No client has ever come to us and asked about certification, insurance, etc. They look at what we've done, talk to our clients, and have candid conversations with me.
Having said that, large enterprises might care about certifications, but I have yet to have any of our huge clients ask for one.
Sincerely,
Robert Dempsey, CEO
Atlantic Dominion Solutions
http://adsdevshop.com
http://twitter.com/rdempsey
Phone: 321-274-4684
Robert Dempsey
> Yes, you would have to pay handsomely to get this
> certification, and the result would not necessarily be what you
> expect.
limit competition in the Rails space
Obie - can you clarify what you mean by this?
- Robert Dempsey
Colin Bartlett
> No client has ever come to us and asked about certification,
> insurance, etc. They look at what we've done, talk to our clients,
> and have candid conversations with me.
Agreed.
Clients do ask about insurance in contracts and whatnot. But no one's
ever demanded certification. To be fair, none exists for Rails, so
there's nothing to ask for. But even when we were an all-Microsoft
shop, we were never Microsoft Certified. But no one cared. We did
great work and that's what mattered.
I say, let the markets decide.
Colin
Colin A. Bartlett
Kinetic Web Solutions
http://blog.kineticweb.com
Obie Fernandez
Sorry, that was only meant to say that you can't *buy* a top-level RMM
grading. That's all.
Obie Fernandez
http://en.wikipedia.org/wiki/Capability_Immaturity_Model
smeade
> shop, we were never Microsoft Certified. But no one cared. We did
> great work and that's what mattered.
>
> I say, let the markets decide.
>
> Colin
because of the marketing power. The Microsoft brand is one of the
most powerful in the world, beating out Coke and GE in some cases.
So, companies go through the hoops and pay the big bucks to become
associated with the brand. It often does little to ensure success of
any given project.
Rails, on the other hand, explicitly and purposefully does not have
such dominant brand awareness in end-users or clients. In
http://www.loudthinking.com/posts/6-why-theres-no-rails-inc David
says, "The growth of the Rails ecosystem has been staggering. There
are so many shops out there offering Rails consulting and training. I
believe part of that proliferation is due to the fact that there's no
core-group monopoly that can dominate the market." You could easily
drop in "Rails certification monopoly" in place of core-group monopoly
and have a similar growth impacting effect.
RMM could be a good idea for shops that want to improve. Continuous
improvement is a good thing and RMM could provide a framework for
that. Yet the certification piece would be completely and 100% a
marketing tool granted by the Rails-shop certification monopoly. In
that light, it starts to sound like Microsoft Gold Partner
certification without the power of the brand.
_ Scott
Chris W
where certifications are almost required in order to get new business
(ISO 9000, CMMI, etc). It is very costly to get certificated and to
keep the certification. There is a whole industry around doing CMMI
based training and assessment. While our site is CMMI level 5, I know
of projects that don't follow CMMI level 1 principles.
Obie, I would read this paper about CMMI and Agile to get an idea of
how the very process oriented companies look at Agile. It would be
interesting to see where Hashrocket might rank within the CMMI scale
(at least a 3 and maybe a 5 in my mind)
http://www.sei.cmu.edu/news-at-sei/whats-new/cmmiagile.html
Maybe a certification is overkill but a website where companies can
register and document the level of process they implement allowing for
customers to provide feedback so they could get ranked. If that kind
of site took off, it would give prospective customers a way to compare
apples to apples.
Just my 2 cents...
Chris
On Feb 13, 9:38 am, Obie Fernandez <obiefernan...@gmail.com> wrote:
bcardarella
you'll find is that people/shops will only be working towards that
model. The strength in the Ruby community has been innovation and the
freedom to try new things and ideas. If clients begin to believe that
the only good shops are those that are RMM compliant you will quickly
see a death to this spirit of innovation.
- Brian Cardarella
On Feb 13, 10:28 am, Chris W <chris.d.willi...@gmail.com> wrote:
> I am a -1 on the wholecertificationidea. I work for a large company
> (ISO 9000, CMMI, etc). It is very costly to get certificated and to
> of projects that don't follow CMMI level 1 principles.
>
> Obie, I would read this paper about CMMI and Agile to get an idea of
> how the very process oriented companies look at Agile. It would be
> interesting to see where Hashrocket might rank within the CMMI scale
> (at least a 3 and maybe a 5 in my mind)http://www.sei.cmu.edu/news-at-sei/whats-new/cmmiagile.html
>
Obie Fernandez
would entail from the concept of official certification, since
certification implies a bunch of bad stuff and politics.
smeade
> But even when we were an all-Microsoft
> shop, we were never Microsoft Certified. But no one cared. We did
> great work and that's what mattered.
can sort that out.
In "Why there's no Rails Inc" http://www.loudthinking.com/posts/6-why-theres-no-rails-inc,
"The growth of the Rails ecosystem has been staggering. There are so
many shops out there offering Rails consulting and training. I believe
part of that proliferation is due to the fact that there's no core-
like you'd like to substitute 'core-group monopoly' with 'Rails-
certification monopoly'.
In any industry, certification is all about and only about marketing.
Firms go through the hoops and pay big bucks to be a Microsoft Gold
partner (which requires certification) because the Microsoft brand is
so strong - stronger than Coke and GE in many cases. RMM could be a
good idea for firms that want a framework for continuos improvement,
certainly. Yet the RMM certification piece seems like Microsoft Gold
without the power of a brand.
Nick Plante
..nap
bcardarella
still implied and the end will still be the same.
I think a better idea would be to follow the model of the medical
community. Have some sort of apprenticeship process for individual
developers. Let the shops worry about the skills of those that are
being hired and let the results build a reputation that speak to the
clients.
This RMM idea feels too much like a 4-star rating process that
companies can brag about.
- Brian Cardarella
On Feb 13, 10:49 am, Obie Fernandez <obiefernan...@gmail.com> wrote:
> Let's disconnect the concept of a Rails Maturity Model and what that
>
> On Fri, Feb 13, 2009 at 10:28 AM, Chris W <chris.d.willi...@gmail.com> wrote:
>
> > I am a -1 on the wholecertificationidea. I work for a large company
> > (ISO 9000, CMMI, etc). It is very costly to get certificated and to
> > of projects that don't follow CMMI level 1 principles.
>
> > Obie, I would read this paper about CMMI and Agile to get an idea of
> > how the very process oriented companies look at Agile. It would be
> > interesting to see where Hashrocket might rank within the CMMI scale
> > (at least a 3 and maybe a 5 in my mind)
> >http://www.sei.cmu.edu/news-at-sei/whats-new/cmmiagile.html
>
Mike Gunderloy
that you can disconnect any sort of maturity model from "bad stuff and
politics" seems rather questionable.
Mike
steve....@gmail.com
individual certifications, as well as certifications of
organizations. As a longtime Microsoft developer I obtained a number
of certifications (MCSD, MCSDBA, MCAD) and while they helped me to get
some more cash out of my employer they were pretty useless. I can't
tell you how many people I interviewed that held all sorts of
certifications but couldn't complete the most basic coding
assignment. At an organizational level my employer at the time was
also a Microsoft Certified Partner. The only reason for applying and
obtaining that level of certification was to get more business. Being
certified didn't make us anymore qualified to build great software
then a guy working out of his basement it just meant we had the money
to buy into the program and send our consultants to get their certs,
and he didn't.
These past experiences make it difficult for me to value any sort of
certification. I see certifications as a way for folks to make money
by giving those who wish to be certified a false sense of
accomplishment and belonging. I don't see a way for it to happen that
doesn't involve some seriously negative ramifications for all those
involved. Sure, it might make someone some money, and sure it might
make a big corporate client feel all warm and fuzzy by working with a
"Rails Certified Master", but at the end of the day, it won't make
anyone more likely to succeed at building and delivering great
software. Building great software is too hard to be compiled down to
a checklist, and I, for one, am 100% against the idea of putting
something together that implies otherwise.
Cheers,
Steve
Jonathan Weiss
> This is a terrible idea. Anytime you create a certification model what
> you'll find is that people/shops will only be working towards that
> model. The strength in the Ruby community has been innovation and the
> freedom to try new things and ideas. If clients begin to believe that
> the only good shops are those that are RMM compliant you will quickly
> see a death to this spirit of innovation.
+1
Jonathan
--
Jonathan Weiss
http://blog.innerewut.de
http://twitter.com/jweiss
Mike Mangino
On Feb 13, 2009, at 10:50 AM, smeade wrote:
>
> On Feb 13, 7:26 am, Colin Bartlett <colinbartl...@gmail.com> wrote:
>> But even when we were an all-Microsoft
>> shop, we were never Microsoft Certified. But no one cared. We did
>> great work and that's what mattered.
>
> I agree with Colin. Great results is what matters and a free market
> can sort that out.
It's a nice thought, but there isn't a well functioning market here.
Transaction costs are non-trivial, the products aren't commodities and
the information asymmetry is huge.
This situation isn't unique to software. I had a horrible time trying
to find an accountant and an attorney because I don't know enough
about either field to judge the worthiness of my potential providers.
Our clients are in the same position. After several mis-starts, I
ended up finding good people via recommendations from people I trust.
That's the same way we get most of our business. It allows us to go
into an engagement with a little trust. Of course, there is a self
selection problem. Most people don't give out references of clients
that weren't happy. (We do, but I think we're in the minority there)
I mention other fields because this problem is universal and isn't
well solved in any field. I got my first attorney from a trade
organization similar to the one proposed here. He wasted a ton of my
money and never delivered the goods. The trade organization is only
helpful if the people who run it our trustworthy. That leads us back
to the same information asymmetry.
I wish I had a brilliant solution to this problem. I would just
recommend that before an organization is created, you look at what
problem you're trying to solve and think closely about whether or not
the organization solves the problem.
Mike
--
Mike Mangino
http://www.elevatedrails.com
Roland Moriz
> bcardarella wrote:
>> This is a terrible idea. Anytime you create a certification model what
>> you'll find is that people/shops will only be working towards that
>> model. The strength in the Ruby community has been innovation and the
>> freedom to try new things and ideas. If clients begin to believe that
>> the only good shops are those that are RMM compliant you will quickly
>> see a death to this spirit of innovation.
>
> +1
>
+1
Roland
--
Moriz GmbH
Hedwig-Dransfeld-Allee 14
80637 München
Tel: 089/78795079 (AB)
Vertretungsberechtigter
Geschäftsführer:
Roland Moriz
Registergericht: Amtsgericht München
Registernummer: HRB 174 294
USt-ID: DE260422784
Phlip
>> +1
> +1
The ultimate argument against certification is decertification.
Do you really want to give so much power to some Ubergeek that they can
_de_certify you, without appeal? For the certification to gain industry
acceptance, it must have teeth. And if clients discover that the best works of
art are hung in the "Salon de Refuses" (Gallery of Rejects), then they will
start shopping there, instead!
Robby Russell
As I mentioned in the other thread, the concept is interesting. The
responses have been very interesting as well. However, with any
process/change, it's important for us to assess the cost that would
result.
Certification (or whatever you want to call it) would costs teams time
and money. This then gets passed on to their clients, which raises
their costs. Now, you could argue that clients who work with shitty
development teams will incur a big cost as well, but we don't have
enough real data to suggest that a certification process would save
clients money in the long run.
Given the situations we face with the economy, we need to be looking
at ways to remove unnecessary costs (time+money) from our processes
and my initial response to the concept.. is that this would have the
opposite result. As others have said, there is no *right* way
(globally) for designing and developing web applications with Ruby on
Rails. I suspect most of our teams work in very different ways and
that's up for debate on whether that is a good thing or not.
What is the problem that this idea is trying to solve? I don't think
that's been clearly established and would be more interested in
learning more about that before we jump to possible solutions.
Robby
--
Robby Russell
Chief Evangelist, Partner
PLANET ARGON, LLC
design // development // hosting w/Ruby on Rails
http://planetargon.com/
http://robbyonrails.com/
http://twitter.com/planetargon
aim: planetargon
+1 503 445 2457
+1 877 55 ARGON [toll free]
+1 815 642 4068 [fax]
Jon Dahl
> Let's disconnect the concept of a Rails Maturity Model and what that
> would entail from the concept of official certification, since
> certification implies a bunch of bad stuff and politics.
>
is a stamp of approval given by a paid auditor, it's going to be an
official certification.
What about disconnecting the RMM idea from certification altogether by
making it a set of practices and not an audited status? In other
words, teams can aspire to a certain level by adopting a set of
practices. This doesn't "guarantee" quality, but then again, neither
does an audit by an official RMM certifier. Instead, they make a claim
about their practices ("We follow X, Y, and Z"), and their customers,
peers, and the team members themselves hold the team accountable.
Removing money from the mix is probably the best way to avoid the
political problems. And for those so inclined, there is probably more
money to be had in training than in certification itself.
Will Sargent
>
> This is a terrible idea. Anytime you create a certification model what
> you'll find is that people/shops will only be working towards that
> model. The strength in the Ruby community has been innovation and the
> freedom to try new things and ideas. If clients begin to believe that
> the only good shops are those that are RMM compliant you will quickly
> see a death to this spirit of innovation.
There's a great book, _Measuring and Measuring Performance in
Organizations_ which makes this point exactly; if you grade people to
a metric, they will optimize for the metric rather than for the
intangible quality that the metric is intended to measure.
Still, according to Tom Gilb, "anything you need to quantify can be
measured in some way that is superior to not measuring it at all."
Will.
Robby Russell
* http://macournoyer.com/rmm/
Interesting to see the commentary on Twitter.
* http://search.twitter.com/search?q=rmm
Robby
Obie Fernandez
>
> An interesting take on the idea.
>
> * http://macournoyer.com/rmm/
>
> Interesting to see the commentary on Twitter.
>
> * http://search.twitter.com/search?q=rmm
>
Hysterical.
David
Maybe a better place to start would be with a (complete-ish) list of
what would be expected of each level. And discussions on how to move
from Level 0 to Level 1, etc.
That way, instead of being exclusive and rejecting some those that
didn't meet some criteria, we would be inclusive and help the entire
ruby cummunity. A nice side effect would be making the "average"
rubyist more proficient.
Who know, maybe book could come out of it...
- David
On Thu, Feb 12, 2009 at 10:45 PM, Obie Fernandez
<obiefe...@gmail.com> wrote:
>
> A couple of Railsconfs ago, Courtenay and I did indeed discuss
> certification and I advocated an organization to set guidelines for
> what the characteristics and measurements that set apart a successful
> and disciplined Rails shop apart from the unwashed masses. Rather than
> trying to certify individuals (too easy to game!) I think it would be
> useful to have a certification process for organizations that involved
> an actual extensive interview process and audit of code and practices,
> with associated scorecards and registration in some sort of official
Matt Aimonetti
think a lot of people raised valid concerns, but I would like to add
to that and mention people/companies outside of the US. (I hear they
*do* exist). Even if, let say, we could agree about a good way of
defining different level of certifications, how would you cover the
developing countries?
I see the certification programs as yet another way to discriminate.
However, I'm not against the idea of partnership which I feel is much
less discriminating.
- Matt
Jeremy McAnally
market and gain respect in that sector will inevitably dominate it,
leading to massive revenues with little incentive to improve the
metrics used and little incentive to avoid it becoming "corrupt."
Seeing any sort of praise for the MS certifications is pretty funny,
given that you can just buy the MS books, memorize them cover to
cover, pay MS the exam fees, to work with MS products for MS clients.
It's just another money maker for them, and they care very little for
how qualified people are.
--Jeremy
http://jeremymcanally.com/
http://entp.com/
http://omgbloglol.com
My books:
http://manning.com/mcanally/
http://humblelittlerubybook.com/ (FREE!)
Josh Ferguson
trying to solve first.
1) What is the current problem?
2) Who is bearing the cost of the current problem?
3) What is that cost?
4) How is the problem currently being solved?
5) What are the benefits of the proposed solution?
6) What is the overall cost to existing and new entrants into the
space?
Lets just ask these questions first. This thread opened up with a
solution in search of a problem
Josh Ferguson
On a personal note it's very clear what the intention here is, to
separate successful rails shops from the "unwashed masses". Lets be
honest here, we were all unwashed masses 3-4 years ago. Most of those
who are passionate about their craft, who drove the adoption of rails
far and wide by sacrificed night and weekend hours, are here because
they wanted to escape the world where this kind of thing was the
status quo.
On an economic note solutions like this always masquerade as "lowering
risk for clients". It attempts to do this of course at the cost of
"raising risks for development shops". The end effect is to raise the
price floor of development services, shrinking the gap between high
priced and medium price consultants, while at the same time increasing
the lag time between entering the market and being able to get *real*
contracts. It causes inefficiency in the market at the expense of the
medium ranged players or really anyone who want to play by their own
rules.
On a developer note attempting to define any company in terms of their
process is flawed. The output of a company is not their process, it is
their products. The only grading that can realistically occur is that
of the final product. We have a mechanism for grading companies by
their products already, it's called the market. Products that are
successful make revenue or are used passionately. The addition of a
further grading scale controlled by anyone *other* than the people
buying the products and services if a flawed concept in itself.
That is all.
On Feb 13, 10:04 am, David <david.lanoue...@gmail.com> wrote:
> There seems to be a lot of resistance to the idea, for various reasons.
>
> Maybe a better place to start would be with a (complete-ish) list of
> what would be expected of each level. And discussions on how to move
> from Level 0 to Level 1, etc.
>
> That way, instead of being exclusive and rejecting some those that
> didn't meet some criteria, we would be inclusive and help the entire
> ruby cummunity. A nice side effect would be making the "average"
> rubyist more proficient.
>
> Who know, maybe book could come out of it...
>
> - David
>
> On Thu, Feb 12, 2009 at 10:45 PM, Obie Fernandez
>
Josh Ferguson
>
> Maybe a better place to start would be with a (complete-ish) list of
> what would be expected of each level. And discussions on how to move
> from Level 0 to Level 1, etc.
>
> That way, instead of being exclusive and rejecting some those that
> didn't meet some criteria, we would be inclusive and help the entire
> ruby cummunity. A nice side effect would be making the "average"
> rubyist more proficient.
>
> Who know, maybe book could come out of it...
>
> - David
>
> On Thu, Feb 12, 2009 at 10:45 PM, Obie Fernandez
>
Joseph
> I'm personally very much against the whole certification system. I
> think a lot of people raised valid concerns, but I would like to add
> to that and mention people/companies outside of the US. (I hear they
> *do* exist). Even if, let say, we could agree about a good way of
> defining different level of certifications, how would you cover the
> developing countries?
>
> I see the certification programs as yet another way to discriminate.
> However, I'm not against the idea of partnership which I feel is much
> less discriminating.
>
> - Matt
This is the worst idea I've seen in a long time, and with everything
the US Congress has been doing, that's saying something.
Joseph
Phlip
> Seeing any sort of praise for the MS certifications is pretty funny,
> given that you can just buy the MS books, memorize them cover to
> cover, pay MS the exam fees, to work with MS products for MS clients.
> It's just another money maker for them, and they care very little for
> how qualified people are.
You can also work for an MS certified software vendor, with the right number of
real MS system administrators and software engineers, and watch in horror as it
leaves a very large crater in the ground, too!
--
Phlip
Robert Fischer
For someone -- even a bunch of someones -- to "grade" another business's process is an inherently
arrogant position, born more of hubris than of any kind of real business statement. You may think
that you know The Right Way to run a development business, but so did all the "enterprisey"
certifications and standards that the Rails community -- at least once upon a time -- thumbed their
nose at. It's a really sad day when Rubyists start wanting to vet each other on their "standardized
business practices".
The only real judgment that can be passed on a business is whether or not they deliver to the
satisfaction of their clients. To equate "successful" or "reliable" or "dependable" (in short:
"good") with some standards-based concept of "maturity" is the same stupid idea that's failed time
and time before. There will be businesses out there which are successful and satisfy clients
despite not using whatever standards you feel they should have, and there will be business out there
that will do everything on your personal "Good Idea" list and still produce piles of nonfunctional,
indecipherable code and leave destroyed clients in their wake. Some of the worst code I've ever
seen had 100% code coverage and still didn't work.
~~ Robert.
~~ Robert Fischer.
Grails Training http://GroovyMag.com/training
Smokejumper Consulting http://SmokejumperIT.com
Enfranchised Mind Blog http://EnfranchisedMind.com/blog
Check out my book, "Grails Persistence with GORM and GSQL"!
http://www.smokejumperit.com/redirect.html
Benjamin Curtis
--
Benjamin Curtis
http://railskits.com/ - Ready-made Rails code
http://catchthebest.com/ - Team-powered recruiting
http://www.bencurtis.com/ - Personal blog
Wes Gamble
> On an economic note solutions like this always masquerade as "lowering
> risk for clients". It attempts to do this of course at the cost of
> "raising risks for development shops". The end effect is to raise the
> price floor of development services, shrinking the gap between high
> priced and medium price consultants, while at the same time increasing
> the lag time between entering the market and being able to get *real*
> contracts. It causes inefficiency in the market at the expense of the
> medium ranged players or really anyone who want to play by their own
> rules.
>
> On a developer note attempting to define any company in terms of their
> process is flawed. The output of a company is not their process, it is
> their products. The only grading that can realistically occur is that
> of the final product. We have a mechanism for grading companies by
> their products already, it's called the market. Products that are
> successful make revenue or are used passionately. The addition of a
> further grading scale controlled by anyone *other* than the people
> buying the products and services if a flawed concept in itself.
>
One question to ask about the motivation behind a "certification body"
of some sort is whether the intent is to
establish a minimum standard of operational efficiency (so that clients
who hire based on it can be sure they aren't
dealing with complete idiots), or whether to establish some objective
measurement of quality (good/better/best/awesome).
It gets me thinking about whether this body would be more of an OSHA
(minimum safety standards), an FDA (reasonable
standards for drugs), or <fill-in-your-government-agency-here>.
I think I see some of the positive intent behind the idea, and I
probably agree with some of those intentions, but given that
no one worth their salt seems to take existing certifications very
seriously, I have a hard time thinking that "we" (the Rails community)
will somehow come up with a certification system that really means
something. Best case, it will establish some very baseline, minimum
standard
that cannot be used to distinguish between more than complete crap and
not-crap. I suspect that he people who came up with CMM, and CNEs and
MCSEs and what-have-you's probably attempted to inject lots of "meaning"
into their certifications, and may have mostly failed.
Also, if the body that provides the certifications is private, there
will be some level of corruption involved in the giving out of
certifications, I am pretty certain.
Wes Gamble
Brian
ago. I left the Java Enterprise world because of the overcomplexity
that became developing software, the red tape of the enterprise, and
the terrible hiring habits of most enterprise companies, including the
dependency on certifications.
From my 15 yrs in Java, it was a safe bet that if someone was hired
with a certification they were a walking encyclopedia and couldn't
code worth a darn. They knew it, and had opted for the certification
in order to get hired. But once the coding situation fell outside
their memorized guidelines they were lost.
On the flip side, the great developers and problem solvers could be
determined by talking with them and seeing their recommendations and
past work, and rarely did they have certifications, because they
weren't the "memorize a textbook" people. They were the "get the job
done, in time, with quality" people.
The problem with certification, is that once its in place, clients now
ignore everything else and only look at that, thinking the industry
has ensured the certification as the ultimate stamp of approval. It
eliminates word of mouth and making your customers or employers happy
with your work, and puts stress on the wrong areas, defocusing from
past success.
Brian Burridge
Agile Nomads
www.agilenomads.com
www.rubyrailsreview.com
On Feb 13, 5:13 pm, Benjamin Curtis <benjamin.cur...@gmail.com> wrote:
> Well said.
> --
> Benjamin Curtishttp://railskits.com/- Ready-made Rails codehttp://catchthebest.com/- Team-powered recruitinghttp://www.bencurtis.com/- Personal blog
mdeiters
Perhaps it would be a good idea to discuss where we can see more
easily track the consensus, i created a uservoice forum at:
http://rmm.uservoice.com/
On Feb 13, 9:42 am, bcardarella <bcardare...@gmail.com> wrote:
> This is a terrible idea. Anytime you create a certification model what
> you'll find is that people/shops will only be working towards that
> model. The strength in the Ruby community has been innovation and the
> freedom to try new things and ideas. If clients begin to believe that
> the only good shops are those that are RMM compliant you will quickly
> see a death to this spirit of innovation.
>
> On Feb 13, 10:28 am, Chris W <chris.d.willi...@gmail.com> wrote:
>
> > I am a -1 on the wholecertificationidea. I work for a large company
> > where certifications are almost required in order to get new business
> > (ISO 9000, CMMI, etc). It is very costly to get certificated and to
> > keep thecertification. There is a whole industry around doing CMMI
> > based training and assessment. While our site is CMMI level 5, I know
> > of projects that don't follow CMMI level 1 principles.
>
> > Obie, I would read this paper about CMMI and Agile to get an idea of
> > how the very process oriented companies look at Agile. It would be
> > interesting to see where Hashrocket might rank within the CMMI scale
> > Maybe acertificationis overkill but a website where companies can
> > register and document the level of process they implement allowing for
> > customers to provide feedback so they could get ranked. If that kind
> > of site took off, it would give prospective customers a way to compare
> > apples to apples.
>
> > Just my 2 cents...
> > Chris
>
>
> > > On Fri, Feb 13, 2009 at 9:15 AM, Robert Dempsey <robertonra...@gmail.com> wrote:
>
> > > > On Feb 12, 10:45 pm, Obie Fernandez <obiefernan...@gmail.com> wrote:
> > > >>certification, and the result would not necessarily be what you
> > > >> expect.
>
Jaime Bellmyer
developers. We're doing the clients a favor, right? What I haven't
seen in this concept is the element of *improving* developers.
Microsoft certification produces a lot of "programmers" who have no
business writing code (along with great devs, certainly), but that
late night radio ad promised them a bigger income, and a better life,
dammit, so it's off to the nearest 6 month program. I'm not a big
fan. But at least that process is based on education: here's what a
MS certified developer should know, and here's the test that
(supposedly) proves that knowledge. They publish books, they offer
classes - yeah they make money at it, but I applaud education being
part of the equation.
They provide the standard, means of attaining that standard (reading/
training), and process of assuring that standard has been met
(testing). RMM, in its current incarnation, is missing that crucial
middle step.
That's really what we want, right? Not just to exclude the unfit, but
to help them *become* fit if they so choose. Not just identify good
developers, but help *create* them. You know what I'd love? A list,
however long it needs to be, of well-defined tasks any pro rails dev
should be able to do, in the allotted time. Something I could
practice on my own, get better at, then do a proxied test. Or, maybe
I skip the test because what I really wanted was to *become* a better
developer, branded certification or not.
This is an unfleshed idea - more thinking out loud. I just thought
I'd offer my perspective :)
Jeremy McAnally
than the "industry associations" we typically think about.
They're all about enrichment, best practices, funding new projects,
getting things done, and promoting their members where possible. The
good people will float to the top in their own right with the help of
other members and the org itself and the bad people will sort of drift
away.
Sounds like Ruby Central on crack to me.
--Jeremy
tmor...@gmail.com
incredibly
vehement comments flashing about the net regarding this thread, it's
clearly
struck a nerve!
It's clear that much of the Rails developer community isn't on board
with this
concept. I hear that loud and clear, and I understand the concerns.
Please
don't make me out to be some fiend for agreeing that there may be room
for
this sort of thing. :-)
I apologize to anyone who thought I was supporting something exclusive
and Java or Microsoft like. And, I apologize to Obie for even needing
to
say that, because I don't believe for a moment that he was suggesting
something exclusive and Java or Microsoft like in the first place. :-(
My largest interest was about growing the developer community to bring
in more developers to increase capacity in the Rails ecosystem itself,
and I think that's 100% clear in my supportive response to Obie's
idea.
Our customers at Engine Yard are screaming for more developer
capacity,
and in fact, TWO previous customers abandoned Rails (one for PHP and
the other for Java) simply because they could not find enough skilled
and
savvy Rails developers to move their projects forward.
What I was supporting was, perhaps, more education than certification:
a
commonly agreed upon set of practices could because a curriculum for
the large influx of new Rails Developers to follow so that us old
timers
in the space don't have to read code with multi-hundred line
controllers, for
instance. :-)
--
-- Tom Mornini
-- Founder and CTO
-- Engine Yard, Inc.
On Feb 12, 10:45 pm, "tmorn...@engineyard.com" <tmorn...@gmail.com>
wrote:
> We've been discussing this sort of thing internally for a few months.
>
> I strongly believe that Rails is being held back by a shortage of
> skilled developers.
>
> One of the most common questions our sales people hear is "Do you know
> any good developers?" :-)
>
> One small nitpick: I don't think it's fair to exclude the skilled
> single developers out of the highest level certification because they
> cannot pair program. I think pairing works, but good code is good code
> at the end of the day. A strong organization with multiple developers
> working on a project is a benefit, to be sure, but I don't see it as
> part of a skills/deeds based certification.
>
> I'd say the same thing about Positive customer testimonials. After
> all, a dev shop with a solid balance sheet is a benefit to many
> customers, but are we going to require open books before
> certification? :-)
>
> --
> -- Tom Mornini
> -- Founder and CTO
> -- Engine Yard, Inc.
Raul Murciano
>
> I really think if we do anything, it needs to be more like the AIGA
> than the "industry associations" we typically think about.
>
> They're all about enrichment, best practices, funding new projects,
> getting things done, and promoting their members where possible. The
> good people will float to the top in their own right with the help of
> other members and the org itself and the bad people will sort of drift
> away.
* About "Ruby/Rails Certification":
I completely agree with Jeremy: imho this is the more constructive
path we could take because it will make us move as a community which
promotes activity, diversity and innovation, and not like a certified
elite group, a bunch of wannabes and a big list of talented developers
disappointed with the separatist way that their community is taking.
I also agree with DrNic and Bryan Liles: I've seen different ways to
develop high quality software. Even in the relatively small Rails
community we have great developers who don't pair all the time and
others that don't entirely buy the "always TDD" approach. So yeah,
your teams are doing great stuff, I'm sure, but please don't call your
practices "The Rails Way" ;) because that is a way that is working for
*your teams*, with *your projects* and *at this precise moment*. I'm
pretty sure that, even with your well proven and high quality
development processes, you are all experimenting with new techniques.
And I you don't I would consider it a bad smell ;) If you find a great
new technique, should we all upgrade our certificates?
The ruby people is well known by its innovation and creativity, not
only applied to their libraries and frameworks but also to their
approaches and techniques. Even the opinionated Rails code is changing
to be more flexible, so why would we like to restrict ourselves to a
strict group of practices assuming that it is the One Way to Go?
* About "Agile Certification":
Some experienced people have critizised the certification concept. I
think that all certifications - even the ones which started as a
motivation to improve and spread best practices by standarization -
become the same on the long term: a business for certifiers, a
memory-exercise-to-win-a-badge for candidates and a wrong selection
criteria.
Take a look at one of your certification materials, any of them. It's
obsolete now, isn't it?
In my humble opinion, agileness is not only a group of techniques but,
also and more importantly, a compromise with a state of mind which
implies activity, reaction, evolution and adaptability. And to me,
certification looks like the opposite.
Best regards,
--
Raul Murciano - Freelance Web Developer
http://raul.murciano.net
Robert Fischer
> What I was supporting was, perhaps, more education than certification:
> a
> commonly agreed upon set of practices could because a curriculum for
> the large influx of new Rails Developers to follow so that us old
> timers
> in the space don't have to read code with multi-hundred line
> controllers, for
> instance. :-)
>
To that extent, an apprenticeship program really your only solid bet. You need to test-infect these
developers, along with getting them hooked on a bunch of other best practices to the point where
they can't conceive of living without them. But, to do that, they need to be in a position where
they're inundated with those practices and the social culture is appropriately corrective of bad
practices.
An apprenticeship style of system is inevitably going to be a drag on existing, successful programs
-- at least in terms of productivity, if not in strict terms of income. It's an investment in the
community. So if you're interested in investing in the community that way, jump on in. The Rails
community will be ecstatic.
Note that these issues aren't unique to Rails. The entire development world is riddled with poor
developers. As Obie and I were talking about on Twitter, what's going on here is that Rails is
hitting the mainstream, so knowing Rails is no longer to validate a developer's skill. Without that
informal check, the tendency is to want to define a formal check (a "maturity model").
Unfortunately, none of those really work.
BTW, it would seem a lot less enterprisey if it wasn't presented as a collection of standards which
teams have to check off. And it was doubly harsh when presented as "I know the right way to write
software, and so if you're not doing it this way, you're not as good as me, and I'd like to
formalize just how not as good as me you are". (See Obie's explanation of his "mwahahaha" for
evidence of how that's working.)
Robert Fischer
> Some experienced people have critizised the certification concept. I
> think that all certifications - even the ones which started as a
> motivation to improve and spread best practices by standarization -
> become the same on the long term: a business for certifiers, a
> memory-exercise-to-win-a-badge for candidates and a wrong selection
> criteria.
> Take a look at one of your certification materials, any of them. It's
> obsolete now, isn't it?
>
My co-blogger, Brian Hurt, pointed out something while we were talking about this yesterday.
Certifications are about *getting* a job, whereas what's important is how you do *in the job*.
Certifications are a way for people who don't have experience in software to prove they know
software to other people who don't know software.
For job-seekers who have experience, they don't need certifications: their experience is more
meaningful than any certification. For those position-fillers who know software, they can ask to
look at code, do a technical interview, or a bit of pair programming.
As far as I'm concerned -- as someone who knows software -- open source can play the role of
certification for me. Show me an interesting/useful project you started, or point me to bugs that
you submitted matches for. *Now* you've got my attention, because it shows you 1) are engaged in
the wider development community, and 2) code well enough to not be afraid of putting your code out
there. And I can take a look at what you did and see both the kind of code you write and the way
you carry yourself in a semi-professional context.
Obie Fernandez
somewhat tongue-in-cheek because it was meant for a small audience of
this mailing list and I wanted to spur vigorous discussion. Once the
meme got out into the wild it's taken on a life of its own. I'll try
to put my thoughts about it into a blog post soon.
Sent from my iPhone
On Feb 14, 2009, at 11:34, Robert Fischer <robert....@smokejumperit.com
Robert Bazinet
I think certifications are such a false tool to measure someones knowledge and abilities. I have seen many more times than not that the person with certs knew so much less than those with practical knowledge.
-Rob
Rob Bazinet
http://stillriversoftware.com
http://accidentaltechnologist.com
InfoQ Ruby Community Editor - http://www.infoq.com
Robert Fischer
Rob's experience with certifications is pretty much the same as mine.
~~ Robert.
Robert Bazinet wrote:
> I agree with Robert here but even certifications for getting a job are a
> bit off as well. I have interviewed and hired many people over the
> years and I noticed a pattern which was easily recognizable...people who
> often had little or no experience would make up for that with
> certifications. I remember interviewing an individual one time that
> had scored a perfect score on the Microsoft certifications but he could
> not answer any questions about software development process or other
> simple software engineering principles. The same day I interviewed
> someone who had no certifications and did not finish college but was
> really bright and had held a position with a company which allowed him
> to learn a lot on the job. Guess which one I hired?
>
> I think certifications are such a false tool to measure someones
> knowledge and abilities. I have seen many more times than not that the
> person with certs knew so much less than those with practical knowledge.
>
> -Rob
>
> On Sat, Feb 14, 2009 at 11:41 AM, Robert Fischer
> <robert....@smokejumperit.com
Obie Fernandez
The worst case I remember was circa 2002 I interviewed a guy who must
have been in his late 30s or early 40s with a long resume and all
three Sun Java certifications. To my astonishment, he could not tell
me the purpose of a Java interface.
I don't think I'm ever going to vigorously defend the idea of
individual certifications. That's a concept that Courtenay thinks he
might have a good angle on, but I don't see it.
Brian
corporate IT with my 9 yrs of Java...and none of it was as fun as my
last year in Rails. But let's keep it fun, work hard to improve our
skills, and produce great software, teach others and bring them into
the group, without enforcing certifications or our own ways of doing
things on them.
CogWiseWill
similar to this as we have been cleaning up a lot of code from other
programmers in the past. There is little encouragement or anything to
compel a business person to understand a technical standard and it's
intrinsic value when looking for programmers. Many times they look at
the bottom line such as "This programmer or firm can get it done for
cheaper sooner and has good references." As programmers, we all know
that people just don't give out the bad references they have and if
someone can do it quicker and cheaper that doesn't mean they are doing
it a maintainable way that meets any Rails bare minimum accepted
standards of quality. So, a person can do it for quicker and cheaper
and side step the programmer that is doing the coding in a responsible
and maintainable way.
Usually the way to solve this is a professional organization such as a
guild that sets standards, enforces standards and modifies standards
as the times change. This guild should be formed by companies who
practice these standards. If any one company does it along it will not
have enough weight to be accepted within the community. It needs to be
accepted within the community and accepted by business people as a
legit organization. I would propose a guild or some kind of
organization of freelancers and firms to propose and vote on
standards.
Such an organization should share the responsibility of evaluating new
members between the organizations and freelancers who have already met
the highest level of the standard within the organization. Membership
would be a responsibility and a reward for being a good programmer who
holds an acceptable level of standards.
Some of the standards you proposed such as "Pair programming" I would
disagree on as a requirement but that's all semantics. When someone
can gather together the member organizations and put together a
charter in order to have rules for voting with a governing body then
that would be decided by the members of such an organization. CogWise
would be willing to participate in such an organization.
I think this is a great idea but it should be done from the ground up
by the community. No one organization could create such a
certification standard... what works for your company doesn't work for
all. Client happiness is the true gauge of how well a company does
it's job no matter what standards they use. But if the client doesn't
know they are sitting on a ticking time bomb full of messy code that
is unmaintainable and impossible to upgrade well... ignorance is bliss.
Brian
in getting Rails developers. I personally spend time every week with
friends and past co-workers trying to encourage them to switch from
Java to Rails. I also spend time teaching Rails to those who have
already made the decision to switch. Through all the discussions, the
number one concern I get from prospective new developers is the fear
that there aren't enough jobs out there compared to Java.
It is growing, but I get 10x more job leads in my inbox than I do
Rails. As demand increases there is no doubt that the supply will too.
In the mean time, we can grow the base, by working with the Activists
to teach others, spread the word, expand the Rails guides, and by
writing on our own blogs.
I do want to see great people come into the community, but what I
don't want to see is what happened to Java after it became the "in"
thing to do. At first Java was great, and was filled with committed
and talented developers. But, as companies began to adopt it, and
needed more help, and the Internet boomed, companies threw heaps of
money at new prospective developers, and in no time at all, everyone
and their mother "did Java". It was insane. I worked with ex-cashiers,
truck drivers...anyone who could turn on a computer began learning
Java and jumping into it for a fast buck, and these new developers
were happy to take a smaller salary then the "experts" so companies
got rid of their experts and/or started hiring less skilled people, in
order to reduce costs.. The market was flooded with amateurs and the
quality of developers across the industry plummeted.
So I understand the desire for certification, but its a wolf in sheeps
clothing, and actually what helped some of these people get into Java.
They got themselves certified in a few months with the help of a class
that taught nothing but how to pass the test.
Instead, we as current Rails developers can work with newcomers and
with companies to spread the word on what is good quality code, how to
interview (hint: don't give a certification type exam!), and what
processes help them best achieve their goals. By these simple means,
and by our own adherence to conventions for code and processes, we can
grow the industry but with excellent quality and a firm foundation.
On Feb 14, 7:02 am, Raul Murciano <r...@murciano.net> wrote:
> 2009/2/14 Jeremy McAnally <jeremymcana...@gmail.com>:
Matt V
> I really think if we do anything, it needs to be more like the AIGA
> than the "industry associations" we typically think about.
I firmly believe certifications bodies are the road to ruin for any
developer community. But maybe the issue could be addressed with
something more like a Hall-of-Fame style organization, with
categories, nominations and ballots. You still run into issues of who
gets to vote, and so forth, but the process is a lot more conducive to
recognizing greatness and encouraging improvement, rather than
marketing and resume-polishing.
I hate any sort of checklist oriented approach, because that is what
fast-food shops need, and what great restaurants don't. There is still
a significant level of artistry and craftsmanship in this industry,
and those are things that do not take well to a by-the-book approach.
-- Matt
Pete Forde
problem.
Unspace was, to the best of my knowledge, the first Rails development
studio that (at the time) bet the farm on something totally bat-shit
crazy: we had multiple people and a leased office working on 100%
Rails in late 2004. We had one of our first clients ask us who would
continue their project if we got hit by a bus, and we realized that we
didn't know a single other Ruby developer in Toronto. That's why we
started Rails Pub Nite (the 34th one is next Monday!) and we met
Hampton Catlin at the inaugural event. He still had his faux-hawk!
There's 350 Rubyists on the mailing list, now.
Almost five years later, we're still compelled to stay small enough
that we can all fit around the same table. The A-Team, if you will let
us get away with that. We are great friends, and we have fun doing
what we do... which is build really sophisticated Rails apps for our
several dozen clients.
Not one time has certification ever come up. Not with a client, not
within our ranks.
We're living the Rails/open source/web 3 dream and I dare say that
we'd never have pulled it off if there was some sort of formal
certification process. Obie, I love the skin your features are printed
on, but this thread could end up doing way more Damage to whatever
"Rails" is than anything Zed could punch up.
My inclusive && alternative proposal to exclusive && expensive
certification is that I tell every recruiter and would-be start-up CEO
that contacts me looking for a referral to come to Rails Pub Nite. Not
only do they get to meet members of the community face-to-face in a
casual setting, but they have the option of standing up on a chair,
pitching their employment or contract offer for a minute or two, and
then they have to buy everyone a round of drinks. It's entirely fair
to say that about half of the recruitment and qualification for Rails
development in Canada happens at a pub, and we like it that way.
Rails is our sunshine. Please don't take our sunshine away!
Pete
A former Microsoft MVP for IIS/Active Server Pages... and he's got the
embossed watch to prove it.
Will Sargent
> don't want to see is what happened to Java after it became the "in"
> thing to do. At first Java was great, and was filled with committed
> and talented developers. But, as companies began to adopt it, and
> needed more help, and the Internet boomed, companies threw heaps of
> money at new prospective developers, and in no time at all, everyone
> and their mother "did Java". It was insane. I worked with ex-cashiers,
> truck drivers...anyone who could turn on a computer began learning
> Java and jumping into it for a fast buck, and these new developers
> were happy to take a smaller salary then the "experts" so companies
> got rid of their experts and/or started hiring less skilled people, in
> order to reduce costs.. The market was flooded with amateurs and the
> quality of developers across the industry plummeted.
Exactly.
One of the uses of a language is to serve as a barrier to entry. It
used to be that knowing Java was only known and used by hotshot
programmers. Now it's Ruby. Next it's probably Erlang.
In each case, you start off with some very highly motivated people,
have some initial successes, get a nice chorus going. And then as the
community matures, you start taking on people who know the words but
not the music, or just sort of hum along... and as more people join
in, you end up with a spectacular cacophony that prompts the early
adopters to leave and set up shop somewhere else. Ultimately Ruby
will go the same way.
I don't think that a Rails Maturity Model, or indeed any kind of
application specific Maturity Model will work, because ultimately
you're not looking for a specific language or an application; you're
looking for a solution. The only solution you're going to get is from
good programmers working as a team.
If there were some easy way to tell the quality of a team or
individual programmer, we'd take it. There are indicators you can
use; if a programmer or company has open source contributions then you
can do reviews of their work. You can take customer testimonials, you
can call competitors or non-local teams (they'll be much more plugged
in to who's doing what) and you can do interviews, or even ask for
prototypes or sample "starter" projects to see how the team operates.
But ultimately, the quality of the review you do is dependent on the
work that you put into it. If you're relying on a piece of paper to
do your review, you're probably not doing enough due diligence in the
first place.
Will.
Matthew Palmer
> > I do want to see great people come into the community, but what I
> > don't want to see is what happened to Java after it became the "in"
> > thing to do. At first Java was great, and was filled with committed
> > and talented developers. But, as companies began to adopt it, and
> > needed more help, and the Internet boomed, companies threw heaps of
> > money at new prospective developers, and in no time at all, everyone
> > and their mother "did Java". It was insane. I worked with ex-cashiers,
> > truck drivers...anyone who could turn on a computer began learning
> > Java and jumping into it for a fast buck, and these new developers
> > were happy to take a smaller salary then the "experts" so companies
> > got rid of their experts and/or started hiring less skilled people, in
> > order to reduce costs.. The market was flooded with amateurs and the
> > quality of developers across the industry plummeted.
And there are those who say that this has already happened. It's getting
too bloody easy to find two dozen plugins that all say they do what you
want, but all of which are utter shite.
> One of the uses of a language is to serve as a barrier to entry. It
> used to be that knowing Java was only known and used by hotshot
> programmers. Now it's Ruby. Next it's probably Erlang.
Gawd, please don't put the mojo on Erlang. I've moved into it to try and
find something that isn't being flooded with the great unwashed. I think
it'd break me if it went the same way as Rails is.
- Matt
--
Logan Shaw's Zen of ASR Computing: free yourself of the desire to have
computers work properly for this is the root of all suffering.
Obie Fernandez
http://blog.obiefernandez.com/content/2009/02/rails-maturity-model.html
Totally backing off the idea of certification, but still think a
defined RMM is a good idea.
Cheers,
Obie
Kapil
Model - Best practices without auditors or certifications based on
community ideas ,http://tinyurl.com/czhtez
regards
Kapil Bhatia
On Feb 13, 8:45 am, Obie Fernandez <obiefernan...@gmail.com> wrote:
> A couple of Railsconfs ago, Courtenay and I did indeed discuss
> certification and I advocated an organization to set guidelines for
> what the characteristics and measurements that set apart a successful
> and disciplined Rails shop apart from the unwashed masses. Rather than
> trying to certify individuals (too easy to game!) I think it would be
> useful to have a certification process for organizations that involved
> an actual extensive interview process and audit of code and practices,
> with associated scorecards and registration in some sort of official
Evan Light
On Feb 14, 4:42 am, "tmorn...@engineyard.com" <tmorn...@gmail.com>
wrote:
> capacity,
> and in fact, TWO previous customers abandoned Rails (one for PHP and
> the other for Java) simply because they could not find enough skilled
> and
> savvy Rails developers to move their projects forward.
Last time I checked, there are many of us (myself included) that are
quite open to part-time or full-time contracting opportunities...
Evan Light
Founder, Ruby DCamp
Triple Dog Dare, LLC (which is so under-utilized that it just points
at my blog ;-) ).
Josh
become too watered down? Being in the top 100 probably helps though
but still I've noticed a slight drop off. :-)
Ken Mayer
Vixie
Many years ago, Paul started a talk about the RBL with this statement.
Then he went on to tell the tale of how he put msn.com on the RBL,
much to the distress of a bunch of people in Redmond.
The argument for and against standards have been beaten into dog food
here. I think most agree that they won't work the way you expect them
to. What everyone *really* wants are better rubyists and more educated
customers about what makes good developers (and why they are worth
it). The apprenticeship angle has been brought up. Also "guilds." Some
people object to the union terms. Others feel the "drain" on the more
experienced workers will be a cut to profitability.
You might want to consider an open sourced training curriculum. Invest
in qualified educators (PhDs in education) to make a verifiable,
educationally valid, training system. Then give it away. Put it on the
web somewhere. Encourage candidates to get themselves "better" even
before they apply for jobs. A group of shops could share costs, create
a RubyU, of sorts. Hold training classes (for a fee, some people
prefer a classroom setting and will pay for the service). Send your
big customers there, so they become more adept. Think RailsCasts,
PeepCode, Pragmatic Screencasts, etc. with a plan.
Obie Fernandez
> in qualified educators (PhDs in education) to make a verifiable,
> educationally valid, training system. Then give it away. Put it on the
> web somewhere. Encourage candidates to get themselves "better" even
> before they apply for jobs. A group of shops could share costs, create
> a RubyU, of sorts. Hold training classes (for a fee, some people
> prefer a classroom setting and will pay for the service). Send your
> big customers there, so they become more adept. Think RailsCasts,
> PeepCode, Pragmatic Screencasts, etc. with a plan.
Great feedback for sure. As I mentioned in the Rubiverse podcast with
Mike, you can take the data that will be freely available on RMM and
use it as the basis of curriculum based firmly in the real practices
of successful Rails shops.
Cheers,
Obie
PS: The link to that podcast is
http://rubiverse.com/podcasts/6-obie-fernandez-on-rails-maturity-model
Steven Bristol
I just posted an alternative RMM. I'd love to hear your thoughts.
http://b.lesseverything.com/2009/2/24/an-alternative-rails-maturity-model
cheers,
steven bristol
Evan Light
As I wrote on the blog:
--- 8< ----
Better but still not quite right. The trouble with such broad labels
is a "Master" is defined by what?
Mastery by individual skill I can believe. However, even then, mastery
is at best temporary and at worst a fallacy as techniques in this
community continuously evolve.
Mastery of Rails? Whoa! Sign me up!
Oh wait....
I may be awesome at designing and writing model classes but master
over everything in the stack? The front end as well to include
JavaScript, CSS, design, et al? I doubt that there are more than
perhaps two dozen people who can claim that level of expertise and
have the chops to back it up.
(What I left out of the blog comment but I'll say here: However, I
more than suspect that there are plenty of people with the egos to
back up such ridiculous claims)
Do you remember Hampton Caitlin's Ruby survey perchance? See http://hamptoncatlin.com/2008/hampton-s-ruby-survey-2008-results
Unfortunately, he doesn't have the statistical results posted but he
has the source YAML. The outcome was laughable! Some ridiculous
number, on the order of 20% IIRC, consider themselves Ruby Masters.
I sincerely doubt that 1-in-5 Rails developers are masters.
Color me skeptical.
Evan
Obie Fernandez
"mastery" is not going to work, particularly if they're ranking
themselves. That's all I'm going to say because I don't want to
discuss the topic anymore until I actually have the site ready to
launch and people can see for themselves what I've been talking about.
Obie
