[OT (kinda)] Language Adoption?
Robert Fischer
measure that, but it's hard to tell which languages have more mindshare -- get different people with
different focuses together, and they seem to have wildly different guesses.
~~ Robert Fischer.
Grails Trainining http://www.smokejumperit.com/grails_training.html
Smokejumper Consulting http://smokejumperit.com
Enfranchised Mind Blog http://enfranchisedmind.com/blog
Check out my book, "Grails Persistence with GORM and GSQL"!
http://www.smokejumperit.com/redirect.html
Ben Evans
Is there any kind of reasonable language adoption metric out there? I'm not sure how one might
measure that, but it's hard to tell which languages have more mindshare -- get different people with
different focuses together, and they seem to have wildly different guesses.
While you could use a number of different measures to help gauge this (number of attendees at language-specific conferences, number of technical books published, search query rates on various sites, various survey results, and many, many more) - this is a highly subjective measurement, and any group which is invested in the question is going to show huge selection bias.
That's even supposing that we all had the same idea of what it means for a language to be "adopted" by a user, and leaving aside the problem that there's a good chance that the group of language users who are not going to show up on any of those metrics will in many cases dwarf the population that does show up.
I'm not sure that after the biases of the selection group, method and questioner were taken into account that there would be any meaningful signal left.
What interests you about the question?
Ben
Brian Schlining
Is there any kind of reasonable language adoption metric out there?
There's http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html
Some folks dispute Tiobe's methods but at least it's a starting point.
Cheers
--
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
Brian Schlining
Charles Oliver Nutter
> Is there any kind of reasonable language adoption metric out there? I'm not sure how one might
> measure that, but it's hard to tell which languages have more mindshare -- get different people with
> different focuses together, and they seem to have wildly different guesses.
Tim O'Reilly has used book sales numbers as a measure a couple times,
and it's probably not a bad measure to use. People that buy a book on a
language or libraries related to the language are making a real,
measurable investment in it. Related to this is probably just the number
of books published on a subject, since it means authors at least believe
there will be a market.
In some of my presentations I've used conference attendance as a
measure. For example, there were more Ruby conferences in 2008 than
Menudo has albums. Most of the conferences were smaller affairs (50-200
people) but RailsConf has gone from 500 people to 1500 people to 2500
people. And most of these conferences cost money, so again, there's a
real measurable investment going on.
I'd like to hear about other metrics too. There's obviously no
scientific way to measure it, so these are all estimate based on some
other concrete fact. But we can probably get pretty close. I think
that's some of the idea behind TIOBE...they're trying to aggregate a
number of metrics to form a rough picture. It's probably not super
accurate, but it does illustrate some interesting trends.
- Charlie
John D. Mitchell
number of job listings mentioning a given language.
However, such a direct, first-order measure misses the importance of
the influence of languages on other languages, tools, etc.
Have fun,
John
Charles Oliver Nutter
I think TIOBE uses job metrics as one of their indicators. It seems like
a reasonably good indicator that a language "has been" adopted to some
level, but probably not much of an indicator of languages on their way
to being adopted. For example, we all know Lisp is going to take over
the programming language world any day now, and it's only in 23rd place
on TIOBE. Meanwhile, COBOL, which is dying a slow death, is in 17th
place. So I'd say job numbers is a lagging indicator at best...
- Charlie
Tim Bray
John Wilson
TIOBE say they use search engine metrics using the search term
"<language> programming" with some language specific post processing
(see http://www.tiobe.com/index.php/content/paperinfo/tpci/tpci_definition.htm).
Once the percentage score for a language falls below 5% I don't think
the numbers are significant. Their longer term trends look to be more
valuable and show just how jittery the metric is (I'm sure the actual
usage of established language does not exhibit this degree of jitter.
What we are seeing is an artefact of the metric. And what happened in
2004! http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html).
"Tim O'Reilly"'s (actually, the latest one I can find is from Mike
Hendrickson http://radar.oreilly.com/2008/03/state-of-the-computer-book-mar-23.html)
analysis of the IT book market is based on hard data but, of course,
is not a direct measure of the usage of languages. There are obvious
problems with correlating book purchase with language use (Somebody
who has been writing Fortran for 20 years is probably not going to be
buying a book on Fortran this quarter - Very few of the people buying
Haskel books this quarter will be using the language for serious
work).
Job adds are to be treated with some suspicion. Recruiters just love
keywords, I'm not sure there's a very high correlation between the
language skills in the ad and the language actually used:)
I think that any method which uses internet searches is going to be
pretty unreliable. You are measuring what people are talking about not
what they are using. For example imagine using that method to measure
mobile phone usage - I think you'd come to the conclusion that 75% of
the population use the iPhone.
For languages which are of direct interest to this list I think the
best metric is mailing list usage (markmail.org is very good for this)
John Wilson
David Welton
> methodology -Tim
Yeah, that's mine, thanks for mentioning it!
I also attempt to grab more sources than TIOBE does, to just give
people a lot of different things to look at. Right now, for instance,
I'm working to add irc channels in the 'talk' portion of the stats.
It's impossible to do really good statistics of this type, but I think
it's fun to see what turns up in any case, and while there are some
problems (which I do note on the site), by and large, I think the
results do reflect what my "nose in the wind" tells me about what's
going on.
John Wilson says:
> For languages which are of direct interest to this list I think the
> best metric is mailing list usage (markmail.org is very good for this)
That's true for smaller languages, which probably includes a lot
developed by people here (like my own Hecl, which is unfortunately
significantly less popular than langpop.com despite being a lot more
work on my part), but for larger ones, is going to be pretty much
impossible to keep track of. I bet there are tons of Italian forums
on PHP, for instance. Multiple that by languages, user groups,
countries, and it's going to be impossible to keep track of.
With anything though, you have to think about biases - lots of people
like to talk about Haskell, for instance (the IRC channel is huge),
but how much is it actually used in production? It's best to just
throw lots of numbers out there and let people make up their own
minds, is the conclusion I reached, and what I strive for with
langpop.com.
What might be an interesting addition to langpop.com is a list of "up
and comers", which could indeed use different, and potentially more
accurate metrics.
--
David N. Welton
John Wilson
[snip]
> John Wilson says:
>
>> For languages which are of direct interest to this list I think the
>> best metric is mailing list usage (markmail.org is very good for this)
>
> That's true for smaller languages, which probably includes a lot
> developed by people here (like my own Hecl, which is unfortunately
> significantly less popular than langpop.com despite being a lot more
> work on my part), but for larger ones, is going to be pretty much
> impossible to keep track of. I bet there are tons of Italian forums
> on PHP, for instance. Multiple that by languages, user groups,
> countries, and it's going to be impossible to keep track of.
>
> With anything though, you have to think about biases - lots of people
> like to talk about Haskell, for instance (the IRC channel is huge),
> but how much is it actually used in production? It's best to just
> throw lots of numbers out there and let people make up their own
> minds, is the conclusion I reached, and what I strive for with
> langpop.com.
>
> What might be an interesting addition to langpop.com is a list of "up
> and comers", which could indeed use different, and potentially more
> accurate metrics.
I'm quite interested in the popularity of JRuby rather than Ruby and
Jython rather than Python (and the popularity of Groovy and Scala as
well).
With these languages and ones like them the mailing lists and IRC
channels are probably a very good source of data which indicate their
popularity. I think you would need to do some processing on the raw
data to get meaningful figures. I think a good indicator would be the
volume of user questions received. So you'd look to count only
messages from new and infrequently posting users and have some way of
counting threads as only one message.
If it was possible to get a metric which was roughly "numbers of user
queries per month" I would imagine that would be quite a good
correlation with popularity. There would, of course, be a problem with
JRuby and Jython because they have a proportion of there users who are
new to the implementation but not new to the language so you would
expect to so a smaller number of messages because the "how to I turn
an int into a float" type of question would have been answered
elsewhere.
John Wilson
Robert Fischer
page of those I'm following[1].
I'm not a fan of conference attendence -- that may speak more to the culture of. Rubyists in
particular are a social bunch -- the Pradipta 416 wouldn't have happened if you hit up OCaml
developers, and I may be going to RubyRX down here in Raleigh, even though I don't really do much
with Ruby these days. Also, who do you chalk the No Fluff, Just Stuff conference attendance up to?
Java? Groovy? Scala?
I'm putting together a blog post. I might do regular tracking of said metrics.
[1] http://enfranchisedmind.com/blog/job-graphs/
~~ Robert.
Thomas E Enebo
>
>
> If it was possible to get a metric which was roughly "numbers of user
> queries per month" I would imagine that would be quite a good
> correlation with popularity. There would, of course, be a problem with
> JRuby and Jython because they have a proportion of there users who are
> new to the implementation but not new to the language so you would
> expect to so a smaller number of messages because the "how to I turn
> an int into a float" type of question would have been answered
> elsewhere.
Yeah and since many people using JRuby will have Ruby problems they
end up asking virtually all general Ruby questions on a general Ruby
mailing list than on JRuby's lists. I think in the last 4 years we
have had approximately 4 pure-Ruby questions asked on our lists. So
even if you are new to both implemention AND language you know there
are richer sources of info available (like ruby-talk).
I am also amazed at how many people I run into at conferences who are
using JRuby in production and:
1. never joined mailing list
2. never joined irc channel
3. never updated our wiki
4. never emailed us
And this is from people who are actually attending conferences! I
think most people who use software never ever get seen directly by the
project offering that software.
Jobs seem like a good metric, but the job listing are a gigantic
over-stuffed pot of acronyms. I don't really trust postings too much.
For us, a job for Ruby on Rails may or may not be us. It may not even
be a Ruby on Rails job. Also head-hunters duplicate job postings on
sites which over-inflates the number of jobs that actually exist. I
think it is a noisy source.
Book sales and probably training attendance (consultants offering
training perhaps) is not a bad thing to look at. If people are
regularly making money training people on software then the money is
likely coming from a company. Companies generally only pay money for
training on things they want to start using. Book sales are mostly
the same, but many programmers buy books for themselves. Still, the
company needs the inspiration to use new software from somewhere. So
book sales may be better indication of what is to come and training
tells you what is here now. Though training also seems to be a
cultural thing so it probably varies a lot (More people probably need
to take PeopleSoft training than Rails training for example).
Conferences seem to depend on the culture of the language too. Ruby
has many cheap local conferences so I think many individuals will pay
to go whether they get their company to pay or not. JavaOne on the
other hand...Attendance has to almost be entirely be funded from
companies. The price is much higher. heh, this is a tough subject :)
babble..babble...Thanks Robert...hopefully this thread help illuminate
this better for us all...
-Tom
--
Blog: http://www.bloglines.com/blog/ThomasEEnebo
Email: en...@acm.org , tom....@gmail.com
John D. Mitchell
> On Wed, Jan 28, 2009 at 5:19 AM, Charles Oliver Nutter
> <charles...@sun.com> wrote:
>>
>> John D. Mitchell wrote:
>>> One metric that is relatively telling in the broader view is the
>>> number of job listings mentioning a given language.
>>>
>>> However, such a direct, first-order measure misses the importance of
>>> the influence of languages on other languages, tools, etc.
>>
>> I think TIOBE uses job metrics as one of their indicators. It seems
>> like
>> a reasonably good indicator that a language "has been" adopted to
>> some
>> level, but probably not much of an indicator of languages on their
>> way
>> to being adopted. For example, we all know Lisp is going to take over
>> the programming language world any day now, and it's only in 23rd
>> place
>> on TIOBE. Meanwhile, COBOL, which is dying a slow death, is in 17th
>> place. So I'd say job numbers is a lagging indicator at best...
They are all "bad" indicators for a variety of reasons...
> TIOBE say they use search engine metrics using the search term
> "<language> programming" with some language specific post processing
> (see http://www.tiobe.com/index.php/content/paperinfo/tpci/tpci_definition.htm)
> .
> Once the percentage score for a language falls below 5% I don't think
> the numbers are significant. Their longer term trends look to be more
> valuable and show just how jittery the metric is (I'm sure the actual
> usage of established language does not exhibit this degree of jitter.
> What we are seeing is an artefact of the metric. And what happened in
> 2004! http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html)
> .
Indeed! Just using raw text search metrics is wildly bad. One nasty
effect is the low end numbers often get lost from the crawling and
indexing.
Also note that the search engines don't get behind all of the gated
areas and so will tend to under-represent some areas and over-
represent others.
On the search engine front, it's more interesting to use one of the
specialized search engines such as Krugle to look at things like the
activity of the projects in the various languages, number of projects
in a language, number of (active) committers, the ever popular SLOC,
etc. [ObDisclosure: I was the Chief Architect.] But, of course,
those numbers are hard to normalize across wildly different languages/
problem domains/etc. Also, of course, a public engine like Krugle.org
is only looking through publicly accessible projects and so that a
skew away from the proprietary areas.
> "Tim O'Reilly"'s (actually, the latest one I can find is from Mike
> Hendrickson http://radar.oreilly.com/2008/03/state-of-the-computer-book-mar-23.html)
> analysis of the IT book market is based on hard data but, of course,
> is not a direct measure of the usage of languages. There are obvious
> problems with correlating book purchase with language use (Somebody
> who has been writing Fortran for 20 years is probably not going to be
> buying a book on Fortran this quarter - Very few of the people buying
> Haskel books this quarter will be using the language for serious
> work).
Indeed.
Book sales are much less driven by actual usage as they are by hype.
So, in that sense they can be useful as a leading indicator but they
aren't good at all for actual usage.
Also, look at how much of what was historically content for technical
books that's now freely available on the web. Reference materials and
Q&A forums abound.
Another issue is the whole domestic US & western markets vs. the rest
of the world. Because of things like cost and translations, the
international books sales are completely out of touch with the usage
numbers.
> Job adds are to be treated with some suspicion. Recruiters just love
> keywords, I'm not sure there's a very high correlation between the
> language skills in the ad and the language actually used:)
I think that's a statistically safe correlation (since the recruiters
are just getting that from the company).
The job posting indicators will over-inflate some numbers (e.g.,
phantom postings and multiple-postings for a single actual position)
and under-represent others (e.g., startups, side-projects, etc.).
> I think that any method which uses internet searches is going to be
> pretty unreliable. You are measuring what people are talking about not
> what they are using. For example imagine using that method to measure
> mobile phone usage - I think you'd come to the conclusion that 75% of
> the population use the iPhone.
>
> For languages which are of direct interest to this list I think the
> best metric is mailing list usage (markmail.org is very good for this)
Thanks for mentioning MarkMail!
Mailing lists stats are particularly interesting to me obviously
[ObDisclosure: Mad Scientist of MarkMail :-)] but be careful with them
as they have a variety of artifacts, too. For example some
communities show tremendous mailing list volume growth that then
flattens and declines even as the language becomes increasingly
popular -- one because they moved to a forum based Q&A without a
gateway with the mailing list. Also, different communities have
different cultures and so it's hard to normalize across them -- e.g.,
some communities have lots of talkers on a regular basis and some are
more sedate and/or bursty.
Have fun,
John
John D. Mitchell
> On Wed, Jan 28, 2009 at 9:48 AM, David Welton
> <davidn...@gmail.com> wrote:
> [snip]
>
>> John Wilson says:
>>
>>> For languages which are of direct interest to this list I think the
>>> best metric is mailing list usage (markmail.org is very good for
>>> this)
>>
>> That's true for smaller languages, which probably includes a lot
>> developed by people here (like my own Hecl, which is unfortunately
>> significantly less popular than langpop.com despite being a lot more
>> work on my part), but for larger ones, is going to be pretty much
>> impossible to keep track of. I bet there are tons of Italian forums
>> on PHP, for instance. Multiple that by languages, user groups,
>> countries, and it's going to be impossible to keep track of.
>>
>> With anything though, you have to think about biases - lots of people
>> like to talk about Haskell, for instance (the IRC channel is huge),
>> but how much is it actually used in production? It's best to just
>> throw lots of numbers out there and let people make up their own
>> minds, is the conclusion I reached, and what I strive for with
>> langpop.com.
In developing the enterprise version of Krugle and it's "management/
information dashboard", one of the areas that we tried to focus on was
the trends and particularly the variations against the trends.
>> What might be an interesting addition to langpop.com is a list of "up
>> and comers", which could indeed use different, and potentially more
>> accurate metrics.
>
> I'm quite interested in the popularity of JRuby rather than Ruby and
> Jython rather than Python (and the popularity of Groovy and Scala as
> well).
>
> With these languages and ones like them the mailing lists and IRC
> channels are probably a very good source of data which indicate their
> popularity. I think you would need to do some processing on the raw
> data to get meaningful figures. I think a good indicator would be the
> volume of user questions received. So you'd look to count only
> messages from new and infrequently posting users and have some way of
> counting threads as only one message.
>
> If it was possible to get a metric which was roughly "numbers of user
> queries per month" I would imagine that would be quite a good
> correlation with popularity. There would, of course, be a problem with
> JRuby and Jython because they have a proportion of there users who are
> new to the implementation but not new to the language so you would
> expect to so a smaller number of messages because the "how to I turn
> an int into a float" type of question would have been answered
> elsewhere.
If you're just looking at trends then thats true but trying to
normalize those across communities (and, as David mentioned, across
cultures, languages, etc.) that doesn't work so well.
Also, one bias that highly connected, heavy online users like us have
is that we really don't grasp just how large a portion of technical
people don't regularly use the 'net or books. Yes, that frightens me
to no end but it's one of those nasty facts.
Take care,
John
Jon Harrop
> > There's also http://langpop.com/ which is very up-front about its
> > methodology -Tim
>
> Yeah, that's mine, thanks for mentioning it!
Can you add F#, Scala and Clojure, please?
--
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e
Attila Szegedi
learning the language without buying a book" factor... :-)
:-)
Attila.
Charles Oliver Nutter
> If it was possible to get a metric which was roughly "numbers of user
> queries per month" I would imagine that would be quite a good
> correlation with popularity. There would, of course, be a problem with
> JRuby and Jython because they have a proportion of there users who are
> new to the implementation but not new to the language so you would
> expect to so a smaller number of messages because the "how to I turn
> an int into a float" type of question would have been answered
> elsewhere.
I'll echo Tom's statement and add a couple of my own.
JRuby's mailing lists have never been extremely busy, partly because
Ruby and Rails and other Ruby libs/frameworks have their own lists (and
really, their own communities), but also because we use IRC pretty
heavily for many discussions. And IRC has been a *huge* boon to the
project; truly interactive support of users, real-time design
discussions...without IRC we would certainly not be where we are today.
Striking a balance between the two is tricky, and I lament the lack of
ML traffic; but the benefits of IRC can't be ignored.
I'll also agree that using ML traffic can be misleading. Ruby probably
had more traffic on the main list a couple years ago, before many
libraries and frameworks moved off to their own lists and users started
native-language lists. So if you considered mailing list traffic (which
can still be misleading), you'd need to consider traffic across many,
many lists. Groovy and Scala differ in this way because they largely
have only a couple separate major lists, where Ruby has mailing lists
for basically all the main projects.
There's also another reason ML traffic can be misleading: if a library
or app works without modification, most people don't ever subscribe. We
get people using JRuby happily for months without us ever knowing about
it. They learn Ruby and Rails (maybe joining those lists or maybe not),
then switch to JRuby without many bumps. So ML traffic becomes a victim
of our focus on Ruby compatibility...if we succeed, nobody should *need*
to join the JRuby ML.
- Charlie
Charles Oliver Nutter
>> TIOBE say they use search engine metrics using the search term
>> "<language> programming" with some language specific post processing
>> (see http://www.tiobe.com/index.php/content/paperinfo/tpci/tpci_definition.htm)
>> .
>> Once the percentage score for a language falls below 5% I don't think
>> the numbers are significant. Their longer term trends look to be more
>> valuable and show just how jittery the metric is (I'm sure the actual
>> usage of established language does not exhibit this degree of jitter.
>> What we are seeing is an artefact of the metric. And what happened in
>> 2004! http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html)
>> .
>
> Indeed! Just using raw text search metrics is wildly bad. One nasty
> effect is the low end numbers often get lost from the crawling and
> indexing.
And here's a more insidious problem with search engine results: they
don't filter out false positives. So for a reasonably common word like
"groovy" you get inflated results. Even "groovy programming" gets hits
from programmers that think programming is groovy. "jruby" on the other
hand has pretty much only one meaning, as does "scala", "clojure", and
"jython". "python" and "ruby" suffer from the same problems as "groovy"
since they're common words, but "python programming" and "ruby
programming" are less commonly false positives...unless you're a
snake-handling jeweler who likes to program.
>> "Tim O'Reilly"'s (actually, the latest one I can find is from Mike
>> Hendrickson http://radar.oreilly.com/2008/03/state-of-the-computer-book-mar-23.html)
>> analysis of the IT book market is based on hard data but, of course,
>> is not a direct measure of the usage of languages. There are obvious
>> problems with correlating book purchase with language use (Somebody
>> who has been writing Fortran for 20 years is probably not going to be
>> buying a book on Fortran this quarter - Very few of the people buying
>> Haskel books this quarter will be using the language for serious
>> work).
>
> Indeed.
>
> Book sales are much less driven by actual usage as they are by hype.
> So, in that sense they can be useful as a leading indicator but they
> aren't good at all for actual usage.
I think what Tim uses book sales numbers for is to indicate uptick in
adoption, rather than overall usage. If a language is great and
basically finished for a decade, it's unlikely people will be buying a
lot of new books for it. But it also skews toward corporate-mandated
"bad" technologies; I bet SOA has sold lots of books.
> Mailing lists stats are particularly interesting to me obviously
> [ObDisclosure: Mad Scientist of MarkMail :-)] but be careful with them
> as they have a variety of artifacts, too. For example some
> communities show tremendous mailing list volume growth that then
> flattens and declines even as the language becomes increasingly
> popular -- one because they moved to a forum based Q&A without a
> gateway with the mailing list. Also, different communities have
> different cultures and so it's hard to normalize across them -- e.g.,
> some communities have lots of talkers on a regular basis and some are
> more sedate and/or bursty.
This backs up what Tom and I were saying about JRuby's lists. Generally
low mailing list traffic for JRuby is a good thing, because it means
people aren't having trouble using it. For Ruby-related stuff, there's
other channels. And the IRC factor plays in heavily.
- Charlie
Charles Oliver Nutter
> The book sales numbers are good -- I'll take a look at those. Job trends are okay, and I've got a
> page of those I'm following[1].
>
> I'm not a fan of conference attendence -- that may speak more to the culture of. Rubyists in
> particular are a social bunch -- the Pradipta 416 wouldn't have happened if you hit up OCaml
> developers, and I may be going to RubyRX down here in Raleigh, even though I don't really do much
> with Ruby these days. Also, who do you chalk the No Fluff, Just Stuff conference attendance up to?
> Java? Groovy? Scala?
That's a good point, but I think the social aspect of Ruby is exactly
why it's spread so quickly over the past couple years. It's also why a
lot of Ruby discussion happens in "real life" or on IRC rather than
mailing lists, skewing the lists downward.
Here's another metric worth considering...user-group attendance. The
Ruby Users of Minnesota got nearly 50 people to come out in 5F weather
for discussions on testing, Cocoa programming, and Merb's merge into
Rails. We don't always talk about Ruby, but it's still the main love for
most people there. But then again UGs are almost always skewed higher
when things are still new, because the people that attend them like
shiny objects. So our Ruby group probably has better attendance than
local Java groups, mainly because there's not a lot to be said about
Java (or too much to track?).
> I'm putting together a blog post. I might do regular tracking of said metrics.
>
> [1] http://enfranchisedmind.com/blog/job-graphs/
These results are almost certainly skewed by "groovy" being a normal and
reasonably common English word. "Looking for a sheep shearer to work in
a groovy and relaxed environment". I think I've even used "groovy" as an
adjective in a job req before the language was well-known.
There's also a problem with keyword searches in any engine, since many
job postings will only list "Rails" or "Grails" and not their languages.
BTW, expect to get flamed by everyone if you post numbers without also
admitting they probably wrong :)
- Charlie
Robert Fischer
>>
>> [1] http://enfranchisedmind.com/blog/job-graphs/
>
> These results are almost certainly skewed by "groovy" being a normal and
> reasonably common English word. "Looking for a sheep shearer to work in
> a groovy and relaxed environment". I think I've even used "groovy" as an
> adjective in a job req before the language was well-known.
>
> There's also a problem with keyword searches in any engine, since many
> job postings will only list "Rails" or "Grails" and not their languages.
>
Groovy job (that's how Carol picked up up), or even just out-and-out hiring Java developers and
presuming they can get trained in on Groovy.
> BTW, expect to get flamed by everyone if you post numbers without also
> admitting they probably wrong :)
>
Of course -- there'll be the pre-emptive self-flagellation if/when that blog post ever actually forms.
Charles Oliver Nutter
> Another popular issue is people looking for "Java and Ruby experience" when they're hiring for a
> Groovy job (that's how Carol picked up up), or even just out-and-out hiring Java developers and
> presuming they can get trained in on Groovy.
We've also had hours of fun gaming the Indeed job trend searches. For
example, you'd be better off investing in garbage than groovy:
http://www.indeed.com/jobtrends?q=groovy%2C+garbage&l=
However groovy has overtaken nonsense:
http://www.indeed.com/jobtrends?q=groovy%2C+nonsense&l=
And still lags far behind beef:
http://www.indeed.com/jobtrends?q=groovy%2C+beef&l=
And notice also the number of "groovy" job posting there is in the 0.01%
range. What's the margin of error?
I'd trust the Indeed results about as far as I can throw them.
- Charlie
Jon Harrop
> Is there any kind of reasonable language adoption metric out there? I'm
> not sure how one might measure that, but it's hard to tell which languages
> have more mindshare -- get different people with different focuses
> together, and they seem to have wildly different guesses.
That is an impossible question to phrase, let alone answer. :-)
For example, you initially said "adoption" which I assume means rate of uptake
but then you said "mindshare" which is more likely to mean sizes of the
current user bases rather than their dynamics.
I have spent a long time investigating similar questions but I am really after
objectively quantifiable metrics that correlate strongly with profit,
preferably with a lead time so I can predict our profits. You may also want
to know where the fun is and which languages have mature libraries.
Surprisingly, I don't believe anyone has mentioned what I consider to be the
best metric: Google Trends. This page graphs the proportion of search volume
on Google for a given search term. If you search for language names that are
not homonyms and take into account that the growth of the internet is
bringing a lot more non-technical people into play, which produces a general
downward trend for all technical search terms, then I think you've got a very
good metric.
We have had products out on OCaml and F# (and several other languages) for
many years and we are widely known within and around those communities. So,
all other things being equal, it is compelling that the cross-over on Google
Trends when F# started to be more searched for than OCaml correlated with F#
starting to pull in more money for us than OCaml:
http://www.google.com/trends?q=ocaml%2C+f%23
Then the jumps when Microsoft made major announcements around F# also
correlated with jumps in our F#-based revenue. So I think there is a strong
correlation. Unfortunately, this breaks when the language name is a homonym,
e.g. Lisp, Scheme, Java, Scala.
However, that correlation with profit appears to have broken down this month,
as F# has all but died for us and OCaml is doing much better despite the
Google trends being unchanged. I assume this is the media with tales of doom
about Microsoft and the global economy (and Obama!) driving people away from
Microsoft and towards OSS.
People have mentioned other metrics. I don't have much faith in TIOBE because
I believe it measures cumulative usage (i.e. the number of webpages ever
written about Fortran is huge but current usage of Fortran is small and its
uptake is tiny) and any estimates of dynamics as rate of change of that (i.e.
new pages about Fortran) will have huge errors.
Book sales would be good if you could get accurate statistics about them.
Alas, O'Reilly's stats are tainted by measuring only units of cheap books
sold, neglecting profits and expensive non-mainstream books. For example,
O'Reilly did not count my book OCaml for Scientists which has huge profit
margins. Then there is the problem that new books have outdated those
statistics. The three existing F# books are selling more copies than books
for almost all other functional languages except the new Haskell book which
is selling even more. Then there is the question of what book sales are
actually measuring. I own several Haskell books but have never written
anything significant in Haskell. I can also tell you that a lot of people buy
our journal subscriptions only to let them expire without ever logging in and
reading anything. So book sales are probably only useful to you if you are in
the book selling market.
Job listings are interesting but people often take only US jobs and the
dynamics very enormously from one country to the next. For example, the US is
relatively static for the major languages but the UK recently saw a dramatic
shift away from C++ and Java and towards C#:
http://www.itjobswatch.co.uk/jobs/uk/c++.do
http://www.itjobswatch.co.uk/jobs/uk/java.do
http://www.itjobswatch.co.uk/jobs/uk/csharp.do
In the open source world you can also get a good estimate of the situation on
Linux from the Debian and Ubuntu package popularity contest results because
those distros cover about 75% of Linux users. That is around a million
Debian-based machines so the statistics are good. For compiled languages, you
can get the most by looking at the number of compiler installs because end
user software does not depend on the compiler. You can even see how many have
been run in the past 30 days! However, that is useless for languages with any
serious use outside Linux (e.g. Java, C#, F#).
I don't like mailing list traffic at all, basically because it does not
correlate with anything and has many obvious flaws, like the huge variations
in signal to noise ratios.
Ultimately, all of these kinds of measures are rendered useless by other
effects. For example, many older-generation functional languages like OCaml
and Haskell have technical shortcomings that undermine the ability to sell
commercial libraries to programmers using those languages whereas, of course,
languages like F# do not have that problem. Consequently, we sell a lot more
F# DLLs than OCaml libraries regardless of their relative popularity. If you
didn't want to know where the money is then you may have wanted to know where
the fun is and, of course, that may well be very different from where the
people are for similar reasons.
On a related note, I like studying success stories, i.e. popular of software
written in different languages rather than the popularity of the languages
themselves. This can require quite some detective work though, as I
discovered in 2007 when I began writing a business management
report "Functional programming in Industry" only to discover that a lot of
the information floating around in certain circles is either grossly
misleading or just plain wrong. I ended up shelving the report. I was also
surprised to discover huge variations in the amount of widely-used code
written in similarly-popular languages, i.e. some language communities churn
out huge numbers of libraries that nobody ever uses.
David MacIver
And here's a more insidious problem with search engine results: they
John D. Mitchell wrote:
>> TIOBE say they use search engine metrics using the search term
>> "<language> programming" with some language specific post processing
>> (see http://www.tiobe.com/index.php/content/paperinfo/tpci/tpci_definition.htm)
>> .
>> Once the percentage score for a language falls below 5% I don't think
>> the numbers are significant. Their longer term trends look to be more
>> valuable and show just how jittery the metric is (I'm sure the actual
>> usage of established language does not exhibit this degree of jitter.
>> What we are seeing is an artefact of the metric. And what happened in
>> 2004! http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html)
>> .
>
> Indeed! Just using raw text search metrics is wildly bad. One nasty
> effect is the low end numbers often get lost from the crawling and
> indexing.
don't filter out false positives. So for a reasonably common word like
"groovy" you get inflated results. Even "groovy programming" gets hits
from programmers that think programming is groovy. "jruby" on the other
hand has pretty much only one meaning, as does "scala", "clojure", and
Robert Fischer
~~ Robert.
> meanings within computing.
>
> >
Jochen Theodorou
>> There's also http://langpop.com/ which is very up-front about its
>> methodology -Tim
>
> Yeah, that's mine, thanks for mentioning it!
>
> I also attempt to grab more sources than TIOBE does, to just give
> people a lot of different things to look at. Right now, for instance,
> I'm working to add irc channels in the 'talk' portion of the stats.
>
> It's impossible to do really good statistics of this type, but I think
> it's fun to see what turns up in any case, and while there are some
> problems (which I do note on the site), by and large, I think the
> results do reflect what my "nose in the wind" tells me about what's
> going on.
hmm... what constraints has a language to fulfill to get on your page...
because Groovy is not on it ;) it is in the TIOBE top 50
bye blackdrag
--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
Joseph Wofford
http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html
It may not be reasonable, but at least it's reasonably objective.
David Welton
those who are interested: http://groups.google.com/group/langpop/ ]
On Wed, Jan 28, 2009 at 6:53 PM, Jon Harrop <j...@ffconsultancy.com> wrote:
>
> On Wednesday 28 January 2009 09:48:09 David Welton wrote:
>> > There's also http://langpop.com/ which is very up-front about its
>> > methodology -Tim
>>
>> Yeah, that's mine, thanks for mentioning it!
>
> Can you add F#, Scala and Clojure, please?
They are probably a bit new/unestablished to do much more than cluster
around the bottom of the chart. However, I'm considering the idea of
an "up and coming" chart where people could track languages like
those.
Someone else writes:
> hmm... what constraints has a language to fulfill to get on your page...
> because Groovy is not on it ;) it is in the TIOBE top 50
It should solidly register on all the metrics used. "Up and comers"
often don't have anything on Freshmeat, Google code, Craigslist, and
maybe only have a book or two, if that.