[erlang-questions] Erlang 10 years of Open Source; it is time for the next step

4 views
Skip to first unread message

Torbjorn Tornkvist

unread,
Mar 17, 2008, 6:38:44 AM3/17/08
to erlang-q...@erlang.org
Hi,

I just got back from QCon 2008 in London.
Lots of interesting presentations, and among them
one by Joe Armstrong about Erlang.

One talk was about the Eclipse team and how they had
gone from a closed development style to a style where
all their internal processes was completely open to public(*).

It is now 10 years (I think...) since Erlang became Open Source.
The last year, the interest in Erlang has exploded and I think
it is time to take the next step; open up the development of Erlang.

Now, I know that Erlang is buried in Clearcase somewhere in the
dungeons of Ericsson and that it probably will be a major undertaking
to change the "corporate policy". I think however that the matter
is worth discussing.

So, suggestions:

- Put Erlang into a modern (distributed) VCS like Mercurial or Git.
- Make it possible to browse the full repository.
- Open up the bug-tracker to become fully public.
- Publish the testserver result, e.g using CruiseControl.
- Make the whole testsuite available for download and deployment.
- Open up any other (as of today) internal dev.tools, wikis etc.

This way, it would be easy for the public to provide well tested
patches that easily can be tracked. It would make it easier both
for the OTP team to cherry-pick patches as well as make it easier
to maintain non-accepted patches in ones own respositories.

Cheers, Tobbe

(*) www.jazz.net


Paul Fisher

unread,
Mar 17, 2008, 6:54:02 AM3/17/08
to Torbjorn Tornkvist, erlang-q...@erlang.org
On Mon, 2008-03-17 at 11:38 +0100, Torbjorn Tornkvist wrote:
> So, suggestions:
>
> - Put Erlang into a modern (distributed) VCS like Mercurial or Git.
> - Make it possible to browse the full repository.
> - Open up the bug-tracker to become fully public.
> - Publish the testserver result, e.g using CruiseControl.
> - Make the whole testsuite available for download and deployment.
> - Open up any other (as of today) internal dev.tools, wikis etc.

+1


--

paul


Gleb Peregud

unread,
Mar 17, 2008, 7:50:24 AM3/17/08
to Paul Fisher, erlang-q...@erlang.org, Torbjorn Tornkvist
Great idea. +1

> _______________________________________________
> erlang-questions mailing list
> erlang-q...@erlang.org
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


--
Gleb Peregud
http://gleber.pl/

Every minute is to be grasped.
Time waits for nobody.
-- Inscription on a Zen Gong

Ulf Wiger

unread,
Mar 17, 2008, 7:53:37 AM3/17/08
to Torbjorn Tornkvist, erlang-q...@erlang.org
2008/3/17, Torbjorn Tornkvist <to...@tornkvist.org>:

>
> So, suggestions:
>
> - Put Erlang into a modern (distributed) VCS like Mercurial or Git.

Did the Eclipse team mention who the legal owner of the Eclipse
code base is?

I ask because one of the problems for any company involved in
Open Source development is the fact that it's legally responsible for
all the code in the repository. This is perhaps a bigger problem for
a large company than a small one, int that large companies make
for juicy targets in IPR lawsuits.


> - Open up the bug-tracker to become fully public.

...under the proviso, of course, that proprietary information given
in bug reports are not revealed to the public.

> - Publish the testserver result, e.g using CruiseControl.
> - Make the whole testsuite available for download and deployment.

Yes.


> - Open up any other (as of today) internal dev.tools, wikis etc.

Same thing here. OTP would have to find a way to keep confidential
information confidential.

BR,
Ulf W

Paul Fisher

unread,
Mar 17, 2008, 8:40:14 AM3/17/08
to Ulf Wiger, erlang-q...@erlang.org
On Mon, 2008-03-17 at 12:53 +0100, Ulf Wiger wrote:
> 2008/3/17, Torbjorn Tornkvist <to...@tornkvist.org>:
> >
> > So, suggestions:
> >
> > - Put Erlang into a modern (distributed) VCS like Mercurial or Git.
>
> Did the Eclipse team mention who the legal owner of the Eclipse
> code base is?
>
> I ask because one of the problems for any company involved in
> Open Source development is the fact that it's legally responsible for
> all the code in the repository. This is perhaps a bigger problem for
> a large company than a small one, int that large companies make
> for juicy targets in IPR lawsuits.

Using git (or mercurial) is more about enabling the acceptance of
community (distributed) development into the core distribution. The
official tree would still reside where it is, just that the tools enable
community contributions more simply and effectively. Having internal
development done via git would promote a workflow where community
contributions could be on equal footing (given that they have merit.)

IPR is a separate issue, and the same efforts would need to be made to
contribution integrated into upstream tree. What is done with patches
submitted today?


--
paul


Convey Christian J NPRI

unread,
Mar 17, 2008, 8:52:54 AM3/17/08
to erlang-q...@erlang.org
> -----Original Message-----
> From: erlang-quest...@erlang.org
> [mailto:erlang-quest...@erlang.org] On Behalf Of Ulf Wiger
> Sent: Monday, March 17, 2008 7:54 AM
> To: Torbjorn Tornkvist
> Cc: erlang-q...@erlang.org
> Subject: Re: [erlang-questions] Erlang 10 years of Open

[snip]

>
> > - Open up any other (as of today) internal dev.tools, wikis etc.
>
> Same thing here. OTP would have to find a way to keep
> confidential information confidential.

One solution I've seen is to have two repositories. One repository contains the open-source core product, which can be packaged and installed on its own.

The other repository has the proprietary component. To build the proprietary component, you must have built the open source component and perhaps also have installed it.

This solution isn't without its problems, but it's probably the cleanest solution I've seen to the problem it was meant to solve.

- Christian

Masklinn

unread,
Mar 17, 2008, 9:31:15 AM3/17/08
to Erlang Questions

On 17 Mar 2008, at 13:52 , Convey Christian J NPRI wrote:
> The other repository has the proprietary component. To build the
> proprietary component, you must have built the open source component
> and perhaps also have installed it.

>

It could be even simpler: the proprietary "component" could "simply"
be a fork of the open-source one, additions to the open-source product
would be merged into the proprietary one as needed but not the other
way around. Mercurial and Git both provide facilities for that kind of
workflow.

Elliot Murphy

unread,
Mar 17, 2008, 10:21:39 AM3/17/08
to erlang-q...@erlang.org
Oops, had intended to send this to the whole list.


---------- Forwarded message ----------
From: Elliot Murphy <elliot...@gmail.com>
Date: Mon, Mar 17, 2008 at 9:44 AM
Subject: Re: [erlang-questions] Erlang 10 years of Open Source; it is
time for the next step
To: Torbjorn Tornkvist <to...@tornkvist.org>


Hi!


On Mon, Mar 17, 2008 at 6:38 AM, Torbjorn Tornkvist
<to...@tornkvist.org> wrote:
> Now, I know that Erlang is buried in Clearcase somewhere in the
> dungeons of Ericsson and that it probably will be a major undertaking
> to change the "corporate policy". I think however that the matter
> is worth discussing.

Corporate policy or not, changing version control systems is a lot of work,
and should not be underestimated. I've helped several large companies
change their
version control system, and I'm happy that I can say it is a
reasonable thing to do.

Much of the work can be done up front by a handful of people - the most
important thing to get right is selecting a tool and doing the history
conversion.


>
> So, suggestions:
>
> - Put Erlang into a modern (distributed) VCS like Mercurial or Git.
> - Make it possible to browse the full repository.

I've recently imported the source tarballs into Bazaar, which is an
adaptive/distributed version control system that we use at Canonical:
https://code.launchpad.net/~erlang-dev/erlang/trunk

I was working from the source drops, without access to ClearCase, so
the history is naturally limited to seeing the changes between releases. Still,
it's a good step in the right direction. I am currently working with other
companies to convert major projects from proprietary VCS to Bazaar, while
preserving all their important history, and I'd be delighted to do the same
for the Erlang project.


> - Open up the bug-tracker to become fully public.
> - Publish the testserver result, e.g using CruiseControl.
> - Make the whole testsuite available for download and deployment.
> - Open up any other (as of today) internal dev.tools, wikis etc.
>
> This way, it would be easy for the public to provide well tested
> patches that easily can be tracked. It would make it easier both
> for the OTP team to cherry-pick patches as well as make it easier
> to maintain non-accepted patches in ones own respositories.

I think that is a great idea. My day job is working on https://launchpad.net
(click on the tour button for more info),
which is a web based development/collaboration suite with mailing lists,
code hosting, distributed bug tracker, support tools, I18N tools, and much
much more. One of the core ideas behind Launchpad is to deal with
dependencies in software - you get a lot of those when building a linux
distribution for example. The place where the bug is reported is not
necessarily
the place where it needs to be fixed.

The other place where we have seen the ideas we've built in Launchpad work
really really well is in communities based on a programming language - we've
seen significant numbers of python projects moving to Launchpad, for example.

All this rambling is to say that I think this is a great idea, and we at
Launchpad would be happy to provide as much support for the Erlang project
as the community would like. Two recent projects that moved are Zope and
Inkscape. I was really happy to see how the community reacted to the Inkscape
move: http://bryceharrington.org/drupal/node/18

One of the tricky bits with opening up a bug tracker is that sometimes you
have gotten customer information mixed into the bug tracker that absolutely
needs to be kept private, while the majority of the bugs are actually about
technical bugs and could easily be available to the public. We've dealt with
that in Launchpad by building a private bug capability, so if something like
migrating the bugs from the internal tracker to a public bug tracker
on Launchpad
was desired we could flag bugs as private as needed.

--
Elliot Murphy

--
Elliot Murphy

Massimo Cesaro

unread,
Mar 17, 2008, 11:39:00 AM3/17/08
to erlang-q...@erlang.org


On Mon, Mar 17, 2008 at 11:38 AM, Torbjorn Tornkvist <to...@tornkvist.org> wrote:

It is now 10 years (I think...) since Erlang became Open Source.
The last year, the interest in Erlang has exploded and I think
it is time to take the next step; open up the development of Erlang.

Please help me to understand: What are the real benefits in doing this?
Erlang and OTP are a good example of conceptual integrity, mostly because they are designed to solve real world problems; what are the benefits in allowing, for example, the forking of the build tree?
The language and its libraries are not static, they are maintained by Ericsson in a win-win situation for the end user. Until now, the Erlang maintainers were quite responsive to the Erlang community requests, and the overall quality of their releases is close to excellence.
Sure, maybe accessing the Erlang bug tracking system could be really useful, but all the bugs signaled to the mailing list are at least acknowledged by the developers.
The EEP initiative also contribute to the development of the language, and although someone criticized it, I still have to find another development tool which is so well documented.
I'm not an expert in the management of open source projects, but what could the Erlang users gain from the bazaar?

Massimo

Michael Campbell

unread,
Mar 17, 2008, 11:48:59 AM3/17/08
to erlang-q...@erlang.org
Elliot Murphy wrote:
> Oops, had intended to send this to the whole list.

*chuckle*

I wish I had a nickel (i.e. 5% of a US Dollar) every time this happened to people.

Matthias Lang

unread,
Mar 17, 2008, 12:06:35 PM3/17/08
to erlang-q...@erlang.org

> > Oops, had intended to send this to the whole list.

> I wish I had a nickel (i.e. 5% of a US Dollar) every time this
> happened to people.

It's often a whole lot more entertaining when the reverse happens.

Matt

Dmitrii Dimandt

unread,
Mar 17, 2008, 12:19:56 PM3/17/08
to Massimo Cesaro, erlang-q...@erlang.org
End users could benefit from an earlie adoption of libraries like Starling (ICU binding) and erlycairo. 

Jim Miller

unread,
Mar 17, 2008, 12:34:27 PM3/17/08
to Massimo Cesaro, erlang-q...@erlang.org
>
> Please help me to understand: What are the real benefits in doing this?
> Erlang and OTP are a good example of conceptual integrity, mostly because
> they are designed to solve real world problems; what are the benefits in
> allowing, for example, the forking of the build tree?
> The language and its libraries are not static, they are maintained by
> Ericsson in a win-win situation for the end user. Until now, the Erlang
> maintainers were quite responsive to the Erlang community requests, and the
> overall quality of their releases is close to excellence.
> Sure, maybe accessing the Erlang bug tracking system could be really
> useful, but all the bugs signaled to the mailing list are at least
> acknowledged by the developers.
> The EEP initiative also contribute to the development of the language, and
> although someone criticized it, I still have to find another development
> tool which is so well documented.
> I'm not an expert in the management of open source projects, but what could
> the Erlang users gain from the bazaar?
>
> Massimo

I would second this question and argue against opening the repository
unless a clear advantage over the existing system, besides the "I want
to contribute", can be demonstrated. Contrary to what we open source
advocates may believe, OS is not the oasis of bug free code, useable,
tested code.

Masklinn

unread,
Mar 17, 2008, 12:46:32 PM3/17/08
to Erlang Questions

On 17 Mar 2008, at 16:39 , Massimo Cesaro wrote:

> On Mon, Mar 17, 2008 at 11:38 AM, Torbjorn Tornkvist <to...@tornkvist.org
> >
> wrote:
>
>>
>> It is now 10 years (I think...) since Erlang became Open Source.
>> The last year, the interest in Erlang has exploded and I think
>> it is time to take the next step; open up the development of Erlang.
>>
>
> Please help me to understand: What are the real benefits in doing
> this?

Note that the following are my guesses, or the benefits I would see

>
> Erlang and OTP are a good example of conceptual integrity, mostly
> because
> they are designed to solve real world problems; what are the
> benefits in
> allowing, for example, the forking of the build tree?

Having a full history, allowing people out of Ericsson to create
patches (that could then be forwarded to the core team for inclusion)
or to create temporary or permanent forks to test out features (that
would probably be much rarer and harder).

The first suggestion is one that is extremely well supported by DVCS:
any user can clone the repository (which is the equivalent of checking
it out in e.g. Subversion), then hack around (patch bugs, try to find
things to cleanup, or even just create documentation patches which are
already invaluable), save everything in his local repository and when
needed "patchbomb" an erlang dev list (basically send revision as
inline patches) so that they can be reviewed and considered for
inclusion.

The patchbomb step could be replaced by "attach patches to bugs" if a
bug tracker is opened (the former is the way Mercurial development
works, the latter is the way Django's works -- in the Python world)

Conceptual integrity would be preserved because only the people who
already have commit rights now would still have it, and these people
would therefore have veto power over patches and suggestions, being
able to enforce design decisions or style issues (and possibly ignore
patches altogether if they don't have the time).

DVCS would just make contributions easier by allowing everybody to
hack around locally (with history) and have an history of the Erlang
development for documentation as well as bug-finding (bisection
search) purposes.

>
> The language and its libraries are not static, they are maintained by
> Ericsson in a win-win situation for the end user. Until now, the
> Erlang
> maintainers were quite responsive to the Erlang community requests,
> and the
> overall quality of their releases is close to excellence.

The quality wouldn't have any reason to drop as the people holding the
"keys to the kingdom" now would still have it. But on the other hand
the people having the desire to contribute patches or documentation
would have an easier time doing it. Likewise, experiments based on
erlang (loosely or tightly) would be easier for those not part of the
core team.


Hal Snyder

unread,
Mar 17, 2008, 2:25:05 PM3/17/08
to Erlang Questions
DVCS is a disruptive technology that improves concurrency. :-)
The improvement is in software process, rather than (near) real-time
networks.

Linus summarized the general advantages of DVCS [git,hg,bzr|...] and
some of the method in the YouTube video
http://youtube.com/watch?v=4XpnKHJAok8
I think the general advantages could apply here.

Jim Miller

unread,
Mar 17, 2008, 2:36:43 PM3/17/08
to Erlang Questions
This is a fine exposition on how the tool can facilitate this process
but fails to address the question of "What is the economic (not money)
argument for making Erlang open source?"

Jim Miller

unread,
Mar 17, 2008, 2:38:51 PM3/17/08
to Dmitrii Dimandt, erlang-q...@erlang.org
On Mon, Mar 17, 2008 at 2:36 PM, Dmitrii Dimandt <dmit...@gmail.com> wrote:

>
>
> On Mar 17, 2008, at 6:37 PM, Jim Miller wrote:
>
> >>
> >> End users could benefit from an earlie adoption of libraries like
> >> Starling
> >> (ICU binding) and erlycairo.
> >
> > What prohibits the type of end users that are early adopters from
> > using these now?
>
> these libraries are all over the place, they are hard to come by. If
> they could all be pulled down from a single repository even from an
> unstable branch, that would be awsome
>

Other communities (Scala and OCaml are currently dealing with this)
address this by setting up a repository of third party extensions that
play well together and hopefully are on a migration path to inclusion
in the baseline. This is a very different beast than opening up the
main repository.

I would completely support and agree with an ErlangX repository for
this purpose.

Massimo Cesaro

unread,
Mar 18, 2008, 4:21:07 AM3/18/08
to Erlang Questions
On Mon, Mar 17, 2008 at 5:46 PM, Masklinn <mask...@masklinn.net> wrote:

On 17 Mar 2008, at 16:39 , Massimo Cesaro wrote:

Having a full history, allowing people out of Ericsson to create
patches (that could then be forwarded to the core team for inclusion)
or to create temporary or permanent forks to test out features (that
would probably be much rarer and harder).
 
Exactly my point. There's no clear advantage in doing this.

The first suggestion is one that is extremely well supported by DVCS:
any user can clone the repository (which is the equivalent of checking
it out in e.g. Subversion), then hack around (patch bugs, try to find
things to cleanup, or even just create documentation patches which are
already invaluable), save everything in his local repository and when
needed "patchbomb" an erlang dev list (basically send revision as
inline patches) so that they can be reviewed and considered for
inclusion.

The patchbomb step could be replaced by "attach patches to bugs" if a
bug tracker is opened (the former is the way Mercurial development
works, the latter is the way Django's works -- in the Python world)

Ok, but I was not questioning about the tools used to allow external development. My concerns are on the opportunity on changing the maintining mode.

Conceptual integrity would be preserved because only the people who
already have commit rights now would still have it, and these people
would therefore have veto power over patches and suggestions, being
able to enforce design decisions or style issues (and possibly ignore
patches altogether if they don't have the time).

DVCS would just make contributions easier by allowing everybody to
hack around locally (with history) and have an history of the Erlang
development for documentation as well as bug-finding (bisection
search) purposes.

Indeed, but still a supervision from the core development team is required. Just like today: my experience with this mailing list is that if somebody has a really useful patch to submit either for the code or for the docs, he/she just inform of the availability of the patch to the fine people at Ericsson thourgh the list. I bet that after a close inspection, valuing the pro and cons of the patch, they'll schedule it for an upcoming release. I don't see how automating a small part of this process can simplify the life of a developer.
If I remember well, I know of a user contributed patch solving a real problem; this was acknowledge by the Erlang team, but was rewritten because it contained bugs/constraints/whatever that would impact on existing code.
They'll find problems because is their job to run regression testing among the other tests.
My personal experience with "truly" open source project is that the part time maintainer has no time/equipment to test it all, so first he just release the stuff, and then lets the bazaar to sort it out.

>
> The language and its libraries are not static, they are maintained by
> Ericsson in a win-win situation for the end user. Until now, the
> Erlang
> maintainers were quite responsive to the Erlang community requests,
> and the
> overall quality of their releases is close to excellence.

The quality wouldn't have any reason to drop as the people holding the
"keys to the kingdom" now would still have it. But on the other hand
the people having the desire to contribute patches or documentation
would have an easier time doing it. Likewise, experiments based on
erlang (loosely or tightly) would be easier for those not part of the
core team.
 
My point is: we (Erlang professional users) should avoid the generation of different Erlang flavors, based on experiments or otherwise. I think that one of the often overlooked task of the Erlang team is keeping it up to date on several platforms, evaluating the risks and the benefits of each single patch, giving a warranty that no existing, deployed code will break.
They (Erlang team) have a formal procedure of doing this; my experience with open source enthusiast is that sometime patches and upgrades are done in a "spray and pray" mood.

Massimo

Mats Cronqvist

unread,
Mar 18, 2008, 4:38:16 AM3/18/08
to Jim Miller, Erlang Questions
i don't understand this question.
firstly, erlang is already open source.
secondly, what does "economic (no money)" mean?
thirdly, argument for whom? Ericsson?

i'll answer this question instead; "why should ericsson let the world
at large see the repos and the bug tracker?"
because it would improve the quality of the erlang based Ericsson
products.

mats

mats_cronqvist.vcf

Massimo Cesaro

unread,
Mar 18, 2008, 5:51:41 AM3/18/08
to Erlang Questions


On Tue, Mar 18, 2008 at 10:26 AM, Matthew Dempsky <mat...@dempsky.org> wrote:
On 3/18/08, Massimo Cesaro <massimo...@gmail.com> wrote:
> My point is: we (Erlang professional users) should avoid the generation of
> different Erlang flavors, based on experiments or otherwise.

Why?  Python has a handful of flavors based on different experiments
(Jython, Stackless, PyPy), and the community has not collapsed.  If
someone really were to implement Jerlang, there's no reason you'd be
forced to use it.
But Jython and the likes are not Python; I don't know about Python, but I think that the Python community is sticking to the kosher Python as defined by Guido van Rossum. If somebody wants to stretch and flex the language to adapt it to a problem domain for which it wasn't designed, it's just their option. I would personally try to find out a language/development environment closer to what is the problem at hand.


Looking at the HiPE website, they mention it was merged into OTP in
2001, but their publications page lists documents from as far back as
1996.  I read into this that HiPE was just such an experimental flavor
for the first 5 years of its life.
I wouldn't call HiPE a flavor. HiPE is a native compiler derived from Erlang specifications which eventually was merged in the language distribution itself. If I remember HiPE history, it wasn't born out of the bazaar model, but rather out of a university research with a closed team. Today, I guess HiPE follows the same quality process of Erlang and OTP.

I'm sorry, I didn't want to start a flame war. It's just that I strongly believe that the Erlang community backed by the Ericsson people is really an example of excellence in the world of open source.

Massimo

Matthew Dempsky

unread,
Mar 18, 2008, 5:26:22 AM3/18/08
to Massimo Cesaro, Erlang Questions
On 3/18/08, Massimo Cesaro <massimo...@gmail.com> wrote:
> My point is: we (Erlang professional users) should avoid the generation of
> different Erlang flavors, based on experiments or otherwise.

Why? Python has a handful of flavors based on different experiments


(Jython, Stackless, PyPy), and the community has not collapsed. If
someone really were to implement Jerlang, there's no reason you'd be
forced to use it.

Looking at the HiPE website, they mention it was merged into OTP in

Christian S

unread,
Mar 18, 2008, 8:49:14 AM3/18/08
to Massimo Cesaro, Erlang Questions
> I'm sorry, I didn't want to start a flame war. It's just that I strongly
> believe that the Erlang community backed by the Ericsson people is really an
> example of excellence in the world of open source.

Are you aware of that many production installations today of OTP are
locally patched versions?

So companies are having the problem of managing their patches and
porting them to new otp releases right now.


As a compromise, could otp start to publish each accepted change-set
between releases? More contained change-sets should make it easier to
fast forward patches and identify what changes in otp clash with your
local patches.

Or an even better compromise: publish them to an open git^W DSCM
repository directly instead of having the community do so.

Matthew Dempsky

unread,
Mar 18, 2008, 8:52:50 AM3/18/08
to Massimo Cesaro, Erlang Questions
On 3/18/08, Massimo Cesaro <massimo...@gmail.com> wrote:
> But Jython and the likes are not Python; I don't know about Python, but I
> think that the Python community is sticking to the kosher Python as defined
> by Guido van Rossum.

The point of projects like Stackless and PyPy are not to get the
community to move to a new platform. They're domains for
experimenting with new ideas without complicating the baseline
implementation. If those experiments prove successful, they're likely
to gain wider attention and use (e.g., a lot of Stackless features
have been integrated into PyPy).

> I wouldn't call HiPE a flavor. HiPE is a native compiler derived from Erlang
> specifications which eventually was merged in the language distribution
> itself.

>From the HiPE manual at
http://www.it.uu.se/research/group/hipe/documents/hipe_manual.pdf:

"The HiPE system is intended as an experimental platform for research
in language implementation and compiler techniques as applied to
Erlang. One goal with this system is that [it] should be a complete
Erlang implementation."

This is exactly what Jython/Stackless/PyPy/IronPython/etc. are.

Jim Miller

unread,
Mar 18, 2008, 9:24:58 AM3/18/08
to Mats Cronqvist, Erlang Questions
Because this is a fun discussion, I'll continue weighing in my thoughts

On Tue, Mar 18, 2008 at 4:38 AM, Mats Cronqvist
<mats.cr...@kreditor.se> wrote:
>
> Jim Miller wrote:
> > This is a fine exposition on how the tool can facilitate this process
> > but fails to address the question of "What is the economic (not money)
> > argument for making Erlang open source?"
> i don't understand this question.
> firstly, erlang is already open source.

Poor choice of words on my part. What I meant to say was "What is the
economic argument for opening the central source repository for access
to others outside of Ericcson?"

> secondly, what does "economic (no money)" mean?

Simply the weighing the costs (time, effort) against the return
(improved software quality, a better product). It can involve money
but doesn't necessarily.

> thirdly, argument for whom? Ericsson?

Of course, its their product

>
> i'll answer this question instead; "why should ericsson let the world
> at large see the repos and the bug tracker?"
> because it would improve the quality of the erlang based Ericsson
> products.
>

Open source does not translate into "better quality" software. One
can simply look at freshmeat or any of the other open source projects
out there to see that for every one or two quality open source
projects there are literally thousands that are horrible. As stated
by a few others on this discussion most open source projects are coded
by people who "pray and spray" and who do not go through the rigors of
true testing for a production environment. I know that not everyone
falls into this category, nor am I categorizing people on this list as
such.

Lets look at the cost of what it would take Ericsson to open up the
repository as is being discussed.

- Employee time spent moving things out of clearcase into another
repository while maintaining source history. Yes, one can argue that
simply taking a snapshot of the current state and starting a new
repository might be acceptable but I doubt that would be acceptable to
them. This takes a serious amount of time from an employee who would
otherwise be working on things that directly provide revenue for a
company (primary purpose of companies is to make money for the
shareholders). This is a one to two week investment just to get the
ball rolling.

- Now, for every patch that is submitted through the process, an
employee has to review the submission, verify whether it is acceptable
or not, approve or reject it, write a note to the submitter if its
been rejected (or check-in comment), and roll it into the baseline if
it is. This is for EVERY submission regardless of whether or not it
is part of the business goals of the company. Based on my experience
with having been the make mom for a bunch of good size projects, this
can take a huge amount of time. Very quickly it gets to the point
where that person is doing nothing but dealing with patches.
Interviews with Linus Torvalds indicate that the bulk of his time on
the kernel isn't spent doing coding, its dealing with submissions.

So the overall cost to start something like this is basically
allocating one whole person to it. In the companies I typically work
with (Washington DC area software shops) this is roughly $200K

What are the benefits:

- Quicker inclusion of new capabilities.
True, but are these capabilities that help Ericsson? Does a new
wrapper around a vector graphics library (libcairo) help a company
that writes the embedded software for network hardware? There are
some fantastic libraries out there but what percentage of them are
suitable for Ericsson's goals?

- Better code quality
Maybe. Most code I've seen written on the net does not do nearly
the job required for testing that I require for production quality
code my team releases. If you're Ericsson, and you're achieving 7 or
9-9s of system uptime, a single bug can introduce serious problems.
Do you trust the most software developers out there to be rigorous
enough to do this? Its tedious, hard, unsexy work and most people
just don't do it.

So, does Ericcson get more than $200K a year benefit out of opening
their repository? I'd have a hard time seeing it. A good read is
"The Cathedral and the Bazaar" where Eric Raymond goes through the
thought processes behind what makes for good, successful open source
software. In almost all of the cases, because the original author is
not making money off of the software, the cost/benefit analysis works
out in favor of opening it up. In this case, because Ericsson is
making money from the way they do it, it doesn't make sense.
Additionally because they are paid to do it, there is the motivation
to do the tedious, hard, unsexy work that makes the language and
platform as stable and robust as it is.

And by comparison, the opening of Java seems to have been a success,
but what was behind it? If one looks at the political and corporate
history around Java, they resisted opening for years and finally only
did so under pressure from the surging of the .NET platform. If Sun
wanted Java to survive, they needed the open source community to fully
embrace Java, which they weren't going to do without Sun opening it
up. Sun's business model and strategy for Java required acceptance of
the language which wasn't happening and they were losing ground to
.NET. (Go back in the business news archives and look at what was
going on between Sun and Microsoft, their business strategies, and how
Sun has been losing money, its very interesting reading). Sun was
quickly finding that they were not making money off of Java or its
products using the previous business model so they changed models. It
would be real interesting to see what type of money Sun had to spend
on their Java team before and after the change.

Ulf Wiger (TN/EAB)

unread,
Mar 18, 2008, 9:39:47 AM3/18/08
to Christian S, Erlang Questions
Christian S skrev:

>
> Are you aware of that many production installations today of OTP are
> locally patched versions?

Surely not nearly as many as there are locally patched FreeBSD copies
out there. ;-) (although I wish there were...)

BR,
Ulf W

Kenneth Lundin

unread,
Mar 18, 2008, 11:42:19 AM3/18/08
to Torbjorn Tornkvist, erlang-q...@erlang.org
Hi,

We have on our agenda to open up step by step in a way that we
think is advantageous for our own job and for the Erlang community.
We want to keep the quality on the same or higher level.
We don't want there to be many dialects of Erlang just because we
don't think that
is good for the community.
But it is a matter of prioritizing this over new features and it is
quite hard to motivate that this has higher priority than implementing
new features or improving performance.

See also comments below.

On 3/17/08, Torbjorn Tornkvist <to...@tornkvist.org> wrote:
> Hi,
>
> I just got back from QCon 2008 in London.
> Lots of interesting presentations, and among them
> one by Joe Armstrong about Erlang.
>
> One talk was about the Eclipse team and how they had
> gone from a closed development style to a style where
> all their internal processes was completely open to public(*).
>
> It is now 10 years (I think...) since Erlang became Open Source.
> The last year, the interest in Erlang has exploded and I think
> it is time to take the next step; open up the development of Erlang.

We have plans to make Erlang available in a SVN repository.
The plan is that we continue development in Clearcase , since we see
absolutely no advantage in changing that. We will synchronize to and
from the SVN repository
automatically at regular intervals (probably daily)
The repository will be available for anyone with read access..
We will be very restrictive with letting in persons as "committers"


>
> Now, I know that Erlang is buried in Clearcase somewhere in the
> dungeons of Ericsson and that it probably will be a major undertaking
> to change the "corporate policy". I think however that the matter
> is worth discussing.

Clearcase is not a problem.


>
> So, suggestions:
>
> - Put Erlang into a modern (distributed) VCS like Mercurial or Git.

I think we should use a widely used VCS , SVN is widely used and probably good
enough for this.


> - Make it possible to browse the full repository.

The whole point with this is that the source should be browsable.

A completely different thing is letting others than us commit to the repository.
This will be very restrictive in order to keep the stability of the product.

It is worth noting that so far 99% of the patches contributed to
erlang-bugs or erlang-patches need to be corrected or rewritten by us
before they make it into a release. And this has nothing with lack of
VCS to do. The reasons are more of the type:
- solution is not done the way we like to have it
- does not work on all platforms
- buggy
- unacceptable dependencies
- unacceptable incompatibility

> - Open up the bug-tracker to become fully public.

This is a different thing not tightly connected to the VCS issue above.
This would involve a lot of work for us and to what benefit?
It would require us to change bug-tracking system or to use 2 different ones
at the same time.
There is 2 sides of this.
1) The Open source user submitting a bug-report and tracking
bug-reports from other users.
2) Our reception of bug-reports, answering etc. From this point of view it is no
big difference for us in continuing as now with bugreports via
erlang-bugs and erlang-questions. There is nothing wrong with 1) but
for us it probably costs more than it tastes.


> - Publish the testserver result, e.g using CruiseControl.

Why?
All tests should be successful before we release anything.


> - Make the whole testsuite available for download and deployment.

We will probably release some of the test suites step by step.

> - Open up any other (as of today) internal dev.tools, wikis etc.

Why?
What do you think you are missing today?


>
> This way, it would be easy for the public to provide well tested
> patches that easily can be tracked. It would make it easier both
> for the OTP team to cherry-pick patches as well as make it easier
> to maintain non-accepted patches in ones own respositories.

See above about the need to rewrite 99% of the patches. Of course the
patches can be better if there are test-suites available, but I don't
expect that most contributors will be able to test and build on 10 to
20 different OS+CPU combinations.
The number of committers must therefore be very restricted. And it
must be possible to restrict commit to specific branches and subtrees
of the code.

A comparision with Eclipse gives:

Eclipse is run as a foundation supported by it's members, I understand is as
this foundation have both people and machinery to run web-sites,
repositories etc.

We could have a foundation for Erlang as well but we are not there
yet. Currently
it is the Erlang group at Ericsson that has to finance this. The
business case for that is not obvious for everyone within Ericsson.

In Eclipse there are a lot of rules and checklists before you can
commit something. I think all these would be relevant for Erlang as
well.

See http://www.eclipse.org/projects/dev_process/index-quick.php
>
> Cheers, Tobbe
>
> (*) www.jazz.net


>
> _______________________________________________
> erlang-questions mailing list
> erlang-q...@erlang.org
> http://www.erlang.org/mailman/listinfo/erlang-questions
>

In summary , we are as said not negative to opening up more.
But there must be a business case for everything we do.
It is not always enough with the motivation, "What's good for Erlang
is good for Ericsson". Even if I mostly agree to tha myself.

/Kenneth Erlang/OTP team Ericsson

Kevin Scaldeferri

unread,
Mar 18, 2008, 12:33:37 PM3/18/08
to Massimo Cesaro, Erlang Questions

On Mar 18, 2008, at 1:21 AM, Massimo Cesaro wrote:

> If I remember well, I know of a user contributed patch solving a
> real problem; this was acknowledge by the Erlang team, but was
> rewritten because it contained bugs/constraints/whatever that would
> impact on existing code.
> They'll find problems because is their job to run regression testing
> among the other tests.


This seems like an argument for opening up the test suite to other
people.

On CPAN, for example, there's a system for automated distributed
testing of new modules. Many volunteers around the world, with many
diverse systems, automatically download, install, and test new
uploads. This has the potential for much more complete and diverse
testing than even a single large company can manage (and at very low
cost).


-kevin

Torbjorn Tornkvist

unread,
Mar 18, 2008, 12:37:11 PM3/18/08
to erlang-q...@erlang.org
Kenneth Lundin wrote:

Thanx Kenneth, a great reply!

Just to make thing clear: my intention has never been to
suggest that Erlang should open up for public commit rights.

See comment below:

This is excellent news!

However, it is really beneficial to be able to clone a repository.
This way anyone can publish their patches which then can be merged
by anyone with a maintained history.

> We will be very restrictive with letting in persons as "committers"
>> Now, I know that Erlang is buried in Clearcase somewhere in the
>> dungeons of Ericsson and that it probably will be a major undertaking
>> to change the "corporate policy". I think however that the matter
>> is worth discussing.
>
> Clearcase is not a problem.
>> So, suggestions:
>>
>> - Put Erlang into a modern (distributed) VCS like Mercurial or Git.
> I think we should use a widely used VCS , SVN is widely used and probably good
> enough for this.
>> - Make it possible to browse the full repository.
> The whole point with this is that the source should be browsable.
>
> A completely different thing is letting others than us commit to the repository.
> This will be very restrictive in order to keep the stability of the product.
>
> It is worth noting that so far 99% of the patches contributed to
> erlang-bugs or erlang-patches need to be corrected or rewritten by us
> before they make it into a release. And this has nothing with lack of
> VCS to do. The reasons are more of the type:
> - solution is not done the way we like to have it
> - does not work on all platforms
> - buggy
> - unacceptable dependencies
> - unacceptable incompatibility

This is just as it should.

>
>
>
>> - Open up the bug-tracker to become fully public.
> This is a different thing not tightly connected to the VCS issue above.
> This would involve a lot of work for us and to what benefit?
> It would require us to change bug-tracking system or to use 2 different ones
> at the same time.
> There is 2 sides of this.
> 1) The Open source user submitting a bug-report and tracking
> bug-reports from other users.
> 2) Our reception of bug-reports, answering etc. From this point of view it is no
> big difference for us in continuing as now with bugreports via
> erlang-bugs and erlang-questions. There is nothing wrong with 1) but
> for us it probably costs more than it tastes.

For me it would be fantastic if I could see, and search, among the
bug-reports. To be able to see the status, and any planned actions.

We have been maintaining our own version of Erlang/OTP ever since
it became Open Source. I know of at least 3 non-Ericsson companies
in Stockholm that do this. It has always been a hassle: to check if
our fixes has been included in the latest release and if not, to
port them into the new code base. This is also connected with the
next paragraph below.

>
>
>> - Publish the testserver result, e.g using CruiseControl.
> Why?

Because if it was possible to track all the commits you do,
and have them verified by the test suites (e.g eccessible via
CruiseControl). It would be much easier for us to introduce,
maintain and discard our own patches. Perhaps, we even could
come up with alternative solutions when testcases break...


> All tests should be successful before we release anything.
>> - Make the whole testsuite available for download and deployment.
> We will probably release some of the test suites step by step.
>
>> - Open up any other (as of today) internal dev.tools, wikis etc.
> Why?
> What do you think you are missing today?

Well, I think that if there is just one focal point
where all the development and supporting processes take
place the easier should it be for everyone. I mean how
much confidential info do you have hidden that touches
your Erlang development processes ? I can't think of
anything... (ok, I left Ericsson 10 years ago :-)
Not having to think about what can be exposed must
make life easer for you, or ?

--Tobbe

Alex Alvarez

unread,
Mar 18, 2008, 3:51:56 PM3/18/08
to erlang-q...@erlang.org
I have to say that I don't see how could anyone at this point in time question the benefits of open sourcing and how it'd be good for Ericsson to open Erlang development more.  Yes, it cannot save a project that is doom to begin with, but in the case of Erlang it can only help improve it.  Now, this does not mean for Ericsson not to have control on how things progress, quite the opposite.  But with the amount of competition out there for developer minds and souls, Erlang is doom if doesn't keep enticing developers, particularly the ones who are willing to put in the time to help develop the language.  It is just a matter of time for the .NETs, the Javas and other platforms to provide the same level of functionality Erlang provides today.  If Erlang cannot munster and keep enough developers around, it's all over!
 
Cheers, Alex

Rick Pettit

unread,
Mar 18, 2008, 6:15:14 PM3/18/08
to Alex Alvarez, erlang-q...@erlang.org

Sorry, but this is the silliest post I've read on this list in years.

-Rick

Rick Pettit

unread,
Mar 18, 2008, 6:30:30 PM3/18/08
to Alex Alvarez, erlang-q...@erlang.org

Ever wish you could take back hitting the send button on your mail client?

I do :-)

Please allow me to apologize--did not mean to come off like such a jerk.

Erlang is a fantastic language and has been for years. I have production
systems requiring 24x7 uptime which I've maintained (fixed bugs, added
features) without ever taking the system down, not even for a second.

I don't see the java community touching the robustness of erlang/OTP anytime
soon--maybe not in our life time. I've talked with my "java friends" about
the differences, and they just can't hold a candle to what you get with erlang
out of the box (let alone if you know how to code OTP). Ditto for the .NET
folks.

Though it might be nice to open the code up more (I'm not giving an opinion
here, btw), I think it's rather far fetched to claim that "it's all over" if
Ericsson doesn't do X and Y. Every few months someone joins the list to remind
us how we all need to find new jobs (because erlang is going to go away real
soon, etc, etc), but that theory doesn't seem to match reality.

-Rick

Elliot Murphy

unread,
Mar 18, 2008, 1:37:43 PM3/18/08
to Torbjorn Tornkvist, erlang-q...@erlang.org
Hi!

Torbjorn Tornkvist wrote:
<snip>


>>> I just got back from QCon 2008 in London.
>>> Lots of interesting presentations, and among them
>>> one by Joe Armstrong about Erlang.
>>>
>>> One talk was about the Eclipse team and how they had
>>> gone from a closed development style to a style where
>>> all their internal processes was completely open to public(*).
>>>
>>> It is now 10 years (I think...) since Erlang became Open Source.
>>> The last year, the interest in Erlang has exploded and I think
>>> it is time to take the next step; open up the development of Erlang.
>> We have plans to make Erlang available in a SVN repository.
>> The plan is that we continue development in Clearcase , since we see
>> absolutely no advantage in changing that. We will synchronize to and
>> from the SVN repository
>> automatically at regular intervals (probably daily)
>> The repository will be available for anyone with read access..
>
> This is excellent news!
>
> However, it is really beneficial to be able to clone a repository.
> This way anyone can publish their patches which then can be merged
> by anyone with a maintained history.
>

We can easily set up a continuous import on Launchpad.net from a
subversion repository into a bazaar branch. This will make it easy for
people to clone the repository using bazaar, and should not require any
additional work. People could then maintain their own branches in
bazaar, and easily generate patches, merge from the official trunk, etc.
without needing any rights to the subversion repo.
--
Elliot Murphy | https://launchpad.net/~statik/

Christian S

unread,
Mar 19, 2008, 6:16:00 PM3/19/08
to Erlang Questions
> > However, it is really beneficial to be able to clone a repository.
> > This way anyone can publish their patches which then can be merged
> > by anyone with a maintained history.
>
> We can easily set up a continuous import on Launchpad.net from a
> subversion repository into a bazaar branch. This will make it easy for

While I am partial to GIT already, the following document should be
read for how development can be done using a DSCM, not only git:

http://erlangish.blogspot.com/2007/10/erlware-and-git-development-model.html

These are the values I see:
* OTP would still receive patches over erlang-patches and have the
last word on what gets committed.
* Anyone gets commit access to _a_ repository so their parallel
branches can be kept up to date with the master OTP branch.
+ Experimental branches can live side by side until they reach OTP-quality.
* DSCM encourages change-sets that map 1:1 with an added
feature/bugfix, so the change history in itself is very readable
and educational as to how future similar changes should be made. (No
daily Clear-case-syncing commits that just bring the public repo up to
date please!)

PS.
GIT is not very small or going unmaintained away anytime soon:

http://git.or.cz/gitwiki/GitProjects

Michael Gebetsroither

unread,
Mar 19, 2008, 7:28:25 PM3/19/08
to erlang-q...@erlang.org
* Christian S <chs...@gmail.com> wrote:

> While I am partial to GIT already, the following document should be
> read for how development can be done using a DSCM, not only git:
>
> http://erlangish.blogspot.com/2007/10/erlware-and-git-development-model.html
>
> These are the values I see:
> * OTP would still receive patches over erlang-patches and have the
> last word on what gets committed.
> * Anyone gets commit access to _a_ repository so their parallel
> branches can be kept up to date with the master OTP branch.
> + Experimental branches can live side by side until they reach OTP-quality.
> * DSCM encourages change-sets that map 1:1 with an added
> feature/bugfix, so the change history in itself is very readable
> and educational as to how future similar changes should be made. (No
> daily Clear-case-syncing commits that just bring the public repo up to
> date please!)

Not to mention the incredible usefull git-bisect to search for bugs.
Especially with a small script that automatically tests if a version is
good or bad.
Eg. a bug is reported -> make a unittest and let bisect search where the
error went in.

This is nearly undoable in svn without a really fast lan connection and
even then it's quite timecomsuming.

cu,
michael
--
It's already too late!


Brian Granger

unread,
Mar 19, 2008, 10:43:17 PM3/19/08
to Jim Miller, Erlang Questions
> Open source does not translate into "better quality" software. One
> can simply look at freshmeat or any of the other open source projects
> out there to see that for every one or two quality open source
> projects there are literally thousands that are horrible. As stated
> by a few others on this discussion most open source projects are coded
> by people who "pray and spray" and who do not go through the rigors of
> true testing for a production environment. I know that not everyone
> falls into this category, nor am I categorizing people on this list as
> such.

Very true, but the same can be said about closed source software. The
reality is that many (most?) open _and_ closed source software has
poor software engineering and testing behind it. One nice thing about
projects that have open development models, is that you can quickly
get a sense of the quality of the software by looking at the test
suite, development practices.

For companies considering using Erlang, this matters. Especially when
it comes to bug tracking and testing. In these areas, transparency
leads to trust. Trust of this form leads to more users.

> Lets look at the cost of what it would take Ericsson to open up the
> repository as is being discussed.
>
> - Employee time spent moving things out of clearcase into another
> repository while maintaining source history. Yes, one can argue that
> simply taking a snapshot of the current state and starting a new
> repository might be acceptable but I doubt that would be acceptable to
> them. This takes a serious amount of time from an employee who would
> otherwise be working on things that directly provide revenue for a
> company (primary purpose of companies is to make money for the
> shareholders). This is a one to two week investment just to get the
> ball rolling.

This is a time consuming thing, but it is a one time cost.

> - Now, for every patch that is submitted through the process, an
> employee has to review the submission, verify whether it is acceptable
> or not, approve or reject it, write a note to the submitter if its
> been rejected (or check-in comment), and roll it into the baseline if
> it is. This is for EVERY submission regardless of whether or not it
> is part of the business goals of the company. Based on my experience
> with having been the make mom for a bunch of good size projects, this
> can take a huge amount of time. Very quickly it gets to the point
> where that person is doing nothing but dealing with patches.
> Interviews with Linus Torvalds indicate that the bulk of his time on
> the kernel isn't spent doing coding, its dealing with submissions.

I'm sure that Linus is completely annoyed to have to deal with all
those contributors to Linux ;-).

But seriously, most software projects (open and closed source) I know
of would _love_ to have more top notch contributors - especially ones
that work for free. I am not talking about hackers that submit buggy,
half baked code, but truly good ones. It is true that as a
development team grows, you have to spend more time coming up with a
good process - for instance one that distributes code review and
testing over a larger set of people.

But, one of the big benefits of opening up development is the
possibility of attracting some really good developers. In my mind,
this is a good thing, not a bad thing.

> So the overall cost to start something like this is basically
> allocating one whole person to it. In the companies I typically work
> with (Washington DC area software shops) this is roughly $200K

Again, I would view it as a truly great thing if people were
submitting so many patches to Erlang that such a person had to be
hired.

> What are the benefits:
>
> - Quicker inclusion of new capabilities.
> True, but are these capabilities that help Ericsson? Does a new
> wrapper around a vector graphics library (libcairo) help a company
> that writes the embedded software for network hardware? There are
> some fantastic libraries out there but what percentage of them are
> suitable for Ericsson's goals?

True, but isn't this a straw man argument. I can also think of many
improvements that Ericsson would potentially be interested in:

- An implementation of the erlang network protocol that runs over high
speed interconnects like infiniband (maybe this already exists?)

- SMP performance optimizations

- Better tools for integrating Erlang with other languages

Opening up the dev process could bring people out of the woodwork to
work on interesting things like this.

> - Better code quality
> Maybe. Most code I've seen written on the net does not do nearly
> the job required for testing that I require for production quality
> code my team releases. If you're Ericsson, and you're achieving 7 or
> 9-9s of system uptime, a single bug can introduce serious problems.
> Do you trust the most software developers out there to be rigorous
> enough to do this? Its tedious, hard, unsexy work and most people
> just don't do it.

I fully agree with this. But, I think this is why an open development
process is important. When people make an important choice - like
what language to use for a project, it is important for them to be
able to judge the quality of the language implementation and the
overall health of the language. With an open source language like
Python, it is very easy to get a sense of both of these things.
People like that.

Brian Granger

unread,
Mar 19, 2008, 10:56:28 PM3/19/08
to Kenneth Lundin, erlang-q...@erlang.org, Torbjorn Tornkvist
> It is worth noting that so far 99% of the patches contributed to
> erlang-bugs or erlang-patches need to be corrected or rewritten by us
> before they make it into a release. And this has nothing with lack of
> VCS to do. The reasons are more of the type:
> - solution is not done the way we like to have it
> - does not work on all platforms
> - buggy
> - unacceptable dependencies
> - unacceptable incompatibility

True, using SVN, git or mercurial won't help this. But, having a well
defined and open development model would help this. In fact, I would
argue that the reason you get such poor quality patches is that there
is not formal, open and well documented way of contributing to Erlang.
If Erlang moved to an open dev model, the Erlang team could simply
document the requirements for all contributed code - things like
coding conventions, documentation, testing, dependencies. This would
encourage people to submit better quality code.

Massimo Cesaro

unread,
Mar 20, 2008, 1:28:25 PM3/20/08
to erlang-q...@erlang.org

I'm afraid that's it's not so straightforward as it sound. I think that the Erlang people has an 850 tons gorilla code base to take care of, eventually running on different hardware and software architectures. It's difficult to imagine that anybody has the will/stamina/resources to do the same in a reasonable time frame without a strong commitment (read: customer driven).
The only case in which opening everything to everybody would make sense is if/when Ericsson give up in supporting Erlang, and given the premises that's unlikely to happen real soon.
I guess that non-Ericsson people from many companies in this list, which leveraged on the tremendous productivity gain derived from Erlang and OTP to create their products, share this common sense approach and appreciate having a company like Ericsson to back up their core technology.
Let's face it: Erlang as it is today *is* open source, with one of the better licenses in the world of open source (i.e. no strings attached, no bigotry, no ideological twist). We do have the EEP to propose enhancements. And we have somebody helping us in avoiding to shoot ourselves in the foot.
Erlang is a (pretty wide) domain specific language. It is not a swiss army knife tool. It's not designed to be a teaching tool. It's not a matter of conquering people's minds or souls. It's about making stuff work even when the other guys stuff breaks.

Massimo

Brian Granger

unread,
Mar 20, 2008, 5:07:44 PM3/20/08
to Massimo Cesaro, erlang-q...@erlang.org
> > > It is worth noting that so far 99% of the patches contributed to
> > > erlang-bugs or erlang-patches need to be corrected or rewritten by us
> > > before they make it into a release. And this has nothing with lack of
> > > VCS to do. The reasons are more of the type:
> > > - solution is not done the way we like to have it
> > > - does not work on all platforms
> > > - buggy
> > > - unacceptable dependencies
> > > - unacceptable incompatibility
> >
> > True, using SVN, git or mercurial won't help this. But, having a well
> > defined and open development model would help this. In fact, I would
> > argue that the reason you get such poor quality patches is that there
> > is not formal, open and well documented way of contributing to Erlang.
> > If Erlang moved to an open dev model, the Erlang team could simply
> > document the requirements for all contributed code - things like
> > coding conventions, documentation, testing, dependencies. This would
> > encourage people to submit better quality code.
>
> I think that the
> Erlang people has an 850 tons gorilla code base to take care of, eventually
> running on different hardware and software architectures. It's difficult to
> imagine that anybody has the will/stamina/resources to do the same in a
> reasonable time frame without a strong commitment (read: customer driven).
> The only case in which opening everything to everybody would make sense is
> if/when Ericsson give up in supporting Erlang, and given the premises that's
> unlikely to happen real soon.

I completely agree with this. I don't think anyone (I hope) is
suggesting that Erlang development be turned into a democratic free
for all. You absolutely need extremely good, devoted people (like the
Erlang team) steering the project and making firm decisions about what
gets in and what does not. But, that doesn't mean the development
process can not be made more open and transparent and still maintain
the full integrity of the project and Ericsson's commercial interests.

> I'm afraid that's it's not so straightforward as it sound.

I do think it is straightforward - if you don't tell people what you
expect (in terms of the quality of code contributions and patches) you
can't expect much from them. Simply laying out the expectations would
go a long way toward improving the quality of code contributions that
the Erlang teams gets from the outside.

What else would people propose to encourage higher quality public
contributions to Erlang?

Cheers,

Brian

Christian S

unread,
Mar 20, 2008, 8:45:24 PM3/20/08
to Brian Granger, erlang-q...@erlang.org
> > I think that the
> > Erlang people has an 850 tons gorilla code base to take care of, eventually
> > running on different hardware and software architectures. It's difficult to
> > imagine that anybody has the will/stamina/resources to do the same in a
> > reasonable time frame without a strong commitment (read: customer driven).
> > The only case in which opening everything to everybody would make sense is
> > if/when Ericsson give up in supporting Erlang, and given the premises that's
> > unlikely to happen real soon.
>
> I completely agree with this. I don't think anyone (I hope) is
> suggesting that Erlang development be turned into a democratic free
> for all. You absolutely need extremely good, devoted people (like the
> Erlang team) steering the project and making firm decisions about what
> gets in and what does not. But, that doesn't mean the development
> process can not be made more open and transparent and still maintain
> the full integrity of the project and Ericsson's commercial interests.

These conclusions here sound quite alien to me.

Open source is a meritocracy. It is free for all to participate and
prove their merit.

It is the product that matters.

When R12-B0 didn't prove good enough, then many deployments stayed
with/at/on[1] their R11 releases waiting for a better release.


In the list, I often see promises that detected issues/bugs will be
fixed in the next OTP release.

Often there is an early fix attached as a patch to the mail. Not a bad
thing. Not at all. Very good indeed.

But the patch is nothing else than an informal branch of the OTP
source tree. With a suitable DSCM it can be handled within source
control management.

The branch can be given a name. Additional work can be done on it,
version controlled. It can be merged with some third-party branch
that makes Erlang work on a new processor architecture equipped with
flux-capacitors. Or macs. Or just local patches / other public
pre-release fixes.

As to submitted patches, overworked OTP members can comment with
suggestions to what is required for them to swallow a patch whole for
OTP inclusion if they dont have time to work it right now.
Version controlled work can progress outside of the OTP team, but the
work can be committed back. Using the same tools the whole time.

> > I'm afraid that's it's not so straightforward as it sound.
>
> I do think it is straightforward - if you don't tell people what you
> expect (in terms of the quality of code contributions and patches) you
> can't expect much from them. Simply laying out the expectations would
> go a long way toward improving the quality of code contributions that
> the Erlang teams gets from the outside.

To learn from other projects:

What documentation does linux have about expected code quality besides
the hint to read and burn the gnu code conventions?

Isn't it about publishing your patches to the mailinglist and rework
them until you stop getting witty remarks of how your brain is more
like a "." than an "O" ? (Or stop being handed lollipops :)

Would it be good if good documentation existed so someone new to the
otp source tree and conventions could write a great patch. Yes indeed.
Has this ever happened for any sufficiently interesting project?

My two öre: The best documentation for what makes a patch good is a
history of previous good patches and their evolution from unacceptable
to acceptable. At least it works to do it this way.

[1]
I'm drunk and english prepositions have very vague rules.
This is much more important to fix than any issues people find that
erlang have with punctuation!

Masklinn

unread,
Mar 21, 2008, 8:14:50 AM3/21/08
to Christian S, Erlang Questions

> As to submitted patches, overworked OTP members can comment with
> suggestions to what is required for them to swallow a patch whole for
> OTP inclusion if they dont have time to work it right now.
> Version controlled work can progress outside of the OTP team, but the
> work can be committed back. Using the same tools the whole time.

I completely agree with this: the core Erlang/OTP team should *not*
have to rewrite "99%" of the patches (Kenneth's number). They should
not, in fact, have to rewrite any patch (unless the rewrite or
corrections are trivial, or the patch submitter doesn't have the
necessary resources e.g. to test the patch on all erlang-supported
archs): it wastes their time that could be better spent, and it
prevents contributors from learning what is good Erlang/OTP style/patch.

In the problems Kenneth listed, 4 ought to be handled by the patch
submitter through feedback by the core team until acceptance of the
patch:

* "Solution is not done the way we like to have it", a better/more
acceptable approach should be suggested, and the submitter (or someone
else) could then go on and rewrite the patch according to this. Would
also teach contributors and would-be contributors the preferred coding
style/approaches.
* "buggy", if the patch doesn't work then Erlang/OTP core has no
reason to accept it
* "unacceptable dependencies", should be listed and the submitter
could go back to the drawing board, remove them and resubmit the
patch, which would also send hints to contributors and would-be
contributors about acceptable and unacceptable dependencies
* "unacceptable incompatibility", should likewise be listed/stated and
the submitter could then go work on debugging his patch.

That's usually the way it's done on e.g. the Mercurial list, and DVCS
have a very good support for that kind of workflows.

Matthias Lang

unread,
Mar 21, 2008, 10:03:19 AM3/21/08
to Masklinn, Erlang Questions
Masklinn writes:

> I completely agree with this: the core Erlang/OTP team should *not*
> have to rewrite "99%" of the patches (Kenneth's number). They should
> not, in fact, have to rewrite any patch (unless the rewrite or
> corrections are trivial, or the patch submitter doesn't have the
> necessary resources e.g. to test the patch on all erlang-supported
> archs):

Take a look at a patch that someone actually submitted, for example:

http://www.erlang.org/pipermail/erlang-questions/2008-January/032345.html

it's a pretty typical user-contributed patch, i.e. it's a small
change, only about five lines of code, it fixes a definite problem,
and, most importantly, it's mostly crap. The guy who submitted the
patch

1. couldn't be bothered testing it properly

and 2. messed with some internals which he most likely doesn't
understand.

and 3. made some arbitrary and undocumented design decisions, e.g.
about how wide the hash should be

and 4. couldn't be bothered coming up with a solution which works
with HIPE and with old versions

i.e. he did the first 5% of the job and then wasn't prepared to put in
the hard work to actually finish. Feedback from the OTP group (your
suggestion) isn't going to fix that.

Matt

(the patch is mine, i.e. I am too lazy to do the job properly. Sorry.)

Masklinn

unread,
Mar 21, 2008, 11:28:32 AM3/21/08
to Matthias Lang, Erlang Questions

On 21 Mar 2008, at 15:03 , Matthias Lang wrote:
> Take a look at a patch that someone actually submitted, for example:
>
> http://www.erlang.org/pipermail/erlang-questions/2008-January/032345.html
>
> it's a pretty typical user-contributed patch, i.e. it's a small
> change, only about five lines of code, it fixes a definite problem,
> and, most importantly, it's mostly crap. The guy who submitted the
> patch
>
> 1. couldn't be bothered testing it properly
>
> and 2. messed with some internals which he most likely doesn't
> understand.
>
> and 3. made some arbitrary and undocumented design decisions, e.g.
> about how wide the hash should be
>
> and 4. couldn't be bothered coming up with a solution which works
> with HIPE and with old versions
>
Then unless starting from that patch is vital (e.g. it's a good start
on fixing something really broken) are entitled to send feedback on
it, and plain refuse it if the submitter doesn't want to fix the
patch. And more importantly (as they're already entitled to do so) the
tools should help them doing it.

> i.e. he did the first 5% of the job and then wasn't prepared to put in
> the hard work to actually finish. Feedback from the OTP group (your
> suggestion) isn't going to fix that.
>

Maybe it wouldn't, maybe it would, maybe it wouldn't in that case but
would in others.

> (the patch is mine, i.e. I am too lazy to do the job properly. Sorry.)

I can't judge that you're too lazy to do the job properly as I'm too
lazy to do it at all.

PS: I fully know that tools and processes can't fix everything, or
more precisely can't fix anything, but they can help change things.

Christian S

unread,
Mar 21, 2008, 11:36:40 AM3/21/08
to Matthias Lang, Erlang Questions
On Fri, Mar 21, 2008 at 3:03 PM, Matthias Lang <matt...@corelatus.se> wrote:
> Take a look at a patch that someone actually submitted, for example:
>
> http://www.erlang.org/pipermail/erlang-questions/2008-January/032345.html
>
> it's a pretty typical user-contributed patch, i.e. it's a small
> change, only about five lines of code, it fixes a definite problem,
> and, most importantly, it's mostly crap. The guy who submitted the
> patch

What I see here is that an automated test-case that exposes this
problem should be written.

And that's the kind of feedback you could have received.

Something that leaves the test failures as a thorn until the cause is
fixed (or the test removed :). If OTP has survived with this hole so
far, it can probably survive many more months.


Now as for your needs:
If your code depend on the behavior that your patch introduces, then
you probably want to make sure it is included each time you compile
OTP?

Wouldn't you want to forward port it as new OTP makes new releases you
intend to use? Back port it to that old system still running R9?

Wouldn't you want to publish this patch so others with the same issue
can look at your attempt, or perhaps inviting HiPE to implement
something equivalent?


Basically I'm arguing for the use of a DSCM. I don't get the feeling
that you argue against that. Are we actually disagreeing about
something? :)


Perhaps the most important issue I see here isn't if GIT or Mercurial
is chosen over Subversion, as both can already import from subversion.

The issue is if the public repository will have a commit log like:

OTP ClearCase changes for 2008-03-20 11:00
OTP ClearCase changes for 2008-03-20 14:00
OTP ClearCase changes for 2008-03-14 16:00
OTP ClearCase changes for 2008-03-13 09:00
OTP ClearCase changes for 2008-03-11 14:00
OTP ClearCase changes for 2008-03-11 11:00


Or if it will be a more lively one-commit-is-one-fixed-feature and
multiple living branches for pre-release fixes. Something more like

http://lwn.net/Articles/140350/

Or a less busy log:

http://www.flickr.com/photos/malcolmtredinnick/1516857444/

Reply all
Reply to author
Forward
0 new messages