[erlang-questions] Erlang http servers

428 views
Skip to first unread message

Serge Aleynikov

unread,
Sep 30, 2012, 5:29:26 PM9/30/12
to erlang-q...@erlang.org
Haven't had a need for an HTTP server in Erlang for a while since I last
used inets-based http server five years back, and now examining what
exists in public domain.

The results are quite surprising - everyone who seems comfortable coding
in Erlang, feels compelled to implement an HTTP server. Previously we
had only one contender to inets - yaws. Now the landscape is quite a
mouthful - inets, yaws, cowboy, musultin, mochiweb, etc. As Steve
Vinoski rightfully points out (*) this doesn't help to pull the blanket
in all directions. From personal experience with inets, I recall that
once I figured out how to use it, it happened to be a very helpful and
powerful HTTP library for building an application server. So why don't
all the authors of those wonderful alternative libraries just patch the
inets to extend it with something they feel missing so that we only have
one good and documented library to use as the common denominator?

Serge


(*) http://steve.vinoski.net/blog/2011/05/09/erlang-web-server-benchmarking
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

Loïc Hoguin

unread,
Sep 30, 2012, 6:22:15 PM9/30/12
to Serge Aleynikov, erlang-q...@erlang.org
Biased Cowboy author here. If I got the wrong idea about another project
please correct me. :)

On 09/30/2012 11:29 PM, Serge Aleynikov wrote:
> So why don't
> all the authors of those wonderful alternative libraries just patch the
> inets to extend it with something they feel missing so that we only have
> one good and documented library to use as the common denominator?

Because it's not about what's missing, but about how you use it, how it
works and what you can do with it.

You could probably classify the servers into two groups, inets+yaws
being monolithic servers with many features (that can be
ignored/stripped down, at least for yaws) and with a more traditional
configuration file based approach (unless you run yaws embedded), and
mochiweb+misultin+cowboy being in the lightweight family, being pretty
much bare bone HTTP (except mochiweb has a few extra unrelated modules
like JSON).

Note that mochiweb is in maintenance mode and misultin has been dropped
in favor of cowboy.

Cowboy itself has a focus on reduced memory use (uses binary), reduced
latency, has long-polling and Websocket support (SPDY soon), while
trying to stay close to OTP principles (long-polling/websocket
connections will be special processes in the next version allowing you
to use sys: and friends to debug your code, the handler modules are
inspired by gen_server, everything is supervised, and so on). It also
has Webmachine-inspired REST support.

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

Toby Thain

unread,
Sep 30, 2012, 8:57:37 PM9/30/12
to Serge Aleynikov, erlang-q...@erlang.org
On 30/09/12 5:29 PM, Serge Aleynikov wrote:
> Haven't had a need for an HTTP server in Erlang for a while since I last
> used inets-based http server five years back, and now examining what
> exists in public domain.
>
> The results are quite surprising - everyone who seems comfortable coding
> in Erlang, feels compelled to implement an HTTP server. Previously we
> had only one contender to inets - yaws.


And pico!
http://jungerl.cvs.sourceforge.net/viewvc/jungerl/jungerl/lib/pico/

--Toby

> Now the landscape is quite a
> mouthful - ...

Michael Truog

unread,
Sep 30, 2012, 11:12:53 PM9/30/12
to Serge Aleynikov, erlang-q...@erlang.org
On 09/30/2012 05:57 PM, Toby Thain wrote:
> On 30/09/12 5:29 PM, Serge Aleynikov wrote:
>> Haven't had a need for an HTTP server in Erlang for a while since I last
>> used inets-based http server five years back, and now examining what
>> exists in public domain.
>>
>> The results are quite surprising - everyone who seems comfortable coding
>> in Erlang, feels compelled to implement an HTTP server. Previously we
>> had only one contender to inets - yaws.
>
>
> And pico!
> http://jungerl.cvs.sourceforge.net/viewvc/jungerl/jungerl/lib/pico/
>
>

And elli!
https://github.com/knutin/elli

Joe Armstrong

unread,
Oct 1, 2012, 4:01:31 AM10/1/12
to Serge Aleynikov, erlang-q...@erlang.org
On Sun, Sep 30, 2012 at 11:29 PM, Serge Aleynikov <se...@aleynikov.org> wrote:
> Haven't had a need for an HTTP server in Erlang for a while since I last
> used inets-based http server five years back, and now examining what
> exists in public domain.
>
> The results are quite surprising - everyone who seems comfortable coding
> in Erlang, feels compelled to implement an HTTP server. Previously we
> had only one contender to inets - yaws. Now the landscape is quite a
> mouthful - inets, yaws, cowboy, musultin, mochiweb, etc. As Steve
> Vinoski rightfully points out (*) this doesn't help to pull the blanket
> in all directions. From personal experience with inets, I recall that
> once I figured out how to use it, it happened to be a very helpful and
> powerful HTTP library for building an application server. So why don't
> all the authors of those wonderful alternative libraries just patch the
> inets to extend it with something they feel missing so that we only have
> one good and documented library to use as the common denominator?

I can think of several reasons why this does not happen:

a) You have to distinguish between functional and non-functional behavior.
All web-servers implement the same protocol (HTTP) - so they are
functionally
equivalent - but their non-functional behavior is very different
- one server
might be optimized for large numbers of short-lived sessions,
another for a few long-lived
sessions. One might cache content, another not. So even though they are
all http servers they have very different behaviour - this is good.

b) Servers have two interfaces - the protocol interface (HTTP) and
the interface towards
Erlang (the API if you like). As far as I can see most severs
have different Erlang APIs.
This is bad.

c) Inets is shipped with OTP ie built and maintained by the OTP group.
This is not really
"core business" as far as the OTP group is concerned, better for
everybody if non-core
applications get taken over by third-part vendors.

d) As things mature - people want more in the way of "service" - ie
good documentation and the ability
to buy support - the problem with software is not so much that of
writing it - which can be
easy or difficult depending upon the problem but with the "burden of support"

Personally I'm quite happy with there being several different http
servers, It gives me several to choose
from. No server in the world will do exactly what I want so they all
need tweaking in one way or another.

What I don't like is that all have different APIs - this make swapping
between severs a pain.
Some degree of standardization of the APIs necessary to setup and
manage the servers would be
highly desirable.

It would also be highly desirable to standardize the relationship
between a http uri and
and erlang function call. For example


http://some.host/erlcall?mod=foo&func=bar&arg1=111

means call foo:bar(...) on the server and return the value as html (or
something)

Currently the http-server side of things seems ok - what I'm missing is the
entire user management cycle - password management, authentication, cookies,
forgotten passwords etc. - this needs far more than an http server, it needs
a persistent database, security and so on ...

Cheers

/Joe

Fred Hebert

unread,
Oct 1, 2012, 8:57:25 AM10/1/12
to Joe Armstrong, erlang-q...@erlang.org
I did not have much to say about the rest (except that different interfaces may emerge based on the use cases that the server optimizes for), and I mostly nodded in agreement to most of your points, but the last one I believe is a very bad idea.

This is a problem in terms of:
  1. Maintenance. You don't want to usually give such an obvious window into your server as it somewhat crystallizes your internal application as a public API, what the URI is usually used for. We should strive to have URIs never change (see W3C - Cool URIs don't change ). Unless you're ready not to change the function calls under the hood, I would not use this.
  2. Security. How do you keep someone from calling http://some.host/erlcall?mod=init&func=stop or http://some.host/erlcall?mod=os&func=cmd&arg1="rm -rf /" ? If you go look at tryerlang.org's repository, you'll see how restrictive Roberto Aloi had to make it to avoid people doing tricky things like passing binaries of funs calling things they shouldn't indirectly just to crash the system. This is absolutely non-trivial as soon as you move out from using a very restrictive white list of valid functions -- at which point you may as well hide them behind URIs.
  3. Clarity. What are the argument types? Should we treat the '111' as a string? a binary? an integer? An IO List? Should the server force all arguments to be of a given type that needs to be converted later so you need to have some kind of intermediary function doing conversion for you? It's not different from what we get right now, but there's a difference in expectations from the developer. A minor issue, I guess.
  4. RPC and HTTP. Whether RPC is still a good way to do things is up to debate (look for Steve Vinoski's essays and posts on the topic). Shouldn't the erlcall URI be extended to also handle timeouts? Or are these server options unrelated to the function call? More than that is the idea of how you write things that may depend on content-type (such as RESTful web services). For example, a browser that sends a request to a page in HTML may be served HTML. If Javascript asks for info in JSON, then the same URI may forward JSON. There is an out-of-URL mechanism to deal with different requests and ideas.
  5. HTTP Spec problems. GET and HEAD requests should be idempotent. PUT, POST, and DELETE can be used in ways that change information on the server. If I use POST to http://some.host/erlcall?mod=foo&func=bar&arg1=111 and it updates data, what should it do when I use GET or HEAD on it? Do we prefer to use the POST body for the requests that are done under the POST method? What if I include both POST arguments and querystring arguments and they are different?
    More than that, HTTP doesn't specify what happens when you have more than one querystring argument being the same (from memory). It should thus be possible to have arg1 four times in the URL, with arg3 being there only once, and arg2 being entirely missing. Same with the module or function name. How do we make this interface behave at that point? Things are way more complex than what they look like.

There are likely more issues with this approach. Ideally, it should be impossible for the user of your server to know what you used to implement it -- this is what should give you the most freedom in terms of implementation when lots of people start using it. At this point I'm thinking we should start using different protocols from HTTP if we're not really willing to respect it, but that's for another discussion.

And yes, more user management would definitely be nice. I've written a tiny library to handle passwords themselves, but it's not close enough to be something a framework would use without surrounding support -- it just handles the core of password hashing and verification. It's currently hosted at https://github.com/ferd/erlpass . Then each platform such as Zotonic likely reimplemented their own, although I don't know how moveable they are outside of the project.

Regards,
Fred.

Dale Harvey

unread,
Oct 1, 2012, 9:32:41 AM10/1/12
to Fred Hebert, erlang-q...@erlang.org
"The standard library is where modules go to die"
http://www.leancrew.com/all-this/2012/04/where-modules-go-to-die/
This is from python world, and is very obviously seen in Erlang.

Alternatively https://npmjs.org/ has 15k+ packages, we dont need to add user management and the kitchen sink to inets, we need to remove it from the standard library and promote userland packages as first class citizens (ie pick a tool to install packages and bundle it)

Kenneth Lundin

unread,
Oct 1, 2012, 10:30:25 AM10/1/12
to Joe Armstrong, erlang-q...@erlang.org
I don't agree with everything Joe writes below:

On 10/1/12, Joe Armstrong <erl...@gmail.com> wrote:
> On Sun, Sep 30, 2012 at 11:29 PM, Serge Aleynikov <se...@aleynikov.org>
> wrote:
>> Haven't had a need for an HTTP server in Erlang for a while since I last
>> used inets-based http server five years back, and now examining what
>> exists in public domain.
>>
>> The results are quite surprising - everyone who seems comfortable coding
>> in Erlang, feels compelled to implement an HTTP server. Previously we
>> had only one contender to inets - yaws. Now the landscape is quite a
>> mouthful - inets, yaws, cowboy, musultin, mochiweb, etc. As Steve
>> Vinoski rightfully points out (*) this doesn't help to pull the blanket
>> in all directions. From personal experience with inets, I recall that
>> once I figured out how to use it, it happened to be a very helpful and
>> powerful HTTP library for building an application server. So why don't
>> all the authors of those wonderful alternative libraries just patch the
>> inets to extend it with something they feel missing so that we only have
>> one good and documented library to use as the common denominator?
>
I agree with Serge that it is somewhat unfortunate that there are so
many initiatives
where people write their own implementation of something we already
have included in
the Erlang/OTP distribution and doing this without even suggesting or
asking us if we could
incorporate some missing feature or improve the current implementation
in various ways.

I think it is better to try to improve the existing functionality
first and only if that is not feasible make another implementation
with a new API.

In Erlang/OTP we have keep strict compatibility with many of our APIs
which is both a strength and sometimes a burden (when we are not happy
with the old API) but there are mostly ways around that if there is
strong case for a change.

> I can think of several reasons why this does not happen:
>
> a) You have to distinguish between functional and non-functional behavior.
> All web-servers implement the same protocol (HTTP) - so they are
> functionally
> equivalent - but their non-functional behavior is very different
> - one server
> might be optimized for large numbers of short-lived sessions,
> another for a few long-lived
> sessions. One might cache content, another not. So even though they
> are
> all http servers they have very different behaviour - this is good.
>
> b) Servers have two interfaces - the protocol interface (HTTP) and
> the interface towards
> Erlang (the API if you like). As far as I can see most severs
> have different Erlang APIs.
> This is bad.
Would be good if there was a kind of standardized API for HTTP-servers
and clients.
>
> c) Inets is shipped with OTP ie built and maintained by the OTP group.
> This is not really
> "core business" as far as the OTP group is concerned, better for
> everybody if non-core
> applications get taken over by third-part vendors.
Joe is right reagarding core business in that there are not much use
of HTTP-servers in
products developed with Erlang inside Ericsson (who is financing ERlang/OTP) and
if used the requirements are not very demanding.
>
> d) As things mature - people want more in the way of "service" - ie
> good documentation and the ability
> to buy support - the problem with software is not so much that of
> writing it - which can be
> easy or difficult depending upon the problem but with the "burden of
> support"
We still actively develop inets with its http client and server and I
am not sure that
all the reasons to develop alternatives still exist.

We are more than happy to receive contributions with bugfixes and new
functionality.

I think it is of great value to have a relative complete functionality
in Erlang/OTP so that
the user can implement basic functionality without having to search
for separate implementations of, with unknown quality and completeness
at least for a beginner.
>
> Personally I'm quite happy with there being several different http
> servers, It gives me several to choose
> from. No server in the world will do exactly what I want so they all
> need tweaking in one way or another.
>
> What I don't like is that all have different APIs - this make swapping
> between severs a pain.
> Some degree of standardization of the APIs necessary to setup and
> manage the servers would be
> highly desirable.
>
> It would also be highly desirable to standardize the relationship
> between a http uri and
> and erlang function call. For example
>
>
> http://some.host/erlcall?mod=foo&func=bar&arg1=111
>
> means call foo:bar(...) on the server and return the value as html (or
> something)
>
> Currently the http-server side of things seems ok - what I'm missing is
> the
> entire user management cycle - password management, authentication,
> cookies,
> forgotten passwords etc. - this needs far more than an http server, it
> needs
> a persistent database, security and so on ...
>
> Cheers
>
> /Joe
>
>
/Kenneth, Erlang/OTP Ericsson

Loïc Hoguin

unread,
Oct 1, 2012, 11:38:52 AM10/1/12
to Kenneth Lundin, erlang-q...@erlang.org
On 10/01/2012 04:30 PM, Kenneth Lundin wrote:
> I agree with Serge that it is somewhat unfortunate that there are so
> many initiatives
> where people write their own implementation of something we already
> have included in
> the Erlang/OTP distribution and doing this without even suggesting or
> asking us if we could
> incorporate some missing feature or improve the current implementation
> in various ways.

It happens a lot. People tend to review available options, and if none
fits their needs, write something themselves without consulting any of
the authors. Sometimes they're right to do so, because the answer
they'll get is obvious, and sometimes not.

There's a few reasons this happens for this specific case:

* HTTP is simple, it's easier to go ahead and experiment on a new
project than on an existing one.
* If the experiment succeeds and the implementation has to differ
significantly from other projects, it's not worth backporting since
you'll just run into a wall of backward compatibility requirements.
* OTP has a long release process. Many companies need their fixes to be
included quickly to avoid maintaining huge branches for their changes.
They generally agree in return to keep track of changes that might
happen during the product's development.
* Inets is seriously well hidden. I only heard about it after releasing
Cowboy. I knew about Yaws/Mochiweb/Misultin but not the inets HTTP
server. Many people I talked to didn't seem to know it existed.
* Sometimes removing features is an improvement. Inets is big, it's only
natural that lightweight alternatives appear. Same happened to Apache,
and it's no surprise that Inets is quite similar to Apache from a user
point of view.

Note that while not everyone uses inets, most servers do use
erlang:decode_packet/3 which does most of the job. Cowboy doesn't
anymore, because it needed backward incompatible changes to
erlang:decode_packet/3. I didn't ask your thoughts, simply because the
experiment I did hinted that doing the parsing in full Erlang was a
better idea: this change results in improved HiPE optimizations and
better reduction management which results in better latency under load.

> In Erlang/OTP we have keep strict compatibility with many of our APIs
> which is both a strength and sometimes a burden (when we are not happy
> with the old API) but there are mostly ways around that if there is
> strong case for a change.

It's a big strength for people looking for stability and a big burden
for people looking for reactivity. Can't really blame the second group
for finding solutions to their problems.

> We still actively develop inets with its http client and server and I
> am not sure that
> all the reasons to develop alternatives still exist.

But like you said the level of support for applications included in OTP
vary widely. It is also not clearly indicated which parts are the focus
of recent works (there is a small roadmap for the most important stuff
but that's about it), nor can it be deduced from reading tickets in a
bug tracker or similar collaboration tool.

Which brings me to a very important point: please start using a public
bug tracker! It doesn't have to be 100% public (as I'm sure you have
some issues that are internal only) but it would help a lot if people
could see the existing issues instead of stumbling upon the same bugs
again and again. It probably also would help with contributions.

> I think it is of great value to have a relative complete functionality
> in Erlang/OTP so that
> the user can implement basic functionality without having to search
> for separate implementations of, with unknown quality and completeness
> at least for a beginner.

I would prefer Erlang/OTP to focus on the core features and leave the
satellite features to third parties. OTP has an HTTP server but doesn't
have proper Unicode support. OTP has OpenGL but doesn't have proper
timezones support. These 2 examples missing from OTP are some of many
things that are too complex for most people to implement, and that would
not benefit from having alternatives.

Let's take another example. While taking a quick glance at the modules
included in OTP I discovered there is a GD implementation in the percept
application. I'm pretty sure most people didn't know it was there, as
they usually end up using a NIF. It's quite unexpected that it's in OTP.

Finding things in OTP unexpectedly is a quite common occurence, though.

If the user had a package management application (perhaps working
similarly to rebar's deps, = per project/release, no site-wide install
by default) along with a package index website, it could quite easily
find what he needs. It wouldn't be of unknown quality or completeness,
simply because adding a comment and rating system to a package index
website is fairly simple to do.

Plus, he can make an opinion for himself, instead of just using whatever
someone decided for him. You might include a project in OTP that fits
your criteria for quality, but it might be crap for someone else. A more
democratic system where people choose what they use based on other
users' feedback and ratings is not only a lot more powerful but also
allows to bring some light on some more obscure projects that are
sometimes quite hard to find armed only with Google.

It makes sense to have a single choice for big corporate environments,
not so much for the rest of the world.

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

Kenneth Lundin

unread,
Oct 1, 2012, 2:11:09 PM10/1/12
to Loïc Hoguin, erlang-q...@erlang.org


Den 1 okt 2012 17:39 skrev "Loïc Hoguin" <es...@ninenines.eu>:
>
> On 10/01/2012 04:30 PM, Kenneth Lundin wrote:
>>
>> I agree with Serge that it is somewhat unfortunate that there are so
>> many initiatives
>> where people write their own implementation of something we already
>> have included in
>> the Erlang/OTP distribution and doing this without even suggesting or
>> asking us if we could
>> incorporate some missing feature or improve the current implementation
>> in various ways.
>
>
> It happens a lot. People tend to review available options, and if none fits their needs, write something themselves without consulting any of the authors. Sometimes they're right to do so, because the answer they'll get is obvious, and sometimes not.
>
> There's a few reasons this happens for this specific case:
>
> * HTTP is simple, it's easier to go ahead and experiment on a new project than on an existing one.
> * If the experiment succeeds and the implementation has to differ significantly from other projects, it's not worth backporting since you'll just run into a wall of backward compatibility requirements.
> * OTP has a long release process. Many companies need their fixes to be included quickly to avoid maintaining huge branches for their changes. They generally agree in return to keep track of changes that might happen during the product's development.

I think we are pretty fast with service releases about every 3rd month and updates on the maint branch in git between that.

> * Inets is seriously well hidden. I only heard about it after releasing Cowboy. I knew about Yaws/Mochiweb/Misultin but not the inets HTTP server. Many people I talked to didn't seem to know it existed.

I agree on that. I have always thought that the name inets is misleading.

> * Sometimes removing features is an improvement. Inets is big, it's only natural that lightweight alternatives appear. Same happened to Apache, and it's no surprise that Inets is quite similar to Apache from a user point of view.

I agree here too. The http server included in OTP should have been a lightweight one leaving for others to implement a more complete or specialized one.


>
> Note that while not everyone uses inets, most servers do use erlang:decode_packet/3 which does most of the job. Cowboy doesn't anymore, because it needed backward incompatible changes to erlang:decode_packet/3. I didn't ask your thoughts, simply because the experiment I did hinted that doing the parsing in full Erlang was a better idea: this change results in improved HiPE optimizations and better reduction management which results in better latency under load.

Interesting, would be good if you could give us more info on this. The http parsing in
decode_packet is there to improve performance. If it is not ( without using HiPE) we should consider removing it!


>
>
>> In Erlang/OTP we have keep strict compatibility with many of our APIs
>> which is both a strength and sometimes a burden (when we are not happy
>> with the old API) but there are mostly ways around that if there is
>> strong case for a change.
>
>
> It's a big strength for people looking for stability and a big burden for people looking for reactivity. Can't really blame the second group for finding solutions to their problems.

I am not blaming people for implementing their own solution, it is good to have a lot to choose from. But I still think the stability is very important for the majority of users making systems with a longer life cycle than a year.
Our experience is that we have to be very compatible in order to convince our
users (often big projects) to step up to our latest release.
By this we can also stop supporting older versions.


>
>
>> We still actively develop inets with its http client and server and I
>> am not sure that
>> all the reasons to develop alternatives still exist.
>
>
> But like you said the level of support for applications included in OTP vary widely. It is also not clearly indicated which parts are the focus of recent works (there is a small roadmap for the most important stuff but that's about it), nor can it be deduced from reading tickets in a bug tracker or similar collaboration tool.

Yes we plan to improve in this area.

I would say that the level of support is equally high for all applications we regard as important and heavily used by Ericsson and or the external Erlang community.


>
> Which brings me to a very important point: please start using a public bug tracker! It doesn't have to be 100% public (as I'm sure you have some issues that are internal only) but it would help a lot if people could see the existing issues instead of stumbling upon the same bugs again and again. It probably also would help with contributions.

We have discussed this as well.

>
>
>> I think it is of great value to have a relative complete functionality
>> in Erlang/OTP so that
>> the user can implement basic functionality without having to search
>> for separate implementations of, with unknown quality and completeness
>> at least for a beginner.
>
>
> I would prefer Erlang/OTP to focus on the core features and leave the satellite features to third parties. OTP has an HTTP server but doesn't have proper Unicode support. OTP has OpenGL but doesn't have proper timezones support. These 2 examples missing from OTP are some of many things that are too complex for most people to implement, and that would not benefit from having alternatives.

What is missing when it comes to timezones? Has it been mentioned in another thread?
I think we rely on the OS configuration here.


>
> Let's take another example. While taking a quick glance at the modules included in OTP I discovered there is a GD implementation in the percept application. I'm pretty sure most people didn't know it was there, as they usually end up using a NIF. It's quite unexpected that it's in OTP.

It was developed as part of Percept just to have something to use there. We don't regard it complete or general enough to be supported as an app of its own.


>
> Finding things in OTP unexpectedly is a quite common occurence, though.

I think most things are documented but the docs could have better search facilities.


>
> If the user had a package management application (perhaps working similarly to rebar's deps, = per project/release, no site-wide install by default) along with a package index website, it could quite easily find what he needs. It wouldn't be of unknown quality or completeness, simply because adding a comment and rating system to a package index website is fairly simple to do.

I agree that a package management system would be good. There are initiatives in that direction.


>
> Plus, he can make an opinion for himself, instead of just using whatever someone decided for him. You might include a project in OTP that fits your criteria for quality, but it might be crap for someone else. A more democratic system where people choose what they use based on other users' feedback and ratings is not only a lot more powerful but also allows to bring some light on some more obscure projects that are sometimes quite hard to find armed only with Google.
>
> It makes sense to have a single choice for big corporate environments, not so much for the rest of the world.
>

Striving in that direction.
> --
> Loďc Hoguin


>
> Erlang Cowboy
> Nine Nines
> http://ninenines.eu

/Kenneth, Erlang/OTP Ericsson

Loïc Hoguin

unread,
Oct 1, 2012, 3:52:53 PM10/1/12
to Kenneth Lundin, erlang-q...@erlang.org
On 10/01/2012 08:11 PM, Kenneth Lundin wrote:
> > * OTP has a long release process. Many companies need their fixes to
> be included quickly to avoid maintaining huge branches for their
> changes. They generally agree in return to keep track of changes that
> might happen during the product's development.
> I think we are pretty fast with service releases about every 3rd month
> and updates on the maint branch in git between that.

But doesn't that imply upgrading the whole of OTP? If it does, that
means more testing and more chances for new issues. If you can build a
release upgrade with one app from one OTP version and the rest from a
previous version, I'd like to know how.

> > Note that while not everyone uses inets, most servers do use
> erlang:decode_packet/3 which does most of the job. Cowboy doesn't
> anymore, because it needed backward incompatible changes to
> erlang:decode_packet/3. I didn't ask your thoughts, simply because the
> experiment I did hinted that doing the parsing in full Erlang was a
> better idea: this change results in improved HiPE optimizations and
> better reduction management which results in better latency under load.
>
> Interesting, would be good if you could give us more info on this. The
> http parsing in
> decode_packet is there to improve performance. If it is not ( without
> using HiPE) we should consider removing it!

Between now and then binaries appeared, got performance improvements,
and considering Cowboy is full binary (no conversion to atom for methods
or header names or anything) and only receives requests, not responses,
that means it can skip many steps that decode_packet/3 can't.

Binaries also mean you can parse the request line and all the headers
using a single match context which seems to be very fast (good job!) and
I'm guessing is the main reason for improved performance under HiPE (I'm
about to start reading the HiPE papers to understand that better).

> >> In Erlang/OTP we have keep strict compatibility with many of our APIs
> >> which is both a strength and sometimes a burden (when we are not happy
> >> with the old API) but there are mostly ways around that if there is
> >> strong case for a change.
> >
> >
> > It's a big strength for people looking for stability and a big burden
> for people looking for reactivity. Can't really blame the second group
> for finding solutions to their problems.
>
> I am not blaming people

I used a word too strong there. :)

> for implementing their own solution, it is good
> to have a lot to choose from. But I still think the stability is very
> important for the majority of users making systems with a longer life
> cycle than a year.
> Our experience is that we have to be very compatible in order to
> convince our
> users (often big projects) to step up to our latest release.
> By this we can also stop supporting older versions.

Yes and I wouldn't want that to stop, that's a strong point. In fact
having both ways is great, it means you can make sure the important
parts of your system are stable while the more bleeding-edge parts are
kept, well, bleeding-edge.

> >> I think it is of great value to have a relative complete functionality
> >> in Erlang/OTP so that
> >> the user can implement basic functionality without having to search
> >> for separate implementations of, with unknown quality and completeness
> >> at least for a beginner.
> >
> >
> > I would prefer Erlang/OTP to focus on the core features and leave the
> satellite features to third parties. OTP has an HTTP server but doesn't
> have proper Unicode support. OTP has OpenGL but doesn't have proper
> timezones support. These 2 examples missing from OTP are some of many
> things that are too complex for most people to implement, and that would
> not benefit from having alternatives.
>
> What is missing when it comes to timezones? Has it been mentioned in
> another thread?
> I think we rely on the OS configuration here.

Date/time conversion from one timezone to another. For example if your
data is saved with UTC time but the user is in Paris you want to convert
it to Europe/Paris.

Perhaps it exists, but I haven't found it.

I think there's a lot of work to be done with regards to localization
and internationalization support, but I haven't had enough time to think
about it in details yet. Unicode (not just Unicode though, but a native
string type abstracting encodings, EEP coming in a few weeks),
timezones, graphical drawing libraries are pretty much what I find lacks
the most for the web side of things. Graphical drawing libraries are
fine outside of OTP for a while (until there's one that can do near
everything and do it well) though.

Thanks for the info on package management and bug tracker. Hope we'll be
involved in the process of creating the former. And thanks for your nice
reply. :)

--
Loïc Hoguin


Erlang Cowboy
Nine Nines
http://ninenines.eu

fr...@circlewave.net

unread,
Oct 1, 2012, 4:29:04 PM10/1/12
to Lo?c Hoguin, erlang-q...@erlang.org
On Mon, Oct 01, 2012 at 05:38:52PM +0200, Lo?c Hoguin wrote:
> Finding things in OTP unexpectedly is a quite common occurence, though.

Not OTP's fault, anybody can do (for example):

$ mandb ~/R15B01/lib/erlang/man
[... noise ...]

$ apropos -M ~/R15B01/lib/erlang/man http
http_uri (3) - URI utility module
httpc (3) - An HTTP/1.1 client
httpd (3) - An implementation of an HTTP 1.1 compliant Web server, as defined in RFC 2616.
httpd_conf (3) - Configuration utility functions to be used by the Erlang Web server API programmer.
httpd_socket (3) - Communication utility functions to be used by the Erlang Web server API programmer.
httpd_util (3) - Miscellaneous utility functions to be used when implementing Erlang Web server API modules.

Standard functionality, really. And there's always find(1) and grep(1)
to assist with detective work -- not perfect, but quite efficient
anyways.

BR,
-- Jachym

Thomas Allen

unread,
Oct 2, 2012, 10:18:25 AM10/2/12
to erlang-q...@erlang.org
> "The standard library is where modules go to die"
> http://www.leancrew.com/all-this/2012/04/where-modules-go-to-die/
> This is from python world, and is very obviously seen in Erlang.

Could you cite some examples of third-party libraries being absorbed into
the Erlang/OTP distribution and "dying"? Perhaps this is not as obvious to
everybody as you think.

> Alternatively https://npmjs.org/ has 15k+ packages, we dont need to add
> user management and the kitchen sink to inets, we need to remove it from
> the standard library and promote userland packages as first class citizens
> (ie pick a tool to install packages and bundle it)

I couldn't disagree with that much more ... npm/Node.js is cluttered with
packages of dubious quality that are not maintained in the slightest, and
while this "Wild West" they prefer may seem exciting, I do not think it
bodes well for the long-term stability of projects that depend on these
packages.

If some particular functionality becomes used in a large body of Erlang
projects, it should certainly be brought into the stdlib. Per your Python
reference, one need only look at the "json" module for an example where
bringing something into the stdlib has made deployment and maintenance of
Python code-bases simpler -- no more installing the "simplejson" package,
docs right there on docs.python.org, and JSON earning first-class status
as a serialization format for Python objects.

To advocate actually removing things from the Erlang stdlib, potentially
breaking working code on a large scale, is madness to me.

Thomas Allen

Serge Aleynikov

unread,
Oct 2, 2012, 11:52:37 AM10/2/12
to erlang-q...@erlang.org
I agree with Joe that having a design of the HTTP server architecture
included in OTP that would offer standardized internal API and allow
developers to either replace selected functionality via providing
callback modules or swap the HTTP servers easily would be a great plus.

Concerning inets, what's really strange is that whenever we see some
benchmarks posted comparing performance of different HTTP
implementations, inets is never included there, though I would think, it
would make more sense in terms of showing how a custom implementation
compares to what's included in OTP. Does it have to do something with
what Loïc referred to as inets being well hidden in OTP? I doubt, but
if not that, then what? There are not even benchmarks comparing yaws to
inets, and despite the fact that both offer a rich set of functionality
make one wonder was was the idea of writing yaws in the first place
given existence of inets' httpd.

Diversity is good, but at some point it creates a burden for the user
who needs to make a selection of an HTTP server for his project.
Instead of solving the application-related problem, now the task becomes
studying several implementations to see which one offers a feature-set
best suited for the problem being solved, and the answer to that
question is not always straightforward.

Perhaps a good initiative from the OTP team would be to spec out the
open architecture of an HTTP server and welcome feedback from authors of
current http libraries, so that the successor of inets would be flexible
enough to accommodate building light-weight and heavy-weight HTTP
servers based on the supported code base included in OTP? This would
also open a possibility of diversity for library writers
replacing/enhancing parts to suit their needs.

Serge

Roberto Ostinelli

unread,
Oct 2, 2012, 12:50:34 PM10/2/12
to Serge Aleynikov, erlang-q...@erlang.org
On Sun, Sep 30, 2012 at 2:29 PM, Serge Aleynikov <se...@aleynikov.org> wrote:
The results are quite surprising - everyone who seems comfortable coding
in Erlang, feels compelled to implement an HTTP server.  

I find this quite offensive to all the people that are putting efforts and releasing their own work open source (and especially in BSD/MIT terms). But I'm sure that you only meant to be provocative. :)

I too believed that too many duplication of efforts were going on, and that's the reason why I've stopped misultin development [1] something like 8 months ago. The main result that I've obtained in closing misultin was to push people to concentrate efforts around cowboy and yaws, and at least as far as cowboy goes, this has been a success (judging by the number of products that I've helped switching over).

However, I believe everyone should do whatever they feel like. Open source and many projects are the result of a bunch of intelligent people desires to bring their own creatures to life., andI see nothing bad in that. I'm actually sorry to read statements such as "diversity is bad" or "too many options".

This. is. open sooooooooooource. [2]

r.


Serge Aleynikov

unread,
Oct 2, 2012, 1:23:30 PM10/2/12
to Roberto Ostinelli, erlang-q...@erlang.org
Roberto,

On 10/2/2012 12:50 PM, Roberto Ostinelli wrote:
> On Sun, Sep 30, 2012 at 2:29 PM, Serge Aleynikov <se...@aleynikov.org
> <mailto:se...@aleynikov.org>> wrote:
>
> The results are quite surprising - everyone who seems comfortable coding
> in Erlang, feels compelled to implement an HTTP server.
>
>
> I find this quite offensive to all the people that are putting efforts
> and releasing their own work open source (and especially in BSD/MIT
> terms). But I'm sure that you only meant to be provocative. :)

I am guilty too of having implemented my own small Erlang HTTP server at
some point in the past before deciding to switch to using inets! :-)

> I too believed that too many duplication of efforts were going on, and
> that's the reason why I've stopped misultin development [1] something
> like 8 months ago. The main result that I've obtained in closing
> misultin was to push people to concentrate efforts around cowboy and
> yaws, and at least as far as cowboy goes, this has been a success
> (judging by the number of products that I've helped switching over).

Concerning misultin, I did previously look at it and thought it was
great. Despite the success, your decision to close it in favor of
promoting other http server initiatives was very much admirable.

> However, I believe everyone should do whatever they feel like. Open
> source and many projects are the result of a bunch of intelligent people
> desires to bring their own creatures to life., andI see nothing bad in
> that. I'm actually sorry to read statements such as "diversity is bad"
> or "too many options".
>
> This. is. open sooooooooooource. [2]

See my previous post. I don't think we disagree in terms of what you
said above.

BR,

Serge

Loïc Hoguin

unread,
Oct 2, 2012, 1:37:14 PM10/2/12
to Serge Aleynikov, erlang-q...@erlang.org
On 10/02/2012 05:52 PM, Serge Aleynikov wrote:
> I agree with Joe that having a design of the HTTP server architecture
> included in OTP that would offer standardized internal API and allow
> developers to either replace selected functionality via providing
> callback modules or swap the HTTP servers easily would be a great plus.

This wouldn't have helped Cowboy, as one of the biggest difference is
the use of binary instead of strings.

There's another solution, which is to write wrappers for other APIs, for
example mochicow allows Mochiweb applications to use Cowboy under the
hood without having to change their code. This means that the switch can
be progressive instead of being brutal like a direct code switch.

> Diversity is good, but at some point it creates a burden for the user
> who needs to make a selection of an HTTP server for his project.

It doesn't!

> Instead of solving the application-related problem, now the task becomes
> studying several implementations to see which one offers a feature-set
> best suited for the problem being solved, and the answer to that
> question is not always straightforward.

Erlang is wonderful in that it allows you to write your application 100%
HTTP-free, and then write another application for the HTTP layer that
will be a thin layer running alongside your application. The
communication layer can be separate from the business logic.

If you go that way then your application's code isn't dependent on the
transports used at all, and you can easily switch servers, change
protocols, serialization formats or anything in your HTTP layer without
this impacting your application in any way.

For example, when SPDY support will be added to Cowboy, only the HTTP
layer will require a change to make use of it (start_http, which only
does HTTP, can be replaced by start_spdy, which does SPDY or HTTP
depending on client capabilities). The application behind it shouldn't
have to care about that.

> Perhaps a good initiative from the OTP team would be to spec out the
> open architecture of an HTTP server and welcome feedback from authors of
> current http libraries, so that the successor of inets would be flexible
> enough to accommodate building light-weight and heavy-weight HTTP
> servers based on the supported code base included in OTP? This would
> also open a possibility of diversity for library writers
> replacing/enhancing parts to suit their needs.

More importantly, do people actually want that?

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

Mike Oxford

unread,
Oct 2, 2012, 2:09:38 PM10/2/12
to Loïc Hoguin, erlang-q...@erlang.org
I would even go so far as to propose that almost every single Erlang project, for the foreseeable future will want an HTTP service.  I would even extend this to "most project in any realm in any language."

Reason:  Even if you're not providing an external service, having a web-status console, admin or REST-based health/monitoring/whatever port has a whole lot of value.

Right now, if you want external services you have to meld server+protocol+client.  The HTTP's strongest suit is that, just like javascript, it's ubiquitous.  It may not be optimal, but it's everywhere and it's easy.  
Would I prefer a world built on client-side(browser!) Erlang, a length-prefixed binary protocol and real sockets (instead of websocket silliness)? Hell yes.  But we don't have that, and likely won't anytime soon, so HTTP+javascript continues to be the lingua franca.

Having too many options is a Bad Thing<tm>, IMO.  Erlang is already enough of a 'big scary monster in the dark' for a lot of people.  If we had a unified, documented, packaged and turnkey solution withing the stdlib so that users could 'drop in and go' it would increase uptake as people at least explored it.  The are more web-coders than ever, many with just enough knowledge to be dangerous.  Apple vs PC back in the day --- snag all of the low-hanging cheap fruit of PC clones instead of insisting on the white-ivory-tower and then achieve market dominance (which is then yours to lose/squander at your leisure... :)

-mox

Claes Wikstrom

unread,
Oct 2, 2012, 5:29:42 PM10/2/12
to erlang-q...@erlang.org
On 10/1/12 4:30 PM, Kenneth Lundin wrote:
>>
> I agree with Serge that it is somewhat unfortunate that there are so
> many initiatives
> where people write their own implementation of something we already
> have included in
> the Erlang/OTP distribution and doing this without even suggesting or
> asking us if we could
> incorporate some missing feature or improve the current implementation
> in various ways.

Why, the more the merrier, let the best library win.

>> b) Servers have two interfaces - the protocol interface (HTTP) and
>> the interface towards
>> Erlang (the API if you like). As far as I can see most severs
>> have different Erlang APIs.
>> This is bad.
> Would be good if there was a kind of standardized API for HTTP-servers
> and clients.


Disagree, in todays webdevelopment the DOM is the really important
interface, and that is standardized.
Switching servers is work, yes, but doable.

Actually, the API between the protocol HTTP and the erlang code
is pretty much what distinguishes the different servers, this api can
be more or less ... well good. Yaws has appmods to do this, other
servers have different mechanisms. This is a matter of preference and
standardizing on this pretty much removes the need for different servers
and we're back on square one - we should all have contributed
to inets instead. This would have lead to a worse world.

-klacke

Michael Turner

unread,
Oct 2, 2012, 10:25:40 PM10/2/12
to Serge Aleynikov, erlang-q...@erlang.org
I recently received a copy of Building Web Applications with Erlang,
which is almost entirely about yaws. The discussion of inets on this
thread spurred me to look in the index under "i" -- except, oops, the
book has no index. It does have an appendix about other webservers and
frameworks in Erlang -- but inets is not listed there.

"Well-hidden" indeed.

-michael turner

Jesper Louis Andersen

unread,
Oct 3, 2012, 4:04:30 AM10/3/12
to tho...@oinksoft.com, erlang-q...@erlang.org
On Oct 2, 2012, at 4:18 PM, "Thomas Allen" <tho...@oinksoft.com> wrote:

>
> I couldn't disagree with that much more ... npm/Node.js is cluttered with
> packages of dubious quality that are not maintained in the slightest, and
> while this "Wild West" they prefer may seem exciting, I do not think it
> bodes well for the long-term stability of projects that depend on these
> packages.
>

Two ideas spring to mind immediately.

First, one should definitely not make the package repository a dump. You want to make package rules. Does it have documentation? Does it have a test suite? Can the test suite be run automatically? Semantic versioning? And so on. Is the package experimental, does it have real world uses?

Furthermore, we could definitely adopt the ideas from the haskell-platform: A set of well-defined and stable packages are made into a basis library. Packages from the dump "graduate".



Jesper Louis Andersen
Erlang Solutions Ltd., Copenhagen

Yurii Rashkovskii

unread,
Oct 3, 2012, 4:11:08 AM10/3/12
to erlang-pr...@googlegroups.com, erlang-q...@erlang.org


On Wednesday, October 3, 2012 1:04:45 AM UTC-7, Jesper Louis Andersen wrote:
 
> I couldn't disagree with that much more ... npm/Node.js is cluttered with
> packages of dubious quality that are not maintained in the slightest, and
> while this "Wild West" they prefer may seem exciting, I do not think it
> bodes well for the long-term stability of projects that depend on these
> packages.
>

Two ideas spring to mind immediately.

First, one should definitely not make the package repository a dump. You want to make package rules. Does it have documentation? Does it have a test suite? Can the test suite be run automatically? Semantic versioning? And so on. Is the package experimental, does it have real world uses?

Furthermore, we could definitely adopt the ideas from the haskell-platform: A set of well-defined and stable packages are made into a basis library. Packages from the dump "graduate".


I believe in giving people simple and flexible tools and let them build the rules later on. This is basically why, after doing my work on Agner, I recently launched EXPM (http://expm.co, http://rashkovskii.com/2012/10/01/expm-or-meet-agner-2/) that is much simpler and much more flexible. EXPM allows to host your own repos very trivially, therefore allowing to build any combinations, including the one suggested by you.

So far I am just running a "dump" index at expm.co to bootstrap this system (after all, publishing got magnitudes easier with expm comparing to Agner), but I can very much see how we can eventually add another repository (say, stable.expm.co) that will only "host" graduated packages.

Yurii.

Motiejus Jakštys

unread,
Oct 3, 2012, 4:18:29 AM10/3/12
to Yurii Rashkovskii, erlang-pr...@googlegroups.com, erlang-q...@erlang.org
On Wed, Oct 03, 2012 at 01:11:08AM -0700, Yurii Rashkovskii wrote:
> I believe in giving people simple and flexible tools and let them build the
> rules later on. This is basically why, after doing my work on Agner, I
> recently launched EXPM (http://expm.co,
> http://rashkovskii.com/2012/10/01/expm-or-meet-agner-2/) that is much
> simpler and much more flexible. EXPM allows to host your own repos very
> trivially, therefore allowing to build any combinations, including the one
> suggested by you.

... And why are you announcing this in a thread about "Erlang http
servers"? Please, make a proper announcement.

> So far I am just running a "dump" index at expm.co to bootstrap this system
> (after all, publishing got magnitudes easier with expm comparing to Agner),
> but I can very much see how we can eventually add another repository (say,
> stable.expm.co) that will only "host" graduated packages.

From first sight it looks like a clone of PyPi (which, IMHO, is
great). Let's see how it goes.

Motiejus
signature.asc

Yurii Rashkovskii

unread,
Oct 3, 2012, 4:24:04 AM10/3/12
to erlang-q...@erlang.org
Motiejus,

>>> I believe in giving people simple and flexible tools and let them build the
>>> rules later on. This is basically why, after doing my work on Agner, I
>>> recently launched EXPM (http://expm.co,
>>> http://rashkovskii.com/2012/10/01/expm-or-meet-agner-2/) that is much
>>> simpler and much more flexible. EXPM allows to host your own repos very
>>> trivially, therefore allowing to build any combinations, including the one
>>> suggested by you.
>>
>> ... And why are you announcing this in a thread about "Erlang http
>> servers"? Please, make a proper announcement.
>
I will, I am just fixing things here and there to make sure it doesn't
have too many embarrassingly stupid bugs when I make a "proper"
announcement.

But I do want to involve early adopters who like playing with new
stuff (that's why I started talking about it, after a long period of
silence on "Agner 2" project)

Yurii

Ingela Andin

unread,
Oct 3, 2012, 4:00:02 PM10/3/12
to Michael Turner, erlang-q...@erlang.org
Hi!

I feel compelled to contribute on the topic of the hidden inets
application. Once a upon a time someone in Ericsson wrote an http
server that was very much inspired by Apache, which probably seemed
sensible at the time but it does not feel like the erlang way of doing
things. It was almost not document but used extensively by
Ericsson projects, hence very hard to get rid of. Years later several
improvements have been made and inets httpd server can be run in a
much more erlangish way without Apache config files, even though we
still have to support them. The documentation is much improved but
still quite sparse. And now I think we come to the negative circle. A
lot of pople seem to think HTTP is easy and roll their own and the
Ericsson projects are mostly content and do not have so many new
requirements. As you know HTTP is not Ericsson core business and most
open source users used something else and hence inets have not been
given the highest priority. And there
has been hardly no user contributions. Most efforts to improve inets
has been to the HTTP client as it is more used. Lesson to be learnt it
is worth doing things properly from the start because other pople may
have to live with your code and bad naming for years and years to
come.

Regards Ingela Erlang/OTP team - Ericsson AB

2012/10/3 Michael Turner <michael.eu...@gmail.com>:

Ulf Wiger

unread,
Oct 3, 2012, 4:23:17 PM10/3/12
to Ingela Andin, erlang-q...@erlang.org

As one of the first users of the Inets web server, I would like to
point out that, at the time, making it Apache-compatible made
perfect sense. It was by no means obvious that it was the best
choice to use an Erlang-based web server in a telecom-class
product, and Apache was a strong contender.

Of course, it would have been nice to have been able to put
a lot of effort into the web server already from the start, but it
was by no means a priority, as we had much bigger fish to
fry with OTP. ;-)

The Inets HTTP server did it's job extremely well given our
requirements. I cannot recall any issues - which is not to say
that there weren't any, but that they were not serious enough
to linger in my mind all this time.

The main challenge for us on the HTTP front was to create a
framework that allowed us to design and maintain over 100
different web pages, with dynamic content, access control and
live SNMP hooks. *That* would have been something to contribute
to the Open Source community. After a wholesale rewrite in
(I think) 1999, it was actually pretty nice, esp. compared to what
was needed and expected at the time.

BR,
Ulf W

On 3 Oct 2012, at 22:00, Ingela Andin wrote:

> Hi!
>
> I feel compelled to contribute on the topic of the hidden inets
> application. Once a upon a time someone in Ericsson wrote an http
> server that was very much inspired by Apache, which probably seemed
> sensible at the time but it does not feel like the erlang way of doing
> things. It was almost not document but used extensively by
> Ericsson projects, hence very hard to get rid of. ...

Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
http://feuerlabs.com

Max Lapshin

unread,
Oct 3, 2012, 4:28:31 PM10/3/12
to Ingela Andin, erlang-q...@erlang.org
I think, it is a problem about HTTP, not about user contributions.

HTTP is an outstanding protocol. It is not yet-another-silly-protocol
like RTMP or LDAP.
It is a basic protocol for lots of projects, starting today. For
example, Dropbox. It could use rsync 10 years ago,
but today it is using only HTTP.


And it is a very complicated protocol that includes many modern
extensions, like websockets, etc.

This is why users require implementation which is very optimized and
fresh. It is really hard to write such a basic code of HTTP server,
that will fit all users requirements for next 3 years.


To understand it better, lets try to think: why there is no SIP + RTP
implementation in OTP? Why telecom platform doesn't include main
telecom protocol? I think, it is because it is almost impossible to
write such a SIP implementation, that will fit all needs.

Joe Armstrong

unread,
Oct 3, 2012, 4:43:21 PM10/3/12
to Max Lapshin, erlang-q...@erlang.org
On Wed, Oct 3, 2012 at 10:28 PM, Max Lapshin <max.l...@gmail.com> wrote:
> I think, it is a problem about HTTP, not about user contributions.
>
> HTTP is an outstanding protocol. It is not yet-another-silly-protocol
> like RTMP or LDAP.
> It is a basic protocol for lots of projects, starting today. For
> example, Dropbox. It could use rsync 10 years ago,
> but today it is using only HTTP.
>
>
> And it is a very complicated protocol that includes many modern
> extensions, like websockets, etc.
>
> This is why users require implementation which is very optimized and
> fresh. It is really hard to write such a basic code of HTTP server,
> that will fit all users requirements for next 3 years.
>
>
> To understand it better, lets try to think: why there is no SIP + RTP
> implementation in OTP? Why telecom platform doesn't include main
> telecom protocol?

Possibly for the same reason that apple doesn't release the
source code for OS-X in X-code :-)

/J

Max Lapshin

unread,
Oct 3, 2012, 4:47:37 PM10/3/12
to Joe Armstrong, erlang-q...@erlang.org
On Thu, Oct 4, 2012 at 12:43 AM, Joe Armstrong <erl...@gmail.com> wrote:
>
> Possibly for the same reason that apple doesn't release the
> source code for OS-X in X-code :-)
>

Yes, I haven't even though about it =)))

But indeed: what can be considered as a "rather generic" implementation of SIP?
What be useful for all?

Ulf Wiger

unread,
Oct 3, 2012, 5:34:05 PM10/3/12
to Max Lapshin, erlang-q...@erlang.org
I talked in Freiburg 2007 about a SIP stack that could have made a fine addition to OTP, but as Joe says...

(page 4, the actual slides are behind the ACM paywall)

BR,
Ulf W

Ulf Wiger, Feuerlabs, Inc.

Max Lapshin

unread,
Oct 3, 2012, 5:48:03 PM10/3/12
to Ulf Wiger, erlang-q...@erlang.org
"A fortunate turn of events allowed us to revive the early SIP exper-
iments, rewrite the software and experiment to find an optimal
architecture"

I consider that this is exactly what is happening now with http
servers in erlang.

Take a look at difference between yaws api, inets api and cowboy api.
They are really different even in such deep things:
* how many processes should serve http request?
* fully async parsing of http or using recv to synchronously parse it
* use return codes or explicit replying via reply module, etc.

This is an experimenting and it is hard to experiment when you need to
think about backward compatibility.

Richard O'Keefe

unread,
Oct 3, 2012, 6:22:42 PM10/3/12
to Ingela Andin, erlang-q...@erlang.org
Part of the issue with inets is that at one's very
first look at the documentation it looks as though
nobody is bothering to maintain the documentation.

Inets is a container for Internet clients and servers.
Currently, an client and server, a TFPT client and server,
^^ s/an/a/
and a FTP client has been incorporated into Inets.
^^^ s/has/have/
The HTTP server and client is HTTP 1.1 compliant
^^ s/is/are/
as defined in 2616.
^ s//RFC /
...
Each client and server in inets is viewed as service.
^s//a /

At the last count, my Swedish vocabulary was a massive 20 words.
I couldn't even document how to boil water in Swedish.
So I'm not criticising anyone at Ericsson for writing this in
almost-English.

What I *am* saying is that practically every sentence on the
first page of text has something about it that's like a poke
in the eye to the point that for me it was less stressful to
find something else than to read this manual.

The text really needs to be revised by a native speaker of
English who is also a good writer.

Of course, getting that done professionally costs money, and
I paid, oh, it must be at least 0 dollars for Erlang...
Making the documentation sources and the tool chain available
means that it's possible now to contribute to people who need
HTTP (or FTP) services in Erlang by improving the inets
documentation, or it will be when the bug I reported (about
spaces dropping out) is fixed.

Ingela Andin

unread,
Oct 4, 2012, 3:56:01 AM10/4/12
to Max Lapshin, erlang-q...@erlang.org
Hi!

2012/10/3, Max Lapshin <max.l...@gmail.com>:
> I think, it is a problem about HTTP, not about user contributions.

No the problem is not that we do not get user contributions, but the
lack of contributions
is a symptom of the problem that the for very long the non existing
documentation
of httpd resulted in that people did not understand how inets
webserver works and dismisses
it, not because it does not work well, but because they do not
understand how it works.
I still think the documentation is lacking but it is relativly much
better than before.
Considering HTTP and all things surrounding it I would still expect
there to be other webserves around. To compare there are people using
the odbc-application and contributing to it, but there are also a lot
of pople that think a more native API is what they need and roll thier
own. Point is the application has a life outside of Ericsson (in this
particulare case it
might not still be here if it had not).

> This is why users require implementation which is very optimized and
> fresh. It is really hard to write such a basic code of HTTP server,
> that will fit all users requirements for next 3 years

Inets tries to be a small core that should be easy to extend and
customize not a big bang webserver that comes with all the bells and
whistles. The big problem with the fist version of inets was not the
Apache approch but rather the lack of documentation. To make
improvments, extensions etc to an application you need to understand
how it is meant to work.

So point taken documentation needs further improvments.

Regards Ingela Erlang/OTP Team - Ericsson AB

David Mercer

unread,
Oct 4, 2012, 9:18:04 AM10/4/12
to erlang-q...@erlang.org
Can someone please summarize this thread for me? If I am going to run a web server in Erlang, which one should I use?

Cheers,

DBM

> -----Original Message-----
> From: erlang-quest...@erlang.org [mailto:erlang-questions-
> bou...@erlang.org] On Behalf Of Serge Aleynikov
> Sent: Sunday, September 30, 2012 16:29
> To: erlang-q...@erlang.org
> Subject: [erlang-questions] Erlang http servers
>
> Haven't had a need for an HTTP server in Erlang for a while since I last used
> inets-based http server five years back, and now examining what exists in
> public domain.
>
> The results are quite surprising - everyone who seems comfortable coding in
> Erlang, feels compelled to implement an HTTP server. Previously we had
> only one contender to inets - yaws. Now the landscape is quite a mouthful -
> inets, yaws, cowboy, musultin, mochiweb, etc. As Steve Vinoski rightfully
> points out (*) this doesn't help to pull the blanket in all directions. From
> personal experience with inets, I recall that once I figured out how to use it, it
> happened to be a very helpful and powerful HTTP library for building an
> application server. So why don't all the authors of those wonderful
> alternative libraries just patch the inets to extend it with something they feel
> missing so that we only have one good and documented library to use as the
> common denominator?
>
> Serge
>
>
> (*) http://steve.vinoski.net/blog/2011/05/09/erlang-web-server-
> benchmarking

tho...@oinksoft.com

unread,
Oct 4, 2012, 9:24:58 AM10/4/12
to David Mercer, erlang-q...@erlang.org
> Can someone please summarize this thread for me? If I am going to run a
> web server in Erlang, which one should I use?

Right now, Cowboy seems to be the favorite. The fellow who wrote MochiWeb
has said as much in past threads (that he would write MochiWeb like Cowboy
if he were to start today). The author of the misultin project shut it
down so that new development efforts could be focused on Cowboy and tools
that integrate with Cowboy.

WebMachine is still popular for writing RESTful web services, but note
that Cowboy provides this too.

Apache users who don't want to proxy to Cowboy probably want to use Yaws.

That is my take.

Thomas Allen

Steve Vinoski

unread,
Oct 5, 2012, 7:53:32 AM10/5/12
to tho...@oinksoft.com, erlang-q...@erlang.org
On Thu, Oct 4, 2012 at 9:24 AM, <tho...@oinksoft.com> wrote:
>> Can someone please summarize this thread for me? If I am going to run a
>> web server in Erlang, which one should I use?
>
> Right now, Cowboy seems to be the favorite.

Favorite among whom? This is a very broad statement that isn't broadly
true. There are many Yaws users out there, many Mochiweb users, and I
bet there are more than a few who still use Misultin.

IMO Cowboy is popular among folks who need a library-like HTTP server
and who also like to be leading edge, but not everyone falls into that
category, not even close. Also, there seems to be some ongoing rumor
that Yaws can't handle library-like HTTP apps, but that is entirely
untrue.

> WebMachine is still popular for writing RESTful web services, but note
> that Cowboy provides this too.

This statement is slightly confusing. The phrase "Cowboy provides
this" could be taken to mean "Cowboy provides Webmachine", which isn't
true. So just to be clear, what you really meant was that Cowboy
provides assistance for writing RESTful web services.

> Apache users who don't want to proxy to Cowboy probably want to use Yaws.

This is another unfair generalization. I've never been an Apache user,
yet I've used Yaws for everything from regular websites to embedded
systems and it's always given me just what I needed.

At about 11 years old and still going strong, Yaws is popular among
users who need a system they can count on to provide years of
production stability, backwards compatibility, and consistent support,
all while keeping up with the latest Web developments such as
streaming, long polling, WebSocket, and Server-Sent Events.

--steve

Thomas Allen

unread,
Oct 5, 2012, 9:04:08 AM10/5/12
to Steve Vinoski, erlang-q...@erlang.org
On Fri, October 5, 2012 7:53 am, Steve Vinoski wrote:
> Favorite among whom? This is a very broad statement that isn't broadly
> true. There are many Yaws users out there, many Mochiweb users, and I
> bet there are more than a few who still use Misultin.

I'm sorry, I made an unfair generalization. I should've been more
specific: Following this mailing list and discussions in the Freenode
channel for some time, it seems to me that the Cowboy project has a great
deal of momentum. This is not to say that the other tools are somehow
unsuitable or will all die out.

I certainly didn't intend to slight Yaws, which is a perfectly stable,
capable web server, as are the other projects. Perhaps I should have
phrased my recommendation in terms of what the others are good for rather
than comparing them to Cowboy: You can't go wrong with Yaws or MochiWeb
which are proven, stable servers. WebMachine is notably well-suited for
writing RESTful services and focuses on HTTP compliance. Misultin is a
good product but is no longer actively developed or maintained by the
original author, so it is a riskier choice.

Thomas Allen

Tristan Sloughter

unread,
Oct 5, 2012, 10:01:44 AM10/5/12
to tho...@oinksoft.com, erlang-q...@erlang.org, Steve Vinoski
This thread is still going? How long till it is the longest thread ever on the mailing list? (I suspect those are in the 100s though actually, haha).

But how has this not just ended with the understanding:

a) Yaws for some, can be embedded or used like a stand alone web server and is known to be stable, fast and production ready
b) Cowboy for http library in your app and you like bleeding edge and a smaller dependency -- and now even smaller in it self but requires ranch
c) Ellis for those who don't care to support http
d) Whatever makes you happy, maybe write your own!

And in my opinion Cowboy REST is now a much better Webmachine than Webmachine, due to some changes around handling POST and a few other things.

Tristan

Motiejus Jakštys

unread,
Oct 5, 2012, 10:23:35 AM10/5/12
to Tristan Sloughter, erlang-q...@erlang.org, Steve Vinoski
On Fri, Oct 5, 2012 at 3:01 PM, Tristan Sloughter
<tristan....@gmail.com> wrote:
> This thread is still going? How long till it is the longest thread ever on
> the mailing list? (I suspect those are in the 100s though actually, haha).

Since you mentioned it. This thread is still quite far from the top
ones with my email being the 41'st one:
http://m.jakstys.lt/tech/2012/02/erlang-most-popular-subjects/

> But how has this not just ended with the understanding:
>
> a) Yaws for some, can be embedded or used like a stand alone web server and
> is known to be stable, fast and production ready
> b) Cowboy for http library in your app and you like bleeding edge and a
> smaller dependency -- and now even smaller in it self but requires ranch
> c) Ellis for those who don't care to support http
> d) Whatever makes you happy, maybe write your own!
>
> And in my opinion Cowboy REST is now a much better Webmachine than
> Webmachine, due to some changes around handling POST and a few other things.

At least on 0.6.0 cowboy had unexpectedly horrible performance issues
handling big POST requests, like discovered by my colleague[1]. Not
sure if this is now fixed. So *always* measure. Always.

[1]: https://github.com/yfyf/cowboy_bench

--
Motiejus Jakštys

Loïc Hoguin

unread,
Oct 5, 2012, 10:28:54 AM10/5/12
to Motiejus Jakštys, erlang-q...@erlang.org
On 10/05/2012 04:23 PM, Motiejus Jakštys wrote:
> At least on 0.6.0 cowboy had unexpectedly horrible performance issues
> handling big POST requests, like discovered by my colleague[1]. Not
> sure if this is now fixed. So *always* measure. Always.
>
> [1]: https://github.com/yfyf/cowboy_bench

It didn't, it just didn't support Expect: 100-continue.

It does in 0.6.1.

If there's still something wrong, please open a ticket. Nobody's
complaining at the moment.

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

David Mercer

unread,
Oct 5, 2012, 10:36:51 AM10/5/12
to Loïc Hoguin, Motiejus Jakštys, erlang-q...@erlang.org
Does Cowboy work on Windows? I ask because I went to download it and give it a whirl, but the first line of the Quick Start section says to use Rebar, which does not work on Windows. Please advise. Thank-you.

Cheers,

DBM


> -----Original Message-----
> From: erlang-quest...@erlang.org [mailto:erlang-questions-

Loïc Hoguin

unread,
Oct 5, 2012, 10:39:17 AM10/5/12
to David Mercer, erlang-q...@erlang.org
It hasn't been tested on Windows, but it should work, with possibly a
few easily fixed bugs in cowboy_static (that's the only Cowboy code
dealing with files).

I have no idea how to build an Erlang project on Windows.

Patches are welcome!

Tuncer Ayaz

unread,
Oct 5, 2012, 10:54:58 AM10/5/12
to David Mercer, erlang-q...@erlang.org
On Fri, Oct 5, 2012 at 4:36 PM, David Mercer wrote:
> Does Cowboy work on Windows? I ask because I went to
> download it and give it a whirl, but the first line of the Quick
> Start section says to use Rebar, which does not work on Windows.
> Please advise. Thank-you.

rebar does work and is used by Erlang developers on Windows.
What specifically didn't work for you? Did you use rebar.cmd?

Dave Cottlehuber

unread,
Oct 5, 2012, 11:37:20 AM10/5/12
to David Mercer, Loïc Hoguin, Motiejus Jakštys, erlang-q...@erlang.org
On 5 October 2012 16:36, David Mercer <dme...@gmail.com> wrote:
> Does Cowboy work on Windows? I ask because I went to download it and give it a whirl, but the first line of the Quick Start section says to use Rebar, which does not work on Windows. Please advise. Thank-you.

Just to confirm, rebar *does* work on Windows, even for compiling
NIFs. Obviously the C code needs to support an appropriate compiler in
the first place, but that's not rebar's fault. So far I have built via
rebar snappy, ejson, jiffy, erica, and a whole plethora of other tools
& projects, without needing a special unix shell, just erlang & rebar.
Many thanks to Basho & Tuncer for making that possible.

Loïc, I'll give cowboy a whirl on Windows sometime next week & will
send an issue or patch if reqd.

A+
Dave

David Mercer

unread,
Oct 5, 2012, 12:47:06 PM10/5/12
to Dave Cottlehuber, Loïc Hoguin, Motiejus Jakštys, erlang-q...@erlang.org
I'll give it another whirl. Tried it about a year ago and it didn't work. Rebar, that is; haven’t tried Cowboy yet.

I'll report back on my experiences so that I don't leave you hanging.

Cheers,

DBM


> -----Original Message-----
> From: Dave Cottlehuber [mailto:d...@jsonified.com]
> Sent: Friday, October 05, 2012 10:37
> To: David Mercer; Loïc Hoguin; Motiejus Jakštys; erlang-
> ques...@erlang.org
> Subject: Re: [erlang-questions] Erlang http servers
>

Jim

unread,
Oct 5, 2012, 2:35:36 PM10/5/12
to David Mercer, <erlang-questions@erlang.org>
I am currently using Cowboy, built with rebar, on Windows 7, 64Bit. I did not have to do anything special to build. And, except for issues of my own creation, have had no problems; although I have on,y used the most basic http_handler to date.

Sent from my iPad

Reply all
Reply to author
Forward
0 new messages