There are likely more issues with this approach. Ideally, it
should be impossible for the user of your server to know what you
used to implement it -- this is what should give you the most
freedom in terms of implementation when lots of people start using
it. At this point I'm thinking we should start using different
protocols from HTTP if we're not really willing to respect it, but
that's for another discussion.
And yes, more user management would definitely be nice. I've
written a tiny library to handle passwords themselves, but it's
not close enough to be something a framework would use without
surrounding support -- it just handles the core of password
hashing and verification. It's currently hosted at
https://github.com/ferd/erlpass . Then each platform such as
Zotonic likely reimplemented their own, although I don't know how
moveable they are outside of the project.
Den 1 okt 2012 17:39 skrev "Loïc Hoguin" <es...@ninenines.eu>:
>
> On 10/01/2012 04:30 PM, Kenneth Lundin wrote:
>>
>> I agree with Serge that it is somewhat unfortunate that there are so
>> many initiatives
>> where people write their own implementation of something we already
>> have included in
>> the Erlang/OTP distribution and doing this without even suggesting or
>> asking us if we could
>> incorporate some missing feature or improve the current implementation
>> in various ways.
>
>
> It happens a lot. People tend to review available options, and if none fits their needs, write something themselves without consulting any of the authors. Sometimes they're right to do so, because the answer they'll get is obvious, and sometimes not.
>
> There's a few reasons this happens for this specific case:
>
> * HTTP is simple, it's easier to go ahead and experiment on a new project than on an existing one.
> * If the experiment succeeds and the implementation has to differ significantly from other projects, it's not worth backporting since you'll just run into a wall of backward compatibility requirements.
> * OTP has a long release process. Many companies need their fixes to be included quickly to avoid maintaining huge branches for their changes. They generally agree in return to keep track of changes that might happen during the product's development.
I think we are pretty fast with service releases about every 3rd month and updates on the maint branch in git between that.
> * Inets is seriously well hidden. I only heard about it after releasing Cowboy. I knew about Yaws/Mochiweb/Misultin but not the inets HTTP server. Many people I talked to didn't seem to know it existed.
I agree on that. I have always thought that the name inets is misleading.
> * Sometimes removing features is an improvement. Inets is big, it's only natural that lightweight alternatives appear. Same happened to Apache, and it's no surprise that Inets is quite similar to Apache from a user point of view.
I agree here too. The http server included in OTP should have been a lightweight one leaving for others to implement a more complete or specialized one.
>
> Note that while not everyone uses inets, most servers do use erlang:decode_packet/3 which does most of the job. Cowboy doesn't anymore, because it needed backward incompatible changes to erlang:decode_packet/3. I didn't ask your thoughts, simply because the experiment I did hinted that doing the parsing in full Erlang was a better idea: this change results in improved HiPE optimizations and better reduction management which results in better latency under load.
Interesting, would be good if you could give us more info on this. The http parsing in
decode_packet is there to improve performance. If it is not ( without using HiPE) we should consider removing it!
>
>
>> In Erlang/OTP we have keep strict compatibility with many of our APIs
>> which is both a strength and sometimes a burden (when we are not happy
>> with the old API) but there are mostly ways around that if there is
>> strong case for a change.
>
>
> It's a big strength for people looking for stability and a big burden for people looking for reactivity. Can't really blame the second group for finding solutions to their problems.
I am not blaming people for implementing their own solution, it is good to have a lot to choose from. But I still think the stability is very important for the majority of users making systems with a longer life cycle than a year.
Our experience is that we have to be very compatible in order to convince our
users (often big projects) to step up to our latest release.
By this we can also stop supporting older versions.
>
>
>> We still actively develop inets with its http client and server and I
>> am not sure that
>> all the reasons to develop alternatives still exist.
>
>
> But like you said the level of support for applications included in OTP vary widely. It is also not clearly indicated which parts are the focus of recent works (there is a small roadmap for the most important stuff but that's about it), nor can it be deduced from reading tickets in a bug tracker or similar collaboration tool.
Yes we plan to improve in this area.
I would say that the level of support is equally high for all applications we regard as important and heavily used by Ericsson and or the external Erlang community.
>
> Which brings me to a very important point: please start using a public bug tracker! It doesn't have to be 100% public (as I'm sure you have some issues that are internal only) but it would help a lot if people could see the existing issues instead of stumbling upon the same bugs again and again. It probably also would help with contributions.
We have discussed this as well.
>
>
>> I think it is of great value to have a relative complete functionality
>> in Erlang/OTP so that
>> the user can implement basic functionality without having to search
>> for separate implementations of, with unknown quality and completeness
>> at least for a beginner.
>
>
> I would prefer Erlang/OTP to focus on the core features and leave the satellite features to third parties. OTP has an HTTP server but doesn't have proper Unicode support. OTP has OpenGL but doesn't have proper timezones support. These 2 examples missing from OTP are some of many things that are too complex for most people to implement, and that would not benefit from having alternatives.
What is missing when it comes to timezones? Has it been mentioned in another thread?
I think we rely on the OS configuration here.
>
> Let's take another example. While taking a quick glance at the modules included in OTP I discovered there is a GD implementation in the percept application. I'm pretty sure most people didn't know it was there, as they usually end up using a NIF. It's quite unexpected that it's in OTP.
It was developed as part of Percept just to have something to use there. We don't regard it complete or general enough to be supported as an app of its own.
>
> Finding things in OTP unexpectedly is a quite common occurence, though.
I think most things are documented but the docs could have better search facilities.
>
> If the user had a package management application (perhaps working similarly to rebar's deps, = per project/release, no site-wide install by default) along with a package index website, it could quite easily find what he needs. It wouldn't be of unknown quality or completeness, simply because adding a comment and rating system to a package index website is fairly simple to do.
I agree that a package management system would be good. There are initiatives in that direction.
>
> Plus, he can make an opinion for himself, instead of just using whatever someone decided for him. You might include a project in OTP that fits your criteria for quality, but it might be crap for someone else. A more democratic system where people choose what they use based on other users' feedback and ratings is not only a lot more powerful but also allows to bring some light on some more obscure projects that are sometimes quite hard to find armed only with Google.
>
> It makes sense to have a single choice for big corporate environments, not so much for the rest of the world.
>
Striving in that direction.
> --
> Loďc Hoguin
>
> Erlang Cowboy
> Nine Nines
> http://ninenines.eu
/Kenneth, Erlang/OTP Ericsson
But doesn't that imply upgrading the whole of OTP? If it does, that
means more testing and more chances for new issues. If you can build a
release upgrade with one app from one OTP version and the rest from a
previous version, I'd like to know how.
> > Note that while not everyone uses inets, most servers do use
> erlang:decode_packet/3 which does most of the job. Cowboy doesn't
> anymore, because it needed backward incompatible changes to
> erlang:decode_packet/3. I didn't ask your thoughts, simply because the
> experiment I did hinted that doing the parsing in full Erlang was a
> better idea: this change results in improved HiPE optimizations and
> better reduction management which results in better latency under load.
>
> Interesting, would be good if you could give us more info on this. The
> http parsing in
> decode_packet is there to improve performance. If it is not ( without
> using HiPE) we should consider removing it!
Between now and then binaries appeared, got performance improvements,
and considering Cowboy is full binary (no conversion to atom for methods
or header names or anything) and only receives requests, not responses,
that means it can skip many steps that decode_packet/3 can't.
Binaries also mean you can parse the request line and all the headers
using a single match context which seems to be very fast (good job!) and
I'm guessing is the main reason for improved performance under HiPE (I'm
about to start reading the HiPE papers to understand that better).
> >> In Erlang/OTP we have keep strict compatibility with many of our APIs
> >> which is both a strength and sometimes a burden (when we are not happy
> >> with the old API) but there are mostly ways around that if there is
> >> strong case for a change.
> >
> >
> > It's a big strength for people looking for stability and a big burden
> for people looking for reactivity. Can't really blame the second group
> for finding solutions to their problems.
>
> I am not blaming people
I used a word too strong there. :)
> for implementing their own solution, it is good
> to have a lot to choose from. But I still think the stability is very
> important for the majority of users making systems with a longer life
> cycle than a year.
> Our experience is that we have to be very compatible in order to
> convince our
> users (often big projects) to step up to our latest release.
> By this we can also stop supporting older versions.
Yes and I wouldn't want that to stop, that's a strong point. In fact
having both ways is great, it means you can make sure the important
parts of your system are stable while the more bleeding-edge parts are
kept, well, bleeding-edge.
> >> I think it is of great value to have a relative complete functionality
> >> in Erlang/OTP so that
> >> the user can implement basic functionality without having to search
> >> for separate implementations of, with unknown quality and completeness
> >> at least for a beginner.
> >
> >
> > I would prefer Erlang/OTP to focus on the core features and leave the
> satellite features to third parties. OTP has an HTTP server but doesn't
> have proper Unicode support. OTP has OpenGL but doesn't have proper
> timezones support. These 2 examples missing from OTP are some of many
> things that are too complex for most people to implement, and that would
> not benefit from having alternatives.
>
> What is missing when it comes to timezones? Has it been mentioned in
> another thread?
> I think we rely on the OS configuration here.
Date/time conversion from one timezone to another. For example if your
data is saved with UTC time but the user is in Paris you want to convert
it to Europe/Paris.
Perhaps it exists, but I haven't found it.
I think there's a lot of work to be done with regards to localization
and internationalization support, but I haven't had enough time to think
about it in details yet. Unicode (not just Unicode though, but a native
string type abstracting encodings, EEP coming in a few weeks),
timezones, graphical drawing libraries are pretty much what I find lacks
the most for the web side of things. Graphical drawing libraries are
fine outside of OTP for a while (until there's one that can do near
everything and do it well) though.
Thanks for the info on package management and bug tracker. Hope we'll be
involved in the process of creating the former. And thanks for your nice
reply. :)
--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu
Concerning inets, what's really strange is that whenever we see some
benchmarks posted comparing performance of different HTTP
implementations, inets is never included there, though I would think, it
would make more sense in terms of showing how a custom implementation
compares to what's included in OTP. Does it have to do something with
what Loïc referred to as inets being well hidden in OTP? I doubt, but
if not that, then what? There are not even benchmarks comparing yaws to
inets, and despite the fact that both offer a rich set of functionality
make one wonder was was the idea of writing yaws in the first place
given existence of inets' httpd.
Diversity is good, but at some point it creates a burden for the user
who needs to make a selection of an HTTP server for his project.
Instead of solving the application-related problem, now the task becomes
studying several implementations to see which one offers a feature-set
best suited for the problem being solved, and the answer to that
question is not always straightforward.
Perhaps a good initiative from the OTP team would be to spec out the
open architecture of an HTTP server and welcome feedback from authors of
current http libraries, so that the successor of inets would be flexible
enough to accommodate building light-weight and heavy-weight HTTP
servers based on the supported code base included in OTP? This would
also open a possibility of diversity for library writers
replacing/enhancing parts to suit their needs.
Serge
The results are quite surprising - everyone who seems comfortable coding
in Erlang, feels compelled to implement an HTTP server.
This wouldn't have helped Cowboy, as one of the biggest difference is
the use of binary instead of strings.
There's another solution, which is to write wrappers for other APIs, for
example mochicow allows Mochiweb applications to use Cowboy under the
hood without having to change their code. This means that the switch can
be progressive instead of being brutal like a direct code switch.
> Diversity is good, but at some point it creates a burden for the user
> who needs to make a selection of an HTTP server for his project.
It doesn't!
> Instead of solving the application-related problem, now the task becomes
> studying several implementations to see which one offers a feature-set
> best suited for the problem being solved, and the answer to that
> question is not always straightforward.
Erlang is wonderful in that it allows you to write your application 100%
HTTP-free, and then write another application for the HTTP layer that
will be a thin layer running alongside your application. The
communication layer can be separate from the business logic.
If you go that way then your application's code isn't dependent on the
transports used at all, and you can easily switch servers, change
protocols, serialization formats or anything in your HTTP layer without
this impacting your application in any way.
For example, when SPDY support will be added to Cowboy, only the HTTP
layer will require a change to make use of it (start_http, which only
does HTTP, can be replaced by start_spdy, which does SPDY or HTTP
depending on client capabilities). The application behind it shouldn't
have to care about that.
> Perhaps a good initiative from the OTP team would be to spec out the
> open architecture of an HTTP server and welcome feedback from authors of
> current http libraries, so that the successor of inets would be flexible
> enough to accommodate building light-weight and heavy-weight HTTP
> servers based on the supported code base included in OTP? This would
> also open a possibility of diversity for library writers
> replacing/enhancing parts to suit their needs.
More importantly, do people actually want that?
--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu
> I couldn't disagree with that much more ... npm/Node.js is cluttered with
> packages of dubious quality that are not maintained in the slightest, and
> while this "Wild West" they prefer may seem exciting, I do not think it
> bodes well for the long-term stability of projects that depend on these
> packages.
>
Two ideas spring to mind immediately.
First, one should definitely not make the package repository a dump. You want to make package rules. Does it have documentation? Does it have a test suite? Can the test suite be run automatically? Semantic versioning? And so on. Is the package experimental, does it have real world uses?
Furthermore, we could definitely adopt the ideas from the haskell-platform: A set of well-defined and stable packages are made into a basis library. Packages from the dump "graduate".
Since you mentioned it. This thread is still quite far from the top
ones with my email being the 41'st one:
http://m.jakstys.lt/tech/2012/02/erlang-most-popular-subjects/
> But how has this not just ended with the understanding:
>
> a) Yaws for some, can be embedded or used like a stand alone web server and
> is known to be stable, fast and production ready
> b) Cowboy for http library in your app and you like bleeding edge and a
> smaller dependency -- and now even smaller in it self but requires ranch
> c) Ellis for those who don't care to support http
> d) Whatever makes you happy, maybe write your own!
>
> And in my opinion Cowboy REST is now a much better Webmachine than
> Webmachine, due to some changes around handling POST and a few other things.
At least on 0.6.0 cowboy had unexpectedly horrible performance issues
handling big POST requests, like discovered by my colleague[1]. Not
sure if this is now fixed. So *always* measure. Always.
[1]: https://github.com/yfyf/cowboy_bench
--
Motiejus Jakštys
It didn't, it just didn't support Expect: 100-continue.
It does in 0.6.1.
If there's still something wrong, please open a ticket. Nobody's
complaining at the moment.
--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu
Cheers,
DBM
> -----Original Message-----
> From: erlang-quest...@erlang.org [mailto:erlang-questions-
I have no idea how to build an Erlang project on Windows.
Patches are welcome!
I'll report back on my experiences so that I don't leave you hanging.
Cheers,
DBM
> -----Original Message-----
> From: Dave Cottlehuber [mailto:d...@jsonified.com]
> Sent: Friday, October 05, 2012 10:37
> To: David Mercer; Loïc Hoguin; Motiejus Jakštys; erlang-
> ques...@erlang.org
> Subject: Re: [erlang-questions] Erlang http servers
>
Sent from my iPad