On Tue, May 17, 2011 at 01:04:58PM +0200, Alexander Krasnukhin wrote: > Hej,
> Why this mailing list doesn't add itself to the 'reply-to' header? I see it > isn't uncommon for subscribers to send message to the author directly > instead of the list.
> I wonder why?
1) This is an old mailing list. It has been around since 1997, which is long before Google et.al started to violate then already accepted mailing list behaviours probably with the intention to unify forum like behaviour with mailing lists. It has always been like this, expect for a short while when I changed mailing list software and accidentally configured the behaviour you ask and the list exploded with dislike. The existing subscribers expect this behaviour.
2) It is according to Internet standards.
3) It lowers the probability of sending privately intended mails to the list.
One might argue that it is not proven that the majority of the currently existing subscribers actually want it this way. That is correct, but it is just a few new subscribers asking this question getting little positive response from the existing subscribers. And we all have other things to do than to dig up already worn out arguments in this futile debate.
If a large number of old reliable subscribers start voicing the opinion that this Reply-To setting is now a bad idea we might change the setting. Such a controversial change would require a _large_ majority.
I anyone tries to start a flamewar about changing the Reply-To setting it will be regarded as off-topic and nonproductive and they will end up ignored or banned.
On Tue, May 17, 2011 at 05:07:50PM +0200, Alexander Krasnukhin wrote: > Yes, yes. I've got this. People from telecom will always rely on standards > instead of people expectations. Good. Right. Understandable.
There are standards and often conflicting standards. We have already made a delibirate choiche of which standards we believe in.
There are different people expecting different things. Newcomers to a list should not expect that all oldtimers expect the same as themselves.
If this list behaviour feels differently than other list it depends which lists you regard as the other. Many lists today are really Google groups, and they seem to have chosen to violate Internet standards since they are more like a forum, and in forums private messages do not exist.
The default setting for both Majordomo and Mailman is to not munge Reply-To, and their documentation discourages changing this. All mailing lists at e.g openbsd.org do not munge Reply-To.
I'd disagree with you on that. I receive an email from the list. I send my emails to the list. It's one sender and one recepient. So I expect Reply-To to send my email back to the list.
Moreover, if you hit "Reply All", you'll end up sending multiple emails to seemingly random people:
Right now this email will be sent To: Richard O'Keefe, CC: Alexandr Krasnukhin, Erlang Questions. Both you and Alexandr will end up with two copies of my letter (one sent directly from me and one sent from the mailing list). This is clearly not the behaviour I'd want when I want to reply to a message from the mailing list.
The number of recepients varies depending on which email you reply to, but the idea is the same. Instead of sending one email to one recepient (the list) I end up spamming people :)
So what do I have do now? I have to delete all the extra emails and move Erlang Questions to the "To:" field. This way I increase the chance that the recepient(s) will also reply to the list and not to me directly.
> On 18/05/2011, at 3:07 AM, Alexander Krasnukhin wrote:
>> Yes, yes. I've got this. People from telecom will always rely on standards instead of people expectations. Good. Right. Understandable.
> That sounds a bit sarcastic. > The thing is that the Erlang mailing list behaves *EXACTLY* the way I expect > a mailing list to work. > I expect "Reply All" to reply to everyone on the list. > I expect "Reply" to go just to the author. > I expect it to work the way mailing lists always used to work.
> The standard in this case is not arbitrary, but part of a coherent > design to ensure a straightforward user experience.
raimo+erlang-questi...@erix.ericsson.se> wrote: > On Tue, May 17, 2011 at 05:07:50PM +0200, Alexander Krasnukhin wrote: > > Yes, yes. I've got this. People from telecom will always rely on > standards > > instead of people expectations. Good. Right. Understandable.
> There are standards and often conflicting standards. We have already > made a delibirate choiche of which standards we believe in.
> There are different people expecting different things. > Newcomers to a list should not expect that all oldtimers > expect the same as themselves.
> If this list behaviour feels differently than other list it > depends which lists you regard as the other. Many lists > today are really Google groups, and they seem to have chosen > to violate Internet standards since they are more like a > forum, and in forums private messages do not exist.
> The default setting for both Majordomo and Mailman is to > not munge Reply-To, and their documentation discourages > changing this. All mailing lists at e.g openbsd.org > do not munge Reply-To.
How about we completely side-step the whole issue and re-structure the various erlang mailing lists a forum instead? And by choosing one of the common and free forum packages we will automatically get a standard (in the sense of being common and much used) behaviour.
> Seriously, I do understand your point. It wasn't a sarcasm or a joke.
> On Wed, May 18, 2011 at 2:30 PM, Raimo Niskanen < raimo+erlang-questi...@erix.ericsson.se > wrote:
> On Tue, May 17, 2011 at 05:07:50PM +0200, Alexander Krasnukhin wrote: > > Yes, yes. I've got this. People from telecom will always rely on standards > > instead of people expectations. Good. Right. Understandable.
> There are standards and often conflicting standards. We have already > made a delibirate choiche of which standards we believe in.
> There are different people expecting different things. > Newcomers to a list should not expect that all oldtimers > expect the same as themselves.
> If this list behaviour feels differently than other list it > depends which lists you regard as the other. Many lists > today are really Google groups, and they seem to have chosen > to violate Internet standards since they are more like a > forum, and in forums private messages do not exist.
> The default setting for both Majordomo and Mailman is to > not munge Reply-To, and their documentation discourages > changing this. All mailing lists at e.g openbsd.org > do not munge Reply-To.
> How about we completely side-step the whole issue and re-structure the > various erlang mailing lists a forum instead? And by choosing one of the > common and free forum packages we will automatically get a standard (in the > sense of being common and much used) behaviour.
I really dislike forums. it's very easy for me to rapidly read and reply to any topics on a mailing list, but with a forum I have to 1) visit a particular page 2) manually click around. I much prefer having things in an email client which is optimized for the job.
The only advantage to forums in my opinion is that the archives tend to be much better than mailing list archives.
> I really dislike forums. it's very easy for me to rapidly read and > reply to any topics on a mailing list, but with a forum I have to 1) > visit a particular page 2) manually click around. I much prefer having > things in an email client which is optimized for the job.
+1
-- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development
"Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ _______________________________________________ erlang-questions mailing list erlang-questi...@erlang.org http://erlang.org/mailman/listinfo/erlang-questions
Um, isn't it client specific and not list specific? Every list I've been a part of has a broad reply. So if I send a reply to erlang-questi...@erlang.org with the "Re: [erlang-questions] 'reply-to' header in this mailing list" subject it goes to everyone on that list.
If I do a wide reply (Emacs: Shft-f) it will send the reply message to the specific person I'm replying to /and/ it will send it to the list (as is happening with this email).
I can also send just the reply to the list, or just a reply to the specific person. They are just different commands.
As it stands, the mailing list does exactly what I expect it to. Maybe your software is what's not living up to your familiarity/intuitive standards?
Michael Turner <michael.eugene.tur...@gmail.com> writes: > Does anyone have any statistics on this question? That is, what > percentage of mailing lists "do it wrong"? It's counterintuitive to me > that a "reply" on this mailing list is only to the individual. "Reply" > is "Reply to the list" on every other mailing list I'm currently on, > and on almost every mailing list I can remember being on. But perhaps > my lifetime mailing list membership doesn't approach statistical > significance.
> Although it may get me accused of "blindly following the herd," let me > assert it anyway: arguments from "intuition" in commonly used software > interfaces only work when you have statistically significant user > support for them, not some purely formal, rule-based argument for your > *personal* intuition. What's "intuitive" to one person may be > counterintuitive to many. As pointed out long ago, "intuitive equals > familiar":
> And, as pointed out even longer ago, "a foolish consistency is the > hobgoblin of small minds." (Emerson.)
> Perhaps to some people, a mailing list is like a noticeboard in a > mostly-empty hallway. My mental model of a mailing list corresponds > more closely to a conversation in a classroom. When you reply to an > open question or comment in such a context, you cannot help but be > heard by more than one person; most likely you'll be heard by everyone > in the room. You actually have to make a special effort (i.e., lean > over and whisper in an ear, or pass a note) to be sure that your reply > is private. I think this corresponds pretty closely to the intuition of > the average mailing list user. But erlangeurs may be different, I don't > know.
> -michael turner
> On Wed, May 18, 2011 at 8:45 AM, Richard O'Keefe <o...@cs.otago.ac.nz> > wrote:
> On 18/05/2011, at 3:07 AM, Alexander Krasnukhin wrote:
> > Yes, yes. I've got this. People from telecom will always rely on > standards instead of people expectations. Good. Right. > Understandable.
> That sounds a bit sarcastic. > The thing is that the Erlang mailing list behaves *EXACTLY* the way > I expect > a mailing list to work. > I expect "Reply All" to reply to everyone on the list. > I expect "Reply" to go just to the author. > I expect it to work the way mailing lists always used to work.
> The standard in this case is not arbitrary, but part of a coherent > design to ensure a straightforward user experience.
On Wed, May 18, 2011 at 16:18, Ivan Uemlianin <i...@llaisdy.com> wrote: > On 18/05/2011 15:11, Jack Moffitt wrote:
>> I really dislike forums. it's very easy for me to rapidly read and >> reply to any topics on a mailing list, but with a forum I have to 1) >> visit a particular page 2) manually click around. I much prefer having >> things in an email client which is optimized for the job.
On Wed, May 18, 2011 at 08:11:17AM -0600, Jack Moffitt wrote: > > How about we completely side-step the whole issue and re-structure the > > various erlang mailing lists a forum instead? And by choosing one of the > > common and free forum packages we will automatically get a standard (in the > > sense of being common and much used) behaviour.
> I really dislike forums. it's very easy for me to rapidly read and > reply to any topics on a mailing list, but with a forum I have to 1) > visit a particular page 2) manually click around. I much prefer having > things in an email client which is optimized for the job.
> The only advantage to forums in my opinion is that the archives tend > to be much better than mailing list archives.
The archive is a plus for a forum.
There does not seem to be any religious user fractions disaggreing on how forums should behave. That is also a plus.
Free forum packages often means PHP, MySQL or other sysadm hazzles. They are more complicated than a mailing list. That is a minus for a forum (from my point of view). There is of course other party forum hosting, but they we loose control of the forum archives...
On Wed, May 18, 2011 at 02:06:37PM +0000, Robert Virding wrote: > How about we completely side-step the whole issue and re-structure > the various erlang mailing lists a forum instead? And by choosing one > of the common and free forum packages we will automatically get a > standard (in the sense of being common and much used) behaviour. > This is also a serious suggestion.
-1. The mailing list works well the way it is.
(As far as my, personal, workflow is concerned, forum software makes it hard work for me to collect the material I will want to look at again and catalogue it to suit my needs. I suppose businesses like Google prefer that because they can serve me new ads each time I look for something I remember seeing last year that might be helpful to me right now. I see that as their problem, not mine.)
From: erlang-questions-boun...@erlang.org [mailto:erlang-questions-boun...@erlang.org] On Behalf Of Robert Virding Sent: Wednesday, May 18, 2011 9:07 AM To: Alexander Krasnukhin Cc: erlang-questi...@erlang.org Subject: Re: [erlang-questions] 'reply-to' header in this mailing list
How about we completely side-step the whole issue and re-structure the various erlang mailing lists a forum instead? And by choosing one of the common and free forum packages we will automatically get a standard (in the sense of being common and much used) behaviour.
This is also a serious suggestion.
[DBM] What’s the difference between a forum and a mailing list? I tend to think of forums as being something like Usenet, but I haven’t used that in years. Some people seem to think they’re like Google Groups, which I don’t really use that much. Can someone please elucidate the distinction between a mailing list and a forum?
On Thu, May 19, 2011 at 11:35:14AM -0500, David Mercer wrote: } Can someone please elucidate the distinction between a mailing } list and a forum?
I have come to the conclusion that what is beneath these sort of debates amounts to a generational gap. Those who grew up with a mouse in their hands have different expectations than those of us who grew up with our hands planted on the keyboard looking at a command prompt. As one of the later my view is that the distinction is:
> On Thu, May 19, 2011 at 11:35:14AM -0500, David Mercer wrote: > } Can someone please elucidate the distinction between a mailing > } list and a forum?
> I have come to the conclusion that what is beneath these sort > of debates amounts to a generational gap. Those who grew up > with a mouse in their hands have different expectations than > those of us who grew up with our hands planted on the keyboard > looking at a command prompt. As one of the later my view is > that the distinction is:
I should like to point out that the *proposal* to change the currently very sensible behaviour of this list has *already* cost me far more time than any change would credibly save me. When you add up all the other people who've been following it, the waste of time is quite scary.
IF YOU DON'T LIKE IT, LEARN TO USE THE "REPLY ALL" BUTTON ALL THE TIME.
> On Tue, May 17, 2011 at 5:25 PM, Todd <t.greenwoodg...@gmail.com> wrote: > 4b. Side note: is anyone concerned about Akka/Typesafe stealing mindshare?
Having read Williamson's "Legion of Space" series, I would feel rather unhappy using AKKA (:-).
> I should like to point out that the *proposal* to change the > currently very sensible behaviour of this list has *already* > cost me far more time than any change would credibly save me. > When you add up all the other people who've been following > it, the waste of time is quite scary.
> IF YOU DON'T LIKE IT, LEARN TO USE THE "REPLY ALL" BUTTON > ALL THE TIME.
>> On Tue, May 17, 2011 at 5:25 PM, Todd <t.greenwoodg...@gmail.com> wrote: >> 4b. Side note: is anyone concerned about Akka/Typesafe stealing mindshare? > Having read Williamson's "Legion of Space" series, I would feel rather > unhappy using AKKA (:-).
Even if these and other libraries (jetlang is the basis for akka, kilim which came before it, in the Java world, etc.) support message passing based scheduling of light-weight processes in a way that facilitates creating Actors, the libraries are unlikely to perform as well as Erlang because: - The amount of research/development that has gone into Erlang - The process-based garbage collection which can not be found through a message passing library
Ok, even if the message based scheduling of light-weight processes is as efficient (comparable) to Erlang (ignoring garbage collection problems), a library will not give you code-level fault-tolerance (unless you go through a lot of painful development with custom code that lacks testing and increases the complexity of your codebase). So, I do not think there will be an Erlang replacement anytime soon. _______________________________________________ erlang-questions mailing list erlang-questi...@erlang.org http://erlang.org/mailman/listinfo/erlang-questions
On Thu, May 19, 2011 at 06:01:31PM -0600, OvermindDL1 wrote: > On May 19, 2011 10:57 AM, "Vance Shipley" <van...@motivity.ca> wrote:
> > On Thu, May 19, 2011 at 11:35:14AM -0500, David Mercer wrote: > > } Can someone please elucidate the distinction between a mailing > > } list and a forum?
> > I have come to the conclusion that what is beneath these sort > > of debates amounts to a generational gap. Those who grew up > > with a mouse in their hands have different expectations than > > those of us who grew up with our hands planted on the keyboard > > looking at a command prompt. As one of the later my view is > > that the distinction is:
> Why can it not just be configurable on a person by person basis by editing > their own user preferences on the mailing list server?
> Certainly the mailing list software is not so poorly designed so as to > require such a person specific option to be set only globally. ;-)
Oh yes it is, I hope "poorly designed" was the reason for the ";-)".
I have run some mailing lists and only looked at running forums, but they are different software packages. Popular mailing list softwares are e.g "Mailman" or "Majordomo", forums "phpBB", "UseBB", "WWWBoard", "YaBB", etc.
A mailing list is centered around the mails. Users posts by sending a mail to the list, the software distributes the mail to all subscirbers and possibly keeps an archive of all posts. Often the origin address of the mail is used to verify that a mail is valid to post. And that is about it.
A forum is centered about the archive/bulletin board. Users have to log on, there has to be a web server handling the user session, the post is written through a web interface, users have more settings e.g an Avatar and there has to be permission handling of which user is allowed to post to which forum section. There has to be moderators that can completely delete threads that are inappropriate (in mailing lists changing the archive is rare done since all subscribers know all posts anyway). You can often get a mail when a thread you are "watching" is updated but you are supposed to read through the web interface. Forums can have mailing list extensions so you can post by sending a mail, possibly also get the posts as mails. I have been on forums where this appeared to be upcoming features but were not enabled...
So, mailing list software is designed so that the forum model is not on the map at all except keeping an archive. Forum software may be designed to act as mailing lists, but I do not know how well they do it.
> On 20 May 2011 05:01, Richard O'Keefe <o...@cs.otago.ac.nz> wrote:
> > I should like to point out that the *proposal* to change the > > currently very sensible behaviour of this list has *already* > > cost me far more time than any change would credibly save me. > > When you add up all the other people who've been following > > it, the waste of time is quite scary.
> > IF YOU DON'T LIKE IT, LEARN TO USE THE "REPLY ALL" BUTTON > > ALL THE TIME.
The Ruby on Rails lists have both a web-based forum and forum-to-email gateway - users can choose to use either interface to create and respond to questions and responses made using either interface are seen on both interfaces as well.
Max
On 5/20/11, Raimo Niskanen <raimo+erlang-questi...@erix.ericsson.se> wrote:
> On Fri, May 20, 2011 at 08:35:28AM +0200, Mazen Harake wrote: >> First thing I said heh. no one listened... :(
>> Looking forward to having the same discussion in a few months time again >> :P
> Oh yes. Made me even consider putting 'reply-to' in the spam filter ;-)
>> On 20 May 2011 05:01, Richard O'Keefe <o...@cs.otago.ac.nz> wrote:
>> > I should like to point out that the *proposal* to change the >> > currently very sensible behaviour of this list has *already* >> > cost me far more time than any change would credibly save me. >> > When you add up all the other people who've been following >> > it, the waste of time is quite scary.
>> > IF YOU DON'T LIKE IT, LEARN TO USE THE "REPLY ALL" BUTTON >> > ALL THE TIME.
On Fri, May 20, 2011 at 2:59 AM, Michael Truog <mjtr...@gmail.com> wrote: > On 05/19/2011 11:10 PM, Richard O'Keefe wrote: >>> On Tue, May 17, 2011 at 5:25 PM, Todd <t.greenwoodg...@gmail.com> wrote: >>> 4b. Side note: is anyone concerned about Akka/Typesafe stealing mindshare? >> Having read Williamson's "Legion of Space" series, I would feel rather >> unhappy using AKKA (:-).
> Even if these and other libraries (jetlang is the basis for akka, kilim which came before it, in the Java world, etc.) support message passing based scheduling of light-weight processes in a way that facilitates creating Actors, the libraries are unlikely to perform as well as Erlang because: > - The amount of research/development that has gone into Erlang > - The process-based garbage collection which can not be found through a message passing library
> Ok, even if the message based scheduling of light-weight processes is as efficient (comparable) to Erlang (ignoring garbage collection problems), a library will not give you code-level fault-tolerance (unless you go through a lot of painful development with custom code that lacks testing and increases the complexity of your codebase). So, I do not think there will be an Erlang replacement anytime soon. > _______________________________________________
If anything Akka will expose more Java folks to Erlang concepts and potentially draw people looking for a way out of neo-COBOL. _______________________________________________ erlang-questions mailing list erlang-questi...@erlang.org http://erlang.org/mailman/listinfo/erlang-questions
On Wed, May 18, 2011 at 02:25, Todd <t.greenwoodg...@gmail.com> wrote:
Let me be quite blunt here...
> 1. In general, what are the most valuable libraries to learn, both within > the Erlang dist and external?
> 2. Is there a consolidated/curated repository of libraries that is industry > standard? I know the erlware folks have a repo...is that both a complete and > accepted authoritative repo? From reading the list, it sounds like there's > also a fair bit of stuff scattered about in github, too.
I think this approach to Erlang is wrong. Rather than ask for a set of "standard" modules to look into you should attack it on a on-demand basis when you find a need for a specific library. Personally, I really like the Agner system which aims to be a system listing available software so you can use it. It intermingles with the Rebar build system in a neat way.
Erlang seeks to provide tools, not solutions. As such, you will find a lot of tools in the OTP distribution and elsewhere which will give you stuff to write your own solutions. But you won't find any prebaked solutions which magically solves the problem you are looking at.
> 3. How does one easily multithread an app? For instance, there's pmap in > clojure and something similar in akka that lets you map a function across a > list, and it allocates threads accordingly...
There is no easy way to multi-thread an app so it gives good speedup when adding more cores. There are some general guidelines you can follow when writing the program, but they do not always yield a speedup. Here is a simple module:
-module(foo). -compile(export_all).
m(X) -> X*2.
test_input() -> lists:seq(1, 10000).
t1(L) -> timer:tc(fun() -> [m(X) || X <- L] end).
t2(L) -> timer:tc(rpc, pmap, [{foo, m}, [], L]).
where t1/1 and t2/1 are our tests. t1 uses a list comprehension and t2 uses the pmap function of the rpc module to execute in parallel on my two cores. A simple experiment in the shell:
Eshell V5.8.4 (abort with ^G) 1> c(foo). {ok,foo} 2> L = test_input(). ** exception error: undefined shell command test_input/0 3> L = foo:test_input(). [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, 23,24,25,26,27,28,29|...] 4> X = foo:t1(L). {2491, [2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42, 44,46,48,50,52,54|...]} 5> Y = foo:t2(L). {65111, [2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42, 44,46,48,50,52,54|...]} 6>
shows how t1 is much much faster than t2. You need to know a lot about the problem at hand to make it faster. If your m/1 function is altered to this:
m(X) -> timer:sleep(3), X*2.
so we in the parallel example can do other work in between, then the numbers are different:
in much in favor of t2. Hopefully this shows you need to know about your problem to make it go faster. There is no magical solution.
It is also important to note that in Erlang, concurrency was added to build fault tolerant programs. Not to make programs run faster. It is neat that it is often the case that it helps on multi-core machines, but it was not the initial goal.
> 4. Along that note, does anyone have any ideas as to how to tackle the > Typesafe 'getting started tutorial?'
calc_pi_worker({Start, N}) -> lists:sum( [4.0 * (1 - (I rem 2) * 2) / (2 * I + 1) || I <- lists:seq(Start, Start+N-1)]).
calc_pi() -> K = 10000, lists:sum([calc_pi_worker({I*K, K}) || I <- lists:seq(0,K-1)]).
if we run it sequentially it gives the correct result, but is fairly slow since we generate a lot of data that subsequently becomes garbage. calc_pi_worker/1 can be optimized by moving stuff out of the main loop and not building the list with lists:seq/2 each time around. calc_pi/0 can be optimized by using the plists library (https://github.com/eveel/plists). So while we are at it, let us parallelize. And let us not create all that boiler plate while doing it! Here is the code:
-module(foo). -compile(export_all).
calc_pi_worker({Start, N}) -> calc_pi_worker(Start, N, 0).
calc_pi_worker(_I, 0, Acc) -> Acc; calc_pi_worker(I, K, Acc) -> S = (1 - (I rem 2) * 2) / (2 * I + 1), calc_pi_worker(I+1, K-1, Acc + S).
calc_pi() -> K = 10000, 4.0 * lists:sum(plists:map(fun foo:calc_pi_worker/1, [{I*K, K} || I <- lists:seq(0, K-1)])).
Yes, we don't have to change anything else. This is parallel using as many cores as you have. It can be tuned some more, but for a start it is magnificent, even though it is slower than the Akka-version.
> 4b. Side note: is anyone concerned about Akka/Typesafe stealing mindshare?
Well, if you want to write all the boiler plate code they have to write, then go ahead. I'd rather not :)
Seriously though, the mindshare we will steal are from Non-concurrent languages, be it Python, Ruby, Java, C# or ... -- We are in the same boat as Scala/Akka, Go and so on. The influx of interested programmers will be large, so we don't have to worry too much about who steals from whom. Erlang has the distinct advantage of being old, tried, battle-hardened and extremely robust. It has a main focus on fault tolerance, which gives it some unique capabilities. Also, its focus on using functional programming is a robustness advantage. You can't just copy part of Erlang to obtain what it provides. You have to have all of it and then some.
> 5. How does one push an app such that it self instantiates it's processes > across the cluster? I can see how OTP is great at managing an app on a > single node, but how do you say something like: "create one of these > processes on each node in the cluster, and restart 1-for-1 if they die"... > or something similar. I see mention of gproc, but honestly, I don't see how > to use it. Likewise, if nodes are added to the cluster, how would you ensure > that the necessary processes are pushed to the new node after it joins the > cluster?
Essentially this is handled by the application if it is written to adhere to some rules of Erlang and if you write application such that it does not assume all of it is present locally on a single VM, but is distributed across multiple. There is no automatic solution here either. For each application you will need to define what to do. For some applications a 'takeover' is enough. There is one running and if the node it runs on crashes, then another node will take over the job and arrange that requests are now redirected to the new node. For other systems, like Riak, all nodes are simultaneously runing and the application on each node talks to the same application on other nodes to internally manage state.
Yes, it is a hard problem. But Erlang provides you with the tools to solve it.
> 6. How do you deploy and live code upgrade in real life? I've been looking > at some of the work by the 'Dukes of Erl' ... is erlrc what folks commononly > use?
I don't. The projects I am working on has the virtue that we can do rolling upgrades by closing down machines and restart them. You will have to ask someone else :)
> 7. Does anyone use dynamic load balancing of demand across a cluster (e.g. > spinning up erlang processes to meet the demand curve?)
I am sure there are people who does this. But I'll let them answer the question. It is not that hard to pull off.
> 8. What's the best way to integrate w/ other code bases. In akka, you'd use > camel as an integration bus. What are the common ways to integrate with > erlang? Is that what ports and nifs are for? Forgive my ignorance, but I > always considered those as simply ways to write code in a different, perhaps > more comfortable language...not as integration mechanisms.
Integration is perhaps Erlangs strength. NIFs are for writing small hotspots in your code in C for speed. The calc_pi_worker/1 from above comes to mind for instance. Ports are used for several things. They are a representation in the Erlang VM of something external. It can be another process, where we have a pipe for communication. It can be a file on disk. It can be a network socket. Or it can be a linked in driver. Another option is to make a node in another language that talks the Erlang distribution protocol. Yet another option is to use a message queue like ZeroMQ or AQMP for communication. Finally, you can do like my own project and simply implement the foreign protocol in Erlang - BitTorrent in my case. Erlang rocks for implementing foreign protocols.
> Also, I've continued to peck away at various newbie tutorials. Any > comments/suggestions/corrections are welcome.
Jesper, thanks for the response. I've been a bit remiss in thanking folks for their responses to my lengthy questions...I'm a bit pressed for time as I'm packing in preparation for moving.
> On Wed, May 18, 2011 at 02:25, Todd<t.greenwoodg...@gmail.com> wrote:
> Let me be quite blunt here...
>> 1. In general, what are the most valuable libraries to learn, both within >> the Erlang dist and external?
>> 2. Is there a consolidated/curated repository of libraries that is industry >> standard? I know the erlware folks have a repo...is that both a complete and >> accepted authoritative repo? From reading the list, it sounds like there's >> also a fair bit of stuff scattered about in github, too.
> I think this approach to Erlang is wrong. Rather than ask for a set of > "standard" modules to look into you should attack it on a on-demand > basis when you find a need for a specific library. Personally, I > really like the Agner system which aims to be a system listing > available software so you can use it. It intermingles with the Rebar > build system in a neat way.
Thanks for mentioning Agner, that looks cool. I'll noodle around and figure out how it links with rebar.
For those of you that have been programming in erlang for some time, it would be useful for those of us, newer to erlang, to understand what libraries are your bread-and-butter, as opposed to domain specific libraries. As an example, if someone were to ask me this about Java, I'd suggest...understand threading semantics, java.util.collections, immutable collections (guava), IOC...and JUnit. The rest would be specific to the problem domain (e.g. Spring Framework, hibernate|JDBC, EJB3, and other more specific libraries. )
> Erlang seeks to provide tools, not solutions. As such, you will find a > lot of tools in the OTP distribution and elsewhere which will give you > stuff to write your own solutions. But you won't find any prebaked > solutions which magically solves the problem you are looking at.
>> 3. How does one easily multithread an app? For instance, there's pmap in >> clojure and something similar in akka that lets you map a function across a >> list, and it allocates threads accordingly...
> There is no easy way to multi-thread an app so it gives good speedup > when adding more cores. There are some general guidelines you can > follow when writing the program, but they do not always yield a > speedup. Here is a simple module:
> where t1/1 and t2/1 are our tests. t1 uses a list comprehension and t2 > uses the pmap function of the rpc module to execute in parallel on my > two cores. A simple experiment in the shell:
> shows how t1 is much much faster than t2. You need to know a lot about > the problem at hand to make it faster. If your m/1 function is altered > to this:
> m(X) -> > timer:sleep(3), > X*2.
> so we in the parallel example can do other work in between, then the > numbers are different:
> in much in favor of t2. Hopefully this shows you need to know about > your problem to make it go faster. There is no magical solution.
I'm pretty familiar with this sort of thing in other languages, but it's nice to see such a clear example. For me, the missing piece was the rpc module that contains the pmap function. I'll have to look closely at that module.
> It is also important to note that in Erlang, concurrency was added to > build fault tolerant programs. Not to make programs run faster. It is > neat that it is often the case that it helps on multi-core machines, > but it was not the initial goal.
>> 4. Along that note, does anyone have any ideas as to how to tackle the >> Typesafe 'getting started tutorial?'
> if we run it sequentially it gives the correct result, but is fairly > slow since we generate a lot of data that subsequently becomes > garbage. calc_pi_worker/1 can be optimized by moving stuff out of the > main loop and not building the list with lists:seq/2 each time around. > calc_pi/0 can be optimized by using the plists library > (https://github.com/eveel/plists). So while we are at it, let us > parallelize. And let us not create all that boiler plate while doing > it! Here is the code:
> -module(foo). > -compile(export_all).
> calc_pi_worker({Start, N}) -> > calc_pi_worker(Start, N, 0).
> calc_pi_worker(_I, 0, Acc) -> Acc; > calc_pi_worker(I, K, Acc) -> > S = (1 - (I rem 2) * 2) / (2 * I + 1), > calc_pi_worker(I+1, K-1, Acc + S).
> Yes, we don't have to change anything else. This is parallel using as > many cores as you have. It can be tuned some more, but for a start it > is magnificent, even though it is slower than the Akka-version.
Wow. That's impressive. I'll take a closer look at the plists module, too. In a way, that's what I was talking about earlier in this email w/ respect to bread-and-butter libs. You pulled plists out of your back pocket...what else have you got there?
>> 4b. Side note: is anyone concerned about Akka/Typesafe stealing mindshare?
> Well, if you want to write all the boiler plate code they have to > write, then go ahead. I'd rather not :)
> Seriously though, the mindshare we will steal are from Non-concurrent > languages, be it Python, Ruby, Java, C# or ... -- We are in the same > boat as Scala/Akka, Go and so on. The influx of interested programmers > will be large, so we don't have to worry too much about who steals > from whom. Erlang has the distinct advantage of being old, tried, > battle-hardened and extremely robust. It has a main focus on fault > tolerance, which gives it some unique capabilities. Also, its focus on > using functional programming is a robustness advantage. You can't just > copy part of Erlang to obtain what it provides. You have to have all > of it and then some.
>> 5. How does one push an app such that it self instantiates it's processes >> across the cluster? I can see how OTP is great at managing an app on a >> single node, but how do you say something like: "create one of these >> processes on each node in the cluster, and restart 1-for-1 if they die"... >> or something similar. I see mention of gproc, but honestly, I don't see how >> to use it. Likewise, if nodes are added to the cluster, how would you ensure >> that the necessary processes are pushed to the new node after it joins the >> cluster?
> Essentially this is handled by the application if it is written to > adhere to some rules of Erlang and if you write application such that > it does not assume all of it is present locally on a single VM, but is > distributed across multiple. There is no automatic solution here > either. For each application you will need to define what to do. For > some applications a 'takeover' is enough. There is one running and if > the node it runs on crashes, then another node will take over the job > and arrange that requests are now redirected to the new node. For > other systems, like Riak, all nodes are simultaneously runing and the > application on each node talks to the same application on other nodes > to internally manage state.
> Yes, it is a hard problem. But Erlang provides you with the tools to solve it.
I have to admit, this is something that really intrigues me... creating an app that can move around in a cluster as nodes go up or down...as well as responding to load and spinning up new processes on nodes to handle the load. One thought I've had is to use erlang to manage os processes to spin up new erlang nodes on remote machines. Or to have erlang spin up an external resource, like a database or message queue and link to it.
>> 6. How do you deploy and live code upgrade in real life? I've been looking >> at some of the work by the 'Dukes of Erl' ... is erlrc what folks commononly >> use?
> I don't. The projects I am working on has the virtue that we can do > rolling upgrades by closing down machines and restart them. You will > have to ask someone else :)
How do you insure that, as you roll a cluster from v1 to v2, the new v2 nodes don't corrupt the data that v1 is using?
On all the large projects I've been involved with (java,jdbc,mysql,etc.), there are typically service level changes that are tightly coupled to sql ddl changes... thereby requiring downing the entire cluster, applying the ddl deltas to the dbs, and
