[erlang-questions] The gen server simplified (how it works)
Joe Armstrong
and often force their problems to fit the gen_sever even though the
gen_server is not appropriate for their problems.
The gen_server is an extremely simple bit of code
which can be easily changed to fit other problems, though people don't
often do this.
In this posting I'll explain the basic idea of how the gen_server works.
To illustrate this I've written mini_gs.erl - this is a mini gen_server
if you understand mini_gs.erl you'll have understood 98% of how the real
gen_server works. The real gen_server just adds a load of "bells and whistles"
to mini_gs.erl.
mini_gs.erl has a compatible interface to gen_server.erl (for a subset of
the gen_server API)
-module(mini_gs).
-export([start_link/4, call/2]).
%% this module behaves just like the gen-server for a sub-set of the gen_server
%% commands
start_link({local,Name}, Mod, Args, _Opts) ->
register(Name, spawn(fun() -> start(Mod, Args) end)).
call(Name, X) ->
Name ! {self(), Ref = make_ref(), X},
receive
{Ref, Reply} -> Reply
end.
start(Mod, Args) ->
{ok, State} = Mod:init(Args),
loop(Mod, State).
loop(Mod, State) ->
receive
{From, Tag, X} ->
case Mod:handle_call(X, From, State) of
{reply, R, State1} ->
From ! {Tag, R},
loop(Mod, State1)
end
end.
There. That wasn't so painful. The client starts by calling
min_gs:start_link({local,Name}, Mod, Args, Opts)
I've ignored Opts in mini_gs, also frozen the name of the server to be of the form
{local, Name} (gen_server has more general arguments for the name of the server)
What happens?
mini_gs calls Mod:init(Args) to initialize the server, this must return {ok, State}
and State becomes the initial state of the server.
Now mini_gs calls loop(Mod, State.) When mini_gs receives a message {From, Tag, X}
it calls Mod:handle_call(X, From, State) which returns {repy, R, State1}. R1 is
sent back to the client, and the server calls loop/2 with the new state State1.
That's it. call/2 is an interface routine to abstract out the interface between the
client and the sever.
Now we can write a simple client application.
-module(kv).
%% These define the client API
-export([start/0, store/2,lookup/1]).
%% these must be defined because they are called by gs
-export([init/1, handle_call/3]).
-define(GS, mini_gs).
%% -define(GS, gen_server).
%% define the client API
start() -> ?GS:start_link({local,someatom}, kv, foo, []).
store(Key,Val) -> ?GS:call(someatom, {putval,Key,Val}).
lookup(Key) -> ?GS:call(someatom, {getval,Key}).
%% define the internal routines
init(foo) -> {ok, dict:new()}.
handle_call({putval, Key, Val}, _From, Dict) ->
{reply, ok, dict:store(Key, Val, Dict)};
handle_call({getval,Key}, _From, Dict) ->
{reply, dict:find(Key, Dict), Dict}.
This module can call either gen_server or mini_gs (just change the define statement)
So now we have turned a single process key-value store (using dict) into a global
key-value store. Note that kv.erl never uses the primitives spawn_link, send, receive
or do on. ie kv.erl is written with pure *sequential* code.
This is *why* we made the gen_server abstraction. You can write well-typed
sequential code (the handle_call and init functions) to parametrize
a concurrent behavior, ie you need to know nothing about concurrency to
get the job done. We've "abstracted out" the concurrency.
Things become problematic when you do not entirely understand the abstraction.
Maybe the abstraction is inappropriate for your needs. I have seen many examples
of code where the gen_server *is* inappropriate. The acid test is "does the
gen_sever code look like spaghetti" if the answer is yes then all you have done
is shoe horn the applications into an inappropriate form. In this case
you should ditch the gen_server and roll-your own.
/Joe
Joe Armstrong
> Things become problematic when you do not entirely understand the
> abstraction.
> Maybe the abstraction is inappropriate for your needs. I have seen many
> examples
> of code where the gen_server *is* inappropriate. The acid test is "does the
> gen_sever code look like spaghetti" if the answer is yes then all you have
> done
> is shoe horn the applications into an inappropriate form. In this case
> you should ditch the gen_server and roll-your own.
Another problem I often see is that newbies tend to regard gen_server
as something that does magic, and fail to understand how the code runs
in the server and the clients (since that code lives in the same
module, usually).
They end up in solutions with parts of the gen server sending messages
back and forth, putting receives that spoil the abstraction
intercepting messages that gen_server code should be handling,
throwing exits in server code and attempting to catch them in the
client code, blocking servers with heavyweight operations, ...
I really consider advising against using gen_server (or any other
abstraction) until one is able to write a client-server and a
supervisor tree without them so that you can understand what's
happening under the hood of gen_server (which, as you say, it's not
rocket science).
Absolutely.
I think the problem stems from the use of IDEs - in eclipse/Xcode etc you click on
a "new project" button and the system automatically generates a load of crap files
full of incomprehensible undocumented nonsense. This leads to a mentality
of "using stuff without understanding it" - All I ever use is emacs, make and a shell.
Usually I put all the files in one directory. I want to understand *everything*
As IDEs get more and more complex and help you more and more
it gets harder and harder to understand what's really going on.
Graphics programming used to be easy - anybody remember Borlands turbo graphics?
- but now it's far more difficult since you have to fight with all the tools and
frameworks that get in the way and make life difficult.
/joe
Cheers
--
Samuel
Attila Rajmund Nohl
> On Wed, Apr 20, 2011 at 10:59 AM, Samuel Rivas
> <samuel...@madsgroup.org>wrote:
>
>> > Things become problematic when you do not entirely understand the
>> > abstraction.
>> > Maybe the abstraction is inappropriate for your needs. I have seen many
>> > examples
>> > of code where the gen_server *is* inappropriate. The acid test is "does
>> the
>> > gen_sever code look like spaghetti" if the answer is yes then all you
>> have
>> > done
>> > is shoe horn the applications into an inappropriate form. In this case
>> > you should ditch the gen_server and roll-your own.
>>
>> Another problem I often see is that newbies tend to regard gen_server
>> as something that does magic, and fail to understand how the code runs
>> in the server and the clients (since that code lives in the same
>> module, usually).
I think this is the most confusing design pattern in Erlang:
send(x,y) ->
gen_server:call(?NAME, {send, x, y}).
handle_call({send, x, y}, State) ->
i_send(x, y).
i_send(x,y) ->
...
One always have to keep in mind that actual code he's looking at is in
the client process (send) or server (i_send). Obviously calling send
from i_send in the above example would just lead to a deadlock. Maybe
it would be a nice tool that I could run on a huge codebase and split
these modules into a client and server part...
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
Mihai Balea
This is *why* we made the gen_server abstraction. You can write well-typed
sequential code (the handle_call and init functions) to parametrize
a concurrent behavior, ie you need to know nothing about concurrency to
get the job done. We've "abstracted out" the concurrency.
Things become problematic when you do not entirely understand the abstraction.
Maybe the abstraction is inappropriate for your needs. I have seen many examples
of code where the gen_server *is* inappropriate. The acid test is "does the
gen_sever code look like spaghetti" if the answer is yes then all you have done
is shoe horn the applications into an inappropriate form. In this case
you should ditch the gen_server and roll-your own.
Jeroen Koops
The most common form of gen _server abuse I have seen (and I plead guilty of it myself) is attempting to turn it into something akin to a state machine. My rule of thumb is if your state record contains a field called state or something like it and you take different actions based on it, then it's time to re-factor into a gen_fsm. It's easy to get into this situation since things usually start simple and build up once more functionality is piled on.
Attila Rajmund Nohl
[...]
> On a related note, I have been moving away from using gen_servers all the
> time. The reason for this is that in a gen_server, the state consists of
> just the state term. In an arbitrary process, the state actually consists of
> the process' entire call-stack, and it makes the flow in code such as the
> almost proverbial:
>
> idle() ->
> receive
> { Number, incoming } ->
> start_ringing(),
> ringing(Number);
> off_hook ->
> start_tone(),
> dial()
> end.
>
> ... much easier to follow than when using a gen_server or gen_fsm.
On the other hand it's far from the functionality of a gen_server,
because e.g. it doesn't send back data to the caller. When you add
this functionality, you essentially reinvent the gen_server.
In other words: gen_server is great to provide safe access to shared
data, maybe it's not so great to control stuff. There's one more great
point in using the gen_* behaviours - they provide usually useful and
detailed crash reports when something goes wrong.
Kevin A. Smith
Abstractions aren't very helpful if you don't understand what they're abstracting.
--Kevin
Torben Hoffmann
The most common form of gen _server abuse I have seen (and I plead guilty of it myself) is attempting to turn it into something akin to a state machine. My rule of thumb is if your state record contains a field called state or something like it and you take different actions based on it, then it's time to re-factor into a gen_fsm. It's easy to get into this situation since things usually start simple and build up once more functionality is piled on.I don't know about this -- I have written numerous gen_servers whose state-record include a state-field (although I usually call it 'current'), and events (calls, casts, infos) are handled differently depending on its value. The reason for not using a gen_fsm is that very often, an event should be handled in one way in one particular state, and in another way in _all_ other states. This is easy to do using pattern-matching when using a gen_server, but when using a gen_fsm you have no choice but to write out every state/event combination, which can become pretty cumbersome.
Then one can use an gen_fsm:send_all_state_event/2 and pattern match on both state and state date that in the modules handle_event/3 function.
I have done that many times and it works rather well.
Cheers,
Torben
On a related note, I have been moving away from using gen_servers all the time. The reason for this is that in a gen_server, the state consists of just the state term. In an arbitrary process, the state actually consists of the process' entire call-stack, and it makes the flow in code such as the almost proverbial:idle() ->receive{ Number, incoming } ->start_ringing(),ringing(Number);off_hook ->start_tone(),dial()end.... much easier to follow than when using a gen_server or gen_fsm.
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
--
http://www.linkedin.com/in/torbenhoffmann
Joe Armstrong
Strong agreement with everything stated. When I teach an "Intro To Erlang" class I spend a lot of time (lecture and labs) working on the basics before we tackle gen_server. In fact, one of the last labs before getting to gen_server requires the class to write their own server very similar to Joe's code.
xcellent
Abstractions aren't very helpful if you don't understand what they're abstracting.
Well said !
One other point - years ago I taught programming (still do occasionally) and at the time I thought
"all I have to do is explain the abstractions" and that the student would immediately understand all
the consequences of the abstractions (good programmers do, which is why it it's nice talking to them,
you tell the story and they fill in the details) but many programmers don't.
I explained translative closures in Prolog (the good old related(A,B) :- ...) later one of the students
was answering the question "what's Prolog good for" they replied "figuring out genealogical relationships"
they were completely oblivious to the fact that the solution to a problem in genealogy was isomorphic to
the solutions to a large class of problems in unrelated problem areas. I then made up loads of
examples.
The point is, you can show people the generalization - but many will not understand the broad applicability
of the technique.
The best way is to ask people what their particular problem is and solve it for them using the
technique you have just outlined (if it is appropriate)
/Joe
Jeroen Koops
On Wed, Apr 20, 2011 at 15:09, Jeroen Koops <koo...@gmail.com> wrote:The most common form of gen _server abuse I have seen (and I plead guilty of it myself) is attempting to turn it into something akin to a state machine. My rule of thumb is if your state record contains a field called state or something like it and you take different actions based on it, then it's time to re-factor into a gen_fsm. It's easy to get into this situation since things usually start simple and build up once more functionality is piled on.I don't know about this -- I have written numerous gen_servers whose state-record include a state-field (although I usually call it 'current'), and events (calls, casts, infos) are handled differently depending on its value. The reason for not using a gen_fsm is that very often, an event should be handled in one way in one particular state, and in another way in _all_ other states. This is easy to do using pattern-matching when using a gen_server, but when using a gen_fsm you have no choice but to write out every state/event combination, which can become pretty cumbersome.
Then one can use an gen_fsm:send_all_state_event/2 and pattern match on both state and state date that in the modules handle_event/3 function.
I have done that many times and it works rather well.
Mazen Harake
Of course, but when you do that for the majority of the events it amounts to the same as using a gen_server with a 'state'-field, doesn't it?
When you say "an event should be handled in one way in one particular state, and in another way in _all_ other states" you are essentially talking about a state flag. That flag is a substate of a larger state and it is wrong to compare a state you are in (e.g. as in a gen_fsm state) and a state which you carry to each state (namely the substate or the "State" variable in the gen_fsm if you will).
This mix of states (where you use the "State" variable in a gen_server as a variant of gen_fsm's "nextstate") is not recommended but I have seen it a lot. I think this boils down to design firstly and laziness secondly; Developers don't want (as you say) to implement 9 other states that handle a specific message just because 1 state should do it in a special way.
What we should have is a gen_fsm which takes "Mod:handle_state(State::atom(
Moral of the story, they are not the same and if you end up in a situation that requires a lot of these "state flags" then perhaps a gen_server is not the solution, but also if you have too many "general" states with very few exceptions then perhaps a state flag is more appropriate.
/M
pmac...@comcast.net
Hi,
I am saving up for a new laptop to run Erlang. Can't drag my tower around. Two questions:
1 - I want to run Ubuntu. Will Erlang work well with this? (Assuming Ubuntu v10/11)
2 - What's a decent laptop manufacturer/model to run this on? I've always bought Dell in the past, but I am not wed to this manufacturer.
--Peter
Rapsey
Sergej
Eric Merritt
that has multiple cores so you get true concurrency off the bat. Just
about every laptop has that now though.
Håkan Mattsson
> Many users seem to use the gen_server for absolutely everything
> and often force their problems to fit the gen_sever even though the
> gen_server is not appropriate for their problems.
>
> The gen_server is an extremely simple bit of code which can
> be easily changed to fit other problems, though people don't
> often do this.
>
> In this posting I'll explain the basic idea of how the gen_server works.
>
> To illustrate this I've written mini_gs.erl - this is a mini gen_server
> if you understand mini_gs.erl you'll have understood 98% of how the
> real gen_server works. The real gen_server just adds a load of "bells
> and whistles" to mini_gs.erl.
I tend to use proc_lib (spawn_link, start_link, init_ack) and sys
(handle_system_msg) a lot. They gives you almost all the bells and
whistles that gen_server has, but it enables you to have the same
simple code structure that you have in your very pedagogic code
snippet below.
/Håkan
PS.
By the way, it seems more appropriate to use spawn_link in a
function called start_link.
Ulf Wiger
I tend to use proc_lib (spawn_link, start_link, init_ack) and sys
(handle_system_msg) a lot. They gives you almost all the bells and
whistles that gen_server has, but it enables you to have the same
simple code structure that you have in your very pedagogic code
snippet below.
Ulf Wiger
I'd love to see how you handle system messages in two lines of code, without reusing some library. :)
Usually, it involves writing your own versions of system_continue/3, system_terminate/4, system_code_change/4, and format_status/2.
BR,
Ulf W
On 21 Apr 2011, at 18:14, Håkan Mattsson wrote:
> I gladly write two lines of code to handle system
> messages, if that enables me to avoid a boring
> template. I really like the simplicity of Joe's code
> snippet. It is easy to understand and does not
> imply any black magic. It is just proc_lib that is
> missing...
>
> /Håkan
>
> PS. You did also have two "boring" lines in your
> template to handle code change.
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Ulf Wiger, CTO, Erlang Solutions, Ltd.
http://erlang-solutions.com
_______________________________________________
Juan Jose Comellas
would focus more on buying a laptop where Ubuntu runs without any
problems. Beware of the ones with NVIDIA Optimus video cards, as they
aren't supported on Linux. Also, you should probably choose one with
an Atheros WiFi chipset or anything else that you know runs without
any trouble.
Olivier Girondel
--
Olivier / http://biniou.net
Banibrata Dutta
ASUS laptops work just fine, both in 32 or 64 bits
As noted by others, the most usual sources of worry for Linux are: Wireless (WiFi) chipsets, graphics accelerator, and sound chipset, so the best approach is to first shortlist a bunch of models, figure out the details for those 3 worrysome parts, check the forums for compatibility issues for your target, and if you don't find any, figure out the HW compatibility list, for the specific version(s) of your favourtie Linux distros. The biggest source of pain is usually the devices which are the lowest end of the market, typically the ones which rely too much on host-CPU's computation power, and themselves do only limited necessary yet light-weight things that CPU can't.
Coming back to Erlang, my initial experience with Erlang with Ubuntu was bit different compared to my experience with Erlang on Windows. My installation of Erlang on Ubuntu 9.04 took a bit more of knowledge of Ubuntu's Erlang packaging, what might be needed (or not), so it is not (or was not) as dumbed-down as it is on Windows. In the end however, the flexibility of package choices was handy when I was trying to minimize disk footprint of installed software, and could get away with much lesser that it might have been otherwise.
Des Small
> Any laptop just about will work. I would suggest you go with something
> that has multiple cores so you get true concurrency off the bat. Just
> about every laptop has that now though.
At the bottom end, I have had occasion to use Erlang on my Asus eee
netbook with 1 Gb of RAM and a single-core Atom processor. (You can get
these for under EUR 200 these days.)
It isn't very fast and I'm very glad it isn't my main development
environment, but it does work OK and it fits neatly into my handluggage
for flights.
Cheers,
Des
--
Des Small,
Scientific Programmer, JIVE
Contact details on http://www.jive.nl/~small
Wes James
> Hi,
>
> I am saving up for a new laptop to run Erlang. Can't drag my tower around.
> Two questions:
>
>
>
> 1 - I want to run Ubuntu. Will Erlang work well with this? (Assuming Ubuntu
> v10/11)
you will have R13B03, and if you try to compile from scratch to get
R14B02, expect to do a bit of work (especially wxwidgets)
You may want to try this:
http://sivieri.wordpress.com/2011/04/08/erlang-r14b02-in-ppa/
if you want R14B02.
I just tried it and it seems to work fine so far.
You can also try this procedure (installing packages) to get R14B02
working on Ubuntu 10.10
http://sites.google.com/site/comptekkia/erlang/how-to-install-erlang-on-ubuntu-10-10
> 2 - What's a decent laptop manufacturer/model to run this on? I've always
> bought Dell in the past, but I am not wed to this manufacturer.
>
>
-wes
Torben Hoffmann
In terms of code evolution I feel that you risk being stuck with gen_server when gen_fsm would be the right choice.
Even if most of your events are send_all_state_event's you do gain more information with gen_fsm due to the structure of the callback functions. I have exploited that to create a little program that can produce a CSP description of a gen_fsm, which is good for communication with people who have a hard time reading Erlang code. I think it would have been harder to write that program using the gen_server + state approach, but that is guess work! (The program was very specific to the code style we used, so there is nothing to share.)
My experience is probably also tainted by the fact that I have mostly implemented telecom protocols and they fit best with gen_fsm and have only a few all state events.
Cheers,
Torben
--
http://www.linkedin.com/in/torbenhoffmann
David Mercer
> Many users seem to use the gen_server for absolutely everything
> and often force their problems to fit the gen_sever even though the
> gen_server is not appropriate for their problems.
>
> . . .
>
> This is *why* we made the gen_server abstraction. You can write well-typed
> sequential code (the handle_call and init functions) to parametrize
> a concurrent behavior, ie you need to know nothing about concurrency to
> get the job done. We've "abstracted out" the concurrency.
>
> Things become problematic when you do not entirely understand the
abstraction.
> Maybe the abstraction is inappropriate for your needs. I have seen many
examples
> of code where the gen_server *is* inappropriate. The acid test is "does
the
> gen_sever code look like spaghetti" if the answer is yes then all you have
done
> is shoe horn the applications into an inappropriate form. In this case
> you should ditch the gen_server and roll-your own.
I am not sure what you are advocating in gen_server's stead. Some others on
this thread seem to be pushing gen_fsm as the alternative, but I don't think
that's what you are arguing, since you have generally preferred lightweight
non-OTP processes. Are you arguing for more use of gen_fsm, more creation
of roll-your-own OTP-compliant processes using proc_lib and sys, or less
reliance on the OTP framework and more use of plain old Erlang?
David Mercer
thread? I think some people are not adequately distinguishing between their
contributions and the existing thread in their replies. Maybe it's just
me... Anyone else?
Mihai Balea
On Apr 25, 2011, at 4:52 PM, David Mercer wrote:
> I am not sure what you are advocating in gen_server's stead. Some others on
> this thread seem to be pushing gen_fsm as the alternative, but I don't think
> that's what you are arguing, since you have generally preferred lightweight
> non-OTP processes. Are you arguing for more use of gen_fsm, more creation
> of roll-your-own OTP-compliant processes using proc_lib and sys, or less
> reliance on the OTP framework and more use of plain old Erlang?
I believe I am the one who brought gen_fsm into this discussion.
My intention was to exemplify a particular case of gen_server abuse, one where gen_fsm would have made a lot more sense.
I did not intend to suggest that gen_fsm be regarded as a generic alternative to gen_server, but rather that each abstraction has its own domain of applicability.
Just thought I'd clear any possible misunderstandings.
Mihai
littlebobby
When you want to run erlang in production, there is no doubt that you want
to take advantage of it's battle tested code (OTP). So it's going to be some
behaviour, right? Well let's see, it's neither gen_event nor gen_fsm, so it
has to be gen_server.
Why is this question even asked? I think it might be because of a lack of
good explanations with lots of reasoning pro / contra per approach.
How is a person learning about OTP supposed to know the implications of
leaving the tested path? Leaving a tested path is generally a bad thing. To
do so you should be aware of the consequences (or whether there even are
any). People lack the knowledge to gauge the consequences. That knowledge is
only gained from experience - which can include writing spaghetti gen_server
code and recognizing it as a mess.
You have that knowledge and that might make it hard for you to comprehend
the impact of uncertainty here.
--
View this message in context: http://erlang.2086793.n4.nabble.com/The-gen-server-simplified-how-it-works-tp3462381p3474996.html
Sent from the Erlang Questions mailing list archive at Nabble.com.
Kenji Rikitake
and FreeBSD.
Kenji Rikitake
++> Des Small <sm...@jive.nl> [2011-04-22 08:13:56 +0200]:
David Mercer
attachment names ATT<nnnnn>.txt with the following contents?