[erlang-questions] Scoped group leaders
Yurii Rashkovskii
I had an idea to play with this long weekend and I want to share some
results and get some feedback.
Basically, I was wondering whether it will be feasible to extend the
concept of group leaders beyond I/O so that you can build your own
primitives around this construct of inherited group leaders. It seemed
to me that tasks like implicit configuration, security groups and such
can benefit from having a more generalized group leadership mechanism.
In the last two days I wrote a quick-n-dirty proof of concept
implementation (https://github.com/spawngrid/otp/tree/group_leader_scope),
it's pretty rough but seems to work. I also wrote an early draft of an
EEP for this feature
(https://github.com/spawngrid/eep/blob/scoped-group-leaders/eeps/eep-scoped-group-leaders.md).
The EEP has some examples, and you can play with that PoC
implementation (group_leader_scope branch). It's not ready for the
prime time, though — it lacks automated tests and the performance
implications have not yet been measured (although this should be fixed
soon). No optimizations were considered at this moment.
The EEP draft itself has not been sent to EEP editors as I plan to
elaborate on motivations and possibly add more examples.
Either way, please let me know if this is of any interest to anybody
and if you have any thoughts about it.
Have a nice weekend,
Yurii.
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
Frédéric Trottier-Hébert
The abstractions put in place in front of the IO server could be used across other resources with potentially different semantics (and thus requiring a different protocol). This could be a nice step towards that.
--
Fred Hébert
http://www.erlang-solutions.com
On 2011-07-04, at 10:15 AM, Eric Merritt wrote:
> I can actually see a ton of uses for this. Not the least of which is
> security, I just did a deep security piece in an erlang app, and it
> turned out well, but I could see where something like this would have
> made it much more elegant. The only thing that would concern me (from
> looking at your implementation) is how much over head is added to
> process creation. I think thats probably the main unknown, there are
> probably some other things as well.
>
> Just out of curiosity does a group leader go away when a group dies?
Eric Merritt
security, I just did a deep security piece in an erlang app, and it
turned out well, but I could see where something like this would have
made it much more elegant. The only thing that would concern me (from
looking at your implementation) is how much over head is added to
process creation. I think thats probably the main unknown, there are
probably some other things as well.
Just out of curiosity does a group leader go away when a group dies?
On Sat, Jul 2, 2011 at 7:38 PM, Yurii Rashkovskii <yra...@gmail.com> wrote:
Ulf Wiger
On 4 Jul 2011, at 16:15, Eric Merritt wrote:
> I can actually see a ton of uses for this. Not the least of which is
> security, I just did a deep security piece in an erlang app, and it
> turned out well, but I could see where something like this would have
> made it much more elegant
How, exactly?
The things that are difficult to do today, in terms of security in Erlang, is to *enforce* security in a way that cannot be subverted.
Group leaders are natural for IO, since IO necessarily involves sending a message to some entity. Some other things are overloaded - e.g. the application controller using the group leader to find out which application a process belongs to. That is a clear case where I think this would be useful: an "application group leader". But for security, I don't see it solving any of the problems I consider difficult - perhaps this is my failing: after having implemented reasonably transparent security in ErlHive, I get all kinds of associations to nasty problems that wouldn't be helped by this suggestion. But perhaps you are viewing it from a different angle?
BR,
Ulf W
Ulf Wiger, CTO, Erlang Solutions, Ltd.
http://erlang-solutions.com
Eric Merritt
>
> On 4 Jul 2011, at 16:15, Eric Merritt wrote:
>
>> I can actually see a ton of uses for this. Not the least of which is
>> security, I just did a deep security piece in an erlang app, and it
>> turned out well, but I could see where something like this would have
>> made it much more elegant
>
> How, exactly?
In the existing model a context is passed through out the system,
identifying the users and their capabilities. In this case, assuming
that all threads related to use where spawned at some level by the
incoming process, a context would not have had to be passed, it could
have been implicit in the group leader. This would have been much less
invasive in the system. I made no attempt to protect against
developers adding malicious code to the system. That is hard
regardless of the model. Perhaps this would have been better called an
access control system rather then a security system given the
assumptions.
>
> The things that are difficult to do today, in terms of security in Erlang, is to *enforce* security in a way that cannot be subverted.
>
> Group leaders are natural for IO, since IO necessarily involves sending a message to some entity. Some other things are overloaded -
>e.g. the application controller using the group leader to find out which application a process belongs to. That is a clear case where I think
>this would be useful: an "application group leader". But for security, I don't see it solving any of the problems I consider difficult - perhaps
>this is my failing: after having implemented reasonably transparent security in ErlHive, I get all kinds of associations to nasty problems
>that wouldn't be helped by this suggestion. But perhaps you are viewing it from a different angle?
>
I am not thinking about this as solving any truly hard problems, not
that can't be solved in some other way. It just would have made the
code created for this project, more elegant, and the security aspects
much less invasive then they currently are, at the cost of slightly
more difficult testing and some knowledge about where the process
group boundries are.
Yurii Rashkovskii
> The only thing that would concern me (from
> looking at your implementation) is how much over head is added to
> process creation. I think thats probably the main unknown, there are
> probably some other things as well.
Judging by my limited "unscientific" performance testing that I did
yesterday, I've got virtually no impact for situations where you do
not create any extra group leaders (i.e. running code that can be done
today). In a case when you have a hundred extra group leaders, it
slows down by about 2.5 times.
Please bear in mind that current implementation is a proof-of-concept
level quality. I can already foresee some techniques for speeding up
group leader allocation by allocating on process creation in bulk,
it's rather simple and will reduce number of allocations to just one.
In fact, this is so easy I might update this PoC implementation rather
soon! It sounds like a good idea.
> Just out of curiosity does a group leader go away when a group dies?
Define group death?
Eric Merritt
>
>> Just out of curiosity does a group leader go away when a group dies?
>
> Define group death?
>
When when all processes in a group have terminated, either normally or
abnormally.
Yurii Rashkovskii
>>
>>
>>> Just out of curiosity does a group leader go away when a group dies?
>>
>> Define group death?
>>
>
> When when all processes in a group have terminated, either normally or
> abnormally.
I have intentionally decided not to overcomplicate current
implementation. Right now nothing will happen.
Robert Virding
> >
> >
> >> Just out of curiosity does a group leader go away when a group
> dies?
> >
> > Define group death?
> >
>
> When when all processes in a group have terminated, either normally
> or
> abnormally.
Today there is nothing in the system which keeps track of the processes in a group, process groups don't really exist at all. A "process group" is those processes which have the same group leader. The group leader has no information about which processes have it as group leader. You can make any process a group leader by setting it as group leader.
Process groups were explicitly added in such a way as NOT to impose any structure or hierarchy on processes, process space is flat.
Robert
Yurii Rashkovskii
<robert....@erlang-solutions.com> wrote:
> Today there is nothing in the system which keeps track of the processes in a group, process groups don't really exist at all. A "process group" is those processes which have the same group leader. The group leader has no information about which processes have it as group leader. You can make any process a group leader by setting it as group leader.
>
> Process groups were explicitly added in such a way as NOT to impose any structure or hierarchy on processes, process space is flat.
That's right. I personally understand them as "ephemeral" notions,
just like you defined. That's why I don't do anything if a group
leader dies. It's a duty of individual process to decide what to do
with this.
Yurii
Eric Merritt
problem. That is group leaders are created but never destroyed. In the
case of the process itself, this is probably ok. The developer will
have to create some mechanism to know when his group leader is no
longer useful and have it die on its own. However, I didn't see any
mechanisms to unregister group leaders and reclaim the memory alloced
when they are created. That could be me missing something when I was
looking at the code, if so then the point is moot, however if not
that's something that would probably need to be added to make this
workable. Of course, that still leaves the problem that if a group
leader dies unexpectedly, or the user mis-codes his application and
forgets to unregister a group leader when it dies, then he has a
memory leak. You could say that its a users problem and go from there,
but leaving around unreferenced memory just doesn't smell right. (I
know, we do that already with the atom table, but it doesn't make it a
good thing)
Eric Merritt
On Tue, Jul 5, 2011 at 10:31 AM, Yurii Rashkovskii <yra...@gmail.com> wrote:
> Right now there is no way to remove a group leader from process, but
> it is not inherently impossible to do so, just was out of scope of
> this prototype.
Yurii Rashkovskii
it is not inherently impossible to do so, just was out of scope of
this prototype.
On Tue, Jul 5, 2011 at 8:05 AM, Eric Merritt <ericbm...@gmail.com> wrote:
Eric Merritt
of the group exist any longer. When a group has no processes, the
group no longer exists, however the memory alloced by this will
continue to exist, thats the problem.
On Tue, Jul 5, 2011 at 6:04 PM, Robert Virding
<robert....@erlang-solutions.com> wrote:
> Conceptually a process is always the member of a group so the very idea of removing a group leader from a process does not make sense. All you can do is assign it another group leader.
>
> Robert
Robert Virding
Robert Virding
Robert
----- "Eric Merritt" <ericbm...@gmail.com> wrote: