How expensive blocking actors are?

[Alexander Gagarin]

Hi!

Documentation states that each blocking actor live in it's own thread of execution, 'not so lightweight' and 'do not scale to large numbers'.

Can you please give a glance explanation why is that? Does above means that spawning new blocking actor (for example, `scoped_actor`) = spawn new thread?

Application of this question is that I want to provide 'regular' class interface (API) for requests to async event-based actor. What I do now:

1. Spawn event-based actor A that do all async work and communicate with other such actors (in a group).

2. Create a `caf::function_view` instance B (that internally spawns `scoped_actor`) used to make blocking requests to actor A.

Steaps 1 & 2 are performed in constructor of _every_ object.

Having that, if some method of object API must return calculation result, I just make blocking request `B(A, ...)` inside that method. Client code that use objects doesn't even aware that there are actors involved, it just calls conventional class API.

This works fine, but I afraid that creating `scoped_actor` instances (contained inside `function_view`) equal to numer of objects (can be large) is not a good idea? If true, then what can you suggest to improve the design?

Thanks beforehand.

[Dominik Charousset]

> Documentation states that each blocking actor live in it's own thread of execution, 'not so lightweight' and 'do not scale to large numbers'.
> Can you please give a glance explanation why is that?

The overhead simply comes from having a std::thread. This means: stack allocation, system calls, signal management, etc. This isn’t terribly expensive, but it adds several kB of RAM overhead to an actor that otherwise only needs a couple hundred Bytes. Spawning a couple dozen blocking actors (=threads) is not a problem. But you wouldn’t want to start thousands of them (unless you have some monster hardware sitting around with thousands of cores to match).

> Does above means that spawning new blocking actor (for example, `scoped_actor`) = spawn new thread?

No. A scoped actor is not really “spawned”. It has no control loop on its own and only exists for ad-hoc communication with other actors. Scoped actors do heap allocations, but otherwise are inexpensive exactly because they don’t have their own thread of execution.

> Application of this question is that I want to provide 'regular' class interface (API) for requests to async event-based actor. What I do now:
> 1. Spawn event-based actor A that do all async work and communicate with other such actors (in a group).
> 2. Create a `caf::function_view` instance B (that internally spawns `scoped_actor`) used to make blocking requests to actor A.
> Steaps 1 & 2 are performed in constructor of _every_ object.
>
> Having that, if some method of object API must return calculation result, I just make blocking request `B(A, ...)` inside that method. Client code that use objects doesn't even aware that there are actors involved, it just calls conventional class API.
>
> This works fine, but I afraid that creating `scoped_actor` instances (contained inside `function_view`) equal to numer of objects (can be large) is not a good idea? If true, then what can you suggest to improve the design?
> Thanks beforehand.

Generally I don’t see a problem with this design. As long as the extra memory allocations and constructing overheads for the function views don’t impact performance. Depending on your API guarantees (safe to call from multiple threads?), maybe you can reduce the number of scoped actors / function views by sharing them between your objects. But I wouldn’t bother before measuring the system. Unless your objects are short-lived, construction overhead probably isn’t an issue.

Hope that helps.

Dominik

[Alexander Gagarin]

Nice and clear explanation as always. Big thanks, Dominik.

[Alexander Gagarin]

> Does above means that spawning new blocking actor (for example, `scoped_actor`) = spawn new thread?

No. A scoped actor is not really “spawned”. It has no control loop on its own and only exists for ad-hoc communication with other actors. Scoped actors do heap allocations, but otherwise are inexpensive exactly because they don’t have their own thread of execution.

Just for clarification - do this apply only to `scoped_actor`? If I spawn 'normal' custom blocking actor with event loop - a separate std::thread will be spawned for it?

[Dominik Charousset]

> > > Does above means that spawning new blocking actor (for example, `scoped_actor`) = spawn new thread?
>
> > No. A scoped actor is not really “spawned”. It has no control loop on its own and only exists for ad-hoc communication with other actors. Scoped actors do heap allocations, but otherwise are inexpensive exactly because they don’t have their own thread of execution.
>
> Just for clarification - do this apply only to `scoped_actor`? If I spawn 'normal' custom blocking actor with event loop - a separate std::thread will be spawned for it?

Correct. Whenever you spawn a blocking actor or an event-based actor with the `detached` flag, you’ll get a std::thread for running it. The scoped_actor is the only exception, because it does not have its own control loop.

Dominik

[Alexander Gagarin]

I just ran a very primitive performance test of design described in this thread.

Conditions are the following.

1. One object created (contains one event-based actor A and one `function_view` B for queries).

2. Object's method is called N times in a loop (large N), does `B(A, E, ...)` inernally.

3. Message handler for E does nothing, just return a value of variable from actor A.

Immediate results confusing me -- even for not so big N (1000, 10000) test loop from step 2 runs for a long and noticable period of time with low CPU usage. If I change implementaion to omit request and instead cast actor handle -> actor A instance -> directly return A's variable then (as expected) loop finishes almost instantly even for very big N (10 millions and more).

I tried to change scheduler from 'steal' to 'sharing'. Then loop executes much faster (order of magnitude I think) and CPU usage is high. But still not even close to 'direct' value return (expectable).

Do CAF work-stealing scheduler sleeps/waits/blocks a lot inside? I mean long execution time and low CPU usage. What can I do to imrove throughput in such workloads (single actor, low compultational cost of message handler, a lot of messages)? Maybe a better choice is to spawn N short-lived actors for each method call instead of passing N messages to single actor? BTW messages are actually processed sequentially, beacuse on each loop iteration we wait until answer is ready.

PS. Will post time measurements later.

[Alexander Gagarin]

Well, I did some searching in this group and found everal topics where people came to same conclusions as I did.

For example, here: https://groups.google.com/d/topic/actor-framework/sm8gZECywl8/discussion

Suggestion is to set sleep duration timess to zero. By doing that I get high CPU usage and major decrease of loop execution time. Will do more experiments.

[Dominik Charousset]

> Well, I did some searching in this group and found everal topics where people came to same conclusions as I did.
> For example, here: https://groups.google.com/d/topic/actor-framework/sm8gZECywl8/discussion
>
> Suggestion is to set sleep duration timess to zero. By doing that I get high CPU usage and major decrease of loop execution time. Will do more experiments.

Would you mind sharing your code for the experiments? If we have a tool to reproduce this behavior reliably, it would make digging into what’s going on much easier. :)

Dominik