Actors: Very poor performance on sending messages

[Alexandre Verri]

Hi, it's my first experience with Akka. I'm used to work with multithreaded applications.

I've done a simple test with 2 actors, in a very simple context: the sender sends a message to the consumer. I've measured the time elapsed to send 1,000,000 messages. I think that the time is too long.

The code is very simple:

public class AkkaExample{

    public static void main(String[] args) {

        final ActorSystem system = ActorSystem.create("mySystem");

        final ActorRef pActor = system.actorOf(Props.create(ProducerActor.class));

        final ActorRef cActor = system.actorOf(Props.create(ConsumerActor.class));

        t0 = System.nanoTime();

        for (int i = 0; i < 1000000; i++) {

            cActor.tell("Hello", pActor);

        }

        total = System.nanoTime() - t0;

        System.out.println("Actor nano seconds: " + total);

        system.shutdown();

    }

    static class ConsumerActor extends UntypedActor {

        ConsumerActor() {

        }

        @Override

        public void onReceive(Object o) throws Exception {

            // Do nothing.

        }

    }

    static class ProducerActor extends UntypedActor {

        ProducerActor() {

        }

        @Override

        public void onReceive(Object o) throws Exception {

            // Do nothing.

        }

    }

}

The result is very bad: 280316892 nano seconds.

Using ExecutorService to coordinate threads, I can send the same 1,000,000 messages in 1084202 nano seconds. The example with Akka is 259 times slow.

I'm using Java 1.7, with Akka 2.2-M3.

<dependency>

    <groupId>com.typesafe.akka</groupId>

    <artifactId>akka-actor_2.10</artifactId>

    <version>2.2-M3</version>

</dependency>

Is there something wrong with my example?

[√iktor Ҡlang]

Hi!

I don't understand your benchmark, what are you trying to measure?

Recommendations:

Change to version 2.2.0-RC2

Change the mailbox of your actors to akka.dispatch.SingleConsumerOnlyUnboundedMailbox

Change the max number of threads of the default mailbox to 1

Change the throughput setting of the default dispatcher to 1024 or similar

Cheers,

√

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://akka.io/faq/
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Viktor Klang
Director of Engineering

Typesafe

Twitter: @viktorklang

[Alexandre Verri]

Hi, I've changed according to your recommendations, but the results is the same.

The code:

 final Config config = ConfigFactory.parseString("akka {\n" +

                "  actor {\n" +

                "    default-dispatcher {\n" +

                "      throughput = 1024\n" +

                "    }\n" +

                "  }\n" +

                "  default-mailbox {     \n" +

                "    mailbox-type = \"akka.dispatch.SingleConsumerOnlyUnboundedMailbox\"\n" +

                "  }  \n" +

                "}");

        final ActorSystem system = ActorSystem.create("mySystem", config);

        final ActorRef pActor = system.actorOf(Props.create(ProducerActor.class));

        final ActorRef cActor = system.actorOf(Props.create(ConsumerActor.class));

        t0 = System.nanoTime();

        for (int i = 0; i < MultiThreaded.SIZE; i++) {

            cActor.tell("Hello", pActor);

        }

        total = System.nanoTime() - t0;

        System.out.println("Actor nano seconds: " + total);

        system.shutdown();

I would like to benchmark the throughput for sending messages between actors. I've read that it's possible to send 50M messages one second between actors, and my test is far below it. 

Actualy, I'm using a simple ConcurrentLinkedQueue is to pass messages between threads is very fast (it took 1084202 nano seconds to send 1M messages between two threads, and Akka took 280316892 nano seconds to send 1M messages to an actor).

Regargds,

Alexandre

[Luis Ángel Vicente Sánchez]

Check this:

http://letitcrash.com/post/20397701710/50-million-messages-per-second-on-a-single-machine

2013/7/1 Alexandre Verri <alexand...@gmail.com>

[Roland Kuhn]

Hi Alexandre,

your measurement only covers enqueueing the messages into the recipient’s mailbox, not passing them to the actor’s behavior. As such it is indeed surprisingly slow, are you sure that you properly warmed up the JVM? It does not look like that is the case, given the code you quote.

Another small mistake: default-mailbox goes into the actor scope, right next to default-dispatcher.

Regards,

Dr. Roland Kuhn
Akka Tech Lead
Typesafe – Reactive apps on the JVM.
twitter: @rolandkuhn

[√iktor Ҡlang]

On Mon, Jul 1, 2013 at 7:38 PM, Alexandre Verri <alexand...@gmail.com> wrote:

Hi, I've changed according to your recommendations, but the results is the same.

The code:

 final Config config = ConfigFactory.parseString("akka {\n" +

                "  actor {\n" +

                "    default-dispatcher {\n" +

                "      throughput = 1024\n" +

                "    }\n" +

                "  }\n" +

                "  default-mailbox {     \n" +

                "    mailbox-type = \"akka.dispatch.SingleConsumerOnlyUnboundedMailbox\"\n" +

                "  }  \n" +

                "}");

default-mailbox goes into the actor scope (beneath or above the default-dispatcher config)

You also forgot to tweak the max threads, inside the default-dispatcher section, add:

fork-join-executor {

  parallelism-max = 2

}

Also, you aren't properly warming up the JVM before you start benching, this has some good reading: http://stackoverflow.com/questions/504103/how-do-i-write-a-correct-micro-benchmark-in-java

Hope that helps!

Cheers,

√

[Alexandre Verri]

Thank you all for help.

I've done the configuration as recommended, and the time is the same.

Now I'm using the following configuration:

akka {

  actor {

    default-dispatcher {

      throughput = 1024

 fork-join-executor {

        parallelism-max = 4

      }

    }

default-mailbox {

      mailbox-type = "akka.dispatch.UnboundedMailbox"

    }

  }  

}

The reference benchmark is the simple case of sending messages between threads using a ConcurrentLinkedQueue, please check below.

public static void main(String[] args) {

        ConcurrentLinkedQueue queue = new ConcurrentLinkedQueue();

        ExecutorService exe = Executors.newFixedThreadPool(4);

        Producer p = new Producer(queue);

        Consumer c = new Consumer(queue);

        long t0 = System.nanoTime();

        exe.submit(p);

        exe.submit(c);

        long total = System.nanoTime() - t0;

        System.out.println("Nano seconds: " + total);

        exe.shutdown();

}

class Producer implements Runnable {

    private final ConcurrentLinkedQueue queue;

    Producer(ConcurrentLinkedQueue queue) {

        this.queue = queue;

    }

    @Override

    public void run() {

        for (int i = 0; i < 1000000; i++)

            queue.add("Hello");

    }

}

class Consumer implements Runnable {

    private final ConcurrentLinkedQueue queue;

    Consumer(ConcurrentLinkedQueue queue) {

        this.queue = queue;

    }

    @Override

    public void run() {

        for (int i = 0; i < 1000000; i++)

            queue.poll();

    }

}

The example I gave is very fast: it took 1.2 ms to pass 1M messages between the threads. With Akka, it took +250 ms. Maybe I'm missing something.

Regards,

Alexandre

[√iktor Ҡlang]

Hi Alexandre!

On Mon, Jul 1, 2013 at 10:00 PM, Alexandre Verri <alexand...@gmail.com> wrote:

Thank you all for help.

I've done the configuration as recommended, and the time is the same.

Now I'm using the following configuration:

akka {

  actor {

    default-dispatcher {

      throughput = 1024

 fork-join-executor {

        parallelism-max = 4

      }

    }

default-mailbox {

      mailbox-type = "akka.dispatch.UnboundedMailbox"

A couple of observations:

1) You aren't using the right mailbox: mailbox-type = "akka.dispatch.SingleConsumerOnlyUnboundedMailbox"

2) There is no need for parallelism-max = 4 since you don't have 4 actors.

3) You are still not warming up the JVM before the benchmark.

4) AAAND, finally:

        long t0 = System.nanoTime();

        exe.submit(p);

        exe.submit(c);

        long total = System.nanoTime() - t0;

        System.out.println("Nano seconds: " + total);

This terminates the program before the producer or consumer has even executed. So you are benchmarking how long it takes to schedule the producer and the consumer to run, but not how long the producer takes to run.

Cheers,

√

[Alexandre Verri]

Hi, you are right of course. The time measurement must be used inside the producer or consumer. I've changed the consumer to:

long t0 = System.nanoTime();

        long counter = 0;

        while (true) {

            final Object poll = queue.poll();

            if (poll != null) {

                counter++;

                if (counter >= 1000000) {

                    break;

                }

            }

        }

        long total = System.nanoTime() - t0;

        System.out.println("Nano seconds: " + total);

With this correction, the time diference between this implementation and Akka is much lower (12 times).

Thank you a lot. I'm really interested in learn Akka, I've downloaded the full documentation to read in my Kindle.

[√iktor Ҡlang]

Hi Alexandre,

and what is the difference with a warmed up JVM/JIT?

(I'm positively surprise that it's only a 12 fold decrease in performance with a non-warmed jvm, considering what kind of features you get with Actors)

Cheers,

√

[Alec Zorab]

I see the numbers looking more like this: https://gist.github.com/AlecZorab/5907515

Ie, something like half the throughput.

[√iktor Ҡlang]

Thanks Alec, I am _very_ impressed by that considering that there are quite more logic involved in the Akka codepath.

There's clearly no reason to roll your own.

Cheers, V

[Rajiv Kurian]

Are there any benchmarks available for the new SingleConsumerOnlyUnboundedMailbox? 

[√iktor Ҡlang]

Here's one: https://github.com/plokhotnyuk/actors/tree/81a3308d6f39b3a6429c4933b2156d5ac0c0a1f4