Thread pool executor

[Alessandro Arcangeli]

Hi there,

I'm trying to implement a pool thread executor using KJ.

I mean an executor that has a list of threads and sends the job to  the "most empty" one.

Or better, when a thread finish his job (or it is blocked by an IO) it automatically pick the next job from the executor.

I looked up in kj's public api but found no way (also combining executors/threads/TaskSets etc).

Is there a way to do something like this using KJ?.

Thanks

Sample Java Equivalent:

import java.util.concurrent.ExecutorService;

import java.util.concurrent.Executors;

public class TestExecutor {

  public static void main(String[] args) {

    ExecutorService service = Executors.newFixedThreadPool(8);

    for (int i = 0; i < 100; i++) {

      service.execute(() -> {

        System.out.println("Running from " + Thread.currentThread().getName());

        try {

          Thread.sleep(100);

        }

        catch (InterruptedException e) {

        }

      });

    }

    service.shutdown();

  }

}

Sample output:

Running from pool-1-thread-2

Running from pool-1-thread-8

Running from pool-1-thread-3

Running from pool-1-thread-5

Running from pool-1-thread-1

Running from pool-1-thread-7

Running from pool-1-thread-4

Running from pool-1-thread-6

Running from pool-1-thread-7

Running from pool-1-thread-6

Running from pool-1-thread-3

Running from pool-1-thread-8

...

[Kenton Varda]

Hi Alessandro,

KJ doesn't currently have any built-in thread pool implementation. The KJ async / Promise framework is centered around single-threaded event loops, so probably isn't what you're looking for.

I guess if I were implementing this I'd write a class that creates threads using kj::Thread, and has some sort of kj::MutexGuarded<Queue<Job>>. Each thread would run a loop where it uses MutexGuarded::when() to wait for the queue to be non-empty, pops the top job off the queue, and runs it.

-Kenton

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/b331a5f4-4162-46b5-a160-ead0ffecdfddn%40googlegroups.com.

[Alessandro Arcangeli]

Hi Kenton, thanks for the response.

I'm currently trying to use an EventPort for fetching jobs from a shared queue.

The problem is that once a job is completed I have no way to return to the caller thread.

I read from another discussion about newCrossThreadPromiseAndFulfiller, this would be really useful (I create it when the job is requested and I fulfill it from the other thread when the job is ended).

Do you have any date for the 0.9 release?

Anyway I can checkout the master branch since my project is in super early stage.