daemon thread FitnessEvaluationWorker

[vsmo...@gmail.com]

Could you help to point out how FitnessEvaluationWorkers can be created as daemon threads?

I inherited some code that utilizes non daemon FitnessEvaluationWorkers.  This setup leaves behind some stale Fitness EvaluationThreads in WAITING state on our continuous integration (Jenkins) machine.

Plus we get a bunch of warning (one per thread) about potential memory leaks every time a test involving Watchmaker stops executing.

I cannot find code that instantiates these Workers in the inherited code. I am assuming that the EvolutionEngine does it under the hood.

Any insight, please?

[Daniel Dyer]

It's been a while since I wrote this code. It looks like the threads are always daemon threads unless you run a standalone FitnessEvaluationWorker by invoking its main method. The API doesn't provide a way for you to specify whether you want daemon threads or not. If the worker is created by the evolution engine you get daemon threads. If it's created via the main method, you get non-daemon threads.

The main method was added to experiment with distributed fitness evaluations; the idea being you could get several machines to share the work load. I had it working as a proof-of-concept with Terracotta but for short-lived fitness evaluations the overhead made it slower. If you are running everything in a single VM, you shouldn't be creating your own worker.

Another option is to bypass the FitnessEvaluationWorker and make everything single-threaded (https://watchmaker.uncommons.org/api/org/uncommons/watchmaker/framework/AbstractEvolutionEngine.html#setSingleThreaded(boolean)). It's not great for taking advantage of multiple cores, but may be useful for some testing scenarios.

I could make the FitnessEvalutionWorker constructor that takes a boolean (daemon/non-daemon) public, or add a stop() method to FitnessEvaluationWorker, but I'd like to understand your use case first.

Dan.

[vsmo...@gmail.com]

Thank you, Dan.  Thank you for the explanation about the "main" method

Having pondered about your reply let me re-phrase my original question: what is the best way to execute "finalize" method of FitnessEvaluationWorker (after  junit tests complete )?

I believe my issue is directly related to ThreadPoolExecutor object internal to FitnessEvaluationWorker not being shutdown and leaving behind its ThreadPool, each Thread in the ThreadPool in WAITING state,

Please correct me if my understanding is wrong:

- FitnessEvaluatorWorker object is created automatically by Watchmaker, specifically by EvolutionEngine (per JavaDoc)

If so, how can I get a reference to this object?  Or may be there is a standard way to trigger execution of this "finalize" method?  Or may be our implementation of TerminationCondition is faulty and

prevents EveolutionEngine from shutting down ThreadPoolExecutor?

Again, thank you for your response. Hopefully, my question makes sense.

Thank you.

VS

[d...@uncommons.org]

On Tue, 5 Dec 2017, at 09:13 PM, vsmo...@gmail.com wrote:

Thank you, Dan.  Thank you for the explanation about the "main" method

Having pondered about your reply let me re-phrase my original question: what is the best way to execute "finalize" method of FitnessEvaluationWorker (after  junit tests complete )?

I believe my issue is directly related to ThreadPoolExecutor object internal to FitnessEvaluationWorker not being shutdown and leaving behind its ThreadPool, each Thread in the ThreadPool in WAITING state,

Please correct me if my understanding is wrong:

- FitnessEvaluatorWorker object is created automatically by Watchmaker, specifically by EvolutionEngine (per JavaDoc)

Yes, unless you're calling the main method somewhere. The AbstractEvolutionEngine class will create the FitnessEvaluationWorker with daemon threads.

If so, how can I get a reference to this object?  Or may be there is a standard way to trigger execution of this "finalize" method?  Or may be our implementation of TerminationCondition is faulty and

prevents EveolutionEngine from shutting down ThreadPoolExecutor?

Currently the FitnessEvalutionWorker is a private static member of the AbstractEvolutionEngine and lazily instantiated. Because it's private, you can't get a reference to it. Because it's static, it won't be garbage collected when the evolution engine is, unless you unload the AbstractEvolutionEngine class, which will require some ClassLoader gymnastics.

Alternatively, you can avoid the FitnessEvaluationWorker ever being instantiated by using single-threaded mode, if you can live with the implications.

I've just had a look at the code and, as currently configured, the ThreadPoolExecutor will retain the idle threads indefinitely since the core thread pool size is set to whatever the number of available hardware threads is. Java 6 added an option to allow core threads to timeout, but this isn't being used.

I think therefore, that unless you can use single-threaded mode for your testing, it will require a change to the API to allow you to shutdown the threads properly.

Dan.

[vsmo...@gmail.com]

Perfect.  Thank you for your time and a detailed explanation.

VS