| The issue is that not always when there is a queue you want to add extra nodes. With the current plugin, however, it will always scale up the nodes as soon as there is a queue. The only way to make sure this does not totally get out of control, is to limit the maximum amount of nodes. In our situation this results in two issues:
- There are often more nodes running then needed.
- There are sometimes not enough nodes, because the max is too low.
Example 1
- There are 5 nodes with 2 executors, all busy.
- There are 10 jobs in the queue.
- You don't want to scale up, because the 5 nodes will be able to finish everything in 10 minutes.
Example 2
- There are 5 nodes with 2 executors, all busy.
- There are 10 jobs in the queue.
- You do want to scale up. More nodes are needed to catch up with the queue.
There are several factors that influence how you want your nodes to scale up.
- The startup time of new nodes + extra tasks only the first execution would do.
- The amount of time an average job takes to complete.
- The distribution of amount of jobs over time (evenly vs peaks).
|