Configuration with an anytime/no budget

[Daniel Watson]

I've been looking for a configurator which does not have a finite budget. This is commonly called "anytime" behavior (and unfortunately overlaps with forum posts here about the target algorithm's anytime behavior).

There are several other AC algorithms and some have anytime behavior but none fit my use-case quite as well as irace.

So I added the following to "scenario.txt":

```

maxExperiments = 999999
nbExperimentsPerIteration = 300
nbIterations = 250

```

This might lean to far into exploration but I hope the soft-restart feature is like a pseudo-ILS.

Is there a better way to approach anytime configuration? Is it possible to build anytime configuration into irace at all?

[Manuel López-Ibáñez]

Hi Daniel,

I am not sure about this definition of "anytime". The definition that I use here (https://lopez-ibanez.eu/publications#LopStu2013ejor) is "return as high-
quality solutions as possible at any moment of their execution." Thus, the algorithm may have a finite budget but it tries really hard to improve solutions as quick as possible.

In that sense, irace is not designed for anytime behavior. It should be possible to have a better anytime behavior by tuning the parameters of irace using the method shown in (https://lopez-ibanez.eu/publications#LopStu2013ejor) but a truly anytime algorithm will require a re-design of irace. If you or someone else would be interested in that, please contact me outside this mailing list.

If you simply want irace to run for a very long budget but do not care about convergence speed, then the setting you give (with an even larger maxExperiments) could work.

If you want irace to converge faster, then you probably need small nbConfigurations (5), or alternatively a larger nbConfigurations (10-20) and a larger minSurvival (0.9 * nbConfigurations), so that the population is small.

(Ideally, nbConfigurations will grow overtime, which may be something easy to implement if you are not afraid to modify irace: https://github.com/MLopez-Ibanez/irace). And maybe elitistNewInstances=0 to avoid having too many instances at the end and also increase softRestartThreshold to trigger restarts more often.  I would not touch nbExperimentsPerIteration or nbIterations and let irace calculate them based on nbConfigurations.

You may wish to look at the output of irace using acviz https://github.com/souzamarcelo/acviz and iraceplot (https://auto-optimization.github.io/iraceplot/) to understand what is going in within irace.

Since these are unusual settings, please don't hesitate to report any crashes or unexpected behaviors that you may observe.

I hope the above helps!

Manuel.