Question of maxExperiments in scenario.txt

[yanfei qian]

Dear irace team,

I am confused about the parameter of maxExperiments in scenario.txt.

In irace-package user guide, 

maxExperiments (--max-experiments): the maximum number of executions of the target algorithm that irace will perform. 

In the paper, The irace package: Iterated racing for automatic algorithm configuration,

maxExperiments is the tuning budegt. 

Combined with the above two points, my understanding of MaxExperiments is the maximum number of cycles of the circle in Fig.A.11. I hope you can correct me if I understanding  is wrong.

I try to set maxExperiments = 30, but there is an error:Insufficient budget: With the current settings, irace will require a value of 'maxExperiments' of at least '96'.

I have no idea about how to set the minimum value of maxExperiments in irace.

and If I want to set MaxExperiments = infinite times，How should I represent it in Scenario？

Best wishes,

Qian Yanfei

20 July

[Manuel López-Ibáñez]

Hi Qian,

Your understanding is correct.

30 runs of the algorithm is too little. Imagine that irace needs to spend at least 5 runs (firstTest) per configuration to do any type of elimination test. And to do any kind of learning of good parameters, it needs to do more than one iteration. The minimum number of iterations depends on the number of parameters, but let's say 3, so 3*5=15 runs per configuration at least. With 30 runs you can only look at 2 parameter configurations. That doesn't make sense. So irace refuses to start. In the past we allowed such ridiculous budget and they often reported that irace was not working. But how can it work at all if it only looks at 2 configurations?

Running with maxExperiments=infinite does not make sense, it will take infinite time to run and you will never get an answer.

I hope this helps,

Manuel.