Some questions regarding the info in the User Guide

[jovanarad...@gmail.com]

Hi,

Sorry to bother you, but I have a few questions I haven't found answers for. I would appreciate it if you could answer.

1. nbIterations in the scenario - by default it is the minimum number of iterations, but if we set a value for it is it still a minimum number or is it the exact number of iterations that will be performed?

2. It says in the User Guide that the default test when capping is enabled is "t-test" , however when I ran an example it still said F-test was being used. Did I do something wrong, or did i misunderstand? You can see a screenshot at https://ibb.co/d7zKZwK.

3. At the bottom of page 32. of the User Guide the last command given is iraceResults$state$.irace$instancesList[index,"seed"], should it say index there or pair.id ?  

4. In the probabilistic model (seen in the "irace.Rdata" file) for numerical parameters, what are the numbers that are seen. Are they the mean and standard deviation for the truncate normal distribution?

5. On page 40. of the User Guide it says "targetRunner

should return the execution time of a configuration instead of solution cost",  when irace is run it says both the solution cost and the time should be returned by targetRunner, as expected.

6. Just to check with capping, on page 40, kij is the max time to spend on the j-th configuration, i is the number of instances seen, and bi is used to calculate that bound and for the domination criterion, it has no other purpose (it is referred to as the elite configurations bound)?

Best regards,

Jovana 

[The irace package: Iterated Racing for Automatic Configuration]

Hi Jovana,

1. You are right. This parameter sets the maximum number (there is no default maximum only default minimum). I have updated the User Guide in the development version: https://mlopez-ibanez.github.io/irace/ Be aware that this is an internal parameter, we do not recommend changing it directly as irace does not try to adjust other parameters according to this value, so results may be not what you expect.

2. Ouch, that is indeed a bug. Fixed in the development version in github that will be released next: https://mlopez-ibanez.github.io/irace/news/index.html#irace-3-5-unreleased Thanks for reporting it!

3. You are right! I have fixed the example in the development version.

4. It depends on the version of irace, but they should be the standard deviation and the value to use as the mean if the parent configuration does not have a value for that parameter.

5. I agree it was not completely clear: I have replaced with with the following: "When using \irace for tuning algorithms that only report computation time to
reach a target, \parameter{targetRunner} should return the execution time of a configuration instead of solution cost. When using \parameter{maxTime} as the budget, this means that \parameter{targetRunner} must return two times the execution time since the first value is the minimization objective and the second value is used to track the budget consumed." Does that clarify it?

6. Yes.

Cheers,

Manuel.

[Jovana Radjenovic]

Dear Manuel,

Thank you, everything is clear now. Thank you for mentioning me, and I am glad to be of help.

Best regards,

Jovana

--
You received this message because you are subscribed to the Google Groups "The irace package: Iterated Racing for Automatic Configuration" group.
To unsubscribe from this group and stop receiving emails from it, send an email to irace-packag...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/irace-package/e0c7ce55-3efa-4680-a5a1-54297ca29af4n%40googlegroups.com.