PDF manual wrong issues

[Thomas Feutrier]

Hello,
I think I've come across a bug in the pdf of the package manual on pages 32-33.
In order to retrieve the instance names, you do:
pair.id <- which(!is.na(all.exp))
AND you say that what we want to retrieve are the rownames that correspond to the Seed instances in instancesList.
However which() doesn't return the rowname but the index if the "all.exp" was a column by column vector, at least on my version of R.
The problem is that too many instances are retrieved in the case of a very large train set, and especially the wrong ones.
To fix this, all you have to do is:
pair.id <- which(!is.na(all.exp), arr.ind=TRUE)[, "row"]
And you've got the rownames and the right number.

I can send the logfile and the executable code to test. Hoping that which works the same way on your computers.

Kind Regards,

Feutrier Thomas  

[Manuel López-Ibáñez]

Hi Thomas,

Please do so. I cannot see where the text says that 'which' returns rownames.

Could you open an issue? https://github.com/MLopez-Ibanez/irace/issues

And point where in the user-guide source file you think the error may be?

This is where the line that you mention is located:

https://github.com/MLopez-Ibanez/irace/blob/35d09d9d0b175b055a5421aac4f46c15cbd152db/vignettes/irace-package.Rnw#L2024

Thanks!

Manuel.

[Thomas Feutrier]

Hello again,

You can try this logFile: https://drive.google.com/file/d/1ihbeKFxCdK9A-ZHGWpZ06zUYmi_Lbjgh/view?usp=sharing

and execute these lines:

all.exp <- iraceResults$experiments[,as.character(c(1,2,3))]

all.exp

pair.id <- which(!is.na(all.exp),arr.ind = TRUE)[,"row"]

pair.id <- which(!is.na(all.exp))

pair.id

#Get index of each instance used

index_instances <- iraceResults$state$.irace$instancesList[pair.id, "instance"]

tmp <- iraceResults$scenario$instances[index_instances]

You will see that with the version in the pdf: you get 25 instances used while the matrix all.exp shows that there are 10 on the first three config.

Feutrier Thomas

[Manuel López-Ibáñez]

Hi Thomas,

The example there is designed to work when all.exp is a vector, so is.na() and which() return vectors.

If you select two configurations or more, then all.exp is a matrix and, you would need to choose of these two options:

# You want instances where all configurations have been evaluated:

pair.id <- which(apply(!is.na(all.exp), 1, all))

# You want instances where at least one configuration has been evaluated:

pair.id <- which(apply(!is.na(all.exp), 1, any))

For the next version of irace, I am going to change this example to make the above clearer.

I hope the above helps!

Manuel