How to handle evaluation failure of the black box function?

Lukas M

unread,

Dec 21, 2015, 2:23:06 AM12/21/15

to BayesOpt discussion

Hi all,

Is it possible to account for evaluation errors and what should be the return value?

Bests,

Lukas

Ruben Martinez-Cantin

unread,

Dec 21, 2015, 7:57:08 AM12/21/15

to BayesOpt discussion

> Is it possible to account for evaluation errors and what should be the
> return value?

That is actually the purpose of the noise parameter. Right now, only
constant (homoscedastic) noise can be considered.

Ruben

>
>
> Bests,
>
> Lukas
>
> --
> You received this message because you are subscribed to the Google Groups
> "BayesOpt discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to bayesopt-discus...@googlegroups.com.
> To post to this group, send email to bayesopt-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/bayesopt-discussion.
<div id="DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2"><table
style="border-top: 1px solid #aaabb6; margin-top: 10px;">
<tr>
<td style="width: 105px; padding-top: 15px;">
<a href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail"
target="_blank"><img
src="https://ipmcdn.avast.com/images/logo-avast-v1.png" style="width:
90px; height:33px;"/></a>
</td>
<td style="width: 470px; padding-top: 20px; color: #41424e;
font-size: 13px; font-family: Arial, Helvetica, sans-serif;
line-height: 18px;">Este correo electrónico se ha enviado desde un
equipo libre de virus y protegido por Avast. <br /><a
href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail"
target="_blank" style="color: #4453ea;">www.avast.com</a> </td>
</tr>
</table>
<a href="#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2" width="1" height="1"></a></div>

José Nogueira

unread,

Dec 21, 2015, 9:40:46 AM12/21/15

to Ruben Martinez-Cantin, BayesOpt discussion

I think I didn't understand your question question correctly.

Are you talking about the error | E[F(x)] - y_observation(x) | over the input space?

Cumprimentos,

José Nogueira

Lukas M

unread,

Dec 21, 2015, 10:14:59 AM12/21/15

to BayesOpt discussion, rmca...@unizar.es

I meant the case where e.g. the black box function crashes and returns an error, hence y_observation(x) = False or something similar.
As Ruben pointed out, this would be heteroscedastic noise and thus cannot be handled right now.

Ruben Martinez-Cantin

unread,

Dec 21, 2015, 12:57:26 PM12/21/15

to BayesOpt discussion

From the point of view of optimization, that's not an error/noise, but
a constrain.

Then you should use the evaluate the function inside the
checkReachability method where you can return True or False depending
on the success. Then, return the computed value in evaluateSample
method.

This way is only available in C++ and might not guarantee convergence
to the optimum.

Ruben<div id="DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2"><table

style="border-top: 1px solid #aaabb6; margin-top: 10px;">
<tr>
<td style="width: 105px; padding-top: 15px;">
<a href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail"
target="_blank"><img
src="https://ipmcdn.avast.com/images/logo-avast-v1.png" style="width:
90px; height:33px;"/></a>
</td>
<td style="width: 470px; padding-top: 20px; color: #41424e;
font-size: 13px; font-family: Arial, Helvetica, sans-serif;
line-height: 18px;">Este correo electrónico se ha enviado desde un
equipo libre de virus y protegido por Avast. <br /><a
href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail"
target="_blank" style="color: #4453ea;">www.avast.com</a> </td>
</tr>
</table>
<a href="#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2" width="1" height="1"></a></div>

Reply all

Reply to author

Forward