having twice the same initial point changes the result
37 views
Skip to first unread message
DevShark
May 13, 2016, 10:21:25 AM5/13/16
to BayesOpt discussion
For testing, I compare what happens when I include twice the same point in the initialization phase. I was expecting that it makes no difference, but it does.
Thanks for the help!
With no duplicate:
| Initial | points: | ||||||||
| X:[4](0.535411,0.0912871,0.936008,0.838236)|Y:-131.892 | |||||||||
| X:[4](0.0615618,0.835183,0.341798,0.385033)|Y:-36.0984 | |||||||||
| X:[4](0.756241,0.597978,0.785665,0.53879)|Y:-11.1789 | |||||||||
| X:[4](0.870247,0.663176,0.846263,0.977768)|Y:-28.8244 | |||||||||
| X:[4](0.210823,0.216738,0.105533,0.438307)|Y:56.5692 | |||||||||
| X:[4](0.494329,0.404284,0.524138,0.661351)|Y:-42.46 | |||||||||
| X:[4](0.903634,0.922184,0.647815,0.705625)|Y:-165.619 | |||||||||
| X:[4](0.172734,0.185965,0.0894092,0.10974)|Y:23.3846 | |||||||||
| X:[4](0.661656,0.712999,0.458534,0.231818)|Y:81.7631 | |||||||||
| X:[4](0.352233,0.312991,0.25264,0.055005)|Y:60.0951 | |||||||||
| X:[4](0.5,0.55,0.51,0.75)|Y:-8.70692 | |||||||||
| Best | point | so | far: | ||||||
| X:[4](0.903634,0.922184,0.647815,0.705625)|Y:-165.619 | |||||||||
| ------ | Optimizing | hyperparameters | ------ | ||||||
| Initial | hyper | parameters: | [4](1,1,1,1) | ||||||
| 1st | opt | 288-> | [4](-1.03086,0.95679,-2.12551,-2.41358) | f() | ->30341.6 | ||||
| 2nd | opt | 32-> | [4](-1.21574,0.990706,-1.89685,-2.33075) | f() | ->30219 | ||||
| Final | hyper | parameters: | [4](-1.21574,0.990706,-1.89685,-2.33075) | ||||||
| State | succesfully | restored | from | file | /tmp/tmpgiUrtA | ||||
| ------ | Optimizing | criteria | ------ | ||||||
| 1st | opt | 1800-> | [4](0.5,0.5,0.5,0.5) | f() | ->-0 | ||||
| 2nd | opt | 200-> | [4](0.5,0.5,0.5,0.5) | f() | ->-0 | ||||
| Near | trial | 20|[4](0.902356,0.92178,0.647397,0.703743)-> | [4](0.903216,0.877531,0.651045,0.707559) | f() | ->-0.0738822 | ||||
| Local | beats | Global | |||||||
| Near | trial | 20|[4](0.902056,0.920996,0.645953,0.705086)-> | [4](0.900649,0.85192,0.65752,0.705086) | f() | ->-0.00753307 | ||||
| Near | trial | 20|[4](0.902394,0.922829,0.647216,0.704843)-> | [4](0.90428,0.870461,0.65136,0.704843) | f() | ->-0.0715464 | ||||
| Near | trial | 20|[4](0.902857,0.922506,0.648081,0.70541)-> | [4](0.902814,0.915126,0.648325,0.70558) | f() | ->-0.0299681 | ||||
| Near | trial | 20|[4](0.904411,0.922004,0.646968,0.706387)-> | [4](0.899342,0.85937,0.646968,0.706387) | f() | ->-5.18832e-06 | ||||
| Iteration: | 1 | of | 20 | | | Total | samples: | 12 | ||
| Query: | [4](24.193,15.1012,6.04182,8.30236) | ||||||||
With duplicates:
| Initial | points: | ||||||
| X:[4](0.535411,0.0912871,0.936008,0.838236)|Y:-131.892 | |||||||
| X:[4](0.0615618,0.835183,0.341798,0.385033)|Y:-36.0984 | |||||||
| X:[4](0.756241,0.597978,0.785665,0.53879)|Y:-11.1789 | |||||||
| X:[4](0.870247,0.663176,0.846263,0.977768)|Y:-28.8244 | |||||||
| X:[4](0.210823,0.216738,0.105533,0.438307)|Y:56.5692 | |||||||
| X:[4](0.494329,0.404284,0.524138,0.661351)|Y:-42.46 | |||||||
| X:[4](0.903634,0.922184,0.647815,0.705625)|Y:-165.619 | |||||||
| X:[4](0.172734,0.185965,0.0894092,0.10974)|Y:23.3846 | |||||||
| X:[4](0.661656,0.712999,0.458534,0.231818)|Y:81.7631 | |||||||
| X:[4](0.352233,0.312991,0.25264,0.055005)|Y:60.0951 | |||||||
| X:[4](0.5,0.55,0.51,0.75)|Y:-8.70692 | |||||||
| X:[4](0.5,0.55,0.51,0.75)|Y:-8.70692 | |||||||
| Best | point | so | far: | ||||
| X:[4](0.903634,0.922184,0.647815,0.705625)|Y:-165.619 | |||||||
| ------ | Optimizing | hyperparameters | ------ | ||||
| Initial | hyper | parameters: | [4](1,1,1,1) | ||||
| 1st | opt | 288-> | [4](-1.03086,0.95679,-2.12551,-2.41358) | f() | ->30335.1 | ||
| 2nd | opt | 32-> | [4](-1.21041,0.990679,-1.8927,-2.32931) | f() | ->30212.4 | ||
| Final | hyper | parameters: | [4](-1.21041,0.990679,-1.8927,-2.32931) | ||||
| State | succesfully | restored | from | file | /tmp/tmpUPjhk4 | ||
| ------ | Optimizing | criteria | ------ | ||||
| 1st | opt | 1800-> | [4](0.5,0.5,0.5,0.5) | f() | ->-0 | ||
| 2nd | opt | 200-> | [4](0.5,0.5,0.5,0.5) | f() | ->-0 | ||
| Near | trial | 20|[4](0.902356,0.92178,0.647397,0.703743)-> | [4](0.903267,0.876181,0.651028,0.707564) | f() | ->-0.0757023 | ||
| Local | beats | Global | |||||
| Near | trial | 20|[4](0.902056,0.920996,0.645953,0.705086)-> | [4](0.90065,0.851928,0.657391,0.705086) | f() | ->-0.0110441 | ||
| Near | trial | 20|[4](0.902394,0.922829,0.647216,0.704843)-> | [4](0.904319,0.870454,0.651318,0.704843) | f() | ->-0.0749613 | ||
| Near | trial | 20|[4](0.902857,0.922506,0.648081,0.70541)-> | [4](0.902815,0.915122,0.648364,0.705584) | f() | ->-0.0322321 | ||
| Near | trial | 20|[4](0.904411,0.922004,0.646968,0.706387)-> | [4](0.904411,0.863507,0.646968,0.706387) | f() | ->-4.28601e-06 | ||
| 24.196 | 15.04724 | ||||||
| Iteration: | 1 | of | 20 | | | Total | samples: | 13 |
| Query: | [4](24.196,15.0472,6.04113,8.30257) | ||||||
As you can see the red line is slightly different. So having an additional point changes the decision that is made for where to explore. Is that expected?
For reference, here's the file I use to load in the no duplicate case:
mCurrentIter=0
mCounterStuck=0
mYPrev=0
mParameters.n_iterations=20
mParameters.n_inner_iterations=500
mParameters.n_init_samples=11
mParameters.n_iter_relearn=50
mParameters.init_method=1
mParameters.surr_name=sGaussianProcess
mParameters.sigma_s=1
mParameters.noise=1e-06
mParameters.alpha=1
mParameters.beta=1
mParameters.sc_type=SC_MAP
mParameters.l_type=L_EMPIRICAL
mParameters.l_all=0
mParameters.epsilon=0
mParameters.force_jump=20
mParameters.kernel.name=kMaternARD5
mParameters.kernel.hp_mean=[1](1)
mParameters.kernel.hp_std=[1](10)
mParameters.mean.name=mConst
mParameters.mean.coef_mean=[1](1)
mParameters.mean.coef_std=[1](1000)
mParameters.crit_name=cEI
mParameters.crit_params=[0]()
mY=[0]()
mX=[11,4](0.5354105885,0.0912870703,0.9360078983,0.8382364499,0.06156182918,0.8351828123,0.3417980209,0.3850325136,0.756241279,0.5979781601,0.7856646714,0.5387904278,0.8702465395,0.6631758463,0.8462626772,0.9777678614,0.2108226998,0.2167380157,0.1055331081,0.4383066001,0.4943287024,0.4042844846,0.524138438,0.6613511022,0.9036337236,0.9221843243,0.6478151672,0.7056251925,0.1727343705,0.1859649223,0.08940923936,0.1097401529,0.6616558479,0.7129987855,0.4585338059,0.2318179702,0.3522334888,0.3129912749,0.2526399577,0.05500500104,0.5,0.55,0.51,0.75)
and in the duplicate case:
mCurrentIter=0
mCounterStuck=0
mYPrev=0
mParameters.n_iterations=20
mParameters.n_inner_iterations=500
mParameters.n_init_samples=12
mParameters.n_iter_relearn=50
mParameters.init_method=1
mParameters.surr_name=sGaussianProcess
mParameters.sigma_s=1
mParameters.noise=1e-06
mParameters.alpha=1
mParameters.beta=1
mParameters.sc_type=SC_MAP
mParameters.l_type=L_EMPIRICAL
mParameters.l_all=0
mParameters.epsilon=0
mParameters.force_jump=20
mParameters.kernel.name=kMaternARD5
mParameters.kernel.hp_mean=[1](1)
mParameters.kernel.hp_std=[1](10)
mParameters.mean.name=mConst
mParameters.mean.coef_mean=[1](1)
mParameters.mean.coef_std=[1](1000)
mParameters.crit_name=cEI
mParameters.crit_params=[0]()
mY=[0]()
mX=[12,4](0.5354105885,0.0912870703,0.9360078983,0.8382364499,0.06156182918,0.8351828123,0.3417980209,0.3850325136,0.756241279,0.5979781601,0.7856646714,0.5387904278,0.8702465395,0.6631758463,0.8462626772,0.9777678614,0.2108226998,0.2167380157,0.1055331081,0.4383066001,0.4943287024,0.4042844846,0.524138438,0.6613511022,0.9036337236,0.9221843243,0.6478151672,0.7056251925,0.1727343705,0.1859649223,0.08940923936,0.1097401529,0.6616558479,0.7129987855,0.4585338059,0.2318179702,0.3522334888,0.3129912749,0.2526399577,0.05500500104,0.5,0.55,0.51,0.75,0.5,0.55,0.51,0.75)
Thanks for the help!
Reply all
Reply to author
Forward
0 new messages