Anyone interested in helping tweak skopt's performance in a benchmarking exercise? #1002
Replies: 2 comments 3 replies
-
|
The leaderboard link 404s. Wouldn't better-performing combinations of parameters depend always solely on the distribution of the underlying data that is being fitted (i.e. no free lunch)? Why not just do grid search over all accepted parameters/values? |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, I fixed the leaderboard URL. If you just want one of them, https://github.com/microprediction/optimizer-elo-ratings/tree/main/results/leaderboards/overall Grid search could be included ... which would probably answer your question. Although remember that these optimizers are being challenged in quite high dimensions sometimes, so it isn't entirely clear what constitutes a grid search. |
Beta Was this translation helpful? Give feedback.
-
|
I meant grid search over viable parameters to |
Beta Was this translation helpful? Give feedback.
-
|
I'm with you now. Just a newbie so not sure how to describe the valid params. |
Beta Was this translation helpful? Give feedback.
-
|
I guess just from the documentation? E.g. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Hello,
I am wondering if someone with more experience than me is interested in helping tweak skopt's performance in a benchmarking exercise I am running (in the open) for my company.
As you can see from the leaderboard skopt is beating a whole slew of the usual suspects. Nice job!
However, I'd like to include a few more sub-choices in gp_minimize and wondering if anyone has favourite choices of parameters. My use of skopt is here. I'm thus far trying the following variations:
'lcbexplore':{"acq_func": "LCB", "kappa":3.0},
'lcb':{"acq_func":"LCB","kappa":1.96},
'lcbexploit':{"acq_func":"LCB","kappa":1.0},
'pi':{"acq_func":"pi"},
'xi02':{"xi":0.02},
'default':{},
'sampling':{'acq_optimizer':'sampling'},
'lbfgs':{'acq_optimizer':'lbfgs'},
'noisy':{'noise':'gaussian'},
'calm':{'noise':1e-8},
'':{}
Pls let me know if something else should be tried.
Peter
Beta Was this translation helpful? Give feedback.
All reactions