Basic Question About Grid Search and Cross Validation

[Lorenzo Isella]

Dear All,
I am back to using h2o (from R) after ages.
I am a caret/mlr user to tune model hyperparameters.
I discovered that now h2o offers cross validation and grid search, but
it is not clear how to combine them for me.
Apologies for now providing any code, but if I could I would not be
asking this question.
Can anyone provide a simple self-contained example of a random Forest
model where cross validation and grid search are used to select the
"best" mtries value?
Many thanks

Lorenzo

[Erin LeDell]

Here are some examples of grid search in H2O R:

• https://github.com/h2oai/h2o-tutorials/blob/master/h2o-open-tour-2016/chicago/grid-search-model-selection.R
• https://github.com/h2oai/h2o-3/blob/master/h2o-r/ensemble/demos/higgs_randomgrid_stack.R#L54
  
• https://github.com/h2oai/h2o-3/blob/master/h2o-r/demos/rdemo.tutorial.eeg.eyestate.ipynb
  

And here is a code snippet which shows how to get the "best" model for a grid.  You first have to decide which metric you care about (this uses AUC):

auc_table <- h2o.getGrid(grid_id = "eeg_demo_gbm_grid", sort_by = "auc", decreasing = TRUE)
best_model <- h2o.getModel(auc_table@model_ids[[1]])
h2o.auc(best_model, valid = TRUE)  

-Erin

On 10/20/16 12:51 PM, Lorenzo Isella wrote:

Dear All, I am back to using h2o (from R) after ages. I am a caret/mlr user to tune model hyperparameters. I discovered that now h2o offers cross validation and grid search, but it is not clear how to combine them for me. Apologies for now providing any code, but if I could I would not be asking this question. Can anyone provide a simple self-contained example of a random Forest model where cross validation and grid search are used to select the "best" mtries value? Many thanks Lorenzo

-- 
Erin LeDell Ph.D.
Statistician & Machine Learning Scientist | H2O.ai