Where can we register the xgboost backend for version >3.12.0.1 of H2O

[Roberto Rösler]

Hi,

since version 3.12.0.1 it looks like xgboost is now available in h2o using a plugin mechanism. I've installed h2o and xgboost. But when I run h2o in python it can't find the backend.

>>> xgboostModel = h2o.estimators.xgboost.H2OXGBoostEstimator()
>>> xgboostModel.available()
Cannot build an XGBoost model - no backend found.
False

Looking at the documentation (http://h2o-release.s3.amazonaws.com/h2o/rel-vapnik/1/docs-website/h2o-py/docs/modeling.html#h2oxgboostestimator, https://github.com/h2oai/h2o-tutorials/blob/master/tutorials/ensembles-stacking/stacked_ensemble_h2o_xgboost.Rmd) I don't see any information on how to register the backend. So the question goes to the h2o guys on how to register it.

Many thanks,

Roberto

[Erin LeDell]

Hi Roberto,

Can you confirm that you are connecting to a 3.12 H2O cluster by printing the output from: h2o.cluster().show_status()

-Erin

--
You received this message because you are subscribed to the Google Groups "H2O Open Source Scalable Machine Learning - h2ostream" group.
To unsubscribe from this group and stop receiving emails from it, send an email to h2ostream+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
Erin LeDell Ph.D.
Statistician & Machine Learning Scientist | H2O.ai

[Roberto Rösler]

Hi Erin,

yes I can confirm this:

>>> h2o.cluster().show_status()
--------------------------  -------------------------------
H2O cluster uptime:         1 hour 48 mins
H2O cluster version:        3.12.0.1
H2O cluster version age:    4 days
H2O cluster name:           H2O_from_python_roroesle_iljtg5
H2O cluster total nodes:    1
H2O cluster free memory:    1.697 Gb
H2O cluster total cores:    8
H2O cluster allowed cores:  2
H2O cluster status:         locked, healthy
H2O connection url:         http://127.0.0.1:54321
H2O connection proxy:
H2O internal security:      False
Python version:             3.5.3 final
--------------------------  -------------------------------

Best,

Roberto

[Erin LeDell]

Roberto,

Thanks for confirming.  I will get back to you on this... 

-Erin

[Tom Kraljevic]

hi,

it would also be helpful to see the actual h2o logs to see if the dynamically linked libraries are loading properly.

(note this is the first release of h2o-3 thats had native libraries packaged, so there could be kinks to work out.  as such, it's billed as a "preview release" right now.)

thanks!

tom

Sent from my iPhone

[Roberto Rösler]

Hi,

please find the logs attached. Interesting to see (h2o_127.0.0.1_54321-4-warn.log):

06-11 20:09:44.168 127.0.0.1:54321       10296  #93302-17 WARN: Cannot initialize XGBoost backend! Xgboost (enabled GPUs) needs:
06-11 20:09:44.168 127.0.0.1:54321       10296  #93302-17 WARN:   - CUDA 8.0
06-11 20:09:44.168 127.0.0.1:54321       10296  #93302-17 WARN: XGboost (minimal version) needs:
06-11 20:09:44.168 127.0.0.1:54321       10296  #93302-17 WARN:   - no req
06-11 20:09:44.168 127.0.0.1:54321       10296  #93302-17 WARN: java.io.IOException: java.io.IOException: java.io.FileNotFoundException: File lib/xgboost4j.dll was not found inside JAR.
06-12 08:34:40.678 127.0.0.1:54321       10296  #93302-19 WARN: Cannot initialize XGBoost backend! Xgboost (enabled GPUs) needs:
06-12 08:34:40.678 127.0.0.1:54321       10296  #93302-19 WARN:   - CUDA 8.0
06-12 08:34:40.678 127.0.0.1:54321       10296  #93302-19 WARN: XGboost (minimal version) needs:
06-12 08:34:40.678 127.0.0.1:54321       10296  #93302-19 WARN:   - no req
06-12 08:34:40.678 127.0.0.1:54321       10296  #93302-19 WARN: java.io.IOException: java.io.IOException: java.io.FileNotFoundException: File lib/xgboost4j.dll was not found inside JAR.

Best,

Roberto

[Michal Malohlava]

Hi Roberto,

it seems that xgboost native libs cannot be loaded since we do not deliver Windows native libraries for XGBoost.

It will be fixed in the following release.

Michal

[Erin LeDell]

Hi Roberto,

Not all platforms are supported (because of binary artifacts we deliver with h2o.ai).  We did the best job we could to make it compatible with old linux distributions, however, if you have gcc4.6 and compatible libg++, XGBoost will not link, and H2O will report that XGBoost is not available.

If you have different specs (and/or if you are running on a mac), let us know, so we can document the systems which are lacking support.  We will try to make a list of coverage and do what we can to address the gaps.

-Erin

[Roberto Rösler]

Hi Michal,

thanks that is the answer I had looked for. If I need to have a special set of libs and bins available  (like g++, gcc, ...) than it would be great if you can add this info to the documentation.

Is there a plan for the next release - still in June?

Best,

Roberto

[Michal Malohlava]

Hi Roberto,

we have aprox 2weeks release cycle for fix release. It can be part of the next one. We are still waiting for a few changes in xgboost.

Best regards,
Michal

[nsri...@gmail.com]

I am getting a similar message in R with H2O. I have the xgboost library installed separately. Should I be linking the two somehow?
[1] "Cannot build a XGboost model - no backend found."
[1] FALSE

Here is the cluster info:

R is connected to the H2O cluster:
H2O cluster uptime: 14 minutes 6 seconds
H2O cluster version: 3.14.0.1
H2O cluster version age: 11 days
H2O cluster name: H2O_started_from_R_MBAUser_nbj966

H2O cluster total nodes: 1

H2O cluster total memory: 8.80 GB
H2O cluster total cores: 4
H2O cluster allowed cores: 4
H2O cluster healthy: TRUE
H2O Connection ip: localhost
H2O Connection port: 54321
H2O Connection proxy: NA
H2O Internal Security: FALSE
H2O API Extensions: Algos, AutoML, Core V3, Core V4
R Version: R version 3.4.1 (2017-06-30)

[Erin LeDell]

No,

You don't need to link the two -- H2O ships with it's own XGboost code,
it should just work.

Can you provide more information about the system that you're running so
we can help debug the issue?

If the XGBoost extension is working, it will show up when you type:

> h2o.clusterInfo()

R is connected to the H2O cluster:

H2O cluster uptime: 3 days 3 hours
H2O cluster version: 3.15.0.99999

H2O cluster version age: 4 days

H2O cluster name: H2O_started_from_R_me_iqv833

H2O cluster total nodes: 1

H2O cluster total memory: 3.24 GB

H2O cluster total cores: 8

H2O cluster allowed cores: 8

H2O cluster healthy: TRUE
H2O Connection ip: localhost
H2O Connection port: 54321
H2O Connection proxy: NA
H2O Internal Security: FALSE

H2O API Extensions: XGBoost, Algos, AutoML, Core V3, Core V4
R Version: R version 3.3.2 (2016-10-31)

[Sriram Narayanamoorthy]

I am using a laptop running Windows 10. R version is 3.4.1. 

I installed this H2O source: http://h2o-release.s3.amazonaws.com/h2o/rel-weierstrass/1/R

Let me know if you need any other info. Thanks for helping troubleshoot!

--

Sriram Narayanamoorthy

[Erin LeDell]

Ok, you're on Windows, which is why it's not working.

As stated in the "Limitations" section of the XGBoost page in the H2O User Guide, XGBoost is not supported on Windows.  This will be supported in the future (not sure when), but you can track the progress here: https://0xdata.atlassian.net/browse/PUBDEV-4581

-Erin

[Erin LeDell]

Actually, it looks like it's slated to be fixed in the next fix release of H2O, 3.14.0.3.  That could still change, but it's a good sign that we will fix it soon.

-Erin

[laurentr...@gmail.com]

Hi Erin, I installed the latest bleeding edge release (3.15.0.4) on linux (rhel 6) and i am getting the same "no backend" error for xgboost, even though xgboost is installed. i am running it in python 2.7

[joni...@gmail.com]

Hi,

Are there any plans or release dates for Xgboost to work on Windows?

Thanks

[leandr...@gmail.com]

Hi people.

Have news about this discussion?

I'm trying to use Automl by XGBoost but I receive the message: AutoML: XGBoost is not available; skipping it.

I also use Windows 10 with Python 3.6 and H2O version 3.28.0.2.