You Are Running 32-bit Python On A 64-bit Os Xgboost

[Eliecer Brathwaite]

Ihave installed xgboost successfully using pip for Python 2.7.16 (I installed this Python version using Homebrew on macOS High Sierra). My problem is that I'm unable to import xgboost in Python, as per the error message below:

I've made sure that both my Python version and OS are 64-bit, so the problem is definitely not the second "likely cause." I'm also pretty sure I have OpenMP installed: looking up instructions, for my case I would do brew install llvm and the install was successful. I'm also not sure why it can't seem to find /usr/local/lib/python2.7/site-packages/xgboost/./lib/libxgboost.dylib, as I can cd into that directory and see libxgboost.dylib just fine. It looks like later on in the message it did find it, but still throws an error? What exactly is going on here, and what can I do to fix this error?

XGBoost provides binary packages for some language bindings. The binary packages supportthe GPU algorithm (device=cuda:0) on machines with NVIDIA GPUs. Please note thattraining with multiple GPUs is only supported for Linux platform. SeeXGBoost GPU Support. Also we have both stable releases and nightly builds, see below for howto install them. For building from source, visit this page.

We also provide experimental pre-built binary with GPU support. With this binary,you will be able to use the GPU algorithm without building XGBoost from the source.Download the binary package from the Releases page. The file name will be of the formxgboost_r_gpu_[os]_[version].tar.gz, where [os] is either linux or win64.(We build the binaries for 64-bit Linux and Windows.)Then install XGBoost by running:

Other than standard CRAN installation, we also provide experimental pre-built binary onwith GPU support. You can go to this page, Find the commitID you want to install and then locate the file xgboost_r_gpu_[os]_[commit].tar.gz,where [os] is either linux or win64. (We build the binaries for 64-bit Linuxand Windows.) Download it and run the following commands:

The SNAPSHOT JARs are hosted by the XGBoost project. Every commit in the master branch will automatically trigger generation of a new SNAPSHOT JAR. You can control how often Maven should upgrade your SNAPSHOT installation by specifying updatePolicy. See here for details.

Install a recent version of Python (3.9 or later at the time of writing) forinstance using Miniforge3. Miniforge provides a conda-based distribution ofPython and the most popular scientific libraries.

--config-settings editable-verbose=true is optional but recommendedto avoid surprises when you import sklearn. meson-python implementseditable installs by rebuilding sklearn when executing import sklearn.With the recommended setting you will see a message when this happens,rather than potentially waiting without feed-back and wonderingwhat is taking so long. Bonus: this means you only have to run the pipinstall command once, sklearn will automatically be rebuilt whenimporting sklearn.

If OpenMP is not supported by the compiler, the build will be done withOpenMP functionalities disabled. This is not recommended since it will forcesome estimators to run in sequential mode instead of leveraging thread-basedparallelism. Setting the SKLEARN_FAIL_NO_OPENMP environment variable(before cythonization) will force the build to fail if OpenMP is notsupported.

Since version 0.21, scikit-learn automatically detects and uses the linearalgebra library used by SciPy at runtime. Scikit-learn has therefore nobuild dependency on BLAS/LAPACK implementations such as OpenBlas, Atlas, Blisor MKL.

Secondly, find out if you are running 64-bit or 32-bit Python. The buildingcommand depends on the architecture of the Python interpreter. You can checkthe architecture by running the following in cmd or powershellconsole:

For Apple Silicon M1 hardware, only the conda-forge method below is known towork at the time of writing (January 2021). You can install the macos/arm64distribution of conda using the miniforge installer

If you get any conflicting dependency error message, try commenting outany custom conda configuration in the $HOME/.condarc file. Inparticular the channel_priority: strict directive is known to causeproblems for this setup.

Installing scikit-learn from source without using conda requires you to haveinstalled the scikit-learn Python development headers and a working C/C++compiler with OpenMP support (typically the GCC toolchain).

Cython and the pre-compiled wheels for the runtime dependencies (numpy, scipyand joblib) should automatically be installed in$HOME/.local/lib/pythonX.Y/site-packages. Alternatively you can run theabove commands from a virtualenv or a conda environment to get fullisolation from the Python packages installed via the system packager. Whenusing an isolated environment, pip3 should be replaced by pip in theabove commands.

If your compiler is not listed here, you can specify it through some environmentvariables (does not work on windows). This sectionof the setuptools documentation explains in details which environment variablesare used by setuptools, and at which stage of the compilation, to set thecompiler and linker options.

I am using Random Forest algorithm for classification using build_forest() and then apply_forest().

As these operations are running on only one process, how could i parallelize these operations?

And how can i generate graph for the same?

If so, it looks like the forest training is already parallelized through the @parallel macro, so you would only have to run addprocs() before training your model and then build_forest should use multiple workers (Check the link below for the package source code).

How can I do parrallel computing with modules imported using PyCall and @pyimport. I am trying to do something like this but it does not work. If I make n_jobs >1, and remove 3rd line of code from top (@everywhere (@pyimport lightgbm as lgb) ) it still uses single processor. I am using julia 1.0 on win 10 64 bit.

@bernhard

Thanks it does starts parrallel processing. However, I am not gaining any improvement in speed by doing this. I have data size of about ( 220K,28). Without parallel processing one single run takes about 10.6s. With parallel processing feature on, it takes about 16s.

I agree that calling python models in Julia may not be as efficient as native Julia models but I have no choice and hence my current exploration. In Python with the help of Cython, I was able to run 10 iterations of 3-fold CV in approx 95 seconds that too using single core.

Hm, I do not know if there is a native xgboost or gbm implementation in Julia. It does not seem to be the case. Maybe you can find something on

But I guess you already searched and opted for PyCall (an alternative would of course by RCall).

I have an experimental Julia package which includes a regression boosting approach, but it is not well documented and may likely not fit your purpose (GitHub - kafisatz/DecisionTrees.jl: Julia Decision Tree Algorithms for Regression )

What I meant was that what changes would be required for using parrallel computing inside a function call to the model being fit. You have already shared a code for wrapping the model fit inside the function call. Where and what code do I put inside or outside the code to use parrallel computing in this case.

It (or rather, the software running on it) will use arbitrary-precision ("bignum") arithmetic. The way this works is basically the same way in which you (probably) learned to do arithmetic on paper at school.

You may have also learned similar pencil-and-paper algorithms for other elementary arithmetic operations, like subtraction, multiplication and even long division (which can also be used for modular reduction). The important thing to realize is that all these calculation methods are based on rules for manipulating strings of digits, and on simple arithmetic operations on single digits. As long as you can do the basic single-digit operations, and know the algorithm for combining them together, you can do arithmetic on paper with numbers as large as you want (or need) to!

So how do computers do it, then? They could of course use exactly the same decimal arithmetic rules as we do, but that would be pretty inefficient. A typical CPU already has fast circuitry to add or multiply together any two 32 or 64 bit (and possibly even larger) numbers with a single machine code instruction, so it makes a lot more sense to treat 32 or 64 bit arithmetic as the basic building block.

Thus, a typical computer implementation of bignum arithmetic effectively works with "digits" that are 32 or 64 bit integers, and represents larger numbers as strings of those smaller integers. The algorithms used are very similar to those we'd use for pencil-and-paper calculation, except that instead of base 10, a computer is far more likely to use base 232 or 264.

A modern CPU might be able to calculate that directly, but let's assume that we only have a 32-bit CPU available. Fortunately, we can break up our numbers into 32-bit chunks and use the same addition algorithm we used before:

I used addition for this simple example, but similar algorithms can be used for other arithmetic operations, including exponentiation and modular reduction (which one would normally combine into a single modular exponentiation algorithm, since it's much more efficient to do them together rather that to first exponentiate and then reduce).

Also note that the arithmetic algorithms used for cryptography tend to be somewhat specialized, since in crypto it's often important to avoid timing attacks and other types of side-channel attacks by making sure that the algorithm takes the same amount of time to run (and consumes roughly the same amount of power, etc.) regardless of what the numbers being added (or multiplied or raised to a power, etc.) are.

Of course the processor cannot process such large numbers directly; this is done though a library such as GMP. See Wikipedia for a list of such libraries, and a good textbook such as that of Gerhard and von zur Gathen for the underlying ideas. The freely available Handbook of Applied Cryptography also talks about this, especially in Chapter 14.

3a8082e126