Cannot install pyarrow in fresh Anaconda 2020.11 installation

[Richard Stanton]

I just installed Anaconda 2020.11 on both a Mac and a Linux machine today (with freshly downloaded installation packages), and find that I cannot install pyarrow on either machine. Here's what happens when I run "conda install pyarrow":

Collecting package metadata (current_repodata.json): done

Solving environment: failed with initial frozen solve. Retrying with flexible solve.

Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.

Collecting package metadata (repodata.json): done

Solving environment: failed with initial frozen solve. Retrying with flexible solve.

Solving environment: / 

Found conflicts! Looking for incompatible packages.

This can take several minutes.  Press CTRL-C to abort.

Examining conflict for matplotlib ...

[... wait a long time...]

failed                                                                                       

UnsatisfiableError: The following specifications were found

to be incompatible with the existing python installation in your environment:

Specifications:

  - alabaster -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - anaconda==2020.11 -> python[version='3.7.9|3.8.5',build='h26836e1_0|h26836e1_1']

  - argon2-cffi -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0']

  - atomicwrites -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - attrs -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - autopep8 -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - babel -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.7,<3.8.0a0|>=3.6,<3.7.0a0']

  - backports -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.7,<3.8.0a0|>=3.6,<3.7.0a0']

  - backports.functools_lru_cache -> python[version='>=2.7,<2.8.0a0']

  - bleach -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - brotlipy -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0']

  - click -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - colorama -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - conda-verify -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - conda[version='>=4.9.2'] -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0']

  - contextlib2 -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']

  - cryptography -> python[version='<=3.3']

  - decorator -> python[version='>=2.7,<2.8.0a0|>=3.7,<3.8.0a0|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0']

  - defusedxml -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']

  - filelock -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - flask -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - glob2 -> python[version='>=2.7,<2.8.0a0|>=3.7,<3.8.0a0|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0']

  - h5py -> python[version='<3']

  - html5lib -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.7,<3.8.0a0|>=3.6,<3.7.0a0']

  - imageio -> python[version='>=3.9,<3.10.0a0']

  - imagesize -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']

  - ipywidgets -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - isort -> python[version='>=3.9,<3.10.0a0']

  - itsdangerous -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - jdcal -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.7,<3.8.0a0|>=3.6,<3.7.0a0']

  - jinja2 -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - jupyter_console -> python[version='>=3.9,<3.10.0a0']

  - matplotlib-base -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0']

  - mock -> python[version='>=3.9,<3.10.0a0']

  - more-itertools -> python[version='>=3.9,<3.10.0a0']

  - nose -> python[version='>=3.9,<3.10.0a0']

  - numba -> python[version='3.4.*|>=3.9,<3.10.0a0']

  - olefile -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']

  - packaging -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - path -> python[version='>=2.7,<2.8.0a0']

  - pickleshare -> python[version='>=3.9,<3.10.0a0']

  - prometheus_client -> python[version='>=2.7,<2.8.0a0|>=3.7,<3.8.0a0|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0']

  - pyarrow -> python[version='>=3.9,<3.10.0a0']

  - pycparser -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0|>=3.7,<3.8.0a0']

  - pyflakes -> python[version='>=3.9,<3.10.0a0']

  - pyopenssl -> python[version='>=3.9,<3.10.0a0']

  - pyparsing -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - pyrsistent -> python[version='>=3.5,<3.6.0a0']

  - python-dateutil -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - pytz -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - qtawesome -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - qtconsole -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - qtpy -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - rope -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - singledispatch -> python[version='>=3.9,<3.10.0a0']

  - snowballstemmer -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']

  - sortedcollections -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - sphinxcontrib-websupport -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']

  - tblib -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']

  - testpath -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - tifffile -> python[version='>=2.7,<2.8.0a0|>=3.9,<3.10.0a0|>=3.5,<3.6.0a0']

  - toml -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - traitlets -> python[version='>=3.9,<3.10.0a0']

  - urllib3 -> python[version='<=3.3|>=3.9,<3.10.0a0']

  - watchdog -> python[version='>=3.5,<3.6.0a0']

  - werkzeug -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.7,<3.8.0a0|>=3.6,<3.7.0a0']

  - wheel -> python[version='>=3.9,<3.10.0a0']

  - wurlitzer -> python[version='<3.4']

  - xlrd -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - xlsxwriter -> python[version='>=2.7,<2.8.0a0|>=3.7,<3.8.0a0|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0']

  - xmltodict -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']

  - yapf -> python[version='>=2.7,<2.8.0a0|>=3.7,<3.8.0a0|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0']

  - zict -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

  - zope.event -> python[version='>=3.5,<3.6.0a0']

Your python: python=3.8

[lots more detail...]

 

[Pete Jemian]

Your installed python version 3.8 (in your base environment) seems to be
the conflict. Here is one example of a package required by pyarrow (or
one of its dependencies) that can't be installed from your list.

On 1/14/2021 4:47 PM, Richard Stanton wrote:
> - alabaster ->
> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']
>

Suggest you might try installing in a new environment for pyarrow:

conda create -n arrow pyarrow


Here's the package list I just got for a linux install with that command:

_libgcc_mutex conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge
_openmp_mutex conda-forge/linux-64::_openmp_mutex-4.5-1_gnu
arrow-cpp
conda-forge/linux-64::arrow-cpp-2.0.0-py38h8084e0b_16_cpu
aws-c-cal conda-forge/linux-64::aws-c-cal-0.4.5-h54aeb68_4
aws-c-common conda-forge/linux-64::aws-c-common-0.4.65-h7f98852_0
aws-c-event-stream
conda-forge/linux-64::aws-c-event-stream-0.2.6-hbb68377_1
aws-c-io conda-forge/linux-64::aws-c-io-0.8.0-h650c6ac_0
aws-checksums conda-forge/linux-64::aws-checksums-0.1.10-h650c6ac_2
aws-sdk-cpp conda-forge/linux-64::aws-sdk-cpp-1.8.122-hf85971d_0
blas pkgs/main/linux-64::blas-1.0-mkl
brotli pkgs/main/linux-64::brotli-1.0.9-he6710b0_2
bzip2 pkgs/main/linux-64::bzip2-1.0.8-h7b6447c_0
c-ares pkgs/main/linux-64::c-ares-1.17.1-h27cfd23_0
ca-certificates
pkgs/main/linux-64::ca-certificates-2020.12.8-h06a4308_0
certifi pkgs/main/linux-64::certifi-2020.12.5-py38h06a4308_0
gflags pkgs/main/linux-64::gflags-2.2.2-he6710b0_0
glog pkgs/main/linux-64::glog-0.4.0-he6710b0_0
grpc-cpp pkgs/main/linux-64::grpc-cpp-1.34.1-h2157cd5_4
intel-openmp pkgs/main/linux-64::intel-openmp-2020.2-254
krb5 pkgs/main/linux-64::krb5-1.18.2-h173b8e3_0
ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.33.1-h53a641e_7
libcurl pkgs/main/linux-64::libcurl-7.71.1-h20c2e04_1
libedit pkgs/main/linux-64::libedit-3.1.20191231-h14c3975_1
libevent conda-forge/linux-64::libevent-2.1.10-hcdb4288_3
libffi pkgs/main/linux-64::libffi-3.3-he6710b0_2
libgcc-ng conda-forge/linux-64::libgcc-ng-9.3.0-h5dbcf3e_17
libgomp conda-forge/linux-64::libgomp-9.3.0-h5dbcf3e_17
libprotobuf pkgs/main/linux-64::libprotobuf-3.14.0-h8c45485_0
libssh2 pkgs/main/linux-64::libssh2-1.9.0-h1ba5d50_1
libstdcxx-ng conda-forge/linux-64::libstdcxx-ng-9.3.0-h2ae2ef3_17
libthrift conda-forge/linux-64::libthrift-0.13.0-h5aa387f_6
libutf8proc conda-forge/linux-64::libutf8proc-2.6.1-h7f98852_0
lz4-c conda-forge/linux-64::lz4-c-1.9.3-h9c3ff4c_0
mkl pkgs/main/linux-64::mkl-2020.2-256
mkl-service pkgs/main/linux-64::mkl-service-2.3.0-py38he904b0f_0
mkl_fft pkgs/main/linux-64::mkl_fft-1.2.0-py38h23d657b_0
mkl_random pkgs/main/linux-64::mkl_random-1.1.1-py38h0573a6f_0
ncurses pkgs/main/linux-64::ncurses-6.2-he6710b0_1
numpy pkgs/main/linux-64::numpy-1.19.2-py38h54aff64_0
numpy-base pkgs/main/linux-64::numpy-base-1.19.2-py38hfa32c7d_0
openssl pkgs/main/linux-64::openssl-1.1.1i-h27cfd23_0
orc conda-forge/linux-64::orc-1.6.6-h7950760_1
parquet-cpp conda-forge/noarch::parquet-cpp-1.5.1-2
pip pkgs/main/linux-64::pip-20.3.3-py38h06a4308_0
pyarrow
conda-forge/linux-64::pyarrow-2.0.0-py38h842ea0c_16_cpu
python pkgs/main/linux-64::python-3.8.5-h7579374_1
python_abi conda-forge/linux-64::python_abi-3.8-1_cp38
re2 pkgs/main/linux-64::re2-2020.11.01-h2531618_1
readline pkgs/main/linux-64::readline-8.0-h7b6447c_0
s2n conda-forge/linux-64::s2n-0.10.24-h9b69904_0
setuptools pkgs/main/linux-64::setuptools-51.1.2-py38h06a4308_4
six pkgs/main/linux-64::six-1.15.0-py38h06a4308_0
snappy pkgs/main/linux-64::snappy-1.1.8-he6710b0_0
sqlite pkgs/main/linux-64::sqlite-3.33.0-h62c20be_0
tk pkgs/main/linux-64::tk-8.6.10-hbc83047_0
wheel pkgs/main/noarch::wheel-0.36.2-pyhd3eb1b0_0
xz pkgs/main/linux-64::xz-5.2.5-h7b6447c_0
zlib pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3
zstd conda-forge/linux-64::zstd-1.4.8-ha95c52a_1

[Richard Stanton]

Thanks! I'll give that a try.

[Richard Stanton]

I really need a lot of the other packages in the full anaconda install (pandas, etc.) as well as just pyarrow itself. Is there a way to install all of anaconda, subject to the version of everything installed being compatible with pyarrow?

Thanks again!

[Pete Jemian]

You can add them to the `arrow` environment. I'm not sure how to
proceed with installing into the base environment. Seems you already
got an answer on that when you tried it.

Got any other ideas, Anaconda support folks?

Pete

[Richard Stanton]

After running “condo update anaconda”, I was able to install pyarrow using conda, but only version 0.15.1. Then I uninstalled pyarrow and tried “pip install pyarrow”. This installed version 2.0.0, which seems to work OK. Not perfect, but at least it’s a work-around.

[Pete Jemian]

Good move, that. Despite other best practice recommendations, this is a
great example when pip comes to the rescue!