broken interaction between ibgd, ligtiff and pyproj

58 views

Skip to first unread message

Chris Barker

unread,

Dec 27, 2022, 3:15:04 PM12/27/22

to conda-forge

I discovered that if I have an environment with pyproj and ;ibgd, and I update libgd, it updates libtiff,and breaks pyproj.

(Note: I'm seeing this on my Mac, and it's also failing in similarly on Linux in out CI)

I've found this in a very complex environment, but I *think* it's only those few packages interacting

fresh, functioning environment has:

libgd 2.3.3 h1e214de_3 conda-forge
libtiff 4.4.0 h6268bbc_5 conda-forge
proj 9.1.0 hcbd9701_0 conda-forge
py_gd 2.0.3 py39ha30fb19_2 conda-forge
pyproj 3.4.1 py39h2798b3d_0 conda-forge
python 3.9.15 h709bd14_0_cpython conda-forge

If I update libgd:

$ conda update libgd

## Package Plan ##

environment location: /Users/chris.barker/miniconda3/envs/junk

added / updated specs:
- libgd

The following packages will be UPDATED:

libgd 2.3.3-h1e214de_3 --> 2.3.3-h4685329_4 None
libtiff 4.4.0-h6268bbc_5 --> 4.5.0-h6268bbc_0 None
libwebp 1.2.4-hfa4350a_0 --> 1.2.4-h70a068d_1 None

The following packages will be SUPERSEDED by a higher-priority channel:

proj conda-forge::proj-9.1.0-hcbd9701_0 --> pkgs/main::proj-8.2.1-hd69def0_0 None
pyproj conda-forge::pyproj-3.4.1-py39h2798b3~ --> pkgs/main::pyproj-3.3.0-py39h823a480_0 None

Note the update of libtiff, and the downgrade / change in channel for Proj

And now:

$ python -c "import pyproj"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/chris.barker/miniconda3/envs/junk/lib/python3.9/site-packages/pyproj/__init__.py", line 49, in <module>
import pyproj.network
File "/Users/chris.barker/miniconda3/envs/junk/lib/python3.9/site-packages/pyproj/network.py", line 10, in <module>
from pyproj._network import ( # noqa: F401 pylint: disable=unused-import
ImportError: dlopen(/Users/chris.barker/miniconda3/envs/junk/lib/python3.9/site-packages/pyproj/_network.cpython-39-darwin.so, 2): Library not loaded: @rpath/libtiff.5.dylib
Referenced from: /Users/chris.barker/miniconda3/envs/junk/lib/libproj.22.2.1.dylib
Reason: image not found

Maybe I have my channel priorities wrong? (though this has worked for ages...)

$ conda config --show channels
channels:
- conda-forge
- defaults

$ conda config --describe channel_priority
# # channel_priority (ChannelPriority)
# # Accepts values of 'strict', 'flexible', and 'disabled'. The default
# # value is 'flexible'. With strict channel priority, packages in lower
# # priority channels are not considered if a package with the same name
# # appears in a higher priority channel. With flexible channel priority,
# # the solver may reach into lower priority channels to fulfill
# # dependencies, rather than raising an unsatisfiable error. With channel
# # priority disabled, package version takes precedence, and the
# # configured priority of channels is used only to break ties. In
# # previous versions of conda, this parameter was configured as either
# # True or False. True is now an alias to 'flexible'.
# #
# channel_priority: flexible

Thanks,

-CHB

Here is the full, working environment, in case there's some other interaction there...

# packages in environment at /Users/chris.barker/miniconda3/envs/junk:
#
# Name Version Build Channel
attrs 22.2.0 pyh71513ae_0 conda-forge
bzip2 1.0.8 h0d85af4_4 conda-forge
c-ares 1.18.1 h0d85af4_0 conda-forge
ca-certificates 2022.12.7 h033912b_0 conda-forge
cell_tree2d 0.3.0 py39hecff1ad_1006 conda-forge
certifi 2022.12.7 pyhd8ed1ab_0 conda-forge
cftime 1.6.2 py39h7cc1f47_1 conda-forge
colander 1.8.3 pyhd8ed1ab_0 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
coverage 7.0.1 py39ha30fb19_0 conda-forge
curl 7.87.0 h6df9250_0 conda-forge
cython 0.29.32 py39h7a8716b_1 conda-forge
exceptiongroup 1.1.0 pyhd8ed1ab_0 conda-forge
expat 2.5.0 hf0c8a7f_0 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 hab24e00_0 conda-forge
fontconfig 2.14.1 h5bb23bf_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
freetype 2.12.1 h3f81eb7_1 conda-forge
geojson 2.5.0 py_0 conda-forge
geos 3.11.1 hf0c8a7f_0 conda-forge
giflib 5.2.1 hbcb3906_2 conda-forge
gitdb 4.0.10 pyhd8ed1ab_0 conda-forge
gitpython 3.1.29 pyhd8ed1ab_0 conda-forge
gridded 0.3.6 pyhd8ed1ab_2 conda-forge
gsw 3.6.16 py39h7cc1f47_4 conda-forge
hdf4 4.2.15 h7aa5921_5 conda-forge
hdf5 1.12.2 nompi_h48135f9_101 conda-forge
icu 70.1 h96cf925_0 conda-forge
iniconfig 1.1.1 pyh9f0ad1d_0 conda-forge
iso8601 1.1.0 pyhd8ed1ab_0 conda-forge
jpeg 9e hac89ed1_2 conda-forge
krb5 1.20.1 h049b76e_0 conda-forge
lerc 4.0.0 hb486fe8_0 conda-forge
libaec 1.0.6 he49afe7_0 conda-forge
libblas 3.9.0 16_osx64_openblas conda-forge
libcblas 3.9.0 16_osx64_openblas conda-forge
libcurl 7.87.0 h6df9250_0 conda-forge
libcxx 14.0.6 hccf4f1f_0 conda-forge
libdeflate 1.14 hb7f2c08_0 conda-forge
libedit 3.1.20191231 h0678c8f_2 conda-forge
libev 4.33 haf1e3a3_1 conda-forge
libffi 3.4.2 h0d85af4_5 conda-forge
libgd 2.3.3 h1e214de_3 conda-forge
libgfortran 5.0.0 11_3_0_h97931a8_27 conda-forge
libgfortran5 11.3.0 h082f757_27 conda-forge
libiconv 1.17 hac89ed1_0 conda-forge
liblapack 3.9.0 16_osx64_openblas conda-forge
libnetcdf 4.8.1 nompi_hc61b76e_106 conda-forge
libnghttp2 1.47.0 h5aae05b_1 conda-forge
libopenblas 0.3.21 openmp_h429af6e_3 conda-forge
libpng 1.6.39 ha978bb4_0 conda-forge
libsodium 1.0.18 hbcb3906_1 conda-forge
libsqlite 3.40.0 ha978bb4_0 conda-forge
libssh2 1.10.0 h47af595_3 conda-forge
libtiff 4.4.0 h6268bbc_5 conda-forge
libwebp 1.2.4 hfa4350a_0 conda-forge
libwebp-base 1.2.4 h775f41a_0 conda-forge
libxml2 2.10.3 hb9e07b5_0 conda-forge
libzip 1.9.2 h6db710c_1 conda-forge
libzlib 1.2.13 hfd90126_4 conda-forge
llvm-openmp 15.0.6 h61d9ccf_0 conda-forge
mapbox_earcut 1.0.0 py39h92daf61_5 conda-forge
ncurses 6.3 h96cf925_1 conda-forge
netcdf4 1.6.2 nompi_py39h0d363ce_100 conda-forge
numpy 1.24.0 py39hdfa1d0c_0 conda-forge
openssl 3.0.7 hfd90126_1 conda-forge
packaging 22.0 pyhd8ed1ab_0 conda-forge
pip 22.3.1 pyhd8ed1ab_0 conda-forge
pluggy 1.0.0 pyhd8ed1ab_5 conda-forge
progressbar 2.5 py_0 conda-forge
proj 9.1.0 hcbd9701_0 conda-forge
psutil 5.9.4 py39ha30fb19_0 conda-forge
py_gd 2.0.3 py39ha30fb19_2 conda-forge
pynucos 3.1.1 pyhd8ed1ab_0 conda-forge
pyproj 3.4.1 py39h2798b3d_0 conda-forge
pyshp 2.3.1 pyhd8ed1ab_0 conda-forge
pytest 7.2.0 pyhd8ed1ab_2 conda-forge
pytest-cov 4.0.0 pyhd8ed1ab_0 conda-forge
pytest-timeout 2.1.0 pyhd8ed1ab_0 conda-forge
python 3.9.15 h709bd14_0_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python_abi 3.9 3_cp39 conda-forge
pyzmq 24.0.1 py39hed8f129_1 conda-forge
readline 8.1.2 h3899abd_0 conda-forge
scipy 1.9.3 py39h8a15683_2 conda-forge
setuptools 65.6.3 pyhd8ed1ab_0 conda-forge
shapely 2.0.0 py39hed42127_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
smmap 3.0.5 pyh44b312d_0 conda-forge
sqlite 3.40.0 h9ae0607_0 conda-forge
tblib 1.7.0 pyhd8ed1ab_0 conda-forge
testfixtures 7.0.4 pyhd8ed1ab_0 conda-forge
tk 8.6.12 h5dbffcc_0 conda-forge
toml 0.10.2 pyhd8ed1ab_0 conda-forge
tomli 2.0.1 pyhd8ed1ab_0 conda-forge
translationstring 1.4 pyh9f0ad1d_0 conda-forge
trimesh 3.17.1 pyhd8ed1ab_0 conda-forge
typing_extensions 4.4.0 pyha770c72_0 conda-forge
tzdata 2022g h191b570_0 conda-forge
wheel 0.38.4 pyhd8ed1ab_0 conda-forge
xz 5.2.6 h775f41a_0 conda-forge
zeromq 4.3.4 he49afe7_1 conda-forge
zlib 1.2.13 hfd90126_4 conda-forge
zstd 1.5.2 hfa58983_4 conda-forge

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA 98115   (206) 526-6317   main reception

Chris....@noaa.gov

Xylar Asay-Davis

unread,

Dec 28, 2022, 5:53:50 AM12/28/22

to Chris Barker, conda-forge

Hi Chris,

I'm not sure why you're getting downgraded instead of upgraded. But I would always recommend that you start with creating a fresh conda (or better yet mamba) environment when you run into solver troubles. I had luck with this (at least in that I get recent versions, as expected):

$ mamba create -y -n test libgd=2.3.3 libtiff=4.5.0 libwebp=1.2.4 proj=9.1.1 pyproj

$ mamba activate test

$ mamba list
# packages in environment at /gpfs/fs1/home/ac.xylar/chrysalis/mambaforge/envs/test:

#
# Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
c-ares 1.18.1 h7f98852_0 conda-forge
ca-certificates 2022.12.7 ha878542_0 conda-forge
certifi 2019.11.28 py27h8c360ce_1 conda-forge
expat 2.5.0 h27087fc_0 conda-forge

font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 hab24e00_0 conda-forge

fontconfig 2.14.1 hc2a2eb6_0 conda-forge

fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge

freetype 2.12.1 hca18f0e_1 conda-forge
giflib 5.2.1 h36c2ea0_2 conda-forge
icu 70.1 h27087fc_0 conda-forge
jpeg 9e h166bdaf_2 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
krb5 1.20.1 hf9c8cef_0 conda-forge
ld_impl_linux-64 2.39 hcc3a1bd_1 conda-forge
lerc 4.0.0 h27087fc_0 conda-forge
libblas 3.9.0 16_linux64_openblas conda-forge
libcblas 3.9.0 16_linux64_openblas conda-forge
libcurl 7.87.0 h6312ad2_0 conda-forge
libdeflate 1.14 h166bdaf_0 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 h516909a_1 conda-forge
libffi 3.2.1 he1b5a44_1007 conda-forge
libgcc-ng 12.2.0 h65d4601_19 conda-forge
libgd 2.3.3 h5aea950_4 conda-forge
libgfortran-ng 12.2.0 h69a702a_19 conda-forge
libgfortran5 12.2.0 h337968e_19 conda-forge
libgomp 12.2.0 h65d4601_19 conda-forge
liblapack 3.9.0 16_linux64_openblas conda-forge
libnghttp2 1.47.0 hdcd2b5c_1 conda-forge
libopenblas 0.3.21 pthreads_h78a6416_3 conda-forge
libpng 1.6.39 h753d276_0 conda-forge
libsqlite 3.40.0 h753d276_0 conda-forge
libssh2 1.10.0 haa6b8db_3 conda-forge
libstdcxx-ng 12.2.0 h46fd767_19 conda-forge
libtiff 4.5.0 h82bc61c_0 conda-forge
libuuid 2.32.1 h7f98852_1000 conda-forge
libwebp 1.2.4 h1daa5a0_1 conda-forge
libwebp-base 1.2.4 h166bdaf_0 conda-forge
libzlib 1.2.13 h166bdaf_4 conda-forge
ncurses 6.3 h27087fc_1 conda-forge
numpy 1.16.5 py27h95a1406_0 conda-forge
openssl 1.1.1s h0b41bf4_1 conda-forge
pip 20.1.1 pyh9f0ad1d_0 conda-forge
proj 9.1.1 h8ffa02c_1 conda-forge
pyproj 1.9.4 py27_0 conda-forge
python 2.7.15 h5a48372_1011_cpython conda-forge
python_abi 2.7 1_cp27mu conda-forge
readline 8.1.2 h0f457ee_0 conda-forge
setuptools 44.0.0 py27_0 conda-forge
sqlite 3.40.0 h4ff8645_0 conda-forge
tk 8.6.12 h27826a3_0 conda-forge
wheel 0.37.1 pyhd8ed1ab_0 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
zlib 1.2.13 h166bdaf_4 conda-forge
zstd 1.5.2 h6239696_4 conda-forge

Let me know if that works for you.

Cheers,

Xylar

--
You received this message because you are subscribed to the Google Groups "conda-forge" group.
To unsubscribe from this group and stop receiving emails from it, send an email to conda-forge...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/conda-forge/CALGmxELa6zq4cDgynHB-BjOxt5s%3DmdG1rBZKErN6hepUxrwYhQ%40mail.gmail.com.

Xylar Asay-Davis

unread,

Dec 28, 2022, 5:54:48 AM12/28/22

to Chris Barker, conda-forge

Oops, I just noticed that `pyproj` is super old. Weird!

Xylar Asay-Davis

unread,

Dec 28, 2022, 5:59:00 AM12/28/22

to Chris Barker, conda-forge

Hi Chris,

Sorry for the confusion. I now understand what the deal is. proj has only been built for libtiff 4.5 as of version 9.1.1. But we decided not to build pyproj for proj 9.1.1, see:

https://github.com/conda-forge/pyproj-feedstock/pull/129

https://github.com/conda-forge/pyproj-feedstock/issues/126

https://github.com/pyproj4/pyproj/issues/1192

An easy solution will be to build proj 9.1.0 with libtiff 4.5. I'll do a PR for that as soon as I can.

Cheers,

Xylar

Chris Barker

unread,

Dec 28, 2022, 2:20:47 PM12/28/22

to Xylar Asay-Davis, conda-forge

On Wed, Dec 28, 2022 at 2:53 AM Xylar Asay-Davis <xylar...@gmail.com> wrote:

I'm not sure why you're getting downgraded instead of upgraded. But I would always recommend that you start with creating a fresh conda (or better yet mamba) environment when you run into solver troubles.

well, yes -- as a rule, I find it works MUCH better to always create a fresh environment with all the dependencies -- or at least pass in all the dependencies when you install or upgrade a package. In fact, in this case, the "working" environment was built from scratch, all at once, with a large pile of dependencies.

But in our CI, we were doing a selective upgrade of py_gd, because of an old bug -- I just removed that and the problem is solved, but this did strike me as mismatch somewhere ...

Which it seems you have found and fixed -- thanks!

All good now :-)

-CHB

Reply all

Reply to author

Forward

0 new messages