numpy 1.13 md5 mismatch

[Matt Craig]

Hi,

A student (on Windows) had reported this to me and I've just confirmed it (on Mac)...with conda 4.3.22 I get this result when I try creating a simple environment with numpy 1.13:

$ conda create -n bad-numpy numpy=1.13 python=3

Fetching package metadata .....................

Solving package specifications: .

Package plan for installation in environment /Users/mcraig/anaconda/envs/bad-numpy:

The following NEW packages will be INSTALLED:

    mkl:        2017.0.1-0

    numpy:      1.13.0-py36_0

    openssl:    1.0.2l-0

    pip:        9.0.1-py36_1

    python:     3.6.1-2

    readline:   6.2-2

    setuptools: 27.2.0-py36_0

    sqlite:     3.13.0-0

    tk:         8.5.18-0

    wheel:      0.29.0-py36_0

    xz:         5.2.2-1

    zlib:       1.2.8-3

Proceed ([y]/n)?

numpy-1.13.0-p 100% |##############################################################| Time: 0:00:00  10.55 MB/s

numpy-1.13.0-p 100% |##############################################################| Time: 0:00:00  11.79 MB/s

numpy-1.13.0-p 100% |##############################################################| Time: 0:00:00  11.77 MB/s

CondaError: MD5MismatchError: Conda detected a mismatch between the expected content and downloaded content

for url 'https://repo.continuum.io/pkgs/free/osx-64/numpy-1.13.0-py36_0.tar.bz2'.

  download saved to: /Users/mcraig/anaconda/pkgs/numpy-1.13.0-py36_0.tar.bz2

  expected md5 sum: d19afd82a434d9c48a8c58aa0de65116

  actual md5 sum: 881e4baa497ea7749dc210db9fabc3b7

CondaError: MD5MismatchError: Conda detected a mismatch between the expected content and downloaded content

for url 'https://repo.continuum.io/pkgs/free/osx-64/numpy-1.13.0-py36_0.tar.bz2'.

  download saved to: /Users/mcraig/anaconda/pkgs/numpy-1.13.0-py36_0.tar.bz2

  expected md5 sum: d19afd82a434d9c48a8c58aa0de65116

  actual md5 sum: 881e4baa497ea7749dc210db9fabc3b7

CondaError: MD5MismatchError: Conda detected a mismatch between the expected content and downloaded content

for url 'https://repo.continuum.io/pkgs/free/osx-64/numpy-1.13.0-py36_0.tar.bz2'.

  download saved to: /Users/mcraig/anaconda/pkgs/numpy-1.13.0-py36_0.tar.bz2

  expected md5 sum: d19afd82a434d9c48a8c58aa0de65116

  actual md5 sum: 881e4baa497ea7749dc210db9fabc3b7

I have not tried with other versions of python...there is no issue making an environment with numpy 1.12.

Matt Craig

[Matt Craig]

I just tried numpy 1.13/python 3.5, no problem there...(on mac)

[Crystal Soja]

Hi Matt,

Thank you for bringing this to our attention.  I was not able to reproduce the problem on my Mac or on a Windows 64-bit VM.  (using conda create -n bad-numpy numpy=1.13 python=3) Looking at https://repo.continuum.io/pkgs/free/osx-64/ - the md5sum looks like it should be d19afd82a434d9c48a8c58aa0de65116 for that file.  Perhaps you have a previous or bad version in your local package cache - can you remove /Users/mcraig/anaconda/pkgs/numpy-1.13.0-py36_0.tar.bz2 and try to create the environment again?

Thanks,

Crystal

[Matt Craig]

Hi Crystal,

It looks like it is a weird local network issue. 

When I download the package while connected to my campus' network, I consistently get the wrong checksum, 881e4baa497ea7749dc210db9fabc3b7. If I download the same file tethered to my phone or from home, I get the correct checksum d19afd82a434d9c48a8c58aa0de65116. 

Oddly, I don't get the mismatch for a couple other packages I spot-checked, but it seems clear the issue is not package but our network, or perhaps to a difference in DNS or something that leads to different servers?

The difference between the two downloads isn't random gibberish (campus- is the numpy package downloaded on our campus network):

diff -r numpy-1.13.0-py36_0/info/index.json campus-numpy-1.13.0-py36_0/info/index.json

6c6

<     "mkl 2017.0.*",

---

>     "mkl 2017.0.1",

Only in numpy-1.13.0-py36_0/info: index.json.org

Only in campus-numpy-1.13.0-py36_0/info: paths.json

Anyway, I understand this is likely a "me" problem, and will report it to our campus IT.

Thanks,

Matt

--
Anaconda Community Support Group Brought to you by Continuum Analytics
---
You received this message because you are subscribed to a topic in the Google Groups "Anaconda - Public" group.
To unsubscribe from this topic, visit https://groups.google.com/a/continuum.io/d/topic/anaconda/0CxPrxjPtqo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to anaconda+unsubscribe@continuum.io.
To post to this group, send email to anac...@continuum.io.
Visit this group at https://groups.google.com/a/continuum.io/group/anaconda/.

[Ian Stokes Rees]

Hi Matt,

This is weird.  I wonder if your campus network has somehow cached the package but not the MD5 file (or vice-versa), and on our end we have somehow updated the package and MD5 file for the same URL.  My sense was that we would never do this: at most we would remove "problem" packages so they are simply not available any more (bad builds, security vulnerability, etc.), but any replacement would have a new URL (probably an incremented build number).

Ian

You received this message because you are subscribed to the Google Groups "Anaconda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to anaconda+u...@continuum.io.

To post to this group, send email to anac...@continuum.io.
Visit this group at https://groups.google.com/a/continuum.io/group/anaconda/.

--

Ian Stokes-Rees, PhD
Computational Scientist
1.617.942.0218
ijst...@continuum.io
@ContinuumIO @ijstokes

[Ian Stokes Rees]

OK, I think we know what's going on here.

The c3b7 hash is from the pre-June-20th version of that package which your campus network has cached (probably based on file size, possibly also based on the caching policy we serve with the content in the HTTP headers).  We discovered that there were errors in the meta-data, so the package was updated to *only* change the meta-data, but since meta-data is embedded in the conda package the MD5 sum needs to be updated, and the 5116 hash is the correct hash for the new package.

Presently we perform these kind of "hotfix" package updates "in-place" so the URL doesn't change.  This means same package version and build number.  While conda package dependencies don't specify the expected MD5 hash they often *do* specify an exact build number.  It is for that reason that we will, under certain limited circumstances, do an inplace "hotfix" of a package and revise its associated MD5 sum.

I hope that helps you understand what's happened here.  I learned something in the process.

And for anyone else reading this: if you spot MD5 sum mismatches, please report them immediately.  Better safe than sorry.

Thanks,

Ian

On 6/28/17 4:54 PM, Matt Craig wrote:

You received this message because you are subscribed to the Google Groups "Anaconda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to anaconda+u...@continuum.io.

To post to this group, send email to anac...@continuum.io.
Visit this group at https://groups.google.com/a/continuum.io/group/anaconda/.

[Matt Craig]

Hi Ian,

That makes sense -- it certainly seemed like a caching issue. Do you know what platforms the hotfix was applied to? Looks like at least mac and win-64, though I haven't checked all of the other platforms.

I'm a little surprised it is caching based on size since the difference is ~2kB, but I'll ask about the caching policies and see if they can clear it.

In the mean time, do you have any advice for working around it? It came up originally in preparing installation instructions for some students and for a scipy tutorial. 

Right now the student can't install the combination numpy 1.13/python 3.6 because of the md5 mismatch. I can, of course, work around it by pinning numpy to 1.12 (which is what I did for the scipy install instructions) but if there is any other way to do it that would be helpful to know.

For the future, perhaps hot fixes could be accompanied by releasing a new build with an incremented build number? That way packages that have pinned to the build number continue to work, but anyone who would otherwise hit a cached copy gets the new build number.

I'm sure you guys have thought a lot more about this than I have and have weighed to cost/benefits of hot fixing vs bumping the build number, but it seems like hot fixes potentially make it more difficult to reproduce work.

Anyway, if you have ideas for a workaround please let me know, and thanks for diggetting into this!

Matt

To unsubscribe from this group and stop receiving emails from it, send an email to anaconda+unsubscribe@continuum.io.

To post to this group, send email to anac...@continuum.io.
Visit this group at https://groups.google.com/a/continuum.io/group/anaconda/.

--

Ian Stokes-Rees, PhD

Computational Scientist
1.617.942.0218
ijst...@continuum.io
@ContinuumIO @ijstokes

--

[Ian Stokes Rees]

Hi Matt,

On 6/28/17 6:17 PM, Matt Craig wrote:

Hi Ian,

That makes sense -- it certainly seemed like a caching issue. Do you know what platforms the hotfix was applied to? Looks like at least mac and win-64, though I haven't checked all of the other platforms.

Unfortunately I do not.  I am not part of the team that looks after those things.  They nominally monitor this mailing list and may chime in with more details.

I'm a little surprised it is caching based on size since the difference is ~2kB, but I'll ask about the caching policies and see if they can clear it.

That is going to be the best course of action for you.

In the mean time, do you have any advice for working around it? It came up originally in preparing installation instructions for some students and for a scipy tutorial.

I understand your problem.  I don't have good advice beyond the strategy you've identified: using the v1.12 of Numpy.

Your suggestion sounds like a good one and I'll pass it on to the relevant people: having a fresh higher build number package that was never hot-fixed, so a "virgin" conda install command will pick it up as the "latest" numpy release.

Thanks, Ian

[ode...@gmail.com]

I had the same problem - Anaconda 4.3.22 - Windows 7 64-Bit

I was able to get around it by connecting to a VPN location out of Chicago. When I run a traceroute on my local machine through my ISP, the final IP is a Cloudflare source. https://whois.arin.net/rest/net/NET-104-16-0-0-1/pft?s=104.16.19.10

It appears that Cloudflare must have an outdated cache of the conda repo.

On Tuesday, June 27, 2017 at 12:28:14 PM UTC-5, Matt Craig wrote:

[Sumeet K]

I can corroborate this. Same issue with numpy, I'm in Madison (close to Chicago). I too avoided the problem for the time being using the VPN trick.

[hay...@gmail.com]

I had the same problem.  Been trying to install geopandas and getting the numpy1.12 checksum problem.  Was able to get around it when I found this thread.  I was going through my home network, but when I switched to my iPhone as a hotspot, I was able to install.

[Matt Craig]

FYI, separate thread started at: https://groups.google.com/a/continuum.io/forum/#!topic/anaconda/V6Hw-iNMikA

Matt