pypi package install fails in nexus-3.2.1

devops

unread,

Sep 12, 2017, 1:23:38 AM9/12/17

to Nexus Users

Hello Nexus,

I have two nexus instances of version -3.2.1.01 (Dev and production env). Created three pypi repositories in each nexus instances given below

* pypi-all - group repository
* pypi-internal - we host internal repos here
* pypi-proxy - it proxies https://pypi.python.org/

Migrated all pypi packages from pypi registry to nexus pypi repository using twine utility

* twine upload -r nexus <package name> --- Pypi package is in the tar.gz format.

when i uploaded pypi packages to nexus dev instance, it compress the fie to gzip format. But for some reason, compression is not happening in prod instance of nexus. so when i do pip install against dev instance, it installs successfully

* pip install <package-name> --trusted-host <hostname>

But pypi package install fails against prod instance. attached exception erros..

 File "/Users/devops/anaconda/lib/python3.6/gzip.py", line 411, in _read_gzip_header
    raise OSError('Not a gzipped file (%r)' % magic)
OSError: Not a gzipped file (b'sh')

Please let me know, if i am doing anything wrong.

Thanks in advance !!!!!

devops

unread,

Sep 12, 2017, 2:16:42 PM9/12/17

to Nexus Users

I am kind of blocked here. can someone help me out to resolve this issue?

Thanks.

Rich Seddon

unread,

Sep 12, 2017, 2:22:18 PM9/12/17

to Nexus Users

I've seen this sort of problem before when something running in front of Nexus has gzip compression configured incorrectly.

Is there a reverse proxy such as nginx or apache running in front of the production instance?

Brian Fox

unread,

Sep 12, 2017, 2:22:31 PM9/12/17

to devops, Nexus Users

Is there a reverse proxy in front of these instances? (or one but not the other)

--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users+unsubscribe@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.
To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/481fc136-5ec9-4bfa-a724-5dc08f2a3d50%40glists.sonatype.com.

For more options, visit https://groups.google.com/a/glists.sonatype.com/d/optout.

devops

unread,

Sep 12, 2017, 2:26:15 PM9/12/17

to Nexus Users

Yes that's right, i have apache running infront of production instance but not in dev instance.

On Monday, September 11, 2017 at 10:23:38 PM UTC-7, devops wrote:

Brian Fox

unread,

Sep 12, 2017, 2:28:55 PM9/12/17

to devops, Nexus Users

There's your answer. The gzip compression often happens at the client based on the accept-type headers sent from the server.

--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users+unsubscribe@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/d7ef98c5-aa3b-45c8-a232-50061f12f87f%40glists.sonatype.com.

devops

unread,

Sep 13, 2017, 2:16:24 PM9/13/17

to Nexus Users, kala...@gmail.com

Thanks for your reply. i tried to configure gzip in apache, couldn't get compressed file in the output.

So, i turned off apache reverse proxy in production and tried to install pypi package without reverse proxy, stilll its says not a gzip file. Let me know if i am doing anything wrong.

Kalai

On Tuesday, September 12, 2017 at 11:28:55 AM UTC-7, brianf wrote:

There's your answer. The gzip compression often happens at the client based on the accept-type headers sent from the server.

On Tue, Sep 12, 2017 at 2:26 PM, devops <kala...@gmail.com> wrote:

Yes that's right, i have apache running infront of production instance but not in dev instance.

On Monday, September 11, 2017 at 10:23:38 PM UTC-7, devops wrote:
Hello Nexus,

I have two nexus instances of version -3.2.1.01 (Dev and production env). Created three pypi repositories in each nexus instances given below

* pypi-all - group repository
* pypi-internal - we host internal repos here
* pypi-proxy - it proxies https://pypi.python.org/

Migrated all pypi packages from pypi registry to nexus pypi repository using twine utility

* twine upload -r nexus <package name> --- Pypi package is in the tar.gz format.

when i uploaded pypi packages to nexus dev instance, it compress the fie to gzip format. But for some reason, compression is not happening in prod instance of nexus. so when i do pip install against dev instance, it installs successfully

* pip install <package-name> --trusted-host <hostname>

But pypi package install fails against prod instance. attached exception erros..

File "/Users/devops/anaconda/lib/python3.6/gzip.py", line 411, in _read_gzip_header raise OSError('Not a gzipped file (%r)' % magic) OSError: Not a gzipped file (b'sh')

Please let me know, if i am doing anything wrong.

Thanks in advance !!!!!

--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users...@glists.sonatype.com.

To post to this group, send email to nexus...@glists.sonatype.com.

Peter Lynch

unread,

Sep 13, 2017, 3:13:55 PM9/13/17

to devops, Nexus Users

Try turning off strict content type validation for the pypi repositories.

https://issues.sonatype.org/browse/NEXUS-10243

If that works, then upgrade to 3.5.2 for the permanent fix.

To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users+unsubscribe@glists.sonatype.com.

To post to this group, send email to nexus...@glists.sonatype.com.

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/26ea6c1f-2bd4-47ed-bfbe-5e95aade357d%40glists.sonatype.com.

Frederick Milens

unread,

Sep 13, 2017, 3:47:26 PM9/13/17

to Peter Lynch, devops, Nexus Users

A thought occurs:

What happens if you attempt to manually download the same file (from the prod Nexus instance) that pip is attempting to download? Are there any interesting clues there, and what exactly is in the file you get back?

I believe running with pip -v should give you verbose enough output to get the actual URLs.

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/26ea6c1f-2bd4-47ed-bfbe-5e95aade357d%40glists.sonatype.com.

For more options, visit https://groups.google.com/a/glists.sonatype.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users...@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/CAC16qPg6pXNrWzOS1Fkg0wFRdXhv-pAR6F%3DTGGfv9YMPib%2B_Lg%40mail.gmail.com.

devops

unread,

Sep 13, 2017, 4:16:48 PM9/13/17

to Nexus Users, ply...@sonatype.com, kala...@gmail.com

File "/Users/devops/anaconda/lib/python3.6/gzip.py", line 411, in _read_gzip_header
raise OSError('Not a gzipped file (%r)' % magic)

OSError: Not a gzipped file (b'ca')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/devops/anaconda/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
File "/Users/devops/anaconda/lib/python3.6/site-packages/pip/commands/install.py", line 335, in run
    wb.build(autobuilding=True)
File "/Users/devops/anaconda/lib/python3.6/site-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
File "/Users/devops/anaconda/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
File "/Users/devops/anaconda/lib/python3.6/site-packages/pip/req/req_set.py", line 620, in _prepare_file
    session=self.session, hashes=hashes)
File "/Users/devops/anaconda/lib/python3.6/site-packages/pip/download.py", line 821, in unpack_url
    hashes=hashes
File "/Users/devops/anaconda/lib/python3.6/site-packages/pip/download.py", line 663, in unpack_http_url
    unpack_file(from_path, location, content_type, link)
File "/Users/devops/anaconda/lib/python3.6/site-packages/pip/utils/__init__.py", line 605, in unpack_file
    untar_file(filename, location)
File "/Users/devops/anaconda/lib/python3.6/site-packages/pip/utils/__init__.py", line 538, in untar_file
    tar = tarfile.open(filename, mode)
File "/Users/devops/anaconda/lib/python3.6/tarfile.py", line 1586, in open
    return func(name, filemode, fileobj, **kwargs)
File "/Users/devops/anaconda/lib/python3.6/tarfile.py", line 1644, in gzopen
    raise ReadError("not a gzip file")
tarfile.ReadError: not a gzip file

Frederick Milens

unread,

Sep 13, 2017, 4:41:01 PM9/13/17

to devops, Nexus Users, ply...@sonatype.com

Thanks for trying that. Next questions:

For the file you downloaded from http://nexusrepo.com/nexus/repository/pypi-all/packages/pypi-detect/0.3.1/pypi-detect-0.3.1.tar.gz, did you open that file (or look at the actual bytes) to see that it is, in fact, a valid gzipped tar file? I would recommend using a command-line tool as opposed to a browser, just in case it tries to decompress for you automatically.

What are the first eight bytes in that downloaded file once you download it?

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/fc6783e5-2510-4e9f-ab0f-fd159775f72e%40glists.sonatype.com.

devops

unread,

Sep 13, 2017, 5:56:22 PM9/13/17

to Nexus Users, kala...@gmail.com, ply...@sonatype.com

Thanks for your reply

used command line tool to download package.

devops@MacBook-Pro:~/Downloads$ wget http://nexusrepo.com/nexus/repository/pypi-hosted/packages/pypi-detect/0.3.1/pypi-detect-0.3.1.tar.gz
--2017-09-13 14:50:49-- http://nexusrepo.com/nexus/repository/pypi-hosted/packages/pypi-detect/0.3.1/pypi-detect-0.3.1.tar.gz
Resolving nexusrepo.com... 10.92.254.84
Connecting to nexusrepo.com|10.92.254.84|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 40960 (40K) [application/x-gzip]
Saving to: ‘pypi-detect-0.3.1.tar.gz’

pypi-detect-0.3.1.tar.gz 100%[=========================================================================================================================================>] 40.00K 154KB/s in 0.3s

2017-09-13 14:50:49 (154 KB/s) - ‘pypi-detect-0.3.1.tar.gz’ saved [40960/40960]

devops@MacBook-Pro:~/Downloads$ file pypi-detect-0.3.1.tar.gz
pypi-detect-0.3.1.tar.gz: POSIX tar archive (GNU)

Please let me know if you need any information from my end.

Thanks

Frederick Milens

unread,

Sep 13, 2017, 6:29:54 PM9/13/17

to devops, Nexus Users, ply...@sonatype.com

Okay, using the file command serves the same purpose as examining the file signature. Based on the output there, it looks more to me like you have a file that's just a tar, not a .tar.gz file, which could be causing the problem.

My next question would be whether or not this is the same as what's currently stored in your production Nexus instance? For that, I would suggest you run shasum (or similar) against the manually-downloaded file to obtain a SHA1 of the file, then compare that to the SHA1 in the asset's attributes in the Nexus 3 UI. (You should be able to drill down to that file in the UI and see the digest there for comparison.) When you do that, what are the digests, and do they match?

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/d93a6d6f-ba0c-4135-b83d-92ba16ee1dac%40glists.sonatype.com.

devops

unread,

Sep 14, 2017, 5:12:45 AM9/14/17

to Nexus Users, kala...@gmail.com, ply...@sonatype.com

Just realized. yes it's not a tar.gz file. Uploaded the tar.gz file again from pypi registry to nexus pypi repository. It worked.

Thanks Frederick & everyone...

Reply all

Reply to author

Forward