YUM repos not syncing with Nexus3, possible net proxy whitelist issue?

[mike_hof...@yahoo.com]

I'm evaluating the use of Nexus at my client site, a security team with strict policies regarding external access by internal systems.

The goal is to have Nexus serve as internal repository for everything, from YUM to Docker. Every repository we wish to sync must be approved by the SecArchs and undergo a policy review and what-not before it gets specifically whitelisted for external access.

I had *thought* I had all lined up and approved and finally opened through firewalls/proxies, but my first attempt at proxying a simple YUM repo fails: I simply do not see any repo data appearing. Online doco and links speak about specific tasks that are kicked off for YUM repost, but they all refer to Nexus 2 when it still ran reposync under the hood. Supposedly, in Nexus 3, it's all built in. However, I still don't see any syncing tasks in the Tasks scheduler.

Is there something I'm missing? I've verified that connection to the intended RPM sources work - in fact, we had been syncing them with reposync/createrepo for years now. I simply configured the same external URL in Nexus, made sure I had the HTTP proxy configured and thought it will all start syncing after a while.

However, nothing is showing up in the Assets or Components and a command line "yum makecache" returns that repodata cannot be found under the Nexus repo URL. I *can* curl to the repo, and receive a "can't browse" error, which doesn't surprise me. .../sonatype-work/nexus3/blobs/default/content is also empty.

Furthermore, looking through the logs, I see messages like this:

2017-10-24 06:16:46,277+0000 INFO  [qtp263965335-47] *UNKNOWN com.sonatype.nexus.plugins.outreach.internal.outreach.SonatypeOutreach - Outreach bundle unavailable http://links.sonatype.com/products/nexus/outreach/oss/3.6.0-02/en/anonymous (403:blocked by proxy)

Which made me do a double-take. What is that connection trying to do? I would have expected to see no connections except to the external repo URL - those that I had to painstakingly get whitelisted. What is links.sonatype.com and what it is trying to get from there?

What else needs to be done to get the YUM repos (and ultimately other repos as well) to work? The docs make it sound like it should just work once the proxy repo is configured.

Mike

[Michael Prescott]

Mike,

Proxy repositories don't act like mirrors which proactively sync the remote repository content. They just cache the content as clients request it.

NXRM 2's yum support was built on top of maven, so there are scheduled tasks to produce the necessary yum metadata. We have pure yum support in NXRM 3, so this isn't necessary.

The request to links.sonatype.com comes from the Outreach plugin, which pulls down the HTML content you see once you log in. It typically includes links to help, information about new releases, and recently some single-question surveys.

If you want to prevent NXRM from fetching outreach content, go into the Capabilities section on the left nav, and disable the "Outreach: Management" plugin.

Michael

--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users+unsubscribe@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.
To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/42c782e0-9540-4b95-b4ec-54efa18a3913%40glists.sonatype.com.
For more options, visit https://groups.google.com/a/glists.sonatype.com/d/optout.

[mike_hof...@yahoo.com]

Thanks for the fast reply!

So, I remain somewhat confused where the repodata, which the YUM client requires is supposed to come from? "yum makecache" didn't create it, "yum update" complains about its lack. Adding '--enablerepo=...' had no effect (not that I expected it). My .repo file in /etc/yum.repos.d is pointing at the URL given in the Repository settings in N3, as I'd said, and I also verified it's a reachable valid URL by itself.

Doing a refresh of the cache data, whether preceded by a cache invalidation or not, also didn't do anything!

Still confused and lost...
Mike

To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users...@glists.sonatype.com.

[Rich Seddon]

Try taking yum out of this to narrow down the issue.

Request "/repodata/repomd.xml" through the proxy repository's URL.  That should result in the file being download into the proxy, and also the other metadata files should get downloaded into the proxy's repodata directory.  If that doesn't happen check the logs, you likely have a network issue.

Rich

[mike_hof...@yahoo.com]

Hi Rich,

Just tested that with a quick curl and that was successful. The repo data file now also appears in the Assets list. So, connectivity principally works.

I then did a "yum clean all && yum makecache" and primary, filelists, etcalso appear. Then, a "yum list" on the repo now correctly return files.

Finally, a "yum install" for a package of the repo goes through: I see the package in both Assets and Components.

So, the issue seems to be that first initialising of the repo which doesn't happen automatically. Having to manually do this first init for every repo would obviously be a bit of a pain! :)

As if something isn't running that should be (this is the RPM based Nexus3 3.50-02 package, with mostly the default parameters - apart from proxy settings, of course).

Cheers
Mike

[Peter Lynch]

On Tue, Oct 24, 2017 at 8:43 PM, mike_hoffman2000 via Nexus Users <nexus...@glists.sonatype.com> wrote:

Hi Rich,

Just tested that with a quick curl and that was successful. The repo data file now also appears in the Assets list. So, connectivity principally works.

I then did a "yum clean all && yum makecache" and primary, filelists, etcalso appear. Then, a "yum list" on the repo now correctly return files.

Finally, a "yum install" for a package of the repo goes through: I see the package in both Assets and Components.

So, the issue seems to be that first initialising of the repo which doesn't happen automatically. Having to manually do this first init for every repo would obviously be a bit of a pain! :)

As if something isn't running that should be (this is the RPM based Nexus3 3.50-02 package, with mostly the default parameters - apart from proxy settings, of course).

If you want to have us take a more detailed look, we need to examine your Nexus settings and logs from the time of the problem.

You can open a private ticket in the NEXUS project at https://issues.sonatype.org. Attach the following:

- Nexus support zip

- complete compressed nexus.log and request.log from Oct 24, the day you did all this testing 

- yum client output and time of test

Alternately, recreate the issue with an identical set of newly created repositories in Nexus and supply the above.

 

To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users+unsubscribe@glists.sonatype.com.

To post to this group, send email to nexus...@glists.sonatype.com.

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/cc03916e-b79a-4358-8c35-df4344b28427%40glists.sonatype.com.

[Michael Hoffmann]

Thanks, Peter. I've submitted a ticket referencing this thread and attaching the files you listed.

[Michael Hoffmann]

While I've been busy (and mostly successful) in adding other proxies to Nexus (YUM, Docker, NuGet), this one is still not working.

Just to track things down better, I installed a separate Nexus server that did not require a proxy for Internet connectivity. With that I added one YUM proxy, directly out of the documentation.

As in:

Minimal configuration steps are:

• Define Name e.g. yum-proxy
• Define URL for Remote storage, e.g. http://mirror.centos.org/centos/
• Pick a Blob store for Storage

and then copy-pasted the nexus.repo file into /etc/yum.repos.d with

[nexusrepo]
name=Nexus Repository
baseurl=https://malbec.centaur.id.au/repository/yum-proxy/$releasever/os/$basearch/
enabled=1
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
priority=1

Only difference: gpgcheck is set to 0, just to remove one other error source. I then ran yum makecache and got this:

 Loaded plugins: fastestmirror
base                                                                                                         | 3.6 kB  00:00:00
epel/x86_64/metalink                                                                                         | 2.9 kB  00:00:00
extras                                                                                                       | 3.4 kB  00:00:00
nexusrepo                                                                                                    | 2.2 kB  00:00:00
updates                                                                                                      | 3.4 kB  00:00:00
nexusrepo/7/x86_64/group_gz    FAILED
https://myserver.mydomain.com/repository/yum-proxy/7/os/x86_64/repodata/9346184be1deb727caf4b1ecf4a7949155da5da74af9b92c172687b290a773df-c7-x86_64-comps.xml.gz: [Errno 14] HTTPS Error 404 - Not Found
Trying other mirror.
To address this issue please refer to the below knowledge base article

https://access.redhat.com/articles/1320623

If above article doesn't help to resolve this issue please create a bug on https://bugs.centos.org/

nexusrepo/7/x86_64/filelists   FAILED
https://myserver.mydomain.com/repository/yum-proxy/7/os/x86_64/repodata/c1561546c684bd06b3a499c2babc35c761b37b2fc331677eca12f0c769b1bb37-filelists.xml.gz: [Errno 14] HTTPS Error 404 - Not Found
Trying other mirror.
nexusrepo/7/x86_64/primary     FAILED
https://myserver.mydomain.com/repository/yum-proxy/7/os/x86_64/repodata/b686d3a0f337323e656d9387b9a76ce6808b26255fc3a138b1a87d3b1cb95ed5-primary.xml.gz: [Errno 14] HTTPS Error 404 - Not Found
Trying other mirror.
nexusrepo/7/x86_64/other       FAILED
https://myserver.mydomain.com/repository/yum-proxy/7/os/x86_64/repodata/a0af68e1057f6b03a36894d3a4f267bbe0590327423d0005d95566fb58cd7a29-other.xml.gz: [Errno 14] HTTPS Error 404 - Not Found
Trying other mirror.
Loading mirror speeds from cached hostfile
 * base: mirror.as24220.net
 * epel: mirror.as24220.net
 * extras: mirror.aarnet.edu.au
 * updates: mirror.as24220.net
https://myserver.mydomain.com/repository/yum-proxy/7/os/x86_64/repodata/b686d3a0f337323e656d9387b9a76ce6808b26255fc3a138b1a87d3b1cb95ed5-primary.xml.gz: [Errno 14] HTTPS Error 404 - Not Found
Trying other mirror.


 One of the configured repositories failed (Nexus Repository),
 and yum doesn't have enough cached data to continue. At this point the only
 safe thing yum can do is fail. There are a few ways to work "fix" this:

     1. Contact the upstream for the repository and get them to fix the problem.

     2. Reconfigure the baseurl/etc. for the repository, to point to a working
        upstream. This is most often useful if you are using a newer
        distribution release than is supported by the repository (and the
        packages for the previous distribution release still work).

     3. Run the command with the repository temporarily disabled
            yum --disablerepo=nexusrepo ...

     4. Disable the repository permanently, so yum won't use it by default. Yum
        will then just ignore the repository until you permanently enable it
        again or use --enablerepo for temporary usage:

            yum-config-manager --disable nexusrepo
        or
            subscription-manager repos --disable=nexusrepo

     5. Configure the failing repository to be skipped, if it is unavailable.
        Note that yum will try to contact the repo. when it runs most commands,
        so will have to try and fail each time (and thus. yum will be be much
        slower). If it is a very temporary problem though, this is often a nice
        compromise:

            yum-config-manager --save --setopt=nexusrepo.skip_if_unavailable=true

failure: repodata/b686d3a0f337323e656d9387b9a76ce6808b26255fc3a138b1a87d3b1cb95ed5-primary.xml.gz from nexusrepo: [Errno 256] No more mirrors to try.
https://myserver.mydomain.com/repository/yum-proxy/7/os/x86_64/repodata/b686d3a0f337323e656d9387b9a76ce6808b26255fc3a138b1a87d3b1cb95ed5-primary.xml.gz: [Errno 14] HTTPS Error 404 - Not Found

When I now look into the Assets for this repo I see one lonely entry named

7/os/x86_64/repodata/repomd.xml

The Nexus logs at that moment show

192.168.20.91 - - [03/Nov/2017:19:15:46 +1100] "GET /repository/yum-proxy/7/os/x86_64/repodata/repomd.xml HTTP/1.1" 200 2264 2 "urlgrabber/3.10 yum/3.4.3"
192.168.20.91 - - [03/Nov/2017:19:15:47 +1100] "GET /repository/yum-proxy/7/os/x86_64/repodata/9346184be1deb727caf4b1ecf4a7949155da5da74af9b92c172687b290a773df-c7-x86_64-comps.xml.gz HTTP/1.1" 404 1853 1 "urlgrabber/3.10 yum/3.4.3"
192.168.20.91 - - [03/Nov/2017:19:15:47 +1100] "GET /repository/yum-proxy/7/os/x86_64/repodata/c1561546c684bd06b3a499c2babc35c761b37b2fc331677eca12f0c769b1bb37-filelists.xml.gz HTTP/1.1" 404 1853 1 "urlgrabber/3.10 yum/3.4.3"
192.168.20.91 - - [03/Nov/2017:19:15:47 +1100] "GET /repository/yum-proxy/7/os/x86_64/repodata/b686d3a0f337323e656d9387b9a76ce6808b26255fc3a138b1a87d3b1cb95ed5-primary.xml.gz HTTP/1.1" 404 1853 1 "urlgrabber/3.10 yum/3.4.3"
192.168.20.91 - - [03/Nov/2017:19:15:47 +1100] "GET /repository/yum-proxy/7/os/x86_64/repodata/a0af68e1057f6b03a36894d3a4f267bbe0590327423d0005d95566fb58cd7a29-other.xml.gz HTTP/1.1" 404 1853 1 "urlgrabber/3.10 yum/3.4.3"
192.168.20.91 - - [03/Nov/2017:19:15:47 +1100] "GET /repository/yum-proxy/7/os/x86_64/repodata/b686d3a0f337323e656d9387b9a76ce6808b26255fc3a138b1a87d3b1cb95ed5-primary.xml.gz HTTP/1.1" 404 1853 1 "urlgrabber/3.10 yum/3.4.3"

So, the repomd.xml is downloaded by the yum client and gets a 200, it "sees" the other files to download *inside* the repomd.xml file - but fails to get them with a 404. Because the Nexus server also only grabbed that one lonely file.

I suppose that's progress in a way, but I'm no closer to a solution when even an example copy-pasted out of the documentation isn't working.

[Niels Bertram]

I recently tried to setup a EPEL proxy which failed with the same symptoms as yours Michael. Nexus will happily resolve/download/serve the repomd.xml file but all listed resources just return a 404.

--

You received this message because you are subscribed to the Google Groups "Nexus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users+unsubscribe@glists.sonatype.com.

To post to this group, send email to nexus...@glists.sonatype.com.

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/dd9f7925-a089-483d-93b6-5ef950f9ee0b%40glists.sonatype.com.

[Joseph Stephens]

This is caused by a known issue https://issues.sonatype.org/browse/NEXUS-14058 which is ready for release in the next Nexus version.

The way around it for now is to point the remote url in your proxy configuration at the server that contains the repodata folder (e.g. http://server/7/os/x86_64/) although that does mean you'll require a repository per architecture / version.

Thanks,

Joe

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/CAPLPygn91vU9J6H5NH8eTQTaNVM38MfzE5TpuNgbfVoVHh%2BC4g%40mail.gmail.com.

For more options, visit https://groups.google.com/a/glists.sonatype.com/d/optout.

--

Joseph Stephens

[Niels Bertram]

Yes what is listed in the ticket makes total sense. Does anyone know where the sources live for this bundle? I checked out the nexus-public code base from github but there is nothing remotely close to YUM repo code in there unless it is just a generic raw proxy implementation. Cheers Niels

[Joseph Stephens]

Hi Niels,

The Yum implementation is a first class citizen (i.e. not built on a generic raw proxy implementation) but the sources are currently private. 

Is there a specific reason you're interested in seeing the sources?

Thanks,

Joe

--

Joseph Stephens

[Niels Bertram]

Yes I’d love to get a fix for this issue pretty soon and doing up a patch and installing it into Karaf is pretty straight forward (or so I hope). Looks like this repo type follows the Docker one where the feature is available but sources are not. Is that a commercial consideration or do you intend to create some stability before turning it to the masses? Kind regards, Niels

[Joseph Stephens]

That's correct that the Yum feature is available to both OSS and Pro users but the sources are closed.

Thanks,

Joe

--

Joseph Stephens

[Michael Hoffmann]

With the help of Joseph Stephens, I now managed to finagle both the Nexus YUM proxy config, as well as the matching .repo file such that it works for a number of YUM mirrors.

As it varies from the documentation, I thought I'd post the solution that works here (until 3.7 comes out and hopefully fixes the issues):

1) In the Nexus3 yum proxy Repository

Make sure you use the full URL, which must point to a directory that contains the repodata sub-dir and those hopefully don't use relative paths (as per the documented error). Make sure the URL has a slash at the end - not sure if that's absolutely needed but why risk it. ;)

2) Point your .repo at that proxy repository in Nexus.

You won't be able to use any variable substitutions or anything.

With that I've made a start with CentOS 7 and with EPEL. Slowly adding more.

NOTE: all working proxies are currently on HTTP. HTTPS is next.