Nexus 3 Docker

[Michael Rumpf]

Hi,

I'm struggeling with the Docker support in Nexus 3, using different connectors and ports for each repository.
In the Maven world we have the following setup:
A release repository, a snapshot repository, and a repository group with the 2 hosted ones for each project, using our shared service of Nexus, SonarQube, and Jenkins.
Similarly I would like to add a Docker repository (private registry), a proxy for Docker Hub and a repository group, combining both.
A user will access the project Docker repository group to search and pull images. Jenkins will be the only one to push to the private repository as we only want images to be pushed that went through the official build process.

In this scenario we would be forced to configure 2 ports per project, having a 60 projects right now, that sums up to 120 ports just for Docker. The amount of ports is technically not an issue, but because we are in a corporate environment, opening so many ports in the different firewalls is a huge pain.

Is this the current setup, forced by the limitation of the Docker client not to allow context-paths?

If yes, the question is could Nexus come up with a solution that is more enterprise friendly?
For example:
There will only be one hosted Docker repository for images from all of the 60 projects. The proxy repository for Docker Hub and a repository group for both. If there is a way to apply permissions on Docker namespaces, we could assign each project a namespace and then limit the permission of the project (defined by a project role) to access only images from this namespace.
In Nexus 2.x Repository Targets could be used to achieve something like that.

I'm not sure, whether this would be possible at all, especially how the Docker client would behave with permissions on namespaces, but maybe this approach would be worth evaluating further to reduce the amount of ports in a corporate environment.

Regards,
     Michael

[Frank Mueller]

Dear Michael,

 

You made the same mistake as myself as I started with docker. Do NOT compare Maven and Docker. This will not work.

 

In Docker you have one Registry which is something like a Repository in Maven.

Within this registry you have multiple Repositories which is some sort of a GroupID in Maven

Under this GroupID you have your docker images and versions.

 

Because you have to address each container with its registry url you end up in Artifacts like

 

your.private-registry.com/projectname/imagename:imageversion

 

To be honest I am not aware of the docker support within nexus. We are using a different product for this but this mechanic should be the same.

If something is hosted in dockerhub you don’t need to add the registry which means with registry it looks into this registry and without it search in dockerhub.

 

Best regards

Frank

--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users...@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.
To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/8a0f027f-ce15-4e00-85e0-b76a8163245c%40glists.sonatype.com.
For more options, visit https://groups.google.com/a/glists.sonatype.com/d/optout.

[Michael Rumpf]

Hi Frank,

OK, understood. But when you create one Docker repository for all projects in Nexus 3, I do not find any way to restrict access to a namespace for a certain role.
The issue is that we have 60 projects and each project wants to restrict project artifact access to the project members only, you either need to separate the repositories or find a way to restrict access to a path in a single repository.

When project A and project B publish their Docker images into one registry:

registry.host.name:5000/projectA/imageA1..An
registry.host.name:5000/projectB/imageB1..Bn

How can I prevent members from projectA to access images from projectB?

In Nexus 2.x there was the concept of a "Repository Target", with which you could restrict the access to a path /projectX/**/*, but it seems as if this feature has been dropped for Nexus 3 and I do not see a way currently to achieve such a permission scheme with Nexus 3 on a single Docker repository... :(

Thanks,
    Michael

[Manfred Moser]

The repository target concept from Nexus 2 was based on a Maven format specific understanding of a directory structure and therefore a path name. With Nexus 3 we are going to expand that to work in a repository format independent way using the component identifier (most likely). E.g. for a Docker image the name containing say projectA/imageX

This has not been implemented yet but is on our todo list. If you have any specific ideas or suggestions, please let us know.

Manfred

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/9e757ec7-8083-449c-a47e-e71f7d8e7d99%40glists.sonatype.com.

[Frank Mueller]

I think the major Problem is located within the docker api itself.

 

While pushing or pulling the docker daemon connects to host:port/v2 without any option to add subdirectories or something like this.

That is the reason why you have to add the ports. There are tools out there that can handle you request but they are not free (and because this is the nexus mailing list, I will not encourage you to change)

 

Nexus is currently a milestone release which means that it might be changes. Regarding you problem while authenticating I would strongly suggest not do drive such an auth structure because of the nature of docker.

 

The biggest advantage is the layering. You can easily use existing layers and put you changes on top of them without changing the below layers. That means if a project already have layers that you also need, you will share them (public layers will also be put into your private repository while pushing you own layer).

 

We are working for multiple clients and decided to let each developer access each other artifact as it might happen that a developer changes projects or may be able to reuse some existing artifacts.

But that is only our approach. If you really need the feature to encapsulate the project artifacts, you will need to create the multiple repositories using different ports and use auth on them. Also you have to create a dockercfg entry for each repository you will handle as for docker the repository private.repo.com:2432 is different that private.repo.com:8573 (for instance)

 

I am sorry but it seems that you currently have to decide which devil you want: Opening many ports in firewall or having all artifacts available.

 

Maybe it is the right way to file a change request within Sonatype Support.

 

Best regards

 

Frank

[Michael Rumpf]

Hi Manfred,

we have just started playing around with the Nexus 3 m5 and faced a few problems (see my other 2 posts) to integrate it into our infrastructure in the same way Nexus 2.x works at the moment. But as it is still in beta phase that is OK and we are happy to report any issues we find to help Nexus 3 to get stable.

My knowledge of Docker (as Frank realized) is at beginner level at the moment. I have read a lot but not used it besides starting a few containers (I struggeled to get docker-machine running on a corporate windows 7 host to be able to start docker containers easily on my local notebook).

So I cannot give any suggestions at the moment on how to implement the Repository Target feature to enable the permission scheme I mentioned.
However our requirements are: I'm working for a large company with many projects on Nexus where we have a hosted Maven repository for each project (people do not switch between projects very often).
With Docker we would need to provide the same kind of separation per project without the need to open 2 ports per project (one for the project private hosted repo and one for the repository group as suggested by the videos "tips from the trenches" http://www.sonatype.org/nexus/2015/09/22/docker-and-nexus-3-ready-set-action/).
Ideally there would be only one private registry with a permission scheme, based on namespaces (maybe implemented by Repository Targets).
A Docker proxy repository group, proxying Docker Hub and a repository group which combines the 2 repositories to a single repository.
Accessing the repository group shows all proxies resources from Docker Hub as well as the project resources from the private part of the hosted repository where the user has access to.

However, I'm currently not sure whether a permission scheme based on namespaces will work for all Docker use-cases. I wanted to configure Nexus 3 and then experiment with the various Maven/Jenkins plugins to build and publish Docker images. Currently I'm stuck with the issue we are discussing here... :(

Regards,
     Michael

[Michael Rumpf]

It seems as if the Docker Registry v2 API supports more path components:
http://docs.docker.com/registry/spec/api/#overview

Regards,
    Michael

[Jeffry Hesse]

Hi there,

A few notes on the current conversation:

- As Manfred mentioned, Repo Targets is missing but only because we are rebuilding it to better serve you all in the future. We are talking amongst ourselves on how we would apply our new approach to Docker images and layers, and the rest of the formats we are going to be supporting.

- In a similar breath we are also reworking security to allow for scenarios such as what you describe at least for images. In some early talks we were curious about security around the layers within an image. If you or others have any thoughts, we'd love to hear them.

Also thanks for the feedback, particularly on the ports and connectors. This is helpful information to the team. The thought in allowing the ports was such that you wouldn't have to setup NGINX, Apache, etc... in front of Nexus and run a reverse proxy. We felt this gave our users the most amount of flexibility and reduced the amount of applications/complexity needed for a production deployment. In particular you are right about not being able to specify a context path in the Docker client. Docker v1 private registries had to be setup this way as well, meaning at the very least it's a common problem. We are going to be taking a look at Docker v2 registries in the near future. In my initial look I hadn't seen any change to allow context path for a registry in say the docker pull or push commands, and everything seems to indicate it still has to be run on a port at least as of the latest version. The change for additional path components on the v2 registry API seems to be mainly around naming of images rather than a full context path, but potentially I'm reading it wrong?

Thanks,

Jeffry

--

You received this message because you are subscribed to the Google Groups "Nexus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users...@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/9f1b9e4e-d045-4d86-9c7b-3c988baaff71%40glists.sonatype.com.

[Michael Rumpf]

Hi Jeffry,

great to hear that you will be working on repository target feature. I hope you will find a way to support private regions in a single registry.
There is no issue with opening one or two ports for the Docker registries, but having 2 ports for each new project would be a pain.

Whether the V2 registry allows for an additional context-path I cannot tell as I'm quite new to Docker. In case there is no context path but only additional path elements for an image, then the repository target could solve the separation issue like this:

myhost.com/
    projectA/
        pathElement*/baseName
    projectB/
        pathElement*/baseName

So the projectA and projectB are part of the image name but are used by the Repository Target to create a privilege based on the path element:

Repository Targets:
    * projectA: projectA/**/*
    * projectB: projectB/**/*

I really hope that you find a solution to make this possible with the final version of Nexus 3.

Regards,
     Michael

[Michael Süß]

Hi Jeffry,

I just wanted to ask about the state of the security enhancements, especially about the functionality to restrict the access within a docker repository.

The docker implementation of milestone release 7 has been announced as ready for production. As far as I can see, there is still no way to create privileges for specific sections of a repository (although it is written on the introduction section of the security documentation). Is this a topic that is still worked on for the final release?

Best regards,

Michael

[Jeffry Hesse]

Hi there Michael,

We are currently working on some new features that will end up replacing the Repo Target functionality that existed in Nexus Repository Manager 2.x, and we are looking at it through the view of how do we apply it to all formats. We won't have anything for Docker specifically in the initial 3.0 release, but the wheels are on the pavement. Stay tuned! Feel free to follow up directly as well, it would be nice to know more about your use cases if you can share them.

Jeffry

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/a912a859-df21-4a0c-af0b-4c2d9f2d93a8%40glists.sonatype.com.

[Michael Süß]

Hi Jeffry,

sorry for my late response. I'm actually in the same team than Michael Rumpf. We provide a centrally hosted platform providing Jenkins, SonarQube and Nexus for other projects.
Those projects develop software for several different customers and therefore must not have access to artifacts of another projects (for legal and security reasons).

I'm looking forward to hear more details about the new mechanism that you mentioned. Is there a rough time schedule when we have to expect those changes?

Best regards,

Michael

[Vincent Berruchon]

Hi,

Working in a big corparate environment too, we'd like to use Nexus 3 as a Docker Registry as we are yet using Nexus v2 for our existing CI chains.

I've been testing Nexus 3 M7, and from what I can read here, it's like it should be easier to organize different Docker "repositories" for different use and teams with according habilitations facilities?

I was wondering about using various DNS aliasses/VIP + NAT redirection to have easy access to the different registries just by the FQDN instead of ports even if it's the same Nexus instance.

And I would have two repos by teams/phase to distinct one for snapshots version we will purge on a regular basis and one for fixed release versions (that's another problem but it implies too easy access to different repos with different rights).

For example I could have: 

- nexus3-docker-tech-snapshot.dom:5000 : for snapshots images of middleware software at disposition of projects

- nexus3-docker-tech.dom:5000 : same but for released version.

- nexus3-docker-dev-snapshot.dom:5000 : for images of the applicatives components, made using "tech" images as base.

- nexus3-docker-dev.dom:5000 : same but for released version.

Would theses strategies be revelant?

Will the v3.0 allow something easier than to have to manage many DNS aliasses with certs and redirections configuration?

A s no date for the v3.0 are annouced, it will be hard to say that we want to wait for it. 

Thank you for your hints

Best regards

Vincent

[Frans Thamura]

Mike

Will sonatype create a step by step installation to use nexus 3 with docker?

from openssl steps :)

so we can follow and setup easily

F
--
Frans Thamura (曽志胜)
Java Champion
Shadow Master and Lead Investor
Meruvian.
Integrated Hypermedia Java Solution Provider.

Mobile: +628557888699
Blog: http://blogs.mervpolis.com/roller/flatburger (id)

FB: http://www.facebook.com/meruvian
TW: http://www.twitter.com/meruvian / @meruvian
Website: http://www.meruvian.org

"We grow because we share the same belief."

> https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/73455e70-954e-46c7-bb11-fcaf0aef8a54%40glists.sonatype.com.

[Peter Lynch]

On Tue, Apr 5, 2016 at 1:01 PM, Frans Thamura <fr...@meruvian.org> wrote:

Mike

Will sonatype create a step by step installation to use nexus 3 with docker?

from openssl steps :)

Docker already has decent openssl related docs for securing the docker daemon so we chose not to duplicate them.

https://docs.docker.com/engine/security/https/

https://docs.docker.com/engine/security/certificates/

We also have our own docs which describe the same using java keytool.

https://support.sonatype.com/hc/en-us/articles/217542177

..and we also have the book for general docker use and it covers configuring Nexus to use SSL .

We would be interested in more specifics on what you think is still missing. 

 

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/CAOeeQyq5tOYS-CJfM49WX-q%3DqeMq6BHnFrGZTfTa9Lx401C2_w%40mail.gmail.com.