Garbage Collection for Docker Registry

[Tim Webster]

Hi,

We're using Nexus 3 for a private hosted Docker registry.  I'm looking at ways to manage the storage.  I can delete image manifests via the Docker Registry HTTP v2 API, but how do I ensure the layers are deleted from the disk afterwards?  Is there an equivalent of docker 'garbage collection' I can run (or can Nexus do this on its own)?

Thanks,

Tim

[Daniel MD]

Hi Tim, you can run the compact blob store but there appear to be some issues currently, track https://issues.sonatype.org/browse/NEXUS-9293

Hopefully 3.2 will bring improvements in terms of blob store cleanup for docker

--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users...@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.
To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/79421d0e-db85-465a-8ae0-5a2204acdcec%40glists.sonatype.com.
For more options, visit https://groups.google.com/a/glists.sonatype.com/d/optout.

[Tim Webster]

Hi - OK thanks for that - that's exactly what we (all) need....

On Thu, Nov 17, 2016 at 5:37 PM, Daniel MD <daniel...@gmail.com> wrote:

Hi Tim, you can run the compact blob store but there appear to be some issues currently, track https://issues.sonatype.org/browse/NEXUS-9293

Hopefully 3.2 will bring improvements in terms of blob store cleanup for docker

On Thu, Nov 17, 2016 at 6:13 PM Tim Webster <tim.w...@gmail.com> wrote:

Hi,

We're using Nexus 3 for a private hosted Docker registry.  I'm looking at ways to manage the storage.  I can delete image manifests via the Docker Registry HTTP v2 API, but how do I ensure the layers are deleted from the disk afterwards?  Is there an equivalent of docker 'garbage collection' I can run (or can Nexus do this on its own)?

Thanks,

Tim

--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users+unsubscribe@glists.sonatype.com.

[Tim Webster]

Hi,

Running the compact blob store definitely helped (it freed up 130 GB of space), but it doesn't get rid of 'deleted' images.  They are still visible in the GUI, but with no content (because the manifest has been deleted).  Rebuilding the index doesn't help either.

So - it's still not clear if these images are in fact being deleted.  The 130 GB could have been from orphaned layers or dangling images.  I only deleted one image using the HTTP API so it definitely wasn't from that.

I suppose I could live with the fact that images are still in the GUI if the actual layers were deleted from the file system (although it's dodgy), but I can't even tell if that's what happened.  The only conclusion I can draw from this as it stands is there is no way to automate deletion from images - you can only do it from the GUI.

The only other thing I can think of is deleting the layers as well (using the same API), but that is pretty risky and complicated and I don't really want to go there...

[Jeffry Hesse]

Hi there,

Signal boosting something one of our community members created:

https://gist.github.com/lukewpatterson/bf9d19410094ea8bced1d4bb0523b67f

Comments on usage are here: https://issues.sonatype.org/browse/NEXUS-9293

The real thing is in progress as well, but just in case this can help you in the meantime :)

Of note this is not supported by Sonatype directly, it's a community contribution.

Jeffry

On Fri, Nov 18, 2016 at 4:48 AM Tim Webster <tim.w...@gmail.com> wrote:

Hi,

Running the compact blob store definitely helped (it freed up 130 GB of space), but it doesn't get rid of 'deleted' images.  They are still visible in the GUI, but with no content (because the manifest has been deleted).  Rebuilding the index doesn't help either.

So - it's still not clear if these images are in fact being deleted.  The 130 GB could have been from orphaned layers or dangling images.  I only deleted one image using the HTTP API so it definitely wasn't from that.

I suppose I could live with the fact that images are still in the GUI if the actual layers were deleted from the file system (although it's dodgy), but I can't even tell if that's what happened.  The only conclusion I can draw from this as it stands is there is no way to automate deletion from images - you can only do it from the GUI.

The only other thing I can think of is deleting the layers as well (using the same API), but that is pretty risky and complicated and I don't really want to go there...

On Thu, Nov 17, 2016 at 5:44 PM, Tim Webster <tim.w...@gmail.com> wrote:

Hi - OK thanks for that - that's exactly what we (all) need....

On Thu, Nov 17, 2016 at 5:37 PM, Daniel MD <daniel...@gmail.com> wrote:

Hi Tim, you can run the compact blob store but there appear to be some issues currently, track https://issues.sonatype.org/browse/NEXUS-9293

Hopefully 3.2 will bring improvements in terms of blob store cleanup for docker

On Thu, Nov 17, 2016 at 6:13 PM Tim Webster <tim.w...@gmail.com> wrote:

Hi,

We're using Nexus 3 for a private hosted Docker registry.  I'm looking at ways to manage the storage.  I can delete image manifests via the Docker Registry HTTP v2 API, but how do I ensure the layers are deleted from the disk afterwards?  Is there an equivalent of docker 'garbage collection' I can run (or can Nexus do this on its own)?

Thanks,

Tim

--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users...@glists.sonatype.com.

To post to this group, send email to nexus...@glists.sonatype.com.
To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/79421d0e-db85-465a-8ae0-5a2204acdcec%40glists.sonatype.com.
For more options, visit https://groups.google.com/a/glists.sonatype.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users...@glists.sonatype.com.

To post to this group, send email to nexus...@glists.sonatype.com.

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/CABrquLziX9N%3D9roFSe15uipwOmD1vvfX6_Bq5hjWSbSWd%2Bw%2BXA%40mail.gmail.com.