Setting up S3 blobstores on Dell ECS

136 views
Skip to first unread message

Olivier Clavel

unread,
May 11, 2021, 3:01:49 AM5/11/21
to Nexus Users
Has anyone here ever configured s3 blobstores on Dell ECS ?

The company I work for recentryl aquired such an infra and we wanted to test using it for Nexus. Creating the blobstore works without problems and using it to store content as well. The problem comes when deleting content which actally never gets cleaned up from the bucket (after correctly waiting for the configured grace period and playing a compact blobstore task on it).

Has anyone experienced the same problem and even better solved it ? Thanks in advance for your returns. Below the procedure we used to create and test.

  1. create a brand new S3 blobstore:
    • select the S3 type
    • Enter a compliant name (tests3)
    • Choose a region (any will do in our implementation but can't be default)
    • Enter a compliant bucket name (tests3-bucket)
    • Set expiration days to 1
    • tick "authentication", enter credentials
    • tick "advanced connection settings", enter the endpoint url (https://our.local.s3.service) and tick the "Use path-style access" (dns style not enabled in our environment)
  2. set an automatic task to compact the blobstore daily
  3. create whatever hosted repository backed by the above blobstore (we tested several formats but raw is easy)
  4. add a content to the repo
  5. make sure the content exist and is available
  6. check with an s3 browser that the content has a corresponding file in the blobstore
  7. remove the content from the repo
  8. make sure the content has disappeared from the repo
  9. check that the file still exist in the blobsore
  10. wait for the expiration period to pass (checking that the compact task is running as expected)

Expected result:: the file has disappeared in the S3 blobstore

Actual result the file is still present in the S3 blobstore

Peter Lynch

unread,
May 12, 2021, 1:15:49 PM5/12/21
to Olivier Clavel, Nexus Users
On Tue, May 11, 2021 at 4:01 AM Olivier Clavel <olivier...@gmail.com> wrote:
Has anyone here ever configured s3 blobstores on Dell ECS ?

 

The company I work for recentryl aquired such an infra and we wanted to test using it for Nexus. Creating the blobstore works without problems and using it to store content as well. The problem comes when deleting content which actally never gets cleaned up from the bucket (after correctly waiting for the configured grace period and playing a compact blobstore task on it).

Has anyone experienced the same problem and even better solved it ? Thanks in advance for your returns. Below the procedure we used to create and test.

  1. create a brand new S3 blobstore:
    • select the S3 type
    • Enter a compliant name (tests3)
    • Choose a region (any will do in our implementation but can't be default)
    • Enter a compliant bucket name (tests3-bucket)
    • Set expiration days to 1
    • tick "authentication", enter credentials
    • tick "advanced connection settings", enter the endpoint url (https://our.local.s3.service) and tick the "Use path-style access" (dns style not enabled in our environment)
  2. set an automatic task to compact the blobstore daily
  3. create whatever hosted repository backed by the above blobstore (we tested several formats but raw is easy)
  4. add a content to the repo
  5. make sure the content exist and is available
  6. check with an s3 browser that the content has a corresponding file in the blobstore
  7. remove the content from the repo
  8. make sure the content has disappeared from the repo
  9. check that the file still exist in the blobsore
  10. wait for the expiration period to pass (checking that the compact task is running as expected)
Compact blobstore task does not affect S3 blobstores, even if AWS s3 implementation was being used.

S3 object deletion works using S3 lifecycle configuration created at time of blobstore creation/update.


Expected result:: the file has disappeared in the S3 blobstore

Actual result the file is still present in the S3 blobstore

--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users...@glists.sonatype.com.
To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/e399ce6b-1053-4e02-8952-d5e193ee6e91n%40glists.sonatype.com.


--
Sonatype
Peter Lynch
Principal Support Engineer

Reply all
Reply to author
Forward
0 new messages