Remove Snapshots from Repository
1,304 views
Skip to first unread message
epiq sty
Jul 12, 2017, 12:07:53 PM7/12/17
to Nexus Users
Hello, all!
We are faced with problem , when old snapshots are groving in our nexus repo , and tried to use Cleanup Unused Snapshots scheduled task to cleanup it.
But , seems it not helps:
| jvm 1 | 2017-07-11 16:19:32,787+0200 INFO [pxpool-1-thread-5] admin org.sonatype.nexus.maven.tasks.UnusedSnapshotRemoverTask - Scheduled task (Cleanup Unused Snapshots) started :: Removing unused snapshots from repository Snapshots |
| jvm 1 | 2017-07-12 12:32:29,326+0200 INFO [pxpool-1-thread-5] admin org.sonatype.nexus.maven.tasks.UnusedSnapshotRemoverTask - Scheduled task (Cleanup Unused Snapshots) finished :: Removing unused snapshots from repository Snapshots (started 2017-07-11T16:19:32+02:00, runtime 20:12:56.538) |
| jvm 1 | 2017-07-12 12:32:37,767+0200 INFO [pxpool-1-thread-7] admin org.sonatype.nexus.tasks.EmptyTrashTask - Scheduled task (Empty Snapshot Trash) started :: Emptying Trash. |
| jvm 1 | 2017-07-12 12:37:18,890+0200 INFO [pxpool-1-thread-7] admin org.sonatype.nexus.tasks.EmptyTrashTask - Scheduled task (Empty Snapshot Trash) finished :: Emptying Trash. (started 2017-07-12T12:32:37+02:00, runtime 0:04:41.122) |
After 20 hours task was finished, and nothing was changed:
| root@vmnexus01:~# ll /opt/nexus-repositories/storage/snapshots/xxx/xxx/xxx-api/ |
| total 900 |
| drwxr-xr-x 14 nexus nexus 8192 Jul 12 16:02 . |
| drwxr-xr-x 73 nexus nexus 20480 Jul 12 12:32 .. |
| drwxr-xr-x 2 nexus nexus 8192 Apr 18 2016 1.0.26-SNAPSHOT |
snapshots from April still there, even after
| Scheduled task (Empty Snapshot Trash) finished |
Can anyone here explain me, please - how we can figure out with it ?
The current setup of scheduled task is:
Our current version it Sonatype Nexus™ 2.11.2-06
Thanks in advance!
Rich Seddon
Jul 12, 2017, 12:55:48 PM7/12/17
to Nexus Users
Your listing only shows a directory:
| drwxr-xr-x 2 nexus nexus 8192 Apr 18 2016 1.0.26-SNAPSHOT |
What is in that directory, and what are the dates of the files?
--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users+unsubscribe@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.
To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/c74a5384-7424-4a30-a1fb-6afcf61cf853%40glists.sonatype.com.
For more options, visit https://groups.google.com/a/glists.sonatype.com/d/optout.
epiq sty
Jul 12, 2017, 1:01:32 PM7/12/17
to Nexus Users
Rich, here is listing :
drwxr-xr-x 2 nexus nexus 8192 Apr 18 2016 .
drwxr-xr-x 15 nexus nexus 8192 Jul 12 17:01 ..
-rw-r--r-- 1 nexus nexus 1209 Apr 18 2016 maven-metadata.xml
-rw-r--r-- 1 nexus nexus 32 Apr 18 2016 maven-metadata.xml.md5
-rw-r--r-- 1 nexus nexus 40 Apr 18 2016 maven-metadata.xml.sha1
-rw-r--r-- 1 nexus nexus 35557548 Apr 14 2016 xxx-api-1.0.26-20160414.144236-1.jar
-rw-r--r-- 1 nexus nexus 32 Apr 14 2016 xxx-api-1.0.26-20160414.144236-1.jar.md5
-rw-r--r-- 1 nexus nexus 40 Apr 14 2016 xxx-api-1.0.26-20160414.144236-1.jar.sha1
-rw-r--r-- 1 nexus nexus 8711 Apr 14 2016 xxx-api-1.0.26-20160414.144236-1.pom
-rw-r--r-- 1 nexus nexus 32 Apr 14 2016 xxx-api-1.0.26-20160414.144236-1.pom.md5
-rw-r--r-- 1 nexus nexus 40 Apr 14 2016 xxx-api-1.0.26-20160414.144236-1.pom.sha1
-rw-r--r-- 1 nexus nexus 31935682 Apr 14 2016 xxx-api-1.0.26-20160414.144236-1-runtime.zip
-rw-r--r-- 1 nexus nexus 32 Apr 14 2016 xxx-api-1.0.26-20160414.144236-1-runtime.zip.md5
-rw-r--r-- 1 nexus nexus 40 Apr 14 2016 xxx-api-1.0.26-20160414.144236-1-runtime.zip.sha1
-rw-r--r-- 1 nexus nexus 170366 Apr 14 2016 xxx-api-1.0.26-20160414.144236-1-sources.jar
-rw-r--r-- 1 nexus nexus 32 Apr 14 2016 xxx-api-1.0.26-20160414.144236-1-sources.jar.md5
-rw-r--r-- 1 nexus nexus 40 Apr 14 2016 xxx-api-1.0.26-20160414.144236-1-sources.jar.sha1
-rw-r--r-- 1 nexus nexus 35563929 Apr 18 2016 xxx-api-1.0.26-20160418.084621-2.jar
-rw-r--r-- 1 nexus nexus 32 Apr 18 2016 xxx-api-1.0.26-20160418.084621-2.jar.md5
-rw-r--r-- 1 nexus nexus 40 Apr 18 2016 xxx-api-1.0.26-20160418.084621-2.jar.sha1
-rw-r--r-- 1 nexus nexus 8711 Apr 18 2016 xxx-api-1.0.26-20160418.084621-2.pom
-rw-r--r-- 1 nexus nexus 32 Apr 18 2016 xxx-api-1.0.26-20160418.084621-2.pom.md5
-rw-r--r-- 1 nexus nexus 40 Apr 18 2016 xxx-api-1.0.26-20160418.084621-2.pom.sha1
-rw-r--r-- 1 nexus nexus 31943238 Apr 18 2016 xxx-api-1.0.26-20160418.084621-2-runtime.zip
-rw-r--r-- 1 nexus nexus 32 Apr 18 2016 xxx-api-1.0.26-20160418.084621-2-runtime.zip.md5
-rw-r--r-- 1 nexus nexus 40 Apr 18 2016 xxx-api-1.0.26-20160418.084621-2-runtime.zip.sha1
-rw-r--r-- 1 nexus nexus 174257 Apr 18 2016 xxx-api-1.0.26-20160418.084621-2-sources.jar
-rw-r--r-- 1 nexus nexus 32 Apr 18 2016 xxx-api-1.0.26-20160418.084621-2-sources.jar.md5
-rw-r--r-- 1 nexus nexus 40 Apr 18 2016 xxx-api-1.0.26-20160418.084621-2-sources.jar.sha1
-rw-r--r-- 1 nexus nexus 35563923 Apr 18 2016 xxx-api-1.0.26-20160418.094012-3.jar
-rw-r--r-- 1 nexus nexus 32 Apr 18 2016 xxx-api-1.0.26-20160418.094012-3.jar.md5
-rw-r--r-- 1 nexus nexus 40 Apr 18 2016 xxx-api-1.0.26-20160418.094012-3.jar.sha1
-rw-r--r-- 1 nexus nexus 8711 Apr 18 2016 xxx-api-1.0.26-20160418.094012-3.pom
-rw-r--r-- 1 nexus nexus 32 Apr 18 2016 xxx-api-1.0.26-20160418.094012-3.pom.md5
-rw-r--r-- 1 nexus nexus 40 Apr 18 2016 xxx-api-1.0.26-20160418.094012-3.pom.sha1
-rw-r--r-- 1 nexus nexus 31943118 Apr 18 2016 xxx-api-1.0.26-20160418.094012-3-runtime.zip
-rw-r--r-- 1 nexus nexus 32 Apr 18 2016 xxx-api-1.0.26-20160418.094012-3-runtime.zip.md5
-rw-r--r-- 1 nexus nexus 40 Apr 18 2016 xxx-api-1.0.26-20160418.094012-3-runtime.zip.sha1
-rw-r--r-- 1 nexus nexus 174258 Apr 18 2016 xxx-api-1.0.26-20160418.094012-3-sources.jar
-rw-r--r-- 1 nexus nexus 32 Apr 18 2016 xxx-api-1.0.26-20160418.094012-3-sources.jar.md5
-rw-r--r-- 1 nexus nexus 40 Apr 18 2016 xxx-api-1.0.26-20160418.094012-3-sources.jar.sha1
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users...@glists.sonatype.com.
Rich Seddon
Jul 12, 2017, 1:27:05 PM7/12/17
to Nexus Users
The "days since last requested" setting in the task should really be called "days since last accessed". Any form of access will update this count, and that includes browsing this directory via the Nexus UI (either via the "browse storage" tab or the html directory listing). Has that sort of access been done in the last 30 days for this directory?
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users+unsubscribe@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.
To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/12fdf447-412e-4b5e-b8eb-40e9f19c2862%40glists.sonatype.com.
epiq sty
Jul 13, 2017, 4:04:50 AM7/13/17
to Nexus Users
Sounds interesting, but which attributes exactly are used to get "days since last accessed" ?
Is it something like access time from :
# stat 1.0.26-SNAPSHOT/
File: `1.0.26-SNAPSHOT/'
Size: 8192 Blocks: 16 IO Block: 65536 directory
Device: 12h/18d Inode: 712719 Links: 2
Access: (0755/drwxr-xr-x) Uid: ( 999/ nexus) Gid: ( 1004/ nexus)
Access: 2017-07-13 05:55:38.713955000 +0200
Modify: 2016-04-18 11:40:16.130537000 +0200
Change: 2016-04-18 11:40:16.130537000 +0200
Birth: -
And if I do not want to keep this files, can probably I adjust access time, using "touch" unility with -a -t <old.timestamp> params?
Rich Seddon
Jul 13, 2017, 11:15:32 AM7/13/17
to Nexus Users
The last requested (accessed) time in kept in the attributes file associated with all files in repository storage. These files can be found in paths like this:
sonatype-work/nexus/storage/<repo-id>/.nexus/attributes/path/to/artifact
Where "path/to/artifact" is the path name of the artifact in the storage directory. The files in the hidden attributes directory are JSON files, and the attribute name where this information is stored is "storageItem-lastRequested". The format of the field is a long, the value is a standard unix timestamp.
Rich
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users+unsubscribe@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.
To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/17e72400-5cf0-4f17-af9f-f8a5b07e5201%40glists.sonatype.com.
epiq sty
Jul 14, 2017, 10:40:03 AM7/14/17
to Nexus Users
Rich, thanks for explanation!
Looks like "storageItem-lastRequested":"1499870453850", are referring to exact time, when we browsed it in nexus repo.
Do you know, is it possible to change Cleanup Unused Snapshots logic to use something like time, when artefact was last time downloaded?
And another question - may be you have any ideas why such task took so much time:
jvm 1 | 2017-07-12 12:32:29,326+0200 INFO [pxpool-1-thread-5] admin org.sonatype.nexus.maven.tasks.UnusedSnapshotRemoverTask - Scheduled task (Cleanup Unused Snapshots) finished :: Removing unused snapshots from repository Snapshots (started 2017-07-11T16:19:32+02:00, runtime 20:12:56.538)
jvm 1 | 2017-07-13 12:26:31,033+0200 INFO [pxpool-1-thread-12] *TASK org.sonatype.nexus.maven.tasks.UnusedSnapshotRemoverTask - Scheduled task (Cleanup Unused Snapshots) finished :: Removing unused snapshots from repository Snapshots (started 2017-07-12T16:16:00+02:00, runtime 20:10:30.965)
jvm 1 | 2017-07-14 10:21:16,988+0200 INFO [pxpool-1-thread-17] *TASK org.sonatype.nexus.maven.tasks.UnusedSnapshotRemoverTask - Scheduled task (Cleanup Unused Snapshots) finished :: Removing unused snapshots from repository Snapshots (started 2017-07-13T16:16:00+02:00, runtime 18:05:16.899)
it's every time about ~20 hours, while Cleanup Snapshots works only for 5-7 minutes:
jvm 1 | 2017-07-12 14:06:00,109+0200 INFO [pxpool-1-thread-13] *TASK org.sonatype.nexus.maven.tasks.SnapshotRemoverTask - Scheduled task (Cleanup Snapshots) finished :: Removing snapshots from repository Snapshots (started 2017-07-12T14:00:00+02:00, runtime 0:05:59.396)
jvm 1 | 2017-07-13 14:07:14,753+0200 INFO [pxpool-1-thread-8] *TASK org.sonatype.nexus.maven.tasks.SnapshotRemoverTask - Scheduled task (Cleanup Snapshots) finished :: Removing snapshots from repository Snapshots (started 2017-07-13T14:00:01+02:00, runtime 0:07:13.370)
jvm 1 | 2017-07-14 14:07:46,553+0200 INFO [pxpool-1-thread-3] *TASK org.sonatype.nexus.maven.tasks.SnapshotRemoverTask - Scheduled task (Cleanup Snapshots) finished :: Removing snapshots from repository Snapshots (started 2017-07-14T14:00:01+02:00, runtime 0:07:45.434)
Thanks in advance!
Rich Seddon
Jul 14, 2017, 12:32:26 PM7/14/17
to Nexus Users
Do you know, is it possible to change Cleanup Unused Snapshots logic to use something like time, when artefact was last time downloaded?
No, there isn't any way to do that.
And another question - may be you have any ideas why such task took so much time:
You need to upgrade. Significant performance improvements have been made to this task since Nexus 2.11.2 was released.
Rich
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users+unsubscribe@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.
To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/a0ced0d7-5a8a-4bb3-b88a-a04cda07d471%40glists.sonatype.com.
epiq sty
Jul 17, 2017, 9:53:18 AM7/17/17
to Rich Seddon, Nexus Users
Rich, thanks a lot!
At moment we solved most of problems by tuning day's thresholds, but if we faced with it again we will perform upgrade .
To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/CALX_hVzCKGMi_tPjZ%3Dj76pCHSuG_YO0qXKmYbxcJkfzKKTaAuw%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages