migrate nearline for old et new studies

[jeff22]

Hello,
Actually I have an ONLINE_STORAGE which is nearly to become full. I would like to migrate all the old studies to a NEARLINE_STORAGE not accessed for X days(1h in my example).

I found this article very helpfull: http://www.dcm4che.org/confluence/display/ee2/Using+Amazon+S3+for+NEARLINE+storage
I just replace type=S3 by type=FileBased for HSM service

I have altered the following settings from the default in the file system management service for ONLINE_STORAGE:
*DeleteStudyIfNotAccessedFor = 1h
*DeleteStudyOnlyIfStorageNotCommited=false
*DeleteStudyOnlyIfCopyOnMedia=false
*DeleteStudyOnlyIfCopyOnReadOnlyFileSystem=false
*ScheduleStudiesForDeletionInterval=10m
*DeleteStudyOnlyIfNotAccessedFor = 1h
*DeleteStudyOnlyIfCopyOnFileSystemOfFileSystemGroup = NEARLINE_STORAGE
*DeleteStudyOnlyIfCopyArchived = true
*ScheduleStudiesForDeletionOnSeriesStored = false

For newly series strored FileCopy service does its job. I have a copy on NEARLINE_STORAGE.

I have in fact two problems:
- Old studies are not moved to NEARLINE_STORAGE and and not deleted from ONLINE_STORAGE
- All my studies are in status ONLINE on web interface

server.log:
2013-07-16 13:49:35,234 INFO  -> (Thread-1342) [org.dcm4chex.archive.mbean.FileSystemMgt2Service] Check file system group LOSSY_STORAGE for deletion of orphaned private files
2013-07-16 13:49:39,546 INFO  -> (Thread-1343) [org.dcm4chex.archive.hsm.SyncFileStatusService] Start SyncFileStatus!
2013-07-16 13:49:39,812 INFO  -> (Thread-1343) [org.dcm4chex.archive.hsm.SyncFileStatusService] SyncFileStatus finished! changed files:0
2013-07-16 13:49:46,625 INFO  -> (Thread-1344) [org.dcm4chex.archive.mbean.FileSystemMgt2Service] Check file system group ONLINE_STORAGE for deletion of orphaned private files
2013-07-16 13:49:46,640 INFO  -> (Thread-1345) [org.dcm4chex.archive.mbean.FileSystemMgt2Service] Check file system group ONLINE_STORAGE for deletion of studies
2013-07-16 13:49:46,640 WARN  -> (Thread-1345) [org.dcm4chex.archive.mbean.FileSystemMgt2Service] No study found for clean up filesystem group ONLINE_STORAGE! Please check your configuration!
2013-07-16 13:49:46,640 WARN  -> (Thread-1345) [org.dcm4chex.archive.mbean.FileSystemMgt2Service] Trigger intervall: 1m
Trigger on SeriesStored:false
All studies not accessed for 1h.
 And studies not accessed for 1h when running out of disk space!
Deleter Criteria:
  1) External Retrievable
  2) Copy on Filesystem Group NEARLINE_STORAGE
  3) Copy must be archived


I really would appreciate any help (and sorry for my english !)

Thanks.

[jeff22]

I finally found a solution...
When you use FileCopy Service. New images stored are copied by this service and I wanted to move only old studies that are not accessed fo X days from ONLINE to NEARLINE.
I used FileMove Service. Studies on ONLINE_STORAGE are moved to NEARLINE_STORAGE where they aren't accessed for X days.
After You can specify parameters on NEARLINE_STORAGE for deleting studies older than X days.
Be carefull, to specifiy DestinationFileStatus to ARCHIVED otherwhise access_time while not be updated on table study_on_fs and so you will not be able to delete studies on nearline storage. By default, when you move a studie the line access_time for studie on table study_on_fs id deleted. So if not acces is done on the study you will not be able to delete studie on nearline. You can do an update via sql to insert one line whith date day for every files moved.
Sql I use in script is like this:
INSERT  INTO study_on_fs (study_fk, filesystem_fk, access_time, mark_to_delete)
SELECT DISTINCT series.study_fk, '2', '2013-07-22 14:33:43','\0' FROM series, instance, files
WHERE series.series_status = 0 AND files.filesystem_fk=2
AND series.pk = instance.series_fk AND instance.pk = files.instance_fk
AND NOT EXISTS (SELECT pk FROM study_on_fs WHERE study_on_fs.study_fk=series.study_fk)

where filesystem_fk=2 is for NEARLINE_STORAGE

What I regrets is that when you use FileMove Only Dicom files are moved. Repositories on source aren't deleted.
But I think it is a really good solution if you want to move only old studies that aren't accessed thin X days to NEARLINE fast and don't need to keep original on source(ONLINE_STORAGE).
I hope this could be usefull for someone

[GOKHAN DILEK]

When you change the destinationFilestatus to ARCHIVED, does that mean you can delete the studies and it deletes the stored files as well?

Thank you.