Need help with Synchronize File Status service

[Rob McLear]

Hi,

I am really hoping to get an answer to this question, it is the last issue remaining that is keeping me from deploying this dcm4chee installation.

I have installed 2.17.3-mysql on an AWS EC2 based instance of Ubuntu 12.04.2LTS. 

There are two configured file systems:

FileSystem[pk=1, archive, groupID=ONLINE_STORAGE, aet=petradpacs, ONLINE, RW+, userinfo=null]
FileSystem[pk=2, tar:/mnt/s3, groupID=NEARLINE_STORAGE, aet=petradpacs, NEARLINE, RO, userinfo=null]

FileCopy is set to copy all incoming studies from the ONLINE file system to the NEARLINE file system. This works.

I have the ONLINE file system configured to delete old studies with the following criteria; status must be "ARCHIVED" and a copy must be on the NEARLINE file system:

Deleter Criteria: 

  1) Copy on Filesystem Group [Ljava.lang.String;@415d24

  2) Copy must be archived

This does NOT work. 

If I remove the requirement that the status must be "ARCHIVED", then deletion occurs appropriately.  So, I do not think that the Synchronize File Status Service is working correctly (i.e. .tar files are not being verified, status is not being changed to ARCHIVED)

The server.log file shows the following errors for each file that the system attempts to synchronize:

2013-04-07 13:34:52,987 INFO  -> (Thread-35825) [org.dcm4chex.archive.hsm.module.HSMCommandModule] queryStatus: mmls /archive/2013/4/7/8/ADBD0797/3299839C/7CCBFC62

2013-04-07 13:34:53,021 ERROR -> (Thread-35825) [org.dcm4chex.archive.hsm.module.HSMCommandModule] Failed to execute mmls /archive/2013/4/7/8/ADBD0797/3299839C/7CCBFC62

These errors are generated using the FileCopyHSMModule with a setting of QueryCommand mmls /%d/%f

I have also tried using settings of mmls %d/%f, mmls /%f mmls %f, and the default of %i/%f; the failures persist, only the format of the resulting error message changes.

I have tried finding information on mmls but I found only one reference from the developers that it is a proprietary program; I've found no other information online which would enable me to troubleshoot the mmls query language.

So, I'm stuck. Is mmls the right command here? If so what is the syntax? Am I completely missing something?

I really appreciate input from anyone who can help me get this working. 

Thanks,

-Rob

[Alessio]

On 04/07/2013 07:44 PM, Rob McLear wrote:
>
> So, I'm stuck. Is mmls the right command here? If so what is the syntax? Am
> I completely missing something?

You must use the right command to verify the existence/consistency of
the "archived" file.
Maybe "mmls" is not suitable in your case.
And you must configure the return of the command.

For example, here we use Qstar HMS to manage tapes.
The configuration in FileCopyHSMModule is the following.
The right command for us, in QueryCommand, with such software, is
"/opt/QStar/bin/mmls %i/%f"
If I launch such command from the command line, I can get
"Archived" or "Out-of-cache", in each case the file is to "archived". If
mmls return any other string, the file is not to be considered archived.
So inside Pattern, as a regular expression, i have:
".*\bArchived\b.*|.*\bOut-of-cache\b.*"

Sorry but I can't be more explicit. I can only suppose that you can use
a script of your own. Or you can use the simple "ls".


Ciao
A

[Rob McLear]

Thanks, that's very helpful. I am not using any kind of HSM system; both file systems are just directly attached to the linux instance, and the nearline system is set up as a tar: system. So what would be the appropriate way to query a directly attached tar archive?

Thanks again,

-Rob

> --
> You received this message because you are subscribed to a topic in the Google Groups "dcm4che" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/dcm4che/UiZ4naVsa6c/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to dcm4che+u...@googlegroups.com.
> To post to this group, send email to dcm...@googlegroups.com.
> Visit this group at http://groups.google.com/group/dcm4che?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Alessio]

On 04/08/2013 03:56 PM, Rob McLear wrote:
> Thanks, that's very helpful. I am not using any kind of HSM system; both file systems are just directly attached to the linux instance, and the nearline system is set up as a tar: system. So what would be the appropriate way to query a directly attached tar archive?

The filesystem is a... filesystem? I mean: like an ext3 filesystem? Or
it is a tape?


Ciao
A

[Rob McLear]

It is Amazon S3 mounted using S3FS at /mnt/s3 .

As far as I know, it behaves like any conventionally attached file system.

Thanks,

-Rob

[Alessio]

On 04/08/2013 03:56 PM, Rob McLear wrote:

> Thanks, that's very helpful. I am not using any kind of HSM system; both file systems are just directly attached to the linux instance, and the nearline system is set up as a tar: system. So what would be the appropriate way to query a directly attached tar archive?
>

Have you solved your problem?

For testing purposes I have configured a dcm4chee to copy files on
another filesystem as tar files.

So I have defined a "tar:/longtermfs" as NEARLINE_STORAGE.
In FileCopy service, as DestinationFileSystem, I've difined
"tar:/longtermfs"
In SyncFileStatus service, as MonitoredFileSystem, I've defined such
file system.
At last, in FileCopyHSMModule.Command, as QueryCommand, I've defined
/usr/local/bin/mmls %i/%f

Such command is a simple bash script:

-----
#!/bin/bash

if [ "x$1" == "x" ]
then
echo Script Error
exit 2
fi

if [ -f $1 ]
then
echo Archived
exit 0
fi

echo Not Archived
exit 1
-----


So, if the file is present on the destination file system, I can assume
that the file has been copied, so I can consider them as archived.

Please note that I used (as it is by default) %i/%f and not %d/%f,
because the path that will be passed to the script will be
tar:/longtermfs/2013/3/2/172781
So %i must be the real path to the file system. Such option can be
configured before the creation of the new nearline storage filling
DefaultUserInformation (in FileSystemMgt.NEARLINE_STORAGE service), or
updating the database

UPDATE filesystem SET user_info='/longtermfs' WHERE
dirpath='tar:/dcm4nearlinefs';


Sorry if I'm a bit confusing. I hope, however, that this stuff may be
useful.
Ciao
A

[Rob McLear]

Thank you for the information. I will try it again using your script and see if I can make it work!

-Rob

[Rob McLear]

Hi,

I have instituted this script to check my NEARLINE tar archives per the very helpful e-mail below. I think that it is almost working, but the SyncFileStatusService seems to not be working correctly yet.

I wanted to start the verification of tar files slowly, as I've had problems with s3fs disconnecting when placed under heavy load, so I set SyncFileStatus to only evaluate a small number of files at first (20).

When the service runs, the log entries are as follows:

2013-07-09 13:05:57,945 INFO -> (Thread-416) [org.dcm4chex.archive.hsm.module.HSMCommandModule] queryStatus: /usr/local/bin/mmls /mnt/s3/2013/3/6/20/27EAC49F/D84C1EA9-2066621.tar
2013-07-09 13:05:58,050 INFO -> (Thread-416) [org.dcm4chex.archive.hsm.SyncFileStatusService] Verify tar file /mnt/s3/2013/3/6/20/27EAC49F/D84C1EA9-2066621.tar
2013-07-09 13:05:58,201 INFO -> (Thread-416) [org.dcm4chex.archive.hsm.module.HSMCommandModule] queryStatus: /usr/local/bin/mmls /mnt/s3/2013/3/6/20/27EAC49F/D84C1EAA-2183859.tar
2013-07-09 13:05:58,303 INFO -> (Thread-416) [org.dcm4chex.archive.hsm.SyncFileStatusService] Verify tar file /mnt/s3/2013/3/6/20/27EAC49F/D84C1EAA-2183859.tar
2013-07-09 13:05:58,441 INFO -> (Thread-416) [org.dcm4chex.archive.hsm.SyncFileStatusService] SyncFileStatus finished! changed files:0

However, the next time the SyncFileStatus process runs, it checks exactly the same group of files, and continues to do so over and over again. So, obviously I have something configured incorrectly.

I appreciate any help the community can offer.

Thanks,

-Rob

[Rob McLear]

Never mind! I tweaked the script so that the 'echo' statements matched the .*\bARCHIVED\b.* statement in the HSM module. Seems to work fine now.

Thanks,

-Rob

> To unsubscribe from this topic, visit https://groups.google.com/d/topic/dcm4che/UiZ4naVsa6c/unsubscribe.

> To unsubscribe from this group and all its topics, send an email to dcm4che+u...@googlegroups.com.
> To post to this group, send email to dcm...@googlegroups.com.

> Visit this group at http://groups.google.com/group/dcm4che.

[bjpl]

Could you paste your working script here?

Thank you,

bjpl

[Rob McLear]

This is my script:

#!/bin/bash

if [ "x$1" == "x" ]
then
echo Script Error
exit 2
fi

if [ -f $1 ]
then

echo ARCHIVED
exit 0
fi

echo NOARCH
exit 1


On Jul 26, 2013, at 11:12 AM, bjpl <bartek...@gmail.com> wrote:

> Could you paste your script here?
>
> Thank you,
> bjpl

>
> W dniu wtorek, 9 lipca 2013 19:26:32 UTC+2 użytkownik Rob McLear napisał:
>>
>> Never mind! I tweaked the script so that the 'echo' statements matched the
>> .*\bARCHIVED\b.* statement in the HSM module. Seems to work fine now.
>>
>> Thanks,
>>
>> -Rob
>>
>>

>> On Jul 9, 2013, at 1:12 PM, Rob McLear <robm...@me.com <javascript:>>

>>> On May 3, 2013, at 10:47 AM, Alessio <ale...@ftgm.it <javascript:>>

>> dcm4che+u...@googlegroups.com <javascript:>.
>>>> To post to this group, send email to dcm...@googlegroups.com<javascript:>.

>>
>>>> Visit this group at http://groups.google.com/group/dcm4che?hl=en.
>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>
>>>>
>>>
>>> --
>>> You received this message because you are subscribed to a topic in the
>> Google Groups "dcm4che" group.
>>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/dcm4che/UiZ4naVsa6c/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to

>> dcm4che+u...@googlegroups.com <javascript:>.
>>> To post to this group, send email to dcm...@googlegroups.com<javascript:>.

[bjpl]

Thanks, but it doesn't work ;-(

Maybe I do something wrong else...

All tar files are on my local filesystem

2013-07-26 17:21:01,740 INFO  -> (Thread-1561) [org.dcm4chex.archive.hsm.SyncFileStatusService] Verify tar file /srv/CT/2012/8/28/10/7B76E2FF/A3928CBA-1488843.tar

2013-07-26 17:21:01,768 INFO  -> (Thread-1561) [org.dcm4chex.archive.hsm.module.HSMCommandModule] queryStatus: /usr/local/bin/mmls tar:/srv/CT/2012/8/28/10/7B76E2FF/87006F96-1489247.tar

2013-07-26 17:21:01,770 ERROR -> (Thread-1561) [org.dcm4chex.archive.hsm.module.HSMCommandModule] Failed to execute /usr/local/bin/mmls tar:/srv/CT/2012/8/28/10/7B76E2FF/87006F96-1489247.tar

org.dcm4chex.archive.hsm.module.HSMException: queryStatus failed! Non-zero exit code(1) of /usr/local/bin/mmls tar:/srv/CT/2012/8/28/10/7B76E2FF/87006F96-1489247.tar

        at org.dcm4chex.archive.hsm.module.AbstractHSMModule.doCommand(AbstractHSMModule.java:131)

        at org.dcm4chex.archive.hsm.module.HSMCommandModule.queryStatus(HSMCommandModule.java:246)

        at sun.reflect.GeneratedMethodAccessor271.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)

        at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)

        at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:133)

        at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)

        at org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:142)

        at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)

        at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)

        at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)

        at org.dcm4chex.archive.hsm.SyncFileStatusService.queryHSM(SyncFileStatusService.java:557)

        at org.dcm4chex.archive.hsm.SyncFileStatusService.check(SyncFileStatusService.java:408)

        at org.dcm4chex.archive.hsm.SyncFileStatusService.check(SyncFileStatusService.java:386)

        at org.dcm4chex.archive.hsm.SyncFileStatusService$1$1.run(SyncFileStatusService.java:133)

        at java.lang.Thread.run(Thread.java:724)

[Rob McLear]

Check permissions? Is it allowed to execute the script? Mine are -rwxr-xr-x

-Rob

On Jul 26, 2013, at 11:26 AM, bjpl <bartek...@gmail.com> wrote:

> Thanks, but it doesn't work ;-(
> Maybe I do something wrong else...
>

> W dniu piątek, 26 lipca 2013 17:17:17 UTC+2 użytkownik Rob McLear napisał:
>>
>> This is my script:
>>
>> #!/bin/bash
>>
>> if [ "x$1" == "x" ]
>> then
>> echo Script Error
>> exit 2
>> fi
>>
>> if [ -f $1 ]
>> then
>> echo ARCHIVED
>> exit 0
>> fi
>>
>> echo NOARCH
>> exit 1
>>
>>

>> On Jul 26, 2013, at 11:12 AM, bjpl <bartek...@gmail.com <javascript:>>

[bjpl]

I have the same permisions: rwxr-xr-x or 100755

Mayby something else?

Thank you,

bjpl

[Rob McLear]

I'm out of ideas, sorry. It looks as if your command it set up correctly, I'm still using /usr/local/bin/mmls %i/%f as my invoked command, and the Regex for the string is .*\bARCHIVED\b.* and it works for me, with the exception of some issues with memory allocation, and that seems to be more of an issue with the VM I'm using it on than with the script itself.

-Rob

On Jul 26, 2013, at 11:34 AM, bjpl <bartek...@gmail.com> wrote:

> I have the same permisions: rwxr-xr-x or 100755
> Mayby something else?
>

> Thank you,
> bjpl

>
> W dniu piątek, 26 lipca 2013 17:27:44 UTC+2 użytkownik Rob McLear napisał:
>>
>> Check permissions? Is it allowed to execute the script? Mine are
>> -rwxr-xr-x
>>
>> -Rob
>>
>>
>>

>> On Jul 26, 2013, at 11:26 AM, bjpl <bartek...@gmail.com <javascript:>>

[bjpl]

I'm using %i/%f  or %d/%f with the same result.

Did you mean the permisions of script or .tar files?

My scrip has chmod as I wrote before but .tar files has -rw-r--r--  

bjpl

[Rob McLear]

I was referring to the script; my tar files are all set as -rw-r--r-- with root as the owner and group.

On Jul 26, 2013, at 11:46 AM, bjpl <bartek...@gmail.com> wrote:

> I'm using %i/%f or %d/%f with the same result.
>
> Did you mean the permisions of script or .tar files?
> My scrip has chmod as I wrote before but .tar files has -rw-r--r--
>
> bjpl
>
> W dniu piątek, 26 lipca 2013 17:38:04 UTC+2 użytkownik Rob McLear napisał:
>>
>> I'm out of ideas, sorry. It looks as if your command it set up correctly,
>> I'm still using /usr/local/bin/mmls %i/%f as my invoked command, and the
>> Regex for the string is .*\bARCHIVED\b.* and it works for me, with the
>> exception of some issues with memory allocation, and that seems to be more
>> of an issue with the VM I'm using it on than with the script itself.
>>
>> -Rob
>>
>>

>> On Jul 26, 2013, at 11:34 AM, bjpl <bartek...@gmail.com <javascript:>>

[bjpl]

Thank you for your answers.

bjpl

[Chaithika Prabudda]

Hi Rob,

i have the same problem as you had. tell me the steps you have followed to fixed your issue.

thank you

chaithika.

[Rob McLear]

It was really just figuring out the mmls script. I've posted it in this thread previously, but here it is:

#!/bin/bash

if [ "x$1" == "x" ]
then
echo Script Error
exit 2
fi

if [ -f $1 ]
then
echo ARCHIVED
exit 0
fi

echo NOARCH
exit 1

-Rob

[David Simic]

Hi guys,

This thread was really useful in helping me figure out backups... the mmls script was the main stumbling block for me as well, so thanks!

I did want to let you know that I wrote up my experience configuring automated backups to AWS S3 on my blog:

http://davidsimic.com/2016/08/08/dcm4chee-automated-backups-archiving-to-the-cloud-or-anyplace-really/

The post includes python scripts which implement the copy, fetch and mmls functionalities expected by dcm4chee (implemented using the s3 boto api). 

Might be of use to someone, even if it is just to have the process restated in a somewhat different manner.

Best,

David