Spectra tape library as a irods resource

[Prashant Shete]

Hello,
I want to implement irods on Spectra tape library which is provided by Spectralogic.
Can anyone suggest me the approach for doing it? As far as I read it is possible through Universal mass storage plugin which is there in /plugins/resources/univmss but one need to modify the script iRODS/server/bin/cmd/univMSSInterface.sh according to storage.

So please help me with the commands for Spectra tape library which will be similar to mv, mkdir, cp , rm etc.

Thanks.

[Jean-Yves Nief]

hello,

are you using a HSM in front of your tape library ? If so,
you certainly have some client interfaces that can be used to fill all
the required funtions in the univMSS plugins.
If you are writing to the tapes directly, then you have to mount the
tape drives as raw devices. But univMSS is not designed to handle this
kind of use cases as your library tapes are not seen as a single
physical resource: you will have to handle this on your own and
functions like mkdir, chmod etc are not relevant in that case.
If you are using LTO tapes, you could use LTFS in order to see each of
your tapes like a filesystem (you would have to pay to have a LTFS
handling lots of tapes for you without handling each of the filesystems:
it could be feasible to do it using iRODS though).
hope this help a bit,
JY

[Dan Bedard]

Hi Jean-Yves,

Have you ever written a UnivMSS script to connect iRODS to the API of a backup/recovery solutions, such as, e.g., EMC Networker?

Dan

> --
> --
> "iRODS: the Integrated Rule-Oriented Data-management System; A community driven, open source, data grid software solution" https://www.irods.org
>
> iROD-Chat: http://groups.google.com/group/iROD-Chat
>
> --- You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Tom Langborg]

Hi Dan

We have (Janos) have written a interface to TSM for iRODS tape storage.

It's still under test but is look good so far.

Remember tape is still tape.

Don't count on high speed with retrieving many files.

regards

tom 

[Jean-Yves Nief]

hello Dan,

when I wrote the UnivMSS driver, I had in mind that people have
command line interface to their HSM. So it cannot use APIs directly.
I think that some people have interfaced iRODS with TSM quite a long
time ago, but not quite sure about this;
cheers,
JY

[Jean-Yves Nief]

Tom Langborg wrote:
> Hi Dan
>
> We have (Janos) have written a interface to TSM for iRODS tape storage.
>
> It's still under test but is look good so far.
>
> Remember tape is still tape.
> Don't count on high speed with retrieving many files.

well, it depends on the number of tape drives you have. And although
there is the overhead of the tape mount and time to seek the right
sector on the tape, the transfer rates are pretty decent (> 200 MB/s
depending on the tape drive techs). We have been able to achieve 150
TBs/day retrieved from tapes.
JY

>
> regards
> tom
>
> Den onsdag 2 mars 2016 kl. 15:30:40 UTC+1 skrev Dan Bedard:
>
> Hi Jean-Yves,
>
> Have you ever written a UnivMSS script to connect iRODS to the API
> of a backup/recovery solutions, such as, e.g., EMC Networker?
>
> Dan
>
> > On Mar 2, 2016, at 4:13 AM, Jean-Yves Nief <ni...@cc.in2p3.fr

> <javascript:>> wrote:
> >
> > hello,
> >
> > are you using a HSM in front of your tape library ?
> If so, you certainly have some client interfaces that can be used
> to fill all the required funtions in the univMSS plugins.
> > If you are writing to the tapes directly, then you have to mount
> the tape drives as raw devices. But univMSS is not designed to
> handle this kind of use cases as your library tapes are not seen
> as a single physical resource: you will have to handle this on
> your own and functions like mkdir, chmod etc are not relevant in
> that case.
> > If you are using LTO tapes, you could use LTFS in order to see
> each of your tapes like a filesystem (you would have to pay to
> have a LTFS handling lots of tapes for you without handling each
> of the filesystems: it could be feasible to do it using iRODS
> though).
> > hope this help a bit,
> > JY
>

> --
> --
> "iRODS: the Integrated Rule-Oriented Data-management System; A
> community driven, open source, data grid software solution"
> https://www.irods.org
>
> iROD-Chat: http://groups.google.com/group/iROD-Chat
>
> ---
> You received this message because you are subscribed to the Google
> Groups "iRODS-Chat" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to irod-chat+...@googlegroups.com

> <mailto:irod-chat+...@googlegroups.com>.

[Tom Langborg]

Den onsdag 2 mars 2016 kl. 16:20:34 UTC+1 skrev Jean-Yves Nief:

Tom Langborg wrote:

> Remember tape is still tape.
> Don't count on high speed with retrieving many files.
well, it depends on the number of tape drives you have. And although
there is the overhead of the tape mount and time to seek the right
sector on the tape, the transfer rates are pretty decent (> 200 MB/s
depending on the tape drive techs). We have been able to achieve 150
TBs/day retrieved from tapes.
JY

Yes the streaming speed is good in tape. 

But winding tape is slooow.

The get command from irods is one per file.

That will give one tsm command per file to the tsm system.

So if you have many file it will be slooow.

It will speed up if we can get a collection of files instead from irods to the tsm.

But it will still be slow but not so slow.

Retrieving many from tape system is always slow comparing to disk.

/tom

  

[Janos Nagy]

Hello,

I can send you my source, it is a univMSS interface calling the TSM API.

Janos