bagIT and AmazonS3

[edwardiglesias]

Hello,

Has anyone been been able to use bagIT's chekpayloadsum on S3
remotely? We are able to use it fine on local data AWS is another
matter.


~~~~~~~~~~~~~
Edward Iglesias
Systems Librarian
Central Connecticut State University

[Brian Vargas]

Edward,

I'm not sure what scenario you're thinking of here. Are you trying to
verify bagged data on S3 using an EC2 instance? Or something else?

Brian

On Nov 4, 2009, at 12:43, edwardiglesias <edwardi...@gmail.com>
wrote:

[Edward Iglesias]

We are still pretty new at this.  We managed to make our S3 buckets appear to show up like a standard network drive.  When we tried to run verify protocols against them they failed.  I suspect it is because S3 only allows read write permissions and there is no way to execute a checksum.


Edward Iglesias

[Brian Vargas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Edward,

If the drive is properly mapped, I can't imagine any reason why you
wouldn't be able to run a verification on it. Of course, the data will
have to get pulled back down from S3, which could be painfully slow for
a bag of any significant size.

But without knowing what tools you're using to map the drive, or a log
file with errors in it, there's not much I can do to help further.

Brian

Edward Iglesias wrote:
> We are still pretty new at this. We managed to make our S3 buckets
> appear to show up like a standard network drive. When we tried to run
> verify protocols against them they failed. I suspect it is because S3
> only allows read write permissions and there is no way to execute a
> checksum.
>
>
> Edward Iglesias
>
>
> On Thu, Nov 5, 2009 at 8:44 AM, Brian Vargas <br...@ardvaark.net
> <mailto:br...@ardvaark.net>> wrote:
>
>
> Edward,
>
> I'm not sure what scenario you're thinking of here. Are you trying to
> verify bagged data on S3 using an EC2 instance? Or something else?
>
> Brian
>
> On Nov 4, 2009, at 12:43, edwardiglesias <edwardi...@gmail.com

> <mailto:edwardi...@gmail.com>>

> wrote:
>
> >
> > Hello,
> >
> > Has anyone been been able to use bagIT's chekpayloadsum on S3
> > remotely? We are able to use it fine on local data AWS is another
> > matter.
> >
> >
> > ~~~~~~~~~~~~~
> > Edward Iglesias
> > Systems Librarian
> > Central Connecticut State University
> >
> > >
>
>
>
>
> >

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: What is this? http://pgp.ardvaark.net
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEAREDAAYFAkry51gACgkQ3YdPnMKx1eNOywCgz0556SsBNF3MpYtJe46Oq5XO
Qz8Anj4b6iszx6hNzLPaYN//G2JKRccm
=luc+
-----END PGP SIGNATURE-----

[Edward Iglesias]

Thanks Brian.  It was the

data will
have to get pulled back down from S3, which could be painfully slow for
a bag of any significant size.

 

part we were trying to avoid. We were trying to run the tools while the data was actually mounted on S3.

Edward Iglesias

[Andy Boyko]

Wouldn't you have to be running the tools on Amazon EC2, if you wanted the checksumming to happen locally to data housed on S3? Merely mapping the drive on your computer doesn't bring the data any closer to you, so it doesn't make the tools any more efficient to run -- it'll still pull every bit down the wire to you.

-Andy