communication failure to S3 - UnknownHostException

[Mia Shani]

James,

I have been uploading lots of data to my S3 buckets, as backup.

I check existence and last update time for each file I am trying to upload.

all went well for a long time, but suddenly I see exception:

 file Jennings_-_Wilson_12-22-2012_217.jpg uploaded to S3 (14:26:49)

failed to examine existance: 433556/jennings_-_wilson_12-22-2012_218.jpg from bucket digilabs.galleries.6

Throwable: (org.jets3t.service.ServiceException):2013-05-11 14:27:01:257

Throwable: Request Error: java.net.UnknownHostException: digilabs.galleries.6.s3.amazonaws.com

at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:574)

at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:281)

at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRestHead(RestStorageService.java:942)

at org.jets3t.service.impl.rest.httpclient.RestStorageService.getObjectImpl(RestStorageService.java:2148)

at org.jets3t.service.impl.rest.httpclient.RestStorageService.getObjectDetailsImpl(RestStorageService.java:2075)

at org.jets3t.service.StorageService.getObjectDetails(StorageService.java:1093)

at org.jets3t.service.StorageService.getObjectDetails(StorageService.java:548)

at org.jets3t.service.StorageService.isObjectInBucket(StorageService.java:485)

at com.digilabs.amazonS3.utils.BucketUtils.s3FileExists(BucketUtils.java:578)

<etc>

I have been uploading several files to this exact same bucket, then this happened for another file in the same bucket, then after a while... (at 14:27:03 by my debug prints) uploads resumed to another bucket. 

Later (17:08:04 by my debug clock) I was able to upload to same bucket a large number of files.

Is it a disconnect from AWS S3 or is it something that I am doing wrong?

incidentally the missed files uploaded successfully the next time I tried.

thanks

Mia

[Mia Shani]

Just wanted to add, in the code, line BucketUtils.java:578 is:

if (s3Service.isObjectInBucket(bucketName, key))

exists = true;

where 

RestS3Service s3Service = new RestS3Service(credentials);

thanks

Mia

[James Murty]

Hi Mia,

It's hard to know for sure what went wrong in this case. I doubt it's anything you did, more likely a problem on Amazon's side or a transient Internet/connectivity hitch which would be almost impossible to debug after the fact.

One thing to watch out for is that long-running Java programs can cache DNS lookups for far too long, potentially breaking Amazon's assumption that server locations can change safely over time because occasional DNS checks will reflect the new location. There's a long, boring and unresolved ticket about this issue here [1]. This would only matter to you if your Java program runs for a long time without restarting.

J

[1]: https://bitbucket.org/jmurty/jets3t/issue/151/do-not-overwrite-networkaddresscache

---
jamesmurty.com

--
You received this message because you are subscribed to the Google Groups "JetS3t Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jets3t-users...@googlegroups.com.
To post to this group, send email to jets3t...@googlegroups.com.
Visit this group at http://groups.google.com/group/jets3t-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Mia Shani]

Thanks James. Actually my Java program IS running for a long time without restarting - I have very long lists of files which I am checking for freshness and possibly uploading. 

The interesting thing is, it resumes for the next file just fine.

I will read teh long and boring discussion and see what action I can take.

all the best 

Mia

[guma...@gmail.com]

hi Mia, I know it's been awhile but did you end up figuring out what the solution was?