Average size of objects that have been successfully uploaded?

[zackp]

I use both AWS S3 and GCS.  With AWS S3, each single upload is limited to 5GB maximum.  I have uploaded objects of a couple of GBs successfully many times.  With its Multipart Uploads, I can exceed the 5GB/object limit and have successfully uploaded objects around ~10GB so far by splitting the data into 500MB chunks.

AFAIK, GCS doesn't have the 5GB/object limit. These days, I do have some objects that are sized around 20GBs that I wish to put in GCS. What is my chance of uploading them successfully?  The fastest bandwidth that is available to me is a 10Mbps connection.  

That also leads me to the Q: what is the average size of objects that have been successfully uploaded into GCS?  Even a rough number would be great. I will use the info to prepare our data objects accordingly.  

Thanks,

-- Zack

[Google Cloud Storage Team]

Hi Zack,

I'll have to go through the appropriate channels to make sure it's safe to share numbers on average object size, but I can tell you that with the resumable upload feature you should be able to successfully upload an arbitrarily sized object, provided you're willing to pay us to store it.  You can have a look at the documentation for resumable uploads here.  If you want to test it out, you should use a recent version of the gsutil tool and we'll automatically use a resumable upload -- there's documentation on that here.  

If your connection can truly handle a 10 megabit sustained upload, a 20 GB file should upload in less than five hours.  That seems entirely reasonable to me.  If you'd like to store it with us, I recommend you just give it a try, and share your results with the forum here. 

If you really want to know how big our average object is, let me know -- I would be surprised if we're comfortable sharing that information, but if it's important to you I'm willing to ask.  

All the best,

Benson

Google Cloud Storage Team

--
You received this message because you are subscribed to the Google Groups "Google Cloud Storage" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gs-discussio...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.