Does S3 to S3 file transfers trigger AWS data egress fees?

[David Nix]

Hello Folks,

The copy documentation states that data transfers between endpoints are routed through one's local Cyberduck instance.  Does this apply to file transfers between S3 buckets in the same zone?  If so, does this trigger AWS egress fees?   Any plans to use the S3 api to avoid the egress fees and speed up the transfers for S3 to S3? (Within S3 transfers with the cli are free.)  We share large datasets (e.g. 10s of TBs) on S3 by copying the files into a users bucket in another account where they can do what they like with the files.  Data egress at this level is expensive ($90/TB), so avoiding these charges is important.

Thanks,

David

Huntsman Cancer Institute

[Hendrik Visage]

Hmm... lets ask you the question this way:
  How will Cyberduck *know* that it's the same S3 provider? - I use "S3" compatibles all the time, and now it'll make special code handling I suspect

  Thus, using the CLI would be much easier, especially if you need to do this more than once, to script/automate it.

[David Nix]

Yes Cyberduck would need to check if the endpoints are on the same platform and in the same “Region" and use the Java AWS SDK for the transfer. I suspect most of the cloud providers make this distinction between internal transfers vs external internet egress and bill differentially. Like AWS, GCP and Azure are free to upload data into their systems and move it internally, but charge when you take it out.  Adding this modality to Cyberduck (if it doesn’t have it) would be a huge boost for big data folks.  It would also speed up the transfers.

Agreed, our bioinformaticians use the AWS CLI for work in the core but for many of our non-cmd line savvy clients, they need a GUI.  

Thx.

--
--
You received this message because you are subscribed to the Google Groups "Cyberduck" group. To post to this group, send email to cybe...@googlegroups.com
To unsubscribe from this group, send email to cyberduck+...@googlegroups.com
For more options, visit this group at https://groups.google.com/group/cyberduck
--
Post bug reports and feature requests
https://trac.cyberduck.io/newticket
--
Support development
https://cyberduck.io/donate/
---
You received this message because you are subscribed to a topic in the Google Groups "Cyberduck" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cyberduck/7s4uLFL9zEY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cyberduck+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cyberduck/2c5c387c-4a5a-470f-9df4-dd7663487158n%40googlegroups.com.