[JIRA] (JENKINS-56740) Artifacts bigger than 5GB can't be uploaded to S3

[bozaro@yandex.ru (JIRA)]

Artem V. Navrotskiy created an issue

Jenkins / JENKINS-56740 Artifacts bigger than 5GB can't be uploaded to S3 Issue Type: Bug Assignee: Unassigned Components: artifact-manager-s3-plugin Created: 2019-03-25 11:53 Priority: Critical Reporter: Artem V. Navrotskiy Unfortunately S3 have 5GB limit for single operation upload object: https://docs.aws.amazon.com/AmazonS3/latest/dev/UploadingObjects.html Upload objects in a single operation—With a single PUT operation, you can upload objects up to 5 GB in size. Upload objects in parts—Using the multipart upload API, you can upload large objects, up to 5 TB. This plugin don't implement multipart upload and can't upload big files to S3. Add Comment

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

[jglick@cloudbees.com (JIRA)]

Jesse Glick started work on JENKINS-56740

Change By: Jesse Glick Status: Open In Progress

Add Comment

[jglick@cloudbees.com (JIRA)]

Jesse Glick assigned an issue to Artem V. Navrotskiy

Jenkins / JENKINS-56740 Artifacts bigger than 5GB can't be uploaded to S3

Change By: Jesse Glick Assignee: Artem V. Navrotskiy

Add Comment

[jglick@cloudbees.com (JIRA)]

Jesse Glick updated JENKINS-56740

Jenkins / JENKINS-56740 Artifacts bigger than 5GB can't be uploaded to S3

Change By: Jesse Glick Status: In Progress Review

Add Comment

[d.stanhope@qsrinternational.com (JIRA)]

D Stanhope commented on JENKINS-56740

Re: Artifacts bigger than 5GB can't be uploaded to S3 I quite like this plugin, and have also run into this issue. For now, I have coded a PowerShell script I can call like a function, using Write-S3Object (Which auto-detects file size and uses multi-part upload as necessary) to upload a file or folder. If this is useful to others I'm happy to post it here.

Add Comment

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)

[carlos@barcenilla.com.ar (JIRA)]

Carlos Barcenilla commented on JENKINS-56740

Re: Artifacts bigger than 5GB can't be uploaded to S3

D Stanhope Could you share the details of your PowerShell workaround?

Add Comment

[d.stanhope@qsrinternational.com (JIRA)]

D Stanhope updated an issue

Jenkins / JENKINS-56740

Artifacts bigger than 5GB can't be uploaded to S3

Change By: D Stanhope Attachment: Copy-ToAwsS3.psm1

Add Comment

[d.stanhope@qsrinternational.com (JIRA)]

D Stanhope commented on JENKINS-56740

Re: Artifacts bigger than 5GB can't be uploaded to S3 Here's a generalised version: Copy-ToAwsS3.psm1 Documentation is in the file. Note that you'll probably want to find a different way of handling credentials than what that file implements, or at least switch to using PowerShell Secure Strings Hope that helps!

Add Comment