AWS S3 Image Upload with Golang

[mak...@gmail.com]

Hi,

I am trying to upload a local image file to an AWS S3 bucket and return the public URL in Golang. This is the core Golang code I have written to interact with my S3 bucket:

    creds := credentials.NewSharedCredentials("/Users/username/.aws/credentials", "default")

    config := &aws.Config{

        Region:      aws.String("us-west-2"),

        Credentials: creds,

    }

    sess := session.New(config)

    uploader := s3manager.NewUploader(sess, func(u *s3manager.Uploader) {

    u.PartSize = 64 * 1024 * 1024 

    u.LeavePartsOnError = true

    })

    fmt.Println(header.Filename)

    fileInfo, _ := out.Stat()

var size int64 =  fileInfo.Size()

fmt.Println("size", size)

buffer := make([]byte, size) 

out.Read(buffer)

fileBytes := bytes.NewReader(buffer)

    result, err := uploader.Upload(&s3manager.UploadInput{

        Bucket: aws.String("bucket-name"),

        Key: aws.String(header.Filename),

        Body: aws.ReadSeekCloser(fileBytes),

        ContentType: aws.String("image/jpeg"),

        ACL: aws.String("public-read"),

    })

    if err != nil || result == nil {

        log.Fatalln("Failed to upload", err)

    }

    log.Println("Successfully uploaded to", result.Location)

With this code, the file is uploaded successfully to my s3 bucket with the correct size in bytes, but when I click on the URL, a black square is displayed instead of the actual image. If I change the ContentType field to allow file types dynamically, when I click on the URL, it downloads the image, but the image cannot be opened because "it may have been damaged." How can I fix this Golang code to just upload an image to an S3 bucket and get the public URL as a result of that upload?

This may be more of a debugging question pertaining to AWS S3 than Golang, but any help would be appreciated. Thank you!

[helloPiers]

It's not clear what the type of "out" is, other than it has Stat() and Read([]byte) methods, but, assuming it's an io.Reader at least, if you examine the return values from Read, like:

  n, err := out.Read(buffer)

you may get a clue to what's going wrong, for example n might not be the full size of the file, there might be an error, etc. Likewise it's worth checking the error returned from Stat. 

It might be that "out" already is a ReadSeekCloser (if it's an *io.File for example) in which case you can set Body in the UploadInput just to "out":

 ...

 Body: out,

 ...

and dispense with all the stuff related to buffer.

[mak...@gmail.com]

Hi,

If I set Body to the out file itself, an item with the name of the outfile is uploaded to the S3 bucket, but that item is 0 bytes in size and when I click on it in S3 it doesn't appear, whereas with the buffer stuff, the image size in bytes is correct, but a black square shows up. This is how I get the out file in the first place:

file, header, err := r.FormFile("upload")

if err != nil {

fmt.Fprintln(w, err)

return

}

defer file.Close()

out, err := os.Create("/path/to/file/" + header.Filename)

if err != nil {

fmt.Fprintf(w, "Unable to create the file for writing. Check your write access privilege")

return

}

defer out.Close()

_, err = io.Copy(out, file)

if err != nil {

fmt.Fprintln(w, err)

}

I'm not sure why setting Body to the out file itself results in a 0 byte upload. Is there another way to upload the image to s3?

Thanks

[Ingo Oeser]

You seem to ignore the error from out.Close()

What is out.Close reporting after the io.Copy?

[mak...@gmail.com]

err = out.Close()

if err != nil {

fmt.Println(err)

}

When I add this code after the io.Copy line, it gives me this error (attached screenshot to the email). It says upload multipart failed, failed to compute body hashes, and file already closed. I moved both the file.Close() and out.Close() lines to be after the io.Copy but I still got the same error.

[Steven Hartland]

If your file out code is above your original code snippet so you have something like:
https://play.golang.org/p/5MVxJamHy4c

Then one of the problems is the file pointer of "out" points to the end of your file not the beginning.

If you don't need the file on disk then something like the following should suffice:
https://play.golang.org/p/GbLdui4NsGK

    Regards
    Steve

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.