Writing files to Amazon S3

[Simon Goldschmidt]

The process of writing a file to Amazon S3 appears very simple, but I haven't been able to make it work. Hope someone can point out where I'm going wrong writing a file to an S3 bucket.

I'm using a new bucket in the Sydney region where Everyone has List and Upload/Delete permissions. I'm using the credentials for an IAM account that is in the Admin group. I have set variables for the access key, secret code and bucket name.

The line:

  <cfset files = directoryList("s3://#access-key#:#secret-code#@s3-ap-southeast-2.amazonaws.com/#bucket-name#")>

returns the error "directory [...directory above...] doesn't exist"

And the line:

  <cffile action="write" output="Sample text" file="s3://#access-key#:#secret-code#@s3-ap-southeast-2.amazonaws.com/#bucket-name#/sample.txt">

returns the error "Parent directory for [...file above...] doesn't exist"

Have I missed something? Lucee 4.5.1.023 with Tomcat 7 and Java 1.8 on Windows 8.1

Simon

[Michael van Leest]

That location might require a newer version signature (V4) which isn't supported at this time. I believe the Germany location requires signature v4 as well.

Try directoryList("s3://#access-key#:#secret-code#@#bucket-name#.s3-ap-southeast-2.amazonaws.com")

If that doesn't work either, try a cfc like https://github.com/mso-net/lucee-aws

Kind regards,

Michael

--
See Lucee at CFCamp Oct 22 & 23 2015 @ Munich Airport, Germany - Get your ticket NOW - http://www.cfcamp.org/
---
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/98d26a5a-ef13-4fa5-be9f-c00fdb918b16%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Michael van Leest

[Simon Hooker]

Hi there,

That sounds like an issue where the S3ResourceProvider can struggle to identify a "directory" because S3 has no concept of directories.  Depending how the directory was created, sometimes it then doesn't quite work as you'd expect.  This compounds with the SignatureV4 issue.

So, as Michael has already mentioned, I'd recommend checking out https://github.com/mso-net/lucee-aws.  It has methods to help you create directories as well as upload files even to newer bucket locations that require Signature V4 :)

Sample code for this can be seen in https://github.com/mso-net/lucee-aws/blob/master/tests/s3.cfc but adapting your example I can write a simple text file

<cfscript>
 // Init the object
 s3 = new aws.s3( account = access-key , secret = secret-code , bucket = bucket-name );


 // Write a file
 s3.putObject(
  key = '/sample.txt',
  object = 'data:text/plain;base64,'&ToBase64( 'Sample text' )
 );
</cfscript>

[Simon Goldschmidt]

I have written a simple script to upload a file to S3 using the REST API and Signature 4.... see the sample below with changed bucket, awsid and awssecret. I was very careful with the formatting, but get a "SignatureDoesNotMatch" response.

I noticed that sometimes the hmac() function did not return the same result as the examples in the AWS documentation. 

Am I doing something wrong here? Could it be that the hmac() function isn't returning what it should?

Simon

<cfif isdefined("form.image")>

  <cfset payload=form.image>

  <cfset time=dateadd("s",GetTimeZoneInfo().UTCTotalOffset,now())>

  <cfset region="ap-southeast-2">

  <cfset hostname="s3-ap-southeast-2.amazonaws.com">

  <cfset bucket="mybucket">

  <cfset filename="test.jpg">

  <cfset awsid="myawsid">

  <cfset awssecret="myawssecret">

  <cfset myrequest="PUT#chr(10)#http://#bucket#.#hostname#/#filename##chr(10)##chr(10)#host:#bucket#.#hostname##chr(10)#x-amz-content-sha256:#lcase(hash(payload,"sha-256"))##chr(10)#x-amz-date:#dateformat(time,"yyyymmdd")#T#timeformat(time,"HHmmss")#Z#chr(10)##chr(10)#host;x-amz-content-sha256;x-amz-date#chr(10)##lcase(hash(payload,"sha-256"))##chr(10)#">

  <cfset mystring="AWS4-HMAC-SHA256#chr(10)##dateformat(time,"yyyymmdd")#T#timeformat(time,"HHmmss")#Z#chr(10)##dateformat(time,"yyyymmdd")#/#region#/s3/aws4_request#chr(10)##lcase(hash(myrequest,"sha-256"))##chr(10)#">

  <cfset string1=tobinary(hmac(dateformat(time,"yyyymmdd"),"AWS4#awssecret#","hmacsha256"))>

  <cfset string2=tobinary(hmac(region,string1,"hmacsha256"))>

  <cfset string3=tobinary(hmac("s3",string2,"hmacsha256"))>

  <cfset mykey=tobinary(hmac("aws4_request",string3,"hmacsha256"))>

  <cfset signature=lcase(hmac(mystring,mykey,"hmacsha256"))>

  <cfhttp url="http://#bucket#.#hostname#/#filename#" method="PUT" timeout="10">

  <cfhttpparam type="header" name="Authorization" value="AWS4-HMAC-SHA256 Credential=#awsid#/#dateformat(time,"yyyymmdd")#/#region#/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=#signature#">

  <cfhttpparam type="header" name="Content-Length" value="#len(payload)#">

  <cfhttpparam type="header" name="Content-Type" value="image/jpeg">

  <cfhttpparam type="header" name="Host" value="#bucket#.#hostname#">

  <cfhttpparam type="header" name="x-amz-content-sha256" value="#lcase(hash(payload,"sha-256"))#">

  <cfhttpparam type="header" name="x-amz-date" value="#dateformat(time,"yyyymmdd")#T#timeformat(time,"HHmmss")#Z">

  <cfhttpparam type="body" value="#payload#">

  </cfhttp>

  <cfoutput>#cfhttp.statuscode#<br><pre>#cfhttp.filecontent#</pre><p></cfoutput><a href="test.cfm">> Return</a>

<cfelse>

  <form action="test.cfm" enctype="multipart/form-data" method="post">

    Upload image: <input name="image" type="file"> <input type="submit" value="Upload">

  </form>

</cfif>

[Denard Springle]

Hey Simon,

   I don't think you need to run the hmac's through toBinary() - looking at the Python example given by AWS I don't see anywhere they are binary encoding the hashes. I'd give it a shot without the toBinary() and see what you get. Not 100% but you may need to lcase() your hmac() results in each step (string1, string2, etc). as well to get a proper signature. Just guesses, mind you... but worth a shot ;) Hope they help!

-- Denny

[John Berquist]

Simon,

This came up on the CFML Slack group recently. Hmac() returns a hex string, so you can't use toBinary() on it. Instead you need to use binaryDecode(string, 'hex'). So you would have:

<cfset string1=binaryDecode(hmac(dateformat(time,"yyyymmdd"),"AWS4#awssecret#","hmacsha256"), "hex")>

 <cfset string2=binaryDecode(hmac(region,string1,"hmacsha256"), "hex")>

 <cfset string3=binaryDecode(hmac("s3",string2,"hmacsha256"), "hex")>

 <cfset mykey=binaryDecode(hmac("aws4_request",string3,"hmacsha256"), "hex")>

 <cfset signature=lcase(hmac(mystring,mykey,"hmacsha256"))>

Note that AWS signature v4 expects a hex encoded result, so you don't binary decode the last line (as you have it already). Also this does mean that the variables names 'stringX' will be misleading, as the signing key for each intermediate step is in binary format.

Hope this helps,

John

On Wednesday, September 23, 2015 at 5:02:16 AM UTC-4, Simon Goldschmidt wrote:

[James Holmes]

Are you aware that "Everyone" literally means everyone with an AWS account? Do you intend for other people to be able to upload anything they want to your bucket?

--