Problems with direct uploads to S3

[Tim Uckun]

Hi. I am building a back end for a mobile  app and I would like to have the app be able to upload files directly to s3. I am following this documentation http://shrinerb.com/rdoc/files/doc/direct_s3_md.html and I need to use the static strategy (strategy B).

The back end is in sinatra.

So I have done this.

require "shrine/storage/s3"
get '/picture/upload' do

  s3_options = {
      access_key_id: Config.AWS_ACCESS_KEY_ID,
      secret_access_key: Config.AWS_SECRET_ACCESS_KEY,
      region: Config.AWS_REGION,
      bucket: Config.AWS_UPLOADS_BUCKET,
  }

  Shrine.storages = {
      cache: Shrine::Storage::S3.new(prefix: "cache", **s3_options),
      store: Shrine::Storage::S3.new(prefix: "store", **s3_options),
  }

  content_type :html
  erb :picture_upload
end

I just copied and pasted the view as well

<%
  presign = Shrine.storages[:cache].presign SecureRandom.hex,
                                            success_action_redirect: '/picture/upload/success',
                                            allow_any: ['utf8', 'authenticity_token']
%>
<form action="<%= presign.url %>" method="post" enctype="multipart/form-data">
  <input type="file" name="file">
  <% presign.fields.each do |name, value| %>
      <input type="hidden" name="<%= name %>" value="<%= value %>">
  <% end %>
  <input type="submit" value="Upload">
</form>

So I get the form but when I submit I get the following error from amazon.

<Error>

<Code>InvalidArgument</Code>

<Message>

Bucket POST must contain a field named 'key'. If it is specified, please check the order of the fields.

</Message>

<ArgumentName>key</ArgumentName>

<ArgumentValue/>

<RequestId>33417C217DC288E2</RequestId>

<HostId>

aMIzKulrJwQgbGn8cMWmXJ0OZUbX2ypRg+PJe0XAqEmJ8cHOazj14ctTztiww5EN5AMXx2N2+aE=

</HostId>

</Error>

I really need to do the static method because the client is using nativescript and it doesn't support s3 very well. 

What can do to make this work?

Thanks.

[Janko Marohnić]

According to this StackOverflow answer, the "file" field should be the last field in the form, not the first. Could you try whether that works? If it does, I'll update the documentation, and note to myself to actually test things before documenting them :)

Kind regards,

Janko

--
You received this message because you are subscribed to the Google Groups "Shrine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ruby-shrine+unsubscribe@googlegroups.com.
To post to this group, send email to ruby-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ruby-shrine/449b32e3-3e54-4ded-a62f-0e7f7a70785e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tim Uckun]

I did that and now am getting this error

<Error>

<Code>AccessDenied</Code>

<Message>

Invalid according to Policy: Policy Condition failed: ["starts-with", "$utf8", ""]

</Message>

<RequestId>E5FF497CA21D2F78</RequestId>

<HostId>

OakGjkmGIhI74j8jOLTh+6qkAaKYdGplFXbYkZvLtCPtfuh5rr99j5SdmrukXPVgtDbWUSoazLo=

</HostId>

</Error>

The following code does work (without shrine)

post '/picture_upload' do
  content_type :text

  file       = params[:file][:tempfile]
  filename   = params[:file][:filename]
   s3_file_name= "api/#{APP_ENV}/pictures/#{filename}"

   s3 = Aws::S3::Resource.new(region: Config.AWS_REGION)
  obj = s3.bucket(Config.AWS_UPLOADS_BUCKET).object(s3_file_name)
  result= obj.upload_file(file.path,  {acl: 'public-read'})
  resul

end

[Janko Marohnić]

Try removing the `allow_any: ['utf8', 'authenticity_token']` option. I added it because Rails form builder generates those automatically, but here you're using Sinatra so you won't have those fields.

To view this discussion on the web visit https://groups.google.com/d/msgid/ruby-shrine/CAGuHJrMt3-HpvVKEfC-1nopkYOmRnBV7oEsA3gSA_6NA8UoYxw%40mail.gmail.com.

[Tim Uckun]

Thanks that worked.

It put the file as a random name (I presume that's the securerandom.hex bit?).  It set the content type as  binary/octet-stream

How does the callback work? It didn't redirect my browser to the URL I specified although I specified a relative URL which I am hoping works.

[Janko Marohnić]

It put the file as a random name (I presume that's the securerandom.hex bit?).  It set the content type as  binary/octet-stream

S3 will always set "Content-Type: binary/octet-stream" by default, regardless of the key (it doesn't look at the file extension of the key). Since the presign field are generated before the file is selected, I'm afraid there is not much you can do about that other than uploading through your app.

Actually, there is one thing you could do, you could tell S3 to forget object fields when copying from temporary to permanent storage:

  plugin :restore_cached_data # extract MIME type of cached file on assignment

  plugin :upload_options, store: -> (io, context) do

    { metadata_directive: "REPLACE" } if io.is_a?(Shrine::UploadedFile)

  end

See shrine#105 for more details on `:metadata_directive`. This would still make the original file uploaded to temporary S3 storage with the default "Content-Type: binary/octet-stream", but when Shrine copies this file to permanent S3 storage it should automatically override Content-Type with the MIME type metadata value extracted by Shrine.

How does the callback work? It didn't redirect my browser to the URL I specified although I specified a relative URL which I am hoping works.

I'm pretty sure that S3 needs to have an absolute URL.

Kind regards,

Janko

[Tim Uckun]

Thanks.

That's kind of a bummer about the absolute URL.  I haven't set up sinatra for that yet.

This is the first non trivial sinatra app I have written and it's clear by now that by the time I get done it's going to be 90% of rails :)

[Janko Marohnić]

That's kind of a bummer about the absolute URL.  I haven't set up sinatra for that yet.

You don't need to set anything up, see the Generating URLs section of the Sinatra README ;)

To view this discussion on the web visit https://groups.google.com/d/msgid/ruby-shrine/CAGuHJrNvPQEbRSCrJE99nuzQM4TysgCe3-FeS6VBcGxwAfhrUg%40mail.gmail.com.