Dragonfly integration for on-the-fly processing

[br...@facca.info]

Thank you for this great gem!

I have followed [these intructions](https://twin.github.io/better-file-uploads-with-shrine-processing) in an attempt to integrate Shrine 2.5.0 with Dragonfly 1.1.1 for on-the-fly processing on Rails 4.2.7 with Active Record.

Here is my Dragonfly initializer:

    require 'dragonfly/s3_data_store'

    Dragonfly.app.configure do

      plugin :imagemagick

      url_format "/processed_images/:job"

      secret ENV.fetch("RAILS_SECRET_KEY")

      datastore :s3,

                bucket_name: ENV.fetch("S3_BUCKET"),

                region: ENV.fetch("S3_REGION"),

                access_key_id: ENV.fetch("S3_ACCESS_KEY_ID"),

                secret_access_key: ENV.fetch("S3_SECRET_ACCESS_KEY")

    end

    Dragonfly.logger = Rails.logger

    Rails.application.config.middleware.use Dragonfly::Middleware

At the view:

<%= image_tag Dragonfly.app.fetch(share_image.image.id).thumb("100x").url %>

The following error occurs:

    Started GET "/processed_images/W1siZiIsIjQwY2MwODcxYzUzNGJiMDQyOGE5OGI3NDc5ZmNjMWY5Il0sWyJwIiwidGh1bWIiLCIxMDB4Il0sWyJwIiwicm90YXRlIiwxODBdXQ?sha=77ef3a20acf1827b" for 10.0.2.2 at 2017-02-13 08:54:15 -0200

    DRAGONFLY: uid 40cc0871c534bb0428a98b7479fcc1f9 not found

    DRAGONFLY: GET /processed_images/W1siZiIsIjQwY2MwODcxYzUzNGJiMDQyOGE5OGI3NDc5ZmNjMWY5Il0sWyJwIiwidGh1bWIiLCIxMDB4Il0sWyJwIiwicm90YXRlIiwxODBdXQ?sha=77ef3a20acf1827b 404

Any idea on what caused the problem?

The only way I could get it to work was by having Shrine provide the image URLs to Dragonfly, like this:

<%= image_tag Dragonfly.app.fetch_url(share_image.image_url).thumb("100x").url %>

I also had to whitelist my S3 bucket host to the Dragonfly initializer:

fetch_url_whitelist [/awakeadmin-uploads-dev\.s3\.amazonaws\.com/]

 Are there any significant disadvantages to this approach?

Thank you.

Bruno Facca

[Janko Marohnić]

I might have forgotten to take into account the configured storage prefix, I think the following should work:

<%= image_tag Dragonfly.app.fetch([*photo.image.storage.prefix, photo.image.id]).thumb("100x").url %>

There aren't any disadvantages to giving Dragonfly the Amazon S3 URL directly, versus giving it only the ID, Dragonfly internally generates the URL anyway to download the file from S3. Since just having Dragonfly fetch from a URL is way simpler (you don't need to configure storage on Dragonfly), I will probably update the blog post to use that.

For filesystem storage you'd still have to use the above approach (so that Dragonfly pulls the file in locally instead of downloading it), so I'll update the blog post to show both cases.

Kind regards,

Janko

--
You received this message because you are subscribed to the Google Groups "Shrine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ruby-shrine+unsubscribe@googlegroups.com.
To post to this group, send email to ruby-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ruby-shrine/a4c1e8a6-99e7-40a9-99f8-b102ca18119b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[br...@facca.info]

Thank you for the explanation and for replying so quickly. I'll stick to providing the full URL to Dragonfly, so we can simplify things by discarding the storage configuration in Dragonfly and the dragonfly-s3_data_store plugin.

Kind Regards,

Bruno Facca

To unsubscribe from this group and stop receiving emails from it, send an email to ruby-shrine...@googlegroups.com.

[br...@facca.info]

I found that Rack::Cache does not work when using the fetch_url method instead of fetch. When using Shrine's S3 storage, the url attribute of an ImageUploader::UploadedFile object changes frequently (the X-Amz-Date and X-Amz-Signature query parameters change because they contain timestamps). Also, when requesting the same image multiple times (using fetch_url), a different ETAG is generated each time. So multiple requests to the same image never hit the cache and Shrine ends up re-processing the image at every request. I believe the same problem would happen with other HTTP reverse proxy caches (e.g., Varnish). 

After replacing the fetch_url call by Dragonfly.app.fetch([share_image.image.storage.prefix, share_image.image.id]).thumb("100x").url, Rack::Cache works as expected.

I suggest you don't mention using fetch_url with S3 storage in the documentation.

Kind Regards,

Bruno

On Monday, February 13, 2017 at 1:38:20 PM UTC-2, Janko Marohnić wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to ruby-shrine...@googlegroups.com.

[Janko Marohnić]

Oh, that makes total sense, thank you for the heads-up, and sorry for suggesting that `#fetch_url` would work equally well as  `#fetch. I agree that if `#fetch_url` won't allow reusing the cache, it shouldn't be used.

The only source of documentation is still the "Processing" blog post of the "Better File Uploads with Shrine" series, and I haven't updated that post since our last discussion, so it still correctly uses `#fetch`. I will just update it with the `prefix` addition.

Kind regards,

Janko

To unsubscribe from this group and stop receiving emails from it, send an email to ruby-shrine+unsubscribe@googlegroups.com.

To post to this group, send email to ruby-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ruby-shrine/4636860a-371b-49bc-92d0-ef8d5bcc368a%40googlegroups.com.

[Janko Marohnić]

Awesome, that's great to hear!

On Tue, Mar 21, 2017 at 11:22 PM, Bruno Facca <br...@facca.info> wrote:

We have finished implementing Shrine in our Rails application, and we're very satisfied with the results. Thank you very much for this great gem and your guidance regarding the implementation.

Kind regards,

Bruno Facca

You received this message because you are subscribed to a topic in the Google Groups "Shrine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ruby-shrine/uzRC8MzauC8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ruby-shrine+unsubscribe@googlegroups.com.

To post to this group, send email to ruby-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ruby-shrine/CAJ8a-RPzdSBog_qO6JO_1jjy5YjaLJb48SH76cGCASmEMGiOCA%40mail.gmail.com.

[Bruno Facca]

We have finished implementing Shrine in our Rails application, and we're very satisfied with the results. Thank you very much for this great gem and your guidance regarding the implementation.

Kind regards,

Bruno Facca

On Sat, Mar 18, 2017 at 2:38 AM Janko Marohnić <janko.m...@gmail.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/ruby-shrine/4636860a-371b-49bc-92d0-ef8d5bcc368a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Shrine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ruby-shrine/uzRC8MzauC8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ruby-shrine...@googlegroups.com.

To post to this group, send email to ruby-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ruby-shrine/CAJ8a-RPzdSBog_qO6JO_1jjy5YjaLJb48SH76cGCASmEMGiOCA%40mail.gmail.com.