S3 timeouts?
nader
I upload an image to S3, the request just hangs for 15 minutes,
locking up that Heroku dyno. Has anyone else experienced this?
Samuel Lown
Yes, in fact I recently helped out a project were this was happening
and causing lots of damage. I'd expect it to be an S3 issue as opposed
to a CarrierWave bug (but I'm out of touch with the latest S3
uploading changes!)
The solution I ended up putting in place was to use delayed_job and
upload the file in the background using manual Timeouts to restart if
uploading failed after 30s. This is recommended anyway as its not a
good idea to block dynos or mongrels while uploading stuff.
Maybe Rackspace's Cloud Files is more reliable?
Good luck!
sam
> --
> You received this message because you are subscribed to the Google Groups "carrierwave" group.
> To post to this group, send email to carri...@googlegroups.com.
> To unsubscribe from this group, send email to carrierwave...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/carrierwave?hl=en.
>
>
nader
modify the Carrierwave S3 storage class? Would you care to share that
change?
Cheers,
Nader
On Dec 2, 11:40 am, Samuel Lown <sam.l...@gmail.com> wrote:
> Hi Nader,
>
> Yes, in fact I recently helped out a project were this was happening
> and causing lots of damage. I'd expect it to be an S3 issue as opposed
> to a CarrierWave bug (but I'm out of touch with the latest S3
> uploading changes!)
>
> The solution I ended up putting in place was to use delayed_job and
> upload the file in the background using manual Timeouts to restart if
> uploading failed after 30s. This is recommended anyway as its not a
> good idea to block dynos or mongrels while uploading stuff.
>
> Maybe Rackspace's Cloud Files is more reliable?
>
> Good luck!
> sam
>
Samuel Lown
up delayed_job or something similar and using the ruby time out class
when you store the file.
I should mention, we were generating PDFs rather than receiving files
from users. Might be a bit more difficult if you're dealing with
user-uploaded content, especially given that Heroku doesn't provide
any type of reliable temporary storage...
Cheers,
sam
Kevin Ansfield / Looking Sideways
I've just started hitting this issue too. With your delayed jobs, do
you have to perform processing as well? I was concerned that the temp
file system on Heroku may not be the best place to have images waiting
around to get processed, especially if a worker gets hung up!
Cheers,
Kevin
On Dec 2, 7:40 pm, Samuel Lown <sam.l...@gmail.com> wrote:
> Hi Nader,
>
> Yes, in fact I recently helped out a project were this was happening
> and causing lots of damage. I'd expect it to be an S3 issue as opposed
> to a CarrierWave bug (but I'm out of touch with the latest S3
> uploading changes!)
>
> The solution I ended up putting in place was to use delayed_job and
> upload the file in the background using manual Timeouts to restart if
> uploading failed after 30s. This is recommended anyway as its not a
> good idea to block dynos or mongrels while uploading stuff.
>
> Maybe Rackspace's Cloud Files is more reliable?
>
> Good luck!
> sam
>
nader
where reliable to cache the files so that they're available to the
delayed job worker.
Given that constraint, it seems like the right way to handle this is
to do the upload without delayed job, but still within a Timeout. The
versions could be processed by delayed job after the initial upload to
S3, by retrieving it from S3 to process. But I'm not sure I want to
mess with Carrierwave that significantly
On Dec 2, 12:32 pm, "Kevin Ansfield / Looking Sideways"
Trevor Turk
Given that constraint, it seems like the right way to handle this is
to do the upload without delayed job, but still within a Timeout.