How do I add intensive processes to my Rails Application?

[David McDonald]

I've started to use wicked_pdf and it works fine.  However, when I generate a PDF files, usually the CPU will get to ~80+% during the generation of the PDF by the wicked_pdf gem.  What's the proper way of handling this in my application?  I've asked around on IRC chat, stack overflow, etc, but I haven't been able to acquire a good answer.  People have told me to use Sidekiq, which I have, but this won't resolve my CPU issue unless I've put the job processes onto another machine it would seem.  Is there another way of handling this using my one server?

If it's helpful here is the stack overflow question that I posted about 12 days ago - it has perhaps a little more detail: 

https://stackoverflow.com/questions/44465332/high-cpu-usage-when-generating-pdfs-in-rails-with-wicked-pdf-gem

Please let me know if I can provide more details on any aspect of this issue. Thanks!

[Dave Aronson]

On Thu, Jun 22, 2017 at 12:08 PM, David McDonald <dave...@gmail.com> wrote:

> when I generate
> a PDF files, usually the CPU will get to ~80+% during the generation

Ouch! The big question is: are you actually doing anything else with
that server? If Rails has the machine all to itself, and you don't
have enough traffic that someone else is probably waiting for a reply,
you may be OK as-is.

Otherwise, a number of other approaches come to mind. As you
mentioned, you could put it on another machine. You could also see if
you could "nice down" that process, i.e., use the "nice" utility (or
some such programming) to set the priority of the PDF-making process
so that others can have a better share.

-Dave

--
Dave Aronson, consulting software developer of Codosaur.us,
PullRequestRoulette.com, Blog.Codosaur.us, and Dare2XL.com.

[David McDonald]

Thanks Dave, I'll look into the "Nice" utility you mentioned -- I'm currently unfamiliar with it.  Also traffic on the application isn't heavy as its only used on an internal network for roughly 60 users at the moment. However, I would prefer to find the "correct" way of doing it.  I'm thinking I may resort to using another machine, but at the moment I don't have those resources.  Rails is the only thing on the server right now... so that helps as well.  With some of the PDFs that time several minutes to run, we have seen a slow-down for other users accessing pages.  Since the PDF generation isn't urgent I don't mind having it last longer as long as it doesn't slow down the rest of the application for the other active users.  Thanks again for the "Nice" recommendation I'll be looking into it shortly.

[Hassan Schroeder]

On Thu, Jun 22, 2017 at 9:08 AM, David McDonald <dave...@gmail.com> wrote:
> I've started to use wicked_pdf and it works fine. However, when I generate
> a PDF files, usually the CPU will get to ~80+% during the generation of the
> PDF by the wicked_pdf gem.

This is exactly the kind of use case AWS Lambda was made for.

You would have to switch to a PDF generator/converter in one of
the Lambda-supported languages, unfortunately, but for a very
occasional task this would be a lot cheaper than provisioning a
separate server that would be idle most of the time.

FWIW,
--
Hassan Schroeder ------------------------ hassan.s...@gmail.com
twitter: @hassan
Consulting Availability : Silicon Valley or remote

[Colin Law]

On 22 June 2017 at 17:08, David McDonald <dave...@gmail.com> wrote:
> I've started to use wicked_pdf and it works fine. However, when I generate
> a PDF files, usually the CPU will get to ~80+% during the generation of the
> PDF by the wicked_pdf gem. What's the proper way of handling this in my
> application? I've asked around on IRC chat, stack overflow, etc, but I
> haven't been able to acquire a good answer. People have told me to use
> Sidekiq, which I have, but this won't resolve my CPU issue unless I've put
> the job processes onto another machine it would seem. Is there another way
> of handling this using my one server?

Why is the fact that PDF generation can use 80% of the processor a
problem? There will still be 20% available for Rails, or is that not
enough to give you the performance you require? Linux is pretty good
at balancing processes and threads and the processor is there to be
used after all. Provided you have shelved it off into a background
process using something like sidekiq then it may well not be an issue.
Try it and see. if it becomes an issue then you can look at more
complex solutions. My philosophy is always to start off using the
KISS principle.

Colin

[David McDonald]

Hey Colin, when I say it gets to ~80+% what I meant to get across is, for a 1 minute PDF Generation, it would start around 20% and quickly built to 80 - then 95 - and eventually go to 100% towards the end. I've noticed that many other users will have the application slow down dramatically during the generation of such a PDF.  It would cause much longer page loads and if they too were going to generate a PDF document then it would make problems worse. I have been attempting to background the process on Sidekiq - and am having another issue with it at the moment preventing me from seeing how it behaves in that procedure.  Though with it all using the same CPU I was guessing it wouldn't change much from what I'm already experiencing.

[Colin Law]

OK, understood. I think it should be possible to set the nice value
for sidekiq so that it only uses the processor when it is not doing
other things, but I don't have experience of doing that, and google
does not immediately come up with anything helpful.

Colin

> --
> You received this message because you are subscribed to the Google Groups
> "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to rubyonrails-ta...@googlegroups.com.
> To post to this group, send email to rubyonra...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/rubyonrails-talk/c3885f81-ea1a-4fc6-a396-f798a2c5862e%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.