Using sidekiq batching

[James Trowbridge]

Hello, I've been trying to implement batching, (i am a pro customer), however I keep getting an uninitialized constant error around Sidekiq::Batch.  I imagine there is some file I need to require that I haven't.  In short, which file do I need to require to get batching to work?

[Altonymous]

What require statements are you using now?

Did you change to require 'sidekiq-pro' instead of require 'sidekiq'  ?

[Mike Perham]

If you just have:

gem 'sidekiq-pro'

in your Gemfile, everything should work as normal.  Otherwise, you might need to require 'sidekiq-pro'.

On Fri, Aug 8, 2014 at 11:32 AM, James Trowbridge <trowbrid...@gmail.com> wrote:

Hello, I've been trying to implement batching, (i am a pro customer), however I keep getting an uninitialized constant error around Sidekiq::Batch.  I imagine there is some file I need to require that I haven't.  In short, which file do I need to require to get batching to work?

--
You received this message because you are subscribed to the Google Groups "Sidekiq" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sidekiq+u...@googlegroups.com.
To post to this group, send email to sid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sidekiq/46166e95-215b-430f-91df-96c02a3bce5c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Mike Perham – Contributed Systems

Smart, effective open source infrastructure for your apps.

[James Trowbridge]

okay i'll try that, in the short term i manually required it with require 'sidekiq/batch'

[James Trowbridge]

So now i'm having a problem where the batch gets created correctly, but when the job finishes the batch still thinks its pending.  I've enqueued 200 jobs in a batch, then they all run correctly, but the batch still says all 200 are pending.  Anyone run into this issue?

[Chris Altman]

Can you give a snippet of your code in a gist?

—
Chris Altman
Twitter: @Altonymous
GitHub: http://www.github.com/altonymous
LinkedIn: http://www.linkedin.com/in/altonymous/

To view this discussion on the web visit https://groups.google.com/d/msgid/sidekiq/b217a392-0af0-44d2-beda-7e9057bc2b12%40googlegroups.com.

[James Trowbridge]

sure thing, here is the file that enqueues the batch, and the class that performs the job.  What this code does is take a pdf then it splits it into each page then creates a sidekiq batch for converting every page in that document into images.  The image converting part is working fine, it's just they never get marked as completed when they finish, they are forever stuck in the pending state.

https://gist.github.com/anonymous/f0a395537487fffbfc4f

Here's a screenshot of the browser showing the jobs as stuck and also the log showing them as processed.  If you need more of either i can create it.

http://imgur.com/o2vfg6f

[Mike Perham]

Have you ever had a batch process successfully?  That's the behavior you'd see if the server-side batch middleware wasn't active.  What's your sidekiq.rb initializer look like?

To view this discussion on the web visit https://groups.google.com/d/msgid/sidekiq/f8004715-ab13-4f3d-a765-99ece38c0eea%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[James Trowbridge]

FWIW the method that enqueus the batch is a class method on the same class that performs the batch, 'ImageConversion' in that case.  I'll have to try it differently and see if that's causing the problem.

[James Trowbridge]

No mike I haven't had a batch that has progressing past the 'pending' state.  Here is my initializer

redis_config = YAML.load_file(File.expand_path("../../redis.yml", __FILE__))

redis_config = redis_config[ENV.fetch("RACK_ENV", 'staging')] || {}

redis_url = ENV.fetch("REDIS_URL", "redis://#{redis_config[:host]}:#{redis_config[:port]}/#{redis_config[:database]}")

client_opts = {:url => redis_url, :size => 1}

if redis_config[:namespace]

  client_opts[:namespace] = redis_config[:namespace]

end

Sidekiq.configure_client do |config|

  config.redis = client_opts

end

server_opts = client_opts.delete(:size) && client_opts

Sidekiq.configure_server do |config|

  config.redis = server_opts

end

[James Trowbridge]

Problem solved.  For future reference the problem was I was using sidekiq via Sinatra and not rails and I was missing a 

Bundler.require(:default)

at the top of my main sinatra file.