delayed_job--what am I missing?

[Phil Darnowsky]

Hi all,

I'm working with delayed_job for the first time, and have run up a trivial test application to play with. But the worker isn't running the jobs properly and I don't know what to make of these errors.

This is the model I'm trying:

class NumberAdder
def add_two(a, b)
result = a + b
Rails.logger.debug "#{a} + #{b} = #{result}"
end
end

And here's what I get when I try to actually run a queued job:

# This looks OK
>> NumberAdder.new.send_later :add_two, 3, 4
=> #<Delayed::Backend::ActiveRecord::Job id: 6, priority: 0, attempts: 0, handler: "--- !ruby/struct:Delayed::PerformableMethod \nobject...", last_error: nil, run_at: "2010-04-02 22:01:46", locked_at: nil, failed_at: nil, locked_by: nil, created_at: "2010-04-02 22:01:46", updated_at: "2010-04-02 22:01:46">

# But then...

phil@beast:~/src/test/delayed_job_test$ rake jobs:work
(in /home/phil/src/test/delayed_job_test)
*** Starting job worker host:beast pid:11824
* [Worker(host:beast pid:11824)] acquired lock on NumberAdder#add_two
* [JOB] host:beast pid:11824 failed with NoMethodError: undefined method `add_two' for #<YAML::Object:0xb76593b8 @ivars={}, @class="NumberAdder"> - 0 failed attempts
* [Worker(host:beast pid:11824)] acquired lock on NumberAdder#add_two
* [JOB] host:beast pid:11824 failed with NoMethodError: undefined method `add_two' for #<YAML::Object:0xb7631bc4 @ivars={}, @class="NumberAdder"> - 1 failed attempts
2 jobs processed at 17.3046 j/s, 2 failed ...

Any advice would be most appreciated.

--Phil

[Dan Croak]

DJ serializes your object as YAML. I think it's related to that. Check
out last_error docs. I tend to store ids instead of AR objects. Also
had issues with string for error field, usually make it a text field.
Otherwise big backtraces crash DJ.

dan croak
thoughtbot

> --
> You received this message because you are subscribed to the Boston
> Ruby Group mailing list
> To post to this group, send email to boston-r...@googlegroups.com
> To unsubscribe from this group, send email to boston-rubygro...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/boston-rubygroup
>
> To unsubscribe, reply using "remove me" as the subject.

[Blake Carlson]

Sometimes there are issues when the worker tries to unmarshal the
object - it doesn't know how to autoload classes that aren't
ActiveRecord. Try just adding "NumberAdder" somewhere in the
initializers or environment.rb to trigger class loading. Or, in
console do ..

>> NumberAdder
>> Delayed::Job.work_off

That'll get you on the right path.

- Blake

[Andrew Kuklewicz]

There are known problems with AR record marshaling, and some known fixes.
(I've run into this a few times with ActiveMessaging, and we have discussed it on our dev list several times over the years.)

Yaml should reconstitute your object by type, not as a '#<YAML::Object...>', it only does that if it doesn't find/recognize the actual class.

Here are a few references on this yaml AR problem, and solution.

http://itsignals.cascadia.com.au/?p=10
http://dev.rubyonrails.org/ticket/7537
http://yaml4r.sourceforge.net/doc/page/type_families.htm

To cut to the chase: put the following in an initializer, and you will be all set:

# fix for yaml serialization
YAML.add_domain_type("ActiveRecord,2008", "") do |type, val|
  klass = type.split(':').last.constantize
  YAML.object_maker(klass, val)
end

class ActiveRecord::Base
  def to_yaml_type
    "!ActiveRecord,2008/#{self.class}"
  end
end

class ActiveRecord::Base
  def to_yaml_properties
    ['@attributes']
  end
end


Now, I have to say I usually try to avoid dealing with this - it works but is generally only necessary if the processor of the message/job does not have access to the database to look up the AR object.

If it does have DB access, I do something like the following to serialize just the type and id:

class Array
  def to_message
    self.collect{|obj|
      if obj.respond_to?(:to_message_hash)
        obj.to_message_hash
      else
        obj
      end
    }.to_yaml
  end
end

class ActiveRecord::Base
  def to_message(include_attributes=[])
    msg = self.to_message_hash
    include_attributes.each{ |attribute| msg[attribute.to_sym] = self.send(attribute.to_sym) }
    msg.to_yaml
  end
 
  def to_message_hash
    {:object_id=>self.id, :class_name=>self.class.name}
  end
end


Then use this code to get the object back:

def from_message(message, options={})
  result = YAML::load(message)
  if result.is_a?(Array)
    result.collect do |obj|
      obj.is_a?(Hash) ? hash_to_obj(obj, options) : obj
    end
  elsif result.is_a?(Hash)
    hash_to_obj(result, options)
  else
    result
  end
end

def hash_to_obj(hsh, options={})
  if (hsh.has_key?(:class_name) && hsh.has_key?(:object_id))
    clazz = hsh[:class_name].constantize
    id = hsh[:object_id].to_i
    if (clazz.paranoid? == true) && options[:with_deleted]
      clazz.find_with_deleted(id)
    else
      clazz.find(id)
    end
  else
    hsh
  end
rescue Object=>err
  logger.error "ApplicationProcessor::hash_to_obj: #{err.class.name} rescued:\n#{err.message}"
  return nil
end

Hope that helps -

Andrew Kuklewicz

[Ryan Angilly]

Or, like Blake said, just drop a reference to the class in an initializer.  This will get Rails' missing constant loader into play and find the class for you.  We ran into this problem at Punchbowl.  Dropping the class name into an initializer is hacky, but short & sweet.

-Ryan

[Keenan Brock]

For us, relying more on the parsing of the yml and less on the yaml .class seemed to fix our problem.

FWIW/

--Keenan

note: ParseObjectFromYaml=/\!ruby\/\w+\:([^\s]+)/ 

job.rb line 191 (YMMV)

      unless handler.respond_to?(:perform)

        #yaml does not namespace the handler.class - so bypass for now

        #used to use handler.class if ! handler.nil?

        if source =~ ParseObjectFromYaml

          handler_class = $1

        end

        attempt_to_load(handler_class || handler.class.to_s)

        handler = YAML.load(source)

      end

[Phil Darnowsky]

Thanks all! Putting a "require 'number_adder'" in environment.rb did the trick.  Now to go try this in some non-toy code... --Phil --- On Fri, 4/2/10, Ryan Angilly <ry...@angilly.com> wrote: