Why does Rails throw an autoloading constant exception when I use threads?

[Andy Rose]

Hey guys, I posted this on Stackoverflow but it's not getting much love.

Note in the question that I've found a solution to my problem. However I'm curious as to what is going on here. Why does Rails encounter a race condition when auto-loading an ActiveRecord model association...?

http://stackoverflow.com/questions/28267513/why-does-rails-throw-an-autoloading-constant-exception-when-i-use-threads

Feel free to answer on SO if you have an account, but here's cool too.

Cheers!

[Xavier Noria]

Hey, I am not on SO.

Constant autoloading is not thread-safe (it can't be). That's one of the reasons by default Rails eager loads the application code in production mode.

There are a few things you can do. The easiest one is to enable eager loading by setting config.eager_load = true (in modern versions of Rails). Another possibility is to load the files with the involved classes using require_dependency in your script before spawning threads.

[Andy Rose]

Hey Xavier, thanks so much! Really appreciate your answer!

Could you maybe describe to me (or point me in the right direction) to how content autoloading happens?

You know what it? I've kinda solved the issue by just referencing the constant explicitly. But I'm really curious as to how Rails ends up throwing that circular dependency exception.

Like... I get what happens when you have a non-threadsafe variable... because actually setting a variable requires multiple steps in the MRI and so while one thread is reading from a variable before setting it, another thread has already set it to something.

But what's going on here.... like... one thread tries to autoload a constant... and then before it's done another thread does the same and then... why does that result in a circular reference exception? Any ideas?

[Xavier Noria]

On Wed, Feb 4, 2015 at 12:07 PM, Andy Rose <thatan...@gmail.com> wrote:

Hey Xavier, thanks so much! Really appreciate your answer!

Could you maybe describe to me (or point me in the right direction) to how content autoloading happens?

Love that you ask, because a new guide on the topic is precisely out :)

    http://edgeguides.rubyonrails.org/autoloading_and_reloading_constants.html

(Pointing to the edge guide because there are some minor enhancements that are going to be out with 4.2.1 but didn't make it for 4.2.0.)

You know what it? I've kinda solved the issue by just referencing the constant explicitly. But I'm really curious as to how Rails ends up throwing that circular dependency exception.

Yes, that's another one. It's analogous to issuing require_dependency calls, except that the file gets loaded via autoloading instead of explicitly.

Like... I get what happens when you have a non-threadsafe variable... because actually setting a variable requires multiple steps in the MRI and so while one thread is reading from a variable before setting it, another thread has already set it to something.

But what's going on here.... like... one thread tries to autoload a constant... and then before it's done another thread does the same and then... why does that result in a circular reference exception? Any ideas?

The two key ideas are:

1) Constant autoloading goes through a regular method call, the method is const_missing (you'll see that in the guide).

2) As far a Ruby is concerned, a class definition is code, there's nothing atomic. So, context switching may happen in the middle of the interpretation of the class definition. Albeit unlikely, it could be the case that context-switching happens even before the constant has been defined, and then another const_missing for the same constant could be triggered by a different thread for example resulting in the same file being loaded twice (which would not be circular indeed in that case, but the error message does not distinguish):

    $ cat app/models/foo.rb 
    sleep 1
    Foo = 1
    $ bin/rails runner '[Thread.new { Foo }, Thread.new { Foo }].each(&:join)'

    ... Circular dependency detected while autoloading constant Foo (RuntimeError)

 See?

[Andy Rose]

Oh wow, thanks, super interesting.

So the issue is that the same file gets autoloaded twice... I see. Thanks so much man. Yeah will check out that article, really appreciate the answer.

So as you said this *shouldn't* be an issue in production since as you said rails will eager load everything. Cool, great to know!