field = self.model._meta.get_field(self.field_name) returns self?

[Brandon]

I have had the error described here (https://code.djangoproject.com/ticket/27365) show-up randomly in mulitple independent environments.

The gross things to notice are:

1. It always occurs in exactly the same way.

2. It is always fixed by an apache restart.

3. It is suppressed by reducing the number of WSGI threads and max_requests.

4. An environment may run for months without it occurring, but once it does occur, that apache child is affected until it dies.

5. The error occurs both inside my custom application and inside third party applications/libraries.

6. It seems to only occur on the reverse of an FK.

7. This *never* happened with 1.7.10 and only began happening with an upgrade to 1.9.9

My current theory is that some part of django/apps/registry.py is not thread-safe. Here is why...

The error always occurs at line #244 of django/db/models/fields/reverse_related.py with field being populated incorrectly. The  field variable is populated via self.model._meta.get_field(self.field_name) which *eventually* uses structures from apps including a .get_models(include_auto_created=True).

From that... two things to note. A traceback of a properly functioning call has the self and field variables in the django/db/models/fields/reverse_related.py:get_related_field( ) function (line 237) as:

repr(field) == <django.db.models.fields.AutoField: id>

repr(self) == <OneToOneRel: subclasscase.subclasscase>

A borked call to *exactly* the same place has those same fields as:

repr(field) == <OneToOneRel: subclasscase.subclasscase>

repr(self) == <OneToOneRel: subclasscase.subclasscase>

but once again... if that apache child cycles out... the calls go back to being correct. But if this error occurs in always occurs in exactly the same place with exactly the same call path once django/db/models/query.py:_filter_or_exclude( ) function (line 808) is reached.

I'm looking for any advice on how to dig deeper into this issue or where to start investigating.

One other thing I'll note is that my application does __import__ statements on django.db.models.Model subclasses during routine operations. In fact subclasscase is a subclass of a django.db.models.Model subclass. Given this comment from django/apps/registry.py

# Mapping of app labels => model names => model classes. Every time a

# model is imported, ModelBase.__new__ calls apps.register_model which

# creates an entry in all_models. All imported models are registered,

# regardless of whether they're defined in an installed application

# and whether the registry has been populated. Since it isn't possible

# to reimport a module safely (it could reexecute initialization code)

# all_models is never overridden or reset.

self.all_models = defaultdict(OrderedDict)

and this post (https://mail.python.org/pipermail/python-list/2010-January/565497.html)

On Jan 25, 11:26 am, Raymond Hettinger <pyt... at rcn.com> wrote:

> On Jan 25, 12:59 am, "Frank Millman" <fr... at chagford.com> wrote:

> 

> > Hi all

> 

> > Is defaultdict thread safe?

> 

> Sometimes.  It depends on whether an operation has callbacks to pure

> Python.

> 

> > Assume I have -

> 

> >     from collections import defaultdict

> >     my_dict = defaultdict(list)

> 

> > If two threads call "my_dict['abc'].append(...)" simultaneously, is it

> > guaranteed that my_dict['abc'] will end up containing two elements?

> 

> Yes.

> 

> But, if the constructor is a user defined class, the pure python code

> runs for the instantiation and all bets are off.

> 

>    class A:

>       def __init__(self):

>           . . .

>    my_dict = defaultdict(A)   # not thread-safe.

> 

> Raymond

Many thanks

Frank

Again makes me think that something in django/apps/registry.py is not thread-safe or perhaps... not sub-interpreter safe? I don't know.

Any help is much appreciated.

Cheers!

--Brandon

[Brandon]

This bug is *still* occurring... And seems to do so in waves. But after fighting it again and again, something struck me.

The bug always occurs here: django/db/models/fields/reverse_related.py in get_related_field at line 244

So always in this function: 

    def get_related_field(self):
        """
        Return the Field in the 'to' object to which this relationship is tied.
        """
        field = self.model._meta.get_field(self.field_name)
        if not field.concrete:

            raise exceptions.FieldDoesNotExist("No related field named '%s'" %
                    self.field_name)
        return field

But specifically is it occurring *because*  self.model._meta.get_field(self.field_name) is not returning a field (e.g. django.db.models.fields.*). Somehow self.model._meta.get_field(self.field_name) is returning a OneToOneRel object.

Should self.model._meta.get_field(self.field_name) ever return *anything* other than some django.db.models.fields.* object?

Cheers!

--Brandon