list_display containing ForeignKey create a dead loop.

11 views
Skip to first unread message

一首诗

unread,
Aug 14, 2006, 12:03:00 PM8/14/06
to Django users
Hi all,

I met a problem.

When I added a Foreign Key field to list_display, django falls into a
dead loop and used up memories.

When I removed the field, it's all OK.

I don't think it's my code's problem because I've tested the __str__
function of both models related by the Foreign Key.

Could anybody tell me how to debug django? I used winpdb to run
command:

manage.py runserver

The server is started but the breakpoints I set at

django\contrib\admin\views.py

do not work.

Hehe, maybe I can figure out what happened and fix a bug of django.

Malcolm Tredinnick

unread,
Aug 14, 2006, 8:18:59 PM8/14/06
to django...@googlegroups.com
On Mon, 2006-08-14 at 09:03 -0700, 一首诗 wrote:
> Hi all,
>
> I met a problem.
>
> When I added a Foreign Key field to list_display, django falls into a
> dead loop and used up memories.
>
> When I removed the field, it's all OK.
>
> I don't think it's my code's problem because I've tested the __str__
> function of both models related by the Foreign Key.

If it really is in a loop, interrupting the development server with ^C
should print a traceback showing where the code is currently executing
(and what methods it called to get there). That will give you some
places to start putting print statements to see what's going on.

Alternatively, can you post a short example that illustrates the problem
so that we can have a look?

Regards,
Malcolm


一首诗

unread,
Aug 15, 2006, 11:45:35 AM8/15/06
to Django users
^C dose not help me.

I think it's because django is multithreaded.

But after I have inserted a lot of print in the codes, I found where
the program stopped.

django.db.models.QuerySet.__repr__

called

repr(self._get_data())

And it stopped there. I will trace down to see what happenned tomorrow.

Karen Tracey

unread,
Aug 15, 2006, 12:55:54 PM8/15/06
to Django users
I ran into this also in my initial run through the tutorial trying what
it suggested against my database. I ignored it at the time because I
wasn't really interested in administering things that way, and I
figured I had done something wrong. But this thread made me look at it
again. This is what I tried:

class Clues(models.Model):

ID = models.AutoField(primary_key=True)

EntryID = models.ForeignKey(Entries, db_column = 'Entry ID')

PuzzleID = models.ForeignKey(Puzzles, db_column = 'Puzzle ID',
edit_inline=models.STACKED, num_in_admin=3)

Clue = models.CharField(maxlength=150, core=True)

Num = models.IntegerField(null=True, blank=True)

Dir = models.CharField(maxlength=1)

class Meta:

ordering = ['Dir', 'Num']

db_table = verbose_name_plural = 'Clues'

def __str__(self):

return self.EntryID.Entry + ': ' + self.Clue


As soon as I put that edit_inline= parameter on the PuzzleID field any
attempt to change a Puzzle in admin would result in the server pegging
the cpu at 100% and slowly chomping up memory. (It happened to be time
to break for dinner when I first hit this so I just let it run and came
back about half an hour later to find it was at approx 1G virtual
memory size, the broswer still saying Loading....)

I went back and tried it again today (more confident that there isn't
anything essentially broken in my models) and got the same result.
Just for grins I tried commenting out the other ForeignKey field
(EntryID) and the problem went away. So at least for my example it's
got something to do with having a 2nd ForeignKey in the model. It's
not immediately obvious to this neophyte why that causes a problem but
I figured I'd post it here in case it could shed some light for others.

Cheers,
Karen

一首诗

unread,
Aug 16, 2006, 2:22:57 AM8/16/06
to Django users
Hah!

I've just found what happened!

django.db.models.fill_cache_table is a function that calls itself until
it processed all table connected by FK.

But if there is a loop in FK relations, it will never stopped!

I will see if I could make a simple patch for it.

Malcolm Tredinnick

unread,
Aug 16, 2006, 6:33:31 AM8/16/06
to django...@googlegroups.com
On Tue, 2006-08-15 at 23:22 -0700, 一首诗 wrote:
> Hah!
>
> I've just found what happened!
>
> django.db.models.fill_cache_table is a function that calls itself until
> it processed all table connected by FK.

For anybody else wondering why they can't this method, it's called
fill_table_cache() (I just spent a few minutes trying to work out why
the original poster felt so confident and I couldn't find the method).

> But if there is a loop in FK relations, it will never stopped!

Wow. Looks like you're right. Good debugging!

That's probably either going to be really easy to fix or really hard.
Hopefully you can think of something. Otherwise, please do file a ticket
about this and we'll have a look at it and fix it.

If you can't find a solution easily, don't worry too much. That's a part
of the code that I'm working on at the moment (rewriting a large chunk
of it), so I'll add your problem to my pile.

Regards,
Malcolm

Karen Tracey

unread,
Aug 16, 2006, 9:01:17 AM8/16/06
to Django users
Malcolm Tredinnick wrote:
> On Tue, 2006-08-15 at 23:22 -0700, 一首诗 wrote:
> > But if there is a loop in FK relations, it will never stopped!
>
> Wow. Looks like you're right. Good debugging!

Hmm, in the case where I hit this I don't believe there was a loop in
the FK relations. The whole model group goes like this:

There are 3 models with no ForeignKeys: Entry, Author, and Publisher
There are 2 models with 2 ForeignKeys each:
Puzzle points to Author and Publisher via ForeignKey
Clue points to a Puzzle and Entry via ForeignKey

So there is an FK chain from Clue to Puzzle that then points to Author
and Publisher, but no loop that I can see?

Just mentioning this in case it is helpful. I don't actually need this
fixed, but will try out any fix that is mentioned here. Alternatively
I can provide my models.py if that would be useful.

Cheers,
Karen

一首诗

unread,
Aug 16, 2006, 9:29:54 AM8/16/06
to Django users
Sorry for my mistake!

Because my code is on a PC without Internet Connection, I can not just
coyp and paste the function name.

I've an idea about how to elminate the loop. Add an optional para to
the function which included all processed Foreign Key RelationShip in
the form of

[ [table1Name, Col1Name, table2Name, ColName], ....]

And before any FK to be processed, it must be check that if it's
already in this para.

Perhaps it's not so efficient, but I think it will work.

I will fill a ticket as soon as possible.

Malcolm Tredinnick

unread,
Aug 16, 2006, 9:34:11 PM8/16/06
to django...@googlegroups.com
Hi Karen,

On Wed, 2006-08-16 at 06:01 -0700, Karen Tracey wrote:
[...]


> Hmm, in the case where I hit this I don't believe there was a loop in
> the FK relations. The whole model group goes like this:
>
> There are 3 models with no ForeignKeys: Entry, Author, and Publisher
> There are 2 models with 2 ForeignKeys each:
> Puzzle points to Author and Publisher via ForeignKey
> Clue points to a Puzzle and Entry via ForeignKey
>
> So there is an FK chain from Clue to Puzzle that then points to Author
> and Publisher, but no loop that I can see?
>
> Just mentioning this in case it is helpful.

We had a perfectly good hypothesis and you went and put a hole in it.
How is that helpful? :-)

Strange that after all this time two people suddenly hit infinite loops
close together and nobody's reported this coherently previously. That
code hasn't changed much recently, either. Still, good to know about.
Thanks to both of you for the information.

> I don't actually need this
> fixed, but will try out any fix that is mentioned here. Alternatively
> I can provide my models.py if that would be useful.

It would be useful to see the models if you have a case that fails
consistently. Maybe just send them to me off-list if you don't want to
post them here (or they are very long). I'll keep the information
confidential -- I just want to be able to construct a failing example so
that I can debug it.

Thanks,
Malcolm


一首诗

unread,
Aug 17, 2006, 10:45:49 AM8/17/06
to Django users
Here is my fix for fill_table_cache

=======================================================

def fill_table_cache(opts, select, tables,
where,old_prefix,cache_tables_seen,processed = {}):
"""
Helper function that recursively populates the select, tables and
where (in
place) for select_related queries.
"""
qn = backend.quote_name
if not processed.has_key(opts):
processed[opts] = []
processed_fields = processed[opts]
for f in opts.fields:
if f.rel and not f.null and f not in processed_fields:
processed_fields.append(f)
db_table = f.rel.to._meta.db_table
if db_table not in cache_tables_seen:
tables.append(qn(db_table))
else: # The table was already seen, so give it a table
alias.
new_prefix = '%s%s' % (db_table,
len(cache_tables_seen))
tables.append('%s %s' % (qn(db_table), qn(new_prefix)))
db_table = new_prefix
cache_tables_seen.append(db_table)
where.append('%s.%s = %s.%s' % \
(qn(old_prefix), qn(f.column), qn(db_table),
qn(f.rel.get_related_field().column)))
select.extend(['%s.%s' % (qn(db_table), qn(f2.column)) for
f2 in f.rel.to._meta.fields])
fill_table_cache(f.rel.to._meta, select, tables, where,
db_table, cache_tables_seen, processed)

=============================================================

And here is models for test

=============================================================

class Domain(models.Model):
name = models.CharField(unique=True, maxlength=96)
parent = models.ForeignKey('self', db_column='parent')
ip = models.IPAddressField()

def __str__(self):
cur = self
path = "/" + cur.name
while(cur != cur.parent):
cur = cur.parent
path = "/" + cur.name + path
return path

def path(self):
return self.__str__()

class Admin:
list_display = ('path',)
search_fields = ['name']
ordering = ('name',)

class Choice(models.Model):
choice = models.CharField(maxlength=200, core=True)
votes = models.IntegerField(core=True)
cat = models.ForeignKey(Domain)

def __str__(self):
return self.choice

class Admin:
list_display = ('choice','votes','cat')

================================================================

一首诗

unread,
Aug 17, 2006, 11:26:17 AM8/17/06
to Django users
Hehe, but again, there would be a infinite loop in

django\db\models\query.py in get_cached_row

That's because this record in

Here's my patch

=====================================
def get_cached_row(klass, row, index_start, processed={}):
"Helper function that recursively returns an object with cache
filled"
if not processed.has_key(klass):
processed[klass] = []
processed_fields = processed[klass]
index_end = index_start + len(klass._meta.fields)
obj = klass(*row[index_start:index_end])
for f in klass._meta.fields:


if f.rel and not f.null and f not in processed_fields:
processed_fields.append(f)

rel_obj, index_end = get_cached_row(f.rel.to, row,
index_end)
setattr(obj, f.get_cache_name(), rel_obj)
return obj, index_end

========================================

It works correctly!

Reply all
Reply to author
Forward
0 new messages