In admin, unable to save many-to-many relation objects with intermediate table

[ristretto.rb]

Hello, I'm stuck.  Any help will be much appreciated.

I followed http://www.djangoproject.com/documentation/models/m2m_intermediary/ to make a
Many-to-many relationship via an intermediary table.

class LeftSide(models.Model):
:

class RightSide(models.Model):
:

class JoinTable(models.Model):
    leftSide = models.ForeignKey(LeftSide,
                edit_inline=models.TABULAR,
                num_in_admin=3, core=True)
    rightSide = models.ForeignKey(RightSide,
                edit_inline=models.TABULAR,
                num_in_admin=3,core=True)
    reference_no = models.CharField(max_length=10, null=True, blank=True)

I can load the LeftSide in the admin, and choose to Change a record.  I get 3 RightSide menu groups at the bottom.  I can choose a RightSide to set as a join.  When I choose Save (any of the 3 save variations on the page) nothing is saved.

There are two cases. 

1)  If the are already records in the join table for that LeftSide, then I get an error when I save.  {'jointable.0.id': [u'Join Table record with this ID already exists.']}  I didn't actually make any changes to that association, so I don't know why it would complain.

2)  If there are no join table records associated with the LeftSide, then there is no error, but nothing is saved.

Am I doing something totally wrong here?  Or is this a bug in the Admin system?

[ristretto.rb]

OK, I took core=True off, and added it to the reference_no field. The
problem seems to go away for the case when no there are no
pre-existing joins. Clearly, I don't understand what core is for.
I'll read the docs again, and see if it makes sense.

However, it still errors with

{'jointable.0.id': [u'Join Table record with this ID already exists.']}

when I tried to save a LeftSide records with existing associations.

thanks for any and all help.

--
Picante Solutions Limited
e: ge...@picante.co.nz
w: 06 757 9488

[ristretto.rb]

More details. I'm using MySQL at the moment (but with a plan to move
to Postgresql.) The association table has a number of rows loaded
through these models, but not through my Django app directly. I used
the Model classes in a script to pre load a bunch of data. Could this
be the problem?

On Jul 9, 1:28 pm, ristretto.rb <ristretto...@gmail.com> wrote:
> OK, I took core=True off, and added it to the reference_no field.  The
> problem seems to go away for the case when no there are no
> pre-existing joins.   Clearly, I don't understand what core is for.
> I'll read the docs again, and see if it makes sense.
>
> However, it still errors with
>
> {'jointable.0.id': [u'Join Table record with this ID already exists.']}
>
> when I tried to save a LeftSide records with existing associations.
>
> thanks for any and all help.
>
>
>
> On Wed, Jul 9, 2008 at 1:09 PM, ristretto. rb <ristretto...@gmail.com> wrote:
> > Hello, I'm stuck.  Any help will be much appreciated.
>
> > I followed

> >http://www.djangoproject.com/documentation/models/m2m_intermediary/to make

[Karen Tracey]

On Tue, Jul 8, 2008 at 9:57 PM, ristretto.rb <ristre...@gmail.com> wrote:

More details.  I'm using MySQL at the moment (but with a plan to move
to Postgresql.)  The association table has a number of rows loaded
through these models, but not through my Django app directly.  I used
the Model classes in a script to pre load a bunch of data.  Could this
be the problem?

On Jul 9, 1:28 pm, ristretto.rb <ristretto...@gmail.com> wrote:
> OK, I took core=True off, and added it to the reference_no field.  The
> problem seems to go away for the case when no there are no
> pre-existing joins.   Clearly, I don't understand what core is for.
> I'll read the docs again, and see if it makes sense.
>
> However, it still errors with
>
> {'jointable.0.id': [u'Join Table record with this ID already exists.']}
>
> when I tried to save a LeftSide records with existing associations.
>
> thanks for any and all help.
>

I have no idea what is going on, but a ticket was recently (4 hours ago) opened reporting the same error message for inline-edited objects:

http://code.djangoproject.com/ticket/7682

It seems like some code is thinking records are supposed to be new and checking for primary key uniqueness when in fact it is existing records that are being updated.  This might be a recently introduced bug, it's a little curious to have two people reporting the same (not common) error message suddenly so close together.

What version are you running?  

Karen

[ristretto.rb]

I'm using the svn version. I updated this morning. I can't remember
if I was getting the error before I updated, or not.

I have spent the day tracking this down (it's taken me all day because
I don't know Django or Python very well, and I haven't been able to
find a comprehensive IDE that is worth much more then just an editor.
Argh.)

In any case, here's what I'm up to

In django-trunk/django/oldforms/__init__.py line around 68, I have put
some logging output. field.get_validation_errors(new_data) is coming
back as a oldforms.HiddenField when it generated the following error.

2008-07-09 04:03:08,227 DEBUG <class 'django.oldforms.HiddenField'>
2008-07-09 04:03:08,227 DEBUG {'jointable.0.id': [u'Join table with
this ID already exists.']}

I don't know why the admin system needs to put this following hidden
in the html

<input type="hidden" id="id_jointable.0.id"
name="jointtable.0.id" value="38" />

Now, I've traced this down to line 474, roughly of django-trunk/django/
oldforms/__init__.py where a the HiddenField class is defined. A
validator is passed in called "_curried". I have no idea what that
is. Going to stop here.

thanks!



On Jul 9, 2:47 pm, "Karen Tracey" <kmtra...@gmail.com> wrote:

> > > >http://www.djangoproject.com/documentation/models/m2m_intermediary/to...

[Karen Tracey]

On Wed, Jul 9, 2008 at 12:57 AM, ristretto.rb <ristre...@gmail.com> wrote:

I'm using the svn version.  I updated this morning.  I can't remember
if I was getting the error before I updated, or not.

That ticket I pointed to identifies r7710 as a revision where this problem did not exist.  With your models (much simpler than the ones in that ticket) I was able to recreate the problem on r7871. Binary search shows that the problem was introduced in r7790:
 
http://code.djangoproject.com/changeset/7790

The change message says it was to "Make sure we only create the minimum number of table indexes for MySQL" but I don't believe this problem has anything to do with creating table indexes.  (I did not recreate the tables when recreating the problem, just attempted to "save and continue editing" a problematic entry.)

I have spent the day tracking this down (it's taken me all day because
I don't know Django or Python very well, and I haven't been able to
find a comprehensive IDE that is worth much more then just an editor.
Argh.)

Well you picked some hairy code to start with.  FWIW I use Eclipse with PyDev which is good enough for stepping through code, setting breakpoints, and examining variables, though it can run into problems with side-effects of the variable display.  For getting some clue about what code path is running it's usually OK enough.

In any case, here's what I'm up to

In django-trunk/django/oldforms/__init__.py line around 68, I have put
some logging output.  field.get_validation_errors(new_data) is coming
back as a oldforms.HiddenField when it generated the following error.

2008-07-09 04:03:08,227 DEBUG <class 'django.oldforms.HiddenField'>
2008-07-09 04:03:08,227 DEBUG {'jointable.0.id': [u'Join table with
this ID already exists.']}

I don't know why the admin system needs to put this following hidden
in the html

          <input type="hidden" id="id_jointable.0.id"
name="jointtable.0.id" value="38" />

That's the primary key value of the inline-edited object.  When the admin code needs to update any of the visible fields, it needs to pull the primary key value out of the hidden field in order to know the correct record to update.
 

Now, I've traced this down to line 474, roughly of django-trunk/django/
oldforms/__init__.py where a the HiddenField class is defined.  A
validator is passed in called "_curried".  I have no idea what that
is.  Going to stop here.

Yeah, the problem is that validator.  Prior to r7790, these hidden fields did not have any validators attached to them.  Now they have the manipulator_validator_unique validator that is run.  Problem is this validator doesn't seem to be appropriate for this case.  Tracing through it it is looking up the inline edited object's primary key value in the parent object's table and if it exists comparing it to the parent object's original primary key value, so you get an error if the primary key value for the inline edited object exists in the parent object's table and differs from the parent primary key value.  I don't believe that validator is supposed to be associated with the hidden input field for the primary key of an inline-edited object.

It's being added here:

(http://code.djangoproject.com/browser/django/trunk/django/db/models/fields/__init__.py#L329)

        if self.unique or (self.primary_key and not rel):
            params['validator_list'].append(curry(manipulator_validator_unique, self, opts, manipulator))

which points to the changes associated with setting unique in r7790 as what has introduced the problem.  I don't understand exactly what that change was supposed to be doing, so I'm not sure what the right fix is.  I'll update #7682 with what I've figured out and hopefully someone who knows the code can fix it.  In the meantime if you drop back to r7789 I think you will find the problem goes away.

Karen
 

[ristretto.rb]

Wow, Karen, you're amazing. Thanks for that. I can say I learned a
lot about Django today with your help.

thanks!!

On Jul 9, 6:38 pm, "Karen Tracey" <kmtra...@gmail.com> wrote:
> On Wed, Jul 9, 2008 at 12:57 AM, ristretto.rb <ristretto...@gmail.com>

> (http://code.djangoproject.com/browser/django/trunk/django/db/models/f...
> )