Validating and cleaning related models ...

[Bernd Wechner]

I find myself in a conundrum with related models and related formsets. I'll try and simplify a rather complex set of relations to illustrate the bare bones gist of my issue. Imagine the standard Djnago docs example:

class Musician(models.Model):
    first_name = models.CharField(max_length=50)
    last_name = models.CharField(max_length=50)
    instrument = models.CharField(max_length=100)

class Album(models.Model):
    artist = models.ForeignKey(Musician, on_delete=models.CASCADE)
    name = models.CharField(max_length=100)
    release_date = models.DateField()
    num_stars = models.IntegerField()

Now imagine I have a form in which I can create (or update, same issues arise) a Musician and their Albums. I have a generic CreateView of Musician and on it a formset of Albums. You can enter a Musician's name and data and their albums and submit and it creates the objects in the database. All this I have working fine. Can even us JS to dynamically alter the formset and allow submission of any number of albums, and updated (with an UpdateView) of such record families.

Further, note that in Musician a field is created, "album_set" that is the set of Albums associated with that Musician.

Now I can put a clean() method in Musician, and it's works really charmingly. If I raise a ValidattionError in clean() the form re-renders with the message displayed (it's available in the context item "messages"). I love this it's very slick and easy.

Now let's say I have a criterion to enforce, say "A musician must have at least one album."

I'd like to test for that and use this neat feedback mechanism to tell the user. Alas when I get to clean() in Musician, the field album_set is None as the Albums have not yet been cleaned and saved.

In my CreateView I can override is_valid() to do some checks as well, but I have the same problem, none of the objects exist yet so I can't check of integrity of the objects,.

I can override form_valid() and in that I save the  submitted Musician and Albums explicitly (this all works a dream) and after that can check for integrity (that the Musician has at least 1 album), but I can't at this point raise a ValidationError anymore to bounce back to the form with a message. Validation is done with in is_valid and in form_valid() is a little late to be doing it.

The Catch 22 I see is that to validate the relationships I need the objects to exist and to save the objects I want the relationships validated.

Ouch. A conundrum.

I can imagine saving them all in a transaction and bailing if something is awry (rolling back), all not a conceptual issue, but again in form_valid() is a tad late, and in is_valid() - which calls clean() - the objects don't exist yet.

Conclusion: I have to validate the request not the objects. But that is not DRY. The code that parses a request and turns it into objects is inside of Django. I don't want to repeat the effort. I want to hook into it at the appropriate place to:

1) Validate the relationships between submitted models (in one HTML form, multiple Django forms)
2) Do it at a time that allows easy fallback on the lovely message system.

And all I've concluded is form_valid() seems too late, is_valid() seems too early but I see nothing between them. Is there place to hook into the post processing that is most appropriate for this kind of need?

Please don't dwell on the specifics of the example, I actually have richer more complex relationships with more formsets than that all working fine I can create, update, list and work with it. All charming. I can validate intra-model fields in clean() easily and that works a dream. But I'm stuck on how to validate relationships, when and where that is best done.

Regards,

Bernd.

[Melvyn Sopacua]

So, this criterion boils down to:

* the submitted inline album form has one or more albums OR

* the musician exists and has one or more albums

 

Order of the logic may differ pending the common case, but validation will be:

• Inspect the form count of the album form set (total minus initial). This may require tweaking (added empty rows, duplicates) but you get the gist. Valid if > 0.

• Get the musician from the database, using whatever uniquely identifies a musician in the main form. Typical queryset.get() in a try block. Valid if exists and album_set is not None.

• Invalid

 

This can be done in the main view's form_valid().

 

I do not see a solid way to handle this at the model level, because you will have to save at least an album to validate the rule and both the artist and an album in the case of a new artist.

If you define a through model (in case this is a ManyToMany), you can query that model directly instead of having to go through a possibly missing attribute.

The same applies to querying the Album model directly (in case it is a reverse foreign key).

 

However this still means that it will yield no results if either the artist does not exist or no albums for the artist exist. The model has no clear path to validate the incoming data of a related model - that's what the form should do.

 

From a higher level - it is trivial to generate a list of artists without albums, so maybe implementing a procedure that generates that list for content maintainers to process is the better way to guard integrity, since that also covers the case of a DBA "deleting all albums before 1970".

 

--

Melvyn Sopacua

[Bernd Wechner]

Melvyn,

Thanks. Alas That was but one example in an apocryphal model set, and I have rather more relationship criteria to test in a more complex nest of models. I'm still drilling down into Django to see what I can uncover. Notably form.instance for the main form and all the related formsets, to see what I can divine from that. But I'm still wondering if there isn't a rather standard solution to the problem of when and where to test for relationships between models I guess, before they are saved, at the point of validation.

It's looking like maybe I have to RY a bit not DRY ;-), as in override and add to some form handling code somewhere. I'm speculating for now. It's a bit of a slog. But form instances may provide some DRY fruit, we'll see, though I have to override post() to access the form in my Generic view, I can't see it in the Model.clean() context and it's too late by the time I'm in form_valid() or form_invalid().