Validating data for use directly in a model (not via forms)

[Yunti]

I have a django project where I want to save web scraped data to the database via the django ORM- which will be used in the django app. 

The data is currently in the form of JSON -> converted to python dict via json.loads

I setup my model with the narrowest/ most constrained field type possible - (e.g. DecimalField with decimal_places=2 and max_digits=4 for prices)

I naively tried to save the data/values from the relevant keys in the JSON directly to the relevant model field, however, this raised errors due to data format.

It looks like data entered via a form is converted to the relevant python object in the form validation - e.g. a date string '24 May 2015' is converted to a datetime object and the date format validated.  

None of this appears to happen when saving directly to a model? (would be good to have my understanding here confirmed?) and so saving '24 May 2015' directly to a DateField in a model produces a format error. 

What validation (if any) does Django do when saving to a database directly via the ORM? - Does it just rely on the type checking in the database (so for sqlite this would be nothing but would for postgres)?

Thanks.

[Simon Charette]

Hi Yunti,

Did you read about model level validation? Calling model_instance.full_clean() triggers validation but it's not implicitly called when you save an instance.

For your date case you'll have to include a layer that feeds Django models with datetime.date objects from your string representation. That's what Django forms do under the hood using the DATE_INPUT_FORMAT setting.

I would suggest you define Model form for your models and use them to perform conversion and validation of your scrapped data.

Simon

[Yunti]

Thanks, yes I did look at model validation in the docs - it looked to be tied to use via forms.  I hadn't realised that it's not called automatically when an instance is created.  So if I was to validate when using get_or_create method would I then manually call full_clean() after get_or_create() ? 

I think you may be right regarding just setting up forms for the models.  I don't need a form but do need the validation.  

Thanks for your help.