IntegrityError when POSTing to ListCreateAPIView with custom Serializer and custom fields

[LaundroMat]

Hi -

I must be getting tired, because I can't for the life of me weed out this problem that's been bugging me all night... Thanks in advance to all of you looking into this.

The code:

class TemplateHours(models.Model):

    employee = models.ForeignKey(Employee)

    weekday = models.SmallIntegerField()

    start = models.TimeField()

    end = models.TimeField()

class TemplateHoursSerializer(serializers.Serializer):

    weekday = serializers.IntegerField()

    start = HourField()

    end = HourField()

    employee = EmployeeSerializer()

    def restore_object(self, attrs, instance=None):

        if instance:

            for attr, value in attrs.items():

                setattr(instance, attr, value)

            return instance

        return TemplateHours(**attrs)

class TemplateHoursRootView(generics.ListCreateAPIView):

    """

        Endpoint for block of hours in a template (TemplateHours model).

    """

    model = TemplateHours

    serializer_class = TemplateHoursSerializer

POSTing 

{

"employee": 

    {

        "id":1,

        "user": {"username":"mathieu", "id":1}

    },

    "weekday":"0",

    "start":"09:00:00",

    "end":"16:0:00"

} to the url results in 

IntegrityError at /api/template-block/

planning_templatehours.employee_id may not be NULL

Environment:


Request Method: POST
Request URL: http://127.0.0.1:8000/api/template-block/

Django Version: 1.4.1
Python Version: 2.7.2
Installed Applications:
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.admin',
 'rest_framework',
 'hours',
 'planning',
 'debug_toolbar',
 'django_extensions',
 'django_coverage')
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'debug_toolbar.middleware.DebugToolbarMiddleware')


Traceback:
File "C:\Users\Mathieu\Development\django_projects\hedron\Lib\site-packages\django\core\handlers\base.py" in get_response
  111.                         response = callback(request, *callback_args, **callback_kwargs)
File "C:\Users\Mathieu\Development\django_projects\hedron\Lib\site-packages\django\views\generic\base.py" in view
  48.             return self.dispatch(request, *args, **kwargs)
File "C:\Users\Mathieu\Development\django_projects\hedron\Lib\site-packages\django\views\decorators\csrf.py" in wrapped_view
  77.         return view_func(*args, **kwargs)
File "C:\Users\Mathieu\Development\django_projects\hedron\Lib\site-packages\rest_framework\views.py" in dispatch
  340.             response = self.handle_exception(exc)
File "C:\Users\Mathieu\Development\django_projects\hedron\Lib\site-packages\rest_framework\views.py" in dispatch
  337.             response = handler(request, *args, **kwargs)
File "C:\Users\Mathieu\Development\django_projects\hedron\Lib\site-packages\rest_framework\generics.py" in post
  117.         return self.create(request, *args, **kwargs)
File "C:\Users\Mathieu\Development\django_projects\hedron\Lib\site-packages\rest_framework\mixins.py" in create
  23.             self.object = serializer.save()
File "C:\Users\Mathieu\Development\django_projects\hedron\Lib\site-packages\rest_framework\serializers.py" in save
  275.         self.object.save()
File "C:\Users\Mathieu\Development\django_projects\hedron\Lib\site-packages\django\db\models\base.py" in save
  463.         self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "C:\Users\Mathieu\Development\django_projects\hedron\Lib\site-packages\django\db\models\base.py" in save_base
  551.                 result = manager._insert([self], fields=fields, return_id=update_pk, using=using, raw=raw)
File "C:\Users\Mathieu\Development\django_projects\hedron\Lib\site-packages\django\db\models\manager.py" in _insert
  203.         return insert_query(self.model, objs, fields, **kwargs)
File "C:\Users\Mathieu\Development\django_projects\hedron\Lib\site-packages\django\db\models\query.py" in insert_query
  1576.     return query.get_compiler(using=using).execute_sql(return_id)
File "C:\Users\Mathieu\Development\django_projects\hedron\Lib\site-packages\django\db\models\sql\compiler.py" in execute_sql
  910.             cursor.execute(sql, params)
File "C:\Users\Mathieu\Development\django_projects\hedron\Lib\site-packages\django\db\backends\util.py" in execute
  40.             return self.cursor.execute(sql, params)
File "C:\Users\Mathieu\Development\django_projects\hedron\Lib\site-packages\django\db\backends\sqlite3\base.py" in execute
  337.             return Database.Cursor.execute(self, query, params)

Exception Type: IntegrityError at /api/template-block/
Exception Value: planning_templatehours.employee_id may not be NULL

[Tom Christie]

Right, there's an awful lot I could say about this, but I'll try to keep it limited for the moment.

First up, the question at hand...

You're restoring and then saving a `TemplateHours` model instance, with an FK to `Employee`.
Because you're using an `EmployeeSerializer` field to create a nested representation, rather than say, a PrimaryKeyRelationField or HyperlinkedRelationField,

the restore_object method cannot set the fk for the target employee.

Throw some debugging into that `restore_object` method and take a look at the attrs it's getting at that point and it should make a bit more sense.

The simplest, and probably most sensible answer would be:

* Don't use nested relationships for writable representations.  (Use pks or hyperlinks instead.)

The documentation around related fields is something I still need to work on before 2.0 goes live.

For me, the open question is what is the best way to approach both the documentation and the behavior around nested relationships.

The course of least resistance would probably be to forcibly ensure that they are non-writable, and throw an obvious exception if you try to deserialize incoming data using nested relationships.

There *are* sensible cases where you might want a nested representation that's write-able, and I can see how REST framework *might* handle those,

but it's a difficult area.

Eg.

* What should the behavior be when a nested representation contains a non-existent pk, do we create a new object or fail?

* How do we handle nested reverse-relationships.

* How do we handle nested to-many relationships that contain a list of instances.  Are they updatable?  If an object is removed from the relationship is it also deleted or not?

* How do we handle HTML forms that map to nested relationships.

There are possible answers to all these questions but you can see it wouldn't be trivial to do well.

  - Tom

[LaundroMat]

Hi Tom - 

Many thanks for your in-depth answer; I wish everyone would be as forward-thinking as you are. I'm quite sure I've now avoided a lot of future problems by going for flat representations, thanks to your answer.

Much appreciated,

Mathieu

[Gabriel Landes]

Howdy, i'm new to this group but am having what seems like a similar issue.

We have an AngularJS app that we are building to use a DRF API backend for all data. Our "Create Account" page needs to create an ACCOUNT object and a USER object at the same time. We are sending this:

1. {"name":"Foobar, Inc.","is_superaccount":true,"is_trial":true,"user":{"fullname":"John Doe","email":"j...@test.com"}}

Note "user" is included. Before I go into greater detail (will probably post as a new topic if needed) my question initially is this: Is this possible/adviseable in interacting with a REST API?