Flattening model relationships (in APIs)

[Ankush Thakur]

I'm using DRF for an API. My problem is that I defined my postal address like this (breaking it up into City, State, Country):

class Country(models.Model):    

    class Meta:

        db_table = 'countries'

    name = models.TextField()

    code = models.TextField(unique=True)

class State(models.Model):    

    class Meta:

        db_table = 'states'

    name = models.TextField()

    code = models.TextField(unique=True)

    country = models.ForeignKey('Country', on_delete=models.CASCADE)

class City(models.Model):    

    class Meta:

        db_table = 'cities'

    name = models.TextField()

    code = models.TextField(unique=True)

    state = models.ForeignKey('State', on_delete=models.CASCADE)

class Address(models.Model):

    class Meta:

        db_table = 'addresses'

    

    building_no = models.TextField()

    street = models.TextField(null=True)

    locality = models.TextField(null=True)

    landmark = models.TextField(null=True)

    pincode = models.TextField(null=True)

    latitude = models.DecimalField(max_digits=9, decimal_places=6, null=True)

    longitude = models.DecimalField(max_digits=9, decimal_places=6, null=True)

    city = models.ForeignKey('City', on_delete=models.CASCADE)

Now, in my system, a venue has an address, and so the serializes are defined like this:

class VenueSerializer(serializers.ModelSerializer):

    address = AddressSerializer()

    offerings = OfferingSerializer(many=True)

    class Meta:

        model = Venue

        fields = ['id', 'name', 'price_per_day', 'status', 'offerings', 'address', 'photos']

which leads us to:

class AddressSerializer(serializers.ModelSerializer):

    city = CitySerializer()

    class Meta:

        model = Address

        fields = ['id', 'building_no', 'street', 'locality', 'landmark', 'pincode', 'latitude', 'longitude', 'city']

which leads us to:

class CitySerializer(serializers.ModelSerializer):

    state = StateSerializer()

    class Meta:

        model = City

        fields = ['id', 'name', 'code', 'state']

which leads us to:

class StateSerializer(serializers.ModelSerializer):

    country = CountrySerializer()

    class Meta:

        model = State

        fields = ['id', 'name', 'code', 'country']

which finally leads to:

class CountrySerializer(serializers.ModelSerializer):

    class Meta:

        model = Country

        fields = ['id', 'name', 'code']

and when this gets serialized, the address is given in the following format:

 "address": {

      "id": 2,

      "building_no": "11",

      "street": "Another Street",

      "locality": "",

      "landmark": "Fortis Hospital",

      "pincode": "201003",

      "latitude": "28.632778",

      "longitude": "77.219722",

      "city": {

        "id": 1,

        "name": "Delhi",

        "code": "DEL",

        "state": {

          "id": 1,

          "name": "Delhi",

          "code": "DEL",

          "country": {

            "id": 1,

            "name": "India",

            "code": "IN"

          }

        }

      }

    }

So there's a hell lot of nesting from city to state to country. If the relationship chain was even deeper, there would be even more nesting, which I feel isn't great for API consumers. What is the best practice here to put state and country at the same level as the city? I think this will also complicate the logic while POSTing data, so I'm interested in knowing about that as well.

I'm sorry if there is too much code, but I couldn't think of a better way to convey the situation than actually post everything.

Regards,

Ankush Thakur

[Mike Dewhirst]

Ankush

I think you might have to provide more than one method for retrieving an
address so applications can request particular subsets. Perhaps a
local-postal address, international-postal address, geo-location,
what-three-words address and of course the full bucket as required.

I think it is always best to normalize as much as possible in the
beginning and de-normalize only if performance becomes an issue.

Also ...

https://hackernoon.com/10-things-i-learned-making-the-fastest-site-in-the-world-18a0e1cdf4a7#.3l8n34dvk

Mike

> --
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users...@googlegroups.com
> <mailto:django-users...@googlegroups.com>.
> To post to this group, send email to django...@googlegroups.com
> <mailto:django...@googlegroups.com>.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CALX%3DrK%2Bgbbro%2BkFYsq%3DFDCSu2kw6KRmsfKRamKLA%2Bisrgj%3DboQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/django-users/CALX%3DrK%2Bgbbro%2BkFYsq%3DFDCSu2kw6KRmsfKRamKLA%2Bisrgj%3DboQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

[Ankush Thakur]

Hey Mike,

Thanks for your thoughts. Perhaps I presented my requirement incorrectly. I don't at all want to denormalize the database. My point was that because of the way models are related, I'm ending up with city containing state containing country. This, I feel, would be a chore for the front-end guys. So my question is, how can I rearrange these fields to give a flat JSON output rather than have objects within objects. I hope it makes more sense now.

~~ Ankush

> <mailto:django-users+unsub...@googlegroups.com>.

[Ankush Thakur]

Also, I don't have any need of providing several types of addresses as of now. For now I'm just sticking with outputting everything when the address is requested.

~~ Ankush

[Melvyn Sopacua]

On Wednesday 22 February 2017 00:39:42 Ankush Thakur wrote:

> I'm using DRF for an API. My problem is that I defined my postal

> address like this (breaking it up into City, State, Country):

 

...

 

> So there's a hell lot of nesting from city to state to country. If the

> relationship chain was even deeper, there would be even more nesting,

> which I feel isn't great for API consumers.

 

class AddressSerializer(serializers.ModelSerializer):

depth = 3

class Meta:

model = Address

fields = ['id', 'building_no', 'street', 'locality',

'landmark', 'pincode', 'latitude', 'longitude', 'city']

 

class DisplayAddressSerializer(AddressSerializer):

depth = 1

 

Now you can add display_address to any serializer using an address and API consumers can use it to render easily. You can then use AddressSerializer to provide the full structured address, so it can be used for writes.

You may need to customize how it flattens the models, but that's what DRF's documentation is for.

 

--

Melvyn Sopacua

[marcin....@gmail.com]

On Tuesday, February 21, 2017 at 8:13:25 PM UTC+1, Ankush Thakur wrote:

If the relationship chain was even deeper, there would be even more nesting, which I feel isn't great for API consumers. What is the best practice here to put state and country at the same level as the city? 

Just follow REST design. 

Forget django models, think about encapsulation.

Think about representation of a resource, not about serializing a model(s).

Make semantic representations, as good as possible.

You are not forced to do any nested nasty things. This has nothing to do with REST api.

You may feel that DRF is limiting you. You'll be on a good path, if so. :)

Good luck!

Marcin

[Ankush Thakur]

Marcin, that's exactly where I'm stuck! I know endpoints should never be 1:1 serialization of models, but I just don't know how to do that. I mean, I've been able to create endpoints like "/customers/1/payments/" where I use model relationships to generate JSON structures where Customer contains a Payments array field. My Address endpoint seems to be an oddity, as API consumers don't expect the city to contain state and the state to contain country as a JSON structure. How can I add these to the top-level Address entity directly while serialization? That's where I have no answers. Would it be possible for you to point me towards some article that does that? Thanks in advance!

Regards,

Ankush

[marcin....@gmail.com]

I'm not sure you want to read my answer, really...  I've finally stopped using DRF and wrote own library which is focused on resources, linking and representations.

I think you can do some customization in DRF, probably you ca declare custom serializer class, but this is a hard way, IMO.

I like simple things, so with my lib I can just do something like that:

payments = api.resource('/payments/')

customer = api.resource('/customers/:pk')

@payments.representation

def payment_as_json(payment, ctx):

    return {

        'customer_address': payment.customer.address,

        'customer_url': ctx.link_to(customer, pk=payment.customer_id), # link resources

        'value': payment.value,  # write here

        'and_so_on': True,  # what do you want

    }

Yes, it does not validate automatically and this example is not restful, but you may follow semantic structures like jsonld without any limitation, and apply validation in a controller:

@payments.post(accept='application/json')

def add_payment(ctx):

    form = PaymentForm(data=ctx.body)

    form.is_valid() and form.save(0

    [...]

Where PaymentForm may be a typical Django Form or ModelForm, or even a Colander Schema. 

There is more code to write, but implementation is more explitic.

PEP20: Explicit is better than implicit. Simple is better than complex. Flat is better than nested. Readability counts.  And so on...

BR,

Marcin

[Ankush Thakur]

Hmmm. That's not an answer I wanted to hear, really, but I like it. I'm myself finding DRF too restrictive once you are past the effort-saving magic. Thank you. I might give it up as it's still early days in the project.

Regards,

Ankush Thakur

--
You received this message because you are subscribed to a topic in the Google Groups "Django users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-users/ttoJbZJOBnU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.

Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/a0b23023-8a5c-4257-ad6d-d72639f85925%40googlegroups.com.

[marcin....@gmail.com]

I was limited by DRF long days ago and I realized that it is not following REST architecutre. It does good job in automation in exposing models over http, but everything else is way complicated.

But there were no problem mix both tools in one project. DRF was for "80%" of work and my lib for the rest. Downsides? Two libs used to get the job done.

Maybe try using DRF's @api_view decorator for function views, and return a pure and manually "flattened" dict?

M.

Regards,

Ankush Thakur

To unsubscribe from this group and all its topics, send an email to django-users...@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

[Ankush Thakur]

I guess this is the library in question: https://github.com/marcinn/restosaur (took some effort to find it!). Thanks, if I decide to stick with the API-first approach, I'll use it. Either way, I've bookmarked it for future use. :-)

Regards,

Ankush Thakur

To unsubscribe from this group and all its topics, send an email to django-users+unsubscribe@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/8dbe0176-7fcd-4736-a0ba-78aee3408ac1%40googlegroups.com.

[Xavier Ordoquy]

Quick question. How will you handle the fields updates ?

Like, what if you get an city_name="Bombay" and city_code="DEL" ?

If you can set them as read-only, then using the source argument is the way to go.

Alternatively, you can define a non model serializer and flatten the data before hand.

You can also decide to drop the serializers and pass a Python structure directly though you would have some work for the validation.

The more strict RESTfull way would be to use the several entry points and use hyperlinks on them (though it would add some extra requests).

You have a couple of options opened to you.

Remember that Django REST framework is no silver bullet.

You should think about how you want to represent your resources first and then write the associated code.

Also remember that easy isn't the same as simple. The framework targets the latter.

Regards,

Xavier.

[marcin....@gmail.com]

On Monday, February 27, 2017 at 3:30:11 PM UTC+1, Ankush Thakur wrote:

I guess this is the library in question: https://github.com/marcinn/restosaur (took some effort to find it!). Thanks, if I decide to stick with the API-first approach, I'll use it. Either way, I've bookmarked it for future use. :-)

Yes. I'm not annoucing it, because of entering early beta stage. But on the other side I've decided to use it in production starting from early alphas, and it just works.

This is a thin glue between web framework and your REST(-ful, -ish, plain http, what you want) services, so there is no big issues at all.

Last commits gave me possibility to run api top on a Flask framework, so it is fun. 

Still alpha-beta, so I can't give you any guarantee of satisfaction ;)

DRF has mature community, support, etc. Use it if it will work for you. Otherwise you're welcome to Restosaur community. 

BR,

Marcin

[marcin....@gmail.com]

On Monday, February 27, 2017 at 3:38:34 PM UTC+1, Xavier Ordoquy wrote:

The more strict RESTfull way would be to use the several entry points and use hyperlinks on them (though it would add some extra requests).

This is just about representation. Nobody forces you to do such things. 

You may include more or less data, require linking or not - it does not matter. Client should just *know* what to do - read *something* or *follow link* to read *something*. 

Everything is about semantic, not about a structure.

The most important is hyperlinking, hypermedia and the state transfer. 

Marcin

[marcin....@gmail.com]

Wanna read something interesting? Look at http://t-code.pl/blog/2016/02/rest-misconceptions-0/

You will realize how many "antipatterns" are propagated in turorials on our World Wide Web, and where they're already implemented.