Creating models from json url feed

[Matt Snelgar]

I am making a django app to browse available products at different markets. I am getting data for my models from a JSON url feed, 

I am not sure on the best approach to turn this data into models as the feed for a single market can contain upwards of 50k items.

Models - feel free to suggest any improvements on these

    # Details of a sellable products
    class Product(models.Model):
        item = models.IntegerField(primary_key=True, auto_created=False)
        name = models.CharField(max_length=40)

    # Market where products can be bought, some markets will cater to multiple suburbs.
    class Market(models.Model):
        id = models.IntegerField(primary_key=True, auto_created=True)
        name = models.CharField(max_length=40)

    # Nearby suburbs can share the one market
    class Suburb(models.Model):
        name = models.CharField(max_length=30)
        market = models.ForeignKey(Market, on_delete=models.SET_NULL, null=True)

    # Actual product for sale
    class ProductForSale(models.Model):
        sale = models.IntegerField(primary_key=True)
        product = models.ForeignKey(Product, on_delete=models.SET_NULL, null=True)
        cost = models.IntegerField(default=1)
        owner = models.CharField(max_length=30)
        quantity = models.IntegerField(default=1)
        market = models.ForeignKey(Market,on_delete=models.SET_NULL, null=True )
    Feed of markets products example

    "market_products": [
        {
            "id":11654,
            "item":123,
            "owner":"Bob",
            "suburb":"Springvale",
            "cost":3,
            "quantity":1,

        },
        {
            "id":11655,
            "item":123,
            "owner":"Sarah",
            "suburb":"Sunnyville",
            "cost":5,
            "quantity":2,

        },

This is how I am trying to populate my ProductForSale models - I don't know if this is the correct approach for the Product FK, or how to link the suburb to its Market FK

    markets = Market.objects.all()
    for market in markets:
        url = "jsonurl"
        response = urllib.request.urlopen(url)
        data = json.loads(response.read())
        for entry in data['market_products']:
                new_product_sale = ProductForSale.objects.create(
                    product = Product.objects.get(item=entry['item']),
                    cost = entry['cost'],
                    owner = entry['owner'],
                    quantity = entry['quantity']
                )
                new_product_sale.save()

Is there a point where I should create a separate model for each markets products, or is it fine to house them all in the one model?

[Ryan Nowakowski]

Do products from different markets have many different fields? Or are most of the fields common?

Do you need to be able to filter on products across markets? Or are most of your product queries specific to a particular market?

[Matt Snelgar]

All of the products will share the same fields.

I would filter on products, but my searches would only be restricted to one market at a time.

[Ryan Nowakowski]

I'd use a single model for all products from any market then.