Update _id field before .save() for ForeignKey

[Maxim Bulatov‏]

Hello,

I have lot of lists like follow:
(Name1): (V1, V2, V3, V4, ..)

And I use many threads to create many db objects:
company = Company(Name)
for x in vector:
    v = CompanyValue(company=company, value=x)
    values.append(v)

class CompanyValue(models.Model):
    company = models.ForeignKey('Company', on_delete=models.CASCADE)
    value = FloatField

This pool of threads return all lists, what I combine, filter and want put to database in one bulk_create call.
Suddenly, I found, company.id is not ready to be linked in CompanyValue, follow code does not works:
Company.objects.bulk_create(companies)
CompanyValue.objects.bulk_create(values)      # value.company_id is null here

No way to save objects in my threads, because I need to filter objects and can do it only when all of them are collected. Also, it has performance issues 100k+ of requests is slowly enough. I have two ways to resolve:
1. I can make intermediate class or tuple and create CompanyValue objects only when companies is saved to db. Hard to support, hard to filter, not so clean.
2. I can use dirty hack to update _id field:
    for v in values:
        v.company = v.company

Can you advice more ways for me? It seems, I don't know something from django features to make it elegant.
I see 11 years old issue here (https://code.djangoproject.com/ticket/9553) and it marked as intended behaviour.

[Integr@te System‏]

Hi guy,

I see this doc fix for u, with builtin pk in models django.

And you can create index in text/char field to optimize performance.

https://docs.djangoproject.com/en/dev/ref/databases/#manually-specified-autoincrement-pk

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/3d62beaf-7fbf-492f-b998-1539f45cd7a0%40googlegroups.com.

[Dvenum‏]

Integr@te System, thanks. If I set pk (id) manually, it works, but hard to specify this id value. Maybe for one thread this way is good.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAP5HUWrC747ZHLWPCf7UK%2BZvZRDAcxDMxk7ewE4d4UhZAHpB7g%40mail.gmail.com.

--

dvenum

dven...@gmail.com

[Charles Lee‏]

I have an idea.

1. `bulk_create` all companies.
2. `Filter`(SELECT) all `Company` and make it dictionary for retrieving company fast.
3. 1. company_dict = {
        ’Name’: ‘pk’,
        ...
      }
4. Create a list of `CompanyValue` with company name and company_dict.
5. `bulk_create` the list.

I'm not sure if it will work because I haven't implemented it, but I hope it helps. 

2019년 11월 29일 금요일 오후 9시 42분 0초 UTC+9, Maxim Bulatov 님의 말:

[Dvenum‏]

Interest idea and it works, but not necessary:
    company = Company()
    #company.id is None

    companies.append(company)
    Company.objects.bulk_create(companies)
    #company.id is 1

Instead to ask pk from database, it can be used from company_value.company.id. Django save pk value to internal meta (models/base.py) and does not update it. This works too:
value.company = value.company     # pk updated

I want to ask, each child object should be created only when parent is saved to db? Maybe django has something to cover this case?

--

You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/af2a0f27-7234-4210-8627-1ad8982da805%40googlegroups.com.

--

dvenum

dven...@gmail.com

[Integr@te System‏]

Hi Dvenun,

Bc of  Django implement accord to python, of cause child inherit from parent existence.

https://docs.python.org/3.7/library/functions.html#id

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAHBRYSd1MC5mL3oWTfzqc%2BXXcWxVnDqLpJ5HsEGk2GHFd917eQ%40mail.gmail.com.

[Dvenum‏]

Integr@te System,

This is True: id(company) == id(company_value.company)
So, we have exact the same object company, .id is ready on .save() call, but internal meta dictionary still has None, because it was none when company_value created.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAP5HUWqW7gZVR1u6-%2BAkMhB5dZeRHiXzLrSQvmiL7O3diT9OfA%40mail.gmail.com.

--

dvenum

dven...@gmail.com

[Integr@te System‏]

Hi ,

Please lookandfeel on 

Moving an object from one database to another¶

https://docs.djangoproject.com/en/2.0/topics/db/multi-db/#moving-an-object-from-one-database-to-another

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAHBRYSf9iMaMmT1osqvP2uiu%2BbyN%2BKDkMOqZXdYcfo8Pt%2B1_Sw%40mail.gmail.com.

[Dvenum‏]

Thanks, its interest to know more about pk management, but this is another case. Now I update pk for child objects and this is fastest way, what I found. If django will reread .pk before save, it will the same.

Want to know, maybe my logic can be altered from the start. How peoples create batch of objects with relations to insert it in one request? I think, this is popular task.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAP5HUWq01D-mbPRA-mV_vfn-a7jKj52n-ayk_mH7_SmWn9GD2A%40mail.gmail.com.

--

dvenum

dven...@gmail.com