Migrating ManyToManyField to bigint?

[Jeremy Lainé]

Hi!

I'm currently looking at how to migrate all my models from AutoField to BigAutoField. For all the explicitly defined models the process seems pretty straightforward:

• change DEFAULT_AUTO_FIELD to BigAutoField
• generate migrations
• apply migrations

However, when I inspect the database schema, I see that the "intermediate" tables for many-to-many relations still have a primary key of type "integer" (on postgresql). This means I'm no closer to avoiding 32bit primary key exhaustion!

Does anyone know how I can address this?

Many thanks in advance,

Jeremy

[Mike Dewhirst]

Not really.

In my case I specify models for all my m2m intermediate tables as a matter of course. It is how I think. In my schemas, most such tables typically carry extra data about the relationships they define - therefore I want models. The bigint migration obviously worked for me.

Maybe you can use manage.py to write out your m2m models so you can adjust then migrate them.

settings.DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField" should work for new intermediate tables.

Another option is to manually adjust all your m2m PostgreSQL id fields to bigint.

It is a lossless change and will never need to be reversed so it might be easier to bite that bullet and move on.

YMMV

Many thanks in advance,

Jeremy

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/de4fefa5-6dcf-4419-91f8-a6451cac781en%40googlegroups.com.

-- 
Signed email is an absolute defence against phishing. This email has
been signed with my private key. If you import my public key you can
automatically decrypt my signature and be sure it came from me. Just
ask and I'll send it to you. Your email software can handle signing.

[Jeremy Lainé]

Hi Mike,

Thanks for the reply. For future reference, it was pointed out to me that I missed both an explicit note in the documentation [1] and an issue tracking a potential change in the future [2].

Do you have any suggestions as to the actual SQL migration which needs to be applied for postgresql on each m2m table? Specifically, is it purely a matter of altering the "id" column type, or does something need to be done to the index?

Cheers,

Jeremy

[1] https://docs.djangoproject.com/en/4.0/ref/settings/#default-auto-field

[2] https://code.djangoproject.com/ticket/32674

[Jeremy Lainé]

Answering myself, AFAICT I need to create migrations which look like the following for every m2m intermediate table. The following assumes a "Book" model which has a ManyToMany relationship to a "Tag" model:

from django.db import migrations


class Migration(migrations.Migration):
    dependencies = [
        ("myapp", "0001_initial"),
    ]

    operations = [
        migrations.RunSQL(
            [
                'ALTER TABLE "myapp_book_tags" ALTER COLUMN "id" TYPE bigint USING "id"::bigint',
                'ALTER SEQUENCE "myapp_book_tags_id_seq" AS bigint',
            ],
            elidable=True,
        ),
    ]

Kind regards,

Jeremy

[Mike Dewhirst]

Jeremy

If it was my project I would explicitly model the through tables.

The other options are covered in the references you gave - which were news to me!

Cheers

Mike

--

(Unsigned mail from my phone)

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/e915618e-274c-4e27-a1d4-8cbc72bcc4a8n%40googlegroups.com.

[Makrand Zare]

First you need to create table structure in the PostgreSQL, then after import the data into the respective tables.

Makrand Zare

+919930102832

makza...@gmail.com 

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/5dd13af9-54e3-e6a0-d8b7-9f736e532d46%40dewhirst.com.au.

[Jeremy Lainé]

I really needed an in-place migration as reloading tens of millions of entries was not practical.

I ended up using the migrations I mentioned with one migration per M2M intermediate table to avoid locking multiple tables at once.

Cheers,

Jeremy