The cotent types framework unreasonably limits model name length.

[Richard Campen]

Hello

I have hit what I feel is an arbitrary limit on the length of a django model class name. I am using the PostgreSQL backend, which smartly truncates table names above a certain size (normally 63 characters) which means in theory a table name can be of indeterminate length, as PostgreSQL will truncate it appropriately. However, if the model class name is greater than 100 characters than an error is thrown when saving the model name to the `django_content_type` model as the `ContentType.model` field uses a CharField with a limit of 100. This arbitrarily restricts the size of the model name when the db backend can handle it fine.

I tried to go back in time to figure out if there was any context in setting the `ContentType.model` field max_length at 100 chars, but it was made before the Django project was migrated to git.

I feel it would be best to switch this field to a TextField, as even 255 characters seems an unreasonable limit to impose when the db backend can support longer names, though perhaps having a smaller (but configurable) max_length on the TextField would still be desirable.

What are peoples feelings on this?

Cheers,

Richard

[אורי‎]

How can a class name be more than 100 characters? Can you give an example?

A limit of 100 characters seems reasonable to me, maybe even 60 would be enough.

אורי

u...@speedy.net

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/26478a0a-5809-449f-b17d-d7223e2cfb3do%40googlegroups.com.

[Kye Russell]

I’ve never needed a Django model with a name that long, but I don’t think it’s the framework’s place to make those sorts of assumptions if it doesn’t need to, especially for what I see as little to no gain.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CABD5YeFdA95UhJex%2BERf1pM-groRiUjO5tyLv300wSRjcWkGSQ%40mail.gmail.com.

[Richard Campen]

I guess I have two answers to that:

1) Specifically, in this instance I am scripting the creation of a whole bunch of Django models as there are too many to do manually. A couple out of the hundred or so models have names that are just over 100 characters (102, 103, etc) which is where I found this limit.

2) More generally, why should Django impose any kind of limit at all on the model length? If python doesn't impose a limit on class names, and the db backend doesn't limit it, (and even the Django ORM doesn't limit it) why should a contrib package like content types limit it? Also to preempt the question, yes I need content types as it is required by other things running in the application.

Side note: If Django truly wishes to limit the Model Name length, doing so directly would be more appropriate in the model metaclass, rather than as a side effect of a content types.

Cheers

R

On Tuesday, 11 August 2020 15:19:00 UTC+12, Uri wrote:

How can a class name be more than 100 characters? Can you give an example?

A limit of 100 characters seems reasonable to me, maybe even 60 would be enough.

אורי

u...@speedy.net

On Tue, Aug 11, 2020 at 6:06 AM Richard Campen <richar...@gmail.com> wrote:

Hello

I have hit what I feel is an arbitrary limit on the length of a django model class name. I am using the PostgreSQL backend, which smartly truncates table names above a certain size (normally 63 characters) which means in theory a table name can be of indeterminate length, as PostgreSQL will truncate it appropriately. However, if the model class name is greater than 100 characters than an error is thrown when saving the model name to the `django_content_type` model as the `ContentType.model` field uses a CharField with a limit of 100. This arbitrarily restricts the size of the model name when the db backend can handle it fine.

I tried to go back in time to figure out if there was any context in setting the `ContentType.model` field max_length at 100 chars, but it was made before the Django project was migrated to git.

I feel it would be best to switch this field to a TextField, as even 255 characters seems an unreasonable limit to impose when the db backend can support longer names, though perhaps having a smaller (but configurable) max_length on the TextField would still be desirable.

What are peoples feelings on this?

Cheers,

Richard

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-d...@googlegroups.com.

[Adam Johnson]

It does seem unreasonable. I think a migration to TextField would be warranted.

You should be able to work around this for now with a migration in your own app that uses RunSQL to alter the column for ContentType.model.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/396d07c8-060d-4d35-98eb-e5bbea502a24o%40googlegroups.com.

--

Adam

[Mariusz Felisiak]

contenttypes should support all backends,  You need to remember about DB-restrictions, e.g. we cannot change `ContentType.model` to a `TextField` because TEXT columns cannot be used in unique constraints on MySQL. See similar discussion about `User.last_name`.

Best,

Mariusz

[charettes]

I agree this limitation of `contrib.contenttypes` is a bit unexpected. Not saying we should lift it but it does seem arbitrary even if having such large table names could be considered bad practice.

One approach we could take that doesn't involve any migration is to fallback to hashing of the overflowing characters.

That's the approach taken by the ORM when generating identifiers (e.g. index names) that might overflow the identifier limit of a backend[0]

Simon

[0] https://github.com/django/django/blob/60626162f76f26d32a38d18151700cb041201fb3/django/db/backends/utils.py#L196-L221

[Shai Berger]

AFAIK Postgres, in these cases, simply truncates the name. This means:

1) Generating models with names longer than 63 characters on postgres
is fragile. You may find yourself with more than one model trying to
use the same table name.

2) Suffix-hashing long names like Simon suggests may not be
backwards-compatible.

My 2 cents,
Shai.

[charettes]

> Suffix-hashing long names like Simon suggests may not be backwards-compatible.

Could you elaborate on that?

Assuming model names > 100 characters never worked wouldn't only suffix-hashing model names > 100 characters be backward compatible?

Simon

[Richard Campen]

> You should be able to work around this for now with a migration in your own app that uses RunSQL to alter the column for ContentType.model.

I tried this approach however the ORM level field validation still kicks as it still expects VARCHAR 100 even if the db column is now a different type.

On Tuesday, 11 August 2020 20:52:17 UTC+12, Adam Johnson wrote:

It does seem unreasonable. I think a migration to TextField would be warranted.

You should be able to work around this for now with a migration in your own app that uses RunSQL to alter the column for ContentType.model.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/396d07c8-060d-4d35-98eb-e5bbea502a24o%40googlegroups.com.

--

Adam

[Richard Campen]

I think a problem with hashing names here is that it would break the whole `apps.get_model(<app_label>, <model_name>)` functionality that content types uses, as the model name stored in the db is not the actual name stored in the apps registry (and modifying the registry functionality feels like the wrong approach here).

> contenttypes should support all backends,  You need to remember about DB-restrictions, e.g. we cannot change `ContentType.model` to a `TextField` because TEXT columns cannot be used in unique constraints on MySQL. See similar discussion about `User.last_name`.

I think a key difference with the `User.last_name` example is that Django provides a way to avoid this all together by setting a custom User model. The ContentType model is much more tightly coupled, so unless we want to implement a similar way to override the ContentType model, it feels like we aught to be more accommodating with the max_length here.

With this in mind (and my awareness that me hitting this issue was an edge case already) perhaps the most straight forward solution would be to leave the `model` field as CharField but just set its max_length to 255. This would be consistent with the approach taken with `Permission.name`. Sort of feels like kicking the bucket down the road, but perhaps it is far enough down said road to no longer be an issue.

[Shai Berger]

On Tue, 11 Aug 2020 10:51:58 -0700 (PDT)
charettes <chare...@gmail.com> wrote:

> > Suffix-hashing long names like Simon suggests may not be
> backwards-compatible.
>
> Could you elaborate on that?
>
> Assuming model names > 100 characters never worked wouldn't only
> suffix-hashing model names > 100 characters be backward compatible?
>

You're right, I was thinking about suffix-hashing table names longer
than 63 on Postgres, the way it has always been done on Oracle.