[Django] #35362: ArrayField with nested CharField with a max_length silently truncates overlong values

[Django]

#35362: ArrayField with nested CharField with a max_length silently truncates
overlong values
-------------------------------------+-------------------------------------
Reporter: Craig de | Owner: nobody
Stigter |
Type: Bug | Status: new
Component: Database | Version: 5.0
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
With the following model

{{{
#!python
class TestModel(models.Model):
arrayfield = ArrayField(
models.CharField(max_length=63, blank=False), blank=False,
null=False
)

}}}

Saving an over-long string value into the arrayfield silently truncates
the value:

{{{
#!python
>>> TestModel.objects.create(arrayfield=['a'*100])
<TestModel: TestModel object (1)>

>>> len(TestModel.objects.get().arrayfield[0])
63
}}}

This is probably because the `ArrayField` is casting its values (to
`varchar(63)[]`) during queries for some reason. The
[https://www.postgresql.org/docs/current/datatype-character.html#DATATYPE-
CHARACTER postgres docs] suggest that casting to `varchar(X)` will
truncate strings to X length.

This seems related to #33647 although it's not quite the same situation
since this isn't `bulk_update`
--
Ticket URL: <https://code.djangoproject.com/ticket/35362>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35362: ArrayField with nested CharField with a max_length silently truncates
overlong values
-------------------------------------+-------------------------------------

Reporter: Craig de Stigter | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: postgres | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* keywords: => postgres
* stage: Unreviewed => Accepted
* version: 5.0 => dev

Comment:

Thank you for the report!
Confirmed behaviour on main, 5.0 and 4.2. Example failing test:
{{{
diff --git a/tests/postgres_tests/test_array.py
b/tests/postgres_tests/test_array.py
index 386a0afa3a..fc932ab19a 100644
--- a/tests/postgres_tests/test_array.py
+++ b/tests/postgres_tests/test_array.py
@@ -148,6 +148,11 @@ class TestSaveLoad(PostgreSQLTestCase):
with self.assertRaises(IntegrityError):
instance.save()

+ def test_char_max_length_exceeded(self):
+ instance = CharArrayModel(field=["a" * 11])
+ with self.assertRaises(IntegrityError):
+ instance.save()
+
def test_nested(self):
instance = NestedIntegerArrayModel(field=[[1, 2], [3, 4]])
instance.save()
}}}

This generates the SQL which has the silent truncation

{{{
INSERT INTO "postgres_tests_chararraymodel" ("field") VALUES
(\'{aaaaaaaaaaa}\'::varchar(10)[]) RETURNING
"postgres_tests_chararraymodel"."id"
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35362#comment:1>

[Django]

#35362: ArrayField with nested CharField with a max_length silently truncates
overlong values
-------------------------------------+-------------------------------------
Reporter: Craig de Stigter | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: postgres | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak):

As for me this should be closed as a duplicate.
--
Ticket URL: <https://code.djangoproject.com/ticket/35362#comment:2>

[Django]

#35362: ArrayField with nested CharField with a max_length silently truncates
overlong values
-------------------------------------+-------------------------------------
Reporter: Craig de Stigter | Owner: nobody

Type: Bug | Status: closed

Component: Database layer | Version: dev
(models, ORM) |

Severity: Normal | Resolution: duplicate

Keywords: postgres | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* resolution: => duplicate
* status: new => closed

Comment:

Duplicate of #33647.

Can see that `ArrayField` is also part of the ticket discussion.
--
Ticket URL: <https://code.djangoproject.com/ticket/35362#comment:3>