[Django] #33240: get_image_dimensions: ValueError: buffer is not large enough

Django

unread,

Oct 29, 2021, 9:41:38 AM10/29/21

to django-...@googlegroups.com

#33240: get_image_dimensions: ValueError: buffer is not large enough
------------------------------------------------+------------------------
Reporter: NKSM | Owner: nobody
Type: Bug | Status: new
Component: File uploads/storage | Version: 2.2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
------------------------------------------------+------------------------
When I try to get the dimensions of the image like below:
{{{
def clean_image(self):
image = self.cleaned_data.get("image")
if not image:
raise forms.ValidationError("No image!")
else:
w, h = get_image_dimensions(image)
if w != 100:
raise forms.ValidationError("The image is %i pixel wide. It's
supposed to be 100px" % w)
if h != 200:
raise forms.ValidationError("The image is %i pixel high. It's
supposed to be 200px" % h)
return image
}}}

I get : `ValueError: buffer is not large enough`

**Error at line**:
https://github.com/django/django/blob/073b7b5915fdfb89a144e81173176ee13ff92a25/django/core/files/images.py#L62

{{{
# Most of the time Pillow only needs a small chunk to parse the image
# and get the dimensions, but with some TIFF files Pillow needs to
# parse the whole file.
chunk_size = 1024
while 1:
data = file.read(chunk_size)
if not data:
break
try:
p.feed(data) ⬅️ **HERE**
except zlib.error as e:
}}}

**Exception Location**: `/PIL/Image.py in frombuffer, line 2605`

**This works well**:

`width, height = PillowImage.open(image).size`

--
Ticket URL: <https://code.djangoproject.com/ticket/33240>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,

Oct 29, 2021, 9:41:55 AM10/29/21

to django-...@googlegroups.com

#33240: get_image_dimensions: ValueError: buffer is not large enough

-------------------------------------+-------------------------------------

Reporter: NKSM | Owner: nobody
Type: Bug | Status: new

Component: File | Version: 2.2
uploads/storage |
Severity: Normal | Resolution:

-------------------------------------+-------------------------------------
Changes (by NKSM):

* Attachment "favicon.ico" added.

File to uplad

Django

unread,

Oct 29, 2021, 10:02:52 AM10/29/21

to django-...@googlegroups.com

#33240: get_image_dimensions: ValueError: buffer is not large enough

-------------------------------------+-------------------------------------

Reporter: NKSM | Owner: nobody
Type: Bug | Status: new

Component: File | Version: 2.2
uploads/storage |
Severity: Normal | Resolution:

-------------------------------------+-------------------------------------
Description changed by NKSM:

Old description:

New description:

**This works well**:

{{{
from PIL import Image as PillowImage
width, height = PillowImage.open(image).size
}}}

--

--
Ticket URL: <https://code.djangoproject.com/ticket/33240#comment:1>

Django

unread,

Oct 30, 2021, 7:22:46 AM10/30/21

to django-...@googlegroups.com

#33240: get_image_dimensions: ValueError: buffer is not large enough

-------------------------------------+-------------------------------------
Reporter: NKSM | Owner:
| ath34-tech
Type: Bug | Status: assigned

Component: File | Version: 2.2
uploads/storage |
Severity: Normal | Resolution:

-------------------------------------+-------------------------------------
Changes (by ath34-tech):

* owner: nobody => ath34-tech
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/33240#comment:2>

Django

unread,

Oct 30, 2021, 7:24:47 AM10/30/21

to django-...@googlegroups.com

#33240: get_image_dimensions: ValueError: buffer is not large enough

-------------------------------------+-------------------------------------
Reporter: NKSM | Owner:
| ath34-tech

Type: Bug | Status: closed

Component: File | Version: 2.2
uploads/storage |

Severity: Normal | Resolution: invalid

-------------------------------------+-------------------------------------
Changes (by ath34-tech):

* status: assigned => closed
* resolution: => invalid

--
Ticket URL: <https://code.djangoproject.com/ticket/33240#comment:3>

Django

unread,

Oct 30, 2021, 7:29:34 AM10/30/21

to django-...@googlegroups.com

#33240: get_image_dimensions: ValueError: buffer is not large enough

-------------------------------------+-------------------------------------
Reporter: NKSM | Owner:
| ath34-tech

Type: Bug | Status: new

Component: File | Version: 2.2
uploads/storage |
Severity: Normal | Resolution:

-------------------------------------+-------------------------------------
Changes (by ath34-tech):

* status: closed => new
* resolution: invalid =>

--
Ticket URL: <https://code.djangoproject.com/ticket/33240#comment:4>

Django

unread,

Nov 1, 2021, 8:03:38 AM11/1/21

to django-...@googlegroups.com

#33240: get_image_dimensions: ValueError: buffer is not large enough

-------------------------------------+-------------------------------------
Reporter: NKSM | Owner: Ath
| Tripathi
Type: Bug | Status: closed

Component: File | Version: 2.2
uploads/storage |

Severity: Normal | Resolution: needsinfo
Keywords: pillow | Triage Stage:

| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):

* keywords: => pillow

* status: assigned => closed

* resolution: => needsinfo

Comment:

Thanks for this report, however it looks like an issue in `Pillow` not in
Django itself. We use `ImageFile.Parser()` as
[https://pillow.readthedocs.io/en/stable/reference/ImageFile.html#example-
parse-an-image documented].

First, I would try to report it in the [https://github.com/python-
pillow/Pillow/issues Pillow issue tracker] . Please reopen the ticket if
you can debug your issue and provide details about why and where Django is
at fault.

--
Ticket URL: <https://code.djangoproject.com/ticket/33240#comment:3>

Django

unread,

Nov 1, 2021, 8:04:58 AM11/1/21

to django-...@googlegroups.com

#33240: get_image_dimensions() raises ValueError on some .ico files.

Keywords: pillow | Triage Stage:

| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------

--
Ticket URL: <https://code.djangoproject.com/ticket/33240#comment:4>

Django

unread,

Oct 31, 2024, 9:26:43 AM10/31/24

to django-...@googlegroups.com

#33240: get_image_dimensions() raises ValueError on some .ico files.
-------------------------------------+-------------------------------------
Reporter: NKSM | Owner: Ath
| Tripathi

Type: Bug | Status: new

Component: File | Version: 2.2
uploads/storage |
Severity: Normal | Resolution:

Keywords: pillow | Triage Stage:

| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by Matthias Kestenholz):

* resolution: needsinfo =>

* status: closed => new

Comment:

I have the same issue with Pillow 11.0.

The trouble is that Pillow's TIFF parser now raises a `ValueError` when
attempting to determine the dimensions of this image here:
https://github.com/matthiask/django-
imagefield/blob/main/tests/testapp/media/python-logo.tiff

This produces the following exception:

{{{
======================================================================
ERROR: test_websafe_versatileimageproxy
(testapp.test_imagefield.Test.test_websafe_versatileimageproxy)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-
packages/django/test/utils.py", line 446, in inner
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/django-imagefield/django-
imagefield/tests/testapp/test_imagefield.py", line 401, in
test_websafe_versatileimageproxy
m = WebsafeImage.objects.create(image="python-logo.tiff")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-
packages/django/db/models/manager.py", line 87, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-
packages/django/db/models/query.py", line 658, in create
obj = self.model(**kwargs)
^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-
packages/django/db/models/base.py", line 572, in __init__
post_init.send(sender=cls, instance=self)
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-
packages/django/dispatch/dispatcher.py", line 189, in send
response = receiver(signal=self, sender=sender, **named)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-
packages/django/db/models/fields/files.py", line 519, in
update_dimension_fields
width = file.width
^^^^^^^^^^
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-
packages/django/core/files/images.py", line 21, in width
return self._get_image_dimensions()[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-
packages/django/core/files/images.py", line 31, in _get_image_dimensions
self._dimensions_cache = get_image_dimensions(self, close=close)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-
packages/django/core/files/images.py", line 64, in get_image_dimensions
p.feed(data)
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-
packages/PIL/ImageFile.py", line 471, in feed
im = Image.open(fp)
^^^^^^^^^^^^^^
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-packages/PIL/Image.py",
line 3515, in open
im = _open_core(fp, filename, prefix, formats)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-packages/PIL/Image.py",
line 3503, in _open_core
im = factory(fp, filename)
^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-
packages/PIL/TiffImagePlugin.py", line 1153, in __init__
super().__init__(fp, filename)
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-
packages/PIL/ImageFile.py", line 144, in __init__
self._open()
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-
packages/PIL/TiffImagePlugin.py", line 1177, in _open
self._seek(0)
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-
packages/PIL/TiffImagePlugin.py", line 1248, in _seek
self._setup()
File "/home/runner/work/django-imagefield/django-
imagefield/.tox/py312-djmain/lib/python3.12/site-
packages/PIL/TiffImagePlugin.py", line 1426, in _setup
raise ValueError(msg)
ValueError: Invalid dimensions
}}}

If I modify `get_image_dimensions` to also ignore `ValueError` exceptions
and try again with a larger chunk size the tests pass.

So, it's not exactly a small self-contained example but the following
shows the breakage:

{{{
git clone https://github.com/matthiask/django-imagefield.git
cd django-imagefield
tox -e py312-dj51
}}}

Downgrading to Pillow<11 works also, but since opening the file with
`PIL.Image` works I'd argue it's not really a Pillow issue, but an issue
with `get_image_dimensions` wanting to do interesting* things with
partially available image files. The variety of exceptions which are
ignored already show that the code tries to guard against various
problems, but it's incomplete.

I don't really have a good idea how to work around this issue since the
exception raised by Pillow is so unspecific.

*: I know the reasons and they make sense to me.
--
Ticket URL: <https://code.djangoproject.com/ticket/33240#comment:5>

Django

unread,

Nov 1, 2024, 7:54:02 AM11/1/24

to django-...@googlegroups.com

#33240: get_image_dimensions() raises ValueError on some .ico files.
-------------------------------------+-------------------------------------
Reporter: NKSM | Owner: Ath
| Tripathi

Type: Bug | Status: closed

Component: File | Version: 2.2
uploads/storage |

Severity: Normal | Resolution: needsinfo
Keywords: pillow | Triage Stage:

| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by Sarah Boyce):

* resolution: => needsinfo
* status: new => closed

Comment:

The error looks like a new error raised after type hints were added to
Pillow 11 see: https://github.com/python-
pillow/Pillow/commit/e6e5ef5c5fbd83ac5dd63301e4d7d6860a7b2d09
I agree with Mariusz here, what we are doing is
[https://pillow.readthedocs.io/en/stable/reference/ImageFile.html#example-
parse-an-image in the Pillow docs] and so should be supported by Pillow
(rather than we are doing anything too interesting/exotic here) - I think
first raise an issue to Pillow and see what they say
--
Ticket URL: <https://code.djangoproject.com/ticket/33240#comment:6>

Django

unread,

Nov 5, 2024, 2:42:10 AM11/5/24

to django-...@googlegroups.com

Keywords: pillow | Triage Stage:

| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Comment (by Sarah Boyce):

See #35884 which was marked as a duplicate

There is an issue raised against Pillow. I have commented there how the
error is raised following their documented example: https://github.com
/python-pillow/Pillow/issues/8530#issuecomment-2456436638. We will wait
for the response to that before pursuing a change in Django.

Also, thank you to Matthais for sharing a file I could test with and
comment on the issue
--
Ticket URL: <https://code.djangoproject.com/ticket/33240#comment:7>

Reply all

Reply to author

Forward