Issue 381 in webp: Excessive allocation (7271)

[d… via monorail]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 381 by d...@git.imagemagick.org: Excessive allocation (7271)
https://bugs.chromium.org/p/webp/issues/detail?id=381

The ImageMagick project is using https://github.com/google/oss-fuzz to find bugs in our own library and in libraries that we use. The fuzzer found an issue and we think this is an issue that should be resolved in the library that we use. This issue is posted under the url https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=7271 that is not publicly visible yet but added as a link for future reference. Below are the details of the issue that can be reproduced using the following technique: https://github.com/google/oss-fuzz/blob/master/docs/reproducing.md

The stacktrace in the bug report cannot be copy pasted because it contains no information. The issue is reporting an out of memory error because of excessive allocation inside the webp library. I stepped through it with a debugger and saw a huge allocation at his line: https://github.com/webmproject/libwebp/blob/f9df0081a71248d4cea893a5e28c1ef03b86cb92/src/dec/vp8l_dec.c#L406. num_htree_groups=65536, table_size=5004 and sizeof(*huffman_tables)=4 resulting in 1311768576 (1.3GB). This looks a bit excessive for an image that is only 33x132. The file that can be used to reproduce the issue is attached.


Attachments:
clusterfuzz-testcase-minimized-encoder_webp_fuzzer-6683760943693824 166 bytes

--
You received this message because:
1. The project was configured to send all issue notifications to this address

You may adjust your notification preferences at:
https://bugs.chromium.org/hosting/settings

[pascal.m… via monorail]

Updates:
Owner: pascal.m...@gmail.com

Comment #1 on issue 381 by pascal.m...@gmail.com: Excessive allocation (7271)
https://bugs.chromium.org/p/webp/issues/detail?id=381#c1

thanks for the report and sample.
I can indeed see the big allocation when using PRINT_MEM_INFO:

-------
> dwebp clusterfuzz-testcase-minimized-encoder_webp_fuzzer-6683760943693824
Mem: 368 (+368)
Mem: 12184 (+11816)
Mem: 12752 (+568)
Mem: 13872 (+1120)
Mem: 12752 (-1120)
Mem: 13152 (+400)
Mem: 1336 (-11816)
Mem: 768 (-568)
Mem: 1792 (+1024)
Mem: 1392 (-400)
Mem: 15264 (+13872)
Mem: 15832 (+568)
Mem: 19000 (+3168)
Mem: 20584 (+1584)
Mem: 19000 (-1584)
Mem: 15832 (-3168)
Mem: 17880 (+2048)
Mem: 18220 (+340)
Mem: 4348 (-13872)
Mem: 3780 (-568)
Mem: 1732 (-2048)
Mem: 1311770308 (+1311768576)
Mem: 1348994756 (+37224448)
Mem: 1349004068 (+9312)
Mem: 1348994756 (-9312)
Mem: 1348994416 (-340)
Mem: 37225840 (-1311768576)
Mem: 1392 (-37224448)
Mem: 368 (-1024)
Mem: 0 (-368)
Decoding of clusterfuzz-testcase-minimized-encoder_webp_fuzzer-6683760943693824 failed.
Status: 3(BITSTREAM_ERROR)

MEMORY INFO:
num calls to: malloc = 10
calloc = 5
free = 15
total_mem: 0
total_mem allocated: 1349039212
high-water mark: 1349004068

-----

Although large, this allocation seems within spec bounds. That's probably an oversight. I'll investigate further, regarding how this could be mitigated.

[bugdro… via monorail]

Comment #2 on issue 381 by bugd...@chromium.org: Excessive allocation (7271)
https://bugs.chromium.org/p/webp/issues/detail?id=381#c2

The following revision refers to this bug:
https://chromium.googlesource.com/webm/libwebp/+/dce5d7643177633ebe3513af492ea8c08c299cf3

commit dce5d7643177633ebe3513af492ea8c08c299cf3
Author: Vincent Rabaud <vra...@google.com>
Date: Thu May 03 11:53:44 2018

Limit memory allocation when reading invalid Huffman codes.

BUG=webp:381

Change-Id: I6b68a33689a3309691eba582b759131b81b612c1

[modify] https://crrev.com/dce5d7643177633ebe3513af492ea8c08c299cf3/src/dec/vp8l_dec.c

[vincent.… via monorail]

Updates:
Status: Fixed

Comment #3 on issue 381 by vincent....@gmail.com: Excessive allocation (7271)
https://bugs.chromium.org/p/webp/issues/detail?id=381#c3

Thx a lot for this sample. This is indeed too much flexibility given by the spec. It is now handled properly. Thx !

[vincent.… via monorail]

Comment #4 on issue 381 by vincent....@gmail.com: Excessive allocation (7271)
https://bugs.chromium.org/p/webp/issues/detail?id=381#c4

BTW, we now get:
Mem: 368 (+368)
Mem: 1488 (+1120)
Mem: 13304 (+11816)
Mem: 13872 (+568)

Mem: 12752 (-1120)
Mem: 13152 (+400)
Mem: 1336 (-11816)
Mem: 768 (-568)
Mem: 1792 (+1024)
Mem: 1392 (-400)

Mem: 4560 (+3168)
Mem: 18432 (+13872)
Mem: 19000 (+568)

Mem: 20584 (+1584)
Mem: 19000 (-1584)
Mem: 15832 (-3168)
Mem: 17880 (+2048)
Mem: 18220 (+340)
Mem: 4348 (-13872)
Mem: 3780 (-568)
Mem: 1732 (-2048)

Mem: 11044 (+9312)
Mem: 20027044 (+20016000)
Mem: 20595044 (+568000)
Mem: 20585732 (-9312)
Mem: 20585392 (-340)
Mem: 569392 (-20016000)
Mem: 1392 (-568000)

Mem: 368 (-1024)
Mem: 0 (-368)

Decoding of /tmp/clusterfuzz-testcase-minimized-encoder_webp_fuzzer-6683760943693824 failed.

Status: 3(BITSTREAM_ERROR)

MEMORY INFO:
num calls to: malloc = 10
calloc = 5
free = 15
total_mem: 0

total_mem allocated: 20630188
high-water mark: 20595044