Issue 334 in webp: android native crash when used libwebp-0.5.2 online

[yanyuet… via monorail]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 334 by yanyuet...@gmail.com: android native crash when used libwebp-0.5.2 online
https://bugs.chromium.org/p/webp/issues/detail?id=334

I used incremental api WebPIAppend() and multi-thread function. After published online, found many devices crashed as below, and most happened in 4.x systems. Is there a solution to fixed it? hurry, cause million of users online maybe encounter this problem. Thanks a lot!

--- --- --- --- --- --- --- --- --- --- --- --- --- --- --- ---
backtrace:
#00 pc 0x026f54 libwebp-v7a.so
#01 pc 0x0066e0 libwebp-v7a.so ((WebPIAppend+692))
#02 pc 0x005139 libwebp-v7a.so
#03 pc 0x0053ff libwebp-v7a.so
#04 pc 0x01e54c libdvm.so ((dvmPlatformInvoke+112))
#05 pc 0x04f9b5 libdvm.so ((_Z16dvmCallJNIMethodPKjP6JValuePK6MethodP6Thread+484))
#06 pc 0x027928 libdvm.so
#07 pc 0x02eeb8 libdvm.so ((_Z11dvmMterpStdP6Thread+76))
#08 pc 0x02c39c libdvm.so ((_Z12dvmInterpretP6ThreadPK6MethodP6JValue+188))
#09 pc 0x062eed libdvm.so ((_Z14dvmCallMethodVP6ThreadPK6MethodP6ObjectbP6JValueSt9__va_list+340))
#10 pc 0x062f11 libdvm.so ((_Z13dvmCallMethodP6ThreadPK6MethodP6ObjectP6JValuez+20))
#11 pc 0x057515 libdvm.so
#12 pc 0x00d650 libc.so ((__thread_entry+72))
--- --- --- --- --- --- --- --- --- --- --- --- --- --- --- ---


--
You received this message because:
1. The project was configured to send all issue notifications to this address

You may adjust your notification preferences at:
https://bugs.chromium.org/hosting/settings

[yanyuet… via monorail]

Comment #1 on issue 334 by yanyuet...@gmail.com: android native crash when used libwebp-0.5.2 online
https://bugs.chromium.org/p/webp/issues/detail?id=334#c1

Another backtrace below. Hope it work

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 00000028
r0 00000018 r1 72105009 r2 00000a60 r3 00000000
r4 00000000 r5 6f2f8f30 r6 69bea9d8 r7 00000000
r8 69be7468 r9 72105008 10 00000003 fp 00000a61
ip 00000001 sp 6346abc8 lr 624196e4 pc 62439f54 cpsr 60000010
#00 pc 00026f54 /mnt/asec/com.xxx.xxx-2/lib/libwebp-v7a.so
#01 pc 000066e0 /mnt/asec/com.xxx.xxx-2/lib/libwebp-v7a.so (WebPIAppend)
--- --- ---
00 pc 00026f54 /mnt/asec/com.xxx.xxx-2/lib/libwebp-v7a.so
01 pc 000066e0 /mnt/asec/com.xxx.xxx-2/lib/libwebp-v7a.so (WebPIAppend+692)
02 pc 00005139 /mnt/asec/com.xxx.xxx-2/lib/libwebp-v7a.so
03 pc 000053ff /mnt/asec/com.xxx.xxx-2/lib/libwebp-v7a.so
04 pc 0001e690 /system/lib/libdvm.so (dvmPlatformInvoke+112)
05 pc 0004e379 /system/lib/libdvm.so (_Z16dvmCallJNIMethodPKjP6JValuePK6MethodP6Thread+500)
06 pc 00027a68 /system/lib/libdvm.so
07 pc 0002bad0 /system/lib/libdvm.so (_Z12dvmInterpretP6ThreadPK6MethodP6JValue+180)
08 pc 000619e3 /system/lib/libdvm.so (_Z14dvmCallMethodVP6ThreadPK6MethodP6ObjectbP6JValueSt9__va_list+282)
09 pc 00061a0d /system/lib/libdvm.so (_Z13dvmCallMethodP6ThreadPK6MethodP6ObjectP6JValuez+20)
10 pc 00055fb9 /system/lib/libdvm.so
11 pc 0000e4a4 /system/lib/libc.so (__thread_entry+72)

[jz… via monorail]

Comment #2 on issue 334 by jz...@google.com: android native crash when used libwebp-0.5.2 online
https://bugs.chromium.org/p/webp/issues/detail?id=334#c2

Thanks for the report. Can you provide a bit more detail about your thread setup and the decode sequence? Are you able to reproduce this locally and provide a failing input as well as the calls to Append that led to this?

[yanyuet… via monorail]

Comment #3 on issue 334 by yanyuet...@gmail.com: android native crash when used libwebp-0.5.2 online
https://bugs.chromium.org/p/webp/issues/detail?id=334#c3

cannot reproduce locally now, the probability of occurrence is very small.I can provide the code follow, "doDecode" maybe be called from 4 threads simultaneously.

----------------------------------------------------------------------------------------------------------
static bool doDecode(JNIEnv *env, BaseStream &inStream, jobject options, uint8_t *output, jbyteArray jOutData) {
if (!output && jOutData) {
output = (uint8_t*)env->GetByteArrayElements(jOutData, 0);
size_t outLen = (size_t)env->GetArrayLength(jOutData);
NULL_RETURN_CHECK(output && outLen > 0, "get element failed from output jbytes");
}

bool ret = false;
uint8_t * chunk;
WebPIDecoder* idec;
volatile uint8_t cancelled = 0;
int status = VP8_STATUS_START_FEATURES;
int sampleSize = -1;
int width, height;
size_t appendSize;
bool neverDecode;
InputType iType;
WebPDecoderConfig* config = new WebPDecoderConfig();

if (!WebPInitDecoderConfig(config)) {
DBE("init WebPDecoderConfig failed");
goto ENDING;
}

width = getIntField(env, options, gOptions_outWidthFieldID);
height = getIntField(env, options, gOptions_outHeightFieldID);
neverDecode = width <= 0 || height <= 0;

if(neverDecode) {
jbyteArray bufferArray = static_cast<jbyteArray>(env->GetObjectField(options, gOptions_headerBufferFieldID));

if (bufferArray) {
uint8_t* bufferAddr = (uint8_t*)env->GetByteArrayElements(bufferArray, 0);
size_t bufferLen = (size_t)env->GetArrayLength(bufferArray);
if (bufferAddr && bufferLen >= REQUEST_MIN_HEADER_BUFFER_SIZE) {
status = WebPGetFeatures(bufferAddr, REQUEST_MIN_HEADER_BUFFER_SIZE, &config->input);
env->ReleaseByteArrayElements(bufferArray, (jbyte*)bufferAddr, 0);
}
}

if (status != VP8_STATUS_OK) {
uint8_t * header;
int size = inStream.nextChunk(&header, REQUEST_MIN_HEADER_BUFFER_SIZE);
if (size > 0) {
status = WebPGetFeatures(header, (size_t) size, &config->input);
inStream.freeChunk(header);
}
inStream.rewind();
if (env->ExceptionCheck()) {
goto ENDING;
}
}

if(status != VP8_STATUS_OK) {
DBE("get webp features failed, err=%d", status);
goto ENDING;
}
}

if (neverDecode) {
width = config->input.width;
height = config->input.height;
sampleSize = getIntField(env, options, gOptions_sampleSizeFieldID);
if (sampleSize > 1) {
width = width / sampleSize;
height = height * width / config->input.width;
}
env->SetIntField(options, gOptions_outWidthFieldID, width);
env->SetIntField(options, gOptions_outHeightFieldID, height);
env->SetBooleanField(options, gOptions_outAlphaFieldID, (jboolean) config->input.has_alpha);
}

ret = getBoolField(env, options, gOptions_justBoundsFieldID);
if (ret || !output) {
goto ENDING;
}

if (sampleSize < 0) {
sampleSize = getIntField(env, options, gOptions_sampleSizeFieldID);
}

if (sampleSize > 1) {
config->options.use_scaling = 1;
config->options.scaled_width = width;
config->options.scaled_height = height;
}


config->options.bypass_filtering = 1;
config->options.no_fancy_upsampling = 0;
config->options.use_threads = 1;

config->output.colorspace = MODE_rgbA;
config->output.is_external_memory = 1;
config->output.u.RGBA.rgba = output;
config->output.u.RGBA.stride = width * 4;
config->output.u.RGBA.size = (size_t) (config->output.u.RGBA.stride * height);

status = VP8_STATUS_START_DECODE;

env->SetLongField(options, gOptions_cancelledPtrFieldID, reinterpret_cast<jlong>(&cancelled));

iType = inStream.inputType();
if (iType == BYTE_ARRAY) {
int size = inStream.nextChunk(&chunk, inStream.available());
if (size > 0) {
status = WebPDecode(chunk, (size_t) size, config);
} else {
DBE("byte array stream available %d", size);
}
} else {
idec = WebPIDecode(NULL, 0, config);
if (idec) {
if (iType == FILE_DESCRIPTOR) {
appendSize = FD_APPEND_CHUNK_SIZE;
} else {
appendSize = STREAM_APPEND_CHUNK_SIZE;
}
chunk = (uint8_t*) calloc(appendSize, sizeof(char));
if (chunk) {
int bytesRead;
do {
bytesRead = inStream.nextChunk(chunk, appendSize);
if (bytesRead < 0) {
break;
}
if (bytesRead > 0) {
//DBD("append decode chunk size=%d", bytesRead);
status = WebPIAppend(idec, chunk, (size_t) bytesRead);
//DBD("append decode result status=%d", status);
}
if (cancelled) {
status = VP8_STATUS_USER_ABORT;
DBW("decoding was cancelled in progress");
break;
}
if (VP8_STATUS_OK == status) { // complete decoding
break;
}
} while (VP8_STATUS_SUSPENDED == status || bytesRead == 0);

free(chunk);
} else {
DBE("calloc new memory failed");
}
WebPIDelete(idec);
} else {
DBE("init webp idec failed");
}
}

if (!cancelled) {
env->SetLongField(options, gOptions_cancelledPtrFieldID, 0);
}

ret = status == VP8_STATUS_OK;
if(!ret) {
DBE("decode webp failed, err=%d, type=%d", status, iType);
}
ENDING:
if (jOutData != NULL) {
env->ReleaseByteArrayElements(jOutData, (jbyte*)output, 0);
}
delete config;
return ret;

[jz… via monorail]

Comment #4 on issue 334 by jz...@google.com: android native crash when used libwebp-0.5.2 online
https://bugs.chromium.org/p/webp/issues/detail?id=334#c4

Thanks for the code, we'll take a look. How is the library being built and do you know what devices you're seeing the crash on?

[pascal.m… via monorail]

Comment #5 on issue 334 by pascal.m...@gmail.com: android native crash when used libwebp-0.5.2 online
https://bugs.chromium.org/p/webp/issues/detail?id=334#c5

Thanks for the code.

I am not sure this is related but i noticed something:

in the following code:

if (bufferArray) {
uint8_t* bufferAddr = (uint8_t*)env->GetByteArrayElements(bufferArray, 0);
size_t bufferLen = (size_t)env->GetArrayLength(bufferArray);
if (bufferAddr && bufferLen >= REQUEST_MIN_HEADER_BUFFER_SIZE) {
status = WebPGetFeatures(bufferAddr, REQUEST_MIN_HEADER_BUFFER_SIZE, &config->input);
env->ReleaseByteArrayElements(bufferArray, (jbyte*)bufferAddr, 0);
}
}

the function "env->ReleaseByteArrayElements()" might not be called if "bufferAddr != NULL" but "bufferLen < REQUEST_MIN_HEADER_BUFFER_SIZE". In this case, GetByteArrayElements() succeeded but you are not calling the
corresponding ReleaseByteArrayElements().

Just to be sure i'd change the code to:

uint8_t* bufferAddr = (uint8_t*)env->GetByteArrayElements(bufferArray, 0);

if (bufferAddr) {

size_t bufferLen = (size_t)env->GetArrayLength(bufferArray);

if (bufferLen >= REQUEST_MIN_HEADER_BUFFER_SIZE) {

status = WebPGetFeatures(bufferAddr, REQUEST_MIN_HEADER_BUFFER_SIZE, &config->input);
}
env->ReleaseByteArrayElements(bufferArray, (jbyte*)bufferAddr, 0);
}

[yanyuet… via monorail]

Comment #6 on issue 334 by yanyuet...@gmail.com: android native crash when used libwebp-0.5.2 online
https://bugs.chromium.org/p/webp/issues/detail?id=334#c6

Thank you for your advice @pascal, yeah GetXXX And ReleaseXXX not be called in pairs. the crash happened all kinds of devices, such as HUAWEI G750-T01,OPPO R7,OPPO R7s,R8207,R7c,vivo X5Max+,R8107,GT-N7100.Here is my libwebp-v7a.so build script:

-----------------------------------------------------------------------------
LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)

LOCAL_MODULE := phxwebp
SRC_ROOT_PATH := $(LOCAL_PATH)

LOCAL_C_INCLUDES := $(SRC_ROOT_PATH) $(NDK_PATH)/sources/cxx-stl/gnu-libstdc++/4.9/include

LOCAL_SRC_FILES := \
$(SRC_ROOT_PATH)/../ByteArrayStream.cpp \
$(SRC_ROOT_PATH)/../FileStream.cpp \
$(SRC_ROOT_PATH)/../RewindStream.cpp \
$(SRC_ROOT_PATH)/../JniMain.cpp \
$(wildcard $(SRC_ROOT_PATH)/*.cpp)


LOCAL_STATIC_LIBRARIES := libwebp_dec

LOCAL_CFLAGS += -DREGISTER_OPTIONS_CANCEL -fvisibility=hidden -D__STDC_CONSTANT_MACROS -fpic -Wno-deprecated-declarations
LOCAL_CPPFLAGS += -std=c++11
LOCAL_LDLIBS := -ldl -llog -landroid -lstdc++ -lz

LOCAL_LDLIBS += -L$(NDK_PATH)/sources/cxx-stl/gnu-libstdc++/4.9/libs/x86
LOCAL_LDLIBS += -L$(SYSROOT)/usr/lib -llog

LOCAL_SHARED_LIBRARIES := gnustl-prebuilt

include $(BUILD_SHARED_LIBRARY)

$(call import-module, webp)

Here is module webp build script:

------------------------------------------------------------------------------
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)

SRC_DIR := libwebp-0.5.2
WEBP_CFLAGS := -Wall -DANDROID -DHAVE_MALLOC_H -DHAVE_PTHREAD -DWEBP_USE_THREAD
WEBP_CFLAGS += -fvisibility=hidden

ifeq ($(APP_OPTIM),release)
WEBP_CFLAGS += -finline-functions -ffast-math \
-ffunction-sections -fdata-sections
ifeq ($(findstring clang,$(NDK_TOOLCHAIN_VERSION)),)
WEBP_CFLAGS += -frename-registers -s
endif
endif

ifneq ($(filter armeabi-v7a, $(TARGET_ARCH_ABI)),)
# Setting LOCAL_ARM_NEON will enable -mfpu=neon which may cause illegal
# instructions to be generated for armv7a code. Instead target the neon code
# specifically.
NEON := c.neon
USE_CPUFEATURES := yes
else
NEON := c
endif

dec_srcs := \
$(SRC_DIR)/src/dec/alpha.c \
$(SRC_DIR)/src/dec/buffer.c \
$(SRC_DIR)/src/dec/frame.c \
$(SRC_DIR)/src/dec/idec.c \
$(SRC_DIR)/src/dec/io.c \
$(SRC_DIR)/src/dec/quant.c \
$(SRC_DIR)/src/dec/tree.c \
$(SRC_DIR)/src/dec/vp8.c \
$(SRC_DIR)/src/dec/vp8l.c \
$(SRC_DIR)/src/dec/webp.c \

dsp_dec_srcs := \
$(SRC_DIR)/src/dsp/alpha_processing.c \
$(SRC_DIR)/src/dsp/alpha_processing_sse2.c \
$(SRC_DIR)/src/dsp/alpha_processing_sse41.c \
$(SRC_DIR)/src/dsp/argb.c \
$(SRC_DIR)/src/dsp/argb_sse2.c \
$(SRC_DIR)/src/dsp/cpu.c \
$(SRC_DIR)/src/dsp/dec.c \
$(SRC_DIR)/src/dsp/dec_clip_tables.c \
$(SRC_DIR)/src/dsp/dec_neon.$(NEON) \
$(SRC_DIR)/src/dsp/dec_sse2.c \
$(SRC_DIR)/src/dsp/dec_sse41.c \
$(SRC_DIR)/src/dsp/filters.c \
$(SRC_DIR)/src/dsp/filters_sse2.c \
$(SRC_DIR)/src/dsp/lossless.c \
$(SRC_DIR)/src/dsp/lossless_neon.$(NEON) \
$(SRC_DIR)/src/dsp/lossless_sse2.c \
$(SRC_DIR)/src/dsp/rescaler.c \
$(SRC_DIR)/src/dsp/rescaler_neon.$(NEON) \
$(SRC_DIR)/src/dsp/rescaler_sse2.c \
$(SRC_DIR)/src/dsp/upsampling.c \
$(SRC_DIR)/src/dsp/upsampling_neon.$(NEON) \
$(SRC_DIR)/src/dsp/upsampling_sse2.c \
$(SRC_DIR)/src/dsp/yuv.c \
$(SRC_DIR)/src/dsp/yuv_sse2.c \

utils_dec_srcs := \
$(SRC_DIR)/src/utils/bit_reader.c \
$(SRC_DIR)/src/utils/color_cache.c \
$(SRC_DIR)/src/utils/filters.c \
$(SRC_DIR)/src/utils/huffman.c \
$(SRC_DIR)/src/utils/quant_levels_dec.c \
$(SRC_DIR)/src/utils/random.c \
$(SRC_DIR)/src/utils/rescaler.c \
$(SRC_DIR)/src/utils/thread.c \
$(SRC_DIR)/src/utils/utils.c \

################################################################################
# libwebpdecoder

include $(CLEAR_VARS)

LOCAL_SRC_FILES := \
$(dec_srcs) \
$(dsp_dec_srcs) \
$(utils_dec_srcs) \

LOCAL_CFLAGS := $(WEBP_CFLAGS)
LOCAL_C_INCLUDES += $(LOCAL_PATH)/$(SRC_DIR)/src
LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/libwebp/src

# prefer arm over thumb mode for performance gains
LOCAL_ARM_MODE := arm

ifeq ($(USE_CPUFEATURES),yes)
LOCAL_STATIC_LIBRARIES := cpufeatures
endif

LOCAL_MODULE := libwebp_dec

include $(BUILD_STATIC_LIBRARY)

ifeq ($(USE_CPUFEATURES),yes)
$(call import-module,android/cpufeatures)
endif

[pascal.m… via monorail]

Comment #7 on issue 334 by pascal.m...@gmail.com: android native crash when used libwebp-0.5.2 online
https://bugs.chromium.org/p/webp/issues/detail?id=334#c7

Without a clear repro case and a stack in debug mode (so we can see where the crash is happening), it's very difficult to guess what is happening here.

Some remarks/questions:
* What is gOptions_cancelledPtrFieldID() function doing exactly? Do you have the code for it?
* The object 'idec' is purely local to the thread, so it's close to impossible that the threads would collide here.
* What is the 'BaseStream inStream' object? Is it thread-safe ? Its content must be accessible to the 'idec' object during the WebPIAppend() call. Could it be that its content is remapped by another thread while it is also used by an idec object?

[jz… via monorail]

Comment #8 on issue 334 by jz...@google.com: android native crash when used libwebp-0.5.2 online
https://bugs.chromium.org/p/webp/issues/detail?id=334#c8

> cannot reproduce locally now, the probability of occurrence is very small.I can provide the code follow, "doDecode" maybe be called from 4 threads simultaneously.

Can you reduce that to 1 to see if you still observe the issue?

[jz… via monorail]

Comment #9 on issue 334 by jz...@google.com: android native crash when used libwebp-0.5.2 online
https://bugs.chromium.org/p/webp/issues/detail?id=334#c9

Any update on this? Have you tried the latest release (0.6.0) or the master branch?

[jz… via monorail]

Updates:
Status: Invalid

Comment #11 on issue 334 by jz...@google.com: android native crash when used libwebp-0.5.2 online
https://bugs.chromium.org/p/webp/issues/detail?id=334#c11

Closing this bug for lack of activity. If you're still experiencing an issue and can add some updates to the open questions please feel free to reopen.