FatalOOM on Android

[Mati Cohen]

Hi,

I've created new builds for my Android app pointing to 14.0.365.4 (I used to be on a version from 2022).

Everything seems to be working properly on production, except from a FatalOOM crash that we are unable to explain. According to Firebase crashing devices have enough memory (more than 400MB) and the crash takes place immediately after we call V8::Initialize().

Here is the full stacktrace:

#00 pc 0x1329140 (v8::base::OS::Abort() [/data/app/~~tQAJjlgMcZP-x-50eIcSag==/com.encircle-f_MMozmE6xmMXhHJsIXUPw==/split_config.arm64_v8a.apk!liben8.so]) (BuildId: df4010b9be1607b6935ef08e8d90830ac0ab4f0b)
#01 pc 0x132c898 (v8::base::FatalOOM(v8::base::OOMType, char const*) [/data/app/~~tQAJjlgMcZP-x-50eIcSag==/com.encircle-f_MMozmE6xmMXhHJsIXUPw==/split_config.arm64_v8a.apk!liben8.so]) (BuildId: df4010b9be1607b6935ef08e8d90830ac0ab4f0b)
#02 pc 0x13396f8 (v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/data/app/~~tQAJjlgMcZP-x-50eIcSag==/com.encircle-f_MMozmE6xmMXhHJsIXUPw==/split_config.arm64_v8a.apk!liben8.so]) (BuildId: df4010b9be1607b6935ef08e8d90830ac0ab4f0b)
#03 pc 0x14c2acc (v8::internal::(anonymous namespace)::GlobalFatalOutOfMemoryHandlerImpl(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char>> const&, v8::SourceLocation const&, cppgc::internal::HeapBase*) [/data/app/~~tQAJjlgMcZP-x-50eIcSag==/com.encircle-f_MMozmE6xmMXhHJsIXUPw==/split_config.arm64_v8a.apk!liben8.so]) (BuildId: df4010b9be1607b6935ef08e8d90830ac0ab4f0b)
#04 pc 0x1e06c60 (cppgc::internal::FatalOutOfMemoryHandler::operator()(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char>> const&, v8::SourceLocation const&) const [/data/app/~~tQAJjlgMcZP-x-50eIcSag==/com.encircle-f_MMozmE6xmMXhHJsIXUPw==/split_config.arm64_v8a.apk!liben8.so]) (BuildId: df4010b9be1607b6935ef08e8d90830ac0ab4f0b)
#05 pc 0x1e0ff2c (cppgc::internal::CagedHeap::CagedHeap(v8::PageAllocator&, unsigned long) [/data/app/~~tQAJjlgMcZP-x-50eIcSag==/com.encircle-f_MMozmE6xmMXhHJsIXUPw==/split_config.arm64_v8a.apk!liben8.so]) (BuildId: df4010b9be1607b6935ef08e8d90830ac0ab4f0b)
#06 pc 0x1e0fca8 (cppgc::internal::CagedHeap::InitializeIfNeeded(v8::PageAllocator&, unsigned long) [/data/app/~~tQAJjlgMcZP-x-50eIcSag==/com.encircle-f_MMozmE6xmMXhHJsIXUPw==/split_config.arm64_v8a.apk!liben8.so]) (BuildId: df4010b9be1607b6935ef08e8d90830ac0ab4f0b)
#07 pc 0x1e06d90 (cppgc::InitializeProcess(v8::PageAllocator*, unsigned long) [/data/app/~~tQAJjlgMcZP-x-50eIcSag==/com.encircle-f_MMozmE6xmMXhHJsIXUPw==/split_config.arm64_v8a.apk!liben8.so]) (BuildId: df4010b9be1607b6935ef08e8d90830ac0ab4f0b)
#08 pc 0x1356aac (v8::V8::Initialize(int) [/data/app/~~tQAJjlgMcZP-x-50eIcSag==/com.encircle-f_MMozmE6xmMXhHJsIXUPw==/split_config.arm64_v8a.apk!liben8.so]) (BuildId: df4010b9be1607b6935ef08e8d90830ac0ab4f0b)
#09 pc 0x13245f0 (Runtime::Runtime(_JNIEnv*) [v8-initialization.h:127]) (BuildId: df4010b9be1607b6935ef08e8d90830ac0ab4f0b)
#10 pc 0x131c090 (Java_com_encircle_en8_Runtime_en8Create [en8.cpp:28]) (BuildId: df4010b9be1607b6935ef08e8d90830ac0ab4f0b)
#11 pc 0x2d357c (art_jni_trampoline [/system/framework/arm64/boot.oat]) (BuildId: 5f2f994dfe4014c6ca8efc57596b26a7e9120ce6)
#12 pc 0x9be120 (com.encircle.jsenv.EventLoop$$ExternalSyntheticLambda7.run [/data/app/~~tQAJjlgMcZP-x-50eIcSag==/com.encircle-f_MMozmE6xmMXhHJsIXUPw==/oat/arm64/base.odex])
#13 pc 0x9e41f4 (com.encircle.util.PrioritizedTask.run [/data/app/~~tQAJjlgMcZP-x-50eIcSag==/com.encircle-f_MMozmE6xmMXhHJsIXUPw==/oat/arm64/base.odex])
#14 pc 0x210b74 (java.util.concurrent.ThreadPoolExecutor.runWorker [/system/framework/arm64/boot.oat]) (BuildId: 5f2f994dfe4014c6ca8efc57596b26a7e9120ce6)
#15 pc 0x214b18 (java.util.concurrent.ThreadPoolExecutor$Worker.run [/system/framework/arm64/boot.oat]) (BuildId: 5f2f994dfe4014c6ca8efc57596b26a7e9120ce6)
#16 pc 0xa5500 (java.lang.Thread.run [/system/framework/arm64/boot.oat]) (BuildId: 5f2f994dfe4014c6ca8efc57596b26a7e9120ce6)
#17 pc 0x32d194 (art_quick_invoke_stub [/apex/com.android.art/lib64/libart.so]) (BuildId: 80d2ab18f9d259d8e546c1e6bae752b1)
#18 pc 0x2de270 (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*) [/apex/com.android.art/lib64/libart.so]) (BuildId: 80d2ab18f9d259d8e546c1e6bae752b1)
#19 pc 0x4bfcf4 (art::Thread::CreateCallback(void*) [/apex/com.android.art/lib64/libart.so]) (BuildId: 80d2ab18f9d259d8e546c1e6bae752b1)
#20 pc 0x4bf940 (art::Thread::CreateCallbackWithUffdGc(void*) [/apex/com.android.art/lib64/libart.so]) (BuildId: 80d2ab18f9d259d8e546c1e6bae752b1)
#21 pc 0x82740 (__pthread_start(void*) [/apex/com.android.runtime/lib64/bionic/libc.so]) (BuildId: 61a049a7ad18156ebc52d8d483539df9)
#22 pc 0x74b98 (__start_thread [/apex/com.android.runtime/lib64/bionic/libc.so]) (BuildId: 61a049a7ad18156ebc52d8d483539df9)

Is there any explanation for this?

Thanks a lot!

[Mati Cohen]

I see some changes related to how much heap is allocated to Oilpan. Now, default seems to be 16GB (whereas it used to be 4GB). Could it be an issue for Android devices?

I am rebuilding now with cppgc_enable_larger_cage=false.

[Mati Cohen]

Hey, unfortunately this issue has become critical in production. I had to halt the release.

Can I get some help please?

I am not sure if my theory is right since I am not able to reproduce the crash to verify it.

Thank you in advance.

[Igor Sheludko]

Hello,

It seems that V8 crashes because it can't reserve (not allocate!) an 4GB of address space for a pointer compression cage for Oilpan. Not sure why this happens in your case (Chrome works just fine on Android), looks like there's some system limit on allowed address space reservations for a process.

As a workaround, you could try to disable Oilpan pointer compression by setting this GN argument: cppgc_enable_pointer_compression=false.

--
--
v8-users mailing list
v8-u...@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups "v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/v8-users/266ccc03-ff09-466a-9cfa-73f6394685c2n%40googlegroups.com.

--

Igor Sheludko

Software Engineer

ish...@google.com

Google Germany GmbH

Erika-Mann-Straße 33

80636 München

Geschäftsführer: Paul Manicle, Liana Sebastian

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich. Falls sie diese fälschlicherweise erhalten haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail an die falsche Person gesendet wurde.

    

This e-mail is confidential. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person.

[Mati Cohen]

Thank you very much Igor. Yes, sorry, I meant reserve.

I thought the issue was that, by default, we are now trying to reserve 16GB, thus my idea of disabling cppgc_enable_larger_cage. Do you suggest to just disable cppgc_enable_pointer_compression (and no changes to the cage configuration)?

Thanks again!

[Igor Sheludko]

Disabling cppgc_enable_pointer_compression will hopefully give you a hint about what/where to look for.

I'm rather suggesting to figure out where that address space limitation is coming from. Maybe there's something special with the Android devices that fail to initialize V8 or something.

Oilpan pointer compression was enabled around 2023 and since we don't test all possible build configurations, this configuration might be undertested.

To view this discussion visit https://groups.google.com/d/msgid/v8-users/a0315db5-c82b-4d46-a546-9735afca8ae6n%40googlegroups.com.

[Mati Cohen]

All my kudos to you Igor!!