AAssetManager_open trigger filemap: Out of memory on 32bit devices

[seal2002]

Hello,

I have an issue with use AAssetManager_open to open the file on my game.

After open some of pack, it started to trigger the error and warning

08-27 23:28:58.316 14778 14975 E filemap : mmap(0,251402641) failed: Out of memory 08-27 23:28:58.316 14778 14975 W asset : create map from entry failed

And my game cannot read data any more - app crash

It happens on device 32bit with FW 7.1.2. After we change from GEF to Play Asset Delivery.

My game has around 10 packs, some of them is large but not excess 300MB.

This message happens after app read  4 packs successful.

I'm not sure it is common issue for all device 32bit or only happens with FW 7.1.2.

Do you have any idea about it?

Thanks

[John Dallman]

I don't have any specific knowledge of these APIs, but there is an obvious possibility. Are you closing the files you have opened, or leaving them all open? If you are leaving them open, they're apparently taking up address space, and it's very possible to run out of that in a 32-bit process. 

If you are leaving them open, try closing them and see if it makes a difference. 

Best,

John

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/1666a61c-a2e4-4373-b43f-96762038672dn%40googlegroups.com.

[seal2002]

Thanks John, I leave them open. As the game, open and read data on the pack, push to GPU, not close them after all. As the game tech design from begining, our team make the engine like it.

And we use GEF, everything is okay, even the code behind is use mmap as well. So that why I have the question, AAssetManager_open should use the same mmap too (I guess from the error log) but why It crash?

Thanks

[John Dallman]

I don't think it's crashing, as such. I think your program is running out of address space, and that problem is causing the mmap() call to fail. 

I suggest that, as an experiment, you should try closing the files to see if the problem goes away. If it does, that is evidence that running out of address space is the problem, and then you know what to look for. 

John

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/661738d5-4483-450d-b166-0184d44c9520n%40googlegroups.com.

[seal2002]

Yes, I text wrong the text above  :(.  Why it crash should be Why it is throw the out of memory).

I tried to switch huge packs - (200MB - 500MB) /pack(which store all resource files) x 8 to asset  invidual  file and the issue has delay. It will be failed when the game reach to out of address space.

And thank for the suggestion, so I should try to close each file after open it. Do you have suggestion how to debug the address space on the android?

Thanks

[John Dallman]

The first step in debugging this is to print the size of each chunk of address space as you map it and free it, and keep a running total of the amount allocated. After that, I don't know: I've never used this style of loading on any operating system. 

John

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/08054d7e-88e8-4f95-867c-4b5f4358c67cn%40googlegroups.com.

[seal2002]

Thank you for your suggestion. Currently I leave the issue as open - because it is small of devices has that trouble... And no free time to debug deeper on it.

[seal2002]

Hi, could you please let me know the function should I use to log the address space?

currently I try with getrlimit(RLIMIT_AS, &rlim); but it is same value for cur and max and the valuable is not reallity `4294967295`
size_t pagesize = getpagesize(); pagesize = 4096

[John Dallman]

I'm sorry, but I don't understand what you are asking. The question and the code don't seem to be related. 

John

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/fdfcc562-85b9-43a5-a0b6-77e67ce78506n%40googlegroups.com.

[seal2002]

Sorry to make you confuse, let me sumup why I do it.

The first step in debugging this is to print the size of each chunk of address space as you map it and free it, and keep a running total of the amount allocated

I thought that I need to print the address space is using by the process. As out process is using  AAssetManager_open, so I look at the https://developer.android.com/ndk/reference/group/asset to find the function to get total amount of address space used but no luck.

So I search around to google, and find the function getrlimit at https://man7.org/linux/man-pages/man2/setrlimit.2.html

RLIMIT_AS This is the maximum size of the process's virtual memory (address space). The limit is specified in bytes, and is rounded down to the system page size. This limit affects calls to brk(2), mmap(2), and mremap(2), which fail with the error ENOMEM upon exceeding this limit. In addition, automatic stack expansion fails (and generates a SIGSEGV that kills the process if no alternate stack has been made available via sigaltstack(2)). Since the value is a long, on machines with a 32-bit long either this limit is at most 2 GiB, or this resource is unlimited.

then I print the value to see the number of limit, but no luck...

I have try with dumpsys - https://developer.android.com/studio/command-line/dumpsys#meminfo and face 2 issues

1/ The game crash so fast, thinking to sleep the process than manual call dumpsys

2/ Still Not understand the value of so.mmap ... total value of it does not equal private & dirty...

Thanks,

[John Dallman]

The simpler way is to record the size of each mmap area as they are created, by adding some reporting code to your existing code that loads the assets. 

John

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/99efbfb2-39da-4843-898b-4fb3dc0d74fen%40googlegroups.com.

[seal2002]

Hi again,

I'm not sure I'm understood your idea, I use the code 

https://developer.android.com/ndk/reference/group/asset#aasset_getlength to print the total size use of each mmap? is it right? or we use another function to do that?

Thanks

[John Dallman]

I don't know. As I said at the start, I have no specific knowledge of these APIs. If you add up the asset sizes, what is the total? 

John

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/834f29b0-c823-41b0-a612-144bf5877a78n%40googlegroups.com.

[seal2002]

Hi,
It is around ~ 1GB, ~1073000000 Byte.

[John Dallman]

1GB is quite a large chunk out of a 32-bit app's address space. What else is taking up memory? 

John

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/93ff1d7b-7d0a-4d81-9771-23d06e5f7605n%40googlegroups.com.

[seal2002]

Hi,

First, I'm very appreciated with your input.

To be honest, I don't know how to log the mmap used correctly. Currently I call adb from my PC to log the mem - it might not exactly because has some delay... I try to call those command from my game but it doesn't effect.

I attached 2 log of them.

1. Mem.log: use `adb shell "cat /proc/meminfo"`

2. MemLogByPMap.log: adb shell pmap -x ProcessID

Thanks,

[John Dallman]

Aha. mem.log makes the answer clear:

MemTotal:        1547544 kB   # About 1.5GB on this device. 
MemFree:          661772 kB   # Android is using about 0.6GB
MemAvailable:     824008 kB   # You have only about 0.8GB free

I'm surprised that you are managing to mmap your 1GB of resources. You won't have any significant space left for anything else. Your choices are to stop taking up so much space with mmap'ed resources, or to only run devices with more memory. 

John

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/628cf2bc-a308-4aec-bf68-d98bb490a9c9n%40googlegroups.com.

[seal2002]

I'm blind with those information .... :(. Very clear now. Thank you, John.