A question about image memory allocation in Vulkan backend

[ChenHz Le]

Hi,

Recently I used angle vulkan backend to do some pressure testing and met a problem about memory allocation. Here it is:

(With my pool graphics card AMD Radeon RX 560 Series)

I rendered a complex scene with multiple large texture. And angle created these texture with memoryFlag DEVICE_LOCAL, which may used up my whole device local heap(HEAP0).

When more textures need to be created,  my local heap has run out so I modified angle a little to create vkImage in non-local heap as a fallback way.  But this time the scene render at a low fps.  I guess this maybe due to the low access performance of these images by the GPU.

Can anybody tell me how to deal with this situation? or Is there some solutions in Angle, like a LRU stragegy that transfer image memory between local and host heaps as needed to gain better performance?

Also, I noticed angle uses the VMA to manager memory for buffers. I wonder that why not use it to manager memory for images as well, and if there is any stragegies like the one mentioned above in it?

[Shahbaz Youssefi]

Hi,

Thank you for the report. I have a question; is HEAP0 only DEVICE_LOCAL, or does it also include HOST_VISIBLE? ANGLE doesn't ask for HOST_VISIBLE when creating image memory, but perhaps it's picking up a small DEVICE_LOCAL|HOST_VISIBLE heap and prematurely runs out of memory. In other words, I'm trying to understand if this is simply a matter of ANGLE choosing a suboptimal heap (i.e. there are other heaps that could have been used), or that your application is actually exhausting the GPU memory. The contents of `vulkaninfo` would be useful here.

> Is there some solutions in Angle, like a LRU stragegy that transfer image memory between local and host heaps as needed to gain better performance?

There is no such strategy in ANGLE at the moment.

> Also, I noticed angle uses the VMA to manager memory for buffers. I wonder that why not use it to manager memory for images as well,

Because it doesn't seem to be helpful. Even for large buffers, we make dedicated allocations so they can be freed up earlier and reduce our memory footprint. Images are usually large enough that by the same strategy practically would rarely be suballocated with VMA anyway.

> and if there is any stragegies like the one mentioned above in it?

If there is, I'm not aware of it.

> Can anybody tell me how to deal with this situation?

There could be ideas once we understand how the memory heaps on your system look like.

Cheers,

[ChenHz Le]

Thanks for your reply!  In my case,  HEAP0 only includes the DEVICE_LOCAL bit. and here is some infos:

[Shahbaz Youssefi]

Thanks. Heap 0 has 4GB of memory, so it seems like this is simply a case of OOM. ANGLE doesn't attempt to work around this by moving the textures back and forth between RAM and VRAM, so you'd need to work around it.

For my information, does the same code work with the GL backend of ANGLE? Or just directly on the native driver? I'm curious to know if the native OpenGL driver deals with OOM more gracefully.

That said, my suggestion is to query the available memory beforehand, and adjust the application accordingly. For example, you could drop mip 0 of large textures to save memory. That's a good strategy even if ANGLE _did_ deal with OOM, because the app would be much faster on a graphics card that's clearly not made to deal with such large amount of memory. As a result, your application would be more friendly to lower-end devices.

[Shahbaz Youssefi]

There was a message that was deleted, I'm not sure if you still expect a reply, but I'll try to give a generic one without referencing it:

First, make sure you use compressed textures as much as possible (for read-only textures). That's not just good for memory, but will also greatly improve performance. If you still have memory issues, my advice is to detect available memory and adjust the application accordingly (e.g. by dropping mip 0 from large textures), rather than attempt to juggle memory between host and device. That's because said juggling is costly, and also a device with low memory is probably not strong enough to give good performance accessing large amount of memory anyway.

If you still need memory to be juggled, you can implement that in the application itself, which knows better when the texture is not going to be used anymore (e.g. moving out of a zone in a game). To that end, you can either:

- Copy the texture to PBO (which will have HOST_VISIBLE memory), delete it. When needed again, reupload the texture from PBO.

- Or, if you only apply this to read-only textures, delete the texture when not needed, and reload it again when needed.

Hope that helps. Cheers,

[ChenHz Le]

Thanks.  your suggestions inspire me a lot and I will try to work around this later!

Cheers.