Chromium process - (Deleted) .org.chromium.Chromium.* files holding disk space

[swasts7]

Hi, 

I am running a puppeteer script on Lambda in headless mode.

After few urls, chromium browser crashes as it runs out of disk space (Lambda has /tmp storage of 512MB - HARD LIMIT)

After some investigation, I found that after only loading first url : "https://about.google/?fg=1&utm_source=google-US&utm_medium=referral&utm_campaign=hp-header", my disk usage jumps from 28% to 64%.

However, running du command, I see only a small portion of disk space has been utilized.

So, I looked into process FD's and found a large number of deleted org.chromium files.

Output after loading above URL: https://about.google/?fg=1&utm_source=google-US&utm_medium=referral&utm_campaign=hp-header

DF (df -m):  Filesystem  1M-blocks   Used  Available    Use%   Mounted on 

             /dev/vdd     526        339     176         66%     /tmp

DU:

8.0K /tmp/puppeteer_dev_chrome_profile-pg9wQc/Default/Code Cache/wasm/index-dir

16K /tmp/puppeteer_dev_chrome_profile-pg9wQc/Default/Code Cache/wasm
8.0K /tmp/puppeteer_dev_chrome_profile-pg9wQc/Default/Code Cache/js/index-dir
148K /tmp/puppeteer_dev_chrome_profile-pg9wQc/Default/Code Cache/js
168K /tmp/puppeteer_dev_chrome_profile-pg9wQc/Default/Code Cache
4.0K /tmp/puppeteer_dev_chrome_profile-pg9wQc/Default/blob_storage/f86b4787-bad6-4642-942c-75d1f898a1de
8.0K /tmp/puppeteer_dev_chrome_profile-pg9wQc/Default/blob_storage
40K /tmp/puppeteer_dev_chrome_profile-pg9wQc/Default/GPUCache
240K /tmp/puppeteer_dev_chrome_profile-pg9wQc/Default
248K /tmp/puppeteer_dev_chrome_profile-pg9wQc
4.0K /tmp/locales
3.2M /tmp/aws/lib
976K /tmp/aws/.fonts/Open_Sans
980K /tmp/aws/.fonts
4.2M /tmp/aws
8.0K /tmp/cache-dir/index-dir
1.5M /tmp/cache-dir
4.8M /tmp/swiftshader
140M /tmp ---------------------> /tmp disk usage is clearly 140MB. But DF shows 339MB is used.

I can see many chromium process FD's holding onto org.chromium.Chromium.* deleted files. These files are up to 4MB in size and i found around 40 such deleted files when listed process FDs. (around 35 of which are 4MBs each). 

Total size of /tmp/.org.chromium.Chromium.* files was found to be 160MB.

One such FD : (Complete list is attached)

{ "name": "/tmp/.org.chromium.Chromium.zGlQxn (deleted)", "fd": "102", "stat": { "dev": 65072, "mode": 33152, "nlink": 0, "uid": 496, "gid": 495, "rdev": 0, "blksize": 4096, "ino": 219, "size": 4198400, "blocks": 8200, "atimeMs": 1595902017893.3027, "mtimeMs": 1595902017937.3027, "ctimeMs": 1595902017937.3027, "birthtimeMs": 1595902017853.3027, "atime": "2020-07-28T02:06:57.893Z", "mtime": "2020-07-28T02:06:57.937Z", "ctime": "2020-07-28T02:06:57.937Z", "birthtime": "2020-07-28T02:06:57.853Z" } },

This looks like a chromium bug? After loading few more such pages, my disk use % increases up to 100% and browser finally crashes.

Why chromium is blocking large disk space even though files are deleted. When i terminate the process at the end of the script, all this extra space is released. But I am not able to run entire script successfully because

disk capacity runs out.

This issue seems to be related to https://bugs.chromium.org/p/chromium/issues/detail?id=333996

Any help is much appreciated. Thank you!