The main out/soong/build.ninja file is too large, making localhost compilation difficult

[Yuxin Hu]

Android 13

PLEASE READ:

Our colleagues are developing on Android 13 and often encounter the problem of compilation process interruption during compilation。

soong_build process returns abnormally shortly after compilation

```
Killed
19:51:22 soong bootstrap failed with: exit status 1
```

The information that can be obtained by analyzing the `dmesg` log of the host is that the soong_build process was killed because of Out of memory. 

After analysis, we believe that one of the main reasons for this problem at the beginning of compilation is that out/soong/build.ninja is too large, which leads to out of memory.

```

huyuxin@nj-builder-01:~/missi_t_qcom/out$ cat combined-missi_phone_cn.ninja

builddir = out
pool highmem_pool
 depth = 31
subninja out/build-missi_phone_cn.ninja
subninja out/build-missi_phone_cn-package.ninja
subninja out/soong/build.ninja
huyuxin@nj-builder-01:~/missi_t_qcom/out$
huyuxin@nj-builder-01:~/missi_t_qcom/out$ ls -lh build-missi_phone_cn.ninja
-rw-rw-r-- 1 huyuxin huyuxin 1.2G Sep 26 14:41 build-missi_phone_cn.ninja
huyuxin@nj-builder-01:~/missi_t_qcom/out$ ls -lh build-missi_phone_cn-package.ninja
-rw-rw-r-- 1 huyuxin huyuxin 43K Sep 26 10:08 build-missi_phone_cn-package.ninja
huyuxin@nj-builder-01:~/missi_t_qcom/out$ ls -lh soong/build.ninja
-rw-rw-r-- 1 huyuxin huyuxin 22G Sep 26 14:36 soong/build.ninja

```

On my host this file size is over 20 GB，My source code is from Qualcomm's distribution. The file may not be this large on the AOSP distribution.

Usually the memory on our developer's work host is not very rich, usually around 32GB.

Can we crop or split this file so that it can be easily compiled locally to verify some module issues? For example framework.jar services.jar Settings.apk etc.

(Our current workaround is to increase the size of the host's swap space, which results in a slow compilation process)

Thx ！

[Mauro Rossi]

I am also having issues, with smaller size of out/soong/build.ninja at about 8.5 Gbytes size,

is there a simple way to reduce the unnecessary overhead in out/soong/build.ninja ?

Thanks for any info

Mauro Rossi

android-x86 team

[palrat...@gmail.com]

I am getting same issue while compileing android 12.could you please help me out how to resolve this?

10:37:07 ************************************************************
10:37:07 You are building on a machine with 15.4GB of RAM
10:37:07
10:37:07 The minimum required amount of free memory is around 16GB,
10:37:07 and even with that, some configurations may not work.
10:37:07
10:37:07 If you run into segfaults or other errors, try reducing your
10:37:07 -j value.
10:37:07 ************************************************************
build/make/core/soong_config.mk:195: warning: BOARD_PLAT_PUBLIC_SEPOLICY_DIR has
 been deprecated. Use SYSTEM_EXT_PUBLIC_SEPOLICY_DIRS instead.
build/make/core/soong_config.mk:196: warning: BOARD_PLAT_PRIVATE_SEPOLICY_DIR ha
s been deprecated. Use SYSTEM_EXT_PRIVATE_SEPOLICY_DIRS instead.
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=12
TARGET_PRODUCT=aosp_car_x86_64
TARGET_BUILD_VARIANT=userdebug
TARGET_BUILD_TYPE=release
TARGET_ARCH=x86_64
TARGET_ARCH_VARIANT=x86_64
TARGET_2ND_ARCH=x86
TARGET_2ND_ARCH_VARIANT=x86_64
HOST_ARCH=x86_64
HOST_2ND_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-5.15.0-69-generic-x86_64-Ubuntu-20.04.6-LTS
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
BUILD_ID=SQ3A.220605.009.B1
OUT_DIR=out
PRODUCT_SOONG_NAMESPACES=device/generic/goldfish device/generic/goldfish-opengl
hardware/google/camera hardware/google/camera/devices/EmulatedCamera
============================================
[100% 250/250] out/soong/.bootstrap/bin/soong_build out/soong/build.ninja
FAILED: out/soong/build.ninja
cd "$(dirname "out/soong/.bootstrap/bin/soong_build")" && BUILDER="$PWD/$(basena
me "out/soong/.bootstrap/bin/soong_build")" && cd / && env -i "$BUILDER"     --t
op "$TOP"     --out "out/soong"     -n "out"     -d "out/soong/build.ninja.d"  
  -t -l out/.module_paths/Android.bp.list -globFile out/soong/.bootstrap/build-g
lobs.ninja -o out/soong/build.ninja --available_env out/soong/soong.environment.
available --used_env out/soong/soong.environment.used Android.bp
Killed
10:38:49 soong bootstrap failed with: exit status 1

#### failed to build some targets (01:46 (mm:ss)) ####

[Mohammad Hossein Heydarchi]

Try to run it with less thread:
make -j4

make -j3

make -j2

make -j1

If the PC does not have enough ram, the build system after some time stops running.

[Yuxin Hu]

Received a related Change https://android-review.googlesource.com/c/platform/build/blueprint/+/2449271, 

and the file size of out/soong/build.ninja was reduced to about 3GB when compiling a single module. 

But at the stage of generating out/soong/build.ninja, the reduction of host memory consumption is not obvious