[Android Q] Question about new constraint for build system

[Niklaus Lin (林洲銓)]

Hi Android build team,

 

This is Android team from Mediatek.

We have a question that needs your advice and suggestion

 

According to https://android.googlesource.com/platform/build/+/master/Changes.md#PATH_Tools,

There is a new constraint for build system in order to ensure the build results are reproducible across different machines.

 

Problem occurs when we try to combine our build flow with Android source code.

When we compile our proprietary modules using ARM GNU toolchains or some unix tool, we meet compiling errors because of the constraint mentioned above.

The error log shows ‘Disallowed PATH Tool “xxx” used …’ , and these tools are make, cache, perl,… , used by ARM GNU toolchains or shell command in build flow.

We found that these tools were disallowed because they were fallen into Forbidden or Missing case, defined in build/soong/ui/build/paths/config.go

the detail analysis can be referenced by attachment : 3rd tools error analysis.docx

 

Is it possible Google create a customization mechanism for vendor to add the proprietary build tool in Android source? e.g. add a new path variable (/vendor/prebuilts/build-tools) to vendor folder then vendor can add vendor build tool in it.

and these vendor build tool can be excepted in constraint rule.

 

for this question, we also have some discussion in this buganizer issue.

https://partnerissuetracker.corp.google.com/u/1/issues/123787813

 

 

BRs,

Feng Lin

************* MEDIATEK Confidentiality Notice ********************
The information contained in this e-mail message (including any 
attachments) may be confidential, proprietary, privileged, or otherwise
exempt from disclosure under applicable laws. It is intended to be 
conveyed only to the designated recipient(s). Any use, dissemination, 
distribution, printing, retaining or copying of this e-mail (including its 
attachments) by unintended recipient(s) is strictly prohibited and may 
be unlawful. If you are not an intended recipient of this e-mail, or believe 
that you have received this e-mail in error, please notify the sender 
immediately (by replying to this e-mail), delete any and all copies of 
this e-mail (including any attachments) from your system, and do not
disclose the content of this e-mail to any other person. Thank you!

[Dan Willemsen]

It's easy to use prebuilt tools from the source directory (or built tools) from your own rules that need them, either just refer to them by name, or you may be able to set up your own directory to be added to PATH inside a rule:

https://android.googlesource.com/device/ti/beagle-x15/+/7001c965a5d6792c04fd94838161cd6116c605cf/uboot.mk#151

https://android.googlesource.com/device/ti/beagle-x15/+/7001c965a5d6792c04fd94838161cd6116c605cf/hostcc#

We're not going to add an extension that allows modifying the environment of every build command -- that causes problems with the attempts to build common system images that aren't influenced by the device. The common build commands should not need anything except our standard tools -- if you've modified them, it should be easy enough to also add the dependency to the new tools as well.

On tools:

• We've deprecated GCC during Android builds, we recommend clang for compilation.
• We do have prebuilts of `make` and some other common tools in prebuilts/build-tools/linux-x86/bin
• ccache is not something that we provide anymore -- we've found it's generally better to use incremental builds, as ccache can slow down your builds with extra I/O, and incremental builds should have relatively few cache hits (unless you're sharing the ccache dir; which runs into even more I/O issues).
• Perl is not something supported at Google, so we don't provide that either -- we do have built in support for python and the ability to use python packages from the source tree (or checked in as a prebuilt, when using embedded_launcher, it's a self-contained binary): https://ci.android.com/builds/submitted/5527387/linux/latest/python.html#python_binary_host

- Dan

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to android-...@googlegroups.com
To unsubscribe from this group, send email to
android-buildi...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-buildi...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Niklaus Lin (林洲銓)]

Hi Dan,

 

Thanks for your help.

We will try this suggestion~

 

BRs,

Feng Lin

[nikla...@mediatek.com]

Hi Dan,

Thanks for your suggestion,

we will try this.

Dan Willemsen於 2019年5月3日星期五 UTC+8下午1時34分50秒寫道：

android-...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.

To unsubscribe from this group and stop receiving emails from it, send an email to android-...@googlegroups.com.

[Matt Rogers]

Dan,

  Related from migrating from Android 8 compiles using documentiation for the now depreciated instructions for jack-server

and your comment below about I/O slowdowns,

On Android 8 compiles I have adjusted jack-server to use available 128 gb of my 288 gb available memory on my dell r810 server ( 64 core ) and have resulted in sucessfull compiles in less than 15 minutes,

1. using ssd source drive

2. using ssd output drive for .ccache and a global OUT_DIR for compiled product

3. using custom settings in ~/.jack-server/config_properties

4. using custom tiered compilations settings

So is there documention related to android 9 compile insturctions for cpu and memory and such for me to evaluate and test ?

there was a few threads related to making adjustments,

In essance, what is the list of all the variables the brunch or whatever command used can be adjusted for performance ?

for instance use_ccache=true still works as does

export CCACHE_DIR=/ssd_home/username/.ccache
export OUT_DIR_COMMON_BASE=/ssd_home/username/OUT_DIR

is there a file or list with all the options available to me ?

Oh by the way I translated all the binaries u recommend in AOSP for Ubuntu to Fedora29 server and fedora30 server

thanks,

Matt

github.com/tripLr

github.com/tirpLr-dev

 On Thursday, May 2, 2019 at 10:34:50 PM UTC-7, Dan Willemsen wrote:

I

On tools:

• We've deprecated GCC during Android builds, we recommend clang for compilation.
•