AOSP downloads just got much bigger!

[chris simmonds]

I just did a repo sync on android-14.0.0_r29 and was surprised that it downloaded almost 900 GB of data! Why?

Answer: before r29, most of the binary blobs in the manifest were marked with clone-depth="1" so you only got the most recent version. Now you get all versions. For example .repo/projects/device/google/bluejay-kernel.git/ has gone from 0.5 GB to 14 GB

Solution: *always* set the clone depth to 1 when you repo init:

$ repo init --depth=1 -u .....

Cheers,

Chris Simmonds, https://aospandaaos.github.io/

[Michael Swartout]

You figured it out, good for you. "--partial-clone" can also be used instead of "--depth=1". We have to manually do lunch now though, no more list to choose from.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to android-...@googlegroups.com
To unsubscribe from this group, send email to
android-buildi...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-buildi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/1e778aa7-f6b3-4e32-aa18-03c60fdea8b2n%40googlegroups.com.

[John W. Blue]

Was there a notification that lunch is now manual?  If so where and can you provide insight as to why?  Mostly just curious ..

 

John

To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/CAEq4fyAeVf1e8xHxH1xm%2BR3r08K0QRphdTa-AJVtBG1g32Wm-A%40mail.gmail.com.

[enh]

"works for me" in AOSP master, so maybe just a bug that it was broken
in the particular snapshot you're using?

note that there's also tab-completion for lunch targets. i personally
find that super useful.

On Tue, May 14, 2024 at 11:28 AM 'John W. Blue' via Android Building

> To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/f1df71694b07446084215c99dbf2ee01%40mail.rrcic.com.

[Mathieu Fluhr]

This issue was triggered by a commit when 14 QPR2 was released: https://android.googlesource.com/platform/manifest/+/88132d440afaa83857b85c0ca49ac3f463a363c9

The main branch remains unaffected, so checking out main still gives reasonable sizes.

Kind Regards,

Mathieu

[enh]

sorry, i meant that "lunch showing a menu" works for me... (i gave up
looking at how much disk space is used by AOSP years ago!)

> To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/caf3ae68-0be0-41b2-8863-474c1a97b065n%40googlegroups.com.

[Dan Willemsen]

I'd still recommend using `--partial-clone` in `repo init`, the downloads will be significantly smaller. But it does make more git operations talk to the server, in order to download historical objects that you didn't download initially.

One tradeoff, which is slightly larger and takes more time to download, but doesn't trigger network operations nearly as often is to use `--partial-clone-exclude=platform/frameworks/base --clone-filter=blob:limit=10M`. We use that in addition to `REPO_ALLOW_SHALLOW=0` to switch from clone-depth to partial-clones. I've filed a bug against source.android.com to document that in the instructions, but it may be a while.

Was there a notification that lunch is now manual?  If so where and can you provide insight as to why?  Mostly just curious ..

I think it's more of a bug that the common lunch choice menu isn't working. But in any case, they've never been comprehensive, and it's always been faster to specify the combination you want on the command line.

- Dan

On Tue, May 14, 2024 at 11:53 AM Mathieu Fluhr <mathie...@gmail.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/caf3ae68-0be0-41b2-8863-474c1a97b065n%40googlegroups.com.

[Mathieu Fluhr]

I think this is one of the first thing that I tried when I encountered this problem, but I have to admit I did not checked the size improvement, since the ‘repo sync’ execution time from a local mirror was still above 25min.

On a general note, I would expect calling repo init / repo sync without any additional arguments to give me a standard source tree with “reasonable” size. I always considered extra arguments as “optimizations”, and not something to always add (…. And here I use “I” since this is my personal opinion 😉)

You received this message because you are subscribed to a topic in the Google Groups "Android Building" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/android-building/q-VkE5Oc9ec/unsubscribe.
To unsubscribe from this group and all its topics, send an email to android-buildi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/CALQgHdnFhTyuCFb35u653K7UDpoFA7BbTgD1N4U1e4mj-HPHqw%40mail.gmail.com.

[Michael Swartout]

I did not read one. I just tried to lunch and the menu came up empty. So I enter this info: lunch aosp_<product_name>-<release>-<build_variant>. It allowed myself to make the os or individual modules.

To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/f1df71694b07446084215c99dbf2ee01%40mail.rrcic.com.

[Mathieu Fluhr]

Just FYI, the download size issue seems to be "fixed" with this commit in the android14-qpr2-release branch:

https://android.googlesource.com/platform/manifest/+/1ee30879e100bc979595739dc802d5e9590bead5

Also the new 14 release tags that have been prepared for the Pixel 8a, i.e. android-14.0.0_r38 to android-14.0.0_r45 are not affected anymore.

To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/CAEq4fyCovrmgYMhJ38Tu6X3SJ9%3D%2BYk5EL8PaOF8CcpVA7jmk9g%40mail.gmail.com.

[chris simmonds]

Hi Mathieu,

Good to know that the problem has gone away

BTW, anyone know why the Build ID has jumped from the new trunk stable Axxx.xxxxxx.xxx format back to the old style Uxxx.xxxxxx.xxx in r38 on wards? Now I'm really confused!

Cheers,

Chris

To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/CAPyJwA-Un5%3DvCkmtjRU%3DfypANN2%2Bhp_ALXczE%2Bx8Fze3kDVfMw%40mail.gmail.com.

[Mathieu Fluhr]

The r38+ builds are specially dedicated for the 8a, and looking at the git log, it looks like they are based indeed on an older release.

I _guess_ that the new trunk stable builds for the 8a will come with the June release cycle, together with other pixel devices.... but this is a rough guess, since I am not working for Google :)