How to keep separate AOSP source tree for our own projects

[Travis James]

Good afternoon!  I am working with a team of developers on an embedded Android project were we are adding system services and daemons to the AOSP build tree.  We would like to come up with a method of keeping our source in Git in sync with updates to the AOSP tree for Android 5.1.1, be we are having trouble structuring our repositories to work fluidly with Google's "repo" tools.

For example, we cannot put parts of AOSP in our own repository and have them show up due to the .gitignore structure.

Our plan was to have our developers use isolated VMWare VM's configured with sync'ed local copies of the AOSP base and check in changes in a way that can be merged later.

What are the best practices around continuous integration and source code control when extending the AOSP code base and integrating it with our own code?  Can someone point me to some documentation or other blog posts or materials?

Thanks in advance.

Travis

[Dave Smith]

Hello Travis -

There are many paths here, with little in the way of an "official" solution. I can share with you the code organization I typically use, and what we teach others to use. I believe it to be the simplest approach:

If you are building an AOSP-based product, you should aim to avoid overlapping with any existing repositories. The more code you change in existing upstream repos, the more merge issues you'll have down the line. Instead, aim to keep all your custom daemons/services in separate repositories that you pull in addition to the upstream. Repo makes this quite easy. If you look at a repo manifest file (e.g. https://android.googlesource.com/platform/manifest/+/master/default.xml), you will see it's just a collection of separate Git repositories that are pulled together at build time.

A convenient place to store all your changes is in a single repository that you clone into the device/ directory, since you will have to create one of those for your custom product definition anyway. But as long as you keep the changes in separate repos, later merges should go off without a hitch. Here's a simple recipe that should at least illustrate the concept:

1. Create a fresh repository for your base product definition (e.g. https://android.googlesource.com/device/lge/bullhead/+/master, which is for the Nexus 5X)

2. Create additional repositories (or subdirectories within the last repository) where your services/daemons/HALs will go.

3. Fork the repo manifest repository (e.g. https://android.googlesource.com/platform/manifest/+/master/default.xml) and add your custom repositories to the packages list.

At this point, the only part of AOSP your are overlapping is the repo manifest, so merging in new versions from upstream just means merging the upstream manifest (single XML file) with your modified version. You avoid forking and putting any of your actual code inside existing AOSP repositories, so you don't have to merge them later.

When you sync your entire source tree (onto the build machine or a dev machine) with repo sync, you would do this against your forked manifest URL instead of https://android.googlesource.com/platform/manifest.

As a final note, my company trains teams like yours on working with Android at the embedded level. If you would like to learn more about all of this, we would be happy to work with you in more detail: https://newcircle.com/instructor-led-training/android-internals

Hope that helps!

Cheers,

--

Dave Smith, PE

@devunwired

[Arne-Christian Blystad]

Hey,

My company has a local clone of AOSP sources, based on following the documentation here: https://source.android.com/source/downloading.html ( Using a local mirror ).

With a local mirror setup, you should create a development branch. We've gone for "masterXX" where XX is short for the code name. E.g. JB = Jelly bean, LP = Lollipop, MM=Marshmellow and so on. 

Then update your manifest, and set this as the default branch:

  <default revision="masterLP"

           remote="REMOTE_NAME"

           sync-j="4" />

(Remote name should be set based on the <remote> XML tag, just above. Probably origin or aosp for you. We have it to be our company name).

With that setup, you should be able to do changes to AOSP.

To update your git servers AOSP sources, ssh to the git server and run "repo sync" (see the "Using a Local Mirror" link again ). We have a Jenkins job that does this periodically.

When we want to upgrade our branch to the latest AOSP release, we create a new branch, merge in AOSP changes, builds it, tests it, and then eventually merge it into our master branch. E.g:

repo forall -c "git checkout -b upgrade_5.1.1_r30"

repo forall -p -c "git merge --no-ff --no-edit android-5.1.1_r30"

# this will probably have some merge conflicts, address those...

# with that done, either push it to your gerrit instance for review, or:

repo forall -p -c "git push origin HEAD:masterLP && git branch -D android-5.1.1_r30"

Regards,

Arne-Christian.

[Stuart Monteith]

Another option you can consider, is to use local manifests.
Here you can create the directory ".repo/local_manifests". You can place xml files in the repo manifest format to pull in additional git projects. See the repo help - it is quite useful.
You should keep the local_manifests in a own git repository to manage them.

You can find the manifest format document in your Android repo checkout here: .repo/repo/docs/manifest-format.txt