Mirroring AOSP

[Jean-Baptiste Queru]

I've added a special manifest that lists all the projects currently in
AOSP (or at least all the ones I currently know about).

Use this command line to init a mirror (notice the different manifest URI):
"repo init --mirror -u https://android.googlesource.com/mirror/manifest"

Thanks to the way the manifests are now written, the mirror is
immediately fully functional. Once you've synced a mirror, you can
initialize regular clients from it, via the local filesystem, or ssh,
or a git server if you have that running:

"repo init -u <path_to_mirror>/platform/manifest.git -b gingerbread"

Hopefully people will find this useful.

If you see that some projects seem to be missing from the mirror,
please let me know, I've tried hard to catch them all but it's
possible that I forgot something.

As a note, once you've initialized a client from a mirror, you can
then make it point to the live server by re-running a repo init
command. This is a great way to e.g. store a mirror on a portable hard
drive or on a DVD, use it to initialize multiple machines, and then
have those machines sync future changes from the AOSP server instead.

This is how things could look like overall:

mkdir /usr/local/android-mirror
cd /usr/local/android-mirror
repo init --mirror -u https://android.googlesource.com/mirror/manifest
repo sync
mkdir /usr/local/android-2.3.7
cd /usr/local/android-2.3.7
repo init -u /usr/local/mirror/platform/manifest.git -b android-2.3.7_r1
repo sync
repo init -u https://android.googlesource.com/platform/manifest
repo sync

JBQ

--
Jean-Baptiste M. "JBQ" Queru
Software Engineer, Android Open-Source Project, Google.

Questions sent directly to me that have no reason for being private
will likely get ignored or forwarded to a public forum with no further
warning.

[Carlos Silva]

On Thu, Oct 20, 2011 at 21:11, Jean-Baptiste Queru <j...@android.com> wrote:

I've added a special manifest that lists all the projects currently in
AOSP (or at least all the ones I currently know about).

Use this command line to init a mirror (notice the different manifest URI):
"repo init --mirror -u https://android.googlesource.com/mirror/manifest"

Thanks JBQ. Jus synced and eveything looks cool.

Just one slight problem with the openvpn project. After sync I run a git bundle create but it fails on openvpn.

"error: revision master in platform/external/openvpn not found"

Some problem on the openvpn project or is this related to anything else "new" ?

[Jean-Baptiste Queru]

Oh, I must have forgotten to create a master branch in that project,
and since it's not referenced by the master manifest that could easily
slip unnoticed. Sorry about that.

I just created a master branch in that project, so that one should be
fixed. It's very possible however that I'll have forgotten to create
master branches in other projects, possibly in all the projects that
were used in eclair but not in later releases. I'll try to spot those,
and if you find some on your side, please let me know.

JBQ

> --
> You received this message because you are subscribed to the "Android
> Building" mailing list.
> To post to this group, send email to android-...@googlegroups.com
> To unsubscribe from this group, send email to
> android-buildi...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/android-building?hl=en

[Jean-Baptiste Queru]

I just fixed 4 more. There could be more.

Eventually, though, we'll end up with some projects in there that
don't have master branches (e.g. the kernel sources), so you might
also want to investigate what it'd take for your script to be able to
handle that case as well.

Thanks,
JBQ

[Carlos Silva]

On Thu, Oct 20, 2011 at 23:25, Jean-Baptiste Queru <j...@android.com> wrote:

I just fixed 4 more. There could be more.

Eventually, though, we'll end up with some projects in there that
don't have master branches (e.g. the kernel sources), so you might
also want to investigate what it'd take for your script to be able to
handle that case as well.

Yeah, I'll take a closer look when you push the kernels out. Right now everything is working (just synced) except for device/htc/dream-sapphire.git that makes git bundle segfault.

Other than that, everything looks ok. (232 projects mirrored)

(For anyone that wants: http://android.git.r3pek.org) (has gitweb, and is syncable just like the official one is, just follow the same instructions changing the hostname on the repo init command)

[Jean-Baptiste Queru]

Indeed, something seems wrong with that project, I'm seeing an error
too when I try to create a bundle out of a freshly cloned
dream-sapphire project.

I'm investigating with our git gurus.

JBQ

[Jean-Baptiste Queru]

What version of git are you using? I can reproduce here with git version 1.7.3.1

JBQ

[Carlos Silva]

On Fri, Oct 21, 2011 at 00:21, Jean-Baptiste Queru <j...@android.com> wrote:

What version of git are you using? I can reproduce here with git version 1.7.3.1

A couple above:

git version 1.7.5.4

[Leif Andersen]

Thanks, I'm sure that once all of the kinks are worked out this will be very useful.

Will the repo init command still need to be run from time to time as more public git repos are added to the server, and thus the manifest?

Thank you in advance.

~Leif Andersen

[Jean-Baptiste Queru]

You should only need to run init when you want to change something on
your side (point to another server or another manifest). In all other
cases, repo sync should be enough.

JBQ

--

[bomjose]

Just FYI.
Seeing this error.
error: The requested URL returned error: 403 while accessing
https://android.googlesource.com//platform/external/xmlwriter/info/refs

> warning.- Hide quoted text -
>
> - Show quoted text -

[bomjose]

Another error.

Fetching projects: 96% (224/233) Receiving objects: 30%
(2919/9618), 94.91 MiException in thread Thread-228:4654), 9.38 MiB |
211 KiB/s
Traceback (most recent call last):
File "/usr/lib/python2.6/threading.py", line 532, in
__bootstrap_inner
self.run()
File "/usr/lib/python2.6/threading.py", line 484, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/gerrit2/repositories/aosp/.repo/repo/subcmds/sync.py",
line 182, in _FetchHelper
success = project.Sync_NetworkHalf(quiet=opt.quiet)
File "/home/gerrit2/repositories/aosp/.repo/repo/project.py", line
926, in Sync_NetworkHalf
if alt_dir is None and self._ApplyCloneBundle(initial=is_new,
quiet=quiet):
File "/home/gerrit2/repositories/aosp/.repo/repo/project.py", line
1444, in _ApplyCloneBundle
exist_dst = self._FetchBundle(bundle_url, bundle_tmp, bundle_dst,
quiet)
File "/home/gerrit2/repositories/aosp/.repo/repo/project.py", line
1508, in _FetchBundle
raise DownloadError('%s: %s ' % (req.get_host(), str(e)))
DownloadError: android.googlesource.com: <urlopen error [Errno 110]
Connection timed out>

On Oct 21, 8:27 am, Jean-Baptiste Queru <j...@android.com> wrote: