Creating a private repo server for Android

6,573 views
Skip to first unread message

Michael

unread,
Mar 30, 2011, 2:39:28 PM3/30/11
to Repo and Gerrit Discussion
Hi - It was suggest by Jean-Baptiste Queru that this group may be
better suited to helping me with my repo questions, so hopefully I've
found a home for this problem. The discussion started with this
thread:

http://groups.google.com/group/android-building/browse_thread/thread/ff93c2d24ca035ae

But basically my question can be distilled down to the last question
by asking if there is a preferred way to set up a "private" repo
server, so that people within a company can pull/push changes to that
server without everyone needing to pull from the public Android
server? The idea is for our company to host a "master" server that
synchronizes with Google's main servers, but all of our other
development machines on the network pull/push from that server.

Thanks for any help you can provide.

Michael

Magnus Bäck

unread,
Mar 31, 2011, 2:30:59 AM3/31/11
to Repo and Gerrit Discussion
On Wednesday, March 30, 2011 at 20:39 CEST,
Michael <mike...@gmail.com> wrote:

Set up a Git server using whichever Git hosting solution you like.
I think Gerrit is an excellent choice (regardless of whether you're
interested in using the code review feature), but you could just go
with any regular machine that users can access via SSH, or another
hosting solution like Gitolite.

Periodically, sync the code from AOSP and push it to your server. You
should never ever allow your users to directly modify the AOSP branches.
Instead, use different namespaces for upstream and your own development.
For example, push all AOSP branches to aosp/* and use the root namespace
for your own branches (whose origin of course is one of the aosp/*
branches). This should sync a workspace looking at the master branch
of AOSP and push everything to your server (where all the gits must
exist):

repo init --mirror -u git://android.git.kernel.org/platform/manifest
repo sync
repo forall -c 'git push ssh://yourserver.example.com/$REPO_PROJECT \
"refs/heads/*:refs/heads/aosp/*"'

Until you're comfortable with the forall and push commands I would
strongly recommend that you try each command on a single git first
so you won't wreck havoc.

Again, create local branches based off of the AOSP branches for your
developers to work on. The first git to branch would be the manifest
git, and you'll want to adjust the fetch and review URLs listed in the
<remote> tag at the top of the file so that they point to your server.
When you want to update your local branches from AOSP, merge the
manifest itself and then each git. Something like this -- run from a
workspace looking at your code -- should work:

repo forall -c 'git merge origin/aosp/master'
repo forall -c 'git push ssh://yourserver.example.com/$REPO_PROJECT \
HEAD:refs/heads/master'

(This examples assumes that all gits have a master branch that you
should work from, which as JBQ said in the original thread in
android-building isn't true.)

No, this isn't completely trivial. Expect to spend time on figuring
this out and getting it working. This question is not limited to the
pure IT issues surrounding the setup of a server for hosting Git --
you need to figure out how your company should manage your source code,
which AOSP update strategy to use, and so on. If you want to keep your
code up to date with AOSP and reduce the maintenance cost for new
releases you need to think about how you make changes to the platform
(you might not need to change the platform at all). Some types of
changes are more expensive than others, but if you contribute changes
back to AOSP your maintenance cost will decrease over time.

--
Magnus B�ck Opinions are my own and do not necessarily
SW Configuration Manager represent the ones of my employer, etc.
Sony Ericsson

Nasser Grainawi

unread,
Mar 31, 2011, 11:47:52 AM3/31/11
to Magnus Bäck, Repo and Gerrit Discussion

That's an excellent summary Magnus.

Michael, I would just like to add that there are several of us on this list that have done this and are pretty willing to help out with any questions you might have (especially if you're using Gerrit). Depending on the size of your developer group it could be days, weeks, or months until you have the infrastructure in place, necessary training completed, and are up to full speed with your development process.

Another thing to consider (if you haven't already) that's tangentially related to Gerrit is some type of Continuous Integration solution. Gerrit makes this easier with a pre-setup "Verified" category that you can use on each change to indicate you've tested the change and it works. It's a reasonably common practice to give an automated system permission to set that category as a method to ensure code being merged to the branch works.

Nasser

>
> --
> Magnus Bäck Opinions are my own and do not necessarily


> SW Configuration Manager represent the ones of my employer, etc.
> Sony Ericsson
>

> --
> To unsubscribe, email repo-discuss...@googlegroups.com
> More info at http://groups.google.com/group/repo-discuss?hl=en

--
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum

Michael

unread,
Mar 31, 2011, 2:55:57 PM3/31/11
to Repo and Gerrit Discussion
Thank you both for your information and your help. Magnus, thank you
for taking the time to write such a nice explanation -- it was
appreciated, and Nasser thank you for your kind offer. With your
input, along with some other information I have found, I think I have
come up with a decent strategy for now.

Michael

Michael

unread,
Apr 1, 2011, 9:42:24 AM4/1/11
to Repo and Gerrit Discussion
I have run into a small snag, hopefully someone can shed some light on
it; I'm probably missing something. Right now, I'm just experimenting;
this approach may not be the one I use. As a first step, I have cloned
the public git repo doing something like this:

repo init --mirror -u git://android.git.kernel.org/platform/manifest.git

From another machine, I run a command like this:
modified-repo init -u ssh://g...@192.168.2.36/home/git/android/platform/manifest.git
-b froyo-plus-aosp

The "modified-repo" script has the REPO_URL changed to:
REPO_URL='ssh://g...@192.168.2.36/home/git/android/tools/repo.git'

Which succeeds, so I then proceed to modify the default.xml (I will be
branching and modifying this file if you use this method, but until
then I am just making the changes by hand). I modify entries in the
file, like the <remote fetch="ssh://g...@192.168.2.36/" ... /> element,
and the prefixing the "name" attributes in the "project" element with
"/home/git/android/", so entries look something like this:

<project path="device/common" name="/home/git/android/device/common" /
>
<project path="device/htc/common" name="/home/git/android/device/htc/
common" />
<project path="device/htc/dream" name="/home/git/android/device/htc/
dream" />
<project path="device/htc/dream-sapphire" name="/home/git/android/
device/htc/dream-sapphire" />
<project path="device/htc/passion" name="/home/git/android/device/htc/
passion" />
<project path="device/htc/passion-common" name="/home/git/android/
device/htc/passion-common" />
<project path="device/htc/sapphire" name="/home/git/android/device/htc/
sapphire" />
<project path="device/sample" name="/home/git/android/device/sample" /
>

I then do a repo sync using my modified script, and everything seems
to sync correctly until it hits the "device/htc/dream" project when it
reports this error:

Initializing project /home/git/android/device/htc/dream ...
fatal: '//home/git/android/device/htc/dream.git' does not appear to be
a git repository
fatal: The remote end hung up unexpectedly
error: Cannot fetch /home/git/android/device/htc/dream

It did sync several other projects successfully before halting on this
one. Does anyone have any ideas why this would fail?

Thanks,
Michael

Michael

unread,
Apr 1, 2011, 10:35:56 AM4/1/11
to Repo and Gerrit Discussion
OK, my last question seems silly to me now. The reason it says the git
repo doesn't exist, is because the repo doesn't actually exist on the
"master" branch. So, do I need to do a separate clone for every
branch?

repo init --mirror -u git://android.git.kernel.org/platform/manifest.git
repo sync

(create a new dir)
-b froyo
repo sync

(create a new dir)
-b froyo-plus-aosp
repo sync
...

Having simply a clone of the "master" branch doesn't seem sufficient,
especially if the projects have been deleted or moved around...

Michael

On Apr 1, 9:42 am, Michael <mikecg...@gmail.com> wrote:
> I have run into a small snag, hopefully someone can shed some light on
> it; I'm probably missing something. Right now, I'm just experimenting;
> this approach may not be the one I use. As a first step, I have cloned
> the public git repo doing something like this:
>
> repo init --mirror -u git://android.git.kernel.org/platform/manifest.git
>
> From another machine, I run a command like this:
> modified-repo init -u ssh://...@192.168.2.36/home/git/android/platform/manifest.git
> -b froyo-plus-aosp
>
> The "modified-repo" script has the REPO_URL changed to:
> REPO_URL='ssh://...@192.168.2.36/home/git/android/tools/repo.git'
>
> Which succeeds, so I then proceed to modify the default.xml (I will be
> branching and modifying this file if you use this method, but until
> then I am just making the changes by hand). I modify entries in the
> file, like the <remote fetch="ssh://...@192.168.2.36/" ... /> element,

Jean-Baptiste Queru

unread,
Apr 1, 2011, 10:39:54 AM4/1/11
to Repo and Gerrit Discussion
Indeed.

You might find this useful:

ssh -p 29418 review.source.android.com gerrit ls-projects

(You'll only need the projects under device/ and platform/ for all
recent versions of Android).

JBQ

--
Jean-Baptiste M. "JBQ" Queru
Software Engineer, Android Open-Source Project, Google.

Questions sent directly to me that have no reason for being private
will likely get ignored or forwarded to a public forum with no further
warning.

Magnus Bäck

unread,
Apr 1, 2011, 10:44:45 AM4/1/11
to Repo and Gerrit Discussion
On Friday, April 01, 2011 at 16:35 CEST,
Michael <mike...@gmail.com> wrote:

> OK, my last question seems silly to me now. The reason it says the git
> repo doesn't exist, is because the repo doesn't actually exist on the
> "master" branch. So, do I need to do a separate clone for every
> branch?
>
> repo init --mirror -u git://android.git.kernel.org/platform/manifest.git
> repo sync
>
> (create a new dir)
> repo init --mirror -u git://android.git.kernel.org/platform/manifest.git
> -b froyo
> repo sync
>
> (create a new dir)
> repo init --mirror -u git://android.git.kernel.org/platform/manifest.git
> -b froyo-plus-aosp
> repo sync
> ...
>
> Having simply a clone of the "master" branch doesn't seem sufficient,
> especially if the projects have been deleted or moved around...

Correct; "repo sync" downloads all branches in all gits, but only gits
currently listed in the manifest you're looking at.

That said, you don't need to initialize completely different workspaces
for each branch. You should also be able to do just this:

repo sync
repo forall -c 'git push ...'

repo init -b froyo
repo sync
repo forall -c 'git push ...'

repo init -b froyo-plus-aosp
repo sync
repo forall -c 'git push ...'

--
Magnus B�ck Opinions are my own and do not necessarily

Magnus Bäck

unread,
Apr 1, 2011, 10:52:25 AM4/1/11
to Repo and Gerrit Discussion
On Friday, April 01, 2011 at 15:42 CEST,
Michael <mike...@gmail.com> wrote:

> I have run into a small snag, hopefully someone can shed some light on
> it; I'm probably missing something. Right now, I'm just experimenting;
> this approach may not be the one I use. As a first step, I have cloned
> the public git repo doing something like this:
>
> repo init --mirror -u git://android.git.kernel.org/platform/manifest.git
>
> From another machine, I run a command like this:
> modified-repo init -u ssh://g...@192.168.2.36/home/git/android/platform/manifest.git
> -b froyo-plus-aosp
>
> The "modified-repo" script has the REPO_URL changed to:
> REPO_URL='ssh://g...@192.168.2.36/home/git/android/tools/repo.git'

You'd be better off just using the --repo-url option of "repo init".

Even better, use insteadOf in your .gitconfig (and have all your users
do this):

[url "ssh://g...@192.168.2.36/home/git/android/"]
insteadOf = git://android.git.kernel.org/

Then all git clone/fetch/pull/etc operations will perform a pattern
replacement and replace the original URL with your local one.

[...]

--
Magnus B�ck Opinions are my own and do not necessarily

Nasser Grainawi

unread,
Apr 1, 2011, 11:59:44 AM4/1/11
to Magnus Bäck, Repo and Gerrit Discussion
On Apr 1, 2011, at 8:52 AM, Magnus Bäck wrote:

> On Friday, April 01, 2011 at 15:42 CEST,
> Michael <mike...@gmail.com> wrote:
>
>> I have run into a small snag, hopefully someone can shed some light on
>> it; I'm probably missing something. Right now, I'm just experimenting;
>> this approach may not be the one I use. As a first step, I have cloned
>> the public git repo doing something like this:
>>
>> repo init --mirror -u git://android.git.kernel.org/platform/manifest.git
>>
>> From another machine, I run a command like this:
>> modified-repo init -u ssh://g...@192.168.2.36/home/git/android/platform/manifest.git
>> -b froyo-plus-aosp
>>
>> The "modified-repo" script has the REPO_URL changed to:
>> REPO_URL='ssh://g...@192.168.2.36/home/git/android/tools/repo.git'
>
> You'd be better off just using the --repo-url option of "repo init".
>
> Even better, use insteadOf in your .gitconfig (and have all your users
> do this):
>
> [url "ssh://g...@192.168.2.36/home/git/android/"]
> insteadOf = git://android.git.kernel.org/
>
> Then all git clone/fetch/pull/etc operations will perform a pattern
> replacement and replace the original URL with your local one.
>
> [...]

I'd stay away from the insteadOf if you're working at all directly with a.g.k.o sources.

Also, you shouldn't need to modify repo at all. We use the stock version of repo and just init to our internal manifest project. If you want to mirror repo for availability reasons (a.g.k.o sometimes has high load and will refuse connections).

For cloning we do what JBQ suggests and set up some cron jobs to sync the output of gerrit ls-projects (though we should teach ls-projects about perm-only projects so we don't mirror those).

Nasser

>
> --
> Magnus Bäck Opinions are my own and do not necessarily


> SW Configuration Manager represent the ones of my employer, etc.
> Sony Ericsson
>

Magnus Bäck

unread,
Apr 1, 2011, 1:15:15 PM4/1/11
to Repo and Gerrit Discussion
On Friday, April 01, 2011 at 17:59 CEST,
Nasser Grainawi <nas...@codeaurora.org> wrote:

> On Apr 1, 2011, at 8:52 AM, Magnus B�ck wrote:
>
> > You'd be better off just using the --repo-url option of "repo init".
> >
> > Even better, use insteadOf in your .gitconfig (and have all your
> > users do this):
> >
> > [url "ssh://g...@192.168.2.36/home/git/android/"]
> > insteadOf = git://android.git.kernel.org/
> >
> > Then all git clone/fetch/pull/etc operations will perform a pattern
> > replacement and replace the original URL with your local one.
>

> I'd stay away from the insteadOf if you're working at all directly
> with a.g.k.o sources.

Definitely -- there's no easy way to disable insteadOf without editing
$HOME/.gitconfig, so people working directly with a.g.k.o wouldn't want
to do this.

> Also, you shouldn't need to modify repo at all. We use the stock
> version of repo and just init to our internal manifest project. If you
> want to mirror repo for availability reasons (a.g.k.o sometimes has
> high load and will refuse connections).

Right. Many companies have firewalls that block port 9418, and then I
guess it's either --repo-url, insteadOf, or distribute a custom repo
binary.

> For cloning we do what JBQ suggests and set up some cron jobs to sync
> the output of gerrit ls-projects (though we should teach ls-projects
> about perm-only projects so we don't mirror those).

Yeah, that does sound better than syncing manifests and using
"repo forall" for iteration.

--
Magnus B�ck Opinions are my own and do not necessarily

Daniel DeFreez

unread,
Apr 4, 2011, 1:29:39 AM4/4/11
to Repo and Gerrit Discussion
This is a great thread! Thanks for the help. I've followed most of it,
but I'm having a little bit of trouble with the separate aosp
namespace. I'm relatively new to Git, so I apologize for missing what
is probably an obvious step.

When I try to clone a repo after pushing with the refspec "refs/heads/
*:refs/heads/aosp/*" I receive the warning "remote HEAD refers to
nonexistent ref, unable to checkout." I assume this is because the
HEAD ref points to master which has been moved to aosp/master. What's
the best way to deal with this?

Right now I'm keeping aosp in the root namespace and using a separate
area to keep the development branches in, then limiting the
permissions with Gitolite. That seems to work just fine, but I'd like
to understand the other way.

Thanks again, and happy Android building.

Daniel

On Mar 30, 11:30 pm, Magnus Bäck <magnus.b...@sonyericsson.com> wrote:
> On Wednesday, March 30, 2011 at 20:39 CEST,
>      Michael <mikecg...@gmail.com> wrote:
>
> > Hi - It was suggest by Jean-Baptiste Queru that this group may be
> > better suited to helping me with my repo questions, so hopefully I've
> > found a home for this problem. The discussion started with this
> > thread:
>
> >http://groups.google.com/group/android-building/browse_thread/thread/...
> Magnus B�ck                   Opinions are my own and do not necessarily

Magnus Bäck

unread,
Apr 4, 2011, 12:26:57 PM4/4/11
to Repo and Gerrit Discussion
On Monday, April 04, 2011 at 07:29 CEST,
Daniel DeFreez <dan...@defreez.com> wrote:

> This is a great thread! Thanks for the help. I've followed most of it,
> but I'm having a little bit of trouble with the separate aosp
> namespace. I'm relatively new to Git, so I apologize for missing what
> is probably an obvious step.
>
> When I try to clone a repo after pushing with the refspec "refs/heads/
> *:refs/heads/aosp/*" I receive the warning "remote HEAD refers to
> nonexistent ref, unable to checkout." I assume this is because the
> HEAD ref points to master which has been moved to aosp/master.

Correct.

> What's the best way to deal with this?

You shouldn't make any changes to any of the upstream branches, so it's
a good thing that Git doesn't check out one of the aosp/* branches for
you. You probably want to create a master branch yourself, based on
aosp/master, and use that as the branch you work on.

If you create such a branch on the server Git will create a local
branch (master) that tracks the upstream branch (origin/master) when
you clone, and you won't get that warning anymore. Of course, upstream
in this case means the master branch on your server, which in turn has
AOSP's master branch as its upstream.

> Right now I'm keeping aosp in the root namespace and using a
> separate area to keep the development branches in, then limiting
> the permissions with Gitolite. That seems to work just fine, but
> I'd like to understand the other way.

You could do things your way too, of course -- technically it doesn't
matter. However, putting the AOSP branches in the root of the namespace
means that you assume you'll never have more than one upstream source
(unless you want to be inconsistent).

If it's a common use case for your developers to clone gits by hand
rather than use Repo it also makes more sense to give them a master
branch they can actually work with rather than one based on a branch
to which they must never ever push.

--
Magnus Bäck Opinions are my own and do not necessarily

Syntax Error

unread,
Apr 4, 2011, 1:03:33 PM4/4/11
to Repo and Gerrit Discussion
Hi Magnus,

I am sorry to bother you mate. But I can not get it to work. I have been trying the whole day without success.
Is there maybe you could show me a step by step guide on how I can create a mirror to our server. And then pushing the changes to our gerrit server so that we can have a versioning check system on our local mirror?
Also what about updating the local mirror to latest google sources? How would that go exactly?

Hope to hear from you soon.

Regards

Magnus Bäck

unread,
Apr 4, 2011, 1:50:32 PM4/4/11
to Repo and Gerrit Discussion
On Monday, April 04, 2011 at 19:03 CEST,
Syntax Error <impor...@gmail.com> wrote:

> I am sorry to bother you mate. But I can not get it to work. I have
> been trying the whole day without success.
> Is there maybe you could show me a step by step guide on how I can
> create a mirror to our server.
> And then pushing the changes to our gerrit server so that we can have
> a versioning check system on our local mirror?

I thought my first message in this thread pretty much gave a step by
step instruction...? It even included command examples. The more
specific question you ask the more specific answer you'll get. Right
now I don't know where to start.

Make sure you have a fairly good understanding of Git before you start
doing this, otherwise everything will just be a series of weird commands
and you'll be doomed when something goes wrong.

> Also what about updating the local mirror to latest google sources?
> How would that go exactly?

This would be done in the same way as the initial push, except that you
won't necessarily have to download everything each time.

Syntax Error

unread,
Apr 5, 2011, 12:27:24 PM4/5/11
to Repo and Gerrit Discussion
Hi,

I created the mirror. Created the git projects manually and did a:
repo forall -c 'git push ssh://gerrithost/$REPO_PROJECT HEAD:refs/heads/master' 
This seems to have pushed everything into my git/gerrit system. Although I can not see any update information in gerrit at the time but I guess this is normal as I am not doing a commit.

When executing:
repo init -u ssh://gerrithost/android.git -b gingerbread

I get:
From ssh://gerrithost/android
 * [new branch]      aosp/eclair -> origin/aosp/eclair
 * [new branch]      aosp/froyo -> origin/aosp/froyo
 * [new branch]      aosp/froyo-stable -> origin/aosp/froyo-stable
 * [new branch]      aosp/gingerbread -> origin/aosp/gingerbread
 * [new branch]      aosp/master -> origin/aosp/master
 * [new branch]      aosp/themes-exp -> origin/aosp/themes-exp
 * [new branch]      master     -> origin/master
error: revision gingerbread in manifests not found

so when I execute:
repo init -u ssh://gerrithost/android.git -b aosp/gingerbread

It is still repo syncing from a remote repository...

Anyone knows what I need to edit and push back so that is does sync from my local server?


Daniel DeFreez

unread,
Apr 6, 2011, 12:39:53 AM4/6/11
to Repo and Gerrit Discussion
Did you edit default.xml in the project you are syncing from
(android.git in your case) to point at your local mirror? There is a
default remote for all of the git projects that needs to be changed.

Daniel
> On Mon, Apr 4, 2011 at 7:50 PM, Magnus Bäck <magnus.b...@sonyericsson.com>wrote:
>
>
>
>
>
>
>
> > On Monday, April 04, 2011 at 19:03 CEST,

Syntax Error

unread,
Apr 6, 2011, 2:56:02 AM4/6/11
to Repo and Gerrit Discussion
Hi,

Yes I changed it to:
  <remote  name="korg"
           fetch="git://android.git.kernel.org/"
           review="review.source.android.com" />

  <remote  name="local"
           fetch="ssh://gerrithost"
           review="mygerritServer.domain.com" />

  <default revision="gingerbread"
           remote="korg" />

I also changed the rest of the file to follow my project_path as:
<project path="bionic" name="MyPath/android_bionic" remote="local" />

Is there something I am missing? btw, How do I push: .repo/manifests/ back to my repo?
Now when I do:

repo forall -c 'git push ssh://gerrithost/$REPO_PROJECT HEAD:refs/heads/master'

I get:
To ssh://gerrithost/android
 ! [rejected]        HEAD -> master (non-fast forward)
error: failed to push some refs to 'ssh://gerrithost/android'
Everything up-to-date
Everything up-to-date
Everything up-to-date
Everything up-to-date
Everything up-to-date

Magnus Bäck

unread,
Apr 6, 2011, 3:48:49 AM4/6/11
to Repo and Gerrit Discussion
On Wednesday, April 06, 2011 at 08:56 CEST,
Syntax Error <impor...@gmail.com> wrote:

> Yes I changed it to:
> <remote name="korg"
> fetch="git://android.git.kernel.org/"
> review="review.source.android.com" />
>
> <remote name="local"
> fetch="ssh://gerrithost"
> review="mygerritServer.domain.com" />
>
> <default revision="gingerbread"
> remote="korg" />
>
> I also changed the rest of the file to follow my project_path as:
> <project path="bionic" name="MyPath/android_bionic" remote="local" />

Do you really need to rename the projects like that? It will be very
confusing. Otherwise this should work, but I'd switch the default
revision to use your new remote. Otherwise you'll have to remember to
update the manifest every time you merge new changes from upstream
(although you'll have to do that anyway since you're renaming the
projects).

> Is there something I am missing? btw, How do I push: .repo/manifests/
> back to my repo?

From a git perspective there's nothing unusual about the manifest git.

git push ssh://gerrithost/platform/manifest \
local-commit-ish:refs/heads/desired-branchname

(And by commit-ish I mean any reference that can be resolved to a
commit, e.g. a symbolic reference like HEAD, a SHA-1, a branch or
a tag.)

> Now when I do:
> repo forall -c 'git push ssh://gerrithost/$REPO_PROJECT
> HEAD:refs/heads/master'
>
> I get:
> To ssh://gerrithost/android
> ! [rejected] HEAD -> master (non-fast forward)
> error: failed to push some refs to 'ssh://gerrithost/android'

Which git is "android"? Anyway, why is this a non-fast forward? What's
HEAD pointing to, a local commit or one from AOSP? What's the master
branch on the server, your development branch or a reference branch
from AOSP?

> Everything up-to-date
> Everything up-to-date
> Everything up-to-date
> Everything up-to-date
> Everything up-to-date

Okay, looks good.

Syntax Error

unread,
Apr 6, 2011, 1:08:04 PM4/6/11
to Repo and Gerrit Discussion
Hi,

Thanks a lot for your feedback guys! :)

On Wed, Apr 6, 2011 at 9:48 AM, Magnus Bäck <magnu...@sonyericsson.com> wrote:
On Wednesday, April 06, 2011 at 08:56 CEST,
    Syntax Error <impor...@gmail.com> wrote:

> Yes I changed it to:
>   <remote  name="korg"
>            fetch="git://android.git.kernel.org/"
>            review="review.source.android.com" />
>
>   <remote  name="local"
>            fetch="ssh://gerrithost"
>            review="mygerritServer.domain.com" />
>
>   <default revision="gingerbread"
>            remote="korg" />
>
> I also changed the rest of the file to follow my project_path as:
> <project path="bionic" name="MyPath/android_bionic" remote="local" />

Do you really need to rename the projects like that? It will be very
confusing. Otherwise this should work, but I'd switch the default
revision to use your new remote. Otherwise you'll have to remember to
update the manifest every time you merge new changes from upstream
(although you'll have to do that anyway since you're renaming the
projects).

 The idea is to have a mixture of git repo's being updated. Most of the git's will be AOSP. And a few should only be work from me.

> Is there something I am missing? btw, How do I push: .repo/manifests/
> back to my repo?

From a git perspective there's nothing unusual about the manifest git.

  git push ssh://gerrithost/platform/manifest \
     local-commit-ish:refs/heads/desired-branchname

(And by commit-ish I mean any reference that can be resolved to a
commit, e.g. a symbolic reference like HEAD, a SHA-1, a branch or
a tag.)

> Now when I do:
> repo forall -c 'git push ssh://gerrithost/$REPO_PROJECT
> HEAD:refs/heads/master'
>
> I get:
> To ssh://gerrithost/android
>  ! [rejected]        HEAD -> master (non-fast forward)
> error: failed to push some refs to 'ssh://gerrithost/android'

Which git is "android"? Anyway, why is this a non-fast forward? What's
HEAD pointing to, a local commit or one from AOSP? What's the master
branch on the server, your development branch or a reference branch
from AOSP?

I have made the changes to the android.git and git commited them manually to: git push ssh://gerrithost/android.git HEAD:refs/heads/master

> Everything up-to-date
> Everything up-to-date
> Everything up-to-date
> Everything up-to-date
> Everything up-to-date

Okay, looks good.

--
Magnus Bäck                   Opinions are my own and do not necessarily
SW Configuration Manager      represent the ones of my employer, etc.
Sony Ericsson

--
To unsubscribe, email repo-discuss...@googlegroups.com

I am still a little bit lost on how to create a full copy of AOSP mainstream and have it correctly executing: repo init -u ssh://gerrithost/android.git -b gingerbread. So that it syncs from my local server and not from the AOSP remote. At this moment the feedback has been very nice but making it more difficult for me to understand it correctly.
My knowledge of git is also not that much at the moment so its starting to get a bit confusing here :)

Magnus Bäck

unread,
Apr 6, 2011, 4:41:16 PM4/6/11
to Repo and Gerrit Discussion
On Wednesday, April 06, 2011 at 19:08 CEST,
Syntax Error <impor...@gmail.com> wrote:

> On Wed, Apr 6, 2011 at 9:48 AM, Magnus Bäck
> <magnu...@sonyericsson.com> wrote:
>
> > Do you really need to rename the projects like that? It will be
> > very confusing. Otherwise this should work, but I'd switch the
> > default revision to use your new remote. Otherwise you'll have
> > to remember to update the manifest every time you merge new
> > changes from upstream (although you'll have to do that anyway
> > since you're renaming the projects).
>
> The idea is to have a mixture of git repo's being updated. Most
> of the git's will be AOSP. And a few should only be work from me.

Ah, okay. If all AOSP gits use the korg remote this is fine. However,
if your goal is to have your users sync everything from your local
server and not contact android.git.kernel.org at all it's not what
you want to do.

> I am still a little bit lost on how to create a full copy of
> AOSP mainstream and have it correctly executing: repo init -u
> ssh://gerrithost/android.git -b gingerbread. So that it syncs
> from my local server and not from the AOSP remote.

Again, if you don't want local clients asking android.git.kernel.org
for anything you must not list the korg remote for any project in
default.xml. Secondly, make sure the revision element of each git
points to a valid branch. You can start off pointing them to aosp/master
(or in some cases aosp/tools_r8 or similar) if you've successfully
pushed refs/heads/* to refs/heads/aosp/*.

When that works I suggest you create your own development branch based
on the AOSP branch for each git and use that branch in the manifest.
To do this you can e.g. use the $REPO_RREV variable in a forall command.
That variable is set to the revision attribute listed in the manifest,
i.e. it'll be "aosp/master" for almost all gits. A little string
manipulation and you have the string you want.

repo forall -c 'git push ssh://.../$REPO_PROJECT \
origin/$REPO_RREV:refs/heads/`echo $REPO_RREV | sed "s/^aosp.//"`'

This should produce a command like this (again, verify by prepending
"echo" to the command before running anything):

git push ssh://.../$REPO_PROJECT origin/aosp/master:refs/heads/master

You should play around some with the forall command and its variables.
For example, try this:

repo forall -c 'echo $REPO_PROJECT uses $REPO_RREV'

> At this moment the feedback has been very nice but making it more
> difficult for me to understand it correctly. My knowledge of git
> is also not that much at the moment so its starting to get a bit
> confusing here :)

The stuff you want to do isn't what you'd usually start with, and it's
not obvious at first. Do take some time to dig into Git by reading a
book and experimenting.

Syntax Error

unread,
Apr 8, 2011, 4:00:19 AM4/8/11
to Repo and Gerrit Discussion
On Wed, Apr 6, 2011 at 10:41 PM, Magnus Bäck <magnu...@sonyericsson.com> wrote:
On Wednesday, April 06, 2011 at 19:08 CEST,
    Syntax Error <impor...@gmail.com> wrote:

> On Wed, Apr 6, 2011 at 9:48 AM, Magnus Bäck
> <magnu...@sonyericsson.com> wrote:
>
> > Do you really need to rename the projects like that? It will be
> > very confusing. Otherwise this should work, but I'd switch the
> > default revision to use your new remote. Otherwise you'll have
> > to remember to update the manifest every time you merge new
> > changes from upstream (although you'll have to do that anyway
> > since you're renaming the projects).
>
>  The idea is to have a mixture of git repo's being updated. Most
> of the git's will be AOSP. And a few should only be work from me.

Ah, okay. If all AOSP gits use the korg remote this is fine. However,
if your goal is to have your users sync everything from your local
server and not contact android.git.kernel.org at all it's not what
you want to do.

Yes indeed sorry for messing things up. I meant that the end goal was to have a mixed repo server.
Most of the gits should checkout the korg git repo's and and for our edited sources to check out our changes from our gerrit server only. 
So this will mean I will have to edit the .repo/manifests/default.xml and adding a new remote fetch url that points to our gerrit git. With the remote project path's I guess?
Thanks Magnus for all your feedback to the community it is highly appreciated.
I'll start off in doing some digging into git as its probably the best before jumping straight into repo :)
 

--
Magnus Bäck                   Opinions are my own and do not necessarily
SW Configuration Manager      represent the ones of my employer, etc.
Sony Ericsson

Magnus Bäck

unread,
Apr 11, 2011, 6:29:35 PM4/11/11
to Repo and Gerrit Discussion
On Friday, April 08, 2011 at 10:00 CEST,
Syntax Error <impor...@gmail.com> wrote:

> On Wed, Apr 6, 2011 at 10:41 PM, Magnus Bäck
> <magnu...@sonyericsson.com>wrote:
>

> > Ah, okay. If all AOSP gits use the korg remote this is fine.
> > However, if your goal is to have your users sync everything from
> > your local server and not contact android.git.kernel.org at all
> > it's not what you want to do.
>
> Yes indeed sorry for messing things up. I meant that the end goal was
> to have a mixed repo server.
> Most of the gits should checkout the korg git repo's and and for our
> edited sources to check out our changes from our gerrit server only.
> So this will mean I will have to edit the .repo/manifests/default.xml
> and adding a new remote fetch url that points to our gerrit git. With
> the remote project path's I guess?

I doubt we're using the same terminology, so it's probably better to
show an example and let the code to the talking.

I would be careful when mixing code straight from android.git.kernel.org
(remote="korg") with code from your own branches where you've made
changes. What if developers sync and get new code from AOSP that's
incompatible with your code? This way of working gets you the latest
AOSP changes immediately, which can be both good and bad. If you have
all people use your own server (remote="localserver") you can have a QA
process for making sure that AOSP changes doesn't break your stuff (i.e.
you won't update the local branches with new AOSP stuff until you've
verified that it works okay). I'm not saying this is necessary, but
depending on the amount of changes your make (and their magnitude) it
could make sense.

[...]

Sridhar Ananthakrishnan

unread,
Feb 27, 2015, 3:18:45 AM2/27/15
to repo-d...@googlegroups.com
Thanks for information, its really helpful for me Mr.  Magnus Back
Reply all
Reply to author
Forward
0 new messages