repo mirror server and Gerrit server

[johnny]

Hi,

I have setup an AOSP mirror server with repo and can use "repo sync"
to sync code from the mirror server to my develop machine internally.
I am now evaluating to setup a Gerrit sever internally. As I read the
documentation of Gerrit. It requires to setup a GIT repository. I
wonder if I can use the GIT repositories in the AOSP mirror server
directly. So the Gerrit will directly push the change to my AOSP
mirror server. Of course, I can put the Gerrit server and AOSP mirror
server at the same physical machine.

Regards,
Johnny Xia.

[Shawn Pearce]

Wow, shame on me. I meant to reply to your post 9 days ago, and it
got lost in the noise. :-|

I wouldn't use the mirror repositories directly, due to open bug
GERRIT-81 [1]. Currently you can't safely modify a repository from
outside of Gerrit. Any modifications require a restart of Gerrit.

Instead, make another set of repositories, e.g.:

repo forall -c 'mkdir -p $(dirname
/srv/gerrit/base/$REPO_PROJECT.git); git clone --bare .
/srv/gerrit/base/$REPO_PROJECT.git'

and configure Gerrit to use /srv/gerrit/base as git_base_path (or
whatever your configuration is).

Create a user account in Gerrit which has Push Branch access in all
projects, and use this user to update after you have done a repo sync
on your mirror. E.g.

cd my_mirror
repo sync
repo forall -c 'git push
ssh://update-user@localhost:29418/$REPO_PROJECT
refs/heads/*:refs/heads/*'

The user account can be inserted by hand if you need to in the
accounts table and account_ssh_keys table (use nextval('account_id')
to get a new unique account_id). There is no requirement that the
account exist in your SSO server, or have an OpenID handle.

FWIW, Google updates android.git.kernel.org via this method, from its
Perforce dump and conversion script.

[1] http://jira.source.android.com/jira/browse/GERRIT-81

[johnny]

Thanks Shawn. So it is a limitation of JGit instead of Gerrit, will
you or the developer of JGit try to fix it in future?

Using 2 repositories is not convenient, as I have to push the change
back and forth between these 2 repositories. After I sync from android
server, I need push change to Gerrit's repository. Once there is a
change in Gerrit's repository, I have to push that change back to
mirror server.

Regards,
Johnny Xia



On Mar 19, 8:11 am, Shawn Pearce <s...@google.com> wrote:

[Shawn Pearce]

On Thu, Mar 19, 2009 at 02:05, johnny <john...@gmail.com> wrote:
>
> Thanks Shawn. So it is a limitation of JGit instead of Gerrit, will
> you or the developer of JGit try to fix it in future?

Its a bug in JGit. I'm the maintainer of JGit. It will be fixed.
Eventually. Its just been a lot lower priority than other issues
lately. I started work on it a few weeks back and got some patches
into JGit to support this, but the work is not yet complete.

> Using 2 repositories is not convenient, as I have to push the change
> back and forth between these 2 repositories. After I sync from android
> server, I need push change to Gerrit's repository. Once there is a
> change in Gerrit's repository, I have to push that change back to
> mirror server.

No, have your team use the Gerrit repository as their mirror server.
They just can't see AOSP updates until after you push them from mirror
to Gerrit. So you only have a flow of:

AOSP ---> mirror ---> Gerrit <---> team

[johnny]

It's fine to have the team to use Gerrit's repository, but there is
still need to modify Gerrit's repository as I have to push the change
from mirror server to Gerrit's repository. Shall I stop the Gerrit
when I push changes from mirror server to Gerrit's repository?

If that is the case, can I do this way? Let Gerrit and mirror server
share the same repository, but when I need to sync the mirror server,
first stop the Gerrit server, once the sync is done, restart Gerrit
server. The only limitation I see here is that it may take a long time
to sync from android server.

On Mar 19, 10:40 pm, Shawn Pearce <s...@google.com> wrote:

[Shawn Pearce]

On Thu, Mar 19, 2009 at 22:05, johnny <john...@gmail.com> wrote:
>
> It's fine to have the team to use Gerrit's repository, but there is
> still need to modify Gerrit's repository as I have to push the change
> from mirror server to Gerrit's repository. Shall I stop the Gerrit
> when I push changes from mirror server to Gerrit's repository?

No need to stop Gerrit. Just make sure you push through Gerrit's SSH
daemon and not through the local filesystem. Meaning use

repo forall -c 'git push

ssh://mirror-user@localhost:29418/$REPO_PROJECT
refs/heads/*:refs/heads/*'

instead of:

repo forall -c 'git push /srv/git/$REPO_PROJECT refs/heads/*:refs/heads/*'

If you push through Gerrit's SSH daemon, Gerrit is making the edits to
the local filesystem, so it knows to keep its caches coherent. The
problem is when some other process writes to the filesystem, Gerrit
doesn't know about it, so its caches are out of date, and it currently
has no way to correct the caches.

> If that is the case, can I do this way? Let Gerrit and mirror server
> share the same repository, but when I need to sync the mirror server,
> first stop the Gerrit server, once the sync is done, restart Gerrit
> server. The only limitation I see here is that it may take a long time
> to sync from android server.

Yes, you could also do this. Stop Gerrit, sync, then start it, but
the downside is the possibly large downtime while the sync occurs.

[J.K Wang]

when I run:

repo forall -c 'mkdir -p $(dirname /srv/gerrit/base/
$REPO_PROJECT.git); git clone --bare . /srv/gerrit/base/
$REPO_PROJECT.git'

I found the following error message:
Initialized empty Git repository in /srv/gerrit/base/platform/packages/
apps/VoiceDialer.git/
find: write error: Broken pipe

On 3月19日, 上午8时11分, Shawn Pearce <s...@google.com> wrote:
> On Tue, Mar 10, 2009 at 08:36, johnny <johnny...@gmail.com> wrote:
>
> > I have setup an AOSPmirrorserver with repo and can use "repo sync"
> > to sync code from themirrorserver to my develop machine internally.
> > I am now evaluating to setup aGerritsever internally. As I read the
> > documentation ofGerrit. It requires to setup a GIT repository. I

> > wonder if I can use the GIT repositories in the AOSPmirrorserver

> > directly. So theGerritwill directly push the change to my AOSP
> >mirrorserver. Of course, I can put theGerritserver and AOSPmirror

> > server at the same physical machine.
>
> Wow, shame on me. I meant to reply to your post 9 days ago, and it
> got lost in the noise. :-|
>

> I wouldn't use themirrorrepositories directly, due to open bugGERRIT-81 [1]. Currently you can't safely modify a repository from
> outside ofGerrit. Any modifications require a restart ofGerrit.

>
> Instead, make another set of repositories, e.g.:
>
> repo forall -c 'mkdir -p $(dirname
> /srv/gerrit/base/$REPO_PROJECT.git); git clone --bare .
> /srv/gerrit/base/$REPO_PROJECT.git'
>

> and configureGerritto use /srv/gerrit/base as git_base_path (or
> whatever your configuration is).
>
> Create a user account inGerritwhich has Push Branch access in all

> projects, and use this user to update after you have done a repo sync

> on yourmirror. E.g.

[Shawn Pearce]

2009/3/30 J.K Wang <jkun...@gmail.com>

when I run:

repo forall -c 'mkdir -p $(dirname /srv/gerrit/base/
$REPO_PROJECT.git); git clone --bare . /srv/gerrit/base/
$REPO_PROJECT.git'

I found the following error message:
Initialized empty Git repository in /srv/gerrit/base/platform/packages/
apps/VoiceDialer.git/
find: write error: Broken pipe

The find | cpio pipeline must have aborted, because cpio aborted, maybe the destination disk is full?

You could also try using the file:// URI when doing the clone, to force a repacking on the fly.  E.g.:

repo forall -c 'mkdir -p $(dirname /srv/gerrit/base/$REPO_PROJECT.git); git clone --bare file://. /srv/gerrit/base/$REPO_PROJECT.git'