Repo mirror

[nictam]

Hi,

Is there any documentation, or message thread, that would explain in
details how repo --mirror works? For example I have some a specific
question:

when I create a mirror, how does repo choose which tree to mirror?
Does it take all the trees in the most recent commit from master
branch? or does it also mirror trees which could be referenced in a
different branch, or an older commit?

I'd like to make sure I understand how it works, so that I can setup a
proper mirror.

thanks.

[Shawn Pearce]

On Wed, Jun 17, 2009 at 14:05, nictam<nic...@gmail.com> wrote:
> when I create a mirror, how does repo choose which tree to mirror?
> Does it take all the trees in the most recent commit from master
> branch?

Yes

> or does it also mirror trees which could be referenced in a
> different branch, or an older commit?

No, it doesn't do any of that.

[nictam]

thanks.

isn't that a problem?

once I setup a mirror, how can I ensure that my mirror's users will be
able to repo sync from any version of the manifest? e.g. let's say
that for 'donut' branch I have a specific git tree which is not in
'master'. If I initialize my repo based on 'master', when someone
tries to repo init from my mirror with '-b donut' then the sync will
fail, right?

So basically my mirror is just a 'partial' mirror of the main site, is
that correct?

[Shawn Pearce]

On Thu, Jun 18, 2009 at 03:34, nictam<nic...@gmail.com> wrote:
>
> thanks.
>
> isn't that a problem?

Yes. :-)

Patches to repo are more than welcome if you have a solution.

> once I setup a mirror, how can I ensure that my mirror's users will be
> able to repo sync from any version of the manifest? e.g. let's say
> that for 'donut' branch I have a specific git tree which is not in
> 'master'. If I initialize my repo based on 'master', when someone
> tries to repo init from my mirror with '-b donut' then the sync will
> fail, right?

One approach is to switch the manifest on the mirror and sync twice:

repo init -b master; repo sync
repo init -b donut; repo sync

Another is to monitor changes made to the manifest branches, and to
create an uber-manifest, one that contains all projects from all of
the manifest branches, and have only your mirror use that
uber-manifest. That way all projects are updated.

[nictam]

On Jun 18, 3:56 pm, Shawn Pearce <s...@google.com> wrote:
> On Thu, Jun 18, 2009 at 03:34, nictam<nic...@gmail.com> wrote:
>
> > thanks.
>
> > isn't that a problem?
>
> Yes.  :-)
>
> Patches to repo are more than welcome if you have a solution.

well, that's an option, you're right ;-) I don't feel comfortable for
now with that, but I'll try to think about it...

>
> > once I setup a mirror, how can I ensure that my mirror's users will be
> > able to repo sync from any version of the manifest? e.g. let's say
> > that for 'donut' branch I have a specific git tree which is not in
> > 'master'. If I initialize my repo based on 'master', when someone
> > tries to repo init from my mirror with '-b donut' then the sync will
> > fail, right?
>
> One approach is to switch the manifest on the mirror and sync twice:
>
>   repo init -b master; repo sync
>   repo init -b donut; repo sync
>
> Another is to monitor changes made to the manifest branches, and to
> create an uber-manifest, one that contains all projects from all of
> the manifest branches, and have only your mirror use that
> uber-manifest.  That way all projects are updated.

Thanks. I like this idea. And in fact I have several projects that I
want to mirror, so I can address both problems with a single top level
manifest dedicated to mirror.

[nictam]

Shawn,

just to make sure I understand well. When doing a repo init --mirror,
repo does:

- get repo from android.git.kernel.org
- get the manifest file, the last commit on the master branch, or
another branch if using -b
- from this manifest, populate the list of all projects, and get their
full names, e.g. 'fetch' address concatenated with their 'name'
attribute. Attributes 'path' and 'revision' are ignored.
- for each project, do a bare git clone in the 'project.name' folder:
git clone -n 'remote.fetch'+'project.name' 'project.name'
- also create a bare git clone of the manifest itself, which is
usually not done without the --mirror option
- if there are 'copyfile' attribute for a project in the manifest,
they are ignored as well.

is that all correct?

thanks!


On Jun 18, 7:46 pm, nictam <nic...@gmail.com> wrote:

> On Jun 18, 3:56 pm, Shawn Pearce <s...@google.com> wrote:
>
> > On Thu, Jun 18, 2009 at 03:34, nictam<nic...@gmail.com> wrote:
>
> > > thanks.
>
> > > isn't that a problem?
>
> > Yes.  :-)
>
> > Patches to repo are more than welcome if you have a solution.
>

> well, that's an option, you're right ;-) I don't feel comfortable for
> now with that, but I'll try to think about it...
>
>
>
>
>

> > > once I setup a mirror, how can I ensure that my mirror's users will be
> > > able to repo sync from any version of the manifest? e.g. let's say
> > > that for 'donut' branch I have a specific git tree which is not in
> > > 'master'. If I initialize my repo based on 'master', when someone
> > > tries to repo init from my mirror with '-b donut' then the sync will
> > > fail, right?
>
> > One approach is to switch the manifest on the mirror and sync twice:
>
> >   repo init -b master; repo sync
> >   repo init -b donut; repo sync
>
> > Another is to monitor changes made to the manifest branches, and to
> > create an uber-manifest, one that contains all projects from all of
> > the manifest branches, and have only your mirror use that
> > uber-manifest.  That way all projects are updated.
>

[Shawn Pearce]

Yes

On Thu, Jun 18, 2009 at 11:15, nictam<nic...@gmail.com> wrote:
>
> Shawn,
>
> just to make sure I understand well. When doing a repo init --mirror,
> repo does:
>
> - get repo from android.git.kernel.org
> - get the manifest file, the last commit on the master branch, or
> another branch if using -b
> - from this manifest, populate the list of all projects, and get their
> full names, e.g. 'fetch' address concatenated with their 'name'
> attribute. Attributes 'path' and 'revision' are ignored.
> - for each project, do a bare git clone in the 'project.name' folder:
> git clone -n 'remote.fetch'+'project.name' 'project.name'
> - also create a bare git clone of the manifest itself, which is
> usually not done without the --mirror option
> - if there are 'copyfile' attribute for a project in the manifest,
> they are ignored as well.
>
> is that all correct?
>
> thanks!
>
>
> On Jun 18, 7:46 pm, nictam <nic...@gmail.com> wrote:

>> On Jun 18, 3:56 pm, Shawn Pearce <s...@google.com> wrote:
>>
>> > On Thu, Jun 18, 2009 at 03:34, nictam<nic...@gmail.com> wrote:
>>
>> > > thanks.
>>
>> > > isn't that a problem?
>>
>> > Yes.  :-)
>>
>> > Patches to repo are more than welcome if you have a solution.
>>

>> well, that's an option, you're right ;-) I don't feel comfortable for
>> now with that, but I'll try to think about it...
>>
>>
>>
>>
>>

>> > > once I setup a mirror, how can I ensure that my mirror's users will be
>> > > able to repo sync from any version of the manifest? e.g. let's say
>> > > that for 'donut' branch I have a specific git tree which is not in
>> > > 'master'. If I initialize my repo based on 'master', when someone
>> > > tries to repo init from my mirror with '-b donut' then the sync will
>> > > fail, right?
>>
>> > One approach is to switch the manifest on the mirror and sync twice:
>>
>> >   repo init -b master; repo sync
>> >   repo init -b donut; repo sync
>>
>> > Another is to monitor changes made to the manifest branches, and to
>> > create an uber-manifest, one that contains all projects from all of
>> > the manifest branches, and have only your mirror use that
>> > uber-manifest.  That way all projects are updated.
>>