Any way to repo sync from a local machine on LAN?

[Ashish S]

any way to do repo sync on a m/c that does the sync from a complete checked out source tree that has been already synced from chromiumos.org?

The idea is to use a local copy of the repo when one has already downloaded from chromium.org to save bandwidth and also not to get banned from google due to 10 developers doing repo sync from chromiun.org?

It does make sense for such a huge source.. that one source tree can act as mirror to local devs?

any hack that can be done on repo tool for achieving this?

Thanks a ton.

[Mike Frysinger]

On Sat, Jun 23, 2012 at 6:09 AM, Ashish S <ash...@gmail.com> wrote:
> any way to do repo sync on a m/c that does the sync from a complete checked
> out source tree that has been already synced from chromiumos.org?
> The idea is to use a local copy of the repo when one has already downloaded
> from chromium.org to save bandwidth and also not to get banned from google
> due to 10 developers doing repo sync from chromiun.org?
>
> It does make sense for such a huge source.. that one source tree can act as
> mirror to local devs?

sounds like you want:
http://dev.chromium.org/chromium-os/how-tos-and-troubleshooting/creating-local-mirrors

and the in-development:
https://gerrit.chromium.org/gerrit/25263
-mike

[Ashish S]

hello,

Thanks for replying...

 the doc site says : to do an initial sync by   ./cros_mirror -r /home/cros/mirror -u git://local-cros-mirror

but i have already downloaded my source in ~/chromiumos... and i need to convert this into a local repo..
do i need to download the source once again for setting up a mirror?

when i did:./cros_mirror -r ~/chromiumos/ -u git://local-cros-mirror
it synced the sources successfully but ended up with this error...



Your sources have been sync'd successfully.
info: cloning manifests
Initialized empty Git repository in /media/abc/chromiumos/manifests/.git/
fatal: './manifest.git' does not appear to be a git repository
fatal: The remote end hung up unexpectedly


Any help?


Thanks.

On Sunday, 24 June 2012 01:01:28 UTC+5:30, Mike Frysinger wrote:

[Mike Frysinger]

On Sun, Jun 24, 2012 at 9:47 AM, Ashish S <ash...@gmail.com> wrote:
> but i have already downloaded my source in ~/chromiumos... and i need to
> convert this into a local repo..

the script/document i mentioned is the only supported method for
creating a local mirror. with an existing `repo init && repo sync`,
all of the git dirs exist in .repo/projects/, so you might be able to
massage something into place, but you're on your own here.

> when i did:./cros_mirror -r ~/chromiumos/ -u git://local-cros-mirror
> it synced the sources successfully but ended up with this error...
>
> Your sources have been sync'd successfully.
> info: cloning manifests
> Initialized empty Git repository in /media/abc/chromiumos/manifests/.git/
> fatal: './manifest.git' does not appear to be a git repository
> fatal: The remote end hung up unexpectedly

when i wrote this script, i was using an older version of repo. looks
like the one pulled in recently (1.9.4) has fixed some dumb things
that i was working around. i'll post an updated CL.
-mike

[Ashish S]

Hello Mike...

Ok... i am moving forward in the official way..

I tried to run this command :

cros_mirror -r . -u git://local-cros-mirror

and ended up in a successful sync.. but thereafter an error came..

----------------------------- --------------------------------------------------------------------------------------------------------------------

Fetching projects: 100% (204/204), done.  

info: updating manifest

usage: git ls-remote [--heads] [--tags]  [-u <exec> | --upload-pack <exec>] <repository> <refs>...

info: rewriting manifest branches to git://local-cros-mirror

error: unknown switch `B'

usage: git checkout [options] <branch>

   or: git checkout [options] [<branch>] -- <file>...

    -q, --quiet           be quiet

    -b <new branch>       branch

    -l                    log for new branch

    -t, --track           track

    -2, --ours            stage

    -3, --theirs          stage

    -f, --force           force

    -m, --merge           merge

    --conflict <style>    conflict style (merge or diff3)

    -p, --patch           select hunks interactively

----------------------------- --------------------------------------------------------------------------------------------------------------------

there might be some issue with the cros_mirror script...

i actually got the script from https://gerrit.chromium.org/gerrit/25263 as mentioned in your reply in this thread..

because the wget method in the documentation doesnt work any more:

wget -O cros_mirror 'http://git.chromium.org/gitweb/?p=chromiumos/chromite.git;a=blob_plain;f=scripts/cros_mirror;hb=HEAD' 

is there something wrong at my end or at scripts end?

Many Thanks.

Ashish

On Monday, 25 June 2012 22:08:16 UTC+5:30, Mike Frysinger wrote:

[Mike Frysinger]

On Fri, Jun 29, 2012 at 5:27 AM, Ashish S wrote:
> usage: git ls-remote [--heads] [--tags]  [-u <exec> | --upload-pack <exec>] <repository> <refs>...
> info: rewriting manifest branches to git://local-cros-mirror
> error: unknown switch `B'
> usage: git checkout [options] <branch>

looks like you're using an old git.  upgrade to 1.7.7 or newer.  i'll
add a check to the script as you aren't the first one to hit this.
-mike

[Ashish S]

Hi,

I updated the git version and it worked.. but when i try to repo init at from a client.. the following error comes..

repo init -u git://local-cros-mirror/chromiumos/manifest-mirror.git --repo-url=git://local-cros-mirror/external/repo.git

Get git://local-cros-mirror/external/repo.git

fatal: unable to connect to local-cros-mirror:

local-cros-mirror: Name or service not known

Thanks.

Ashish.

[Ashish S]

P.S i have already setup the git daemon.. as documented..

[Brian Harring]

On Sat, Jun 30, 2012 at 1:35 AM, Ashish S <ash...@gmail.com> wrote:

Hi,

I updated the git version and it worked.. but when i try to repo init at from a client.. the following error comes..

repo init -u git://local-cros-mirror/chromiumos/manifest-mirror.git --repo-url=git://local-cros-mirror/external/repo.git

Get git://local-cros-mirror/external/repo.git

fatal: unable to connect to local-cros-mirror:

local-cros-mirror: Name or service not known

Keep in mind I've not been following this conversation closely, but the line above- the "local-cros-mirror: Name or service not known" is fairly indicative of your issue.

You need to use the hostname you passed for cros_mirror --url; that name needs to be resolvable (either dns, or IP, your choice).  For a *temporary* hack to test it, you could set /etc/hosts w/ an appropriate mapping- but I'd strongly suggest using a properly resolvable name rather than something that requires host hardcoding.

Cheers-

~brian

[Ashish S]

Oh Thanks Brian.. that was the only mistake..

i am able to repo sync from the mirror now.. 

Thanks so much for the help..

Ashish

On Saturday, June 30, 2012 4:34:48 PM UTC+5:30, Brian Harring wrote:

[Ashish S]

Hello,

Thanks for helping.. i have been able to make a mirror repo and its working fine..

now i have a problem.. suppose we have started working in mirrored source checked out from this local repo.. and repo sync on the mirrored source is working fine..

what if the  mirrored repo's path changes.. for example its ip over local LAN changes from X to Y??

i tried changing the fetch paths of the full.xml amd minilayout.xml manifests files placed in .repo dir on one of the mirrored source.. but that didnot work.. repo sync was unable to sync..?

any support for such a scenario?

Thanks.

[Mike Frysinger]

use DNS then instead of raw IPs

the only way to have clients change where they sync from is to re-run `repo init ...` in their checkouts

-mike

--
Chromium OS Developers mailing list: chromiu...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-os-dev?hl=en

[Ashish S]

Hello,

Using a local repo mirror thats been synced to latest release, is it possible to checkout a previous release e.g R18 on a client?

will this: gclient config https://src.chromium.org/chrome/releases/18.0.1001.1 

help? if yes when should it be run,before repo  init?

Thanks

Ashish.

[Brian Harring]

Might be missing something about your request, but...

repo init <usual args> -b <the-desired-branch> && repo sync # should do it.

For example, to checkout R22;

repo init -u https://git.chromium.org/git/chromiumos/manifest -b release-R22-2723.B && repo sync

If you want to checkout a specific build from the past that isn't  a specific branch head, then you grab that build manifest and plunk it at .repo/manifest.xml, then trigger the sync.

~brian

[Ashish S]

i tried your suggestion..

the syncing fails at :

From file:///home/xyz/cctar/tar/cc/external/google-breakpad/src

 * [new branch]      master     -> cros/master

Fetching projects:   2% (4/186)  fatal: '/home/xyz/cctar/tar/cc/chromium/support' does not appear to be a git repository

fatal: The remote end hung up unexpectedly

fatal: '/home/xyz/cctar/tar/cc/chromium/support' does not appear to be a git repository

fatal: The remote end hung up unexpectedly

error: Cannot fetch chromium/support

any clues?

Thanks.

[Mike Frysinger]

the repo mirror behavior only syncs repos in the active manifest.
since the cros_mirror script uses repo to create the mirror, and it
uses the ToT manifest, you can really only safely check out the same
branch the mirror was created with. e.g. ToT.

when you try to checkout an older branch that refers to a repo that no
longer exists (because we punted it in ToT), you'll get an error as
the local mirror did not sync that repo.

repo did not seem to offer a way of merging multiple manifests into
one massive mirror so that you could have one to serve them all :(.

my older cros_mirror based on pure git could be extended to do this,
but that has obvious tradeoffs (parsing the manifest directly vs
letting repo do it).
-mike

[Ashish S]

Hi,

ok so i get the point that once a local repo has been made for a particular branch.. that means its history doesnot date back to the start of time.. clients can pull only the branch that the repo reffers to.

fine,but this fails too...

./cros_mirror -r . -u file:///home/xyz/cctar/tar/r18/ -b release-R18-1660.B   

this is failing,telling no -b option 

if i change it to -m release-R18-1660.B

it ends up saying fatal: manifest release-R18-1660.B not found

how to know the correct manifest for this branch??

Thanks for helping.

[Mike Frysinger]

that depends on what you mean by "its history". any git repo that
gets synced by repo will have its full history. the only thing that
is missing are repos that exist in older branches but not the ToT.

cros_mirror does not currently support mirroring branches. the -m
option is not what you want.
-mike

[Ashish S]

so isint there a way to create a local repo for R18? i am not specific to a branch.. i just need R18 any suggestions?

thanks.

[Mike Frysinger]

you've got the source to the cros_mirror script. you're free to hack
it for your needs.
-mike