Request to create project mirror/manifest on the Gerrit box

[Scott Fan]

Hi David,

Do you have the privilege to create a new project on the https://gerrit-review.googlesource.com/ site?

If so, could you please create a new project named mirror/manifest whose description is "Manifest for mirroring that contains all the projects that are hosted on the Gerrit server."?

I wanna push a manifest file to this project at later.

Thanks.
Scott Fan

[Pursehouse, David]

What exactly is the purpose of this mirror manifest?  You want to include all the projects that are hosted on gerrit-review?

 

Yes, I am able to create projects, however I think I would prefer to leave it to one of the more longer-standing maintainers to decide if it would be appropriate to host such a project on gerrit-review.

[Edwin Kempin]

2013/2/27 Pursehouse, David <David.Pu...@sonymobile.com>

What exactly is the purpose of this mirror manifest?  You want to include all the projects that are hosted on gerrit-review?

 

Yes, I am able to create projects, however I think I would prefer to leave it to one of the more longer-standing maintainers to decide if it would be appropriate to host such a project on gerrit-review.

I think Shawn needs to approve this.
 

 

 

From: Scott Fan [mailto:fanc...@gmail.com]
Sent: Wednesday, February 27, 2013 4:25 PM
To: Pursehouse, David
Cc: repo-discuss
Subject: Request to create project mirror/manifest on the Gerrit box

 

Hi David,

Do you have the privilege to create a new project on the https://gerrit-review.googlesource.com/ site?

If so, could you please create a new project named mirror/manifest whose description is "Manifest for mirroring that contains all the projects that are hosted on the Gerrit server."?

I wanna push a manifest file to this project at later.

Thanks.
Scott Fan

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en
 
---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Scott Fan]

On Wed, Feb 27, 2013 at 6:01 PM, Edwin Kempin <edwin....@gmail.com> wrote:

2013/2/27 Pursehouse, David <David.Pu...@sonymobile.com>

What exactly is the purpose of this mirror manifest?  You want to include all the projects that are hosted on gerrit-review?

YES.
I expect we can create a local mirror for all projects on gerrit-review just like AOSP.
Thus, we need the project https://gerrit.googlesource.com/mirror/manifest for Gerrit,
like the project https://android.googlesource.com/mirror/manifest for AOSP .

 

Yes, I am able to create projects, however I think I would prefer to leave it to one of the more longer-standing maintainers to decide if it would be appropriate to host such a project on gerrit-review.

I think Shawn needs to approve this.

 

Please forgive me I don't know who should be to create a new project on gerrit-review.

[Martin Fick]

This feels a bit unnecessary since you can just get the list of projects using the REST API and then loop over them, this is what we do,

-Martin

----- Reply message -----
From: "Scott Fan" <fanc...@gmail.com>
To: "Edwin Kempin" <edwin....@gmail.com>
Cc: "Pursehouse, David" <David.Pu...@sonymobile.com>, "repo-discuss" <repo-d...@googlegroups.com>
Subject: Request to create project mirror/manifest on the Gerrit box

[Shawn Pearce]

I agree with Martin. If you want a list of the projects just use the
REST API to get the list and loop over it yourself. I don't want to
maintain a repo manifest.

[Scott Fan]

Hi guys,

My real purpose is to clone all git projects on the https://gerrit.googlesource.com/ site as my local MIRROR repositories, not only just to get a list of the projects. We do have a lot of ways to achieve this goal, including using the REST API. But all of those, we need write some scripts, or need do a lot of steps.

However, DO NOT forget the great repo tool, why not to use it?
With repo, only two commands can finish those, need not to write any extra scripts.
$ repo init -u https://gerrit.googlesource.com/mirror/manifest --mirror
$ repo sync

Right now, we just need a new project named mirror/manifest hosting on the gerrit site. It only needs a 'master' branch, and a manifest file named default.xml(see attached pls).

It's so simple, no need of frequent maintenance.

With those, all work will be quite simple.

Thanks.

Scott Fan

[Saša Živkov]

On Thu, Feb 28, 2013 at 12:46 PM, Scott Fan <fanc...@gmail.com> wrote:

Hi guys,

My real purpose is to clone all git projects on the https://gerrit.googlesource.com/ site as my local MIRROR repositories, not only just to get a list of the projects. We do have a lot of ways to achieve this goal, including using the REST API. But all of those, we need write some scripts, or need do a lot of steps.

However, DO NOT forget the great repo tool, why not to use it?
With repo, only two commands can finish those, need not to write any extra scripts.
$ repo init -u https://gerrit.googlesource.com/mirror/manifest --mirror
$ repo sync

Right now, we just need a new project named mirror/manifest hosting on the gerrit site. It only needs a 'master' branch, and a manifest file named default.xml(see attached pls). 

It's so simple, no need of frequent maintenance.

The main questions is who will maintain it?

Obviously, when a new project is created someone needs to remember to update

this manifest file. No one of Gerrit maintainers want this additional step.

Is it so hard to use curl, and parse a json output?

It is one time effort (on your side) vs updating the manifest every time a new project is created (on our side).

[Scott Fan]

> The main questions is who will maintain it?

Maintenance is not a problem, I think, if no one is willing to take on this matter, then I can do it.

> Is it so hard to use curl, and parse a json output?

Repo is enough, I think. No need to do any more.

Scott Fan

[Thomas Swindells (tswindel)]

Perhaps somebody needs to create a Gerrit plugin which automatically exposes a (set of) manifest dynamically?

 

Thomas

[Scott Fan]

> Perhaps somebody needs to create a Gerrit plugin which automatically exposes a (set of) manifest dynamically?

  This sounds very good, but it needs a period of time. Who has the time to implement it?

  Meanwhile, I hope that could work well with repo.

Scott Fan

[Dave Borowitz]

On Thu, Feb 28, 2013 at 3:46 AM, Scott Fan <fanc...@gmail.com> wrote:

Hi guys,

My real purpose is to clone all git projects on the https://gerrit.googlesource.com/ site as my local MIRROR repositories, not only just to get a list of the projects. We do have a lot of ways to achieve this goal, including using the REST API. But all of those, we need write some scripts, or need do a lot of steps.

I guess this qualifies as a script, but looping over all the projects in bash is a one-liner:

for project in $(curl 'https://gerrit.googlesource.com/?format=TEXT'); do

  git clone https://gerrit.googlesource.com/$project

done

You spent more time writing that email than I spent writing this script :)

[Scott Fan]

Dave said good. I do spent too much time writing the emails.

Cloning mirrors just is my first step, the more important for me is to keep mirrors latest ASAP. I prefer repo more than everything.
Maybe I want to be a lazybone, as a result, I became a target. Haha ...

Nobody agree my suggestion, I have to abandon my request here.
Sorry to bother everybody, please close this conversation.

Thanks.

Scott Fan

[Mihai Rusu]

On Thu, Feb 28, 2013 at 07:46:09PM +0800, Scott Fan wrote:
> Hi guys,
>
> My real purpose is to clone all git projects on the
> https://gerrit.googlesource.com/ site as my local MIRROR repositories, not
> only just to get a list of the projects. We do have a lot of ways to
> achieve this goal, including using the REST API. But all of those, we need
> write some scripts, or need do a lot of steps.
>
> However, DO NOT forget the great repo tool, why not to use it?
> With repo, only two commands can finish those, need not to write any extra
> scripts.
> $ repo init -u https://gerrit.googlesource.com/mirror/manifest --mirror
> $ repo sync

I see your point but it's really simple, you'll spend more time trying
to get it your way than just doing this Python script:
>>> import urllib2
>>> import json
>>> headers = {'Accept': 'application/json'}
>>> req = urllib2.Request('https://gerrit-review.googlesource.com/projects/', headers=headers)
>>> result = urllib2.urlopen(req)
>>> result.next() # skip the first line
")]}'\n"
>>> json.load(result).keys()
... list of repos...

> <?xml version="1.0" encoding="UTF-8"?>
> <manifest>
>
> <remote name="gerrit"
> fetch=".." />
> <default revision="master"
> remote="gerrit"
> sync-j="4" />
>
> <project name="executablewar" />
> <project name="gcompute-tools" />
> <project name="gerrit" />
> <project name="gerrit-attic" />
> <project name="git-repo" />
> <project name="gitiles" />
> <project name="gs-maven-wagon" />
> <project name="gwtexpui" />
> <project name="gwtjsonrpc" />
> <project name="gwtorm" />
> <project name="plugins/approval-extension" />
> <project name="plugins/auth-htpasswd" />
> <project name="plugins/avatars/gravatar" />
> <project name="plugins/branch-network" />
> <project name="plugins/change-head" />
> <project name="plugins/change-labels" />
> <project name="plugins/commit-message-length-validator" />
> <project name="plugins/commit-validator-sample" />
> <project name="plugins/delete-project" />
> <project name="plugins/gitblit" />
> <project name="plugins/gitgroups" />
> <project name="plugins/google-apps-group" />
> <project name="plugins/helloworld" />
> <project name="plugins/hooks" />
> <project name="plugins/hooks-audit" />
> <project name="plugins/hooks-bugzilla" />
> <project name="plugins/hooks-its" />
> <project name="plugins/hooks-jira" />
> <project name="plugins/multi-master" />
> <project name="plugins/rename-project" />
> <project name="plugins/replication" />
> <project name="plugins/reviewnotes" />
> <project name="plugins/server-log-viewer" />
> <project name="plugins/singleusergroup" />
> <project name="plugins/zookeeper-refdb" />
> <project name="prolog-cafe" />
> <project name="training/gerrit" />
> <project name="training/sample" />
> <project name="zookeeper-refdb" />
>
> </manifest>


--
Mihai Rusu

[Shawn Pearce]

On Thu, Feb 28, 2013 at 6:13 PM, Mihai Rusu <di...@google.com> wrote:
> On Thu, Feb 28, 2013 at 07:46:09PM +0800, Scott Fan wrote:
>> Hi guys,
>>
>> My real purpose is to clone all git projects on the
>> https://gerrit.googlesource.com/ site as my local MIRROR repositories, not
>> only just to get a list of the projects. We do have a lot of ways to
>> achieve this goal, including using the REST API. But all of those, we need
>> write some scripts, or need do a lot of steps.
>>
>> However, DO NOT forget the great repo tool, why not to use it?
>> With repo, only two commands can finish those, need not to write any extra
>> scripts.
>> $ repo init -u https://gerrit.googlesource.com/mirror/manifest --mirror
>> $ repo sync
>
> I see your point but it's really simple, you'll spend more time trying
> to get it your way than just doing this Python script:
>>>> import urllib2
>>>> import json
>>>> headers = {'Accept': 'application/json'}
>>>> req = urllib2.Request('https://gerrit-review.googlesource.com/projects/', headers=headers)

Or

req = urllib2.Request('https://gerrit.googlesource.com/', headers=headers)

which is usually a bit faster to respond for these sorts of requests.

[Scott Fan]

Mihai, thank you for your advice.

I'm not hating to write scripts myself, I just hate to maintain a pile of scripts written by myself.

In my opinion, the less I write scripts myself, the more convenience to me.

Scott Fan