Report: Gerrit Hackathon #3, May 7-11 2012
조회수 1,039회
읽지 않은 첫 메시지로 건너뛰기
Martin Fick
2012. 5. 21. 오후 5:43:1612. 5. 21.
받는사람 Repo and Gerrit Discussion
Gerrit Hackathon #3, May 7-11 2012
A great thanks goes out to Google for hosting the 3rd Gerrit
Hackathon last week at their Mountain View campus, it was
well attended and very productive. This hackathon was a bit
more focused than previous hackathons which most of us felt
was needed. After everyone had time to meet the new faces
and catch up with each other we sat down and got to business
rather quickly. In the fall Luca suggested that Gerrit
needed plugins to grow its community. Since this seed was
planted, it had been growing in the back of several of the
core devs minds and we quickly agreed to focus on adding a
plugin framework to Gerrit during the upcoming week.
After having made that decision, we discussed a few areas
which could really benefit from plugins (authentication,
hook, events, UI...) and then we identified ssh commands as
a possible quick prototype objective. Almost immediately
thereafter we all started madly hacking towards that end
with the desire to get a prototype completed by the end of
the day or early the next morning. After discovering some
minor pitfalls of intense collaboration using Gerrit, which
we took the time to discuss over lunch, we realized that we
could make good use of the new Gerrit builtin rebase button.
Later that afternoon, the rebase button did indeed come in
handy for many of us to avoid overwriting each other’s
interdependent changes with rebases. But we quickly noticed
that simply having the owner and submitters allowed to
rebase would not be adequate. This led Edwin to propose and
upload a new rebase permission, this has been backported to
be in the 2.4 release since that is the first release with
the new rebase feature.
-
There were other minor Gerrit UI quirks discussed and fixes
proposed for some of these during the following days, for
example: user links now link to the owner:user search
instead of to the user’s dashboard, most of us found that
this is what we really wanted most of the time. Deniz fixed
the annoying lack of reviewer visibility in the case of
draft reviews and his set-account ssh command was also
merged. The former quirk was annoying enough that it lead
to our new IRC headline. Nasser to Shawn:
- Shawn, can you upload that code for review so I can
see how you did that?
- Uh, this code is ugly, I don’t want to display it
to the world.
- How about just uploading it as a draft, then adding
me as a reviewer so I can see it?
- OK. Uploaded, now let me add you as a reviewer.
Hmm your name doesn’t autocomplete, I can’t type
Grainawi!
- Right, that’s the bug that Denitz just uploaded a
fix for, Martin merged it.
- Darn, I think it will be easier for me to just
upgrade gerrit-review with Deniz's fix than to
type your name correctly
Despite Nasser's disappointment, I think he managed to find
this humorous enough to chuckle along with the rest of us.
:)
-
Now back to plugins, while we completed our first prototype
for ssh commands by Monday night, it did not quite work and
we decided to retire anyway since many of the attendees were
struggling to stay awake due to jetlag. But jetlag works
both ways, the previous night's bug was fixed bright and
early before we even assembled the next morning; we were on
our way to a plugin framework. After some further
discussion, we decided to break plugin concepts into 2 main
types: plugins with full access to Gerrit’s internals (and
bound to a specific Gerrit version), and extensions which
would only be bound to light stable interfaces.
More work was done to accommodate both use cases, plugins
and extensions both got their own classloaders, but
extensions were isolated from gerrit classes. Brad and
Gustaf helped create helloworld sample plugins and repos
were created on gerrit-review for several of these. The
intent is to allow users to host plugins alongside Gerrit as
distinct projects. Shawn created two jar files to compile
plugins and extensions against, the plugin jar is one
massive jar with Gerrit and all its dependencies in it.
This large jar file should make it possible for plugin devs
to create plugins quickly without having to download the
many dependencies independently using maven. The extensions
jar file consists mainly of the new fairly sparse top level
gerrit-extensions-api package. Plugins also got their own
Guice injectors. After getting ssh commands working, Shawn
quickly got httpServlet plugin support in Gerrit. Nasser
adapted pegdown so that it could be used by plugin
developers to create lightweight documentation for their
plugins. Shawn crafted a plugin command to be able to
operate on and view plugins in a live Gerrit instance.
With an initial plugin infrastructure in place, initial
rework of the Gerrit core started by expressing part of it
in terms of plugin: Shawn refactored the replication as
plug-in and introduced the ability to create aliases in
order to use plugins commands as surrogates of existing
commands. Luca added the ability to capture JSON/RPC and
SSH actions generate audit-trails and later in the week the
ability to add extra custom commit validation through
plugins.
Additional plugins have been created to provide long awaited
functionalities in Gerrit, such as the delete project
command created by Brad, an on-line server viewer accessible
through a web link contributed by Nasser. As the number of
plugins is expected to grow quickly Luca proposed to define
a simple but published process to allow people to search
through existing plugins and host in Gerrit contributed
plugins and their descriptive wikis.
Allowing plugins to tie into some of the menus (top level,
project/group tabs) by registering links was also proposed
but this has yet to be attempted. Some time was spent
discussing how to go about allowing plugins to determine
approval retention policies after certain actions on a
change. For example: it would be desirable to customize
whether or which approvals get reapplied after using the new
builtin rebase button. This discussion led Shawn to propose
a very generic event extension mechanism to be used by more
than just “approval” extensions. This led to much
infrastructure being built, but there is still work to be
done to make this fully configurable the way intended.
-
The Google facilities are excellent, our workday generally
began with great discussions of the previous day's
achievements and forward looking thoughts for the rest of
the week all while savoring breakfast in one of the many
Google cafeterias. Lunches were similar to breakfast and
often also included general insights into each other's
operational uses of Gerrit. Throughout the day much
amusement was had remarking how cold it was in our
conference room. Some of our European guests took the
opportunity to buy more clothing at the Google store (cold
conference rooms are sadly not new for American workers,
most of us were better prepared/accustomed). Our evenings
consisted mostly of trips to Castro street restaurants in
downtown Mountain View where the food selection and quality
was great.
-
Since we did focus fairly well on our primary task of
creating a plugin infrastructure most of the week (there
were over 10 devs in a room so we should probably be
forgiven for straying a bit here and there), by the last
day there seemed to be a consensus that several other
important Gerrit topics still needed to be discussed. We
took some time to review how topic branches might be well
approached in Gerrit.
The now year old changes under review to add support for
change labels was also discussed eventually. It was
concluded that while these labels are highly coveted, we did
not want to add this additional data to the database since
we are attempting to migrate away from the db. To put this
data into git notes, it will be necessary to have an
indexing system. Shawn decreed that this would need to be
done (or attempted) with Lucene. This could then pave the
path for greater use of Lucene in Gerrit. It is recognized
though that Lucene may need some work to operate smoothly
with minor frequent updates to metadata.
Scalability was naturally on many of our minds also. We
spent some time rehashing many of the performance issues
particular to git and Gerrit. At one point git maintainer
Junio C Hamano (who also works for Google) joined us to
patiently listen to our complaints about the many hacks
being implemented and the many more being proposed to deal
with the git protocol's ref advertisement performance
impacts. Android Gerrit users seem to push many of the
original git use case assumptions beyond their original
intents, especially when it comes to ref counts. We were
delighted to get Junio's approval to propose non backwards
compatible changes to the git protocol to make our use cases
more efficient. Shawn also supported breaking the git
protocol and suggested to only send refs/heads/ and
capabilities in the initial server response. Now someone
"just" needs to step up and do it, Nasser/Martin are
evaluating whether this is something Quic could tackle.
We also spent some time outlining an incremental approach to
making open source Gerrit clusterable (the version running
for android-review and gerrit-review is already clustered
but uses much of Google's proprietary technologies such as
GFS and BigTable). Several incremental steps were outlined
on how to move Gerrit in that direction. The first step
which is needed by most other ideas is to support a very
simple master/master installation of Gerrit where both (or
all if more than 2) masters share a common filesystem
backend (likely a high end NFS server) and a common db.
Two issues were identified here which need to be resolved
before this is possible: 1) Cache coherency and 2) Submit
conflict resolution. A naive approach to #1 is to simply
use really short cache times, but this sort of defeats the
purpose of caching. To solve this properly, some sort of
eviction protocol will need to be developed for masters to
inform their peers of a needed eviction. #2 could be easily
solved by manually determining a submit master and later
upgrading to some sort of voting mechanism among peer
masters to choose a submit master, these would be
incremental approaches. A simple ssh connection between
peers was deemed sufficient in most cases to accomplish both
#1 and #2. A third open problem not actually brought up
during the hackathon is replication from multimasters.
We spent some time discussing ways to make very remote sites
(different continents) with high latencies able to get a web
UI front end only (ssh would be forwarded to the “master”
continent), and found that this would be a lot of work and
ultimately still likely require first solving multi site ref
updates without conflicts since even the UI can update refs
(create branches). So this idea was given up and we moved
on to discussing multi site masters with separate backends.
The main additional problem with separate backends is:
resolving ref updates in a globally safe way. In Google’s
implementation, this is solved by placing the refs in
BigTable. ZooKeeper seemed like a good free/open source
alternative since it is Java based and under the Apache
license. The other piece to solve is moving object data
across sites, it was suggested that ZooKeeper would likely
be involved in helping to coordinate this, but details were
not really discussed. Finally, it was felt that once multi
sites were conquered, that a distributed filesystem may
eventually be needed to scale the git repos effectively,
Hadoop DFS was proposed for this.
So while this is by no means a full run down of the week, it
should give you some insights into the progress which we
made during this week and also into some of the plans for
future Gerrit directions. With the beginnings of a solid
plugin infrastructure now in place, we hope that the Gerrit
community will pick up the task of providing shared plugins
for common tasks and host these new plugins alongside the
gerrit project at the gerrit-review site.
Some Fun Stats
--------------
During the 5 days of the Hackathon, there were (external
contributions counted also):
.115 New Changes Created
.353 New PatchSets Uploaded
.143 Changes Touched
.63 Changes Submitted
During the week since the Hackathon (many carryovers):
.43 Changes submitted
Attendees
---------
The attendees and their sponsors were (as pictured from top
left to bottom right):
.Nasser Grainawi, Qualcomm Innovation Center
.David Borowitz, Google
.Luca Milanesio, GerritForge
.Brad Larson, Garmin
.Deniz Türkoglu, Spotify
.Magnus Bäck, Google
.Gustaf Lundh, Sony Mobile Communications
.Conley Owens, Google
.Saša Živkov, SAP (Submitter)
.Shawn Pearce, Google (Lead / Founder)
.Martin Fick, Qualcomm Innovation Center (Submitter)
.Edwin Kempin, SAP (Submitter)
.Colby Ranger, Google (not pictured)
--
Employee of Qualcomm Innovation Center, Inc. which is a
member of Code Aurora Forum
A great thanks goes out to Google for hosting the 3rd Gerrit
Hackathon last week at their Mountain View campus, it was
well attended and very productive. This hackathon was a bit
more focused than previous hackathons which most of us felt
was needed. After everyone had time to meet the new faces
and catch up with each other we sat down and got to business
rather quickly. In the fall Luca suggested that Gerrit
needed plugins to grow its community. Since this seed was
planted, it had been growing in the back of several of the
core devs minds and we quickly agreed to focus on adding a
plugin framework to Gerrit during the upcoming week.
After having made that decision, we discussed a few areas
which could really benefit from plugins (authentication,
hook, events, UI...) and then we identified ssh commands as
a possible quick prototype objective. Almost immediately
thereafter we all started madly hacking towards that end
with the desire to get a prototype completed by the end of
the day or early the next morning. After discovering some
minor pitfalls of intense collaboration using Gerrit, which
we took the time to discuss over lunch, we realized that we
could make good use of the new Gerrit builtin rebase button.
Later that afternoon, the rebase button did indeed come in
handy for many of us to avoid overwriting each other’s
interdependent changes with rebases. But we quickly noticed
that simply having the owner and submitters allowed to
rebase would not be adequate. This led Edwin to propose and
upload a new rebase permission, this has been backported to
be in the 2.4 release since that is the first release with
the new rebase feature.
-
There were other minor Gerrit UI quirks discussed and fixes
proposed for some of these during the following days, for
example: user links now link to the owner:user search
instead of to the user’s dashboard, most of us found that
this is what we really wanted most of the time. Deniz fixed
the annoying lack of reviewer visibility in the case of
draft reviews and his set-account ssh command was also
merged. The former quirk was annoying enough that it lead
to our new IRC headline. Nasser to Shawn:
- Shawn, can you upload that code for review so I can
see how you did that?
- Uh, this code is ugly, I don’t want to display it
to the world.
- How about just uploading it as a draft, then adding
me as a reviewer so I can see it?
- OK. Uploaded, now let me add you as a reviewer.
Hmm your name doesn’t autocomplete, I can’t type
Grainawi!
- Right, that’s the bug that Denitz just uploaded a
fix for, Martin merged it.
- Darn, I think it will be easier for me to just
upgrade gerrit-review with Deniz's fix than to
type your name correctly
Despite Nasser's disappointment, I think he managed to find
this humorous enough to chuckle along with the rest of us.
:)
-
Now back to plugins, while we completed our first prototype
for ssh commands by Monday night, it did not quite work and
we decided to retire anyway since many of the attendees were
struggling to stay awake due to jetlag. But jetlag works
both ways, the previous night's bug was fixed bright and
early before we even assembled the next morning; we were on
our way to a plugin framework. After some further
discussion, we decided to break plugin concepts into 2 main
types: plugins with full access to Gerrit’s internals (and
bound to a specific Gerrit version), and extensions which
would only be bound to light stable interfaces.
More work was done to accommodate both use cases, plugins
and extensions both got their own classloaders, but
extensions were isolated from gerrit classes. Brad and
Gustaf helped create helloworld sample plugins and repos
were created on gerrit-review for several of these. The
intent is to allow users to host plugins alongside Gerrit as
distinct projects. Shawn created two jar files to compile
plugins and extensions against, the plugin jar is one
massive jar with Gerrit and all its dependencies in it.
This large jar file should make it possible for plugin devs
to create plugins quickly without having to download the
many dependencies independently using maven. The extensions
jar file consists mainly of the new fairly sparse top level
gerrit-extensions-api package. Plugins also got their own
Guice injectors. After getting ssh commands working, Shawn
quickly got httpServlet plugin support in Gerrit. Nasser
adapted pegdown so that it could be used by plugin
developers to create lightweight documentation for their
plugins. Shawn crafted a plugin command to be able to
operate on and view plugins in a live Gerrit instance.
With an initial plugin infrastructure in place, initial
rework of the Gerrit core started by expressing part of it
in terms of plugin: Shawn refactored the replication as
plug-in and introduced the ability to create aliases in
order to use plugins commands as surrogates of existing
commands. Luca added the ability to capture JSON/RPC and
SSH actions generate audit-trails and later in the week the
ability to add extra custom commit validation through
plugins.
Additional plugins have been created to provide long awaited
functionalities in Gerrit, such as the delete project
command created by Brad, an on-line server viewer accessible
through a web link contributed by Nasser. As the number of
plugins is expected to grow quickly Luca proposed to define
a simple but published process to allow people to search
through existing plugins and host in Gerrit contributed
plugins and their descriptive wikis.
Allowing plugins to tie into some of the menus (top level,
project/group tabs) by registering links was also proposed
but this has yet to be attempted. Some time was spent
discussing how to go about allowing plugins to determine
approval retention policies after certain actions on a
change. For example: it would be desirable to customize
whether or which approvals get reapplied after using the new
builtin rebase button. This discussion led Shawn to propose
a very generic event extension mechanism to be used by more
than just “approval” extensions. This led to much
infrastructure being built, but there is still work to be
done to make this fully configurable the way intended.
-
The Google facilities are excellent, our workday generally
began with great discussions of the previous day's
achievements and forward looking thoughts for the rest of
the week all while savoring breakfast in one of the many
Google cafeterias. Lunches were similar to breakfast and
often also included general insights into each other's
operational uses of Gerrit. Throughout the day much
amusement was had remarking how cold it was in our
conference room. Some of our European guests took the
opportunity to buy more clothing at the Google store (cold
conference rooms are sadly not new for American workers,
most of us were better prepared/accustomed). Our evenings
consisted mostly of trips to Castro street restaurants in
downtown Mountain View where the food selection and quality
was great.
-
Since we did focus fairly well on our primary task of
creating a plugin infrastructure most of the week (there
were over 10 devs in a room so we should probably be
forgiven for straying a bit here and there), by the last
day there seemed to be a consensus that several other
important Gerrit topics still needed to be discussed. We
took some time to review how topic branches might be well
approached in Gerrit.
The now year old changes under review to add support for
change labels was also discussed eventually. It was
concluded that while these labels are highly coveted, we did
not want to add this additional data to the database since
we are attempting to migrate away from the db. To put this
data into git notes, it will be necessary to have an
indexing system. Shawn decreed that this would need to be
done (or attempted) with Lucene. This could then pave the
path for greater use of Lucene in Gerrit. It is recognized
though that Lucene may need some work to operate smoothly
with minor frequent updates to metadata.
Scalability was naturally on many of our minds also. We
spent some time rehashing many of the performance issues
particular to git and Gerrit. At one point git maintainer
Junio C Hamano (who also works for Google) joined us to
patiently listen to our complaints about the many hacks
being implemented and the many more being proposed to deal
with the git protocol's ref advertisement performance
impacts. Android Gerrit users seem to push many of the
original git use case assumptions beyond their original
intents, especially when it comes to ref counts. We were
delighted to get Junio's approval to propose non backwards
compatible changes to the git protocol to make our use cases
more efficient. Shawn also supported breaking the git
protocol and suggested to only send refs/heads/ and
capabilities in the initial server response. Now someone
"just" needs to step up and do it, Nasser/Martin are
evaluating whether this is something Quic could tackle.
We also spent some time outlining an incremental approach to
making open source Gerrit clusterable (the version running
for android-review and gerrit-review is already clustered
but uses much of Google's proprietary technologies such as
GFS and BigTable). Several incremental steps were outlined
on how to move Gerrit in that direction. The first step
which is needed by most other ideas is to support a very
simple master/master installation of Gerrit where both (or
all if more than 2) masters share a common filesystem
backend (likely a high end NFS server) and a common db.
Two issues were identified here which need to be resolved
before this is possible: 1) Cache coherency and 2) Submit
conflict resolution. A naive approach to #1 is to simply
use really short cache times, but this sort of defeats the
purpose of caching. To solve this properly, some sort of
eviction protocol will need to be developed for masters to
inform their peers of a needed eviction. #2 could be easily
solved by manually determining a submit master and later
upgrading to some sort of voting mechanism among peer
masters to choose a submit master, these would be
incremental approaches. A simple ssh connection between
peers was deemed sufficient in most cases to accomplish both
#1 and #2. A third open problem not actually brought up
during the hackathon is replication from multimasters.
We spent some time discussing ways to make very remote sites
(different continents) with high latencies able to get a web
UI front end only (ssh would be forwarded to the “master”
continent), and found that this would be a lot of work and
ultimately still likely require first solving multi site ref
updates without conflicts since even the UI can update refs
(create branches). So this idea was given up and we moved
on to discussing multi site masters with separate backends.
The main additional problem with separate backends is:
resolving ref updates in a globally safe way. In Google’s
implementation, this is solved by placing the refs in
BigTable. ZooKeeper seemed like a good free/open source
alternative since it is Java based and under the Apache
license. The other piece to solve is moving object data
across sites, it was suggested that ZooKeeper would likely
be involved in helping to coordinate this, but details were
not really discussed. Finally, it was felt that once multi
sites were conquered, that a distributed filesystem may
eventually be needed to scale the git repos effectively,
Hadoop DFS was proposed for this.
So while this is by no means a full run down of the week, it
should give you some insights into the progress which we
made during this week and also into some of the plans for
future Gerrit directions. With the beginnings of a solid
plugin infrastructure now in place, we hope that the Gerrit
community will pick up the task of providing shared plugins
for common tasks and host these new plugins alongside the
gerrit project at the gerrit-review site.
Some Fun Stats
--------------
During the 5 days of the Hackathon, there were (external
contributions counted also):
.115 New Changes Created
.353 New PatchSets Uploaded
.143 Changes Touched
.63 Changes Submitted
During the week since the Hackathon (many carryovers):
.43 Changes submitted
Attendees
---------
The attendees and their sponsors were (as pictured from top
left to bottom right):
.Nasser Grainawi, Qualcomm Innovation Center
.David Borowitz, Google
.Luca Milanesio, GerritForge
.Brad Larson, Garmin
.Deniz Türkoglu, Spotify
.Magnus Bäck, Google
.Gustaf Lundh, Sony Mobile Communications
.Conley Owens, Google
.Saša Živkov, SAP (Submitter)
.Shawn Pearce, Google (Lead / Founder)
.Martin Fick, Qualcomm Innovation Center (Submitter)
.Edwin Kempin, SAP (Submitter)
.Colby Ranger, Google (not pictured)
--
Employee of Qualcomm Innovation Center, Inc. which is a
member of Code Aurora Forum
Swindells, Thomas
2012. 5. 22. 오전 11:15:3712. 5. 22.
받는사람 Martin Fick, repo-d...@googlegroups.com
I'd just like to say thank you very much for this report.
It was very well written, informative, detailed and entertaining to read.
It looks like you all worked really hard and that there are loads of great features to enjoy soon.
In particular it looks like the plugin support will make it easier for non-gerrit gurus to contribute
and expand the functionality, though I'm sure that will introduce some interesting
challenges on trying to keep track and locate the best plugins. Something like Jenkins
does for tracking their plugins would be good (eg https://wiki.jenkins-ci.org/display/JENKINS/Gerrit+Trigger)
Provides a good level of details on history, usage, change log etc.
Thomas
> --
> To unsubscribe, email repo-discuss...@googlegroups.com
> More info at http://groups.google.com/group/repo-discuss?hl=en
**************************************************************************************
This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postm...@nds.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by NDS for employment and security purposes. To protect the environment please do not print this e-mail unless necessary.
NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18 4EX, United Kingdom. A company registered in England and Wales. Registered no. 3080780. VAT no. GB 603 8808 40-00
**************************************************************************************
It was very well written, informative, detailed and entertaining to read.
It looks like you all worked really hard and that there are loads of great features to enjoy soon.
In particular it looks like the plugin support will make it easier for non-gerrit gurus to contribute
and expand the functionality, though I'm sure that will introduce some interesting
challenges on trying to keep track and locate the best plugins. Something like Jenkins
does for tracking their plugins would be good (eg https://wiki.jenkins-ci.org/display/JENKINS/Gerrit+Trigger)
Provides a good level of details on history, usage, change log etc.
Thomas
> To unsubscribe, email repo-discuss...@googlegroups.com
> More info at http://groups.google.com/group/repo-discuss?hl=en
**************************************************************************************
This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postm...@nds.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by NDS for employment and security purposes. To protect the environment please do not print this e-mail unless necessary.
NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18 4EX, United Kingdom. A company registered in England and Wales. Registered no. 3080780. VAT no. GB 603 8808 40-00
**************************************************************************************
PR
2012. 5. 24. 오전 5:13:5112. 5. 24.
받는사람 repo-d...@googlegroups.com
Hello,
Thanks for this extremely informative account of what appears to have been a very productive week.
Currently we are evaluating the implementation of a cross-repo-dependency solution. The absence of this functionality is something we are struggling with.
I have a number of questions:
(1) Was this topic discussed at all throughout the week?
(2) From an design/implementation perspective, would this functionality be seen by the community as "core code" or something that could be better realised as a plugin?
(3) Related to (1), is this functionality already on a roadmap (somewhere)?
Thanks in advance,
PR.
Thanks for this extremely informative account of what appears to have been a very productive week.
Currently we are evaluating the implementation of a cross-repo-dependency solution. The absence of this functionality is something we are struggling with.
I have a number of questions:
(1) Was this topic discussed at all throughout the week?
(2) From an design/implementation perspective, would this functionality be seen by the community as "core code" or something that could be better realised as a plugin?
(3) Related to (1), is this functionality already on a roadmap (somewhere)?
Thanks in advance,
PR.
Martin Fick
2012. 5. 29. 오후 5:45:0312. 5. 29.
받는사람 repo-d...@googlegroups.com, PR
On Thursday, May 24, 2012 03:13:51 am PR wrote:
> Currently we are evaluating the implementation of a
> cross-repo-dependency solution. The absence of this
> functionality is something we are struggling with.
>
> I have a number of questions:
>
> (1) Was this topic discussed at all throughout the week?
No
> Currently we are evaluating the implementation of a
> cross-repo-dependency solution. The absence of this
> functionality is something we are struggling with.
>
> I have a number of questions:
>
> (1) Was this topic discussed at all throughout the week?
> (2) From an design/implementation perspective, would this
> functionality be seen by the community as "core code" or
> something that could be better realised as a plugin?
> (3) Related to (1), is this functionality already on a
> roadmap (somewhere)?
currently. The closest thing to it was this series:
https://gerrit-
review.googlesource.com/#/q/status:open+project:gerrit+branch:master+topic:cross-
repo-dependencies,n,z
-Martin
Chad Horohoe
2012. 5. 29. 오후 6:31:5412. 5. 29.
받는사람 Repo and Gerrit Discussion, Martin Fick
On Tue, May 29, 2012 at 5:45 PM, Martin Fick <mf...@codeaurora.org> wrote:
> On Thursday, May 24, 2012 03:13:51 am PR wrote:
>> (3) Related to (1), is this functionality already on a
>> roadmap (somewhere)?
>
> On roadmap, no one actively working on it that I know of
> currently. The closest thing to it was this series:
>
> https://gerrit-
> review.googlesource.com/#/q/status:open+project:gerrit+branch:master+topic:cross-
> repo-dependencies,n,z
>
Speaking of roadmaps--is there a page somewhere that outlines
> On Thursday, May 24, 2012 03:13:51 am PR wrote:
>> (3) Related to (1), is this functionality already on a
>> roadmap (somewhere)?
>
> On roadmap, no one actively working on it that I know of
> currently. The closest thing to it was this series:
>
> https://gerrit-
> review.googlesource.com/#/q/status:open+project:gerrit+branch:master+topic:cross-
> repo-dependencies,n,z
>
the current roadmap?
-Chad
Bruce Zu
2012. 6. 11. 오전 2:29:3712. 6. 11.
받는사람 Repo and Gerrit Discussion
some notes of Martin's report of Gerrit Hackathon #3, May 7-11 2012
which giving you some insights into the progress and also into some of
the 3rd Gerrit Hackathon focus on adding a plugin framework, in a few
areas
-replication
-hook
-events
-ssh commands.
...
beginnings of a solid plugin infrastructure now in place.
Plugin concepts have 2 main types:
-plugins with full access to Gerrit’s internals (and bound to a
specific Gerrit version),
-extensions which would only be bound to light stable interfaces.
Plugins and extensions both got their own classloaders, but extensions
The plugin jar is one massive jar with Gerrit and all its dependencies
-Plugins own Guice injectors.
-httpServlet plugin support in Gerrit.
-adapted pegdown so that it could be used by plugin developers to
Gerrit instance.
rework of the Gerrit core:
-refactored the replication as plug-in
-introduced the ability to create aliases in order to use plugins
trails
-the ability to add extra custom commit validation through plugins.
another interesting items:
a very generic event extension mechanism to be used by more than just
“approval” extensions. This led to much
infrastructure being built, but there is still work to be done to make
this fully configurable the way intended.
...
...
--------------------------------
discuss old changes under review:
add support for change labels:
get Junio's approval to propose non backwards compatible changes to
the git protocol to make our use cases
more efficient. Shawn also supported breaking the git protocol and
suggested to only send refs/heads/ and capabilities in the initial
server response. Now
someone "just" needs to step up and do it, Nasser/Martin are
evaluating whether this is something Quic could tackle.
outline an incremental approach to making open source Gerrit
#1 Cache coherency
--some sort of eviction protocol will need to be developed for masters
-- could be easily solved by manually determining a submit master and
discuss ways to make very remote sites (different continents) with
move to discuss multi site masters with separate backends whose
problem is:
resolving ref updates in a globally safe way.
Google's implementation is placing the refs in BigTable. ZooKeeper
seemed like a good free/open source alternative.
which giving you some insights into the progress and also into some of
the plans for future Gerrit directions.
--------------------------------
the 3rd Gerrit Hackathon focus on adding a plugin framework, in a few
areas
-replication
-hook
-events
-ssh commands.
...
beginnings of a solid plugin infrastructure now in place.
Plugin concepts have 2 main types:
-plugins with full access to Gerrit’s internals (and bound to a
specific Gerrit version),
-extensions which would only be bound to light stable interfaces.
Plugins and extensions both got their own classloaders, but extensions
were isolated from gerrit classes.
Two jar files are created to compile plugins and extensions against.
The plugin jar is one massive jar with Gerrit and all its dependencies
in it.
This large jar file should make it possible for plugin devs
to create plugins quickly without having to download the
many dependencies independently using maven.
The extensions jar file consists mainly of the new fairly sparse top
level
gerrit-extensions-api package.
initial plugin infrastructure
This large jar file should make it possible for plugin devs
to create plugins quickly without having to download the
many dependencies independently using maven.
The extensions jar file consists mainly of the new fairly sparse top
level
gerrit-extensions-api package.
-Plugins own Guice injectors.
-httpServlet plugin support in Gerrit.
-adapted pegdown so that it could be used by plugin developers to
create lightweight documentation for their plugins.
-a plugin command to be able to operate on and view plugins in a live
Gerrit instance.
rework of the Gerrit core:
-refactored the replication as plug-in
-introduced the ability to create aliases in order to use plugins
commands as surrogates of existing commands.
-added the ability to capture JSON/RPC and SSH actions generate audit-
trails
-the ability to add extra custom commit validation through plugins.
another interesting items:
a very generic event extension mechanism to be used by more than just
“approval” extensions. This led to much
infrastructure being built, but there is still work to be done to make
this fully configurable the way intended.
...
--------------------------------
discuss old changes under review:
add support for change labels:
we did not want to add this additional data to the database since we
are attempting to migrate away from the db.
To put this data into git notes, it will be necessary to have an
indexing system.
Shawn decreed that this would need to be done (or attempted) with
Lucene. This could then pave the path for greater use of Lucene in
Gerrit. It is
recognized though that Lucene may need some work to operate smoothly
with minor frequent updates to metadata.
Scalability:
are attempting to migrate away from the db.
To put this data into git notes, it will be necessary to have an
indexing system.
Shawn decreed that this would need to be done (or attempted) with
Lucene. This could then pave the path for greater use of Lucene in
Gerrit. It is
recognized though that Lucene may need some work to operate smoothly
with minor frequent updates to metadata.
get Junio's approval to propose non backwards compatible changes to
the git protocol to make our use cases
more efficient. Shawn also supported breaking the git protocol and
suggested to only send refs/heads/ and capabilities in the initial
server response. Now
someone "just" needs to step up and do it, Nasser/Martin are
evaluating whether this is something Quic could tackle.
clusterable (the version running for android-review and gerrit-review
is already clustered
but uses much of Google's proprietary technologies such as GFS and
BigTable). The first step which is needed by most other ideas is to
is already clustered
but uses much of Google's proprietary technologies such as GFS and
support a very
simple master/master installation of Gerrit where both (or
all if more than 2) masters share a common filesystem backend (likely
a high end NFS server) and a common db.
Two issues need to be resolved before this is possible:
simple master/master installation of Gerrit where both (or
all if more than 2) masters share a common filesystem backend (likely
a high end NFS server) and a common db.
#1 Cache coherency
--some sort of eviction protocol will need to be developed for masters
to inform their peers of a needed eviction.
#2 Submit conflict resolution.
-- could be easily solved by manually determining a submit master and
later
upgrading to some sort of voting mechanism among peer masters to
choose a submit master, these would be
incremental approaches.
A simple ssh connection between peers was deemed sufficient in most
cases to accomplish both #1 and #2.
#3 replication from multimasters.
upgrading to some sort of voting mechanism among peer masters to
choose a submit master, these would be
incremental approaches.
A simple ssh connection between peers was deemed sufficient in most
cases to accomplish both #1 and #2.
discuss ways to make very remote sites (different continents) with
high latencies able to get a web
UI front end only (ssh would be forwarded to the “master”
continent),
...require first solving multi site ref updates without conflicts.
UI front end only (ssh would be forwarded to the “master”
continent),
move to discuss multi site masters with separate backends whose
problem is:
resolving ref updates in a globally safe way.
seemed like a good free/open source alternative.
The other piece to solve is moving object data across sites, it was
suggested that ZooKeeper would likely be involved in helping to
coordinate this, but details were not really discussed.
Finally, a distributed filesystem may eventually be needed to scale
suggested that ZooKeeper would likely be involved in helping to
coordinate this, but details were not really discussed.
전체답장
작성자에게 답글
전달
새 메시지 0개