[2.8] Error when pushing config or rebasing through UI and jenkins communication
591 views
Skip to first unread message
Grégory Bougeard
Dec 6, 2013, 9:22:40 AM12/6/13
to repo-d...@googlegroups.com
Hi,
we just updated our gerrit installation from 2.6.1 to 2.8 and now I'm encountering some troubles :
- I' m getting remote: error: internal error while processing changes java.lang.NoSuchMethodError: com.google.gerrit.extensions.events.GitReferenceUpdatedListener$Event.getUpdates()Ljava/util/List;
when I push to HEAD:refs/meta/config
- I'm getting 500 com.google.gerrit.extensions.events.GitReferenceUpdatedListener$Event.getUpdates()Ljava/util/List; when rebasing a change through the UI.
- The communication between gerrit & jenkins seems broken : gerrit trigger does not work anymore.
Did I miss something while updating?
Grégory Bougeard
Dec 6, 2013, 9:31:08 AM12/6/13
to repo-d...@googlegroups.com
Here is the full stack for the failed rebase :
[2013-12-06 15:29:45,564] WARN org.eclipse.jetty.servlet.ServletHandler : Error for /changes/34214/revisions/45894848ffcc60353301759d92a7478a93d7dbe0/rebase java.lang.NoSuchMethodError: com.google.gerrit.extensions.events.GitReferenceUpdatedListener$Event.getUpdates()Ljava/util/List; at com.googlesource.gerrit.plugins.branchnetwork.data.GitCommitCacheRefresh.onGitReferenceUpdated(GitCommitCacheRefresh.java:49) at com.google.gerrit.server.extensions.events.GitReferenceUpdated.fire(GitReferenceUpdated.java:59) at com.google.gerrit.server.extensions.events.GitReferenceUpdated.fire(GitReferenceUpdated.java:48) at com.google.gerrit.server.change.PatchSetInserter.insert(PatchSetInserter.java:228) at com.google.gerrit.server.changedetail.RebaseChange.rebase(RebaseChange.java:313) at com.google.gerrit.server.changedetail.RebaseChange.rebase(RebaseChange.java:134) at com.google.gerrit.server.change.Rebase.apply(Rebase.java:67) at com.google.gerrit.server.change.Rebase.apply(Rebase.java:39) at com.google.gerrit.httpd.restapi.RestApiServlet.service(RestApiServlet.java:304) at javax.servlet.http.HttpServlet.service(HttpServlet.java:722) at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:278) at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:268) at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:180) at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:93) at com.google.gwtexpui.server.CacheControlFilter.doFilter(CacheControlFilter.java:70) at com.google.gerrit.httpd.RunAsFilter.doFilter(RunAsFilter.java:113) at com.google.gerrit.httpd.RequireSslFilter.doFilter(RequireSslFilter.java:68) at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:64) at com.google.gerrit.httpd.AllRequestFilter$FilterProxy.doFilter(AllRequestFilter.java:57) at com.google.gerrit.httpd.RequestContextFilter.doFilter(RequestContextFilter.java:75) at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:120) at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:132) at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:129) at com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:206) at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:129) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:365) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:856) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:627) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:51) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:701) [2013-12-06 15:29:45,564] ERROR com.google.gerrit.pgm.http.jetty.HiddenErrorHandler : Error in POST /changes/34214/revisions/45894848ffcc60353301759d92a7478a93d7dbe0/rebase java.lang.NoSuchMethodError: com.google.gerrit.extensions.events.GitReferenceUpdatedListener$Event.getUpdates()Ljava/util/List; at com.googlesource.gerrit.plugins.branchnetwork.data.GitCommitCacheRefresh.onGitReferenceUpdated(GitCommitCacheRefresh.java:49) at com.google.gerrit.server.extensions.events.GitReferenceUpdated.fire(GitReferenceUpdated.java:59) at com.google.gerrit.server.extensions.events.GitReferenceUpdated.fire(GitReferenceUpdated.java:48) at com.google.gerrit.server.change.PatchSetInserter.insert(PatchSetInserter.java:228) at com.google.gerrit.server.changedetail.RebaseChange.rebase(RebaseChange.java:313) at com.google.gerrit.server.changedetail.RebaseChange.rebase(RebaseChange.java:134) at com.google.gerrit.server.change.Rebase.apply(Rebase.java:67) at com.google.gerrit.server.change.Rebase.apply(Rebase.java:39) at com.google.gerrit.httpd.restapi.RestApiServlet.service(RestApiServlet.java:304) at javax.servlet.http.HttpServlet.service(HttpServlet.java:722) at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:278) at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:268) at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:180) at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:93) at com.google.gwtexpui.server.CacheControlFilter.doFilter(CacheControlFilter.java:70) at com.google.gerrit.httpd.RunAsFilter.doFilter(RunAsFilter.java:113) at com.google.gerrit.httpd.RequireSslFilter.doFilter(RequireSslFilter.java:68) at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:64) at com.google.gerrit.httpd.AllRequestFilter$FilterProxy.doFilter(AllRequestFilter.java:57) at com.google.gerrit.httpd.RequestContextFilter.doFilter(RequestContextFilter.java:75) at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:120) at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:132) at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:129) at com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:206) at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:129) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:365) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:856) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:627) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:51) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:701) [2013-12-06 15:29:45,879] WARN com.google.gerrit.sshd.GerritServerSession : Exception caught java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcher.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:251) at sun.nio.ch.IOUtil.read(IOUtil.java:224) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:254) at org.apache.mina.transport.socket.nio.NioProcessor.read(NioProcessor.java:273) at org.apache.mina.transport.socket.nio.NioProcessor.read(NioProcessor.java:44) at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:690) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:664) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:653) at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:67) at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1124) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:701)
David Pursehouse
Dec 6, 2013, 10:01:37 AM12/6/13
to Grégory Bougeard, repo-discuss
[replying again with repo-discuss in the loop this time]
For the Jenkins trigger plugin, you need to grant the Stream Events capability [1] to the group that the Jenkins user belongs to. This was introduced in 2.7 so if you're updating directly from 2.6.1 to 2.8 you might have missed that.
--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en
---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Grégory Bougeard
Dec 6, 2013, 10:08:03 AM12/6/13
to repo-d...@googlegroups.com, Grégory Bougeard
Yes, I removed the branch-network plugin and I created a group with the Stream Events capability .
I now see jenkins trying to send message to gerrit :
Dec 06, 2013 4:03:44 PM INFO com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.ToGerritRunListener onRetriggered
Project [GERRIT-it-xsl] re-triggered by Gerrit-User: [[ManualPatchsetCreated Change: Change: 34239 PatchSet: PatchSet: 1]]
Dec 06, 2013 4:03:44 PM INFO com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritTrigger schedule
Project GERRIT-it-xsl Build Scheduled: true By event: 34239/1
Dec 06, 2013 4:03:44 PM INFO com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.ToGerritRunListener onStarted
Gerrit build [GERRIT-it-xsl #15] Started for cause: [GerritCause: [ManualPatchsetCreated Change: Change: 34239 PatchSet: PatchSet: 1] silent: false].
but on Gerrit logs I'm getting IOExceptions :
[2013-12-06 16:05:08,364] WARN com.google.gerrit.sshd.GerritServerSession : Exception caught java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcher.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:251) at sun.nio.ch.IOUtil.read(IOUtil.java:224) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:254) at org.apache.mina.transport.socket.nio.NioProcessor.read(NioProcessor.java:273) at org.apache.mina.transport.socket.nio.NioProcessor.read(NioProcessor.java:44) at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:690) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:664) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:653) at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:67) at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1124) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:701)
David Pursehouse
Dec 6, 2013, 10:29:05 AM12/6/13
to Grégory Bougeard, repo-discuss
The connection reset by peer issue has been reported before [1] on 2.6 and 2.7
Not really sure if the solution described there is relevant if you've already added the Jenkins user in the group that has StreamEvents capability. Are there other users trying to use the event stream, without having the capability?
Grégory Bougeard
Dec 6, 2013, 10:37:23 AM12/6/13
to repo-d...@googlegroups.com, Grégory Bougeard
On the IRC channel dougk told me I needed this commit in the jenkins plugins :
In fact, I just manually change the deprecated approve by review in the Gerrit Verified Commands.
And it solved it!
David Pursehouse
Dec 6, 2013, 10:46:46 AM12/6/13
to Grégory Bougeard, repo-discuss
On Sat, Dec 7, 2013 at 12:37 AM, Grégory Bougeard <gbou...@gmail.com> wrote:
On the IRC channel dougk told me I needed this commit in the jenkins plugins :In fact, I just manually change the deprecated approve by review in the Gerrit Verified Commands.And it solved it!
Ah, yes. The 'approve' alias was deprecated several releases ago, but was not actually removed until 2.8
There should have been a warning about this in the release notes.
Joshua J. Kugler
Dec 6, 2013, 4:06:51 PM12/6/13
to repo-d...@googlegroups.com, David Pursehouse, Grégory Bougeard
On Saturday, December 07, 2013 00:46:46 David Pursehouse wrote:
> > In fact, I just manually change the deprecated approve by review in the
> > Gerrit Verified Commands.
> > And it solved it!
>
> Ah, yes. The 'approve' alias was deprecated several releases ago, but was
> not actually removed until 2.8
>
> There should have been a warning about this in the release notes.
I tried this command on 2.8rc3, and it worked fine. Was it removed between rc3
> > In fact, I just manually change the deprecated approve by review in the
> > Gerrit Verified Commands.
> > And it solved it!
>
> Ah, yes. The 'approve' alias was deprecated several releases ago, but was
> not actually removed until 2.8
>
> There should have been a warning about this in the release notes.
and release? I do not have lots of software experience, but isn't it common
that if no bugs are found in the last rc, then it it released, as is? If
there were changes between rc3 and release, should there not have been rc4?
j
--
Joshua J. Kugler - Fairbanks, Alaska
Azariah Enterprises - Programming and Website Design
jos...@azariah.com - Jabber: peda...@gmail.com
PGP Key: http://pgp.mit.edu/ ID 0x73B13B6A
David Pursehouse
Dec 11, 2013, 7:37:34 PM12/11/13
to Joshua J. Kugler, repo-d...@googlegroups.com, David Pursehouse, Grégory Bougeard
On 12/07/2013 06:06 AM, Joshua J. Kugler wrote:
> On Saturday, December 07, 2013 00:46:46 David Pursehouse wrote:
>>> In fact, I just manually change the deprecated approve by review in the
>>> Gerrit Verified Commands.
>>> And it solved it!
>>
>> Ah, yes. The 'approve' alias was deprecated several releases ago, but was
>> not actually removed until 2.8
>>
>> There should have been a warning about this in the release notes.
>
> I tried this command on 2.8rc3, and it worked fine. Was it removed between rc3
> and release?
>
Yes, the change went into stable-2.8 between rc3 and the final build,
> On Saturday, December 07, 2013 00:46:46 David Pursehouse wrote:
>>> In fact, I just manually change the deprecated approve by review in the
>>> Gerrit Verified Commands.
>>> And it solved it!
>>
>> Ah, yes. The 'approve' alias was deprecated several releases ago, but was
>> not actually removed until 2.8
>>
>> There should have been a warning about this in the release notes.
>
> I tried this command on 2.8rc3, and it worked fine. Was it removed between rc3
> and release?
>
along with a few others.
> I do not have lots of software experience, but isn't it common
> that if no bugs are found in the last rc, then it it released, as is? If
> there were changes between rc3 and release, should there not have been rc4?
>
sure there was a clearer warning in the release notes. The release
notes have since been updated to include such a warning.
Sorry if this has caused inconvenience.
Joshua J. Kugler
Dec 11, 2013, 8:01:41 PM12/11/13
to David Pursehouse, repo-d...@googlegroups.com, David Pursehouse, Grégory Bougeard
On Thursday, December 12, 2013 09:37:34 David Pursehouse wrote:
> > I tried this command on 2.8rc3, and it worked fine. Was it removed between
> > rc3 and release?
>
> Yes, the change went into stable-2.8 between rc3 and the final build,
> along with a few others.
>
> > I do not have lots of software experience, but isn't it common
> > that if no bugs are found in the last rc, then it it released, as is? If
> > there were changes between rc3 and release, should there not have been
> > rc4?
>
> You have a valid point. I should have either made another RC, or made
> sure there was a clearer warning in the release notes. The release
> notes have since been updated to include such a warning.
>
> Sorry if this has caused inconvenience.
Not a lot of inconvenience, just a bit of surprise, and a bit of rescheduling
> > I tried this command on 2.8rc3, and it worked fine. Was it removed between
> > rc3 and release?
>
> Yes, the change went into stable-2.8 between rc3 and the final build,
> along with a few others.
>
> > I do not have lots of software experience, but isn't it common
> > that if no bugs are found in the last rc, then it it released, as is? If
> > there were changes between rc3 and release, should there not have been
> > rc4?
>
> You have a valid point. I should have either made another RC, or made
> sure there was a clearer warning in the release notes. The release
> notes have since been updated to include such a warning.
>
> Sorry if this has caused inconvenience.
as we have to update another application before the Gerrit upgrade, something
we weren't expecting to do right now. Didn't expect a command that worked in
rc3 to not work in release. :) Thank you for following up.
Marc
Nov 14, 2014, 1:18:52 PM11/14/14
to repo-d...@googlegroups.com, gbou...@gmail.com
I didn't see a warning anywhere about this and I lost several hours because all I could find was about the stream events and nothing about the "approve" changed into "review"... All I had to do was edit the gerrit-trigger.xml in my jenkins home folder but as all I was getting was the following, it was hard to figure out:
WARN com.google.gerrit.sshd.GerritServerSession : Exception caught
java.io.IOException: Connection reset by peer
Thank you for this thread, I think a big warning could still be added somewhere for people who upgrade.
Edwin Kempin
Nov 16, 2014, 12:20:50 PM11/16/14
to Marc, Repo and Gerrit Discussion, Grégory Bougeard
2014-11-14 19:18 GMT+01:00 Marc <marc.de...@googlemail.com>:
I didn't see a warning anywhere about this and I lost several hours because all I could find was about the stream events and nothing about the "approve" changed into "review"... All I had to do was edit the gerrit-trigger.xml in my jenkins home folder but as all I was getting was the following, it was hard to figure out:
WARN com.google.gerrit.sshd.GerritServerSession : Exception caught
java.io.IOException: Connection reset by peer
Thank you for this thread, I think a big warning could still be added somewhere for people who upgrade.
Actually by now there is a warning about this in the 2.8 release notes [1]:
"WARNING: The deprecated approve alias for the review SSH command has been removed. This is important for all users of the Jenkins Gerrit Trigger Plugin since this plugin by default uses the approve command to vote and comment on changes in Gerrit. If you use the Gerrit Trigger Plugin, go to its global configuration in Jenkins and adapt the Gerrit commands to use the review command instead of the approve command."
[1] https://gerrit-documentation.storage.googleapis.com/ReleaseNotes/ReleaseNotes-2.8.html
"WARNING: The deprecated approve alias for the review SSH command has been removed. This is important for all users of the Jenkins Gerrit Trigger Plugin since this plugin by default uses the approve command to vote and comment on changes in Gerrit. If you use the Gerrit Trigger Plugin, go to its global configuration in Jenkins and adapt the Gerrit commands to use the review command instead of the approve command."
[1] https://gerrit-documentation.storage.googleapis.com/ReleaseNotes/ReleaseNotes-2.8.html
On Friday, 6 December 2013 16:46:46 UTC+1, David Pursehouse wrote:On Sat, Dec 7, 2013 at 12:37 AM, Grégory Bougeard <gbou...@gmail.com> wrote:
On the IRC channel dougk told me I needed this commit in the jenkins plugins :In fact, I just manually change the deprecated approve by review in the Gerrit Verified Commands.And it solved it!Ah, yes. The 'approve' alias was deprecated several releases ago, but was not actually removed until 2.8There should have been a warning about this in the release notes.
--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en
---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages