[Announce] Gerrit Analytics are now online

676 views
Skip to first unread message

Luca Milanesio

unread,
Jan 2, 2018, 8:41:18 AM1/2/18
to Repo and Gerrit Discussion
Hi all,
I am pleased to announce the availability of the Gerrit Analytics web-site at:
https://gerrit-analytics.gerritforge.com.

Read the full announcement at:
https://gitenterprise.me/2018/01/02/gerrit-analytics/

I am requesting the agreement to the community to officially create the Spark Job component that powers the analytics pipeline to the gerrit-review site:

Project name: apps/analytics-etl
Project Description: Spark ETL to extra analytics data from Gerrit Projects using the Analytics plugin

The Spark Job extracts on a regular basic the aggregated statistics of the commits on all the Gerrit Code Review related repositories, including all the plugins.

It automatically manages the aggregation on:
- organizations
- authors with different emails
- year / month / day / hour
- project name
- commit size

You can either display the data graphically on the Kibana dashboards (see below the default one):


It should be easy from now on to extract statistics about the Gerrit Code Review projects .... just use the Kibana UX to select the interval, projects and you get nice and colourful graphs :-)
You can as well extract the raw data and format differently.

The dashboards are automatically updated using the latest commit data coming from Gerrit @Gerrit-Review.

Have a look and let us know what you think :-)

Luca.

Edwin Kempin

unread,
Jan 2, 2018, 9:00:00 AM1/2/18
to Luca Milanesio, Repo and Gerrit Discussion
Thanks Luca!
Finally statistics across all Gerrit-related projects. Looks great!

--
--
To unsubscribe, email repo-discuss+unsubscribe@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Luca Milanesio

unread,
Jan 4, 2018, 2:58:50 AM1/4/18
to Edwin Kempin, Repo and Gerrit Discussion
Thanks Edwin, there are some more tweaks to do. Most urgently the differentiation between merge and non-merge commits :-)

Any feedback (agreement or disagreement) on creating the analytics-etl project on Gerrit-Review?

Project name: apps/analytics-etl
Project Description: Spark ETL to extra analytics data from Gerrit Projects using the Analytics plugin

Thanks.

Luca.

On 2 Jan 2018, at 13:59, Edwin Kempin <eke...@google.com> wrote:

Thanks Luca!
Finally statistics across all Gerrit-related projects. Looks great!
On Tue, Jan 2, 2018 at 2:40 PM, Luca Milanesio <luca.mi...@gmail.com> wrote:
Hi all,
I am pleased to announce the availability of the Gerrit Analytics web-site at:
https://gerrit-analytics.gerritforge.com.

Read the full announcement at:
https://gitenterprise.me/2018/01/02/gerrit-analytics/

I am requesting the agreement to the community to officially create the Spark Job component that powers the analytics pipeline to the gerrit-review site:

Project name: apps/analytics-etl
Project Description: Spark ETL to extra analytics data from Gerrit Projects using the Analytics plugin

The Spark Job extracts on a regular basic the aggregated statistics of the commits on all the Gerrit Code Review related repositories, including all the plugins.

It automatically manages the aggregation on:
- organizations
- authors with different emails
- year / month / day / hour
- project name
- commit size

You can either display the data graphically on the Kibana dashboards (see below the default one):

<Screen Shot 2018-01-02 at 13.36.53.png>

It should be easy from now on to extract statistics about the Gerrit Code Review projects .... just use the Kibana UX to select the interval, projects and you get nice and colourful graphs :-)
You can as well extract the raw data and format differently.

The dashboards are automatically updated using the latest commit data coming from Gerrit @Gerrit-Review.

Have a look and let us know what you think :-)

Luca.

Edwin Kempin

unread,
Jan 4, 2018, 3:12:43 AM1/4/18
to Luca Milanesio, Repo and Gerrit Discussion
On Thu, Jan 4, 2018 at 8:58 AM, Luca Milanesio <luca.mi...@gmail.com> wrote:
Thanks Edwin, there are some more tweaks to do. Most urgently the differentiation between merge and non-merge commits :-)
Ah, that differentiation would be interesting indeed .
 

Any feedback (agreement or disagreement) on creating the analytics-etl project on Gerrit-Review?

Project name: apps/analytics-etl
Project Description: Spark ETL to extra analytics data from Gerrit Projects using the Analytics plugin
As long as it has a friendly licence it would be nice to host it on gerrit-review.
As maintainer you can create the project yourself :-)
Just make sure it inherits from Public-Projects.

Luca Milanesio

unread,
Jan 4, 2018, 3:20:26 AM1/4/18
to Edwin Kempin, Repo and Gerrit Discussion

On 4 Jan 2018, at 08:11, Edwin Kempin <eke...@google.com> wrote:



On Thu, Jan 4, 2018 at 8:58 AM, Luca Milanesio <luca.mi...@gmail.com> wrote:
Thanks Edwin, there are some more tweaks to do. Most urgently the differentiation between merge and non-merge commits :-)
Ah, that differentiation would be interesting indeed .
 

Any feedback (agreement or disagreement) on creating the analytics-etl project on Gerrit-Review?

Project name: apps/analytics-etl
Project Description: Spark ETL to extra analytics data from Gerrit Projects using the Analytics plugin
As long as it has a friendly licence it would be nice to host it on gerrit-review.

Yes, it is Apache 2.0 and includes everything we used for creating gerrit-analytics.gerritforge.com.
It means, anyone could create their "in-house Analytics" on Gerrit.

As maintainer you can create the project yourself :-)

Oh yes, I forgot that I joined the "force" now :-)

Just make sure it inherits from Public-Projects.

Yes, will do. Thanks.

Duft Markus

unread,
Jan 8, 2018, 1:29:26 AM1/8/18
to Luca Milanesio, Repo and Gerrit Discussion

Hey,

 

This looks interesting indeed. Is there a docker container or something similar somewhere I could use as a starting point if I wanted to create something similar for our in-house Gerrit?

 

Cheers,

Markus

--
--
To unsubscribe, email repo-discuss...@googlegroups.com


More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.


For more options, visit https://groups.google.com/d/optout.


SSI Schäfer IT Solutions GmbH | Friesachstrasse 15 | 8114 Friesach | Austria
Registered Office: Friesach | Commercial Register: 49324 K | VAT no. ATU28654300
Commercial Court: Landesgericht für Zivilrechtssachen Graz

Luca Milanesio

unread,
Jan 8, 2018, 3:52:17 AM1/8/18
to Duft Markus, Repo and Gerrit Discussion

On 8 Jan 2018, at 06:29, Duft Markus <Marku...@ssi-schaefer.com> wrote:

Hey,
 
This looks interesting indeed. Is there a docker container or something similar somewhere I could use as a starting point if I wanted to create something similar for our in-house Gerrit?

Yes, it is 100% Docker-ized and the source can be found on the Gerrit-Review site:

You can contribute as well, as with any other Gerrit-related project under the Apache 2.0 license.

Luca.

 
Cheers,
Markus
 
From: repo-d...@googlegroups.com [mailto:repo-d...@googlegroups.com] On Behalf Of Luca Milanesio
Sent: Tuesday, January 2, 2018 2:41 PM
To: Repo and Gerrit Discussion <repo-d...@googlegroups.com>
Subject: [Announce] Gerrit Analytics are now online
 
Hi all,
I am pleased to announce the availability of the Gerrit Analytics web-site at:
https://gerrit-analytics.gerritforge.com.

Read the full announcement at:
https://gitenterprise.me/2018/01/02/gerrit-analytics/

I am requesting the agreement to the community to officially create the Spark Job component that powers the analytics pipeline to the gerrit-review site:
 
Project name: apps/analytics-etl
Project Description: Spark ETL to extra analytics data from Gerrit Projects using the Analytics plugin
 
The Spark Job extracts on a regular basic the aggregated statistics of the commits on all the Gerrit Code Review related repositories, including all the plugins.
 
It automatically manages the aggregation on:
- organizations
- authors with different emails
- year / month / day / hour
- project name
- commit size
 
You can either display the data graphically on the Kibana dashboards (see below the default one):
 
<image001.png>
 
It should be easy from now on to extract statistics about the Gerrit Code Review projects .... just use the Kibana UX to select the interval, projects and you get nice and colourful graphs :-)
You can as well extract the raw data and format differently.
 
The dashboards are automatically updated using the latest commit data coming from Gerrit @Gerrit-Review.
 
Have a look and let us know what you think :-)
 
Luca.
-- 
-- 
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Duft Markus

unread,
Jan 8, 2018, 4:09:12 AM1/8/18
to Luca Milanesio, Repo and Gerrit Discussion

Perfect, thanks a lot!

Patrick Hiesel

unread,
Jan 8, 2018, 11:42:35 AM1/8/18
to Duft Markus, Luca Milanesio, Repo and Gerrit Discussion
Really cool, Luca!!

Is there a way to exclude merge commits or filter them out?

-- 
-- 
To unsubscribe, email repo-discuss+unsub...@googlegroups.com


More info at http://groups.google.com/group/repo-discuss?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss+unsubscribe@googlegroups.com.


For more options, visit https://groups.google.com/d/optout.


SSI Schäfer IT Solutions GmbH | Friesachstrasse 15 | 8114 Friesach | Austria
Registered Office: Friesach | Commercial Register: 49324 K | VAT no. ATU28654300
Commercial Court: Landesgericht für Zivilrechtssachen Graz


SSI Schäfer IT Solutions GmbH | Friesachstrasse 15 | 8114 Friesach | Austria
Registered Office: Friesach | Commercial Register: 49324 K | VAT no. ATU28654300
Commercial Court: Landesgericht für Zivilrechtssachen Graz

--
--
To unsubscribe, email repo-discuss+unsubscribe@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss+unsubscribe@googlegroups.com.

Luca Milanesio

unread,
Jan 8, 2018, 12:10:48 PM1/8/18
to Patrick Hiesel, Duft Markus, Repo and Gerrit Discussion
We are *exactly* working on that and a new Change on the app/analytics-etl should pop-up in a few hours :-)

Another important thing we are doing is the group-by-branch of the contributions.
So that you could ask "how big is 2.15 compared to 2.14?"

And you could get:
- number of commits
- size of the commits
- number of impacted files 

:-)

Makson Lee

unread,
Jan 18, 2018, 3:58:21 AM1/18/18
to Repo and Gerrit Discussion
our internal gerrit instance is username and password protected, we got the following error while running spark ETL job :(

Exception in thread "main" java.io.IOException: Server returned HTTP response code: 401 for URL: http://gerrit:8080/a/projects/
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
        at java.net.URL.openStream(URL.java:1045)
        at scala.io.Source$.fromURL(Source.scala:141)
        at scala.io.Source$.fromURL(Source.scala:131)
        at com.gerritforge.analytics.job.Job$class.run(Main.scala:85)
        at com.gerritforge.analytics.job.Main$.run(Main.scala:25)
        at com.gerritforge.analytics.job.Main$.delayedEndpoint$com$gerritforge$analytics$job$Main$1(Main.scala:67)
        at com.gerritforge.analytics.job.Main$delayedInit$body.apply(Main.scala:25)
        at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
        at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
        at scala.App$$anonfun$main$1.apply(App.scala:76)
        at scala.App$$anonfun$main$1.apply(App.scala:76)
        at scala.collection.immutable.List.foreach(List.scala:381)
        at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
        at scala.App$class.main(App.scala:76)
        at com.gerritforge.analytics.job.Main$.main(Main.scala:25)
        at com.gerritforge.analytics.job.Main.main(Main.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:775)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Luca Milanesio

unread,
Jan 18, 2018, 4:05:16 AM1/18/18
to Makson Lee, Repo and Gerrit Discussion
You should include the credentials in the Gerrit URL.

For example, using the Spark invocation from readme [1], that should be amended to:

bin/spark-submit \
    --conf spark.es.nodes=es.mycompany.com \
    $JARS/SparkAnalytics-assembly-1.0.jar \
    --since 2000-06-01 \
    --aggregate email_hour \
    --url http://user:pass...@gerrit.mycompany.com \
    -e gerrit/analytics

Where 'user' and 'password' are the credentials to access your Gerrit instance.

HTH

Luca.


More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.

Makson Lee

unread,
Jan 18, 2018, 4:21:25 AM1/18/18
to Repo and Gerrit Discussion
thanks for quick response, but i still can't get it work, i found something here [1], do we need to add the username and password into request header?

Exception in thread "main" java.io.IOException: Server returned HTTP response code: 401 for URL: http://username:password@gerrit:8080/a/projects/


On Thursday, January 18, 2018 at 5:05:16 PM UTC+8, lucamilanesio wrote:
You should include the credentials in the Gerrit URL.

For example, using the Spark invocation from readme [1], that should be amended to:

bin/spark-submit \
    --conf spark.es.nodes=es.mycompany.com \
    $JARS/SparkAnalytics-assembly-1.0.jar \
    --since 2000-06-01 \
    --aggregate email_hour \
    --url http://user:password@gerrit.mycompany.com \
    -e gerrit/analytics

Luca Milanesio

unread,
Jan 18, 2018, 4:36:48 AM1/18/18
to Makson Lee, Repo and Gerrit Discussion

On 18 Jan 2018, at 09:21, Makson Lee <cdle...@gmail.com> wrote:

thanks for quick response, but i still can't get it work, i found something here [1], do we need to add the username and password into request header?

Exception in thread "main" java.io.IOException: Server returned HTTP response code: 401 for URL: http://username:password@gerrit:8080/a/projects/

Not really: the java.net.URL parses the user info from the URL. The problem could be basic vs. digest auth.
Have you tried making the HTTP call with CURL -v and check the output?

Luca.

Makson Lee

unread,
Jan 18, 2018, 4:43:07 AM1/18/18
to Repo and Gerrit Discussion
curl -v http://username:password@gerrit:8080/a/projects/ does work.

Luca Milanesio

unread,
Jan 18, 2018, 4:45:42 AM1/18/18
to Makson Lee, Repo and Gerrit Discussion
... and ... what's the output?

Luca Milanesio

unread,
Jan 18, 2018, 4:46:17 AM1/18/18
to Makson Lee, Repo and Gerrit Discussion
... I mean, the headers of the response :-) I don't need the full list of projects :-D

Makson Lee

unread,
Jan 18, 2018, 4:50:54 AM1/18/18
to Repo and Gerrit Discussion
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying xxx.xxx.xxx.xxx...
* Connected to gerrit (xxx.xxx.xxx.xxx) port 8080 (#0)
* Server auth using Basic with user 'xxx'
> GET /a/projects/ HTTP/1.1
> Host: gerrit:8080
> Authorization: Basic xxx
> User-Agent: curl/7.47.0
> Accept: */*
>
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0< HTTP/1.1 200 OK
< Date: Thu, 18 Jan 2018 09:47:00 GMT
< Content-Disposition: attachment
< X-Content-Type-Options: nosniff
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: Mon, 01 Jan 1990 00:00:00 GMT
< Content-Type: application/json;charset=utf-8
< Content-Length: 503579
<
{ [14167 bytes data]
100  491k  100  491k    0     0   793k      0 --:--:-- --:--:-- --:--:--  793k
* Connection #0 to host gerrit left intact

Luca Milanesio

unread,
Jan 18, 2018, 4:59:38 AM1/18/18
to Makson Lee, Repo and Gerrit Discussion
Yep, it is actually a BasicAuth. java.net.URL should work.

Let me try a sample standalone Java App ...

Luca Milanesio

unread,
Jan 18, 2018, 5:56:15 AM1/18/18
to Makson Lee, Repo and Gerrit Discussion
Yeah, same problem:


scala> val is = url.openStream()
java.io.IOException: Server returned HTTP response code: 401 for URL: http://admin:secret@localhost:8080/a/projects/
  at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1626)
  at java.net.URL.openStream(URL.java:1037)
  ... 33 elided

Luca Milanesio

unread,
Jan 18, 2018, 5:56:43 AM1/18/18
to Makson Lee, Repo and Gerrit Discussion
Can you provide a fix / patchset on gerrit-review for it?

Luca.

Makson Lee

unread,
Jan 18, 2018, 6:09:23 AM1/18/18
to Repo and Gerrit Discussion
okay, i will try it later.

Luke Engle

unread,
Feb 5, 2018, 11:34:59 PM2/5/18
to Repo and Gerrit Discussion
Our internal Gerrit server strictly listens to https, and when running the etl spark job I'm getting an "unable to find valid certification path" error. I've tried passing in paths to the keystore and truststore using the --conf syntax to no avail. All examples I see are using http, do you know of an easy fix for this?

Stacktrace:
Exception in thread "main" javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1959)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1514)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:961)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1072)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1546)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:240)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1496)
... 37 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
... 43 more

Luca Milanesio

unread,
Feb 6, 2018, 8:56:18 AM2/6/18
to Luke Engle, Repo and Gerrit Discussion
Hi Luke,
at the moment there is no option in the Spark job to disable SSL certificate validation.

You may look at adding the CA and intermediate certs by hand into the JRE/lib/security trust-store with keytool or similar.

HTH

Luca.

Message has been deleted
Message has been deleted
Message has been deleted

Makson Lee

unread,
Mar 14, 2018, 3:58:08 AM3/14/18
to Repo and Gerrit Discussion
actually, i am using logstash jdbc input plugin now to collect data from gerrit eventsdb [1].

[1] https://gerrit.googlesource.com/plugins/events-log/

On Wednesday, March 14, 2018 at 2:18:22 PM UTC+8, Seheon wrote:
Hello Makson Lee
Did you resolve your issue? I have same issue as you,
"Server returned HTTP response code: 401 for URL: http://user:password@gerrit_server:8110/a/projects/"

Seheon

2018년 1월 18일 목요일 오후 8시 9분 23초 UTC+9, Makson Lee 님의 말:

Luca Milanesio

unread,
Mar 14, 2018, 4:03:20 AM3/14/18
to Makson Lee, Luca Milanesio, Repo and Gerrit Discussion
Why not using Kafka Stream Events then?

Makson Lee

unread,
Mar 14, 2018, 4:30:26 AM3/14/18
to Repo and Gerrit Discussion
thanks for the suggestion, we will give it a try.


On Wednesday, March 14, 2018 at 4:03:20 PM UTC+8, lucamilanesio wrote:
Why not using Kafka Stream Events then?
On 14 Mar 2018, at 07:58, Makson Lee <cdle...@gmail.com> wrote:

actually, i am using logstash jdbc input plugin now to collect data from gerrit eventsdb [1].

[1] https://gerrit.googlesource.com/plugins/events-log/

On Wednesday, March 14, 2018 at 2:18:22 PM UTC+8, Seheon wrote:
Hello Makson Lee
Did you resolve your issue? I have same issue as you,
"Server returned HTTP response code: 401 for URL: http://user:password@gerrit_server:8110/a/projects/"

Seheon

2018년 1월 18일 목요일 오후 8시 9분 23초 UTC+9, Makson Lee 님의 말:
okay, i will try it later.

--
--

Seheon

unread,
Mar 14, 2018, 6:35:19 AM3/14/18
to Repo and Gerrit Discussion

thanks for the suggestion too.

Philip Stefanov

unread,
Aug 13, 2018, 11:51:21 PM8/13/18
to Repo and Gerrit Discussion
Wanted to try out this plugin with spark-2.3.1-bin-hadoop2.7.tgz.
But instead of one spark-assembly.jar, i have 200+ jars which one should i use?

Luca Milanesio

unread,
Aug 14, 2018, 12:38:10 AM8/14/18
to Philip Stefanov, Luca Milanesio, Repo and Gerrit Discussion
Try this URL if you are willing to use with Gerrit 2.15:

There is only one jar, you can't miss it :-)

Luca.

Philip Stefanov

unread,
Aug 14, 2018, 1:48:13 AM8/14/18
to Repo and Gerrit Discussion
Hello Luca,

I meant something else example spark job from doc looks like:
bin/spark-submit \
 
--conf spark.es.nodes=es.mycompany.com \
 $JARS
/SparkAnalytics-assembly.jar \
...

I thought $JARS/SparkAnalytics-assembly.jar is spark-assembly.jar, which was build with Spark it self before.

So this should be ${GERRIT_HOME}/plugins/analytics.jar instead right?
We use 2.13.7 and not likely to migrate any time soon analytics version i have installed is v1.0-1-g5a0cc4a.

Luca Milanesio

unread,
Aug 14, 2018, 1:56:28 AM8/14/18
to Philip Stefanov, Luca Milanesio, Repo and Gerrit Discussion

On 13 Aug 2018, at 22:48, Philip Stefanov <ph.st...@gmail.com> wrote:

Hello Luca,

I meant something else example spark job from doc looks like:
bin/spark-submit \
 
--conf spark.es.nodes=es.mycompany.com \
 $JARS
/SparkAnalytics-assembly.jar \
...

I thought $JARS/SparkAnalytics-assembly.jar is spark-assembly.jar, which was build with Spark it self before.

The example from the documentation is a bit obsolete and needs fixing.
The Spark Job JAR file is:

So this should be ${GERRIT_HOME}/plugins/analytics.jar instead right?

Nope, you need to execute the Spark job and not the Gerrit plugin.

We use 2.13.7 and not likely to migrate any time soon analytics version i have installed is v1.0-1-g5a0cc4a.

Current Gerrit version is 2.15 and (hopefully soon) 2.16/3.0 will be out.
Any reasons why you guys are still stuck on 2.13? (-2 versions from the latest)

The Spark ELT may work as well against v2.13, but I haven't done any testing there :-(

Philip Stefanov

unread,
Aug 14, 2018, 3:52:07 AM8/14/18
to Repo and Gerrit Discussion
Ahh i completely forgot analytics-etl.jar :)
Well we are with 2.13.7 since it's release date, and kinda gives us everything we need so far.
But looking at the bugfixes right now makes me rethink what i say before :)
2.13.7 also use digest auth and this plugin use basic.

Matthias Sohn

unread,
Aug 14, 2018, 4:39:24 AM8/14/18
to ph.st...@gmail.com, Repo and Gerrit Discussion
On Tue, Aug 14, 2018 at 9:52 AM Philip Stefanov <ph.st...@gmail.com> wrote:
Ahh i completely forgot analytics-etl.jar :)
Well we are with 2.13.7 since it's release date, and kinda gives us everything we need so far.
But looking at the bugfixes right now makes me rethink what i say before :)
2.13.7 also use digest auth and this plugin use basic.

you should consider to update at least to the latest 2.13.11 bugfix release, see release notes here
 
To unsubscribe, email repo-discuss...@googlegroups.com

Frank Huang

unread,
Sep 4, 2018, 7:35:30 AM9/4/18
to Repo and Gerrit Discussion
hi,Luca

I found it is not support to get the branch of  project, that'is say this app does not support the option 'extract-branches'.

It is anyway to make this? May I need to change the source code and build by myself?


Reply all
Reply to author
Forward
0 new messages