Re: gerrit kafka-events and gerrit 2.13.1

[Luca Milanesio]

Hi Pierre,

please write to the mailing list :-)

As mentioned in the README.md (see [1]) you need to specify the bootstrapServers configuration.

Example:

[plugin "kafka-events"]
    bootstrapServers = localhost:9092

HTH

Luca.

On 7 Nov 2016, at 15:02, Pierre Tardy <tar...@gmail.com> wrote:

Hi Luca,

I am trying to evaluate gerrit kafka-events as a mean to build microservices around Gerrit activities.

I spend a fair amount of time struggling with the configuration of the server, and finally find out that the plugin does not sound to be compatible with gerrit 2.13.1.

I downloaded the plugin jar from here:

https://gerrit-ci.gerritforge.com/job/plugin-kafka-events-stable-2.13/lastSuccessfulBuild/artifact/buck-out/gen/plugins/kafka-events/kafka-events.jar

If I swap the gerrit.war with a freshly build gerrit.war from gerrit trunk, then I dont have issue anymore with the configuration

with 2.13.1, and the same config (as per README) I get the error:

gerrit_1     | [2016-11-07 14:45:44,133] [main] INFO  com.googlesource.gerrit.plugins.kafka.config.KafkaProperties : applying config: com.google.gerrit.server.config.PluginConfig@467ef400

gerrit_1     | [2016-11-07 14:45:44,186] [main] INFO  com.googlesource.gerrit.plugins.kafka.session.type.KafkaSession : Connect to null...

gerrit_1     | [2016-11-07 14:45:44,233] [main] WARN  com.google.gerrit.server.plugins.PluginLoader : Cannot load plugin kafka-events

gerrit_1     | org.apache.kafka.common.config.ConfigException: Missing required configuration "bootstrap.servers" which has no default value.

gerrit_1     | at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:421)

gerrit_1     | at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:55)

gerrit_1     | at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:62)

Is this something you are aware of?

Regards

Pierre

[Pierre Tardy]

Hi Luca,

Sorry for contacting you directly.  I wasn't sure as this is a plugin that this was the right place.

I indeed did configure the plugin like you are saying (and understood the weird camelCase to dot hierarchy configuration parameters from the source)

It just does not work for me with the 2.13.1 pr 2.13.2 gerrit.war

I tripled check either on a osx development environment or in a dockerfile, and with the two 2.13 releases, it just won't load the plugin configuration.

I use the exact same configuration file and working environment, just swapping the gerrit.war.

I also noticed that the schema135 migration script won't work if All-Project is not created:

Migrating data to schema 135 ...

Exception in thread "main" com.google.gwtorm.server.OrmException: org.eclipse.jgit.errors.RepositoryNotFoundException: repository not found: /Users/ptardy/dev/gerrit_testsite/git/All-Projects

--- 2.13.2--

0 % ./bin/gerrit.sh daemon

Running Gerrit Code Review:

[2016-11-07 16:33:05,122] [main] INFO  com.google.gerrit.server.cache.h2.H2CacheFactory : Enabling disk cache /Users/ptardy/dev/gerrit_testsite/cache

[2016-11-07 16:33:06,293] [main] INFO  com.google.gerrit.server.config.ScheduleConfig : gc schedule parameter "gc.interval" is not configured

[2016-11-07 16:33:06,293] [main] INFO  com.google.gerrit.server.config.ScheduleConfig : changeCleanup schedule parameter "changeCleanup.interval" is not configured

[2016-11-07 16:33:07,019] [main] WARN  com.google.gerrit.sshd.SshDaemon : Disabling cipher aes192-ctr: Unable to initialize cipher BaseCipher[AES,16,24,AES/CTR/NoPadding]

[2016-11-07 16:33:07,019] [main] WARN  com.google.gerrit.sshd.SshDaemon : Disabling cipher aes256-ctr: Unable to initialize cipher BaseCipher[AES,16,32,AES/CTR/NoPadding]

[2016-11-07 16:33:07,021] [main] WARN  com.google.gerrit.sshd.SshDaemon : Disabling cipher arcfour256: Illegal key size or default parameters; try installing unlimited cryptography extension

[2016-11-07 16:33:07,030] [main] WARN  com.google.gerrit.sshd.SshDaemon : Disabling cipher aes192-cbc: Unable to initialize cipher BaseCipher[AES,16,24,AES/CBC/NoPadding]

[2016-11-07 16:33:07,030] [main] WARN  com.google.gerrit.sshd.SshDaemon : Disabling cipher aes256-cbc: Unable to initialize cipher BaseCipher[AES,16,32,AES/CBC/NoPadding]

[2016-11-07 16:33:07,099] [main] WARN  com.google.gerrit.server.config.GitwebCgiConfig : gitweb not installed (no /usr/lib/cgi-bin/gitweb.cgi found)

[2016-11-07 16:33:07,464] [main] INFO  org.eclipse.jetty.util.log : Logging initialized @6833ms

[2016-11-07 16:33:07,562] [main] INFO  com.google.gerrit.server.git.LocalDiskRepositoryManager : Defaulting core.streamFileThreshold to 456m

[2016-11-07 16:33:07,854] [main] INFO  com.google.gerrit.server.plugins.PluginLoader : Loading plugins from /Users/ptardy/dev/gerrit_testsite/plugins

[2016-11-07 16:33:07,968] [main] INFO  com.googlesource.gerrit.plugins.kafka.config.KafkaProperties : applying config: com.google.gerrit.server.config.PluginConfig@6996bbc4

[2016-11-07 16:33:08,016] [main] INFO  com.googlesource.gerrit.plugins.kafka.session.type.KafkaSession : Connect to null...

[2016-11-07 16:33:08,047] [main] WARN  com.google.gerrit.server.plugins.PluginLoader : Cannot load plugin kafka-events

org.apache.kafka.common.config.ConfigException: Missing required configuration "bootstrap.servers" which has no default value.

at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:421)

at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:55)

at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:62)

at org.apache.kafka.clients.producer.ProducerConfig.<init>(ProducerConfig.java:336)

at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:188)

at com.googlesource.gerrit.plugins.kafka.session.type.KafkaSession.connect(KafkaSession.java:55)

at com.googlesource.gerrit.plugins.kafka.message.MessagePublisher.start(MessagePublisher.java:47)

at com.googlesource.gerrit.plugins.kafka.Manager.start(Manager.java:50)

at com.google.gerrit.lifecycle.LifecycleManager.start(LifecycleManager.java:89)

at com.google.gerrit.server.plugins.ServerPlugin.startPlugin(ServerPlugin.java:214)

at com.google.gerrit.server.plugins.ServerPlugin.start(ServerPlugin.java:155)

at com.google.gerrit.server.plugins.PluginLoader.runPlugin(PluginLoader.java:508)

at com.google.gerrit.server.plugins.PluginLoader.rescan(PluginLoader.java:435)

at com.google.gerrit.server.plugins.PluginLoader.start(PluginLoader.java:339)

at com.google.gerrit.lifecycle.LifecycleManager.start(LifecycleManager.java:89)

at com.google.gerrit.pgm.Daemon.start(Daemon.java:311)

at com.google.gerrit.pgm.Daemon.run(Daemon.java:212)

at com.google.gerrit.pgm.util.AbstractProgram.main(AbstractProgram.java:64)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at com.google.gerrit.launcher.GerritLauncher.invokeProgram(GerritLauncher.java:161)

at com.google.gerrit.launcher.GerritLauncher.mainImpl(GerritLauncher.java:102)

at com.google.gerrit.launcher.GerritLauncher.main(GerritLauncher.java:59)

at Main.main(Main.java:25)

[2016-11-07 16:33:08,073] [main] INFO  com.google.gerrit.server.change.ChangeCleanupRunner : Ignoring missing changeCleanup schedule configuration

[2016-11-07 16:33:08,111] [main] INFO  com.google.gerrit.sshd.SshDaemon : Started Gerrit SSHD-CORE-1.2.0 on localhost:29418

[2016-11-07 16:33:08,120] [main] INFO  org.eclipse.jetty.server.Server : jetty-9.2.14.v20151106

[2016-11-07 16:33:08,794] [main] INFO  org.eclipse.jetty.server.handler.ContextHandler : Started o.e.j.s.ServletContextHandler@7598d675{/,null,AVAILABLE}

[2016-11-07 16:33:08,805] [main] INFO  org.eclipse.jetty.server.ServerConnector : Started ServerConnector@126254ec{HTTP/1.1}{localhost:8081}

[2016-11-07 16:33:08,807] [main] INFO  org.eclipse.jetty.server.Server : Started @8178ms

[2016-11-07 16:33:08,809] [main] INFO  com.google.gerrit.pgm.Daemon : Gerrit Code Review 2.13.2 ready

--trunk: --

0 % ./bin/gerrit.sh daemon

Running Gerrit Code Review:

[2016-11-07 16:35:38,349] [main] INFO  com.google.gerrit.server.cache.h2.H2CacheFactory : Enabling disk cache /Users/ptardy/dev/gerrit_testsite/cache

[2016-11-07 16:35:38,883] [main] WARN  com.google.gerrit.server.documentation.QueryDocumentationExecutor : No index available

[2016-11-07 16:35:39,709] [main] INFO  com.google.gerrit.server.config.ScheduleConfig : gc schedule parameter "gc.interval" is not configured

[2016-11-07 16:35:39,709] [main] INFO  com.google.gerrit.server.config.ScheduleConfig : changeCleanup schedule parameter "changeCleanup.interval" is not configured

[2016-11-07 16:35:40,306] [main] WARN  com.google.gerrit.sshd.SshDaemon : Disabling cipher aes192-ctr: Unable to initialize cipher BaseCipher[AES,16,24,AES/CTR/NoPadding]

[2016-11-07 16:35:40,306] [main] WARN  com.google.gerrit.sshd.SshDaemon : Disabling cipher aes256-ctr: Unable to initialize cipher BaseCipher[AES,16,32,AES/CTR/NoPadding]

[2016-11-07 16:35:40,307] [main] WARN  com.google.gerrit.sshd.SshDaemon : Disabling cipher arcfour256: Illegal key size or default parameters; try installing unlimited cryptography extension

[2016-11-07 16:35:40,317] [main] WARN  com.google.gerrit.sshd.SshDaemon : Disabling cipher aes192-cbc: Unable to initialize cipher BaseCipher[AES,16,24,AES/CBC/NoPadding]

[2016-11-07 16:35:40,317] [main] WARN  com.google.gerrit.sshd.SshDaemon : Disabling cipher aes256-cbc: Unable to initialize cipher BaseCipher[AES,16,32,AES/CBC/NoPadding]

[2016-11-07 16:35:40,376] [main] WARN  com.google.gerrit.server.config.GitwebCgiConfig : gitweb not installed (no /usr/lib/cgi-bin/gitweb.cgi found)

[2016-11-07 16:35:40,732] [main] INFO  org.eclipse.jetty.util.log : Logging initialized @7114ms

[2016-11-07 16:35:40,848] [main] INFO  com.google.gerrit.server.git.LocalDiskRepositoryManager : Defaulting core.streamFileThreshold to 456m

[2016-11-07 16:35:41,356] [main] INFO  com.google.gerrit.server.plugins.PluginLoader : Loading plugins from /Users/ptardy/dev/gerrit_testsite/plugins

[2016-11-07 16:35:41,466] [main] INFO  com.googlesource.gerrit.plugins.kafka.config.KafkaProperties : applying config: com.google.gerrit.server.config.PluginConfig@474ce6b8

[2016-11-07 16:35:41,467] [main] INFO  com.googlesource.gerrit.plugins.kafka.config.KafkaProperties : applying config: bootstrapServers localhost:9092

[2016-11-07 16:35:41,471] [main] INFO  com.googlesource.gerrit.plugins.kafka.session.type.KafkaSession : Connect to localhost:9092...

[2016-11-07 16:35:41,486] [main] INFO  org.apache.kafka.clients.producer.ProducerConfig : ProducerConfig values:

metric.reporters = []

metadata.max.age.ms = 300000

reconnect.backoff.ms = 50

sasl.kerberos.ticket.renew.window.factor = 0.8

bootstrap.servers = [localhost:9092]

ssl.keystore.type = JKS

sasl.mechanism = GSSAPI

max.block.ms = 60000

interceptor.classes = null

ssl.truststore.password = null

client.id =

ssl.endpoint.identification.algorithm = null

request.timeout.ms = 30000

acks = all

receive.buffer.bytes = 32768

ssl.truststore.type = JKS

retries = 0

ssl.truststore.location = null

ssl.keystore.password = null

send.buffer.bytes = 131072

compression.type = none

metadata.fetch.timeout.ms = 60000

retry.backoff.ms = 100

sasl.kerberos.kinit.cmd = /usr/bin/kinit

buffer.memory = 33554432

timeout.ms = 30000

key.serializer = class org.apache.kafka.common.serialization.StringSerializer

sasl.kerberos.service.name = null

sasl.kerberos.ticket.renew.jitter = 0.05

ssl.trustmanager.algorithm = PKIX

block.on.buffer.full = false

ssl.key.password = null

sasl.kerberos.min.time.before.relogin = 60000

connections.max.idle.ms = 540000

max.in.flight.requests.per.connection = 5

metrics.num.samples = 2

ssl.protocol = TLS

ssl.provider = null

ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]

batch.size = 16384

ssl.keystore.location = null

ssl.cipher.suites = null

security.protocol = PLAINTEXT

max.request.size = 1048576

value.serializer = class org.apache.kafka.common.serialization.StringSerializer

ssl.keymanager.algorithm = SunX509

metrics.sample.window.ms = 30000

partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner

linger.ms = 1

[2016-11-07 16:35:41,588] [main] INFO  org.apache.kafka.clients.producer.ProducerConfig : ProducerConfig values:

metric.reporters = []

metadata.max.age.ms = 300000

reconnect.backoff.ms = 50

sasl.kerberos.ticket.renew.window.factor = 0.8

bootstrap.servers = [localhost:9092]

ssl.keystore.type = JKS

sasl.mechanism = GSSAPI

max.block.ms = 60000

interceptor.classes = null

ssl.truststore.password = null

client.id = producer-1

ssl.endpoint.identification.algorithm = null

request.timeout.ms = 30000

acks = all

receive.buffer.bytes = 32768

ssl.truststore.type = JKS

retries = 0

ssl.truststore.location = null

ssl.keystore.password = null

send.buffer.bytes = 131072

compression.type = none

metadata.fetch.timeout.ms = 60000

retry.backoff.ms = 100

sasl.kerberos.kinit.cmd = /usr/bin/kinit

buffer.memory = 33554432

timeout.ms = 30000

key.serializer = class org.apache.kafka.common.serialization.StringSerializer

sasl.kerberos.service.name = null

sasl.kerberos.ticket.renew.jitter = 0.05

ssl.trustmanager.algorithm = PKIX

block.on.buffer.full = false

ssl.key.password = null

sasl.kerberos.min.time.before.relogin = 60000

connections.max.idle.ms = 540000

max.in.flight.requests.per.connection = 5

metrics.num.samples = 2

ssl.protocol = TLS

ssl.provider = null

ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]

batch.size = 16384

ssl.keystore.location = null

ssl.cipher.suites = null

security.protocol = PLAINTEXT

max.request.size = 1048576

value.serializer = class org.apache.kafka.common.serialization.StringSerializer

ssl.keymanager.algorithm = SunX509

metrics.sample.window.ms = 30000

partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner

linger.ms = 1

[2016-11-07 16:35:41,590] [main] INFO  org.apache.kafka.common.utils.AppInfoParser : Kafka version : 0.10.0.1

[2016-11-07 16:35:41,591] [main] INFO  org.apache.kafka.common.utils.AppInfoParser : Kafka commitId : a7a17cdec9eaa6c5

[2016-11-07 16:35:41,591] [main] INFO  com.googlesource.gerrit.plugins.kafka.session.type.KafkaSession : Connection established.

[2016-11-07 16:35:41,592] [main] INFO  com.google.gerrit.server.plugins.PluginLoader : Loaded plugin kafka-events, version a00708d-dirty

[2016-11-07 16:35:41,593] [main] INFO  com.google.gerrit.server.change.ChangeCleanupRunner : Ignoring missing changeCleanup schedule configuration

[2016-11-07 16:35:41,623] [main] INFO  com.google.gerrit.sshd.SshDaemon : Started Gerrit SSHD-CORE-1.2.0 on localhost:29418

[2016-11-07 16:35:41,629] [main] INFO  org.eclipse.jetty.server.Server : jetty-9.3.11.v20160721

[2016-11-07 16:35:42,118] [main] INFO  org.eclipse.jetty.server.handler.ContextHandler : Started o.e.j.s.ServletContextHandler@1d0dad12{/,null,AVAILABLE}

[2016-11-07 16:35:42,127] [main] INFO  org.eclipse.jetty.server.AbstractConnector : Started ServerConnector@7296fe0b{HTTP/1.1,[http/1.1]}{localhost:8081}

[2016-11-07 16:35:42,130] [main] INFO  org.eclipse.jetty.server.Server : Started @8513ms

[2016-11-07 16:35:42,131] [main] INFO  com.google.gerrit.pgm.Daemon : Gerrit Code Review 2.13.2-1409-gfd0a87d-dirty ready

[Luca Milanesio]

Hi Pierre,

kafka-events stable-2.13 and master are *exactly the same version* :-)

Have you actually tried getting the master binary and put into a 2.13? The API haven't changed so it should work out-of-the-box.

Luca.

[Pierre Tardy]

I will try to bisect tomorrow but the 2.13.2 cannot be built with

Le lun. 7 nov. 2016 17:13, Luca Milanesio <luca.mi...@gmail.com> a écrit :

Hi Pierre,

kafka-events stable-2.13 and master are *exactly the same version* :-)

Yes, I have seen that.

 

Have you actually tried getting the master binary and put into a 2.13? The API haven't changed so it should work out-of-the-box.

I have tried the master binary, and the one that I built myself to insert some worthless  debug prints.

I haven't found a plugin binary built against the 2.13 api jar.

I am pretty sure this is an issue that was fixed in Gerrit master branch. I've built the 2.13.2 from source and can reproduce the problem.

Its annoying as the source code is not compatible with latest buck

I'll try tomorrow to automate a test, and do the git bisect.

 

Regards

Pierre

[Pierre Tardy]

Hi Luca,

I did the git bisect script, and it found me the culprit. I would suggest to port this patch for gerrit 2.13.3.

kafka-event do use PluginConfig.getNames to convert the camelCase to dotted values, as per this snippet:

    for (String name : config.getNames()) {

      Object value = config.getString(name);

      LOGGER.info("applying config: {} {}", name, value);

      String propName =

          CaseFormat.LOWER_CAMEL.to(CaseFormat.LOWER_HYPHEN, name).replaceAll(

              "-", ".");

      put(propName, value);

    }

0 % git show 3ad1233a66518d4eef6a741576f5b48dbbaf05e2

commit 3ad1233a66518d4eef6a741576f5b48dbbaf05e2

Author: Luca Milanesio <luca.mi...@gmail.com>

Date:   Thu Oct 13 23:07:07 2016 +0100

    Use recursive name lookup for PluginConfig

    PluginConfig is based on Gerrit's Config and delegates its

    variable resolution to its base config object.

    When resolving the configuration names, we do need to perform

    a recursive resolution otherwise we would always get an

    empty names list.

    Change-Id: Iff7a0de4530597fbe45c7aef753363ab6132ecfa

diff --git a/gerrit-server/src/main/java/com/google/gerrit/server/config/PluginConfig.java b/gerrit-server/src/main/java/com/google/gerrit/server/config/PluginConfig.java

index c7390d7..d8485fe 100644

--- a/gerrit-server/src/main/java/com/google/gerrit/server/config/PluginConfig.java

+++ b/gerrit-server/src/main/java/com/google/gerrit/server/config/PluginConfig.java

@@ -151,6 +151,6 @@ public class PluginConfig {

   }

   public Set<String> getNames() {

-    return cfg.getNames(PLUGIN, pluginName);

+    return cfg.getNames(PLUGIN, pluginName, true);

   }

 }

for reference, here is the bisect script I used

#!/bin/bash

REF=`git log -n1 --pretty=format:%h HEAD`

if git grep source_under_test >/dev/null

then

git cherry-pick 0b774c478eca6e8501e8433281715d790b92cb83

fi

while [ -f .git/CHERRY_PICK_HEAD ]

do

    echo "please fix cherry-pick issues"

    sleep 1

done

echo buck build gerrit

if ! buck build gerrit

then

git reset --hard $REF

exit 125

fi

git submodule update --init

echo buck build plugins/kafka-events

buck build plugins/kafka-events

../gerrit_testsite_dic/bin/gerrit.sh stop

killall GerritCodeReview

rm -rf ../gerrit_testsite_dic

java -jar buck-out/gen/gerrit/gerrit.war init --dev -b --install-plugin kafka-events -d ../gerrit_testsite_dic

cp buck-out/gen/plugins/kafka-events/kafka-events.jar ../gerrit_testsite_dic/plugins

rm -f ../gerrit_testsite_dic/logs/error_log

cat kafka_config.txt >> ../gerrit_testsite_dic/etc/gerrit.config

../gerrit_testsite_dic/bin/gerrit.sh restart

while ! grep 'com.google.gerrit.pgm.Daemon : Gerrit Code Review' ../gerrit_testsite_dic/logs/error_log >/dev/null

do

echo "waiting startup"

sleep 1

done

../gerrit_testsite_dic/bin/gerrit.sh stop

if grep "Missing required configuration" ../gerrit_testsite_dic/logs/error_log

then

    RES=0

else

    RES=1

fi

git reset --hard $REF

exit $RES

[Luca Milanesio]

Hi Pierre,

apologies for the misunderstanding, I thought you were saying the plugin was different for stable-2.13 compared to master.

Yes, I do remember the bug in Gerrit, which I fixed on master.

Luca.

[Pierre Tardy]

Le mar. 8 nov. 2016 à 15:01, Luca Milanesio <luca.mi...@gmail.com> a écrit :

Hi Pierre,

apologies for the misunderstanding, I thought you were saying the plugin was different for stable-2.13 compared to master.

Yes, I do remember the bug in Gerrit, which I fixed on master.

Do you intend to port this fix in stable-2.13?

If not, shall I send a patch to the kafka-event plugin's README so that it states that this plugin is compatible with 2.14 and later (this would have save me a lot of time)

Regards,

Pierre

[Luca Milanesio]

Porting the fix to stable-2.13 should be quite easy, let me do it ...

Luca.

[Luca Milanesio]

Done:

https://gerrit-review.googlesource.com/#/c/91074/

Luca.

On 8 Nov 2016, at 16:26, Pierre Tardy <tar...@gmail.com> wrote:

[Pierre Tardy]

Le mar. 8 nov. 2016 à 18:15, Luca Milanesio <luca.mi...@gmail.com> a écrit :

Porting the fix to stable-2.13 should be quite easy, let me do it ...

Thank you!

FWIW, I finally was able to see some message transmitted to kafka0.10.  \o/

Pierre

[luca.mi...@gmail.com]

Great, glad you managed to get it work :-)

Luca

Sent from my iPhone

[Luca Milanesio]

The fix on 2.13 was merged. 

Luca.