[Docker]Gerrit with LDAP Installation

[Gabriel Pascual]

Hi everyone,

Since several days, I try to make an installation of Gerrit 2.14.1 with LDAP. The purpose is to make a Docker installation base on Gerritforge, OpenLDAP and a PostgreSQL images. 

My configuration works fine at 90% but I met several problems:

• When Gerrit intialises the database, an exception is thrown:
  
  
   Exception in thread "main" com.google.gwtorm.server.OrmException: Cannot initialize schema

         at com.google.gerrit.server.schema.SchemaUpdater.update(SchemaUpdater.java:110)

         at com.google.gerrit.pgm.init.BaseInit$SiteRun.upgradeSchema(BaseInit.java:386)

         at com.google.gerrit.pgm.init.BaseInit.run(BaseInit.java:143)

         at com.google.gerrit.pgm.util.AbstractProgram.main(AbstractProgram.java:61)

         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

         at java.lang.reflect.Method.invoke(Method.java:498)

         at com.google.gerrit.launcher.GerritLauncher.invokeProgram(GerritLauncher.java:204)

         at com.google.gerrit.launcher.GerritLauncher.mainImpl(GerritLauncher.java:108)

         at com.google.gerrit.launcher.GerritLauncher.main(GerritLauncher.java:63)

         at Main.main(Main.java:24)

   Caused by: java.io.IOException: Failed to initialize All-Projects refs:

   BatchRefUpdate[

     CREATE: 0000000000000000000000000000000000000000 eec10fe522048b7121d1cc45c55a8b03f1dd08b4 refs/meta/config  (LOCK_FAILURE)

   ]

         at com.google.gerrit.server.schema.AllProjectsCreator.execute(AllProjectsCreator.java:241)

         at com.google.gerrit.server.schema.AllProjectsCreator.initAllProjects(AllProjectsCreator.java:203)

         at com.google.gerrit.server.schema.AllProjectsCreator.create(AllProjectsCreator.java:120)

         at com.google.gerrit.server.schema.SchemaCreator.create(SchemaCreator.java:93)

         at com.google.gerrit.server.schema.SchemaUpdater.update(SchemaUpdater.java:108)

         ... 11 more
• Then I make the first try to log in with an account, I have this exception:
  
   com.google.gerrit.httpd.auth.ldap.LdapLoginServlet : LDAP authentication failed

   java.util.NoSuchElementException

         at java.util.ArrayList$Itr.next(ArrayList.java:854)

         at com.google.gerrit.server.account.AccountManager.create(AccountManager.java:263)

         at com.google.gerrit.server.account.AccountManager.authenticate(AccountManager.java:118)

         at com.google.gerrit.httpd.auth.ldap.LdapLoginServlet.doPost(LdapLoginServlet.java:122)

         at javax.servlet.http.HttpServlet.service(HttpServlet.java:648)

         at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)

         at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)

         at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)

         at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)

         at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)

         at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)

         at com.google.gerrit.httpd.raw.StaticModule$PolyGerritFilter.doFilter(StaticModule.java:451)

         at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)

         at com.google.gerrit.httpd.GetUserFilter.doFilter(GetUserFilter.java:75)

         at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)

         at com.google.gerrit.httpd.UniversalWebLoginFilter.doFilter(UniversalWebLoginFilter.java:74)

         at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)

        at com.google.gerrit.httpd.RunAsFilter.doFilter(RunAsFilter.java:111)

         at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)

         at com.google.gwtexpui.server.CacheControlFilter.doFilter(CacheControlFilter.java:70)

         at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)

         at com.google.gerrit.httpd.RequestMetricsFilter.doFilter(RequestMetricsFilter.java:57)

         at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)

         at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:133)

         at com.google.gerrit.httpd.AllRequestFilter$FilterProxy.doFilter(AllRequestFilter.java:135)

         at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)

         at com.google.gerrit.httpd.RequestContextFilter.doFilter(RequestContextFilter.java:72)

         at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)

         at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:120)

         at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135)

         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)

         at com.googlesource.gerrit.plugins.ootb.FirstTimeRedirect.doFilter(FirstTimeRedirect.java:63)

         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)

         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)

         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)

         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)

         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)

         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)

         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)

         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)

         at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:56)

         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)

         at org.eclipse.jetty.server.Server.handle(Server.java:534)

         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)

         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)

         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)

         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)

         at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)

         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)

         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)

         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)

         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)

         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)

         at java.lang.Thread.run(Thread.java:748)
• This first try blocks all other attempt to log in with the same account. This message appears in log :
  
  'user1' failed to sign in: Cannot assign external ID "gerrit:user1" to account 1000005; external ID already in use.
• Then I try to log with a second account, It fails the first time with this message : 
  
  'user2' failed to sign in: Cannot assign external ID "gerrit:user2" to account 1000003; external ID already in use.
  
• Finally, only the second account can log in Gerrit and the first account which must have Administrator rights is broken.

My analysis is that the first error implies all others. To reproduce my use case, I made a Github project with a getting started (Wiki page ): here

Does someone meet the same problems ?

Regards,

Gabriel PASCUAL

[Luca Milanesio]

Hi Gabriel,

the problems you have shown have nothing to do with the GerritForge's Gerrit Docker image on DockerHub.

Can you share *your* Dockerfile?

Luca.

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Gabriel Pascual]

Hi Luca,

You can find the Dockerfile on the develop branch of the Github repository: https://github.com/Gabitchov/docker-gerrit/blob/develop/image/Dockerfile

Regards,

Gabriel

[Luca Milanesio]

Where is https://github.com/Gabitchov/docker-gerrit/blob/develop/image/gerrit-entrypoint.sh coming from?

You are basically translating all the Gerrit config into environment variables?

Isn't simpler to just include a config file?

Luca.

[Gabriel Pascual]

I made this script to facilitate me the use of Docker Compose. 

Include a config file was my first solution but it was not enough usable because when the config file is injected in the image, this one becomes dependent of the config. So, I put all config into variables.

Gabriel

[Luca Milanesio]

As all the Gerrit config is in variables, it is very hard to tell anything from just the gerrit-entrypoint.sh without having those variables.

Why don't you just exec a shell inside the Docker VM and then troubleshoot from there?

So that you can see what is the generated gerrit.config and try to check some basic connectivity.

Luca.

[Gabriel Pascual]

You can find the values of variable in this file : https://github.com/Gabitchov/docker-gerrit/blob/develop/docker-compose.yml

In the gerrit-entrypoint.sh, I print the final config file. If you make the Getting Started, you could access to log with:

docker-compose logs gerrit

So, the generated configuration must look like this:

 [gerrit]

       basePath = git

       serverId = 8a531847-5f00-478c-a2a9-08b33993ebe7

       canonicalWebUrl = http://localhost:8080/

 [database]

       type = postgresql

       database = reviewdb

       hostname = gerrit-db

       port = 5432

       username = gerrit2

       password = gerrit

 [index]

       type = LUCENE

 [auth]

       type = LDAP

 [sendemail]

       smtpServer = localhost

 [sshd]

       listenAddress = *:29418

 [httpd]

       listenurl = http://*:8080/

       filterClass = com.googlesource.gerrit.plugins.ootb.FirstTimeRedirect

       firstTimeRedirectUrl = /login/%23%2F?account_id=1000000

 [cache]

       directory = cache

 [plugins]

       allowRemoteAdmin = true

 [receive]

       enableSignedPush = false

 [container]

       user = gerrit

       javaHome = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-3.b12.el7_3.x86_64/jre

 [ldap]

       localUsernameToLowerCase = true

       server = ldap://ldap-host

       username = cn=admin,dc=example,dc=org

       referral = follow

       groupsVisibleToAll = true

       accountBase = ou=people,ou=gerrit,dc=example,dc=org

       accountPattern = (&(objectClass=posixAccount)(uid=${username}))

       accountFullname = cn

       groupBase = ou=groups,ou=gerrit,dc=example,dc=org

       groupPattern = (cn=${groupname})

       groupMemberPattern = (&(objectClass=posixGroup)(|(memberUid=${username})(gidNumber=${gidNumber})))

Gabriel

[Luca Milanesio]

The standard Docker image is already initialised using development_become_any_account authentication, which creates the groups and the admin user in the DB and in the All-Users / All-Groups repos.

If you change completely the auth system, then you would need to scrap everything (all the All-* repos and the DBMS) and restart from scratch.

HTH

Luca.

[Joshua Olson]

Hi,

We've run into this issue and if you setup LDAP (or other external account validations like OAUTH) on gerrit then existing accounts and/or accounts created via the REST API will not be able to login via LDAP because the login validation code expects externally validated account to have an existing row in the the ACCOUNT_EXTERNAL_IDS table (or in All-Users.git if external ids have been migrated there) having the external_id value "gerrit:{username}".  If gerrit doesn't find this row it attempts to create a new account.  Gerrit expects all externally validated accounts to be created upon first login.

You can solve this by running this on the DB for any account you expect to login using external auth.

   `insert into account_external_ids (account_id, email_address, external_id) values ({account_id}, '{username}@{domain}', 'gerrit:{username}');`

Cheers,

Josh

[Gabriel Pascual]

Hi Luca,

Thank's a lot. I added a step to clean all "All-*" repos. This resolved my problem with the first logged account which had not Administrator rights.

I have an other question:

  -  What are folders/files that I have not to touch to restart correctly in the same state ?

Regards,

Gabriel

[Gabriel Pascual]

Hi Josh,

I already made this solution to inject the good value into the DB but I found the workaround enough dirty. Luca found the principal reason why Gerrit wasn't initialised correctly. Now, I must to found what folders/files must be persistent to preserve state of Gerrit.

Cheers,

Gabriel

[Luca Milanesio]

On 26 Jun 2017, at 14:45, Gabriel Pascual <gabriel...@gmail.com> wrote:

Hi Luca,

Thank's a lot. I added a step to clean all "All-*" repos. This resolved my problem with the first logged account which had not Administrator rights.

I have an other question:

  -  What are folders/files that I have not to touch to restart correctly in the same state ?

Which "same state"? Can you elaborate?

[Gabriel Pascual]

Hi,

If I correctly understood, there is a link between User Accounts and "All-*" repos. If I want to keep this link, I must not clean repos for each restart.

Today, the state contains only a good ACL configuration. I would like to know if there are other important locations in Gerrit installation ?

Gabriel

[Luca Milanesio]

On 26 Jun 2017, at 16:03, Gabriel Pascual <gabriel...@gmail.com> wrote:

Hi,

If I correctly understood, there is a link between User Accounts and "All-*" repos.

All-Users ... the name tells what it does, isn't it ?

If I want to keep this link, I must not clean repos for each restart.

With Docker VMs, if you want to keep the state you need to use either external or internal volumes.

Today, the state contains only a good ACL configuration. I would like to know if there are other important locations in Gerrit installation ?

Gerrit has:

- DB

- Cache

- Index

- Repos

- Sessions

You should keep all of them across restarts IMHO

Luca.