[Sonar LDAP Plugin] SonarQube service start fails if LDAP server is not available.

[Stuart Stephen]

Hi all,

We have a Windows network and we often need to apply Windows Updates to keep the systems up-to-date. It is therefore reasonable to expect servers to restart on occasion. Sadly, if the domain controller which hosts the LDAP is slower to restart than our SonarQube server, then this means that when starting the SonarQube service it will fail to start and the SonarQube process will exit. This then causes our long running overnight sonar tasks to all fail, so it's quite an inconvenience for us.

I feel that it would be preferable for the LDAP plugin to be lenient in this instance and remain up, it could then keep attempting to connect to LDAP until it succeeds without taking down the entire server as it currently does. 

I'd be interested in hearing your thoughts.

Logs as follows:

2018.02.03 22:23:17 INFO  web[][o.s.p.ProcessEntryPoint] Starting web

2018.02.03 22:23:18 INFO  web[][o.a.t.u.n.NioSelectorPool] Using a shared selector for servlet write/read

2018.02.03 22:23:19 INFO  web[][o.e.p.PluginsService] no modules loaded

2018.02.03 22:23:19 INFO  web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.index.reindex.ReindexPlugin]

2018.02.03 22:23:19 INFO  web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.join.ParentJoinPlugin]

2018.02.03 22:23:19 INFO  web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.percolator.PercolatorPlugin]

2018.02.03 22:23:19 INFO  web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin]

2018.02.03 22:23:21 INFO  web[][o.s.s.e.EsClientProvider] Connected to local Elasticsearch: [127.0.0.1:9001]

2018.02.03 22:23:21 INFO  web[][o.s.s.p.LogServerVersion] SonarQube Server / 6.7.1.35068 / 426519346f51f7b980a76f9050f983110550509d

2018.02.03 22:23:21 INFO  web[][o.sonar.db.Database] Create JDBC data source for jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance

2018.02.03 22:23:22 INFO  web[][o.s.s.p.ServerFileSystemImpl] SonarQube home: D:\sonarqube-6.7.1

2018.02.03 22:23:22 INFO  web[][o.s.s.u.SystemPasscodeImpl] System authentication by passcode is disabled

2018.02.03 22:23:23 INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin Flex / 2.3 / 79167f762219bea1fd181ff5ef6ca4b34abd1675

2018.02.03 22:23:23 INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin LDAP / 2.2.0.608 / 79dc3fa4393a29667673c70182f3016288b548b7

2018.02.03 22:23:23 INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarC# / 6.5.0.3766 / 6011a1e9fb002edb044ae3092d92bcfb7d9d8a2d

2018.02.03 22:23:23 INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarJS / 4.0.0.5862 / 7c3166ea3f0526e74aa53e521d2fee2290fa0820

2018.02.03 22:23:23 INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarJava / 5.1.0.13090 / 5d2c41b3da0a3a6c11417a6d17d011d7a1fe8ba2

2018.02.03 22:23:23 INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarPHP / 2.11.0.2485 / 741861a29e5f9a26c6c99c06268facb6c4f4a882

2018.02.03 22:23:23 INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarPython / 1.8.0.1496 / 3fe3bc4d0273a5721ea2fb368dc45b1bb82fede3

2018.02.03 22:23:23 INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarQube :: Plugins :: SCM :: Git / 1.3.0.869 / 4da53e3f9e55f4f2e5796625cb0c5768ed152079

2018.02.03 22:23:23 INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarQube :: Plugins :: SCM :: SVN / 1.6.0.860 / 2111fdbd1dddda4ad6d4ed6486fd0b18c1010d3b

2018.02.03 22:23:23 INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarTS / 1.1.0.1079 / 042c9e65239a47d92d305f9767f730b3cc1e5ed3

2018.02.03 22:23:23 INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarXML / 1.4.3.1027 / 39588245cecf538bb27be4e496ff303b0143d20b

2018.02.03 22:23:23 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.WebServiceFilter@64d77889 [pattern=UrlPattern{inclusions=[/api/system/migrate_db/*, ...], exclusions=[/api/properties*, ...]}]

2018.02.03 22:23:23 INFO  web[][o.s.s.a.EmbeddedTomcat] HTTP connector enabled on port 9000

2018.02.03 22:23:25 INFO  web[][o.s.s.p.UpdateCenterClient] Update center: https://update.sonarsource.org/update-center.properties (no proxy)

2018.02.03 22:23:27 INFO  web[][o.s.s.s.LogServerId] Server ID: AWFSSlae4lSiiR-0j9my

2018.02.03 22:23:27 INFO  web[][org.sonar.INFO] Security realm: LDAP

2018.02.03 22:23:27 INFO  web[][o.s.p.l.LdapSettingsManager] User mapping: LdapUserMapping{baseDn=OU=Staff,DC=isys,DC=tracegroup,DC=com, request=(&(objectClass=user)(sAMAccountName={0})), realNameAttribute=displayName, emailAttribute=mail}

2018.02.03 22:23:27 INFO  web[][o.s.p.l.LdapSettingsManager] Groups will not be synchronized, because property 'ldap.group.baseDn' is empty.

2018.02.03 22:23:48 INFO  web[][o.s.p.l.LdapContextFactory] Test LDAP connection: FAIL

2018.02.03 22:23:48 ERROR web[][o.s.s.p.Platform] Background initialization failed. Stopping SonarQube

org.sonar.plugins.ldap.LdapException: Unable to open LDAP connection

at org.sonar.plugins.ldap.LdapContextFactory.testConnection(LdapContextFactory.java:211)

at org.sonar.plugins.ldap.LdapRealm.init(LdapRealm.java:63)

at org.sonar.server.user.SecurityRealmFactory.start(SecurityRealmFactory.java:84)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

at java.lang.reflect.Method.invoke(Unknown Source)

at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.invokeMethod(ReflectionLifecycleStrategy.java:110)

at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.start(ReflectionLifecycleStrategy.java:89)

at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.start(AbstractInjectionFactory.java:84)

at org.picocontainer.behaviors.AbstractBehavior.start(AbstractBehavior.java:169)

at org.picocontainer.behaviors.Stored$RealComponentLifecycle.start(Stored.java:132)

at org.picocontainer.behaviors.Stored.start(Stored.java:110)

at org.picocontainer.DefaultPicoContainer.potentiallyStartAdapter(DefaultPicoContainer.java:1016)

at org.picocontainer.DefaultPicoContainer.startAdapters(DefaultPicoContainer.java:1009)

at org.picocontainer.DefaultPicoContainer.start(DefaultPicoContainer.java:767)

at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:134)

at org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:90)

at org.sonar.server.platform.platformlevel.PlatformLevel4.start(PlatformLevel4.java:584)

at org.sonar.server.platform.Platform.start(Platform.java:211)

at org.sonar.server.platform.Platform.startLevel34Containers(Platform.java:185)

at org.sonar.server.platform.Platform.access$500(Platform.java:46)

at org.sonar.server.platform.Platform$1.lambda$doRun$0(Platform.java:119)

at org.sonar.server.platform.Platform$AutoStarterRunnable.runIfNotAborted(Platform.java:371)

at org.sonar.server.platform.Platform$1.doRun(Platform.java:119)

at org.sonar.server.platform.Platform$AutoStarterRunnable.run(Platform.java:355)

at java.lang.Thread.run(Unknown Source)

Caused by: javax.naming.CommunicationException: tidc:389

at com.sun.jndi.ldap.Connection.<init>(Unknown Source)

at com.sun.jndi.ldap.LdapClient.<init>(Unknown Source)

at com.sun.jndi.ldap.LdapClientFactory.createPooledConnection(Unknown Source)

at com.sun.jndi.ldap.pool.Connections.<init>(Unknown Source)

at com.sun.jndi.ldap.pool.Pool.getPooledConnection(Unknown Source)

at com.sun.jndi.ldap.LdapPoolManager.getLdapClient(Unknown Source)

at com.sun.jndi.ldap.LdapClient.getInstance(Unknown Source)

at com.sun.jndi.ldap.LdapCtx.connect(Unknown Source)

at com.sun.jndi.ldap.LdapCtx.<init>(Unknown Source)

at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(Unknown Source)

at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(Unknown Source)

at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(Unknown Source)

at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(Unknown Source)

at javax.naming.spi.NamingManager.getInitialContext(Unknown Source)

at javax.naming.InitialContext.getDefaultInitCtx(Unknown Source)

at javax.naming.InitialContext.init(Unknown Source)

at javax.naming.ldap.InitialLdapContext.<init>(Unknown Source)

at org.sonar.plugins.ldap.LdapContextFactory.createInitialDirContext(LdapContextFactory.java:134)

at org.sonar.plugins.ldap.LdapContextFactory.createBindContext(LdapContextFactory.java:96)

at org.sonar.plugins.ldap.LdapContextFactory.testConnection(LdapContextFactory.java:207)

... 26 common frames omitted

Caused by: java.net.ConnectException: Connection timed out: connect

at java.net.DualStackPlainSocketImpl.connect0(Native Method)

at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)

at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)

at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)

at java.net.AbstractPlainSocketImpl.connect(Unknown Source)

at java.net.PlainSocketImpl.connect(Unknown Source)

at java.net.SocksSocketImpl.connect(Unknown Source)

at java.net.Socket.connect(Unknown Source)

at java.net.Socket.connect(Unknown Source)

at java.net.Socket.<init>(Unknown Source)

at java.net.Socket.<init>(Unknown Source)

at com.sun.jndi.ldap.Connection.createSocket(Unknown Source)

... 46 common frames omitted

2018.02.03 22:23:48 INFO  web[][o.s.p.StopWatcher] Stopping process

Thanks,

Stuart

[Scott B.]

Hi.

Why don't you configure the SonarQube service to restart on failure?

[Stuart Stephen]

Hi Scott,

It's a good idea. I'll do that, but I still feel that the LDAP plugin shouldn't cause a fatal error and make the server go down because of the LDAP server being unavailable at a given point in time.

Cheers,

Stuart

--
You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/f3YPieE4u3I/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/aa6e8a8e-2e3f-4baf-9c9d-b1d470a0ad92%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

stuart....@gmail.com

[G. Ann Campbell]

Hi Stuart,

In general we favor a fail-fast approach. If you've misconfigured your LDAP connection, should we let that ride and simply fail to log people in for... hours? days? ... hopefully not weeks.

Because there's no way for us to tell the difference between "you've misconfigured your LDAP connection" and "your LDAP server's not back up yet". So we fail. And we think that's a good thing.

HTH,

Ann

[Stuart Stephen]

Thanks for the response, Ann.

It appears that the cause was not only windows update, but an intermittent "connection timed out" error caused by java's IPv6 networking. I've set -Djava.net.preferIPv4Stack=true to all the start commands and it appears to be connecting to the LDAP far more robustly.

Additionally, whilst attempting to fix the issues, we found the properties -Dcom.sun.jndi.ldap.connect.pool.timeout=3600000 -Dcom.sun.jndi.ldap.connect.pool.prefsize=0 made no difference whatsoever to the timeout period as it was using the Sockets default timeout.

Hope this helps someone.

Thanks,

Stuart

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/e5f8a08f-cd67-4989-a204-54f35c51c4cc%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

stuart....@gmail.com

[G. Ann Campbell]

Thanks for the followup @Stuart!

---

G. Ann Campbell | SonarSource

Product Manager

@GAnnCampbell

https://sonarsource.com