OBA not starting anymore

61 views
Skip to first unread message

kossavot

unread,
May 13, 2021, 12:43:10 PM5/13/21
to onebusaway-developers
Hi,
I've been running an instance of oba v2 for months without issues. Tomcat is restarted almost daily to allow to reload the updated bundle. No issues for months as I said.

Today I noticed that one of the server was down and I looked at the logs.

This is what I found:

2021-05-13 18:20:49,809 INFO  [ContextLoader.java:304] : Root WebApplicationContext: initialization started
2021-05-13 18:20:50,054 INFO  [AbstractApplicationContext.java:583] : Refreshing Root WebApplicationContext: startup date [Thu May 13 18:20:50 CEST 2021]; root of context hierarchy
2021-05-13 18:20:50,114 INFO  [XmlBeanDefinitionReader.java:317] : Loading XML bean definitions from class path resource [org/onebusaway/transit_data_federation/application-context-webapp.xml]
2021-05-13 18:20:50,202 INFO  [XmlBeanDefinitionReader.java:317] : Loading XML bean definitions from class path resource [org/onebusaway/transit_data_federation/application-context.xml]
2021-05-13 18:20:50,216 INFO  [XmlBeanDefinitionReader.java:317] : Loading XML bean definitions from class path resource [org/onebusaway/transit_data_federation/application-context-services.xml]
2021-05-13 18:20:50,233 INFO  [XmlBeanDefinitionReader.java:317] : Loading XML bean definitions from class path resource [org/onebusaway/transit_data_federation/application-context-common.xml]
2021-05-13 18:20:50,243 INFO  [XmlBeanDefinitionReader.java:317] : Loading XML bean definitions from class path resource [org/onebusaway/container/application-context.xml]
2021-05-13 18:20:50,254 INFO  [XmlBeanDefinitionReader.java:317] : Loading XML bean definitions from class path resource [org/onebusaway/container/application-context-hibernate.xml]
2021-05-13 18:20:50,265 INFO  [XmlBeanDefinitionReader.java:317] : Loading XML bean definitions from class path resource [org/onebusaway/container/application-context-caching.xml]
2021-05-13 18:20:50,278 INFO  [XmlBeanDefinitionReader.java:317] : Loading XML bean definitions from class path resource [org/onebusaway/container/application-context-common.xml]
2021-05-13 18:20:50,522 INFO  [DefaultListableBeanFactory.java:826] : Overriding bean definition for bean 'extendedCalendarServiceImpl' with a different definition: replacing [Generic bean: class [org.onebusaway.transit_data_federation.impl.ExtendedCale
ndarServiceImpl]; scope=singleton; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in URL [jar:f
ile:/opt/tomcat/webapps/onebusaway-transit-data-federation-webapp/WEB-INF/lib/onebusaway-transit-data-federation-2.0.71-cs.jar!/org/onebusaway/transit_data_federation/impl/ExtendedCalendarServiceImpl.class]] with [Generic bean: class [org.onebusaway.tra
nsit_data_federation.impl.ExtendedCalendarServiceImpl]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodNam
e=null; defined in class path resource [org/onebusaway/transit_data_federation/application-context-services.xml]]
2021-05-13 18:20:50,522 INFO  [DefaultListableBeanFactory.java:826] : Overriding bean definition for bean 'jmxNamingStategy' with a different definition: replacing [Generic bean: class [org.springframework.jmx.export.naming.MetadataNamingStrategy]; scop
e=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in class path resource [org/onebusaway/conta
iner/application-context.xml]] with [Generic bean: class [org.onebusaway.container.spring.jmx.ServletContextAwareMetadataNamingStrategy]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; f
actoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in class path resource [org/onebusaway/transit_data_federation/application-context-webapp.xml]]
2021-05-13 18:20:50,523 INFO  [XmlBeanDefinitionReader.java:317] : Loading XML bean definitions from class path resource [data-sources.xml]
2021-05-13 18:20:50,533 INFO  [DefaultListableBeanFactory.java:826] : Overriding bean definition for bean 'dataSource' with a different definition: replacing [Generic bean: class [org.springframework.jdbc.datasource.DriverManagerDataSource]; scope=; abs
tract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in class path resource [org/onebusaway/transit_data
_federation/application-context-common.xml]] with [Generic bean: class [org.springframework.jdbc.datasource.DriverManagerDataSource]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; facto
ryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in class path resource [data-sources.xml]]
2021-05-13 18:20:50,800 INFO  [AutowiredAnnotationBeanPostProcessor.java:155] : JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
2021-05-13 18:20:51,078 INFO  [EhCacheConfigurationFactoryBean.java:74] : Initializing EHCache CacheManager
2021-05-13 18:20:51,099 INFO  [EhCacheConfigurationFactoryBean.java:77] : diskStorePath=/tmp/cache
2021-05-13 18:20:51,100 INFO  [EhCacheConfigurationFactoryBean.java:80] : diskStorePath (translated)=/tmp/cache
2021-05-13 18:20:51,160 INFO  [EhCacheManagerFactoryBean.java:75] : Initializing EHCache CacheManager
2021-05-13 18:20:51,400 INFO  [DriverManagerDataSource.java:133] : Loaded JDBC driver: com.mysql.jdbc.Driver
2021-05-13 18:20:51,561 INFO  [Version.java:37] : HCANN000001: Hibernate Commons Annotations {4.0.1.Final}
2021-05-13 18:20:51,567 INFO  [Version.java:41] : HHH000412: Hibernate Core {4.0.1.Final}
2021-05-13 18:20:51,569 INFO  [Environment.java:239] : HHH000206: hibernate.properties not found
2021-05-13 18:20:51,571 INFO  [Environment.java:342] : HHH000021: Bytecode provider name : javassist
2021-05-13 18:20:52,355 WARN  [JdbcServicesImpl.java:171] : HHH000342: Could not obtain connection to query metadata : The server time zone value 'CEST' is unrecognized or represents more than one time zone. You must configure either the server or JDBC
driver (via the 'serverTimezone' configuration property) to use a more specific time zone value if you want to utilize time zone support.
2021-05-13 18:20:52,356 WARN  [AbstractApplicationContext.java:551] : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactor
y' defined in class path resource [org/onebusaway/container/application-context-hibernate.xml]: Invocation of init method failed; nested exception is org.hibernate.HibernateException: Connection cannot be null when 'hibernate.dialect' not set
2021-05-13 18:20:52,356 INFO  [EhCacheManagerFactoryBean.java:112] : Shutting down EHCache CacheManager
2021-05-13 18:20:52,360 ERROR [ContextLoader.java:350] : Context initialization failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory' defined in class path resource [org/onebusaway/container/application-context-hibernate.xml]: Invocation of init method failed; nested exception is or
g.hibernate.HibernateException: Connection cannot be null when 'hibernate.dialect' not set
       at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1631)
       at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
       at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
       at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
       at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
       at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
       at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
       at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:742)
       at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
       at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
       at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
       at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
       at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
       at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4716)
       at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5177)
       at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
       at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:717)
       at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:690)
       at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:706)
       at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1133)
       at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1866)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
       at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
       at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:1045)
       at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:429)
       at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1576)
       at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:309)
       at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)
       at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:423)
       at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:366)
       at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:936)
       at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:843)
       at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
       at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1384)
       at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1374)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
       at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
       at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:909)
       at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:262)
       at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
       at org.apache.catalina.core.StandardService.startInternal(StandardService.java:434)
       at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
       at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:930)
       at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
       at org.apache.catalina.startup.Catalina.start(Catalina.java:772)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:342)
       at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:473)
Caused by: org.hibernate.HibernateException: Connection cannot be null when 'hibernate.dialect' not set
       at org.hibernate.service.jdbc.dialect.internal.DialectFactoryImpl.determineDialect(DialectFactoryImpl.java:97)
       at org.hibernate.service.jdbc.dialect.internal.DialectFactoryImpl.buildDialect(DialectFactoryImpl.java:67)
       at org.hibernate.engine.jdbc.internal.JdbcServicesImpl.configure(JdbcServicesImpl.java:172)
       at org.hibernate.service.internal.StandardServiceRegistryImpl.configureService(StandardServiceRegistryImpl.java:75)
       at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:159)
       at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:131)
       at org.hibernate.cfg.SettingsFactory.buildSettings(SettingsFactory.java:71)
       at org.hibernate.cfg.Configuration.buildSettingsInternal(Configuration.java:2270)
       at org.hibernate.cfg.Configuration.buildSettings(Configuration.java:2266)
       at org.springframework.orm.hibernate4.LocalSessionFactoryBuilder.buildSettings(LocalSessionFactoryBuilder.java:343)
       at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1735)
       at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1775)
       at org.springframework.orm.hibernate4.LocalSessionFactoryBuilder.buildSessionFactory(LocalSessionFactoryBuilder.java:372)
       at org.springframework.orm.hibernate4.LocalSessionFactoryBean.buildSessionFactory(LocalSessionFactoryBean.java:454)
       at org.springframework.orm.hibernate4.LocalSessionFactoryBean.afterPropertiesSet(LocalSessionFactoryBean.java:439)
       at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1689)
       at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1627)
       ... 53 more

Any idea on what's going on ?

Thanks in advance.

Sheldon A. Brown

unread,
May 14, 2021, 10:16:12 AM5/14/21
to onebusaway...@googlegroups.com
Best guess based on the stack trace and some issues I've been troubleshooting:

your JDBC driver is trying to establish an SSL connection on your
behalf, and failed to do so when it loaded the time zone. You need to
specify the java timezone more specifically, such as
"America/New_York" instead of "EST". You can do this via a jvm arg or
set it at the O/S level, either one should fix it. An upgrade of your
database server likely caused this situation, which is why it was a
surprise to you.

Sheldon
> --
> You received this message because you are subscribed to the Google Groups "onebusaway-developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to onebusaway-devel...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/onebusaway-developers/2d7bdcad-686f-4937-a1df-b09f789e07e2n%40googlegroups.com.

kossavot

unread,
May 14, 2021, 12:19:12 PM5/14/21
to onebusaway-developers
Thanks Sheldon,
you're right. I've fixed it by appending the timezone to the various url properties which set the jdbc:mysql address.
The weird thing is that I haven't updated the db server myself unless it was updated by the system with an unattended update.

Best regards
Reply all
Reply to author
Forward
0 new messages