problems after reboot

[paulo rodrigues]

xnat was working fine, but now after a reboot this happens:

HTTP Status 500 -

---

type Exception report

message

description The server encountered an internal error () that prevented it from fulfilling this request.

exception

java.lang.IllegalStateException: No WebApplicationContext found: no ContextLoaderListener registered?
	org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:159)

note The full stack trace of the root cause is available in the Apache Tomcat/7.0.26 logs.

and this in the log:

Nov 23, 2013 1:58:13 AM org.apache.catalina.core.StandardWrapperValve invoke

SEVERE: Servlet.service() for servlet [xnat] in context with path [/xnat] threw exception

java.lang.IllegalStateException: No WebApplicationContext found: no ContextLoaderListener registered?

at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:159)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)

at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)

at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)

at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:987)

at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579)

at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:724)

What got broken???

[Herrick, Rick]

When you get a "No WebApplicationContext found" error, that means a catastrophic failure of some sort and even the most basic semblance of a web application couldn't be created. Somewhere or another, likely in your catalina.out or maybe the localhost log, you'll find another error message that will indicate the root cause of the error.

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

(314) 827-4250

---

From: xnat_di...@googlegroups.com [xnat_di...@googlegroups.com] on behalf of paulo rodrigues [kha...@gmail.com]
Sent: Friday, November 22, 2013 7:00 PM
To: xnat_di...@googlegroups.com
Subject: [XNAT Discussion] problems after reboot

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/groups/opt_out.

---

The material in this message is private and may contain Protected Healthcare Information (PHI). If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

[paulo rodrigues]

hmm seems that the postgres port 5432 was still open, but not really postmaster running?

I killed the postgres that was running and started it again. seemed to work.

But I got these when restart tomcat7

 * Stopping Tomcat servlet engine tomcat7                                                                                                                                                            [ OK ]

 * Starting Tomcat servlet engine tomcat7                                                                                                                                                            [ OK ]

/var# ERROR:  column "xnat_projectData_resource_id" of relation "xnat_projectdata_resource" does not exist

STATEMENT:  SELECT pg_get_serial_sequence('xnat_projectData_resource','xnat_projectData_resource_id') AS col_name

ERROR:  column "xnat_projectData_investigator_id" of relation "xnat_projectdata_investigator" does not exist

STATEMENT:  SELECT pg_get_serial_sequence('xnat_projectData_investigator','xnat_projectData_investigator_id') AS col_name

ERROR:  relation "xnat_projectdata_investigator_xnat_projectdata_investigator_id_" does not exist at character 15

STATEMENT:  SELECT * FROM xnat_projectData_investigator_xnat_projectData_investigator_id_seq

ERROR:  relation "xnat_projectdata_investigator_xnat_projectdata_investigator__se" does not exist at character 15

STATEMENT:  SELECT * FROM xnat_projectData_investigator_xnat_projectData_investigator__seq

ERROR:  column "xnat_subjectData_resource_id" of relation "xnat_subjectdata_resource" does not exist

STATEMENT:  SELECT pg_get_serial_sequence('xnat_subjectData_resource','xnat_subjectData_resource_id') AS col_name

ERROR:  column "xnat_experimentData_resource_id" of relation "xnat_experimentdata_resource" does not exist

STATEMENT:  SELECT pg_get_serial_sequence('xnat_experimentData_resource','xnat_experimentData_resource_id') AS col_name

ERROR:  relation "xnat_experimentdata_resource_xnat_experimentdata_resource_id_se" does not exist at character 15

STATEMENT:  SELECT * FROM xnat_experimentData_resource_xnat_experimentData_resource_id_seq

ERROR:  column "xnat_datatypeProtocol_fieldGroups_id" of relation "xnat_datatypeprotocol_fieldgroups" does not exist

STATEMENT:  SELECT pg_get_serial_sequence('xnat_datatypeProtocol_fieldGroups','xnat_datatypeProtocol_fieldGroups_id') AS col_name

ERROR:  relation "xnat_datatypeprotocol_fieldgroups_xnat_datatypeprotocol_fieldgr" does not exist at character 15

STATEMENT:  SELECT * FROM xnat_datatypeProtocol_fieldGroups_xnat_datatypeProtocol_fieldGroups_id_seq

ERROR:  relation "xnat_datatypeprotocol_fieldgro_xnat_datatypeprotocol_fieldgro_s" does not exist at character 15

STATEMENT:  SELECT * FROM xnat_datatypeProtocol_fieldGro_xnat_datatypeProtocol_fieldGro_seq

ERROR:  relation "activemq_msgs" already exists

STATEMENT:  CREATE TABLE ACTIVEMQ_MSGS(ID BIGINT NOT NULL, CONTAINER VARCHAR(250), MSGID_PROD VARCHAR(250), MSGID_SEQ BIGINT, EXPIRATION BIGINT, MSG BYTEA, PRIMARY KEY ( ID ) )

ERROR:  current transaction is aborted, commands ignored until end of transaction block

STATEMENT:  CREATE INDEX ACTIVEMQ_MSGS_MIDX ON ACTIVEMQ_MSGS (MSGID_PROD,MSGID_SEQ)

ERROR:  current transaction is aborted, commands ignored until end of transaction block

STATEMENT:  CREATE INDEX ACTIVEMQ_MSGS_CIDX ON ACTIVEMQ_MSGS (CONTAINER)

ERROR:  current transaction is aborted, commands ignored until end of transaction block

STATEMENT:  CREATE INDEX ACTIVEMQ_MSGS_EIDX ON ACTIVEMQ_MSGS (EXPIRATION)

ERROR:  current transaction is aborted, commands ignored until end of transaction block

STATEMENT:  CREATE TABLE ACTIVEMQ_ACKS(CONTAINER VARCHAR(250) NOT NULL, SUB_DEST VARCHAR(250), CLIENT_ID VARCHAR(250) NOT NULL, SUB_NAME VARCHAR(250) NOT NULL, SELECTOR VARCHAR(250), LAST_ACKED_ID BIGINT, PRIMARY KEY ( CONTAINER, CLIENT_ID, SUB_NAME))

ERROR:  current transaction is aborted, commands ignored until end of transaction block

STATEMENT:  CREATE TABLE ACTIVEMQ_LOCK( ID BIGINT NOT NULL, TIME BIGINT, BROKER_NAME VARCHAR(250), PRIMARY KEY (ID) )

ERROR:  current transaction is aborted, commands ignored until end of transaction block

STATEMENT:  INSERT INTO ACTIVEMQ_LOCK(ID) VALUES (1)

ERROR:  current transaction is aborted, commands ignored until end of transaction block

STATEMENT:  ALTER TABLE ACTIVEMQ_MSGS ADD PRIORITY BIGINT

ERROR:  current transaction is aborted, commands ignored until end of transaction block

STATEMENT:  CREATE INDEX ACTIVEMQ_MSGS_PIDX ON ACTIVEMQ_MSGS (PRIORITY)

ERROR:  current transaction is aborted, commands ignored until end of transaction block

STATEMENT:  ALTER TABLE ACTIVEMQ_MSGS ADD XID BYTEA

ERROR:  current transaction is aborted, commands ignored until end of transaction block

STATEMENT:  ALTER TABLE ACTIVEMQ_ACKS ADD PRIORITY BIGINT DEFAULT 5 NOT NULL

ERROR:  current transaction is aborted, commands ignored until end of transaction block

STATEMENT:  ALTER TABLE ACTIVEMQ_ACKS ADD XID BYTEA

ERROR:  current transaction is aborted, commands ignored until end of transaction block

STATEMENT:  ALTER TABLE ACTIVEMQ_ACKS DROP CONSTRAINT "activemq_acks_pkey"

ERROR:  current transaction is aborted, commands ignored until end of transaction block

STATEMENT:  ALTER TABLE ACTIVEMQ_ACKS ADD PRIMARY KEY (CONTAINER, CLIENT_ID, SUB_NAME, PRIORITY)

[paulo rodrigues]

And again this happened.. and now my previous 'fix' or miraculous 'it works again' is not working...

Anybody experiences this stuff?

[Mohana Ramaratnam]

Did you change your project name from xnat to something else? Looks like there is no folder called xnat in TOMCAT_HOME/webapps*.

If that is the case, modify the TOMCAT_HOME/conf/server.xml to set the correct context.

[paulo rodrigues]

There is such a folder and I didn't change the name of it. It was fine, but then after the reboot, not anymore..

[Herrick, Rick]

If the Tomcat context configuration is OK, then likely what's happening is that the xnat application just isn't getting initialized properly. Usually this means there's a bad value in one of the configuration files located in WEB-INF/conf under your installed web app. And usually when it fails so catastrophically that the web app isn't getting initialized, there's an error message in the Tomcat logs, since the error happens before there really is an application to do any logging of its own.

Try looking in the logs folder under your Tomcat installation--not the xnat web app, mind you, but the Tomcat installation, so e.g. /usr/share/tomcat6/logs or /var/lib/tomcat7/logs or whatever depending on your platform and Tomcat version. I'd give a 95% chance that whatever's messed up is being logged in there somewhere.

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

(314) 827-4250

---

From: xnat_di...@googlegroups.com [xnat_di...@googlegroups.com] on behalf of Mohana Ramaratnam [mohana...@gmail.com]
Sent: Wednesday, January 08, 2014 7:07 PM
To: xnat_di...@googlegroups.com
Subject: Re: [XNAT Discussion] Re: problems after reboot

[Herrick, Rick]

Oh, one other thing worth mentioning: the error you're looking for will most likely be in the Tomcat logs, but it will NOT be something that happens everytime you try to access the application. Instead it'll occur on Tomcat restart. So a good way do remove a lot of the noise in the logs folder is just to delete everything, e.g.:

sudo service tomcat6 stop

rm -r /usr/share/tomcat6/logs/*

rm -r /usr/share/tomcat6/webapps/xnat/logs/*

sudo service tomcat6 start

Now you can look in the logs and see what scary messages arise. You shouldn't even need to access the web app if I'm correct, since what you're interested in would already be there.

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

(314) 827-4250

---

From: xnat_di...@googlegroups.com [xnat_di...@googlegroups.com] on behalf of Herrick, Rick [herr...@mir.wustl.edu]
Sent: Wednesday, January 08, 2014 8:08 PM
To: xnat_di...@googlegroups.com
Subject: RE: [XNAT Discussion] Re: problems after reboot

[paulo rodrigues]

hmmm, the errors I get are related to the xnat database.. weird?

Jan 09, 2014 11:05:33 PM org.apache.catalina.core.StandardContext loadOnStartup

SEVERE: Servlet /xnat threw load() exception

org.postgresql.util.PSQLException: FATAL: no pg_hba.conf entry for host "127.0.0.1", user "xnat01", database "xnat", SSL off

at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:291)

at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)

at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)

at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)

at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)

at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)

at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:30)

at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)

at org.postgresql.Driver.makeConnection(Driver.java:393)

...