Trouble upgrading 1.6.2 -> 1.6.4

[Michael Stauffer]

Mint 14

Hi,

We're trying to upgrade an 1.6.2 install to 1.6.4. We moved the old install directly (~/xnat_fresh) and have installed 1.6.4 in the same directory, keeping all build.properties settings the same, except xdat.mail.server and xdat.mail.admin.

Following the instructions under "New Release / Old Database" on https://wiki.xnat.org/display/XNAT16/How+to+Upgrade+XNAT :

- setup.sh runs fine (after manually downloading persistence-api-1.0.2.jar)

- no project customizations

- deleted old XNAT web app

- update.sh (have tried both with and without  -Ddeploy=true)

- ran project-update.sql

After starting tomcat7, the website gives an error:

    An error has occurred.

    Please contact your IT staff or the application logs for more information.

Below is the output of log files in /var/lib/tomcat7/webapps/xnat_fresh/logs that seem most relevant. We think there might be something wrong in our database that's getting caught by 1.6.4 and not 1.6.2.

If anyone has suggestions for debugging this, we'd be very grateful.

application.log 

2015-08-18 16:54:27,859 [http-bio-8080-exec-6] ERROR org.hibernate.util.JDBCExceptionReporter - ERROR: null value in column "id" violates not-null constraint

2015-08-18 16:54:28,170 [http-bio-8080-exec-6] ERROR org.hibernate.util.JDBCExceptionReporter - ERROR: null value in column "id" violates not-null constraint

2015-08-18 16:54:28,427 [http-bio-8080-exec-6] ERROR org.hibernate.util.JDBCExceptionReporter - ERROR: null value in column "id" violates not-null constraint

prearchive.log 

2015-08-18 16:42:38,436 [localhost-startStop-1] ERROR org.nrg.xnat.helpers.prearchive.PrearcUtils - /Library/XNAT/prearchive/PEC_CLINICAL/20150402_182333521/1004236988.xml is empty.

restlet.log

2015-08-18 16:52:12,598 [localhost-startStop-1] ERROR org.nrg.xnat.restlet.servlet.XNATRestletServlet - Unable to initialize prearchive database

java.lang.NullPointerException

at org.nrg.xnat.helpers.prearchive.PrearcDatabase.getPrearchiveFolderTimestamps(PrearcDatabase.java:531)

at org.nrg.xnat.helpers.prearchive.PrearcDatabase.pruneDatabase(PrearcDatabase.java:521)

at org.nrg.xnat.helpers.prearchive.PrearcDatabase.initDatabase(PrearcDatabase.java:123)

at org.nrg.xnat.helpers.prearchive.PrearcDatabase.initDatabase(PrearcDatabase.java:67)

at org.nrg.xnat.restlet.servlet.XNATRestletServlet.init(XNATRestletServlet.java:126)

at javax.servlet.GenericServlet.init(GenericServlet.java:160)

at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1266)

at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1185)

at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1080)

at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5027)

at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5314)

at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)

at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)

at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)

at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:618)

at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1100)

at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1618)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)

at java.util.concurrent.FutureTask.run(FutureTask.java:166)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)

at java.lang.Thread.run(Thread.java:722)

sql.log

2015-08-18 16:51:56,298 - SELECT pg_get_serial_sequence('xnat_projectData_resource','xnat_projectData_resource_id') AS col_name

2015-08-18 16:51:56,306 - SELECT pg_get_serial_sequence('xnat_projectData_investigator','xnat_projectData_investigator_id') AS col_name

2015-08-18 16:51:56,315 - SELECT * FROM xnat_projectData_investigator_xnat_projectData_investigator_id_seq

2015-08-18 16:51:56,316 - SELECT * FROM xnat_projectData_investigator_xnat_projectData_investigator__seq

2015-08-18 16:51:56,322 - SELECT pg_get_serial_sequence('xnat_subjectData_resource','xnat_subjectData_resource_id') AS col_name

2015-08-18 16:51:56,327 - SELECT pg_get_serial_sequence('xnat_experimentData_resource','xnat_experimentData_resource_id') AS col_name

2015-08-18 16:51:56,328 - SELECT * FROM xnat_experimentData_resource_xnat_experimentData_resource_id_seq

2015-08-18 16:51:56,341 - SELECT pg_get_serial_sequence('xnat_datatypeProtocol_fieldGroups','xnat_datatypeProtocol_fieldGroups_id') AS col_name

2015-08-18 16:51:56,342 - SELECT * FROM xnat_datatypeProtocol_fieldGroups_xnat_datatypeProtocol_fieldGroups_id_seq

2015-08-18 16:51:56,343 - SELECT * FROM xnat_datatypeProtocol_fieldGro_xnat_datatypeProtocol_fieldGro_seq

turbine.log

2015-08-18 16:54:27,862 [http-bio-8080-exec-6] ERROR org.apache.turbine.Turbine - Turbine.handleException: 

org.hibernate.exception.ConstraintViolationException: could not insert: [org.nrg.config.entities.ConfigurationData]

<snip>

Caused by: org.postgresql.util.PSQLException: ERROR: null value in column "id" violates not-null constraint

<snip>

2015-08-18 16:54:28,173 [http-bio-8080-exec-6] ERROR org.apache.turbine.services.velocity.TurbineVelocityService - Class org.nrg.xdat.display.DisplayManager.getSingularDisplayNameForImageSession threw Exception

org.hibernate.exception.ConstraintViolationException: could not insert: [org.nrg.config.entities.ConfigurationData]

<snip>

Caused by: org.postgresql.util.PSQLException: ERROR: null value in column "id" violates not-null constraint

<snip>

2015-08-18 16:54:28,429 [http-bio-8080-exec-6] ERROR org.apache.turbine.services.velocity.TurbineVelocityService - Class org.nrg.xdat.display.DisplayManager.getPluralDisplayNameForImageSession threw Exception

org.hibernate.exception.ConstraintViolationException: could not insert: [org.nrg.config.entities.ConfigurationData]

<snip>

Caused by: org.postgresql.util.PSQLException: ERROR: null value in column "id" violates not-null constraint

<snip>

velocity.log

2015-08-18 16:54:28,465 [http-bio-8080-exec-6] ERROR velocity - Left side ($data.getSession().getAttribute("user").getUsername()) of '==' operation has null value. If a reference, it may not be in the context. Operation not possible. navigations//DefaultTop.vm [line 138, column 64]

2015-08-18 16:54:28,465 [http-bio-8080-exec-6] ERROR velocity - Left side ($sessionCount) of '>' operation has null value. Operation not possible. navigations//DefaultTop.vm [line 157, column 21]

2015-08-18 16:54:28,465 [http-bio-8080-exec-6] ERROR velocity - Left side ($sessionIpCount) of '>' operation has null value. Operation not possible. navigations//DefaultTop.vm [line 157, column 44]

2015-08-18 16:54:28,483 [http-bio-8080-exec-6] ERROR velocity - Left side ($data.getSession().getAttribute("user").getUsername()) of '==' operation has null value. If a reference, it may not be in the context. Operation not possible. navigations//DefaultTop.vm [line 4, column 76]

2015-08-18 16:54:28,484 [http-bio-8080-exec-6] ERROR velocity - Left side ($data.getSession().getAttribute("user").getUsername()) of '==' operation has null value. If a reference, it may not be in the context. Operation not possible. navigations//DefaultTop.vm [line 4, column 76]

2015-08-18 16:54:28,485 [http-bio-8080-exec-6] ERROR velocity - Left side ($data.getSession().getAttribute("user").getUsername()) of '==' operation has null value. If a reference, it may not be in the context. Operation not possible. navigations//DefaultTop.vm [line 4, column 76]

2015-08-18 16:54:28,485 [http-bio-8080-exec-6] ERROR velocity - Left side ($data.getSession().getAttribute("user").getUsername()) of '==' operation has null value. If a reference, it may not be in the context. Operation not possible. navigations//DefaultTop.vm [line 4, column 76]

2015-08-18 16:54:28,486 [http-bio-8080-exec-6] ERROR velocity - Left side ($data.getSession().getAttribute("user").getUsername()) of '==' operation has null value. If a reference, it may not be in the context. Operation not possible. navigations//DefaultTop.vm [line 4, column 76]

2015-08-18 16:54:28,486 [http-bio-8080-exec-6] ERROR velocity - Left side ($data.getSession().getAttribute("user").getUsername()) of '==' operation has null value. If a reference, it may not be in the context. Operation not possible. navigations//DefaultTop.vm [line 4, column 76]

xdat.log

2015-08-18 16:52:09,098 [localhost-startStop-1] ERROR org.nrg.xnat.turbine.utils.ArcSpecManager - 

java.lang.NullPointerException

at org.nrg.xnat.helpers.prearchive.PrearcDatabase.getPrearchiveFolderTimestamps(PrearcDatabase.java:531)

<snip>

[Michael Stauffer]

Hi again,

Some more information:

postgres version: 9.1.8

We tried a parallel clean install, i.e. not using an old database, and that works. At least the xnat webpage comes up.

Below is the output from /var/log/tomcat7/catalina.out

And here are permissions for /tomcat-users.xml (it was owned by root:tomcat7, but we changed to tomcat7:tomcat7 to see if that would help)

-rw-r----- 1 tomcat7 tomcat7 1530 May 16  2010 /var/lib/tomcat7/conf/tomcat-users.xml

Aug 18, 2015 5:08:33 PM org.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler ["http-bio-8080"]

Aug 18, 2015 5:08:33 PM org.apache.catalina.startup.Catalina load

INFO: Initialization processed in 1215 ms

Aug 18, 2015 5:08:33 PM org.apache.naming.NamingContext lookup

WARNING: Unexpected exception resolving reference

java.io.FileNotFoundException: /var/lib/tomcat7/conf/tomcat-users.xml (Permission denied)

at java.io.FileInputStream.open(Native Method)

at java.io.FileInputStream.<init>(FileInputStream.java:138)

<snip>

Aug 18, 2015 5:08:33 PM org.apache.catalina.mbeans.GlobalResourcesLifecycleListener createMBeans

SEVERE: Exception processing Global JNDI Resources

javax.naming.NamingException: /var/lib/tomcat7/conf/tomcat-users.xml (Permission denied)

at org.apache.naming.NamingContext.lookup(NamingContext.java:860)

at org.apache.naming.NamingContext.lookup(NamingContext.java:154)

<snip>

Aug 18, 2015 5:08:33 PM org.apache.catalina.core.StandardService startInternal

INFO: Starting service Catalina

Aug 18, 2015 5:08:33 PM org.apache.catalina.core.StandardEngine startInternal

INFO: Starting Servlet Engine: Apache Tomcat/7.0.30

Aug 18, 2015 5:08:33 PM org.apache.naming.NamingContext lookup

WARNING: Unexpected exception resolving reference

java.io.FileNotFoundException: /var/lib/tomcat7/conf/tomcat-users.xml (Permission denied)

at java.io.FileInputStream.open(Native Method)

at java.io.FileInputStream.<init>(FileInputStream.java:138)

at org.apache.catalina.users.MemoryUserDatabase.open(MemoryUserDatabase.java:430)

at org.apache.catalina.users.MemoryUserDatabaseFactory.getObjectInstance(MemoryUserDatabaseFactory.java:104)

at org.apache.naming.factory.ResourceFactory.getObjectInstance(ResourceFactory.java:143)

at javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:321)

at org.apache.naming.NamingContext.lookup(NamingContext.java:843)

<snip>

Aug 18, 2015 5:08:33 PM org.apache.catalina.realm.UserDatabaseRealm startInternal

SEVERE: Exception looking up UserDatabase under key UserDatabase

javax.naming.NamingException: /var/lib/tomcat7/conf/tomcat-users.xml (Permission denied)

at org.apache.naming.NamingContext.lookup(NamingContext.java:860)

at org.apache.naming.NamingContext.lookup(NamingContext.java:168)

at org.apache.catalina.realm.UserDatabaseRealm.startInternal(UserDatabaseRealm.java:255)

<snip>

Aug 18, 2015 5:08:33 PM org.apache.catalina.realm.CombinedRealm startInternal

SEVERE: Failed to start "org.apache.catalina.realm.UserDatabaseRealm/1.0" realm

org.apache.catalina.LifecycleException: Failed to start component [Realm[UserDatabaseRealm]]

at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:154)

<snip>

at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:451)

Caused by: org.apache.catalina.LifecycleException: No UserDatabase component found under key UserDatabase

at org.apache.catalina.realm.UserDatabaseRealm.startInternal(UserDatabaseRealm.java:264)

at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)

... 17 more

Aug 18, 2015 5:08:33 PM org.apache.catalina.startup.HostConfig deployWAR

INFO: Deploying web application archive /var/lib/tomcat7/webapps/xnat_fresh.war

WEBAPP:/xnat_fresh

SOURCE: /var/lib/tomcat7/webapps/xnat_fresh/

[Thread-5] ERROR db.DBAction  - Preparing to convert table xhbm_alias_token to use sequence for default value.

[Thread-5] ERROR db.DBAction  - Queries prepared for conversion:

[Thread-5] ERROR db.DBAction  -  *** CREATE SEQUENCE xhbm_alias_token_id_seq

[Thread-5] ERROR db.DBAction  -  *** ALTER TABLE xhbm_alias_token ALTER COLUMN id SET DEFAULT nextval('xhbm_alias_token_id_seq')

[Thread-5] ERROR db.DBAction  -  *** ALTER TABLE xhbm_alias_token ALTER COLUMN id SET NOT NULL

[Thread-5] ERROR db.DBAction  -  *** ALTER SEQUENCE xhbm_alias_token_id_seq OWNED BY xhbm_alias_token.id

[Thread-5] ERROR db.PoolDBUtils  - [CREATE SEQUENCE xhbm_alias_token_id_seq, ALTER TABLE xhbm_alias_token ALTER COLUMN id SET DEFAULT nextval('xhbm_alias_token_id_seq'), ALTER TABLE xhbm_alias_token ALTER COLUMN id SET NOT NULL, ALTER SEQUENCE xhbm_alias_token_id_seq OWNED BY xhbm_alias_token.id]

[Thread-5] ERROR db.PoolDBUtils  - Batch entry 3 ALTER SEQUENCE xhbm_alias_token_id_seq OWNED BY xhbm_alias_token.id was aborted.  Call getNextException to see the cause.

[Thread-5] ERROR db.PoolDBUtils  - 

org.postgresql.util.PSQLException: ERROR: sequence must have same owner as table it is linked to

at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2102)

at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1835)

<snip>

[Herrick, Rick]

Your second error is definitely because of a permissions problem. I’m surprised you’re getting anything out of Tomcat given that it can’t open that tomcat-users.xml file. I’d recommend doing something like:

sudo chmod -h tomcat7:tomcat7 /var/lib/tomcat7/*

sudo chmod -RH tomcat7:tomcat7 /var/lib/tomcat7

I think this is right, but I always have to mess with it to make sure. The first changes the ownership and group of all the symlinks in the tomcat7 folder, while the second changes the ownership and group of all of the files and folders from /var/lib/tomcat7 on down (the -H flag tells chmod to recurse into symlinked folders).

As for your original problem, the easiest thing to do would be to shut down Tomcat, drop your prearchive table altogether, and then restart. XNAT will rebuild the prearchive table for you from the files in the corresponding directory:

sudo service tomcat7 stop

psql -U xnat -c “drop table xdat_search.prearchive"

sudo service tomcat7 start

If you’re not attached to the data in the prearchive folder (like if you can just resend it or it’s old and just never got cleared out), you can also delete some or all of that to make the process of regenerating the prearchive data table go more quickly.

One other aside: we are literally days away from releasing 1.6.5, so you may want to wait for that. I am bound and determined to get this thing out by this Friday!

-- 

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

(314) 740-5961

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/d/optout.

---

The material in this message is private and may contain Protected Healthcare Information (PHI). If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

[Paul Yushkevich]

Hi Rick

Thank you for your help. I was able to figure it out this morning. The problem was actually something else, related to the contents of our PSQL database. There were two issues.

1. The ownership of some tables in our database was "xnat" and others "xnat_fresh". Appears that at some point the database user for the XNAT project was switched but table ownership not updated. However, I suspect this is not the real issue, as the error messages did not complain about access to tables. However, I went and changed the ownership of everything to xnat_fresh (by editing the sql dump file and restoring from it).

2. I turned on full logging (by changing all ERROR and WARN strings to DEBUG in WEB_INF/conf/log4j.properties. Then, I could see that there was an SQL insert statement for xhbm_configuration_data that was passing 5 values, and was causing an exception org.postgresql.util.PSQLException: ERROR: null value in column "id" violates not-null constraint that Michael reported yesterday. The table has 6 columns. I compared the database schema of this table in my database and in the clean XNAT install database, and found that they were different. The difference was that the clean install defined "id" as the auto-generated key, i.e., included 

CREATE SEQUENCE xhbm_configuration_data_id_seq

    START WITH 1

    INCREMENT BY 1

    NO MINVALUE

    NO MAXVALUE

    CACHE 1;

This was not present in my old database. So I altered my old database, and now the login window comes up and at the first look things are running ok.

I wonder if this is an issue that might affect others, and whether this should be added to the update-xnat.sql script?

Thanks again,

Paul

[Herrick, Rick]

Hey Paul,

 

I’m glad you got it working! Depending on the database configuration, the discrepancy in ownership can certainly cause errors that range from subtle and weird to disastrous and disabling, so it’s definitely something you’d want to correct. Ownership and permissions, across the database, service users (i.e. postgres, tomcat7) and groups, and folder permissions and ownership can wreak havoc.

 

As for the xhbm_configuration table, I’m at a loss to explain how that happened. Fiirst, that’s not really something we can add to the xnat-update.sql script: the configuration stuff is part of a set of libraries that we use that are based on Hibernate, while the update SQL scripts are generated by XFT, which is the core XNAT data-type management framework. They share the same database and a few other things, but for the most part exist in two separate worlds inside of XNAT. That means that the code that generates the xnat-update.sql script is simply unaware of the Hibernate-based classes and has no idea that anything in there has even changed.

 

Now, for the *most* part, Hibernate will actually update its tables on its own at start-up time. There are some occasional glitches in this that require manual intervention. If, for example, you change the type of a column from text to integer, Hibernate won’t update the column, on the not-unreasonable basis that there’s a high chance of losing data in the conversion.

 

That said, this is rather weird, since the configuration data type uses the standard NRG library persistence framework built on top of XNAT. How the ID is generated is actually built into the base framework and not into the configuration library itself. That means, if something changed in the ID generation, it should affect ALL Hibernate-based entities, of which we have quite a few, and not just one particular class. Since it’s not doing that, I can’t really say what might have happened to make that get out of sync. Generally, I haven’t seen any issue with the limited changes we made to the entity framework on the update. Obviously something happened in this case… not sure!

[Paul Yushkevich]

Rick

Although I got to the login screen and have some functionality, when I click on the Administer->Configure option, i get an error regarding not being able to load the configuration. 

I thought there might be other tables that through the upgrade might have become incompatible. So I wrote a short script pasted below, which compares the schemas between a clean install of xnat (xnat_dummy) and the upgrade of our 1.6.2 system (which itself was upgraded from 1.5 a year or so ago). 

#!/bin/bash

# List all tables in the new xnat

psql -d xnat_dummy -t -A -F ' ' -c "select tablename from pg_catalog.pg_tables where schemaname='public'" > /tmp/tables.txt

# Compare schema for each table

for table in $(cat /tmp/tables.txt); do

pg_dump -t $table --schema-only xnat_dummy > /tmp/dummy.sql

pg_dump -t $table --schema-only xnat_fresh > /tmp/fresh.sql

diff /tmp/dummy.sql /tmp/fresh.sql > /tmp/diff.txt

if [[ $? -ne 0 ]]; then

echo Table $table differs

mkdir -p diff/$table

rm -rf diff/$table/*

cp /tmp/diff.txt /tmp/dummy.sql /tmp/fresh.sql diff/$table

else

echo Table $table ok

fi

done

When I run this, I find that there are about 30 tables with different schemas.

prov_processstep                              xdat_user_history        xhbm_subscription

prov_processstep_history                      xhbm_alias_token         xhbm_user_registration_data

wrk_xnatexecutionenvironment_notify           xhbm_category            xhbm_xdat_user_auth

xdat_element_action_type                      xhbm_channel             xhbm_xft_field_exclusion

xdat_element_action_type_history              xhbm_configuration       xnat_pvisitdata

xdat_element_security_listing_action          xhbm_configuration_data  xnat_pvisitdata_history

xdat_element_security_listing_action_history  xhbm_definition          xnat_validationdata

xdat_searches                                 xhbm_notification        xnat_validationdata_history

xdat_user                                     xhbm_subscriber          xs_par_table

What would you recommend? Should I repair these (i.e., copy schemas from the xnat_dummy install and hope that the data can be still copied over)?

Or is there an alternative to import the most important data (projects/subjects/users) into a clean XNAT install?

Thank you very much

Paul

--
You received this message because you are subscribed to a topic in the Google Groups "xnat_discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/xnat_discussion/f2iq3_20CZ0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to xnat_discussi...@googlegroups.com.

To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/d/optout.

--

Paul A. Yushkevich, Ph.D.
Associate Professor
Penn Image Computing and Science Laboratory
Department of Radiology
University of Pennsylvania

[Michael Stauffer]

Paul, you rock, hunting down the problems. I'm pretty clueless as to what's going on at this point.

-M

[Herrick, Rick]

Can you look at the file WEB-INF/conf/orm-config.xml and tell me what value is set for the property hibernate.hbm2ddl.auto?

-- 

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

(314) 740-5961

[Herrick, Rick]

BTW, I meant to add… in your list of tables below, the Hibernate tables are all of those that start with the “xhbm_” prefix. The others are all within the XFT space and should be updated by the xnat-update.sql script.

-- 

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

(314) 740-5961

[Paul Yushkevich]

Thanks, Rick,

the value is "update"

I looked through the differences between the two database schemas and they seem insignificant, ranging from order of fields in a table, to some fields being NOT NULL in one version but not another.

For some reason the admin message did not come back and things seem to be working. I still get errors on startup about prearchive - in xdat.log (the prearchive directory is empty). Since that java code is part of the xnat tree, i am going to put some debug printfs there and see what's up.

java.lang.NullPointerException

        at org.nrg.xnat.helpers.prearchive.PrearcDatabase.getPrearchiveFolderTimestamps(PrearcDatabase.java:531)

        at org.nrg.xnat.helpers.prearchive.PrearcDatabase.pruneDatabase(PrearcDatabase.java:521)

        at org.nrg.xnat.helpers.prearchive.PrearcDatabase.initDatabase(PrearcDatabase.java:123)

        at org.nrg.xnat.helpers.prearchive.PrearcDatabase.initDatabase(PrearcDatabase.java:67)

        at org.nrg.xnat.turbine.utils.ArcSpecManager.GetInstance(ArcSpecManager.java:221)

        at org.nrg.xnat.turbine.utils.ArcSpecManager.GetInstance(ArcSpecManager.java:60)

        at org.nrg.xnat.security.FilterSecurityInterceptorBeanPostProcessor.initializeArcSpecInstance(FilterSecurityInterceptorBeanPostProcessor.java:103)

        at org.nrg.xnat.security.FilterSecurityInterceptorBeanPostProcessor.isRequiredLogin(FilterSecurityInterceptorBeanPostProcessor.java:110)

        at org.nrg.xnat.security.FilterSecurityInterceptorBeanPostProcessor.postProcessAfterInitialization(FilterSecurityInterceptorBeanPostProcessor.java:54)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsAfterInitialization(AbstractAutowireCapableBeanFactory.java:406)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1428)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:518)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:455)

        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)

        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)

        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)

        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192)

        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)

        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895)

        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425)

        at org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:467)

        at org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:483)

        at org.springframework.web.servlet.FrameworkServlet.initWebApplicationContext(FrameworkServlet.java:358)

        at org.springframework.web.servlet.FrameworkServlet.initServletBean(FrameworkServlet.java:325)

        at org.springframework.web.servlet.HttpServletBean.init(HttpServletBean.java:127)

[Paul Yushkevich]

Rick,

It seems that I found a small bug in PrearchDatabase.java:531. When the prearchive path contains symbolic links, the File.listFiles() command returns null. That causes the message in my previous email. See this post on how to fix it (if you think it needs fixing)

http://stackoverflow.com/questions/2471805/listfiles-of-file-not-working-on-symbolic-links

I will stop pointing to a symlink (we have /Library/XNAT pointing to another path) and hopefully that will solve some problems!

Paul

[Herrick, Rick]

Yeah, this is actually a known issue. Sometimes the canonical path will work, sometimes it won’t. Part of the issue is that symlinks vary in their implementation across different flavors of *nix and can even vary on the same flavor based on system settings. But it’s all really exacerbated by the fact that Windows has no idea what a symlink is and many of the things you can do to work around symlinks on other platforms will ruin compatibility with Windows servers. Generally it’s best to stay away from symlinks within the archive and prearchive structures if at all possible.

[Paul Yushkevich]

Rick,

One more question - for some reason after the upgrade I don't see the "DICOM headers" button on the prearchive or under the tools menu. Not even for a clean xnat install. Do I need to configure this tool somewhere? What we see is 

[View]   [Refresh] [Check] [Uncheck] ...

but we should see

[View] [Actions] [Dicom Headers]

Thanks!

Paul

[Tim Olsen]

I believe the old bulk DICOM Headers view in the prearchive was removed in 1.6.3 due to usability issues.   However, you can now review DICOM headers on a per-session basis.  Highlight a session in the prearchive, click the Details button in the right panel, and the detailed view page should support viewing the dicom headers for each scan (as well as snapshots and downloading the files).

 

Tim