Datatype suddenly missing from create/search interface

[santy...@gmail.com]

Hi!

I have a custom datatype that has suddenly disappeared from the create experiment and search listings. The element is missing in the api call to xapi/access/displays/browseableCreatable.

I cannot pinpoint a possible cause because I haven't made any changes related to it or datatype administration. It is configured to be accessible, secured, searchable and browseable, and I have never changed it.

Can I do something else to investigate the cause and bring it back?

Thanks,

Santi

[Mohana Ramaratnam]

Have you restarted tomcat as a user who does not have permissions to read the plugins directory?

Can you see the plugin installed from Administer - Site Admin - Installed Plugins

Regards,
Mohana Ramaratnam,

 moh...@xnatworks.io
 xnatworks.io | Find us on LinkedIn! 

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/xnat_discussion/d8ae1afb-29c4-472c-a2f7-bed4ebd51e5dn%40googlegroups.com.

[santy...@gmail.com]

Hi Mohana,

We have a lot of custom datatypes and all work fine, I forgot to mention. The plugin shows on the installed plugins page.

I tried making changes to the datatype config, like making it not browseable and back, but it didn't have any effect.

Santi

[Timothy Olsen]

Data types only show up if there is data present in the database and the user has access to some of them.  Is there data present in the system for this data type and does your user have access to it?

Tim

[santy...@gmail.com]

Hi Tim,

Yes, that is the odd thing. There is data for this datatype, it has been working fine until now. (But, shouldn't it appear in the new experiment list even without data? There must have been a first time).

I tried with the admin account to rule out that it is a user problem and same result. It should appear right after the amypet report.

Santi

[Rick Herrick]

What happens when you go to an existing experiment of that data type? Do you have a custom report (VM template) for the data type and, if so, is that what displays when you have an experiment open?

Rick Herrick 

Senior Software Developer

ri...@xnatworks.io

https://xnatworks.io | Find us on LinkedIn! 

To view this discussion visit https://groups.google.com/d/msgid/xnat_discussion/b4c73f66-6900-48be-8b0e-7c5a673111c9n%40googlegroups.com.

[santy...@gmail.com]

Hi Rick,

Yes, I have custom screens, and the report and edit screens work fine for existing experiments. I can review and edit data without problems.

I just checked in my local dev installation and it is not listed here either, so it is not happening in production only.

This datatype hasn't changed in years. The only recent change was a file mode set to +x, but that was also done to the rest of the plugin files because of a transfer, and the rest of the datatypes are listed as creatableBrowseable.  All schemas files are -rwx------

No reference to the datatype in any of the logs. I really don't know what to inspect to find the problem. I'm attaching the schema and plugin definition just in case.

Thanks a lot,

Santi

[Mohana Ramaratnam]

HI Santy,

Based on all the previous correspondence, I am summarizing so that we can solve the issue. 

1. You have not upgraded anything on the system (not even added any other new plugin)

2. Tomcat user which owns the Tomcat process has read permissions on the Plugins folder. 

3. You have not made any change on the production system and one fine day the custom datatype disappears from the Add Experiment interface.   

4. You can see the Plugin listed in the Installed Plugins listing

5. You can see the DataType listed in the Admin -> DataTypes interface

T1: Is your datatype, arep:AssessmentReportData,   listed when you do a GET on the endpoint - 

curl -X GET "YOUR_HOST/xapi/schemas/datatypes" -H "accept: application/json"  

T2: You have created a new project, added a subject to the project and then for the subject, from Actions -> Add Experiment you do not see 

Assessment report

Regards,
Mohana Ramaratnam,

 moh...@xnatworks.io
 xnatworks.io | Find us on LinkedIn! 

To view this discussion visit https://groups.google.com/d/msgid/xnat_discussion/6dee2262-d231-4554-90fc-0a0ba3fcbc3bn%40googlegroups.com.

[santy...@gmail.com]

Hi Mohana,

Correct.

For T1 I can see the datatype listed in the response:

But missing in the JSON response from xapi/access/displays/browseableCreatable

For T2, I created a test project and subject, and the datatype is not listed in the new experiment action.

We are running version 1.8.9.2.

Thanks,

Santi

[Rick Herrick]

Has this issue persisted across restarts of the application server? I'm wondering if it got cached (or not cached) incorrectly somehow. If so, a restart should correct the problem. You can test this out by resetting your user cache:

1. Click on your username in the upper-right corner where it says Logged in as: <username>.
2. Click the Manage Cached Resources tab.
3. Click the Reset Access Cache button.

Now see if your data type shows up in the expected locations.

You can also try a more targeted version of the call Mohana recommended:

curl "YOUR_HOST/xapi/schemas/datatypes/elements/arep:AssessmentReportData"

This should dump out something similar to this (this is for xnat:mrSessionData):

{
    "xnat_mrSessionData": {
        "defaultKey": {
            "xnat_mrSessionData_id": "xs:integer"
        },
        "fields": {
            "coil": {
                "prefix": "xnat",
                "xmlPath": "coil",
                "xmlType": "xs:string"
            },
            "fieldStrength": {
                "prefix": "xnat",
                "xmlPath": "fieldStrength",
                "xmlType": "xs:string"
            },
            "imageSessionData": {
                "prefix": "xnat",
                "xmlPath": "imageSessionData",
                "xmlType": "xnat:imageSessionData"
            },
            "marker": {
                "prefix": "xnat",
                "xmlPath": "marker",
                "xmlType": "xs:string"
            },
            "meta": {
                "prefix": "xnat",
                "xmlPath": "meta",
                "xmlType": "xnat:mrSessionData_meta_data"
            },
            "stabilization": {
                "prefix": "xnat",
                "xmlPath": "stabilization",
                "xmlType": "xs:string"
            }
        },
        "formattedName": "xnat_mrSessionData",
        "label": "mrSessionData",
        "name": "mrSessionData",
        "namespacePrefix": "xnat",
        "namespaceURI": "http://nrg.wustl.edu/xnat",
        "properName": "MRSession",
        "sqlName": "xnat_mrSessionData",
        "type": "xnat:mrSessionData",
        "xmlName": "xnat:mrSessionData"
    }
}

This output doesn't include the data-type configuration (i.e. what gets set when you go to Administer -> Data Types), but I'm wondering if anything weird shows up in that output.

Rick Herrick 

Senior Software Developer

ri...@xnatworks.io

https://xnatworks.io | Find us on LinkedIn! 

To view this discussion visit https://groups.google.com/d/msgid/xnat_discussion/6604db7f-56fb-4fdf-9e18-4d45f48595f9n%40googlegroups.com.

[santy...@gmail.com]

Hi Rick,

Yes, I have tried restarting, but since it also happens in my dev container it must be something else.

The JSON response (attached) looks fine to me.

I think I will duplicate the datatype, changing the element name and see what happens.

Santi

[santy...@gmail.com]

Hi again,

I have tried creating a test datatype from scratch and the behaviour is the same. XNAT detects it, updates the database, and it is configured in the administration page, with all checkboxes checked.

This is quite worrying because it prevents us from creating new experiments.

The schema:

<?xml version="1.0" encoding="UTF-8"?>

<xs:schema targetNamespace="http://nrg.wustl.edu/testdt" xmlns:xs="http://www.w3.org/2001/XMLSchema"

xmlns:xdat="http://nrg.wustl.edu/xdat" xmlns:xnat="http://nrg.wustl.edu/xnat"

xmlns:prov="http://www.nbirn.net/prov" xmlns:testdt="http://nrg.wustl.edu/testdt"

elementFormDefault="qualified" attributeFormDefault="unqualified">

<xs:import namespace="http://nrg.wustl.edu/xnat" schemaLocation="../xnat/xnat.xsd"></xs:import>

<xs:element name="TestElement" type="testdt:testData"></xs:element>

<xs:complexType name="testData">

<xs:annotation>

<xs:documentation>A test datatype</xs:documentation>

</xs:annotation>

<xs:complexContent>

<xs:extension base="xnat:subjectAssessorData">

<xs:sequence>

<xs:element name="test_float" type="xs:float" maxOccurs="1" minOccurs="0"></xs:element>

</xs:sequence>

</xs:extension>

</xs:complexContent>

</xs:complexType>

</xs:schema>

The model entry.

@XnatDataModel(value = "testdt:testData",

singular = "test",

plural = "tests",

code = "test")

 

The database change is picked up.

===========================
Database out of date... updating


CREATE TABLE testdt_testData_history (
testdata_info  INTEGER
, change_user  INTEGER
, id  VARCHAR(255)
, previous_change_date  TIMESTAMP
, history_id  serial  NOT NULL
, xft_version  INTEGER
, change_date  TIMESTAMP
, test_float  FLOAT
, new_row_id  VARCHAR(255)
, PRIMARY KEY (history_id) );


CREATE TABLE testdt_testData_meta_data (
xft_version  VARCHAR(255)
, last_modified  TIMESTAMP
, status  VARCHAR(255)
, activation_date  TIMESTAMP
, row_last_modified  TIMESTAMP
, insert_date  TIMESTAMP
, activation_user_xdat_user_id  INTEGER
, insert_user_xdat_user_id  INTEGER
, origin  VARCHAR(255)
, modified  INTEGER
, shareable  INTEGER
, meta_data_id  serial  NOT NULL
, PRIMARY KEY (meta_data_id) );


CREATE TABLE testdt_testData (
id  VARCHAR(255) NOT NULL DEFAULT ''
, test_float  FLOAT
, testData_info  INTEGER
, PRIMARY KEY (id) );


ALTER TABLE testdt_testData_meta_data
 ADD FOREIGN KEY (activation_user_xdat_user_id) REFERENCES xdat_user(xdat_user_id) ON DELETE  SET NULL ON UPDATE CASCADE;


ALTER TABLE testdt_testData_meta_data
 ADD FOREIGN KEY (insert_user_xdat_user_id) REFERENCES xdat_user(xdat_user_id) ON DELETE  SET NULL ON UPDATE CASCADE;


CREATE INDEX testdt_testdata_meta_data_activation_user_xdat_user_id1 ON testdt_testData_meta_data(activation_user_xdat_user_id);


CREATE INDEX testdt_testdata_meta_data_insert_user_xdat_user_id1_hash ON testdt_testData_meta_data USING HASH (insert_user_xdat_user_id);


CREATE INDEX testdt_testdata_meta_data_activation_user_xdat_user_id1_hash ON testdt_testData_meta_data USING HASH (activation_user_xdat_user_id);


CREATE INDEX testdt_testdata_meta_data_meta_data_id1_hash ON testdt_testData_meta_data USING HASH (meta_data_id);


CREATE INDEX testdt_testdata_meta_data_insert_user_xdat_user_id1 ON testdt_testData_meta_data(insert_user_xdat_user_id);


ALTER TABLE testdt_testData
 ADD FOREIGN KEY (id) REFERENCES xnat_subjectAssessorData(id) ON DELETE  CASCADE ON UPDATE CASCADE;


ALTER TABLE testdt_testData
 ADD FOREIGN KEY (testData_info) REFERENCES testdt_testData_meta_data(meta_data_id) ON DELETE  SET NULL ON UPDATE CASCADE;


CREATE INDEX testdt_testdata_id1_hash ON testdt_testData USING HASH (id);


CREATE INDEX testdt_testdata_testdata_info1 ON testdt_testData(testData_info);


CREATE INDEX testdt_testdata_testdata_info1_hash ON testdt_testData USING HASH (testData_info);


CREATE INDEX testdt_testdata_id2_hash ON testdt_testData USING HASH (id);


CREATE INDEX testdt_testdata_id1 ON testdt_testData(id);
===========================

Is there a specific place where the datatypes are filtered that we could debug? 

Thanks,

Santi

[santy...@gmail.com]

There is another thing worth mentioning. I'm not 100% sure, but this could have happened after I changed laptops, set up the build environment, and took the chance to update the build configuration.

We were using an old gradle definition that was working and stuck with the saying "if it works, don't touch it". But since I was configuring Java, gradle, docker, etc. in the new laptop, I also changed the build.gradle to match the latest from the repository.

I'm attaching the two versions so you can see what changed and if it may impact in any way. 

I have diff the jar contents, and, apart from the screens, displays etc. I only see more information in the MANIFEST and that the new jar doesn't include the property files of the bean definitions (e.g. xnat-plugin-arep-bean-definition.properties), which I see map schema element strings to bean classes.