uploading XML assessors in XNAT 1.8.9

Ben Wagner

unread,

Aug 17, 2023, 12:06:01 PM8/17/23

to xnat_discussion

Hi All,

Is anyone else having trouble uploading XML assessors to (custom) datatypes in XNAT 1.8.9. I've got a script that uploads Freesurfer information this way and it has stopped uploading since the update. Running the XML upload by hand I get this...but the field is in the XML.

Ben

$ curl -n -X PUT "https://XXX:8443/xnat/data/projects/PPP/subjects/SSSS/experiments/EEEE/assessors/AAAA_Freesurfer74r1?xsiType=fs74r1:Freesurfer74r1&allowDataDeletion=true" --data-binary @AAAAA_freesurfer74r1.xml
<html>
<head>
<title>Status page</title>
</head>
<body>
<h3>The content of element 'fs74r1:fsData' is not complete. '{"http://XXXX/fs74r1":fs_version}' Required Field
</h3><p>You can get technical details <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.1">here</a>.<br>
Please continue your visit at our <a href="/">home page</a>.
</p>
</body>
</html>

$ grep fs_version AAAA_freesurfer74r1.xml
<fs74r1:fs_version>freesurfer-linux-centos7_x86_64-7.4.1-20230613-7eb8460</fs74r1:fs_version>

Charlie Moore

unread,

Aug 17, 2023, 1:06:40 PM8/17/23

to xnat_discussion

Hi Ben,

What version are you upgrading from? That particular curl call isn't working for me even on 1.8.8.2.

Thanks,

Charlie Moore

Ben Wagner

unread,

Aug 17, 2023, 1:11:07 PM8/17/23

to xnat_discussion

Hi Charlie,

1.8.8.2. Last upload from our pipelines was 8/14. Upgraded to 1.8.9 on the 15th.

Ben

Timothy Olsen

unread,

Aug 17, 2023, 2:22:15 PM8/17/23

to xnat_discussion

Ben,

Can you send me a link to the plugin you are using? Or even just the schema.

And is there any sample xml you could send so I can be testing the right thing?

Thanks,

Tim

Charlie Moore

unread,

Aug 17, 2023, 2:24:58 PM8/17/23

to xnat_discussion

One other thing to clarify: is this for creating a new experiment entirely or updating the assessor in-place? The REST tests we maintain look like they're doing a POST to the same URL but stopping at "assessors", and then main reference I see for that URL is for updating an assessor that already exists.

Thanks,

Charlie Moore

Timothy Olsen

unread,

Aug 17, 2023, 3:21:32 PM8/17/23

to xnat_discussion

Using a different data type, I was able to get this working by adding req_format=xml or inbody=true. Without these, XNAT doesn't know you are uploading an XML and isn't very smart at guessing it. So, it knows from the xsiType in the url to create an object of that type, but then the actual object is empty because its ignoring the xml. If your datatype has any required fields, then it will fail out with a 'missing required field' message like you are seeing. Any chance those parameters were missing from your curl call? Either should do the trick.

If that's not the case here, I'll need the schema and some sample data to reproduce it. Then we can track it down for you.

I don't immediately see anything that changed in this code., but I could be missing something.

Tim

Ben Wagner

unread,

Aug 17, 2023, 4:21:27 PM8/17/23

to xnat_di...@googlegroups.com

Hi Charlie and Tim,

I've been turning this around a dozen ways the last two days. The original script creates the assessor (by a VERY simplified XML put -- which works), uploads some files, creates the detailed XML, and then does a PUT on the update XML. See the create and update XML in the attached ZIP. So, I reset my testing and the incomplete data error happens when I tried to do an upload to a non-existing assessor (aka, create). Depending on how I test I see this oddball error also. Trying the inbody and/or req_format options didn't help either.

$ curl -w "\nHTTP_CODE: %{http_code}\nTIME: %{time_total}" -b /tmp/tp0ddf39c2_0d85_4fd3_8831_15bc0df63f5f -c /tmp/tp0ddf39c2_0d85_4fd3_8831_15bc0df63f5f -X PUT "https://ansirxnat.biohpc.swmed.edu:8443/xnat/data/projects/DHMS/subjects/ANSIRUTSW_S05843/experiments/DHMS_3107148_20230815/assessors/DHMS_3107148_20230815_Freesurfer71r1?xsiType=fs71r1:Freesurfer71r1&allowDataDeletion=true&req_format=xml&inbody=true" -F "filename.txt=@./update.xml"

<html>
<head>
<title>Status page</title>
</head>
<body>

<h3>Invalid character in experiment label.</h3><p>You can get technical details <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.18">here</a>.<br>

Please continue your visit at our <a href="/">home page</a>.
</p>
</body>
</html>

HTTP_CODE: 417
TIME: 0.262

I've attached the schema and some sample logs at https://cloud.biohpc.swmed.edu/index.php/s/SRTt3q4YsfjpyY3 . Hopefully they help.

Ben

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/d38b06bf-412b-4e7d-ba3d-97d91b44f4c6n%40googlegroups.com.

Timothy Olsen

unread,

Aug 17, 2023, 4:28:22 PM8/17/23

to xnat_discussion

I saw that error when I was uploading the xml and it had the wrong label in it. When I fixed the label to match the one in the url it worked.

Tim

Timothy Olsen

unread,

Aug 17, 2023, 5:06:09 PM8/17/23

to xnat_discussion

Actually, it looks like its throwing that message because the ID in the xml is null. At least that's whats happening in my qc test. Let me see if I can find anything that changed in there.

Tim

Timothy Olsen

unread,

Aug 17, 2023, 6:40:56 PM8/17/23

to xnat_discussion

We've been digging in here and so far no luck turning up any new bugs and can't find anything that changed in this release. We'll continue testing and see what we can find.

But, from what I can tell, based on the xmls you sent, I don't see why its failing. If the ID or label in the xml was empty or had non-standard characteristics, then you could get that message. Or, if the inbody=true was missing, it would fail. But, I've tested in 1.8.8.2 and 1.8.9 and can't find any differences there.

Tim

Ben Wagner

unread,

Aug 18, 2023, 10:14:04 AM8/18/23

to xnat_di...@googlegroups.com

Ok, so I'm dumbing my testing down more. I have been able to get both the creation and update XML to upload, but here's the oddball catch...I've had to take the time off the </prov:timestamp> tags.

< <prov:timestamp>2023-08-16T11:27:37</prov:timestamp>
---
> <prov:timestamp>2023-08-16</prov:timestamp>

I'm getting an error in restlet.log:

2023-08-18 09:06:57,966 [http-nio-198.215.54.40-9080-exec-10] ERROR org.nrg.xnat.restlet.resources.SecureResource -
org.postgresql.util.PSQLException: Bad value for type timestamp/date/time: {1}
at org.postgresql.jdbc.TimestampUtils.parseBackendTimestamp(TimestampUtils.java:369)
at org.postgresql.jdbc.TimestampUtils.toTimestamp(TimestampUtils.java:400)
at org.postgresql.jdbc.PgPreparedStatement.setObject(PgPreparedStatement.java:615)
at org.postgresql.jdbc.PgPreparedStatement.setObject(PgPreparedStatement.java:908)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:529)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:529)
at org.nrg.xft.db.PoolDBUtils.executePS(PoolDBUtils.java:964)
at org.nrg.xft.db.PoolDBUtils.executeSelectPS(PoolDBUtils.java:493)
at org.nrg.xft.XFTTable.ExecutePS(XFTTable.java:48)

<snip>

Caused by: java.lang.NumberFormatException: Trailing junk on timestamp: 'T23:29:43Z'
at org.postgresql.jdbc.TimestampUtils.parseBackendTimestamp(TimestampUtils.java:359)
... 166 common frames omitted

Digging further...

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/e1d1f9d3-0123-40f8-ab4a-2f34dfdd6031n%40googlegroups.com.

Ben Wagner

unread,

Aug 18, 2023, 10:56:38 AM8/18/23

to xnat_di...@googlegroups.com

Weird, I don't know why but updating code to do timestamps in "YYYY-MM-DD HH:MM:SS" instead of "YYYY-MM-DDTHH:MM:SSZ" works. It's a way forward, but still odd...

Ben

Timothy Olsen

unread,

Aug 18, 2023, 11:13:49 AM8/18/23

to xnat_di...@googlegroups.com

Thanks for this Ben. We'll create a ticket and get a fix to support the other format.

Tim

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/CADVL1QwNpoVN83v8QQ9K5xzOH%2BaPYm1n3QPXWCWdzJxbVuz-zw%40mail.gmail.com.

Timothy Olsen

unread,

Aug 23, 2023, 5:23:26 PM8/23/23

to xnat_discussion

Ben,

The fix for this is through QA and will be released with 1.8.9.1 tomorrow.

Tim

Reply all

Reply to author

Forward