How to create image assessor with api ?

[國庭]

Dear All ,

I have a `dicom` as the assessor resource file , which can be used to upload.

And I guess it is `PUT - /data/experiments/{session-id}/assessors/{assessor-id}/{in|out}/resources/{resource-id}/files/{filename}`

But , I need a `{assessor-id}`

How can I create the assessor ,
is this `POST - /data/experiments/{session-id}/assessors` ?

But what the parameter `$file` should I input ?

The related Api documentation :
- Upload Assessor Dicom : https://wiki.xnat.org/xnat-api/image-assessor-resource-api#ImageAssessorResourceAPI-UploadANewExperimentResourceFile
- Create Assessor : https://wiki.xnat.org/xnat-api/image-assessor-api#ImageAssessorAPI-UploadANewImageAssessor



Sincerely

[John Flavin]

Hi 國庭,

The short answer to your question is contained in your screenshot from the Image Assessor API page:

The most reliable method for creating new assessors via the APl is to upload an XML definition file via a POST.

But I realize that short answer probably will not actually be very helpful. I'll speak generally first, and then get to information that is hopefully more specific to your problem.

Most broadly, XNAT organizes its data according to data types—for example, MR Session is a data type. Each of these data types can have instances with specific data, and what kind of data it can have is defined by a schema—in our example, the schema for MR Session is xnat:mrSessionData. To create an instance of one of those data types—like a new MR Session—you need to have all the required data formatted in a way that XNAT can understand. You upload that data to XNAT and it creates the new instance.

Now, for things like Subjects and Sessions, most of the time you aren't manually writing data into an XML file that conforms to a schema definition. Often you just upload data files (like DICOM files from a scan, for instance) and XNAT knows how to read properties from there and define all the Subjects, Sessions, and Scans.

Now, to your specific question. What you have is a DICOM file, and what you want to make is an assessor. I have a few questions that will hopefully lead us to the right course of action.

If you are creating an assessor, you will need to upload a structured file (usually XML) that conforms to the assessor's data type for XNAT to know how to read the data. What data do you have? Is there a data type that already exists for this data? Where did the data come from?

Are you sure that what you have is really an "assessor"? Assessors are typically used for things like "derived data", meaning some process takes the session/scan data as an input and produces new data as an output. This could be something manual like a QC Assessor, where a person looks at the session's scans and marks them as "usable" or not; or it could be an automated process like Freesurfer which processes a set of scans into a large number of new images and metrics which can be organized into a structured assessor XML and uploaded. Is what you have an assessor like that? Is this file derived from the original session's data somehow?

Alternatively, is this file an additional scan? If so, you can add a scan to your session and upload the file to that scan's resources. Or could this file be stored in the session's resources? If so, creating a new resource and uploading the file to it is simple using the Manage Files interface.

Hopefully this will help you figure out what to do with your data. But if you are still confused, or need more assistance, don't hesitate to ask.

Thanks for reaching out!

John Flavin

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/31498b9e-d5cb-4c37-83b0-6a315ddc0831n%40googlegroups.com.

[國庭]

Hi John Flavin ,

Thank you so much.

In my case , I really have the assessor (dicom) can be uploaded via Xnat RoiUploadAssistant.

But I have a job needed to upload many assessor (dicoms) automatically via calling api.

I follow your recommendation ,
writing a xml file to try creating the assessors ,
but I still meet some problems.

My curl command (user/pass is fake)
```
curl -X POST 'http://xnat-nginx/data/experiments/XNAT_E00003/assessors?inbody=true' -u fakeUser:fakePass -H 'Content-Type: application/json' -d '{"file": "/var/www/storage/xml/test.xml"}'
```

Should I follow the "Image Assessor Data REST XML Path Shortcuts" , is it right ?
`https://wiki.xnat.org/xnat-api/image-assessor-data-rest-xml-path-shortcuts`

The xml I used is below , but I am not sure is it correct ?
```
<?xml version="1.0" encoding="UTF-8"?>
<xnat xmlns:xnat="http://xnat-nginx">
    <xnat:visit_id>visit_id</xnat:visit_id>
    <xnat:date>2024-03-14</xnat:date>
    <xnat:time>10:32:27</xnat:time>
    <xnat:note>note</xnat:note>
    <xnat:pi_firstname>test</xnat:pi_firstname>
    <xnat:pi_lastname>test</xnat:pi_lastname>
    <xnat:validation_method>test</xnat:validation_method>
    <xnat:validation_status>test</xnat:validation_status>
    <xnat:validation_date>2024-03-14</xnat:validation_date>
    <xnat:validation_notes>test</xnat:validation_notes>
    <xnat:last_modified>10:32:27</xnat:last_modified>
    <xnat:insert_date>2024-03-14</xnat:insert_date>
    <xnat:insert_user>test</xnat:insert_user>
</xnat>
```

I don't know what the "xmlns:xnat={url}" should be ,
but I know `http://xnat-nginx` can point to my xnat server, same as my curl command.

It returns HTTP 422 Unprocessable Entity
```
<html>
<head>
   <title>Status page</title>
</head>
<body>
<h3>org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.</h3><p>You can get technical details <a href="http://www.webdav.org/specs/rfc2518.html#STATUS_422">here</a>.<br>
Please continue your visit at our <a href="/">home page</a>.
</p>
</body>
</html>
```

`restlet.log` :
```
2024-03-14 07:47:34,681 [http-nio-8080-exec-2] ERROR org.nrg.xnat.restlet.resources.SecureResource - An error occurred parsing the XML
org.xml.sax.SAXParseException: Content is not allowed in prolog.
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1239)
```

The detail of `restlet.log` is attached below.

Hope these can help a bit.

Sincerely

唐國庭

John Flavin 在 2024年3月13日 星期三凌晨12:19:27 [UTC+8] 的信中寫道：

[John Flavin]

Thank you for that information. Knowing that you have used the ICR ROI Upload Assistant to upload your data in the past is very helpful!

I'm not incredibly familiar with the ROI Upload Assistant or the details of what it does. That tool isn't maintained by the XNAT team but by the ICR. There are ICR members who read this board, so hopefully they can chime in to help out. But in the meantime, I'll do what I can to help.

What I can see from the ROI Upload Assistant's README file is that it does several helpful things on your behalf to make the upload process easier.

RUA [ROI Upload Assistant] will attempt to find a unique session within the project by using the Study Instance UID read from each ROI collection. Once the session is identified RUA will further check if the session contains the Series Instance UID and all SOP Instance UIDs referenced in the ROI collection. If all conditions are satisfied, RUA will upload the ROI collection to XNAT using the viewer REST API.

That last sentence is very important. The ROI Upload Assistant does not upload the data using any APIs provided by XNAT itself; instead, it uses APIs that were written by ICR and included in their XNAT OHIF Viewer Plugin. So for you to reproduce the upload process of the ROI Upload Assistant, you will want to use whatever those APIs are. Unfortunately, I think that the APIs used by the ROI Upload Assistant are not documented on the XNAT OHIF Plugin API page. And that's as far as my knowledge on the ROI Upload Assistant and the OHIF Viewer goes.

I think that if you want to write a script that will automatically perform the same functions as the ROI Upload Assistant, using the same APIs, someone from ICR will have to help out. They will know much more about the APIs and how to use them, since they're the people who wrote them and maintain them. Alternatively, I could try to help you reverse engineer the process and reproduce it using only built-in XNAT APIs. But I think that would be a lot more work, and would probably produce less consistent results, than using the APIs that the ROI Upload Assistant uses.

John Flavin

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/f129bb5c-994b-4c3b-b055-7263582f2a2an%40googlegroups.com.

[國庭]

Hi John Flavin ,

Thanks for your information !

In fact , I had tried xnat-ohif plugin to export label (ROI-Collection) ,
then I can see the assessor and assessorData (dicom format) is stored in Xnat.







Therefore , I guessed that the api - "PUT - /data/experiments/{session-id}/assessors/{assessor-id}/{in|out}/resources/{resource-id}" ,
may could be used to upload the dicom (assessorData)

So , I have some questions about these :

- 1st : If the dicom and api can be used ,
as previous discussion (2024/03/14) , when I try to create assessor by xml (attached test.xml) 
I will meet `SAXParseException` with HTTP 422 (attached restlet.log) ,

would you have idea about this ? 

my command is
```
curl -X POST 'http://xnat-nginx/data/experiments/XNAT_E00003/assessors?inbody=true' -u admin:admin -H 'Content-Type: application/json' -d '{"file": "/var/www/storage/xml/test.xml"}'
```



- 2nd : If the dicom generated from ROI may cannot be used ,
how can I generate the assessorData ?

Sincerely

唐國庭

John Flavin 在 2024年3月15日 星期五凌晨1:09:01 [UTC+8] 的信中寫道：

[John Flavin]

The reason your test XML fails to parse is because it isn’t formatted according to the schema for the ROI Collection assessor data type. I think it would be helpful to look at an example ROI Collection XML so you can see what fields are there. On the assessor report page for any one of your ROI Collection assessors, you should be able to download its XML in the actions box (View > View XML). 

What you would need to do is create a new XML document, providing values for all the fields you see in the example XML. Most of them can likely be read from your dicom file. Some, like the IDs and labels you have some freedom to make up your own values. But for consistency you should probably follow the pattern established by the IDs and labels created by the ROI Upload Assistant.

I don’t know what language you’re using, but if it is Python, I highly recommend using lxml.etree to create the XML document. https://lxml.de/tutorial.html …actually now that I think about it, I wrote a script for a container years ago which may do a lot of what you’re trying to do: https://github.com/NrgXnat/docker-images/blob/master/rt-struct-assessor/make-rt-struct-assessor.py Take a look at that and see if it helps. I don’t recommend using that script as-is—it is quite old for one thing, and I don’t even know if it would run today—but if it seems to be along the lines of what you want to do, use it as a starting point for your own use case. Also, I’m not even sure if this script maps exactly to what you’re doing.

You can view the schema definition as well. That will technically tell you everything you need to know about what goes into a valid XML document of a given data type, but it is quite difficult to understand a schema by reading its raw definition. I include the references here for completeness. If they are confusing (which I suspect they will be) then feel free to ignore them. 

This is the schema for the ROI Collection assessor: https://bitbucket.org/xnatdev/xnat-data-models/src/master/src/main/resources/schemas/roi/roi.xsd

It extends the image assessor schema, so your XML would need to follow this too: https://bitbucket.org/xnatdev/xnat-data-models/src/master/src/main/resources/schemas/xnat/xnat.xsd#lines-3153

John Flavin

On Mar 14, 2024, at 11:40 PM, 國庭 <tkt9...@gmail.com> wrote:

Hi John Flavin ,

Thanks for your information !

In fact , I had tried xnat-ohif plugin to export label (ROI-Collection) ,
then I can see the assessor and assessorData (dicom format) is stored in Xnat.

<img_1.png>

<img_2.png>

<img_4.png>

Therefore , I guessed that the api - "PUT - /data/experiments/{session-id}/assessors/{assessor-id}/{in|out}/resources/{resource-id}" ,
may could be used to upload the dicom (assessorData)

So , I have some questions about these :

- 1st : If the dicom and api can be used ,
as previous discussion (2024/03/14) , when I try to create assessor by xml (attached test.xml) 
I will meet `SAXParseException` with HTTP 422 (attached restlet.log) ,

would you have idea about this ? 

my command is
```
curl -X POST 'http://xnat-nginx/data/experiments/XNAT_E00003/assessors?inbody=true' -u admin:admin -H 'Content-Type: application/json' -d '{"file": "/var/www/storage/xml/test.xml"}'
```

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/3ee1262a-9311-4439-9d21-e9dbd454f217n%40googlegroups.com.

<test.xml>

<img_5.png>

<img_2.png>

<restlet.log>

<img_1.png>

<img_4.png>

[國庭]

Thank You Very Much !
I succeeded after following your advice.

The xml I used is downloaded from (View -> View XML)

Here I list the curl command I succeed.

The 1st curl command can use to create assessor
```
curl -X POST 'http://127.0.0.1/data/experiments/{experimentId}/assessors?inbody=true' -H 'Content-Type: application/xml' -d '<?xml ...>'
```
The `<?xml ...>` is content of the xml file.


The 2nd curl command can use to upload assessor dicom
```
curl -X PUT http://127.0.0.1/data/experiments/{experimentId}/assessors/{assessorId}/out/resources/SEG/files/test.dcm?format=json -u admin:admin -F "file=test.dcm"
```
In my case , the resource label is `SEG` instead of `DICOM` , and it is related to the xml I used

Hope these curl command can help others who meet the same problem in the future.

John Flavin 在 2024年3月15日 星期五晚上10:57:57 [UTC+8] 的信中寫道：

[James d'Arcy]

Hi  國庭

The recommended method of uploading ROI collections like RTSTRUCT and SEG is to use the XAPI provided by the viewer plugin instead of the more general XNAT built-in REST API. This is what the ROI Upload Assistant and the viewer JS

use. One of the main benefits is that the plugin's XAPI is a single call and all the metadata handling is dealt with internally so you need to supply just three things: 1) the DICOM file as the body of the request and 2) the type of the DICOM

file (RTSTRUCT or SEG) and 3) a unique label.

The XAPI can be viewed from the Swagger page via Administration -> Site Administration -> Miscellaneous -> Swagger or directly (https://your-xnat-host/xapi/swagger-ui.html). You want the ohif-roi-api section. The call for uploading an ROI collection is:

PUT https://your-xnat-host/xapi/roi/projects/{projectId}/sessions/{sessionId}/collections/{label}
The Swagger entry also shows you the available query parameters such as "overwrite" and "type". The label must be supplied by the user and be unique. The ROI Upload Assistant autogenerates this for you but if you are using curl or XNATPy or anything else, you will need to implement it yourself. You can see the general pattern if you have used ROI Upload Assistant in the past.

There are various possible failure modes for the upload and the response code and message should be helpful here. The viewer plugin tries to maintain data integrity by rejecting an upload that does not match what is already in XNAT. The checks are:

1. Check the StudyInstanceUID of the XNAT session is referenced in the uploaded file.
2. Check the SeriesInstanceUID referenced in the uploaded file is present in one of the scans within the session.
3. Check the SOPInstanceUIDs referenced in the uploaded file are present in the scan corresponding to the referenced SeriesInstanceUID.

If one of these checks fails you will get an error return and a message about "Missing dependencies" and the UID type that failed the check. A successful upload should return 200 and the ID of the ROI collection.

If you are attempting to reupload an ROI collection, i.e. same label, the plugin will prevent overwriting by default and you will need to set the overwrite query parameter.

The ROI Upload Assistant uses a couple of additional XAPI endpoints to perform the above checks on the client side before attempting the actual upload. This allows more immediate feedback in the UI, disambiguation where the same StudyInstanceUID is found in multiple sessions and user choice to overwrite an existing ROI collection. I can explain their operation if you wish.

Hope this is helpful

James