Problem to extend freesurfer schema

[Alexis Barbot]

Hi all,

I have some trouble to extend the freesurfer schema embedded in xnat 1.4. I have attached the file I have started to write. Basically what I want is to store results from freesurfer 5.
The problem is with aseg and aparc data. As there are different/more fields than in the schema from xnat, I have tried to extend the datatypes. The problems I had:
 1- I first created a fs.xsd schema in our imagen schemas folder. But it raised datatypes dependency errors while compiling and when I succeeded after changing order in InstanceSettings.xml, my datatypes just didn't appear. So it seems it's not possible to have 2 schemas with the same file name even if the folder is different ? ( I also took care to set a different targetnamespace )
 2- so I changed the schema filename. Then I created the datatypes ASEGRegionAnalysis and APARCRegionAnalysis, as extensions of the datatypes with the same name in fs.xsd. Then again it didn't appear at all, so it seems it's not possible to set two datatypes with the same name in two distinct schemas?
 3- Finally I changed the names to ASEG and APARC. Now I can compile, deploy and configure security for these elements. Problem: when I try to store a xml, I get the message " Invalid field 'imageSession_ID' " ( see xdat_log.txt file for more details ). This doesn't occur with the native fs datatypes, nor with the elements which are simple mrassessor extensions, but just happen for the two extensions of fs elements.

So, do I have an error in my schema? What could I do to fix this?

Thanks,

Alexis

[Mohana Ramaratnam]

Hi Alexis,

We at Washington University in St Louis have created a new schema
representation to represent the Freesurfer data. Freesurfer data can
be cross-sectional and longitudinal. The new representation contains
both the ASEG and the APARC measurements in one assessor.

I have attached the new schema. The schema contains
fs:asegRegionAnalysis and fs:aparcRegionAnalysis which are deprecated
in our instance.

Regards
Mohana

> --
> You received this message because you are subscribed to the Google Groups
> "xnat_discussion" group.
> To post to this group, send email to xnat_di...@googlegroups.com.
> To unsubscribe from this group, send email to
> xnat_discussi...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/xnat_discussion?hl=en.
>

[Satrajit Ghosh]

hi mohana,

thanks very much for this. 

i've going through the shell script you had sent a while earlier. in the script the entire directory is zipped up and sent to the server. is there a way to get that enter directory back? also i'm less interested in the derived data for the moment, but can i run a query to find all left hemisphere white matter surfaces and all left hemisphere sulc files?

i don't understand xnat well enough, so here are some questions

XNATRestClient -host $host $connect_str -local ${freesurfer_out}/${session}_freesurfer5.xml -remote $restPath -m PUT

i presume the xml file comes from a pipeline in xnat. what if i did my freesurfer processing separately from xnat pipelines. is this statement necessary?

could i simply do these two (most likely with pyxnat) to upload the data and be able to run the above query?

XNATRestClient -host $host $connect_str -m PUT -remote "/data/archive/experiments/${session}/assessors/${fs_id}/resources/DATA"

XNATRestClient -host $host $connect_str -m PUT -remote "/data/archive/experiments/${session}/assessors/${fs_id}/resources/DATA/files?extract=true&content=DATA" -local ${freesurfer_out}/${sessLabel}.zip

cheers,

satra

[Mohana Ramaratnam]

Hi Satra,

> i've going through the shell script you had sent a while earlier. in the
> script the entire directory is zipped up and sent to the server. is there a
> way to get that enter directory back? also i'm less interested in the
> derived data for the moment, but can i run a query to find all left
> hemisphere white matter surfaces and all left hemisphere sulc files?
> i don't understand xnat well enough, so here are some questions

The way to get a catalog of files back would be:

GET request on
/data/archive/experiments/EXPT_ACCESSION_ID/assessors/FS_ASSESSOR_ID/resources/DATA/files?format=zip

Depending on the way you choose to catalog the files, you could get a
sub-folder, which would contain the required files or a specific
file. So for example, to get the recon-all.log you would:

GET /data/archive/experiments/EXPT_ACCESSION_ID/assessors/FS_ASSESSOR_ID/resources/DATA/files/EXPT_LABEL/scripts/recon-all.log

(EXPT_LABEL happens to be the root folder when the files are uploaded,
if you create the zip in another format, you would create an
appropriate URL).

>
> XNATRestClient -host $host $connect_str -local
> ${freesurfer_out}/${session}_freesurfer5.xml -remote $restPath -m PUT
>
> i presume the xml file comes from a pipeline in xnat. what if i did my
> freesurfer processing separately from xnat pipelines. is this statement
> necessary?

XML file is generated by a perl script after the FS processing is
complete. You could invoke the perl script independent of the
pipeline.

The XML file is inserted into XNAT to represent a FS assessor. After
the assessor is inserted, files (catalog(s) of file(s)) can be
associated with the assessor. So you would run the following two
calls after the assessor exists in XNAT.

[Alexis]

Hi Mohana,

Thanks for the schema. So if I understand correctly the schema, to
store freesurfer data from an individual subject, I just need the
"Freesurfer" datatype? Does it mean if I wanted to have all files
indexed, I have to put all of them as resources of the same elements?
If you are using this shema, maybe you could also share a display xml
document, and probably some java action files to download images?

Alexis

On 27 sep, 19:07, Mohana Ramaratnam <mohanakann...@gmail.com> wrote:
> Hi Alexis,
>
> We at Washington University in St Louis have created a new schema
> representation to represent the Freesurfer data. Freesurfer data can
> be cross-sectional and longitudinal. The new representation contains
> both the ASEG and the APARC measurements in one assessor.
>
> I have attached the new schema.  The schema contains
> fs:asegRegionAnalysis and fs:aparcRegionAnalysis which are deprecated
> in our instance.
>
> Regards
> Mohana
>

>  fs.xsd
> 26KAfficherTélécharger

[Mohana Ramaratnam]

Hi Alexis,

The Freesurfer datatype is fs:fsdata for representing cross-sectional
data. We create a catalog of the entire output of the Freesurfer.

We download the images using REST calls. (ref: my prev post in this thread).

I am attaching the template file.

Regards
Mohana

[Alexis Barbot]

Thanks Mohana,

So you create one big catalog for all output files, and have you written some classes to allow users to download one specific file for a list of subject? (let's say someone want to download all aseg.stats files, or all brainmask.mgz ... ). Thanks for the report display file, I would have been more interested by a listing file because our users nearly don't use report pages, only listings of all subjects.
And last: would it be relevant to create one catalog per directory? (ie one for the stats dir, one for the mri dir, one for the surf... ). The next task I'll do would be to add columns in the listing to display the availability of files, so users can immediately know and filter which subjects have a specific file.

Alexis

2011/9/28 Mohana Ramaratnam <mohana...@gmail.com>

[Olsen, Tim]

Mohana,

 

Don’t we also have tools which will parse a stat file and generate the XNAT freesurfer XML?  What is that written in?

 

If that tool works consistently, then we could bundle it as part of a FreesurferImporter.  It could be an additional implementation of the ‘import’ service available via REST.  That way, a user could just push up the Freesurfer files, and XNAT could handle generating the xml, catalogs, etc.  Post 1.6, this could be easily integrated into XNAT servers using the module structure (or plugins, post-2.0… next year).  Is there any other info needed that is only known on the client side?  Provenance?

 

Regarding potential catalog structures, we’ve gone around and around on handling Freesurfer files for years.  You certainly could separate the files into separate catalog files.  The benefit here is that the catalog properties (label, format, content, etc) are stored in the database and thus easily query-able.   Versus file level properties which are stored in the catalog xml stored on the file system, and much more complicated to query across sessions.  On the other hand, you certainly wouldn’t want a catalog for each file as this would quickly overwhelm your xnat_resource table.  Finding the right solution to this has been a popular debate topic in our group for years, with the penultimate solution still undecided.

 

If you use the single catalog strategy, you could use custom variables to store whether or not certain files are expected to be present.  Or, you could separate the common files (stat files, brainmask, etc) into separate catalogs and lump everything else into one big catalog.  The benefit to option 1 is that querying custom variables across sessions is already supported in XNAT.  Whereas, option 2 wouldn’t be query-able across sessions out of the box (though it could easily be configured via a custom SQL View in the display docs).

 

Tim

---

The material in this message is private and may contain Protected Healthcare Information (PHI). If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

[Satrajit Ghosh]

hi tim,

If that tool works consistently, then we could bundle it as part of a FreesurferImporter.  It could be an additional implementation of the ‘import’ service available via REST.  That way, a user could just push up the Freesurfer files, and XNAT could handle generating the xml, catalogs, etc.  Post 1.6, this could be easily integrated into XNAT servers using the module structure (or plugins, post-2.0… next year). 

that would be awesome!

Is there any other info needed that is only known on the client side?  Provenance?

 

just spoke to bruce fischl and company and we should have recon-all provenance output in openprovenancemodel format hopefully within a month or two. since this will be just another file within the freesurfer tree, xnat could easily parse this into the provenance entry for freesurfer resource. btw, dave keator is incorporating this into xcede as a replacement for the BIRN provenance.

Regarding potential catalog structures, we’ve gone around and around on handling Freesurfer files for years.  You certainly could separate the files into separate catalog files.  The benefit here is that the catalog properties (label, format, content, etc) are stored in the database and thus easily query-able.   Versus file level properties which are stored in the catalog xml stored on the file system, and much more complicated to query across sessions.  On the other hand, you certainly wouldn’t want a catalog for each file as this would quickly overwhelm your xnat_resource table.  Finding the right solution to this has been a popular debate topic in our group for years, with the penultimate solution still undecided.

 

If you use the single catalog strategy, you could use custom variables to store whether or not certain files are expected to be present.  Or, you could separate the common files (stat files, brainmask, etc) into separate catalogs and lump everything else into one big catalog.  The benefit to option 1 is that querying custom variables across sessions is already supported in XNAT.  Whereas, option 2 wouldn’t be query-able across sessions out of the box (though it could easily be configured via a custom SQL View in the display docs).

how efficient is the query of custom variables? is it really bad? now that we have tools to efficiently add custom variables, if the querying is not super slow, that might really be a good middle ground.

cheers,

satra