Freesurfer

[Tim Olsen]

We are doing another round of freesurfer work.  I’m curious what the latest and greatest out there is. 

 

Flavin, can you share a link to some of your recent Freesurfer stuff?  Has anyone else recently made use of freesurfer on their XNAT’s?

 

Tim

Radiologics, Inc

[Flavin, John]

Getting any of my pipelines installed on a generic XNAT would be pretty tricky. I’m working towards making them all as general as possible, but there will almost certainly be spots I haven’t considered. If all you want to do is see my Freesurfer 5.3 pipeline XML, it’s here: https://bitbucket.org/nrg_customizations/nrg_pipeline_freesurfer53/src/aa63f28abdaed4a5efe83b2c7154aedf4ce24e54/templates/misc/catalog/Freesurfer/Freesurfer_5.3.xml?at=default.

I think to get my Freesurfer 5.3 pipeline(s) running you’d have to use the following repos (on top of xnat_builder_whatever and pipeline_whatever repos):

https://bitbucket.org/nrg_customizations/nrg_webapp_pipelinecore

https://bitbucket.org/nrg_customizations/nrg_webapp_freesurfercommon

https://bitbucket.org/nrg_customizations/nrg_pipeline_freesurfercommon

https://bitbucket.org/nrg_customizations/nrg_webapp_freesurfer53

https://bitbucket.org/nrg_customizations/nrg_pipeline_freesurfer53

You’d need to customize BuildAction.java from cnda_xnat_dev (https://bitbucket.org/nrg/cnda_xnat_dev/src/cca99c5fe83a14d3bfbad5b42fb694fcbdd0401e/src/java/org/nrg/xnat/turbine/modules/actions/BuildAction.java?at=default). Remove all the methods that launch pipelines you don’t have. (I plan to dismantle this piece of the pipeline infrastructure at some point; it makes the process of installing and managing pipelines more difficult.)

A lot of those pipeline resources need setup scripts. Currently ours aren’t managed in any repo (we keep them locally on all our machines) but that’s something I will change very soon. But even if I had examples those would still need to be customized per site, so you would change the setup script to point to the location where, say, your Freesurfer installation lives.

It’s not a simple process. But, like I said, if all you want to do is read the Freesurfer 5.3 pipeline XML (and, of course, its associated resources) I gave the link above.

Flavin

-- 
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/d/optout.

---

The material in this message is private and may contain Protected Healthcare Information (PHI). If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

[Tim]

Here’s a list of the additional changes I had to make to get Freesurfer working.

On the XNAT side:

• templates/screens/PipelineScreen_Freesurfer_53.vm

  • modify cmdPrefix to remove the arc-grid-queue text

• /java/org/nrg/pipeline/launchers/Freesurfer53Launcher.java

  • remove reference the to cnda-ops at nrg.wustl.edu

• /java/org/nrg/pipeline/launchers/FreesurferRelauncher.java

  • remove reference the to cnda-ops at nrg.wustl.edu

• /java/org/nrg/xnat/turbine/modules/screens/DefaultPipelineScreen.java

  • modify the code after String pValues = pipelineParam.getCsvvalues();.  It throws a NullPointerExpception if you don’t select T2 and FLAIR scans.  Not sure why.  But, I just added a null check and ignored the parameter entirely if that value was null.

• Added the following files from various CNDA public repositories.  These could probably be moved to the freesurfer webapp repos.

  • /scripts/slideshow.js

  • templates/layouts/FreesurferQC.vm

  • templates/layouts/FreesurferQC_64.vm

  • templates/layouts/Fs_fsData_QC_64.vm

  • templates/layouts/PipelineScreen_FS5_64_upload_files.vm

  • templates/layouts/PipelineScreen_FS5_upload_files.vm

  • templates/macros/xnatMacros.vm

  • /java/org/nrg/xnat/turbine/modules/actions/DownloadFreesurferAction.java

  • /java/org/nrg/xnat/turbine/modules/screens/FreesurferQC.java

  • /java/org/nrg/xnat/turbine/modules/screens/FreesurferQC_64.java

On the pipeline side

• Commented out setup scripts (freesurfer, etc) and removed FS_VERSION stuff.  I only had 1 version of freesurfer and it was setup by default for the user account.  I might have to do it differently in a production system.

• I had to manually find and replace a few PIPELINE_HOME references in .sh files.

• I had to find and replace references to cnda-ops at nrg.wustl.edu.

• I had to do a chmod +x on many of the .*sh files and a .pl files.

To get everything running, I had to register both the Freesurfer pipeline and the Freesurfer_Snapshots pipeline.  The Freesurfer pipeline had a custom start page of PipelineScreen_Freesurfer_53.vm.  Freesurfer_Snapshots didn’t have one.

On the command prompt, getting the actual Freesurfer pipeline running was easy.  Getting the Freesurfer_Snapshots pipeline was more of a challenge.  It has a lot of dependencies (for generating images without a DISPLAY).  I think these are the commands I had to run:

• yum install -y tcsh libgomp bc perl-core

• yum install turbojpeg

• yum install VirtualGL

• vglserver_config -config +s +f +t  (hat tip to Matt House for recommending this)

• yum install ImageMagick

• yum install Xvfb

Once all that was in place, I was running Freesurfer J.

 

I’ve made some changes to how Freesurfer can be launched from the webapp.  A more streamlined option.   I’m going to clean those up a bit, then I’ll give them to Flavin to see if he wants to incorporate them.

Tim 

[Flavin, John]

Thanks for doing this, Tim. I’ll take all these changes and incorporate them into the repos.

There are a few things that I had known about and have fixed since you first pulled the code.

* I fixed the bug in DefaultPipelineScreen where it chokes on parameters with no defaults. It’s in xnat_builder_1_6dev changeset 50995be.

* In the various setup scripts for the pipeline tools, I had the wrong maven template name. “@PIPELINE_DIR_HOME@“ is wrong and I have replaced it with the correct template string “@PIPELINE_DIR_PATH@“. With that change your pipeline setup script would be able to automatically fill those in. (In your case it won’t matter if you just hard-code the path since you know what it is.)

* I had defined a couple velocity macros in cnda_xnat_dev that got used in the pipeline launch/add templates; I have since moved those to TurbineMacros in xnat_builder_1_6dev in changeset 83b821c.

Flavin

[Ben Wagner]

Hi John and Tim,

   Would this still be the most recent/recommended FreeSurfer pipeline setup routine?

Ben

[Flavin, John]

Ben,

The very short answer is “yes”. 

I had written a longer response, but I decided it would make more sense to turn that into a wiki document. I’ll let you know when I put it up.

Flavin

CNDA Pipelines Developer

fla...@mir.wustl.edu

@cndapipelines

[Kate]

On the FS topic, is there any plan for an XNAT viewer that could open FS images for interactive review (I know the pipeline has snapshots)?

Thanks,

Kate

[Daniel Marcus]

Kate, 

I would love that, particularly if you could do edits directly in the viewer.  I even have a post-it on my desk reminding me that we should implement that. But alas I don't think we have capacity to make it happen.  But if your group or someone else wants to take a stab at it, we'd be happy to provide support.

-Dan

[Ben Wagner]

Is there an open project somewhere where I can see where the newest FS pipeline outputs?  I didn't see anything on Central.  It looked like the older schema.

Ben

[Sanket Gupte]

Hi Flavin,

Is the said Wiki document up yet ?

Also, on the topic,  what's this : http://freesurfer.net/fswiki/QdecProject/Xnat  . Is this module/pipeline available ?

Thanks a bunch :)
Happy Friday
Sanket

[Flavin, John]

No, I still haven’t written any documentation of this. Here’s the outline:

# Installing the modules

You will need to download several repositories, all of which are available at https://bitbucket.org/nrg_customizations/.

* XNAT module repos

** nrg_webapp_freesurfercommon

** nrg_webapp_freesurfer53

* pipeline module repos

* nrg_pipeline_freesurfercommon

* nrg_pipeline_freesurfer53

The XNAT modules are made by zipping up the repos with their respective src/ directories as the root. Put the zips wherever your XNAT expects to see modules and run the update.sh. 

The pipeline modules are easier. With the updates to the pipeline engine as of a few months ago, you can just download the above repos into the engine’s modules directory and it will handle building everything. Or put them into some other directory somewhere else, and provide that directory to the engine when you build. There is a sample gradle.properties file in the pipeline engine that will show you how to do that.

# Providing your system-specific information

Within the pipeline module repos, there will be sample setup scripts (in nrg_pipeline_foo/scripts/scripts). You should make a copy of each of these called foo_setup.sh; inside each will be some path to a tool like Freesurfer or FSL or whatever that you will need to provide.

# Running the pipelines

## Add the pipelines to your XNAT. 

Go to Admin > Pipelines and hit “Add pipelines to site”. 

To add Freesurfer, you would enter

* Path to pipeline XML: </wherever/your/pipelines/get/installed>/catalog/Freesurfer/Freesurfer_5.3.xml

* Custom screen: PipelineScreen_Freesurfer_53.vm

You will also need to add the Freesurfer_Snapshots pipeline

* Path to pipeline XML: </wherever/your/pipelines/get/installed>/catalog/Freesurfer/Freesurfer_Snapshots.xml

* Custom screen: <leave this blank>

## Add both pipelines to a project. 

There aren’t any parameters for Snapshots. For Freesurfer itself, it needs to know what scan types to look for to identify a T1 scan, so you’ll have to enter that as a parameter. If you have the dev tip of XNAT, you’ll be able to enter a regular expression; if not, then you need to enter the literal strings that any T1 scan has for its scan type.

## Go to a session and launch the pipeline

Then you wait for a while.

I am 100% certain there will be problems in that process. Let me know / post to the list when things aren’t working. We can either work on getting you past some difficult point or perhaps fix some bug you’ve found.

Regarding http://freesurfer.net/fswiki/QdecProject/Xnat, I don’t know anything about that. That page says it was last updated in 2008. So… ¯\_(ツ)_/¯.

Flavin

CNDA Pipelines Developer

fla...@mir.wustl.edu

@cndapipelines

[Sanket Gupte]

Okay. Where can I get the AutoFsAparcregionanalysisRegion class .

These are the errors btw.

-------------
    [exec] /opt/xnat/xnat_builder/deployments/NIDAxnat/target/src/org/nrg/xdat/om/base/BaseFsAparcregionanalysisRegion.java:18: error: cannot find symbol
    [exec] public abstract class BaseFsAparcregionanalysisRegion extends AutoFsAparcregionanalysisRegion {
    [exec]                                                               ^
    [exec]   symbol: class AutoFsAparcregionanalysisRegion
    [exec] /opt/xnat/xnat_builder/deployments/NIDAxnat/target/src/org/nrg/xnat/turbine/modules/screens/PipelineScreen_Freesurfer_53.java:60: error: cannot find symbol
    [exec]             ArrayList<XnatImagescandata> mrScans = getScansFromScantypeConfig(SCANTYPE_CONFIG_TOOL_NAME, SCANTYPE_CONFIG_FILE_NAME);
    [exec]                                                    ^
    [exec]   symbol:   method getScansFromScantypeConfig(String,String)
    [exec]   location: class PipelineScreen_Freesurfer_53
    [exec] /opt/xnat/xnat_builder/deployments/NIDAxnat/target/src/org/nrg/xnat/turbine/modules/screens/PipelineScreen_Freesurfer_53.java:61: error: cannot find symbol
    [exec]             ArrayList<XnatImagescandata> t2Scans = getScansFromScantypeConfig(T2_CONFIG_TOOL_NAME, T2_CONFIG_FILE_NAME);
    [exec]                                                    ^
    [exec]   symbol:   method getScansFromScantypeConfig(String,String)
    [exec]   location: class PipelineScreen_Freesurfer_53
    [exec] /opt/xnat/xnat_builder/deployments/NIDAxnat/target/src/org/nrg/xnat/turbine/modules/screens/PipelineScreen_Freesurfer_53.java:62: error: cannot find symbol
    [exec]             ArrayList<XnatImagescandata> flairScans = getScansFromScantypeConfig(FLAIR_CONFIG_TOOL_NAME, FLAIR_CONFIG_FILE_NAME);
    [exec]                                                       ^
    [exec]   symbol:   method getScansFromScantypeConfig(String,String)
    [exec]   location: class PipelineScreen_Freesurfer_53
    [exec] Note: Some input files use or override a deprecated API.
    [exec] Note: Recompile with -Xlint:deprecation for details.
    [exec] Note: Some input files use unchecked or unsafe operations.
    [exec] Note: Recompile with -Xlint:unchecked for details.
    [exec] 4 errors

--------------

[Flavin, John]

I can solve these specific problems, and if you’re interested I have laid out below the steps to do that. But your difficulty using this module raises a broader point that I hadn’t considered. The modules I write are written against the dev tip of XNAT. If I need a particular new XNAT feature to make a module do something, I just make the change I need and check it into XNAT, and expect that the module will be built along with my XNAT changes. These errors you are seeing are a result of that: your XNAT version is not the latest development version. Nor should it be! I wouldn’t expect you to pull the latest builds all the time. But up to now I haven’t had a need to support older versions of XNAT with my module code, so I can guarantee that the module won’t build properly out of the box against an older XNAT. This, as I said before, is something I had not yet considered.

If you want to solve these particular problems, here are the steps to do so. But I know that there will be more errors after that. (For instance, the pipeline won’t send any email when it is done, because I moved the email infrastructure into XNAT and you don’t have those classes. It is possible that it won’t compile without those classes; not sure.) It may be that this is a futile exercise, maybe not. I guess that depends on how much work you want to put into the cycle of build, find problems, ask for solution, repeat. It is a finite process, and will end at some point. And I’ll keep helping as long as you want to keep trying.

So, as to the specific error message. There are two kinds of errors I can see:

1. The build can’t find the AutoFsFoo class. This is (I think) because of the change I made here: https://bitbucket.org/nrg/xnat_builder_1_6dev/commits/04294f5572571b7501f8c4d0983e012e82cf3968. In short, I removed an older, outdated freesurfer schema definition out of XNAT in favor of the up-to-date schema in the module. You will need to remove the directory <your_xnat_builder>/plugin-resources/project-skeletons/xnat/src/schemas/fs and all its contents.

2. The Freesurfer screen PipelineScreen_Freesurfer_53 can’t find the method getScansFromScantypeConfig(String,String). This is because I moved this method into DefaultPipelineScreen in this commit: https://bitbucket.org/nrg/xnat_builder_1_6dev/commits/5881c13ec9779bf388a4433880c350bf5179a166. You should copy both of the files from that commit into your XNAT.

The second problem will be solved with the proposed solution; I am certain of that. The first problem, well, I think the solution will work but I’m not 100% sure. It certainly won’t hurt anything.

Flavin

[Sanket Gupte]

Hey thanks
I will keep fighting till the end. 4 Xnat instances and 5 bosses will be really pleased if this starts working.

I believe I am using a fairly recent version of the dev repo.

There is no fs in plugin-resources/project-skeletons/xnat/src/schemas/fs  :(

I did update the two files from point 2 though.

Now I am down to just 1 error. The first one. That;s progress..
------------
   [exec] /opt/xnat/xnat_builder/

deployments/NIDAxnat/target/src/org/nrg/xdat/om/base/BaseFsAparcregionanalysisRegion.java:18: error: cannot find symbol
    [exec] public abstract class BaseFsAparcregionanalysisRegion extends AutoFsAparcregionanalysisRegion {
    [exec]                                                               ^
    [exec]   symbol: class AutoFsAparcregionanalysisRegion

-------------
Any other suggestions ?

[Flavin, John]

It looks like you need to clear out the old Base… class that is sitting around from a previous build, referencing an Auto… class that didn’t get generated.

When you built with the module, did you just run the update.sh? You may need to blow away your projects and deployments and start over with setup.sh.

[Sanket Gupte]

Ok. You were right. So, now everything compiles. (I think) But I don't see a thing to start a pipeline from the session.
I added a pipeline globally. Then added it to the project. But when i go to a session, I can't see anything to make it run.

[Flavin, John]

Sounds like you don’t have the “Build” action in the actions box. See this thread, and the wiki page linked therein. https://groups.google.com/d/topic/xnat_discussion/0bcqg6JX1YI/discussion

> On Aug 19, 2015, at 5:24 PM, Sanket Gupte <gupte...@umbc.edu> wrote:
>
> Ok. You were right. So, now everything compiles. (I think) But I don't see a thing to start a pipeline from the session.
> I added a pipeline globally. Then added it to the project. But when i go to a session, I can't see anything to make it run.

[Sanket Gupte]

Hi Flavin,

I think you would be able to answer this quickly, probably from the top of your head, rather than me trying to figure out what I have to change where.

My freesurfer pipeline runs, as expected. It just doesn't upload the final XML back into xnat.
The error I got is :

Message: XML Validation Exception.

• fs:fsData/imageAssessorData/derivedData/provenance/processStep[0]/machine : Required Field

And I know it means, of the 4 steps in the pipeline, the first step isn't getting the info about my xnat machine. While all the rest have the correct info in <prov:machine> tag.
How do I go ahead in fixing that .

Thanks a lot
Sanket

[Flavin, John]

I don’t know what is happening off the top of my head. I haven’t seen this before.

1. How are you generating your assessor XML? If you use the script that comes packaged with the pipeline (stats2xml_fs5.3.pl) it should be able to pull out everything it needs from the recon-all log.

2. You could just write a script to insert that one field for you if it doesn’t exist. I would probably use python, myself. Import lxml.etree, read in the assessor, check for the existence of the ‘machine' tag using the exact xpath that you gave, if it doesn’t exist insert it and save the assessor back out.

Flavin

[Sanket Gupte]

Got it to work.

So, of all the provenance steps, the first one is getting the info from recon-all.env file whereas the next steps are getting their provenance tags from the aseg.stats file.

My recon-all.env  didn't have "HOSTNAME" . Instead it had "HOST" .

So, simple fix. But not sure why the difference. May be versions.
Anyway, for others who stumble upon this problem.

In stats2xml_fs5.3.pl file
below the line : my $host_str = `grep HOSTNAME $recon_all_env | sort -u`;
Just write:
    if ($host_str==''){
        my $host_str = `grep HOST= $recon_all_env | sort -u`;
    }

--
You received this message because you are subscribed to a topic in the Google Groups "xnat_discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/xnat_discussion/SbIPPLLC6lg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to xnat_discussi...@googlegroups.com.

[Sanket Gupte]

EDIT:
if ($host_str==''){

        $host_str = `grep HOST= $recon_all_env | sort -u`;
}

[Flavin, John]

Thanks! I can make this change in the repo and push it up.  

You received this message because you are subscribed to the Google Groups "xnat_discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.

[Sanket Gupte]

Hi Flavin/Others,

I didn't want to bother anyone with this. But I tried all I could to get to the bottom of this, and I am not able to get any further. When you get a chance, if you could help, it would be much appreciated.

So, my Freesurfer pipeline runs perfectly. Except, the Snapshots pipeline fails at 85.* % .

Build log doesn't have much.

Following is it's tail.

WorkDirectory /data/build/123/20151003_121115/restlaunch/MyXnat_E01740_freesurfer_20150930025050/112233-1
Executing: /pipeline_1_6/build/pipeline/xnat-tools/XnatDataClient -u dc2fa8b9-e976-427e-8252-ae9313657b74 -p ######## -r "https://myxnat.nida.nih.gov/data/experiments/MyXnat_E01740/assessors/MyXnat_E01740_freesurfer_20150930025050/resources/SNAPSHOTS/files?overwrite=true&label=VOLUME_SNAPSHOTS&content=VOLUME_SNAPSHOTS&reference=/data/build/123/20151003_121115/restlaunch/MyXnat_E01740_freesurfer_20150930025050/temp&event_id=24698" -m PUT

And the pipeline log has this in the tail (among other things).
You tried to PUT to this service, but didn't provide any data (found request entity size of 0). Please check the format of your service request.

I am guessing it is failing in the last step of the snapshots pipeline. And In the log I can see it is able to convert things into the formats it needs.

Not sure how you could help, but any hints on approaching this, and how to figure out where exactly it's failing, would be much appreciated.

[Flavin, John]

I’m not immediately sure why that failed. I do have two things for you to try:

1. Check if there were actually any snapshots created. You can find the path as the value after “reference=” in the PUT query params. If there aren’t any files there, that would explain why the PUT failed, but then we’d have to track down that problem.

2. Maybe try adding an “&inbody=true” at the end of the PUT and just run that command manually on the command line. You will need to put in your username and password in place of the alias token. If that works, then to make the fix permanent you’d have to edit the pipeline; only there you would need to add “&amp;inbody=true” to properly XML-escape the ampersand.

Flavin

[Sanket Gupte]

Yeah. I see the snapshots directory is empty.

And something is failing somewhere. I am not able to figure out what and where.

Before I send you the error and/or log files, and waste your time in going through, let me ask you this .

What all programs are needed by the snapshots pipeline ? I have Imagemagick installed, and it's in the path. I have defined all the necessary environment-variables as well. And ofcourse I have freesurfer and fsl installed as well.

Am I missing any programs?

[Flavin, John]

The only things you need are Freesurfer, X11 (to run xvfb, so we can run tkmedit’s GUI in memory), and ImageMagick (to convert the .rgb output to .gif). 

Though, now that I think about it, you may not have the Freesurfer QA tools. Do you have anything at $FREESURFER_HOME/QATools? If you don’t have that, you can get it from http://freesurfer.net/fswiki/QATools. We use some tcl scripts that come with that package to automate loading the various slices and taking the snapshots of them.

If you do already have the QA tools, then go ahead and send me your logs. I may be able to spot the problem quickly.

Flavin

[Sanket Gupte]

All right. Since you insist so much. I'll send you my logs :D

(The QA Tools was already installed.)

Thanks a bunch. Really appreciate all your help

[Flavin, John]

Yep, that’s a bug.

The snapshot script expects the ImageMagick tool convert to be on the PATH, but I wasn’t sourceing the ImageMagick_setup script before running the snapshot script. I’ve pushed a change to nrg_pipeline_freesurfercommon that remedies that problem. 

Make sure you have an ImageMagick_setup.sh in your <pipline_home>/scripts directory. There is a sample setup script in freesurfercommon/scripts/scripts that you can just rename and fill in the proper path to your ImageMagick. Then when you run the pipeline build, it will be copied to the right place.

Flavin

<MyXnat_E01740_freesurfer_20150930025050.err><MyXnat_E01740_freesurfer_20150930025050.log>

[Sanket Gupte]

'convert' was already in my PATH.

Anything else that would stand out from the logs ?

[Sanket Gupte]

xnat@myxnat:~$ which convert
/usr/bin/convert
xnat@myxnat:~$
xnat@myxnat:~$ echo $PATH
/home/xnat/bin:/usr/local/freesurfer/bin:/usr/local/freesurfer/fsfast/bin:/usr/local/freesurfer/tktools:/usr/share/fsl/4.1/bin:/usr/local/freesurfer/mni/bin:/usr/lib/jvm/java-7-openjdk-amd64/bin:/usr/lib/postgresql/9.1/bin:/opt/xnat/xnat_builder/xnat-165/bin:/opt/xnat/pipeline_installer/build/pipeline/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:$JAVA_HOME:$JRE_HOME:/usr/lib/fsl/4.1:/usr/lib/fsl/4.1

[Flavin, John]

The reason I suggested that convert is not on the path when the pipeline runs is this:

1. I can see from the .log that all the .rgb files were created correctly. There are a sequence of lines like “Converting foo surface: xx% done” which proceed up to 100%. That looks like the snapshot script can run, tkmedit can run, and the tcl scripts can run. All signs point to the actual snapshotting process running properly.

2. I can see from the .err log a boatload of lines like 

/usr/local/freesurfer/QAtools/data_checker:/usr/local/freesurfer/QAtools:/usr/local/freesurfer/bin:/usr/local/freesurfer/fsfast/bin:/usr/local/freesurfer/tktools:/usr/local/freesurfer/mni/bin:/home/xnat/bin:/usr/local/freesurfer/bin:/usr/local/freesurfer/fsfast/bin:/usr/local/freesurfer/tktools:/usr/local/freesurfer/mni/bin:/usr/lib/jvm/java-7-openjdk-amd64/bin:/usr/lib/postgresql/9.1/bin:/xnat_builder/xnat-165/bin:/pipeline_installer/build/pipeline/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:$JAVA_HOME:$JRE_HOME/bin/convert: Command not found.

This looks to me like convert cannot run, because it isn’t on the path. 

Why the script can’t find it but your shell can is a mystery to me. Did you recently add that convert to /usr/bin? Is anything tricky going on in your .bashrc or .bash_profile?

Flavin

[Sanket Gupte]

I hate to elongate this freesurfer chain this long. :( 

I have good news, and I have not so good news.

Good News: You were right, there was problems in finding convert. It was looking for convert in /usr/bin/bin , instead of /usr/bin  . Nothing to change in your freesurfercommon repo. Hard-coding on my end.

Not So Good News: It runs everything. 100% . But the snapshots generated are kind of .. blank. See attached. :(  From your experience, you know any reason why that may be ?

Again. Thanks a lot. Your help is much appreciated by me as well as, at least 85 muggles (read: neuroscientists), who will be very excited to see the Freesurfer results so easily visible AND searchable on a 'website' , and that too 'automatic'  :)   [ pfff   these muggles ]

[Flavin, John]

I don’t know why the snapshots are coming up blank. I can make a few guesses though, and give you some avenues to explore.

Here’s a note from the QATools site (http://freesurfer.net/fswiki/QATools):

When taking snapshots, the tkmedit and tksurfer windows that appear can't be blocked by other windows, and the screensaver can't be activated while these scripts are running. Otherwise, the script will take snapshots of the screensaver or the interfering windows. The monitor can be physically turned off while the scripts are running, however, to save power.

I would think that using xvfb (as the pipeline does) would alleviate this problem, but maybe it doesn’t. Could there be a screensaver running on whatever machine you’re using for processing the snapshots?

If you can try doing the process manually, that could give some extra information. Log into the processing machine, download the DATA resource for some freesurfer, and run the snapshot script. Maybe try both with and without the xvfb_wrapper to isolate that variable.

Sorry I can’t be of more help here.

Flavin

<ScreenShot.JPG>

[Sanket Gupte]

Manually... It definitely doesn't run without the xvfb-wrapper script. Runs into the X11 GLUT errors. It 'runs' with the wrapper but it went into a blackhole. or something. Coz it kept running for 4 hours, and then I killed it.

But I dug really deep, and haven't been able to figure out the issue.

But from my little experience, (which is very very 'little' ) , I think there should be easier ways to get snapshots out of mgz files. And I think it might be much faster than the tkmedit and tksurfer scripts. I will ponder and probe, and if successful, will share the code. :)

Thanks for all the help.

[Flavin, John]

I went down this same path myself a few months ago. I could send you some scripts. 

Reading the volume images is easy; I used nibabel in python but I’m sure there are a ton of other libraries that can read mgz. The problem I found, and the reason I stopped pursuing the idea, was with the surfaces. (I’m just going off my vague memories, so this may not be 100% true.) The surfaces are represented as values on a network of nodes, whereas the volume files are represented as values on a regular grid. Freesurfer does provide a tool to convert a surface file into a volume, so my script would do that conversion and layer the “surface volume” onto whatever other volume I wanted. It looked awful. The resolution of the surface post-conversion was the same as the voxel size of whatever volume I layered it onto, whereas when tkmedit did the layering itself, the line representing the surface is much thinner than the voxels of the volume.

So, yeah, if you want to pursue the idea I could give you what I have. And if you get it to work, I would love to use it. A natively scripted solution would have to be better than the GUI scripting we have to use now. But I couldn’t see it panning out for me.

Flavin

[Sanket Gupte]

Yes please. If you have some scripts, I could definitely try to use them. If not entirely, it would be a good foundation.

[Flavin, John]

Turns out I don’t have as much as I thought I did.

I don’t have any of the commands or scripts I used to do the surface to volume conversion. I am pretty sure I used mri_surf2vol (http://freesurfer.net/fswiki/mri_surf2vol).

I also don’t have the script I used to load the images, layer them, and save the result.

What I do have is:

1. A sample jpg snapshot to show how poor my results were, and

2. A small script of notes I kept on how to load the volumes, slice them for snapshotting, and display them. (But that’s entirely just knowing how to use numpy to manipulate the data and matplotlib to show them.) You’d probably want to run that script line-by-line in an interpreter, otherwise it will display an image and them immediately overwrite it with another.

Good luck!

[Spiro Pantazatos]

Hi Flavin,

I would like to try and get the Freesurfer 5.3 pipeline running on my XNAT 1.7 instance. Do you recommend I start with your documentation outline in this thread (dated 8/14/15), or given that my instance is XNAT 1.7 and your instructions were for 1.6, am I better off writing my own xml pipeline (i.e. adapting the dcm2nii.xml pipeline). 

Thanks for any advice, 

Spiro

[John Flavin]

We haven't modified the pipeline engine at all with 1.7, so the Freesurfer 5.3 pipeline itself, (in bitbucket.org/nrg_customizations/nrg_pipeline_freesurfer53 and other things in bitbucket.org/nrg_customizations/nrg_pipeline_freesurfercommon that are necessary) should still work with XNAT 1.7. I don't think you need to worry at all about the pipeline.

Unfortunately, the corresponding webapp modules (in nrg_webapp_freesurfer53 and nrg_webapp_freesurfercommon) will not yet work with XNAT 1.7. The code that is in those repos should still work fine, but XNAT 1.7 wouldn't be able to read them. The source files would need to be re-arranged first so they can be built into a proper 1.7-style plugin. If those repos just contained the pipeline launch pages, I would say you could probably live without those. But the Freesurfer datatype schema is in there as well; that's necessary to have if you’re trying to store Freesurfer data into XNAT.

Until I (or some other brave soul) can convert the code in the nrg_webapp_freesurfer* repos over to a 1.7-style plugin, then the Freesurfer pipeline(s) won't work in 1.7. There just isn't a way I know of to get the custom datatype without doing that. 

Give me a little time. I'll see what I can do.

Flavin

Visit this group at https://groups.google.com/group/xnat_discussion.