Running C-PAC pipeline on ABIDE dataset

[Federico]

Hi cpac team,

I'm trying to reproduce ABIDE-preprocessed dataset, for the purpose of running the same C-PAC pipeline on ABIDE I and ABIDE II datasets.

I set up a yaml configuration pipeline file following the steps showed on ABIDE-preprocessed website,
and for a more detailed parameters setting I compared my pipeline with the yaml configuration file on Abide-preprocessed github page.

However there are still some differences between my outputs and Abide's time-series namely:

using for example CC200 atlas I get a very similar temporal evolution (with some differences, expecially in height of peaks), and the Pearson correlation coefficient, calculated between the ROI's time series of my output and Abide-preproc's, is nearly one.

But, when I calculate Pearson correlation between two different ROIs (let's say ROI A and ROI B) on my dataset, and the same (ROI A and ROI B) on abide's timeseries I get consistently different results.

Am I doing something wrong? What could I do to improve my outputs similarity with Abide's?


I upload as attachment my pipeline configuration file.

Running CPAC 1.8.1

on singularity 3.9.1

Ubuntu 20.04 focal fossa

Thanks for your support
Federico

[sgia...@gmail.com]

Hi Federico,

To double-check, you designed the pipeline based off of this correct?

http://preprocessed-connectomes-project.org/abide/cpac.html

Steve

[Jon Clucas, MIS]

Hi Federico,

Just regarding the freezing, that freezing behavior occurs when C-PAC underestimates the resources it will need and ends up deadlocking, waiting for a dead process to return.
Try C-PAC 1.8.2, which includes "improved memory management for multi-core node allocation," unless you have a specific reason to use v1.8.1.

As far as opening another issue on this forum, it's fine either way, as long as you're able to keep track of whether we've addressed your issues or not.

If you're able to share log files from failed attempts, that can help us debug, but I do suspect (and hope) this freezing behavior is one that is resolved by the memory management upgrade in v1.8.2.

Thanks,

Jon

On Sunday, January 16, 2022 at 5:09:18 AM UTC-5 Federico wrote:

Also, my cpac run gets stuck during preprocessing after it has preprocessed 3-4 subjects (or 3-4 groups of subjects if I set "number of participants to be run in parallel" > 1). It doesn't return me any error, it just freezes. Should I check some pipeline's parameters or computer settings?..But I probably have to open an other issue on this forum, right?

[FEDERICO CAMPO]

Hi Jon,

Thanks for your suggestion, I tried to run CPAC 1.8.1 on Docker in place of Singularity and it worked! I started the run of the whole ABIDE I dataset reserving 15 Gb RAM for the process.

It's still running without any errors so far.

Thank you again for your support!

Federico

[Jon Clucas, MIS]

Hi Federico,

I'm sorry I wasn't clearer ― I didn't intend to suggest to use Docker instead of Singularity. Based on your experience, I suspect your system config is more forgiving of Docker using more resources than expected than it is of Singularity doing the same thing.

In any case, I'm glad it's working for you now!