C-PAC v.1.6.2 SameFile Error and time point missmatch error

[Annika Mayer]

Hello,

I am using C-PAC 1.6.2 and unfortunately I still have a few problems where I don't know what to do. First of all the program spits out several SameFile Errors, it seems that this refers to the same named files in the working and output directory, but these are created by C-PAC itself and therefore I cannot influence them. Below is an excerpt of the error message:

Traceback (most recent call last):

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/utils/filemanip.py", line 472, in copyfile

    shutil.copyfile(originalfile, newfile)

  File "/usr/local/miniconda/lib/python3.7/shutil.py", line 104, in copyfile

    raise SameFileError("{!r} and {!r} are the same file".format(src, dst))

shutil.SameFileError: '/output/working/resting_preproc_sub-C002JOBA_ses-1/anat_preproc_afni_0/anat_reorient/sub-C002JOBA_T1w_resample.nii.gz' and '/output/output/pipeline_Pipeline_new_freq-filter_nuisance/sub-C002JOBA_ses-1/anatomical_reorient/sub-C002JOBA_T1w_resample.nii.gz' are the same file

My second problem and I think that's the reason why the processing of the data breaks down: Files created by C-PAC contain only 299 time points, my input was a file with 300 volumes (conversion with dcm2nii):

Standard error:

++ 3dTcorr1D: AFNI version=AFNI_20.1.01 (Apr 14 2020) [64-bit]

 + reading dataset file /output/working/resting_preproc_sub-C002JOBA_ses-1/sca_roi_1/_scan_rest/_selector_CSF-2mmE-M_aC-CSF+WM-2mm-DPC5_M-SDB_P-2_BP-B0.01-T0.1/_mask_striatum_mask_file_..Data_preproc..Masks..striatum.nii.gz/3dTCorr1D/bandpassed_demeaned_filtered.nii.gz

 + reading 1D file /output/working/resting_preproc_sub-C002JOBA_ses-1/roi_timeseries_for_sca_1/_scan_rest/_selector_CSF-2mmE-M_aC-CSF+WM-2mm-DPC5_M-SDB_P-2_BP-B0.01-T0.1/_mask_striatum_mask_file_..Data_preproc..Masks..striatum.nii.gz/clean_roi_csv/roi_stats.csv

[7m** FATAL ERROR:[0m 1D file /output/working/resting_preproc_sub-C002JOBA_ses-1/roi_timeseries_for_sca_1/_scan_rest/_selector_CSF-2mmE-M_aC-CSF+WM-2mm-DPC5_M-SDB_P-2_BP-B0.01-T0.1/_mask_striatum_mask_file_..Data_preproc..Masks..striatum.nii.gz/clean_roi_csv/roi_stats.csv has 299 time points, but dataset has 300 values

I have attached my log file and the pipeline I used. I would be very happy if someone has ideas and could help me with my problems.

[Jon Clucas, MIS]

Hi Annika,

Thank you for reaching out! 

We expect to release C-PAC v1.7.0 this week (🤞). Both of your issues ('SameFileError' - crash file, 🐛 3dTCorr1D: "roi_stats.csv has 1599 time points, but dataset has 1600 values") should be resolved (🚑 Fix SameFileError, 🚑 Fix CSV has x-1 time points, but dataset has x values) in the new version. We'll respond on here when v1.7.0 is available.

If you don't want to wait for the official release, tonight's nightly build will contain both fixes. You can get that development build with 

docker pull fcpindi/c-pac:nightly

or

singularity pull docker://fcpindi/c-pac:nightly

as soon as tomorrow morning.

[Annika Mayer]

Hi Jon, 

I'm glad to hear that and look forward to getting your message when the new version is ready. Thanks for the quick help and the work on this error. 

[Annika Mayer]

I tried the nightly version today. It seems that the time point error is fixed :), but the SameFile errors are still appearing. I have attached the log file.

Am Dienstag, 21. Juli 2020 23:44:31 UTC+2 schrieb Jon Clucas, MIS:

[Jon Clucas, MIS]

Hi Annika,

Thanks for keeping us updated! The update is mostly encouraging. I'm glad the time point error is resolved.

The persistent SameFileError bug is annoying but doesn't cause any other problems as far as we know.

We're beginning the process of restructuring C-PAC's outputs to comply with the recently published BIDS Derivatives standard, so the possibility of the SameFileError occurring should disappear in a future release.

Please let us know if you encounter any more issues.

Thanks again!

[Annika Mayer]

Hi Jon, 

Thanks a lot for your answer. I actually still have a problem. Since using the nightly version, my processing interrupts with an operating system error (OSError), which is no longer in the log file, but I see it in the terminal (both in the appendix). I had assumed that the OSError might be caused by the SameFile errors, apparently this is not the case. Do you have any idea what the problem might be? 

Kind regards.

[Jon Clucas, MIS]

Hi Annika, 

I'm sorry C-PAC is still giving you trouble.

To help us figure out what this error is about, could you please share as much of the following information as you're able to / comfortable with?

• Are you still using the pipeline configuration file you shared at the start of this thread?
• What architecture is running to produce this error? Specifically:
• Docker or Singularity (and version)?
• on a single machine or a cluster?
• if you're connecting to external drives or servers, using what protocol(s)?
• what operating system(s)? (those screenshots look like macOS, but if you're running on a server or cluster, what OS there as well?)
• what command (including bindings) are you using to launch C-PAC?

I realize some of these answers may contain information that you can't share. Where that's the case, we'll just try to figure it out without that information.

In the meantime, I subscribed to what looks like a related issue on the Nypipe GitHub repository. I mention that issue mostly for future reference, although if this comment

For future reference, we had a similar issue on one of our servers recently (the same "Cannot obtain lock" error), which turned out to be a relatively straightforward fix. server1 was trying to run a pipeline and save outputs to an NFS share hosted by server2, which we had recently upgraded. It turns out that the /etc/fstab on server1 specified the NFS version in the options columns, and was mounting NFS 4 shares as NFS 3 shares. We were able to fix this by replacing nfsvers=3 with nfsvers=4 in the options column for the affected volumes.

seems relevant to your use-case, the issue might also be useful now.

Thanks,

Jon

[Annika Mayer]

Hi Jon,

No problem. I'm glad you're helping. 

I'm now using the pipeline generated by the nightly version, but it differs only slightly from the previous one (new pipeline version in the appendix). I use docker to start C-PAC with the command:

docker run -i --rm -v /tmp/${USER}:/tmp -v /data_tina/Annika/Data/High_resolution_rs-fMRI_Striatum/Data_test/Analysis:/Data_preproc -v /data_tina/Annika/Data/High_resolution_rs-fMRI_Striatum/Data_test/output:/output fcpindi/c-pac:nightly /Data_preproc /output participant --n_cpus 20 --mem_gb 200 --pipeline_file /output/Pipeline_new.yml --participant_label C002 --save_working_dir

We pulled the docker image locally to a server, which I acess with secure shell (ssh). The operating system we use on the computer is Ubuntu 18.04.4 LTS, the one of the server is Ubuntu 16.04.6 LTS. I don't think it matters, but the computers do not have internet access. 

Kind regards,

Annika

[Jon Clucas, MIS]

Hi Annika,

We recently released C-PAC v1.8.0. If you're still trying to use C-PAC and experiencing the OSError mentioned above, you could try the new version and see if the issue persists. The update includes several updates to dependencies and improvements in resource management. I was never able to replicate the OSError (sorry for the lack of response all this time), but I'm hopeful that the upgrades will have resolved the issue nonetheless.

Please let us know if you try C-PAC 1.8 and if your issue is resolved or still occurs.

Thanks,

Jon