From a CSV connectivity matrix to a dconn.nii

[Ruben Sanchez]

Hi,

I have a connectivity matrix for cortical vertices in csv format. How can I transform it into a dconn.nii file to visualize it in wb_view.

Thanks,

Ruben Sanchez-Romero

Cole Lab

Rutgers-Newark

[Glasser, Matthew]

What are the dimensions of your file?

Matt.

--
You received this message because you are subscribed to the Google Groups "HCP-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hcp-users+...@humanconnectome.org.
To view this discussion on the web visit https://groups.google.com/a/humanconnectome.org/d/msgid/hcp-users/d1e26bcb-0fd3-4da7-b108-f338c5923a29n%40humanconnectome.org.

 

---

The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

[Ruben Sanchez]

Hi Matt,

The matrix is 59412 x 59412, all cortical vertices for both hemispheres. I do not have connectivity for the subcortical structures.

Ruben

ps. Michael Cole and Ravi Mill say hi.

You received this message because you are subscribed to a topic in the Google Groups "HCP-Users" group.
To unsubscribe from this topic, visit https://groups.google.com/a/humanconnectome.org/d/topic/hcp-users/nbw44D6cT5E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to hcp-users+...@humanconnectome.org.
To view this discussion on the web visit https://groups.google.com/a/humanconnectome.org/d/msgid/hcp-users/36D37FDF-A0A7-43FA-9670-C85F3AD22378%40wustl.edu.

[Tim Coalson]

That is quite a csv file, and would probably be substantially smaller (and less of a CPU bottleneck) if you can get the software to write it as cifti directly.

In order to convert the data to cifti, you first need to make a cifti file that has the correct indices in the correct order.  Assuming your csv follows the conventions of the 91282 grayordinates, such that left cortex comes first, and it uses the same medial wall ROIs, you can do wb_command -cifti-create-dense-scalar with those ROIs to create a matched surface-only dscalar file.  You can then use -cifti-merge to concatenate it with itself, to give you a 2-map file, which is the bare minimum to be able to run -cifti-correlation, which is the simplest command-line way to get a dconn file.  You can then use -cifti-convert with the -from-text option to convert the csv.

Alternatively, if you happen to be writing this file from matlab or octave, you can use cifti-matlab and the cifti_write_from_template function to directly generate a (square) dconn file, even if the only cifti file you have loaded is dscalar or dtseries (it just needs to have the same 59412 indices as your output):

https://github.com/Washington-University/cifti-matlab/blob/master/cifti_write_from_template.m

We also have a cifti library for c++, and for python, nibabel supports cifti files.

Tim

To view this discussion on the web visit https://groups.google.com/a/humanconnectome.org/d/msgid/hcp-users/CA%2BK2rtckBXxPRYYmCd-BDLTrEPAkerrzAosBmhBGgWFSi4TTPQ%40mail.gmail.com.

[Ruben Sanchez]

Thanks Tim,

I will try this approach. It looks similar to what I tried:

(1) I used a 91282 grayordinates timeseries dataset and run cifti-correlation, only to get a template *.dconn.nii file (This gave me a ~30G file).

(2) Then I used -cifti-convert -from-text, with my only-cortex FC matrix CSV file (which is also 30G, because I padded it with zeros for the subcortical voxels, to have a 91282 x 91282 size matrix), and use the dconn.nii file from (1) as template. (This gave me again a ~30G dconn.nii file)

In my 16G Mac, step (2) took around 3 hours! That is why I thought it may not be a very efficient approach.

Also, what you suggest would create a dconn file only for the 59412 cortical vertices, correct? That should save some space and speed up the conversion, correct?

Thanks again,

Ruben

To view this discussion on the web visit https://groups.google.com/a/humanconnectome.org/d/msgid/hcp-users/CANm3QU6BEQHeCCor1yemx4-QWgD%3D9qr8DMpOiLZ3arPN7t6bcw%40mail.gmail.com.

[Tim Coalson]

Yes, what I said would create a cortex-only dconn, assuming you don't supply volume data to -cifti-create-dense-scalar.  I would expect the csv to be larger than the cifti file, as arbitrary floating point values should usually take more than 4 characters on average to encode (especially when you include the comma separator).  Parsing text characters into floating point values isn't a particularly fast process, so I would expect that to be the part that is slow, but if you avoid padding on both dimensions, you get the benefit twice, so it may take slightly less than half the time (and disk space).  Avoiding csv (or other text-based formats) entirely would probably speed it up by a factor of 10 or more.

Tim

To view this discussion on the web visit https://groups.google.com/a/humanconnectome.org/d/msgid/hcp-users/CA%2BK2rtfcbZtUN12XvH2Q4UGScF7%3DMtGw9cSrfT8n62GHi7Czug%40mail.gmail.com.

[Ruben Sanchez]

I see. Will give it a try.

We are using CSV because we are computing FC matrices in Python, and then transforming those arrays into CSVs to send them to cifit-convert. 

Could there be a way around it?

Ruben

To view this discussion on the web visit https://groups.google.com/a/humanconnectome.org/d/msgid/hcp-users/CANm3QU7-q3R%2BBAkGBHuuKc1jrD7AVJuCLXQVj6w%3D_WknwopGcQ%40mail.gmail.com.

[Tim Coalson]

You should be able to use nibabel to read and write cifti files directly in python.

Tim

To view this discussion on the web visit https://groups.google.com/a/humanconnectome.org/d/msgid/hcp-users/CA%2BK2rtewuCB2LTzbDO1jj3crgiG0PiGVPa3dq2GjVKGhSgZ-Hw%40mail.gmail.com.

[Tim Coalson]

I am also assuming you are doing something more complicated than correlation or covariance, or else you would be using the -cifti-correlation command (or using the "dynconn" feature in wb_view to display correlation maps without computing a dconn in advance).

Tim

[Ruben Sanchez]

python array --> cifti dconn? 

will check it out, I guess writing a cifti scalar..

Thanks for the help, first time working with dconn files

To view this discussion on the web visit https://groups.google.com/a/humanconnectome.org/d/msgid/hcp-users/CANm3QU6Q5vZ_7Xcnx8i6J41Kw0BhPL%3DGFGV0ODcXoS3h%2BW5oOg%40mail.gmail.com.

[Tim Coalson]

You'll need to read some input as a cifti file with nibabel to get the information about how indices map to vertices, and then figure out how to copy that info to both dimensions of a new nibabel cifti file structure, but then you should be able to just set the data array and write the nibabel cifti structure as a dconn file directly.  But, that means any intermediate steps will need to keep the cifti header info around, so writing data to disk as headerless numpy arrays may not be the best long-term plan for working with cifti data.

Tim

To view this discussion on the web visit https://groups.google.com/a/humanconnectome.org/d/msgid/hcp-users/CA%2BK2rteh51CXUWsCKN6AV4K%3D0%2BCpsQxKiC0FbswaF9A4T7zZxA%40mail.gmail.com.