Downloading Task fMRI 3T Recommended

[Funda Yılmaz]

Dear all, 

We are currently working with the Working Memory task from the HCP fMRI dataset. Since the dataset is very large (~9TB), we are trying to download it directly through a Jupyter Notebook to Google Cloud.

Is there a way to download the data directly without first downloading it to our local machines? Do you provide any solutions for doing this kind of direct transfer to Google Drive?

Best,

[Tim Coalson]

If you plan to run your analyses on cloud computing, "google drive" may not be the correct target to put the data into, as you will probably want it in a compute-oriented filesystem for performance reasons (which would likely require copying it a second time if you copied it to "google drive" first).

I don't have experience transferring our data to "google drive".  If you can run an interactive instance with a desktop on google's compute infrastructure, and run Aspera inside that, it would avoid needing to use your local machine's internet bandwidth and storage.  I don't know much about the tools for mounting "google drive" as a network drive, to avoid needing to allocate a large amount of storage on the interactive compute instance.  I assume google cloud computing has something equivalent to AWS S3 ("Cloud Storage", probably?), which is where I would expect to be the right place to store the data for cloud processing, and is probably better integrated into the cloud infrastructure than "google drive".

Unfortunately, we currently do not have non-interactive ways to download the new release of the HCP-YA data.  There is an older version of the data (which doesn't have the newer fMRI cleanup, etc) on AWS S3, but we don't currently have an interface for generating the needed API keys, since we shut down ConnectomeDB.  We plan to implement these in the near future.

Tim

--
You received this message because you are subscribed to the Google Groups "HCP-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hcp-users+...@humanconnectome.org.
To view this discussion visit https://groups.google.com/a/humanconnectome.org/d/msgid/hcp-users/9a6f3c40-f0b0-4a4f-994a-c42662152d8cn%40humanconnectome.org.