Hi all,
I'm working on a project that will be utilizing the CHILDES data set as well as the Talkbank ASD set. We're doing our computation primarily on a linux slurm cluster that only offers CLI access.
In order to support reproducibility/transparent data pipelines, I was hoping to download the necessary files using a command line tool such as CURL or WGET, but I'm having difficulty. When I log in in the browser I'm able to use the download links successfully; however with the javascript based authentication, I'm not able to use CURL to download the files. Given the amount of data we need to download and organize, using a browser to download everything and then transfer the data to the cluster is not scalable.
Is there documentation on how to authenticate with CURL or WGET, or is there another supported means for programmatic data download.
Thanks,
Ben