Hi,
My name is Ratan Sebastian. I'm a PhD student at Leibniz University Hannover. I'm trying to download the 2019, 2020 and 2021 datasets for a research project that I have going on.
I'm downloading with aria2 using the maximum of 16 connections and downloads in parallel. I've managed to download a fair bit of the dataset but now the downloader fails with connection aborted error messages. Aria can continue downloads from where it last left off but I think it makes a bunch of header requests for all the files that it has downloaded to verify the size. It seems like it fails at this point. For the 2019 set for instance, it has already downloaded 432 files and as it goes through requesting the http URL and being redirected to the https url it starts erroring out after a while before it actually gets to files that it needs to download. I was wondering if there are some rate limits that might be causing this? Are there any connection limits that I need to be aware of?
Thanks,
- Ratan