Idrive Ftp
Libby Cowen
I am an absolut newbie when it comes to rclone, so that's only the most basic version of it. It was of help for me though, so I put the most important things together, it might be of help to someone else, especially as iDrive e2 support is pretty much non-existent. I am using iDrive e2 as an offsite backup location for WordPress sites. Those serve as additional backups, to have independent backups from the hosting provider.
An iDrive e2 Storage account can have different buckets on different locations. Because of a problem I could not solve, I was forced to migrate over from iDrive e2 Frankfurt to iDrive e2 Paris. In my particular case that meant, moving over all existing backups (folders, subfolders, files) as well.
The web interface from iDrive e2 does not support copying/moving folders and files from one bucket to another bucket. You can only copy/move individual files within a bucket, therefore it's pretty much useless for what I needed.
From my experience this worked, however it was unreliable, that means some files could not get copied, I needed to skip them and manually compare later on. I did only a very rough calculation on speed, I would not say it was actually slower than rclone, but I probably only moved 20GBs. However, I am not sure if this is a server to server job, or if that's download/upload in the end
ad 3:
The official iDrive e2 documentation only shows how to copy the entire content of the full bucket. The following command shows how it works for a subfolder including folder names that come with a space:
It worked perfectly fine for my use case which is uploading daily backups at night. However, my hosting provider decided to give up on the hosting location in Germany, and focus on UK only. So my entire WHM account including all cPanel accounts got migrated over to another server/infrastructure in the UK. From then on I had daily problems.
After tons of investigation and testing, it was 100% clear the only problem that is left is somewhere between iDrive e2 Frankfurt and the new server in the UK (based on OVH infrastructure). The old server in Germany was based on Hetzner infrastructure.
It might be the fault of the UK server. It might be the fault of iDrive e2 Frankfurt. Both parties say everything works, but it didn't. Whitelisting didn't change anything. In the end after 50 hours testing, I was ready to let iDrive e2 Frankfurt go. Still waiting for the iDrive support by the way (worst ever unfortunately)
Thanks Markus!
it is good to know
as if I would go later on with iDrive e2, my server is also at Hetzner, and I would have added the bucket in Germany
but I guess I red here somewhere that someone else had also problems with the iDrive location in Germany, so not 100% sure what would make sense
speed is not a question, but if transactions fail that is bad
Fact is, for me it worked very reliable until that change of my hosting provider. I am not able to tell on which end the actual problem occurs. Either iDrive e2 has a problem with the new server/location/network. Or the other way round. Everyone tells me "all is good" which obviously is not.
Moving Backups Between iDrive e2 Locations: A Beginner's Guide with rclone
This guide is for complete beginners who want to migrate their WordPress backups stored on iDrive e2 from one location (e.g., Frankfurt) to another (e.g., Paris). Since the iDrive e2 web interface doesn't support inter-bucket transfers, we'll be using a powerful tool called rclone.
Download & Upload (Unreliable): Easy but time-consuming and prone to errors, especially for large backups.
Third-Party Client (Uncertain): Tools like Forklift might work, but reliability can be a concern. It's also unclear if the transfer happens server-to-server.
rclone (Recommended): This is the most efficient and reliable option. It allows server-to-server transfers and detailed progress monitoring.
Let's Use rclone!
Install rclone: Refer to the official rclone documentation for installation instructions.
Set Up Remote Locations: Here's how to configure rclone for your iDrive e2 buckets:
Run rclone config in your terminal.
Choose "n) New remote" and enter a name (e.g., "idrive-frankfurt").
Select "Amazon S3 Compliant Storage Providers."
Choose "/IDrive e2" as the S3 provider.
Enter your iDrive e2 access credentials (access_key_id and secret_access_key).
Repeat this process to set up a separate remote for your Paris location (e.g., "idrive-paris").
Moving Your Backups:
rclone copy: This initiates the copy operation.
--verbose: Provides detailed progress information during the transfer.
idrive-frankfurt: Source location name (replace with yours).
bucketname: The name of your bucket on the Frankfurt location.
'folder with space': The folder you want to copy (including spaces).
subfolder: The specific subfolder within the chosen folder (optional).
idrive-paris: Destination location name (replace with yours).
Important Notes:
Replace bucketname, folder with space, and subfolder names with your actual details.
Test the command with a smaller folder first to ensure everything works as expected.
This guide assumes you have the necessary permissions to access both iDrive e2 buckets.
Conclusion:
This all happened before my ventures with iDrive e2... which gave me the same results. Frankfurt was going way too slowly way too often, so I moved everything to Paris. It went well for a while, but Paris too gave me one incident of slowness, which for me is one too many.
Click on the odrive icon and select Manage Storage (this will open a webpage)
Click on the + Plus sign at the bottom
Select WebDAV
Enter The URL, Username and password as prompted and link the account
I know this is an old thread, but even 3 years later, I would greatly appreciate direct support of iDrive from within odrive. WebDAV was never a real alternative, as they dis-allow WebDAV to transfer multiple files at once. They (now) explicitly state that WebDAV support is meant solely to open/save individual files, such as an Office document after opening and editing it.
The problem with idrive is that their SSL certificate is out of date for their webdav connection. the URL dav.idrivesync.com does not have a valid certificate. All they need to do is fix this and it would work with odrive. I opened a ticket, but got no response from idrive. Everyone who sees this should open a ticket with idrive about how their SSL cert for their webdav link is invalid.
I thought that would work until I returned to my computer the following morning, with still lots of data to restore, and I found Backblaze had unpaused itself and deleted almost 20TB of data from my cloud backup. This amount of data takes years to accumulate, and although there may have been a way to have them restore my data from a backup, I decided that I would look for an alternative cloud backup at the same time as figuring out my new local storage strategy.
I decided to buy a Synology NAS (Network Attached Storage) drive, which is more a server than just a hard drive, and I also bought a 10Gbps switching hub so that the information between my iMac and my Synology drive would be as fast as possible, and importantly, faster than my Drobo drives over Thunderbolt. My WIFI network speeds limit what I can do from my MacBook Pro, but I can also access the NAS over the network, which is a big benefit over the Drobo, which had to be plugged into one machine. Of course, I could access files on the Drobo via my iMac if it was turned on, but the transfer speeds were pretty poor compared to my Synology and my current network.
For my cloud storage, after a false start when I was told that they did not have a plan with enough storage space for my requirements, I ended up working with a company called iDrive, who released a 50TB plan shortly after we spoke, and they had point-blank refused to create one. This was annoying, but iDrive was realistically my only option, both from the data volume perspective and because they have a service that they call Express Backup, which enables users to request a hard drive and copy their backup data to the drive and post it back to iDrive so that they can add it to your storage space, and you can then continue to incrementally backup your data.
Once the offline data transfer was finished, I followed their instructions to start my backup, which was supposed to be incremental, just adding new files, but the validation of my data was and still is painfully slow, and for more than four months, it froze every few days. I would then have to restart the iDrive service on my Synology box and start the backup again, but when I do that, it starts from the beginning again. I eventually removed a large chunk of the data I wanted to back up so that the transfer would get to my new photos from this year, which were not yet backed up to the cloud.
So essentially, I am coming up to my eighth month of using iDrive, and I have 46% of my data confirmed as being backed up, despite most of the remaining data being stored on my remote storage already. I doubt this initial network backup will finish by the end of the year, and I believe there is still a chance it may never finish, as I will continue to add batches of images regularly.
If, like me, you need a way to transfer your initial backup on physical drives, you may have no option but to try iDrive, as I did. If that is the case, I wish you better luck than I have had. Let me know how your experience goes if you try them. Also, if anyone, either now or in the future, finds a better alternative, please let me know. I have considered AWS and Synology C2 but cannot do my initial data transfer offline.
I am still using BackBlaze which has been working well for me (with the exception of the time I switched from PC to Mac and they told me that everything has to be uploaded again since they store the two backups in completely different formats). But I am surprised you cannot use Amazon Glacier. As far as I have seen, you used to be able to send them a harddrive with your initial data. Maybe this has changed with AWS, but I am sure their infrastructure is much better than idrive so even if you have to upload everything it still might be faster.
4a15465005