Cyberduck+Cryptomater+OneDrive=Corrupted files

[cybergoose]

Hello Cyberduck fans and devs.

I am in dire need of some help.

I recently moved a bunch of video files for archival into a cryptomator vault on ondrive for business (sharepoint).

I used cyberduck for that, because the official onedrive client is very finecky when working with cryptomator and having files stored online only. I thaught I did my due diligence and actually tested some of the files before deleting them, approximately the first 3 and another 5 out of the next 20. There are approximately 150 video files, 650GiB in total now.

I evven did the sha256 check with 7zip, files were identical. So I trusted Cyberduck and carried on.

Apparently I had bad luck with my randomized samples, because now when I went to restore a larger batch of them, approximately 20% of them fail.

The files in question are still present at full size (encrypted) on OneDrive. But when downloading them through cryptomator, they always abort at the same point (different for each file respectively), with the following error, except the chunk number which is varying from file to file:

 For those who don't speak German(and who does really), a translation:

Error

!Acces denied

Authentication of chunk 2853 failed.

->Try again

^Show details   [Abort]

!Help

Chunk 2853 seems strange at this point, because it aborts at the 89MiB mark of a 7.0GiB file.

I can still download the whole 7.0 encrypted GiB though, via Cyberduck with closed vault, the web interface or the onedrive client. So there is something on the OneDrive at least, and acces does not seem to be the problem.

What the problematic files have in common are a tendency for more special character and a "space" before their mp4 ending, eg "video !_# .mp4".

Im testing them all right now if that is a rule or only an inkling of mine.

I appreciate any help. Some of the videos are unique and not recoverable but dear to my heart.

[John Navas]

1. Until virtual retrieval is fixed, you should still be able to download to local storage, and then decrypt your precious files, no?
2. This underscores the importance of not putting your eggs in one basket. I recommend the 3-2-1 backup strategy. Without a local copy, I always have at least 2 copies in different remote storage.
3. You should avoid certain characters in files names. See Characters to Avoid in Directories and Filenames | Michigan Tech.

[cybergoose]

1. What is "virtual retrieval"? I will try to download the whole vault once I made place for it. I am checking first now which files are corrupted as well. Not every local copy is deleted yet. The one test I did with the OneDrive Software and Cryptomator (standalone), it did not work.

2.I am fully aware of the correct backup strategies. Due to worldy constraints, sometimes you just can't oblige. I ran out of space and had to save different stuff.

3. So basically only numbers, small letters and underscore.

additionally:

4.I tried the mountain duck. It was actually able to, seemingly fully, recover one of the files, and download a second one that came out heavily corrupted. Some recovery software quenched some fitting audio and video out of even the second file though.

5.The whole combination of cryptomator vaults, one drive and cyberduck is shaky. I also have massive problems downloading files when using the segmented downloads.

[John Navas]

1. Virtual retrieval is accessing a decrypted file from the cloud; i.e., decryption on the fly.
The alternative is to download the encrypted file, and then decrypt it locally (which is more secure).
2. Storage is dirt cheap. If you allow yourself to run out of storage, then, and with respect, your data must not be terribly valuable.

4. Direct download from OneDrive is simple and reliable. If corruption occurs, then you probably have a compromised system.

5. Your apparent issue is virtual access. Instead, download before decryption.

6. For alternatives to Cryptomator, see Cryptomator Alternatives | AlternativeTo. I recommend VeraCrypt.

[cybergoose]

Thank you for all the spite. Apparently you are much better aware of my situation. Just send me your money. I will order drives.

1. I thought virtual retrieval is exactly what Cyberduck does in combination with cryptomator.

2. Everything is a matter of perspective.

3. Is a bad number.

4.I am trying that next. My system is not compromised in any matter, but thank you for the suggestion. That some files work others don't, the same repeatable behaviour from three systems make me confident of that.

5. I am trying that next.

6. Have you tried veracrypt with a cloud yet? Great fun to update a 650GB container when you changed line in a textfile. The whole reason cryptomator and boxcryptor were written in the first place.

In all seriousness, I hoped someone could teach me how to maybe decrypt singular files with the commandline, ignoring the errors and at least retreiving something. Videos can take some corruption. I would rather have a couple corrupted frames then nothing at all.

[John Navas]

Your rudeness is uncalled for, but you're welcome nonetheless, and even though you don't appreciate my help and advice, others may benefit. 

1. Yes, and that virtual retrieval is the apparent source of your problem. Don't you want to solve your problem?

2. True, but Backblaze offers unlimited backup for only $70 per year. Recommended.

If you prefer local storage, Newegg has Seagate portable hard drives on sale.

4. It's better than even odds that your system is compromised.

5. Good.

6. I manage cloud storage more granularly. Large container files are a recipe for disaster.

"Hope is not a strategy." In all seriousness, follow my advice (after thoroughly checking for compromise), and you will almost certainly recover your files.

[cybergoose]

I didn't consider it rudeness, I considered it equal amount of snarkiness, with all due respect. I do appreciate every help, so thank you and apologies.

1.I did a limited testrun with one of the files (official OneDrive client beeing tricky), and it first asked me for administrative rights, and then failed. That is why I have limited expectancies. But I will try it.

2. I am a Backblaze costumer for years, even reserved the next year for 70bucks with the coming price increase. It only covers files though that you still have locally + 30 days. Also Newegg is not a thing across the pond.

4. The odds of any of my systems being compromised is very small. Either point me the way to confirm that they are, otherwise we assume innocent until proven otherwise.

5. I have to organize a drive with 650gigs of free, continous storage space, and then download the whole ordeal. While hoping that nothing else happens.

6. Exactly what I said, what I pointed to. This is why cryptomater and boxcryptor are for clouds (and other single files encryptions), while veracrypt is for local applications.

Hope is no strategy, but part of tactics. I had no confirmation that someone here will certainly know the right commandline to just "AES-magic" the "slightly" corrupted files back to slightly corrupted videos, but I had to try. Hope is formulaic.

I know a thing or two about backups, and data management. Not professionally, but I usually am the one configuring everything for friends and parents uncles grandmothers for whom a backup is a second copy, preaching 3-2-1 for years.
But sometimes you don't follow best practises, and you try to cut costs, and it bites you. Do as I say, not as I do...

[John Navas]

I assure you there's no snarkiness in my responses, and I'm sorry you got that impression.

1. How is the OneDrive client tricky for you? I use it here, as do many of my clients, without issues.
Regardless, Filezilla Pro is another option for OneDrive.

2. As you may know, Backblaze offers Extended Version History.

There are external hard drive sellers in other markets, including Amazon.

4. I recommend these malware scanners:

• HitmanPro
• AdwCleaner
• Spybot
• Malwarebytes
• Microsoft Safety Scanner

5. It's not necessary for the space to be continuous. You can even span multiple drives.
Regardless, I personally would use an external hard drive that can be formatted and tested.

6. With respect, you're missing my point. I have multiple terabytes in the cloud, with no file in excess of a few gigabytes. I would never use a giant container file. I have secure data organized in cloud folders, not containers, with individual files encrypted, and when there is no local copy, it is replicated on multiple cloud providers (not just one), for which VeraCrypt works well. 

There is no magic.™ You cannot decrypt properly encrypted files that have been corrupted. Good encryption is designed to foreclose that possibility as a matter of security.

[cybergoose]

So I have the file offline now.

It won't copy.

When I point Cyberduck to my local fault, it tells me the same things as witht the on the fly decrpytion. "Authentication of chunk" failed.

When I use the official Cryptomator application, it's Windows Explorer that gives me some very generic messages.

First "Acces denied, administrative rights required", and when those are granted, it says the vault has been disconnected. It is directly accessible again though.

Some of the files get through with the mountainduck trial, but they are corrupted to varying degrees.

My inkling with the filenames seems to be mostly coincidental. There are files with freaking smileys in the name that work fine.

So what logs can I post where? Is there anything left to do?

[cybergoose]

So I did some further testing.

A large mp4 file, only small letters, underscores and number in the filename, uploaded in one go into a onedrive cryptomator vault.

Tried downloading it. Fails. Something is off here, I'm now afraid for all of my data. Also, I always recommended cryptomator and cyberduck to make clouds finally usable, now I'm not so sure anymore.

[Emre]

Hey,

Outside main discussion here with necessary key and everything without folder structure of the vault. Is it possible to decrypt the files. If it is what's the simplest way to go through that. I spot while we are uploading several big chunk files around the same size files appear on the cloud with some increase of size with encryption. If I just pull a single file without anything else. We keep checking the files vs original files and other copies but because of how cloud providers process the data sometimes a very small part of data gets altered even though it does not affect our operations we replace them before getting worse. It can happen while downloading or uploading. We have tape drives, a storage server and drive for most active used files between regions. If there is another way It can increase our throughput. For some reason, we can't make Cyberduck or mountain duck operations faster.

[John Navas]

1. What does "fails" mean?
2. What does "it won't copy" mean? Do you have a local file system fault?
3. Did you create a local vault, and then upload it, or a remote vault on OneDrive?
4. Did you download your vault to local storage before accessing it?
5. Did you check the vault for corruption with an SHA hash? Before uploading? After downloading?
6. Why are you trying to use Cyberduck instead of Cryptomator? 

--
--
You received this message because you are subscribed to the Google Groups "Cyberduck" group. To post to this group, send email to cybe...@googlegroups.com
To unsubscribe from this group, send email to cyberduck+...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/cyberduck
--
Post bug reports and feature requests
http://trac.cyberduck.ch/newticket
--
Support development
http://cyberduck.ch/donate/

---
You received this message because you are subscribed to the Google Groups "Cyberduck" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cyberduck+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cyberduck/34f6f8ca-65b3-457d-a25b-68c655386678n%40googlegroups.com.