S3 droplet: Amazon object lock / wasabi.com compliance issue

[Brahim Raddahi]

I see that in https://github.com/bareos/bareos/blob/Release/19.2.7/core/src/stored/backends/droplet_device.cc a call to dpl_mkdir is used.

There is also a related FIXME in the same file which explains that a check for "does this directory exist" does not work as expected:

/*
	* FIXME: With the current version of libdroplet a dpl_getattr() on a
	* directory fails with DPL_ENOENT even when the directory does exist. All
	* other operations succeed and as walk_chunks() does a dpl_chdir() anyway
	* that will fail if the directory doesn't exist for now we should be
	* mostly fine.
	*/

The issue:

If you have a policy on S3 which prevents overwriting or deleting objects, the directory is still created multiple times, which is seen as an "overwrite"

On such buckets, the operation fails with the message "Job XXXXXXXXXXXXXXXXXXXX is waiting. Cannot find any appendable volumes."

The error that is triggered initially is this one: https://github.com/bareos/bareos/blob/Release/19.2.7/core/src/stored/backends/droplet_device.cc#L477

The problem seems to be related to the FIXME note: it never sees that the directory already exists, and just creates it again, violating the bucket policy

Note: this happens after the first chunk, when trying to create the second chunk (first create of the directory works, second fails, because it is an "update")

Is there a reason to create the directory? AFAIK S3 does not require the parent directory to exist when you create an object.

If I download the source rpm and add a "break;" right after https://github.com/bareos/bareos/blob/Release/19.2.7/core/src/stored/backends/droplet_device.cc#L390 and reinstall the rpm, it works as expected.

If there is no other reason to have the directory specifically be created, I think this can be an easy fix: just never create the directory?

Note: I had to do this with the source rpm of 19.2.6, because 19.2.7 had just been released 3 hours earlier, and there was no source rpm yet :)

[Frank Ueberschar]

Thank you for your important note. We will have discuss this in one of our next developer meetings.

Besides, there is no point in reading the code of 19.2.7 because we did not change that, recently.

Am 16.04.20 um 19:23 schrieb Brahim Raddahi:

--
You received this message because you are subscribed to the Google Groups "bareos-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bareos-devel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-devel/e3a8cac7-106c-4e77-9885-c5981a618a6b%40googlegroups.com.

-- 
Mit freundlichen Grüßen

 Frank Ueberschar                          frank.ue...@bareos.com
 Bareos GmbH & Co. KG                      Phone: +49 221 63 06 93-88
 http://www.bareos.com                     Fax:   +49 221 63 06 93-10

 Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
 Geschäftsführer: S. Dühr, M. Außendorf, J. Steffens, P. Storz