See amount of dirty cache question and backgrounding issue

[Peter Brunnengräber]

Hello all,

  My apologies if I'm missing the obvious here, but I had a few questions on using s3backer...

  I'm very much considering this for a production system as a way to store older data, so to ensure integrity, I've been running some trials to see under what cases I can create data inconsistency. I plan to try and document the ways in which things go wrong with different block sizes, cache sizes, dirty blocks allowed, etc.

  All this lead me to a question... How do I go about determining the number of the blocks in the cache are dirty?

  And one more question... If I start s3backer with '-f' to keep it in foreground (which is how I've been operating) everything is fine.  If I don't use the '-f' it exits straight away and does leave a running process, but the 'file' and in the mount directory never appears.

Thank you kindly!

P.S.   I've also been testing on Oracle's s3 object storage service and things are working well thus far.

[Archie Cobbs]

On Friday, March 26, 2021 at 10:43:22 AM UTC-5 Peter Brunnengräber wrote:

I'm very much considering this for a production system as a way to store older data, so to ensure integrity, I've been running some trials to see under what cases I can create data inconsistency. I plan to try and document the ways in which things go wrong with different block sizes, cache sizes, dirty blocks allowed, etc.

All this lead me to a question... How do I go about determining the number of the blocks in the cache are dirty?

The stats file has this information. In particular, the block_cache_dirty_ratio multiplied by the block_cache_current_size should give you the number of dirty blocks.

 

And one more question... If I start s3backer with '-f' to keep it in foreground (which is how I've been operating) everything is fine.  If I don't use the '-f' it exits straight away and does leave a running process, but the 'file' and in the mount directory never appears.

What may be happening is that you are specifying the mount directory with a relative pathname... try using an absolute pathname. This would matter because when s3backer becomes a daemon it changes to the root directory

If that's not the problem, make sure you have the latest version (1.6.1), and also look for any error messages in /var/log/messages.

 

-Archie

[Peter Brunnengräber]

Hello Archie,

Ah, yes...  I didn't even recognize that could be a plain-text file.  Yes, I see it now... this is perfect.   Sorry for my ignorance on that.

I'm running the debian package which is v1.5.0 at the moment.  I am using the full path, but I'll check more on this and test the newer version.

Thank you!