Amount of data doesn't fit sizing guide

[Caroline Kiel]

Hi all,

I've just created a new SDFS installation on Windows.

The volume is used for backup purposes and writes data to S3.

The cache is set to 512 GB, the amount of data written to the SDFS file system is (raw) 5780 GB, deduplicated to 1130 GB (5.11:1).

Unfortunately, we see far more "administrative" data on the disk, than mentioned in the admin guide:

There it states:

0.21% of raw data for File Meta data (/files and /ddb)

0.2% (or on another page 0.4%) of deduped data for Hash table (/hdb)

This would mean in our situation:

0.21% * 5780 GB = 12.14 GB Meta

0.4% * 1130 GB = 4.52 GB Hash

512 GB Cache

Sum: 528.66 GB

In reality, we see a usage on the disk of 642 GB (646 GB on disk)

In detail:

/files: 3.31 GB (24.5 GB on disk)

/ddb: 115 GB (118 GB on disk)

/hdb: 9.33 GB (9.87 GB on disk)

Cache: 517 GB

Can anyone explain, why this happens?

Is the sizing guide wrong or where does this behavior come from?

Best regards,

Caroline

[Caroline Kiel]

No one having an idea? The database (/ddb) has already grown to 326 GB now..

[Rinat Camal]

I think this is same issue as

https://github.com/opendedup/sdfs/issues/88

"chunks" folder is not being cleared correctly.

[Caroline Kiel]

Hmm, we don’t have any issues with the /chunks folder, but with the database /ddb folder.
And, since running on Windows, the option of using any other file system is quite difficult...