Backup fo deduplicated Windows Disk too small

[Andreas Grabner]

Hi,

i plan to switch the whole backupsystem in my company to bareos and do some tests. I backed up a windows Server 2016 machine where the disk uses dedup. There are 7TB user data on the disk but the first Full backup is just arround 1TB in size (without deduplication or compression on bareos side).

I found out, that some (large) files have zero filesize in bareos, also the restored file has zero bytes.

Using the tool treesize on Windows shows for a sample file a size of 170GB but allocated 0 Bytes. Files where allocated shows 170GB get restored correct.

Is there an setting in bareos(-fd) i have missed.

Kind regards
Andreas

[Andreas Rogge]

Am 27.08.25 um 10:18 schrieb Andreas Grabner:
> Hi,

Hi Andreas,

> i plan to switch the whole backupsystem in my company to bareos and do
> some tests. I backed up a windows Server 2016 machine where the disk
> uses dedup. There are 7TB user data on the disk but the first Full
> backup is just arround 1TB in size (without deduplication or compression
> on bareos side).

I'm glad to hear that you consider Bareos.
However, the behaviour you're seeing is pretty weird. When dedup is
enabled you should actually see more data in Bareos than there is on disk.

> Using the tool treesize on Windows shows for a sample file a size of
> 170GB but allocated 0 Bytes. Files where allocated shows 170GB get
> restored correct.

That is really weird. Which file system are you using?
I could only imagine that for some reason Bareos determined the file is
empty (i.e. looking at what is reported as "allocated") and thus doesn't
even bother backing up the file contents.
This sounds like something we should investigate right now!
Are you able to provide any insight on how to reproduce a file system
with such content (i.e. files reporting 0 allocated bytes) so we can
find out what is going wrong here?

> Is there an setting in bareos(-fd) i have missed.

I wouldn't know about anything to set here.

One thing you could try is to confgure a fileset that only contains a
single one of these files that are missing data. If with that fileset
Bareos still doesn't pick up the data, you could run the job with
tracing in the fd enabled on at least level 450 like this:

setdebug client=<your-fd> trace=1 level=450
run job=<your-test-job> level=full
wait
setdebug client=<your-fd> trace=0 level=0

And then provide the generated tracefile (preferable here on-list, but
you can also send it to me directly). Maybe this already shows why
Bareos did not pick up that data.

Thank you for the effort!

Best Regards,
Andreas
--
Andreas Rogge andrea...@bareos.com
Bareos GmbH & Co. KG Phone: +49 221-630693-86
http://www.bareos.com

Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Komplementär: Bareos Verwaltungs-GmbH
Geschäftsführer: Stephan Dühr, Jörg Steffens, Philipp Storz

[Andreas Grabner]

Hi Andreas

thanks for the quick answer und the intrest in this topic!

Andreas Rogge schrieb am Mittwoch, 27. August 2025 um 12:28:08 UTC+2:

Am 27.08.25 um 10:18 schrieb Andreas Grabner:

> Using the tool treesize on Windows shows for a sample file a size of
> 170GB but allocated 0 Bytes. Files where allocated shows 170GB get
> restored correct.
That is really weird. Which file system are you using?
I could only imagine that for some reason Bareos determined the file is
empty (i.e. looking at what is reported as "allocated") and thus doesn't
even bother backing up the file contents.
This sounds like something we should investigate right now!
Are you able to provide any insight on how to reproduce a file system
with such content (i.e. files reporting 0 allocated bytes) so we can
find out what is going wrong here?

I'm not sure. The person who configured this left the company, so i have to investigate it if there are further details needed
Filesystem-Type is NTFS

Dedup setting is "Allgemeiner Dateiserver"

Age to dedup ist 3 days

Excluded File-extentions are: ebd,jrs

(see attached Image)

 

One thing you could try is to confgure a fileset that only contains a
single one of these files that are missing data. If with that fileset
Bareos still doesn't pick up the data, you could run the job with
tracing in the fd enabled on at least level 450 like this:

setdebug client=<your-fd> trace=1 level=450
run job=<your-test-job> level=full
wait
setdebug client=<your-fd> trace=0 level=0

And then provide the generated tracefile (preferable here on-list, but
you can also send it to me directly). Maybe this already shows why
Bareos did not pick up that data.

See attached trace. 

Some other notes. 

Tried it with and without VSS, same result.

The File size shown in webui ist 170GB, on sd and recovered the file has zero Bytes.

The backup takes just a few seconds.

Just ask for details if needed!

Kind regards
Andreas Grabner

FENZ-Software GmbH | necta Österreich GmbH | Steinbrückl 7a/11 | 7531 Kemeten
Tel.: +43 3352 22 626 |  www.necta-group.com

[Andreas Rogge]

Am 27.08.25 um 17:07 schrieb Andreas Grabner:

> Hi Andreas
> thanks for the quick answer und the intrest in this topic!

I'm always happy to get reports like this, as we tend to overlook things
like this that are rarely used.

> See attached trace.
Thank you.
The trace just shows a "normal" file backup.

> Some other notes.
> Tried it with and without VSS, same result.

> The File size shown in webui ist 170GB, on sd and recovered the file has
> zero Bytes.
> The backup takes just a few seconds.

The basic file attributes are picked up correctly, thus you see 170 GB.
Nevertheless, the FD only picks up a stub-file, which then leads to 0
bytes on restore.

I looked into how deduplication on Windows works (at least on the
documentation end) and I'm honestly not sure how we're supposed to back
something like this up.

Basically, Windows deduplication turns the file into a reparse point and
removes data from it. When you access the file normally (i.e. without
backup semantics) the filesystem will redirect access to the
deduplicated data. However, when using backup semantics (like Bareos
does) you only get the reparse point and not the data.

Could you try to reconfigure your fileset and set the option "Portable =
yes"? This should back up the data without backup semantics. While this
is doomed to miss a lot of attributes and acl, it should actually pick
up the data.

Also, when you tested a restore, did you restore to the very same file
system? Basically, my assumption is that if we back up and recover the
reparse point data correctly (I'm not sure we do), you should get the
same stub-file recovered and it should point to the same deduplicated
data chunks.
While this is pointless for back up (you obviously want the data stored
and not just references to it), it would be interesting to know.

[Andreas Grabner]

Hi Andreas

Andreas Rogge schrieb am Donnerstag, 28. August 2025 um 14:59:15 UTC+2:

Am 27.08.25 um 17:07 schrieb Andreas Grabner:

> The File size shown in webui ist 170GB, on sd and recovered the file has
> zero Bytes.
> The backup takes just a few seconds.
The basic file attributes are picked up correctly, thus you see 170 GB.
Nevertheless, the FD only picks up a stub-file, which then leads to 0
bytes on restore.

I looked into how deduplication on Windows works (at least on the
documentation end) and I'm honestly not sure how we're supposed to back
something like this up.

Basically, Windows deduplication turns the file into a reparse point and
removes data from it. When you access the file normally (i.e. without
backup semantics) the filesystem will redirect access to the
deduplicated data. However, when using backup semantics (like Bareos
does) you only get the reparse point and not the data.

Could you try to reconfigure your fileset and set the option "Portable =
yes"? This should back up the data without backup semantics. While this
is doomed to miss a lot of attributes and acl, it should actually pick
up the data.

With Portable=yes  it backups the data. So it is possible to restore to a different system.
This would work for me.

Also, when you tested a restore, did you restore to the very same file
system? Basically, my assumption is that if we back up and recover the
reparse point data correctly (I'm not sure we do), you should get the
same stub-file recovered and it should point to the same deduplicated
data chunks.
While this is pointless for back up (you obviously want the data stored
and not just references to it), it would be interesting to know.

Yes it is pointless but/and Windows is a funny thing.

When i restore the  "reparse point data" i can access the file it is "normal" with data in it. This also works when restore to an other drive on the Windows host (e.g. C: an D: both works). This was unexpected.
I have not tried to delete other files (which hold the Databolcks) to see what is happening. There would be some intresting tests considering the scrubbing, optimization and so on, but i will not invest my time in this.

Thanks a lot for your help! I will continue to evaluate bareos. For this one fileset Portable is ok for us.

Kind regards

Andreas