Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

tar or gzip dry run?

7,271 views
Skip to first unread message

Lars Uffmann

unread,
Jul 28, 2009, 12:11:38 PM7/28/09
to
Does anyone know of a way to determine the final file size of a tar
archiving process without actually writing anything to disk? Like a dry run?

Let's say I want to know if my final archive will fit on a given drive
*before* actually creating it? And then directly create it at the
destination, if it fits, otherwise log an error message and not archive
anything.

I have found the --list option for gzip, which lists the compressed file
size, and -c to output the file to stdout instead of disk, which I could
suppress - however that does not include the tar archiving process
(which should come first), and those two options seem to not work
together...

Any ideas?

Best Regards,

Lars

jellybean stonerfish

unread,
Jul 28, 2009, 12:28:59 PM7/28/09
to

You could pipe the stdout of tar to a command to count characters. There
is no reason

tar --gzip --create --file - directory | wc -c


Lars Uffmann

unread,
Jul 28, 2009, 1:11:48 PM7/28/09
to
jellybean stonerfish wrote:
> You could pipe the stdout of tar to a command to count characters. There
> is no reason
>
> tar --gzip --create --file - directory | wc -c
>
I assume you meant to add "[..] wouldn't work" :) And you are so right!

I did not know about the wc tool, and I did not find the information how
to redirect tar output to stdout. You covered both :D

Thanks a lot!

Lars

Chris Davies

unread,
Jul 28, 2009, 5:38:18 PM7/28/09
to
jellybean stonerfish wrote:
> You could pipe the stdout of tar to a command to count characters. There
> is no reason
> tar --gzip --create --file - directory | wc -c

Lars Uffmann <ar...@nurfuerspam.de> wrote:
> I assume you meant to add "[..] wouldn't work" :) And you are so right!

It looks like there may be some sarcasm there. I'm not entirely sure
why, because jellybean stonerfish's suggestion works perfectly. At least
for me:

$ tar --gzip --create --file - /boot | wc -c
tar: Removing leading `/' from member names
37972847
$

So, archiving /boot on my system would require approximately 38MB.

I must admit that, personally, I'd have used the older flags and done
it this way, but the effect is exactly the same:

$ tar czf - /boot | wc -c

Chris

jellybean stonerfish

unread,
Jul 29, 2009, 2:55:21 AM7/29/09
to
On Tue, 28 Jul 2009 22:38:18 +0100, Chris Davies wrote:

> jellybean stonerfish wrote:
>> You could pipe the stdout of tar to a command to count characters.
>> There is no reason
>> tar --gzip --create --file - directory | wc -c
>
> Lars Uffmann <ar...@nurfuerspam.de> wrote:
>> I assume you meant to add "[..] wouldn't work" :) And you are so right!

I remember trying to say that tar would have to read everything with this
method, and there is probably a better way, but decided to make my post
short. I don't have a clue where the string "There is no reason" came
from. I don't remember typing that.

>
> It looks like there may be some sarcasm there. I'm not entirely sure
> why, because jellybean stonerfish's suggestion works perfectly. At least
> for me:
>
> $ tar --gzip --create --file - /boot | wc -c tar: Removing leading
> `/' from member names 37972847
> $
>
> So, archiving /boot on my system would require approximately 38MB.
>
> I must admit that, personally, I'd have used the older flags and done it
> this way, but the effect is exactly the same:
>
> $ tar czf - /boot | wc -c
>
> Chris

I tested it with "-czf -" and use that style of options myself. I posted
the long options to be pedantic.

Lars Uffmann

unread,
Jul 29, 2009, 1:40:21 PM7/29/09
to
Chris Davies wrote:
>> You could pipe the stdout of tar to a command to count characters. There
>> is no reason
>> tar --gzip --create --file - directory | wc -c
>
> Lars Uffmann <ar...@nurfuerspam.de> wrote:
>> I assume you meant to add "[..] wouldn't work" :) And you are so right!
>
> It looks like there may be some sarcasm there.

Nope, I was serious - I was completing the sentence from above the
suggestion :)

Lars Uffmann

unread,
Jul 29, 2009, 1:44:41 PM7/29/09
to
jellybean stonerfish wrote:
> I remember trying to say that tar would have to read everything with this
> method, and there is probably a better way, but decided to make my post
> short.
Actually I wouldn't know how there could be a better way - gzip doesn't
really know the filesize it will reach by compressing until it has
processed the whole data, I guess...

> I don't have a clue where the string "There is no reason" came from. I don't remember typing that.

OMG, it's gotta be *them*! Watch out, they are everywhere ;)

> I tested it with "-czf -" and use that style of options myself. I posted
> the long options to be pedantic.

Actually, as a friend pointed out, -f - is merely giving the --file
option it's default value. So
# tar -cz * | wc -c
works just as fine. Glad to have such a steep learning curve still -
maybe I'm not so old yet after all ^^

Thanks again for your help, it was your solution (with -czf) that I
implemented to get things working last night - a neat little backup
script which has big room for improvement, but I will work on that
tomorrow...

Best Regards,

Lars

Lars Uffmann

unread,
Jul 29, 2009, 1:46:35 PM7/29/09
to
Lars Uffmann wrote:
> option it's default value. So
^^^^
its! I meant to type its! :)

jellybean stonerfish

unread,
Jul 29, 2009, 6:25:08 PM7/29/09
to

If it belongs to it, it is it's, if you have a more than one it, it is
its, and if it belongs to more than one it, it is its'.

Lars Uffmann

unread,
Jul 29, 2009, 7:11:37 PM7/29/09
to
jellybean stonerfish wrote:
>> Lars Uffmann wrote:
>>> option it's default value. So
>> ^^^^
>> its! I meant to type its! :)
> If it belongs to it, it is it's, if you have a more than one it, it is
> its, and if it belongs to more than one it, it is its'.

I was trying to differ from "it is"... but yes I guess in genitive case,
it's also "it's"...

Chris Davies

unread,
Aug 4, 2009, 11:07:42 AM8/4/09
to
Lars Uffmann <ar...@nurfuerspam.de> wrote:
> Actually, as a friend pointed out, -f - is merely giving the --file
> option it's default value. So

That's only true for certain instances of default. By providing the "-"
explicitly you guarantee the tar will (more probably) do what you expect.

Chris

Lars Uffmann

unread,
Aug 10, 2009, 7:33:23 AM8/10/09
to
Chris Davies wrote:
> That's only true for certain instances of default. By providing the "-"
> explicitly you guarantee the tar will (more probably) do what you expect.

(back from a business trip, sorry for the late reply) Could you
elaborate on "certain instances of default"? Otoh I did a little math
and found out I'll probably need to do a different approach since a
whole tgz archive creation will need many hours for ~ 500GB - and I
probably cannot afford to zip it all up for the backup, much less do
that twice, only to know how much disk space is needed.

I might just do an unpacked backup, that will give me a good idea of how
much disk space I need and I'll be fine as long as my backup media are
big enough...

Best Regards and thanks for your help!

Lars

0 new messages