Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

tar and tgz size

1 view
Skip to first unread message

mel

unread,
Dec 23, 2001, 4:04:34 PM12/23/01
to
Hello there,

I've read somwhere that a tar file is limited to 2Gb. Is it true. What
does it means. Is it only a tape limit or a software limit?

Does a tgz file is also limited?

Thank and merry Xmas !

Michel


Tony Lawrence

unread,
Dec 23, 2001, 4:57:49 PM12/23/01
to
mel wrote:
>
> Hello there,
>
> I've read somwhere that a tar file is limited to 2Gb. Is it true. What
> does it means. Is it only a tape limit or a software limit?
>
> Does a tgz file is also limited?


Dependent upon the OS and file system in use, an individual file and
sometimes even entire file systems can be limited to 2GB. That's not
at all uncommon, in fact. Nor is it unusual to have a file system that
may support terabytes but have individual files still stuck at 2GB.

When the OS and file system do support larger sizes, some utilities will
still have limitations just because they expected those limitations so
the programmer saw no need to store numbers in more bits. Therefore
it is also not unusual to have an OS and FS that supports much larger
files and yet still find certain utilities that will fail if invoked on
files larger than 2GB.

--
Tony Lawrence
SCO/Linux Support Tips, How-To's, Tests and more: http://pcunix.com

mel

unread,
Dec 25, 2001, 3:51:18 PM12/25/01
to
Thank you Tony,

The goal is to make a daily backup of a server's home directory on a
second HD, and a weekly on cdrom.

I feel that I will wait to reach the 2Gb limit and next I will create
a separate tgz file for each user. Fortunatelly it's a small and
stable organisation so I will not have to make change often.

Thanks again.

Mel

PS: I use Mandrake 8.1

On Sun, 23 Dec 2001 21:57:49 GMT, Tony Lawrence <to...@pcunix.com>
wrote:

Charles Sullivan

unread,
Dec 25, 2001, 8:43:26 PM12/25/01
to

2 GB is the size limit for any file, tar or otherwise, on the Linux ext2
file system. Other types of Linux-compatible file systems may have
greater (or lesser) file size limits.

Tape drives have no file system per se and a tar file written to a tape
drive is limited only by the capacity of the tape.

A tgz is just a compressed tar file. It's compressed on the fly as it's
being written so it's only the final size of the compressed file that
counts.

mel

unread,
Dec 26, 2001, 5:10:12 AM12/26/01
to
Thanks Charles,

The 2GB limit is about the compressed tar file only ... It's less
worst :-)

Have a nice day.

Michel

ynotssor

unread,
Dec 26, 2001, 1:14:16 PM12/26/01
to
"mel" <mel...@mailandnews.com> quoted and wrote in message
news:3c29a0d...@news.wanadoo.be...

> Thanks Charles,
>
> The 2GB limit is about the compressed tar file only ... It's less
> worst :-)
...

One can "split" the output of tar into multiple files, each of which is less
than 2^31-1 bytes. Consider this scenario, where we round that number down to
the nearest billion for typographical ease:

cd /big/disk
tar czvPf - /some/big/dir | split -b 2000000000

This will create multiple files of name xaa,xab,xac,...,xba,xbb,xbc,... each of
which is within the filesystem size limitations. The order of the files is
easily addressed using ASCII collation, so to recover <anything> is simple:

cat x* | tar xvzf - <anything>

HTH,


tony

-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----

mel

unread,
Dec 27, 2001, 2:57:18 AM12/27/01
to
That's also a nice idea.

Thank Tony

ynotssor

unread,
Dec 27, 2001, 3:21:53 AM12/27/01
to
"mel" <mel...@mailandnews.com> quoted and wrote in message
news:3c2ad44...@news.wanadoo.be...

> That's also a nice idea.
>
> Thank Tony
[...]

> >One can "split" the output of tar into multiple files, each of which is less
> >than 2^31-1 bytes. Consider this scenario, where we round that number down
to
> >the nearest billion for typographical ease:
> >
> > cd /big/disk
> > tar czvPf - /some/big/dir | split -b 2000000000

One can simplify mo' bettah with "-b 2000m".

> >This will create multiple files of name xaa,xab,xac,...,xba,xbb,xbc,... each
of
> >which is within the filesystem size limitations. The order of the files is
> >easily addressed using ASCII collation, so to recover <anything> is simple:
> >
> > cat x* | tar xvzf - <anything>

[...]

Note that the EOF markers are considered "trailing garbage" but are well dealt
with by gnu tar; exit status is 2, however, despite successful file
restorations.

0 new messages