Compression recommendation for Windows
Paul Hemans
fairly large files, the total file count is around 7,000. Can I use a
combination of TAR and GZIP to deal with this? Winzip ver 6 creates a
compressed file of 440MB but I can only access the first 360 files!
Aslan
"Paul Hemans" <pauln.o...@laberg.com.au>, haber iletisinde şunları
yazdı:44d13dd7$1...@news.iprimus.com.au...
would need to decompress the whole tar ball first and then extract the
single file. If that's not a problem for you yes you can.
I am surprised that Winzip cannot handle 7000 files. The number files that
can be added to the zip archive should be up to 65536 (SHORT_MAX).
jasen
we have .tar.gz files 2.5G in size with 75000 files...
tar is industrial strength, but for your task zip should work too.
I think winzip is ignoring some of your files.
I've never been a fan of those GUI archive tools
Bye.
Jasen
Paul Hemans
access any files after the "A"s. I have tried using both pkunzip and winzip.
I have gone back through a number of these zip files and some have worked
and some seem to have stopped at a fairly early stage. It gave me no
warnings while the backup was occuring. The specific location where it stops
changes.
There must be a logical explanation for it. This ran on a new laptop where I
copied winzip over to it without registering it, so now I have registered,
it work properly.
"Aslan" <aslans...@yahoo.com> wrote in message
news:eas204$bqj$1...@emma.aioe.org...
BORG Xx(°..º)xX
Before WinRAR I had always used LHA, then ARJ.
WinRAR works great, both GUI & command line.
I gave up on Zip ages ago...
> Bye.
> Jasen
cr88192
> "BORG Xx(°..º)xX" Xx@xX> wrote in message
news:44d3...@news.comindico.com.au...
note, however, that rar is not free, nor is it an open format.
tar, gzip, and zip, however, are all open formats, and there is a much
larger selection of compressors and decompressors available.
this also gives a sense of security for the future ability to easily
decompress these formats.
or such...
>> Bye.
>> Jasen
BORG Xx(°..º)xX
>
>> "BORG Xx(°..º)xX" Xx@xX> wrote in message
> news:44d3...@news.comindico.com.au...
>> "jasen" ja...@free.net.nz> wrote in message
>> news:da5.44d1a...@gonzo.homenet...
>>> On 2006-08-03, Paul Hemans <pauln.o...@laberg.com.au> wrote:
>>>> I have a large directory structure (1.2GB) multiple sub-directories and
>>>> some
>>>> fairly large files, the total file count is around 7,000. Can I use a
>>>> combination of TAR and GZIP to deal with this? Winzip ver 6 creates a
>>>> compressed file of 440MB but I can only access the first 360 files!
>>>
>>> we have .tar.gz files 2.5G in size with 75000 files...
>>> tar is industrial strength, but for your task zip should work too.
>>> I think winzip is ignoring some of your files.
>>>
>>> I've never been a fan of those GUI archive tools
>>
>> Before WinRAR I had always used LHA, then ARJ.
>>
>> WinRAR works great, both GUI & command line.
>> I gave up on Zip ages ago...
>>
>
> note, however, that rar is not free, nor is it an open format.
>
> tar, gzip, and zip, however, are all open formats, and there is a much
> larger selection of compressors and decompressors available.
There are FREE decompressors for most (pretty much all)
types of OS (current & ancient eg: Amiga) & the
decompression source & decompressors
are freely available to DL on the www.rarlab.com website
(and assorted P2P networks).
> this also gives a sense of security for the future ability to easily
> decompress these formats.
>
> or such...
With freely available decompression source it will never die
unless all the lights go out :) And it craps all-over most
other archivers, especially when using "Solid" compression
and the "Best" compression settings.
I have tried Soooooooo,ooooo,ooo many other programs
looking for anything better yet I am still using WinRAR.
(WinRAR can also make ZIP archives & extracts many types)
-Cheers.
>>> Bye.
>>> Jasen
cr88192
"BORG Xx(°..º)xX" <Xxo@oxX> wrote in message
news:44d4...@news.comindico.com.au...
"cr88192" <cr8...@NOSPAM.hotmail.com> wrote in message
news:af63$44d3f7dc$ca83a8d6$23...@saipan.com...
>
>> "BORG Xx(°..º)xX" Xx@xX> wrote in message
> news:44d3...@news.comindico.com.au...
>> "jasen" ja...@free.net.nz> wrote in message
>> news:da5.44d1a...@gonzo.homenet...
>>> On 2006-08-03, Paul Hemans <pauln.o...@laberg.com.au> wrote:
>>>> I have a large directory structure (1.2GB) multiple sub-directories and
>>>> some
>>>> fairly large files, the total file count is around 7,000. Can I use a
>>>> combination of TAR and GZIP to deal with this? Winzip ver 6 creates a
>>>> compressed file of 440MB but I can only access the first 360 files!
>>>
>>> we have .tar.gz files 2.5G in size with 75000 files...
>>> tar is industrial strength, but for your task zip should work too.
>>> I think winzip is ignoring some of your files.
>>>
>>> I've never been a fan of those GUI archive tools
>>
>> Before WinRAR I had always used LHA, then ARJ.
>>
>> WinRAR works great, both GUI & command line.
>> I gave up on Zip ages ago...
>>
>
> note, however, that rar is not free, nor is it an open format.
>
> tar, gzip, and zip, however, are all open formats, and there is a much
> larger selection of compressors and decompressors available.
<fake indent, lazy>
> There are FREE decompressors for most (pretty much all)
> types of OS (current & ancient eg: Amiga) & the
> decompression source & decompressors
> are freely available to DL on the www.rarlab.com website
> (and assorted P2P networks).
this does not change that it is not technically an open format however.
the existance of a few free tools does not necissarily mean it is an open
format.
there is a much wider selection of tools for zip, in fact, I have written
code for encoding/decoding zip files as well...
my main complaint about zip is that it is designed in a way that messes up
the possibility of implementing full read/write access, which required me to
make my own format for cases where this might be necessary (app usage of a
filesystem emulation layer).
if I were to revise the format, I would probably drop a few features which
are unlikely to be all that useful for my uses (fragmentation), and maybe
allow compression of the central directory (however, this would risk
reducing the practical limits of the format in the name of a potentially
minor reduction in total filesize).
it works though...
>> this also gives a sense of security for the future ability to easily
>> decompress these formats.
>>
>> or such...
>
> With freely available decompression source it will never die
> unless all the lights go out :) And it craps all-over most
> other archivers, especially when using "Solid" compression
> and the "Best" compression settings.
>
> I have tried Soooooooo,ooooo,ooo many other programs
> looking for anything better yet I am still using WinRAR.
> (WinRAR can also make ZIP archives & extracts many types)
>
> -Cheers.
winrar is itself, a program, not only that, one where people are expected to
pay for it.
the rar format itself, is a format, but it is effectively (at least
currently afaik) tied to the existance of winrar. even if freely available
decompressors exist, freely available compressors need to exist as well, and
the format needs to be maintained as an open standard.
zip, tar.gz, and tar.bz2 are imo safer bets for long term usability.
of course, tar is crufty on windows, making zip generally the best bet for
general uses (even if compression is not always all that great with zip...).
nearly anyone can write programs to encode or decode zip archives, and the
specifications are commonly available (even if, albeit, some newer features
make the format crufty, or may risk encroaching on existing patents...).
or such...
>>> Bye.
>>> Jasen
BORG Xx(°..º)xX
I wonder why that happens sometimes & not always. Weird.
>> There are FREE decompressors for most (pretty much all)
>> types of OS (current & ancient eg: Amiga) & the
>> decompression source & decompressors
>> are freely available to DL on the www.rarlab.com website
>> (and assorted P2P networks).
>
> this does not change that it is not technically an open format however.
> the existance of a few free tools does not necissarily mean it is an open
> format.
If you want to use their compression you pay a license fee I assume.
Not much of anything good and worthwhile is free these days.
They need to eat too :)
If you want to write something Free & Open that is as good or better
than WinRAR then please do. I might even use it if you succeed.
> there is a much wider selection of tools for zip, in fact, I have written
> code for encoding/decoding zip files as well...
Plenty of people like to re-invent the wheel. Good for learning I suppose.
Very few do better, like WinRAR has done. Those that do compress
better need enormous quantities of ram and a near-supercompter to
compress/decompress large amounts of data, and even then they
only very slightly beat WinRAR & take forever... + their features
usually SUCK.
> my main complaint about zip is that it is designed in a way that messes up
> the possibility of implementing full read/write access, which required me to
> make my own format for cases where this might be necessary (app usage of a
> filesystem emulation layer).
>
> if I were to revise the format, I would probably drop a few features which
> are unlikely to be all that useful for my uses (fragmentation), and maybe
> allow compression of the central directory (however, this would risk
> reducing the practical limits of the format in the name of a potentially
> minor reduction in total filesize).
>
> it works though...
Had you ever used Stacker 4.1? It did quite well for its time
with a few tweaks to the compression settings, even on OLD &
S L O W computers when on the maximum compression setting.
I still have it here somewhere - no source though, you'd have to
reverse engineer it to learn from it. It crapped all over the M$ toys.
>>> this also gives a sense of security for the future ability to easily
>>> decompress these formats.
>>>
>>> or such...
>>
>> With freely available decompression source it will never die
>> unless all the lights go out :) And it craps all-over most
>> other archivers, especially when using "Solid" compression
>> and the "Best" compression settings.
>>
>> I have tried Soooooooo,ooooo,ooo many other programs
>> looking for anything better yet I am still using WinRAR.
>> (WinRAR can also make ZIP archives & extracts many types)
>>
>> -Cheers.
>
> winrar is itself, a program, not only that, one where people are expected to
> pay for it.
Welcome to the present day :) I am a 'closet hippie' at heart so I do understand.
Nevertheless, if you want the best and it isn't free & open then you just have to
either pay for it or steal it or invent it yourself or con someone else into doing it
or use out-dated fossil software :) We need something new, capable & flexible
> the rar format itself, is a format, but it is effectively (at least
> currently afaik) tied to the existance of winrar. even if freely available
> decompressors exist, freely available compressors need to exist as well, and
> the format needs to be maintained as an open standard.
"needs" & Prefers are two different things ;)
I like FREE & Open Source too, yet if I have to pay for something better
then I will if the cost is not too great. Zip is just too lacking in compression,
amongst other things.
> zip, tar.gz, and tar.bz2 are imo safer bets for long term usability.
> of course, tar is crufty on windows, making zip generally the best bet for
> general uses (even if compression is not always all that great with zip...).
"compression is not always all that great with zip" is an understatement.
These days size (& compression ratios) does matter with the vast
amounts of files/data to be compressed & solid archives make
a BIG difference.
> nearly anyone can write programs to encode or decode zip archives, and the
> specifications are commonly available (even if, albeit, some newer features
> make the format crufty, or may risk encroaching on existing patents...).
>
> or such...
Write something new.
If it is better & free, has the right features plus usability
for anyone on any system then persist & it will take off,
as long as it decompresses other common formats too.
- Cheers & Get To Work ;)
>>>> Bye.
>>>> Jasen
cr88192
"BORG Xx(°..º)xX" <Xxo@oxX> wrote in message
dunno...
>> There are FREE decompressors for most (pretty much all)
>> types of OS (current & ancient eg: Amiga) & the
>> decompression source & decompressors
>> are freely available to DL on the www.rarlab.com website
>> (and assorted P2P networks).
>
> this does not change that it is not technically an open format however.
> the existance of a few free tools does not necissarily mean it is an open
> format.
> If you want to use their compression you pay a license fee I assume.
> Not much of anything good and worthwhile is free these days.
> They need to eat too :)
>
compression tools don't need to cost money, and many don't.
> If you want to write something Free & Open that is as good or better
> than WinRAR then please do. I might even use it if you succeed.
>
I don't need to.
ever try of 7zip?...
>
> > there is a much wider selection of tools for zip, in fact, I have
> > written
> > code for encoding/decoding zip files as well...
>
>
> Plenty of people like to re-invent the wheel. Good for learning I
> suppose.
> Very few do better, like WinRAR has done. Those that do compress
> better need enormous quantities of ram and a near-supercompter to
> compress/decompress large amounts of data, and even then they
> only very slightly beat WinRAR & take forever... + their features
> usually SUCK.
>
my code usually uses the zip format as-is.
the reason is partly because the format is common and good enough.
my apps though are generally not archivers, more I like packaging up my data
so I don't need a bunch of files lingering about in directories (and taking
up extra space on the hd).
>> my main complaint about zip is that it is designed in a way that messes
>> up
>> the possibility of implementing full read/write access, which required me
>> to
>> make my own format for cases where this might be necessary (app usage of
>> a
>> filesystem emulation layer).
>>
>> if I were to revise the format, I would probably drop a few features
>> which
>> are unlikely to be all that useful for my uses (fragmentation), and maybe
>> allow compression of the central directory (however, this would risk
>> reducing the practical limits of the format in the name of a potentially
>> minor reduction in total filesize).
>>
>> it works though...
>
>
> Had you ever used Stacker 4.1? It did quite well for its time
> with a few tweaks to the compression settings, even on OLD &
> S L O W computers when on the maximum compression setting.
> I still have it here somewhere - no source though, you'd have to
> reverse engineer it to learn from it. It crapped all over the M$ toys.
>
couldn't find any real documentation on the format, so it didn't do me any
real good.
I am using the deflate algo (the same algo used in zip, gzip, and png).
had also messed some with a BWT based algo, but BWT is slower.
>>> this also gives a sense of security for the future ability to easily
>>> decompress these formats.
>>>
>>> or such...
>>
>> With freely available decompression source it will never die
>> unless all the lights go out :) And it craps all-over most
>> other archivers, especially when using "Solid" compression
>> and the "Best" compression settings.
>>
>> I have tried Soooooooo,ooooo,ooo many other programs
>> looking for anything better yet I am still using WinRAR.
>> (WinRAR can also make ZIP archives & extracts many types)
>>
>> -Cheers.
>
>> winrar is itself, a program, not only that, one where people are
>> >expected to
>> pay for it.
>
>
> Welcome to the present day :) I am a 'closet hippie' at heart so I do
> understand.
> Nevertheless, if you want the best and it isn't free & open then you
> just have to
> either pay for it or steal it or invent it yourself or con someone else
> into doing it
> or use out-dated fossil software :) We need something new, capable &
> flexible
>
what exists works.
there is 7zip, and 7zip's speed and ratios are competitive with winrar.
zip is much faster, and much more standardized.
>> the rar format itself, is a format, but it is effectively (at least
>> currently afaik) tied to the existance of winrar. even if freely
>> available
>> decompressors exist, freely available compressors need to exist as well,
>> and
>> the format needs to be maintained as an open standard.
>
> "needs" & Prefers are two different things ;)
> I like FREE & Open Source too, yet if I have to pay for something better
> then I will if the cost is not too great. Zip is just too lacking in
> compression,
> amongst other things.
the difference in ratios is minor most of the time.
that is where tar.bz2 wins out, it compresses better.
>> zip, tar.gz, and tar.bz2 are imo safer bets for long term usability.
>> of course, tar is crufty on windows, making zip generally the best bet
>for
>> general uses (even if compression is not always all that great with
>zip...).
>
>
> "compression is not always all that great with zip" is an
> understatement.
> These days size (& compression ratios) does matter with the vast
> amounts of files/data to be compressed & solid archives make
> a BIG difference.
so, then, why not tar.bz2, or 7zip?...
tar is both solid, and bzip2 compresses much better than zip (or gzip).
>> nearly anyone can write programs to encode or decode zip archives, and
>> the
>> specifications are commonly available (even if, albeit, some newer
>> features
>> make the format crufty, or may risk encroaching on existing patents...).
>>
>> or such...
>
>
> Write something new.
> If it is better & free, has the right features plus usability
> for anyone on any system then persist & it will take off,
> as long as it decompresses other common formats too.
>
> - Cheers & Get To Work ;)
>
mostly, I don't write archivers.
my main project is more likely to end up being a 3d modeling/rendering
package (if anything).
the specialized format I implemented, however, is a little odd. it combined
aspects from things like zip, wad2, pack, and ntfs...
as implemented, it more or less has a filename limit of 32 chars, but as it
is intended to be used, this shouldn't be too big of a problem. an idea here
is that the format is multifoked, so a longer name could be embedded in a
fork...
or such...
>>>> Bye.
>>>> Jasen