Data Compression versus Encryption of Paradox Tables
Joeldo Holanda
Hello everyone,
I'm working with a Paradox Table and I want to have it compressed for
backup purposes (Compression is being done by PKLITE Zip Utilities).
However, if I protect the table using password encryption I cannot
perform compression with a compress rate bigger than 10%. Without the
password, the table can be compressed at rates up to 80%.
I imagine that it happens because the password widens the ASCII range
of the file by scrambling the characters, therefore leaving less grounds
for a higher compression.
The table is too big, so I don't think it's feasible to decrypt the
whole table before performing the backup. Is there any way to
simultaneously compress and encrypt a Paradox Table without the need of
using an external program/routine such as PKLITE? Something like
reading/decompressing/decrypting and writing/compressing/encrypting in a
record level? It would slow down the database handling, but would solve
this problem. Other solution would be the use of an encryption routine
that doesn't degrade the compress rates. It would be a better solution
because the database handling would be as fast as it is now. Is there
such a routine though, and a way to implement it?
The database is being handled by Delphi code.
Many thanks for any suggestion.
Joeldo
Please make sure to remove the underscore from my e-mail address in
case you wish to reply me via e-mail. I'm sorry, i had to add it to
avoid automatic spamming. :)
Steven Green
Joeldo: your encryption options are exactly as
you suggested.. either encrypt a table or zip
a table.. that's one of the reasons that we
usually don't encrypt tables.. plus there is
the possibility of additional problems if a
table becomes damaged.. and it's just a general
PITA.. <smile>.. if you maintain absolute
control of the tables, there is no need for a
password at that level..
and besides.. there are programs out there that
will easily determine the password.. given the
time, *you* could write the code that does it,
yourself..
--
Steve Green - Diamond software Group - Waldorf Maryland USA
CIS:71333...@compuserve.com - A member of Borland's TeamB
and Corel's CTech tech support staffs but I don't represent
Borland International or Corel in any official capacity.
Greg Williamson
In article <33393E18.5763@PoBo_x.Com>,
Joeldo Holanda <Jo_...@PoBox.Com> wrote:
>Hello everyone,
>
> I'm working with a Paradox Table and I want to have it compressed for
>backup purposes (Compression is being done by PKLITE Zip Utilities).
>However, if I protect the table using password encryption I cannot
>perform compression with a compress rate bigger than 10%. Without the
>password, the table can be compressed at rates up to 80%.
<snip>
Compression works by finding blocks of matching characters that can
be represented by a shorter "token" -- it depends on lots of redundancy;
files with no patterns will not compress well. Encryption does its best
to destroy patterns (at least anything more complicated than a simple
Ceaser [substitute one letter for another] cipher does). Compression
will yield poor results on a encrypted data if the encryption algorithm
is decent.
> The table is too big, so I don't think it's feasible to decrypt the
>whole table before performing the backup. Is there any way to
>simultaneously compress and encrypt a Paradox Table without the need of
>using an external program/routine such as PKLITE? Something like
>reading/decompressing/decrypting and writing/compressing/encrypting in a
>record level? It would slow down the database handling, but would solve
>this problem. Other solution would be the use of an encryption routine
>that doesn't degrade the compress rates. It would be a better solution
>because the database handling would be as fast as it is now. Is there
>such a routine though, and a way to implement it?
<snip>
If you only need it for backup, can you extract data, compress it and
then encrypt that? If you have lots of blobs this may be a real problem
so you'd want to encrypt the tables themselves ... the record level
read/decompress/decrypt will be slower but at least it protects data...
I admit that I don't see any gracefull ways to go ...
Greg Williamson
---
TRW Financial Systems doesn't speak for me, and I don't speak for them.
Herb Sutter
g...@mailhub.tfs.com (Greg Williamson) wrote:
>In article <33393E18.5763@PoBo_x.Com>,
>Joeldo Holanda <Jo_...@PoBox.Com> wrote:
>>However, if I protect the table using password encryption I cannot
>>perform compression with a compress rate bigger than 10%. Without the
>>password, the table can be compressed at rates up to 80%.
>
>be represented by a shorter "token" -- it depends on lots of redundancy;
>files with no patterns will not compress well. Encryption does its best
>to destroy patterns (at least anything more complicated than a simple
>Ceaser [substitute one letter for another] cipher does). Compression
>will yield poor results on a encrypted data if the encryption algorithm
>is decent.
You should be worried if encrypted files are still compressible by as much as
10%. It means that your encryption algorithm is leaking a lot of information
about the plaintext, and is therefore insecure (although it's likely still
fine for casual security). Encrypted data is supposed to look as random as
possible, and should therefore be incompressible.
BTW, running encrypted data through a compression program is a common way to
quickly identify bad crypto. If the ciphertext is compressible, you toss the
crypto program that produced it. If it's not compressible, you continue with
other further testing (since there may still be other problems and
weaknesses), but at least just doing the compressibility test will usually
weed out 80% of bad crypto right away.
By way of comparison, try taking that same file and encrypting it with PGP (a
popular strong encryption program available on all sorts of websites... do a
web search and you'll find it). I'll bet you can't compress the resulting
ciphertext, because PGP uses Blowfish and other strong ciphers.
---
Herb Sutter (mailto:he...@cntc.com)
Current Network Technologies Corp. (http://www.cntc.com)
2695 North Sheridan Way, Suite 150, Mississauga ON Canada L5K 2N6
Tel 416-805-9088 Fax 905-822-3824