Need a better way to Compress and Encrypt

[Graham Hemingway]

I have a program that compresses and encrypts thousands of individual chunks of data.  Each chunk must be compressed and encrypted separately.  Everything works, but there must be a more optimal way to go about it than I am currently.  Here is the code I execute for each chunk to compress it.

// Create the compressor and load it up

CryptoPP::ZlibCompressor compressor;

compressor.Put( buffer, sizeB );

compressor.MessageEnd();

// Get the new size

newSizeB = compressor.MaxRetrievable();

// Get the data

compressor.Get( newBuffer, newSizeB); 

As you can see, I allocate a new compressor for each chunk.  This is done in a big loop, so I figured that I could "new" a compressor before the loop to save on the cost of creating the compressor for each chunk.  But, this does not work.  The first chunk that passes through the compressor works fine, but I always get a MaxRetrievable of 0 for all later chunks.  I have tried putting Flush(true) in all over the place but nothing seems to work yet.  The same issue holds for the encryption pass.  Any ideas?

One other thing.  I am working on OS X 10.6 with XCode 3.2.2.  I compiled Crypto++ as a static lib and everything went well.  When I compile my app I get two warnings from the included Crypto++ header files:

../../Include/cryptopp/algparam.h:26:0 ../../Include/cryptopp/algparam.h:26: warning: unused variable 'cryptopp_assert_26'

from line 26 of algparam.h

../../Include/cryptopp/algparam.h:322:0 ../../Include/cryptopp/algparam.h:322: warning: unused variable 'p'

from line 322 of algparam.h

I have seen some other posts along these lines.  What should I change to get rid of these?

Thanks,

   Graham

[Jeffrey Walton]

On Jan 19, 10:48 am, Graham Hemingway <graham.heming...@gmail.com>
wrote:

> I have a program that compresses and encrypts thousands of individual chunks
> of data.  Each chunk must be compressed and encrypted separately.
>  Everything works, but there must be a more optimal way to go about it than
> I am currently.  Here is the code I execute for each chunk to compress it.
>
> // Create the compressor and load it up
>
> CryptoPP::ZlibCompressor compressor;
>
> compressor.Put( buffer, sizeB );
>
> compressor.MessageEnd();
>
> // Get the new size
>
> newSizeB = compressor.MaxRetrievable();
>
> // Get the data
>
> compressor.Get( newBuffer, newSizeB);
>
> As you can see, I allocate a new compressor for each chunk.  This is done in
> a big loop, so I figured that I could "new" a compressor before the loop to
> save on the cost of creating the compressor for each chunk.  But, this does
> not work.  The first chunk that passes through the compressor works fine,
> but I always get a MaxRetrievable of 0 for all later chunks.  I have tried
> putting Flush(true) in all over the place but nothing seems to work yet.
>  The same issue holds for the encryption pass.  Any ideas?

Use filters. See for example, http://www.cryptopp.com/wiki/Filter.

> One other thing.  I am working on OS X 10.6 with XCode 3.2.2.  I compiled
> Crypto++ as a static lib and everything went well.  When I compile my app I
> get two warnings from the included Crypto++ header files:
>
> ../../Include/cryptopp/algparam.h:26:0 ../../Include/cryptopp/algparam.h:26:
> warning: unused variable 'cryptopp_assert_26'
>
> from line 26 of algparam.h
>
> ../../Include/cryptopp/algparam.h:322:0
> ../../Include/cryptopp/algparam.h:322: warning: unused variable 'p'
>
> from line 322 of algparam.h
>
> I have seen some other posts along these lines.  What should I change to get
> rid of these?

http://www.cryptopp.com/wiki/Ios is written for iOS, but it should
apply to Mac OS X also:

Another setting which is useful, but not required, is
-Wall -Wno-unused -Wno-unknown-pragmas under
GCC 4.2 Warnings, Other Warning Flags. Be careful
of -Wextra since GCC 4.2 will flag signed/unsigned
comparisons due to C++ templates. -Wno-type-limits
will suppress the spurious template warnings, but the
option is only available in GCC 4.3 and above. See
Missing "warning: comparison is always false due to
limited range of data type".

Jeff

[Mouse]

Jeffrey,

I'm not sure how Filter functions - but I thought its internal buffering (and therefore what it passes to the encryter & when) is not related to or dependent on the chunks that it is fed by the caller?

Also, it was not clear to me whether the OP needed each chunk to be encrypted independently - aka with a different key?

Or am I barking up the wrong tree?

--
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-user...@googlegroups.com.
More information about Crypto++ and this group is available at http://www.cryptopp.com.

--
Regards,
Uri