Transferring hash state

33 views
Skip to first unread message

Sands, Daniel N.

unread,
Oct 24, 2024, 5:24:32 PM10/24/24
to openss...@openssl.org

We have an application that performs hashing across multiple hosts in a kind of round-robin form.  In the past, we could reach into whatever hash we were using and pull its state, and write it into the hash state on the next host to continue processing.  With the new opaque hash structures, that ability seems to be lost.  Is that the case, or is it still somehow possible?

Viktor Dukhovni

unread,
Oct 24, 2024, 10:32:53 PM10/24/24
to openss...@openssl.org
If you're using, e.g., SHA256, and willing to bypass the EVP layer
abstractions, then the (deprecated, but still available in OpenSSL 3.x)
functions that take a concrete SHA256_CTX admit serialising the context,
because that structure is not opaque.

<openssl/sha.h>

typedef struct SHA256state_st {
SHA_LONG h[8];
SHA_LONG Nl, Nh;
SHA_LONG data[SHA_LBLOCK];
unsigned int num, md_len;
} SHA256_CTX;

OSSL_DEPRECATEDIN_3_0 int SHA256_Init(SHA256_CTX *c);
OSSL_DEPRECATEDIN_3_0 int SHA256_Update(SHA256_CTX *c,
OSSL_DEPRECATEDIN_3_0 int SHA256_Final(unsigned char *md, SHA256_CTX *c);

If you want "algorithm agility", then unless I'm mistaken, I'm afraid
that indeed the abstract interface currently lacks a way to serialise
and deserialise the internal state, to allow to move "out of process".
If some day implemented, This would surely require new "provider"
mechanisms, and would be available only for provider/algorithm
combinations that support such serialisation. Right now that set of
provider/algorithm combinations is empty.

--
Viktor.

Tomas Mraz

unread,
Oct 25, 2024, 3:40:52 AM10/25/24
to openss...@openssl.org
We have this issue discussing this RFE:

https://github.com/openssl/openssl/issues/14222


Tomas Mraz, OpenSSL
--
Tomáš Mráz, OpenSSL

Sands, Daniel N.

unread,
Oct 25, 2024, 3:23:55 PM10/25/24
to openss...@openssl.org
I see.  Well, I'll comment here since I don't have a Github presence yet.

I have no issues with using provider-specific param names.  But as far as the OpenSSL provider (fips=yes or fips=no), I'd say that most of the digest algorithms are pretty stable by now, especially the old fogies.  It would be useful to keep providing the OpenSSL internal (currently deprecated) state structures, but only as a method of interpreting the internal state after calling for a pointer to it.  So as long as I know which digest I'm using, we can still interpret and manually serialize the state data as we have already been doing, once we get a pointer to it.  Just taking a glance at the 3.0.1 source code, I see that MD5, SHAx, and Blake2 seem to still use the deprecated interfaces under the covers.
I suppose MACs can be more complicated, but I'm only asking for digests.

From: Tomas Mraz <to...@openssl.org>
Sent: Friday, October 25, 2024 1:40 AM
To: openss...@openssl.org <openss...@openssl.org>
Subject: [EXTERNAL] Re: Transferring hash state
 
We have this issue discussing this RFE:

Sands, Daniel N.

unread,
Oct 1, 2025, 5:59:49 PM (14 hours ago) Oct 1
to openss...@openssl.org

The docs state this:

EVP_DecryptInit_ex2(), EVP_DecryptInit_ex(), EVP_DecryptUpdate() and EVP_DecryptFinal_ex()

 

These functions are the corresponding decryption operations. EVP_DecryptFinal() will return an error code if padding is enabled and the final block is not correctly formatted. The parameters and restrictions are identical to the encryption operations except that if padding is enabled the decrypted data buffer out passed to EVP_DecryptUpdate() should have sufficient room for (inl + cipher_block_size) bytes unless the cipher block size is 1 in which case inl bytes is sufficient.

Why does the decrypted buffer need to be “inl + cipher block size”?  The ciphertext will be padded to a multiple of cipher block size, but the amount of decrypted text will be less than the amount of ciphertext given.

In addition, there seems to be a restriction which is NOT documented here (or is it a bug?)  I’ll put together a simple example:

uint8_t *buf; <function arg, buffer the size of the expected plaintext>

size_t     bufsize; <function arg, expected size>

 

uint8_t pad[16];

 

<Read the ciphertext into buf and the final block into pad.  Do the usual setup for AES-128 CBC.>

 

int len;

EVP_DecryptUpdate(ctx, buf, &len, buf, bufsize & ~15);

 

int padlen;

EVP_DecryptUpdate(ctx, pad, &padlen, pad, 16);

 

int finallen;

EVP_DecryptFinal_ex(ctx, <any destination buffer>, &finallen) will fail.  This seems to be because pad was modified by in-place decryption.

 

 

If I instead do the following (very slight tweak here):

 

int len;

EVP_DecryptUpdate(ctx, buf, &len, buf, bufsize & ~15);

 

int padlen;

EVP_DecryptUpdate(ctx, buf + len, &padlen, pad, 16);

 

int finallen;

EVP_DecryptFinal_ex(ctx, <any destination buffer including pad>, &finallen) will succeed.  Oddly enough, it will succeed here even if pad was erased just before this call.

 

<Assert that len + padlen + finallen equals the buffer size>

 

memcpy(buf + len + padlen, pad, finallen); // Assumes that pad was the destination for EVP_DecryptFinal_ex

 


Final question:  Is the latter sequence considered couth for reading a stream into a buffer that is only the expected decrypted size, or does this depend on functionality that might change?

 

Reply all
Reply to author
Forward
0 new messages