Encryption / AesCryptoStreamProvider and Initialization Vector handling

[Mhano Harkness]

I'm looking for best practices on encrypting messages in mass transit.

In looking at the AesCryptoStreamProvider it appears there is no way to communicate the initialization vector from the publisher to the consumer other than an extrernal key issuer that issues per stream key/iv pairs. The IV is obtained from the key provider not the stream during decryption. Given this is on the serializer, I gather this is done per message?

https://github.com/MassTransit/MassTransit/blob/699b167645b7c6f2df10483378420f586b358dc4/src/MassTransit/Serialization/AesCryptoStreamProvider.cs

Stream ICryptoStreamProvider.GetDecryptStream(Stream stream, string keyId, CryptoStreamMode streamMode)

So (if my understanding is correct), each publish of a message requires a stateful external key provider that issues key/IV pairs and stores the IV for retrieval later by one or more consumers.

It is common practice to send the IV (not encrypted) at the beginning of the stream / message specific encrypted payload as the IV has randomness but not secrecy requirements (encrypting the IV with key/null-IV and then including it at the front of the stream encrypted with key/IV is not an option as it drastically reduces the security ~ long story I won't go into).

https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation

An initialization vector (IV) or starting variable (SV)^[5] is a block of bits that is used by several modes to randomize the encryption and hence to produce distinct ciphertexts even if the same plaintext is encrypted multiple times, without the need for a slower re-keying process.^[6]

An initialization vector has different security requirements than a key, so the IV usually does not need to be secret. However, in most cases, it is important that an initialization vector is never reused under the same key. For CBC and CFB, reusing an IV leaks some information about the first block of plaintext, and about any common prefix shared by the two messages.

Is there a reason the IV is not transmitted with the stream?

The interface/classes provided would appear to encourage developers to create a static and re-used IV. If my understanding is correct and the serialization is per-message, this is weak encryption as the IV is re-used many times.

It is possible to encode the IV along with the key identifier (and add that to the encryptionkeyid header) or pehaps add an extra header for the IV?

// on the publisher

busConfigCallback.UseEncryptedSerializer(new AesCryptoStreamProvider(new MyKeyProvider(), MyKeyProvider.DefaultKey));

publishCallback.Headers.Set(EncryptedMessageSerializer.EncryptionKeyHeader, MyKeyProvider.GetIdWithIV(someConfig)); 

// ISymmetricKeyProvider.TryGetKey would take requests for keys, split the keyid and iv out of the id parameter

// on consumer

busConfigCallback.UseEncryptedSerializer(new AesCryptoStreamProvider(new MyKeyProvider(), MyKeyProvider.DefaultKey));

// ISymmetricKeyProvider.TryGetKey would take requests for keys, split the keyid and iv out of the id parameter

Alternatively AesCryptoStreamProvider is not overly complex and an implementing an alternative that supports serializing the IV without overloading the keyid string might be the best appraoch?

Any other suggestions? Does any sample code for aws kms or azure keyvault & masstransit's AesCryptoStreamProvider exist?

[Mhano Harkness]

Interestingly...

PreSharedKeyEncryptedMessageSerializer from MassTransit V2 transmitted the IV with the message.

https://github.com/MassTransit/MassTransit/blob/v2-develop/src/MassTransit/Serialization/PreSharedKeyEncryptedMessageSerializer.cs

Is there a reason anyone can recall this is no longer done?

By using the interface ISymmetricKeyProvider, AesCryptoStreamProvider now really requires a key provider with out of band storage / transmission of the IV.

• Publisher
• Setup bus callback => supply key provider
• Setup publish callback => add encryption key header with key id value from key provider call
• key provider must out of band store the key id + IV (or be able to deterministically generate it later - making it less random and less secure)
• Consumers
• Setup bus callback => supply key provider
• Consume message => ask key provider for key based on id in header (this is where ISymmetricKeyProvider assumes the iv has been stored out of band, as it needs to be able to provide the key and iv and support multiple retrievals).

[Chris Patterson]

I don't see any reason not to just fix this and send the IV along with the message in a transport header. The symmetric key provider would need to provide a unique IV for each call though. 

We use keyvault so I'm so getting a sample into the codebase someplace should by easy enough. It's a separate set of references though so it won't be in the MassTransit service Bus assembly. 

__
Chris Patterson

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/cfcecde5-dba8-465d-90c5-e2ccda126ae3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mhano Harkness]

Thanks Chris,

You suggestion a MT code change to fix?

My workaround to simply overload the encryption key id to also transmit the IV worked, but breaks the whole concept of a default key that the interfaces have:

cb.Headers.Set(EncryptedMessageSerializer.EncryptionKeyHeader, publisherCfg.GetEncryptionKeyNameAndNewIv());

Thus when the (de) serializer finds the AesCrypto stream and key provider attached it asking for the key by id actually gives it the key provider the IV allowing it to return the key + message specific iv.

Cheers,

Mhano

[Chris Patterson]

Did you enter an issue on https://github.com/MassTransit/MassTransit?

--

You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.
To post to this group, send email to masstransit-discuss@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/9b843e58-7dba-4595-abcf-39994a582c8b%40googlegroups.com.

[Mhano Harkness]

Not yet, working on an alternate crypto stream provider... (class AesCryptoStreamProviderWithRandomIV : ICryptoStreamProvider)

On Sunday, February 12, 2017 at 2:44:08 AM UTC+11, Chris Patterson wrote:

Did you enter an issue on https://github.com/MassTransit/MassTransit?

On Mon, Feb 6, 2017 at 2:50 PM, Mhano Harkness <mhano....@gmail.com> wrote:

Thanks Chris,

You suggestion a MT code change to fix?

My workaround to simply overload the encryption key id to also transmit the IV worked, but breaks the whole concept of a default key that the interfaces have:

cb.Headers.Set(EncryptedMessageSerializer.EncryptionKeyHeader, publisherCfg.GetEncryptionKeyNameAndNewIv());

Thus when the (de) serializer finds the AesCrypto stream and key provider attached it asking for the key by id actually gives it the key provider the IV allowing it to return the key + message specific iv.

Cheers,

Mhano

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

[Tim Thompson]

G'day Mhano, how'd you get on with this?

We're trying to get started with encrypted messages over the bus (Healthcare environment) but the lack of worked examples and your concerns about the fixed IV are making for slow going :(

If I can help at all let me know.

Cheers,

Tim

[Mhano Harkness]

Our solution in the end was simply to implement our own ICryptoStreamProvider (back by an aes crypto stream and IV generation) - this takes a key provider interface that doesn't supply an IV (or expect to have anything to do with the IV).

/// <summary>
/// A crypto stream provider that encrypts the contents using AES encryption
/// with a random IV for every message.
/// </summary>
public class AesCryptoStreamProviderWithRandomIV : ICryptoStreamProvider, IProbeSite
{
    private readonly IKeyProvider _keyProvider;
    private readonly PaddingMode _paddingMode;


    public AesCryptoStreamProviderWithRandomIV(IKeyProvider keyProvider, PaddingMode paddingMode = PaddingMode.PKCS7)
    {
        _paddingMode = paddingMode;
        _keyProvider = keyProvider;
    }


    public Stream GetEncryptStream(Stream stream, string keyId, CryptoStreamMode streamMode)
    {
        if (stream == null)
            throw new ArgumentNullException(nameof(stream));


        if (string.IsNullOrEmpty(keyId))
            throw new ArgumentException("Encryption Key not specified", nameof(keyId));


        byte[] key;
        if (!_keyProvider.TryGetKey(keyId, out key))
            throw new SerializationException("Encryption Key not found: " + keyId);


        // Generate random IV to be used for this message.
        // (AesCryptoServiceProvider generates a new one each time if not set)


        using (var provider = new AesCryptoServiceProvider
        {
            Padding = _paddingMode,
            Key = key // Supply Key, but not IV.
        })
        {
            ICryptoTransform encryptor = provider.CreateEncryptor();


            // Write the IV to the begining of the stream.
            WriteIvToStream(stream, provider.IV);


            return new DisposingCryptoStream(stream, encryptor, streamMode);
        }
    }


    public Stream GetDecryptStream(Stream stream, string keyId, CryptoStreamMode streamMode)
    {
        if (stream == null)
            throw new ArgumentNullException(nameof(stream));


        if (string.IsNullOrEmpty(keyId))
            throw new ArgumentException("Encryption Key not specified", nameof(keyId));


        byte[] key;
        if (!_keyProvider.TryGetKey(keyId, out key))
            throw new SerializationException("Encryption Key not found: " + keyId);




        using (var provider = new AesCryptoServiceProvider
        {
            Padding = _paddingMode,
            Key = key, // Supply Key.
        })
        {
            // Supply the IV from the begining of the stream.
            provider.IV = ReadIvFromStream(stream, GetIvLength(provider));


            ICryptoTransform encryptor = provider.CreateDecryptor();


            return new DisposingCryptoStream(stream, encryptor, streamMode);
        }
    }


    /// <summary>
    /// Returns the IV length based on the algorithm's block size.
    /// </summary>
    /// <param name="provider"></param>
    /// <returns></returns>
    private static int GetIvLength(SymmetricAlgorithm provider)
    {
        return provider.BlockSize / 8;
    }


    /// <summary>
    /// Write IV to the begining of the stream.
    /// </summary>
    /// <param name="stream"></param>
    /// <param name="iv"></param>
    public void WriteIvToStream(Stream stream, byte[] iv)
    {
        // Leave the stream open when we dispose the writer
        using (var writer = new BinaryWriter(stream, new UTF8Encoding(), true))
        {
            writer.Write(iv);
        }
    }


    /// <summary>
    /// Read IV from the begining of the stream.
    /// </summary>
    /// <param name="stream"></param>
    /// <param name="ivLength"></param>
    /// <returns></returns>
    public byte[] ReadIvFromStream(Stream stream, int ivLength)
    {
        // Leave the stream open when we dispose the reader
        using (var reader = new BinaryReader(stream, new UTF8Encoding(), true))
        {
            return reader.ReadBytes(ivLength);
        }
    }


    public void Probe(ProbeContext context)
    {
        context.Add("paddingMode", _paddingMode.ToString());
    }




    // Copied directly out of the Mass Transit code.
    private class DisposingCryptoStream :
        CryptoStream
    {
        Stream _stream;
        ICryptoTransform _transform;


        public DisposingCryptoStream(Stream stream, ICryptoTransform transform, CryptoStreamMode mode)
            : base(stream, transform, mode)
        {
            _stream = stream;
            _transform = transform;
        }


        protected override void Dispose(bool disposing)
        {
            if (!disposing)
                return;


            base.Dispose(true);


            if (_stream != null)
            {
                _stream.Dispose();
                _stream = null;
            }


            if (_transform != null)
            {
                _transform.Dispose();
                _transform = null;
            }
        }
    }
}


public interface IKeyProvider
{
    bool TryGetKey(string id, out byte[] key);
}