Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
My own encryption program
41 views
Skip to first unread message
Frederick Gotham
Mar 18, 2020, 1:59:58 PM3/18/20
to
Yesterday I started writing my own encryption program. With all this talk of so-called 'quantum' computing and the possibility that our best algorithms might become brute-forceable, I've decided to start writing something new.
Having worked developing a Linux embedded device with a 32-Bit ARM processor, very limited hard disk space and very limited RAM, I have seen the true convenience and power of programs that are fully-functional when reading from stdin. 'tar' and 'openssl' are good examples as they don't seek on their input even when doing complex tasks.
On one of the devices I'm developing, tar is used to exract a file to stdout, where openssl then decrypts it to stdout, where tar then extracts another inner file. Because of very limited hard disk space and RAM, this wouldn't be possible without the piping capabilities of tar and openssl.
My encryption program will encrypt blocks of 16 bytes at a time, and the scheme will be similar to CBC, however I'll be doing something new which I haven't seen done before (at least not publicly).
One thing I'm taking into account is that I want my encryption and decryption to be /at least/ a third as fast as the mainstream algorithms (Twofish, AES, 3DES). So if AES can encrypt something in 5 minutes, I want mine to be able to do it within 15 minutes.
If I'm going to be encrypting big files then I will want to take the hash of the input as I'm processing it, so that you can check after the encryption procedure that the file wasn't corrupt at some point.
Where possible I will use multiple threads on multiple CPU cores. This actually isn't possible with CBC, but if my idea works as I think it will, then parallel encryption will be possible while getting around the problem of discernable patterns (i.e. the 'Bitmap' problem in encryption).
In 'main', I immediately start a second thread for performing the hashing of the input. This 'hasher' thread will infinitely call the 'consume_all' method on an object of type 'boost::lockfree::spsc_queue'.
The main thread will infinitely read from stdin, and will push 16-byte-data-chunks into the aforementioned spsc_queue.
So, in the main thread, data will be repeatedly read in 16-byte chunks from stdin, and each chunk will be passed imediately to the hasher thread for asychronous processing.
The main thread, working in parallel with the hasher thread, will then encrypt the data and spit it out to stdout. To avoid discernable patterns in the output, this is where the CBC scheme is normally used -- however the problem here is that CBC can only be done by one thread, it cannot be parallelised.
I have two new ideas: One that will allow something similar to CBC that can be done in parallel without creating a pattern, and Another idea that will make it difficult (if not impossible) to perform a brute-force even if a technique is devised of brute-forcing the likes of AES, Twofish, 3DES.
Does anyone have any thoughts on this?
Is spsc_queue suitable for doing the hashing in parallel with the encryption?
Having worked developing a Linux embedded device with a 32-Bit ARM processor, very limited hard disk space and very limited RAM, I have seen the true convenience and power of programs that are fully-functional when reading from stdin. 'tar' and 'openssl' are good examples as they don't seek on their input even when doing complex tasks.
On one of the devices I'm developing, tar is used to exract a file to stdout, where openssl then decrypts it to stdout, where tar then extracts another inner file. Because of very limited hard disk space and RAM, this wouldn't be possible without the piping capabilities of tar and openssl.
My encryption program will encrypt blocks of 16 bytes at a time, and the scheme will be similar to CBC, however I'll be doing something new which I haven't seen done before (at least not publicly).
One thing I'm taking into account is that I want my encryption and decryption to be /at least/ a third as fast as the mainstream algorithms (Twofish, AES, 3DES). So if AES can encrypt something in 5 minutes, I want mine to be able to do it within 15 minutes.
If I'm going to be encrypting big files then I will want to take the hash of the input as I'm processing it, so that you can check after the encryption procedure that the file wasn't corrupt at some point.
Where possible I will use multiple threads on multiple CPU cores. This actually isn't possible with CBC, but if my idea works as I think it will, then parallel encryption will be possible while getting around the problem of discernable patterns (i.e. the 'Bitmap' problem in encryption).
In 'main', I immediately start a second thread for performing the hashing of the input. This 'hasher' thread will infinitely call the 'consume_all' method on an object of type 'boost::lockfree::spsc_queue'.
The main thread will infinitely read from stdin, and will push 16-byte-data-chunks into the aforementioned spsc_queue.
So, in the main thread, data will be repeatedly read in 16-byte chunks from stdin, and each chunk will be passed imediately to the hasher thread for asychronous processing.
The main thread, working in parallel with the hasher thread, will then encrypt the data and spit it out to stdout. To avoid discernable patterns in the output, this is where the CBC scheme is normally used -- however the problem here is that CBC can only be done by one thread, it cannot be parallelised.
I have two new ideas: One that will allow something similar to CBC that can be done in parallel without creating a pattern, and Another idea that will make it difficult (if not impossible) to perform a brute-force even if a technique is devised of brute-forcing the likes of AES, Twofish, 3DES.
Does anyone have any thoughts on this?
Is spsc_queue suitable for doing the hashing in parallel with the encryption?
Paavo Helde
Mar 18, 2020, 3:31:41 PM3/18/20
to
On 18.03.2020 19:59, Frederick Gotham wrote:
> Yesterday I started writing my own encryption program. With all this talk of so-called 'quantum' computing and the possibility that our best algorithms might become brute-forceable, I've decided to start writing something new.
Just curious, what makes you think you can beat the professional
> Yesterday I started writing my own encryption program. With all this talk of so-called 'quantum' computing and the possibility that our best algorithms might become brute-forceable, I've decided to start writing something new.
cryptographers who have already worked on this exact problem for at
least a decade (and have already produced some results like elliptic
curves)?
> Having worked developing a Linux embedded device with a 32-Bit ARM processor, very limited hard disk space and very limited RAM, I have seen the true convenience and power of programs that are fully-functional when reading from stdin. 'tar' and 'openssl' are good examples as they don't seek on their input even when doing complex tasks.
>
> On one of the devices I'm developing, tar is used to exract a file to stdout, where openssl then decrypts it to stdout, where tar then extracts another inner file. Because of very limited hard disk space and RAM, this wouldn't be possible without the piping capabilities of tar and openssl.
support piping will somehow make the algorithm harder to crack for a $$$
quantum computer?
> My encryption program will encrypt blocks of 16 bytes at a time, and the scheme will be similar to CBC, however I'll be doing something new which I haven't seen done before (at least not publicly).
considered quite safe against quantum computers, one just needs to
increase the key sizes. It is the public key algorithms which are under
the threat.
IOW, you are trying to solve a non-problem while having no
qualifications for doing that. Seems a good match to me!
Frederick Gotham
Mar 18, 2020, 4:23:32 PM3/18/20
to
Pavo wrote:
> Just curious, what makes you think you
> can beat the professional cryptographers
> who have already worked on this exact
> problem for at least a decade (and have
> already produced some results like
> elliptic curves)?
<snip>
> Just curious, what makes you think you
> can beat the professional cryptographers
> who have already worked on this exact
> problem for at least a decade (and have
> already produced some results like
> elliptic curves)?
> Do you indeed believe that the additional
> constraint on the algorithm to
support
> piping will somehow make the algorithm
> harder to crack for a $$$
quantum computer?
<snip>
> constraint on the algorithm to
support
> piping will somehow make the algorithm
> harder to crack for a $$$
quantum computer?
> If I understand correctly, CBC is a symmetric
> cipher; these are considered quite safe
> against quantum computers, one just needs to
> increase the key sizes. It is the public
> key algorithms which are under
the threat.
<snip>
> cipher; these are considered quite safe
> against quantum computers, one just needs to
> increase the key sizes. It is the public
> key algorithms which are under
the threat.
> IOW, you are trying to solve a non-problem
> while having no qualifications for
> doing that. Seems a good match to me
I encourage independent ideas in myself and also in others. I think it's okay to compare apples and oranges, and I think it's okay to re-invent the wheel.
> while having no qualifications for
> doing that. Seems a good match to me
I really think it's a bad idea to discourage a person from doing something simply because somebody else did something similar, or because another person (or a group of people) put more work into a similar idea and produced something good.
Irrespective of what any group of people did a thousand years ago, or ten years ago, I'm writing my own encryption program.
In my day job I do some work on encrypted communication, for example recently I implemented the OSDP protocol in fully-portable C++98 code (even accounting for 9-bit bytes and padding bytes within integer types).
I think I'll do a goood job on my encryption program, and I reckon there's a small chance that people might notice. I think this because I've been reading up on encryption, looking through libraries like Crypto++, and I can't find any implementations of the ideas I've come up with. I think I have two new ideas that haven't been explored yet.
With regard to qualifications, and whether one person or another person is licensed to put forward their ideas on a particular topic, I really think that's a poor attitude.
Frederick Gotham
Mar 18, 2020, 4:28:38 PM3/18/20
to
I wrote:
> (even accounting for 9-bit bytes and
> padding bytes within integer types).
I meant "padding bits".
> (even accounting for 9-bit bytes and
> padding bytes within integer types).
Also allowing for BigEndian, LittleEndian, One's Complement, Sign Magnitude.
Real Troll
Mar 18, 2020, 4:48:03 PM3/18/20
to
On 18/03/2020 17:59, Frederick Gotham wrote:
> <tl;dr>
experienced C++ programmers (Kuyper, Montero or Helde to name just a
few) , you need to provide snippets of your code. There is no point in
asking anything if you are not able to show us what exactly you have
done so far.
> <tl;dr>
>
> Does anyone have any thoughts on this?
>
Assuming you are not a troll, to get any advice or views from
> Does anyone have any thoughts on this?
>
experienced C++ programmers (Kuyper, Montero or Helde to name just a
few) , you need to provide snippets of your code. There is no point in
asking anything if you are not able to show us what exactly you have
done so far.
Frederick Gotham
Mar 18, 2020, 6:09:49 PM3/18/20
to
Real Troll wrote:
> you need to provide snippets of your code
I'm still drawing pictures. I'll have code in a day or two.
> you need to provide snippets of your code
Chris M. Thomasson
Mar 18, 2020, 6:10:32 PM3/18/20
to
Chris M. Thomasson
Mar 18, 2020, 6:11:53 PM3/18/20
to
their expertise.
David Brown
Mar 19, 2020, 4:00:53 AM3/19/20
to
means both "I can't see a way to crack this" and "I can't see if there
is a fundamental weakness". For an encryption system to be used by
others or for real applications, being understood fully by other
encryption experts is an absolute requirement.
That shouldn't stop you (Chris and Frederick) playing around, having
fun, trying stuff, learning stuff. But it /should/ stop you using your
own encryption stuff for anything serious.
Chris M. Thomasson
Mar 19, 2020, 3:42:33 PM3/19/20
to
On 3/19/2020 1:00 AM, David Brown wrote:
> On 18/03/2020 23:11, Chris M. Thomasson wrote:
>> On 3/18/2020 3:10 PM, Chris M. Thomasson wrote:
>>> On 3/18/2020 3:09 PM, Frederick Gotham wrote:
>>>> Real Troll wrote:
>>>>
>>>>> you need to provide snippets of your code
>>>>
>>>>
>>>> I'm still drawing pictures. I'll have code in a day or two.
>>>>
>>>
>>> Make sure to give the folks over on sci.crypt a chance to take a look.
>>
>> Fwiw, I am at a point where they are telling me that my cipher is
>> beyond their expertise.
>
> Note that this might be a good thing, or it might be a bad thing. It
> means both "I can't see a way to crack this" and "I can't see if there
> is a fundamental weakness". For an encryption system to be used by
> others or for real applications, being understood fully by other
> encryption experts is an absolute requirement.
Agreed. I know a lot about HMAC, but not enough to say my work with it
> On 18/03/2020 23:11, Chris M. Thomasson wrote:
>> On 3/18/2020 3:10 PM, Chris M. Thomasson wrote:
>>> On 3/18/2020 3:09 PM, Frederick Gotham wrote:
>>>> Real Troll wrote:
>>>>
>>>>> you need to provide snippets of your code
>>>>
>>>>
>>>> I'm still drawing pictures. I'll have code in a day or two.
>>>>
>>>
>>> Make sure to give the folks over on sci.crypt a chance to take a look.
>>
>> Fwiw, I am at a point where they are telling me that my cipher is
>> beyond their expertise.
>
> Note that this might be a good thing, or it might be a bad thing. It
> means both "I can't see a way to crack this" and "I can't see if there
> is a fundamental weakness". For an encryption system to be used by
> others or for real applications, being understood fully by other
> encryption experts is an absolute requirement.
is okay. I need HMAC experts to jump on board wrt trying to literally
tear it apart at every point possible. Or, I might have to pay somebody.
> That shouldn't stop you (Chris and Frederick) playing around, having
> fun, trying stuff, learning stuff. But it /should/ stop you using your
> own encryption stuff for anything serious.
name of my cipher:
Experimental HMAC Cipher
http://funwithfractals.atspace.cc/ct_cipher
therefore, it is lingering in this experimental state. Fun times! ;^o
0 new messages