Blowfish[2] failed to encrypt big text file of 24MB

[might1]

Dear Vimmers:

Because of the interesting vim encryption capability, I tried both vim 7.4.608 downloaded from http://sourceforge.net/projects/cream/files/Vim/ and vim 7.4 downloaded from www.vim.org. I tried both of vim 7.4.608's blowfish2 and vim 7.4's blowfish, all failed to encrypt a large text file(24MB). The platform is Windows XP 32bit. At first time, I got an error of many lines 'E831： bf_key_init()...empty password' but I really had entered password twice. Then I quitted and re-opened that big file, the encryption appeared successful without error out. But the encrypted file both growed to over 100MB. After decryption, its size remained over 100MB, which was nearly entirely different from the original except that several head lines are same. When I tried a small text file(about 1MB), the encryption and decryption are both ok. So, if there is a file size limit for vim to encrypt with blowfish[2]?  I attached(is that possible?) that compressed big file whose size is now about 4MB for you to test.

It seems that it is far for vim to be encryption safe.

Mr. Fritz wrote:
> I doubt very much anyone would decide to use or not use Vim based solely on its encryption capabilities.
Vim has been a very very greatly excellent editor. If strong encryption mehtods were integrated into it, vim will certainly spread more as people often need to encrypt what they write like diary, project plans. The good encryption in vim could eliminate the needs of users having to seek another encryption tool like GPG and trust it, install it and learn it.

> Vim's blowfish2 encryption method is already a strong encryption method. Adding more methods serves little purpose.
At least one more strong encryption method should be implemented in vim. How about encrypting with one strong method an encrypted file that has already been encryted with another strong one, in which case user must prepare two differrent passwords for those two methods respectively? In this way, we can avoid attackers analysing the text after decryption to see if it looks like words.

> And at what cost? Without this magic text, Vim does not know immediately that the file is encrypted.
And Mr. Moolenaar the Vim author wrote:
> If you would really want this, I think we would need a special option for that. The user would then have to enter both the password and the crypt method.
Vim should add an option, say, 'set magicstring' and 'set nomagicstring' to toggle if prepend the magic string to encrypted file. The default is 'set magicstring'.  User could input, say, ':Z blowfish' in vim and be prompted for password to decrypt. Also, user could just input ':Z' to decrypt with default method, which can be set in vimrc. In a nut shell, it should be up to user how to encrypt and decrypt.


Best wishes,
Tora (Tiger in Japanese)

[Christian Brabandt]

Am 2015-02-13 08:48, schrieb might1:
> Because of the interesting vim encryption capability, I tried both vim
> 7.4.608 downloaded from
> http://sourceforge.net/projects/cream/files/Vim/ and vim 7.4
> downloaded from www.vim.org. I tried both of vim 7.4.608's blowfish2
> and vim 7.4's blowfish, all failed to encrypt a large text file(24MB).
> The platform is Windows XP 32bit. At first time, I got an error of
> many lines 'E831： bf_key_init()...empty password' but I really had
> entered password twice. Then I quitted and re-opened that big file,
> the encryption appeared successful without error out. But the
> encrypted file both growed to over 100MB. After decryption, its size
> remained over 100MB, which was nearly entirely different from the
> original except that several head lines are same. When I tried a small
> text file(about 1MB), the encryption and decryption are both ok. So,
> if there is a file size limit for vim to encrypt with blowfish[2]? I
> attached(is that possible?) that compressed big file whose size is now
> about 4MB for you to test.

Can you reproduce this starting from vim -u NONE -N?
I have only a windows plain 7.4 version available, but that did
successfully encrypt using the blowfish cryptmethod your provided file.

> At least one more strong encryption method should be implemented in
> vim. How about encrypting with one strong method an encrypted file
> that has already been encryted with another strong one, in which case
> user must prepare two differrent passwords for those two methods
> respectively? In this way, we can avoid attackers analysing the text
> after decryption to see if it looks like words.

Perhaps you want to use a specialized encryption software like gpg
to reencrypt the file afterwards? Bram has already stated, that there
are not many developers that have the skills to implement that securly.
And since Vim is mainly a text editor, I do not see the sense in adding
another strong encryption method.

Best,
Christian

[might1]

At 2015-02-13 17:02:00，"Christian Brabandt" <cbl...@256bit.org> wrote:

> Can you reproduce this starting from vim -u NONE -N? I have only a windows plain 7.4 version available, but that did successfully encrypt using the blowfish cryptmethod your provided file.

Right now, I just tried vim7.4 downloaded from www.vim.org. According to your way, I tried in commanld line 'vim -u NONE -N test.txt'   and  'gvim -u NONE -N test.txt' to encrypt with blowfish method, both failed. The resulting files all growed to 113MB. Really strange! No error jumped out.

[might1]

At 2015-02-13 18:00:32，"might1" <mig...@126.com> wrote:
> Right now, I just tried vim7.4 downloaded from www.vim.org. According to your way, I tried in commanld line 'vim -u NONE -N test.txt'   and  'gvim -u NONE -N test.txt' to encrypt with blowfish method, both failed. The resulting files all growed to 113MB. Really strange! No error jumped out.

Do you need I attach the resulting encrypted file whose size remains 113MB after max compression?

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[might1]

I just installed vim7.4 on another win7 machine. At the first time I used vim to encrypt that big file with blowfish method, the error   'E831： bf_key_init() called with empty password' displayed but I really had entered the password. After I reopened the file and encrypted it, no error displayed and the encrypted file became 113MB. Anyone knows what is the problem?

[Ben Fritz]

On Friday, February 13, 2015 at 4:16:01 AM UTC-6, might1 wrote:
> At 2015-02-13 18:00:32，"might1" <mig...@126.com> wrote:
> > Right now, I just tried vim7.4 downloaded from www.vim.org.
> According to your way, I tried in commanld line 'vim -u NONE -N
> test.txt'   and  'gvim -u NONE -N test.txt' to encrypt with blowfish
> method, both failed. The resulting files all growed to 113MB. Really
> strange! No error jumped out.
>
> Do you need I attach the resulting encrypted file whose size remains 113MB after max compression?
>

Actually, attaching the already encrypted file is pretty much useless.

I experimented some, with the attached toy input file (compressed with 7-zip to avoid a huge attachment), and found that the issue seems to be related to writing *new* files somehow.

If I create an encrypted output file as follows, everything works fine:

:e encrypt_in.txt
:saveas encrypt_blowfish2.txt
:X
password
password
:w

However, if I encrypt as follows, I see the E381 error repeated a bunch of times, and Vim seems to hang if I quit that error and try the password again and write, presumably creating that giant file the OP mentioned:

:r encrypt_in.txt
ggdd
:X
password
password
:w encrypt_blowfish2.txt

Oddly enough, this also fails:

:r encrypt_in.txt
ggdd
:w encrypt_blowfish2.txt
:X
password
password
:w

But this succeeds (after a really long time):

:r encrypt_in.txt
ggdd
:w encrypt_blowfish2.txt
:e
:X
password
password
:w

It seems to be related to encrypting the swapfile for the new file. This fails:

:set swapfile noundofile
:r encrypt_in.txt
ggdd
:X
password
password
:w encrypt_blowfish2.txt

But this succeeds:

:set noswapfile undofile
:r encrypt_in.txt
ggdd
:X
password
password
:w encrypt_blowfish2.txt

Using the following Vim version on Windows 7:

VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Feb 4 2015 10:16:41)
MS-Windows 64-bit GUI version with OLE support
Included patches: 1-608
Compiled by {my user name}
Huge version with GUI. Features included (+) or not (-):
+acl +cryptv +file_in_path +listcmds +persistent_undo +statusline +visual
+arabic +cscope +find_in_path +localmap -postscript -sun_workshop +visualextra
+autocmd +cursorbind +float -lua +printer +syntax +viminfo
+balloon_eval +cursorshape +folding +menu +profile +tag_binary +vreplace
+browse +dialog_con_gui -footer +mksession +python/dyn +tag_old_static +wildignore
++builtin_terms +diff +gettext/dyn +modify_fname -python3 -tag_any_white +wildmenu
+byte_offset +digraphs -hangul_input +mouse +quickfix -tcl +windows
+cindent -directx +iconv/dyn +mouseshape +reltime -tgetent +writebackup
+clientserver -dnd +insert_expand +multi_byte +rightleft -termresponse -xfontset
+clipboard -ebcdic +jumplist +multi_lang -ruby +textobjects -xim
+cmdline_compl +emacs_tags +keymap -mzscheme +scrollbind +title -xterm_save
+cmdline_hist +eval +langmap +netbeans_intg +signs +toolbar +xpm_w32
+cmdline_info +ex_extra +libcall +ole +smartindent +user_commands
+comments +extra_search +linebreak +path_extra -sniff +vertsplit
+conceal +farsi +lispindent -perl +startuptime +virtualedit

[Ben Fritz]

On Friday, February 13, 2015 at 1:51:37 AM UTC-6, might1 wrote:
> > And at what cost? Without this magic text, Vim does not know immediately that the file is encrypted.
> And Mr. Moolenaar the Vim author wrote:
> > If you would really want this, I think we would need a special option for that. The user would then have to enter both the password and the crypt method.
> Vim should add an option, say, 'set magicstring' and 'set nomagicstring' to toggle if prepend the magic string to encrypted file. The default is 'set magicstring'.  User could input, say, ':Z blowfish' in vim and be prompted for password to decrypt. Also, user could just input ':Z' to decrypt with default method, which can be set in vimrc. In a nut shell, it should be up to user how to encrypt and decrypt.
>

But, WHY?

If someone already knows your password, and knows or guesses you are a Vim user, it will be trivial to try the three different encryption methods.

You're basically multiplying the complexity of your password by 3 by removing that from the file. In terms of security, that's a completely irrelevant factor. The strength of the cryptography does NOT come from not knowing the encryption method. It comes from the fact that the encryption method itself is not breakable, even knowing the details of the implementation. OpenSSL is generally considered secure (minus the occasional high-profile bug) and yet everyone knows exactly how it is implemented, and it's an easy guess that it's in use when you're visiting a webpage on a wide range of sites. Hiding the fact that it's using OpenSSL, or hiding the size of the key in use, doesn't make a website more secure. The algorithm *itself* is secure.

Your idea of using TWO encryption methods back-to-back is not a terrible one. That way, if a flaw is discovered in the implementation of one, you'd still be protected by the second. But hiding the magic string does nothing at all to enhance security, its only effect would be causing a hassle for the user to remember which crypt method was used.

There is still the problem of not having very many available cryptography experts contributing to Vim. So I doubt this will be implemented soon, and it's irrelevant if there are no additional vulnerabilities hiding in the blowfish code.

[Bram Moolenaar]

[...]

Since it works fine on Linux, could it be related to "long" not being
able to store a pointer? It's a common problem for porting a C program
to Windows.

--
hundred-and-one symptoms of being an internet addict:
227. You sleep next to your monitor. Or on top of it.

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[might1]

Mr. Fritz wrote:

> It seems to be related to encrypting the swapfile for the new file.

Hooray! Problem solved! Acute observation.

I tested again for many times and am sure the issue lies in the swapfile's creation and/or encryption. Before this issue is fixed, we have to use:

gvim -n file_to_encrypt_or_decrypt

to properly encrypt and decrypt.

Also, we could dirctedly open a file to encrypt and use:

:set noswapfile

to get rid of swapfile, and every thing goes fine.

Note that, when we decrypt a file, we have to use command line: gvim -n file because vim will immediately prompt for password thus give no chance to use ':set noswapfile' for correct decryption.

Mr. Moolenaar wrote:

> Since it works fine on Linux...It's a common problem for porting a C program to Windows.

It is so easy to make mistakes in a twisted OS.

Mr. Fritz wrote:

> The strength of the cryptography does NOT come from not knowing the encryption method. It comes from the fact that the encryption method itself is not breakable.

> Hiding the fact that it's using OpenSSL, or hiding the size of the key in use, doesn't make a website more secure.

> Hiding the magic string does nothing at all to enhance security.

I agree with you that blowfish algorithm is secure. But at the same time, it requires an ENOUGH LONG password for the encrypted file to be enough safe. What I mean is more encryption methods in Vim could FURTHER GREATLY enhance security.

The prerequisite for a cracker to decrypt is he knows the encryption method by which a file is encrypted. Without knowing the encryption method at first, how would he decrypt?  Say, one uses CAST5 to encrypt a file. If the cracker doesn't know the method and uses blowfish to brute-force decrypt, he will never succeed.  The cracker will think that the password must be very long and he will continue to try and try. This way, hiding the encryption method certainly help enhance security. But for now, vim has only one strong encryption method and leaving out magic string makes no sense if the cracker could guess file is created with vim.  It seems that you are suggesting the encryption method like blowfish or CAST5 can be found from encrypted file maybe because there are some encryption method specific patterns in that messed file. If this is the case, you are right and hiding the encryption method has no meaning. But is it really possbile to find the encryption method from an encryptedly messed file(except weak methods like zip encrypted file. I mean strong ones like blowfish.)? If the answer is no, then encrypting with one strong method an encrypted file that has already been encryted with another strong one will OF COURSE make sense as the attacker will never know that cracked file is another encrypted file or a real mess, and we can eliminate attackers analysing the text after decryption to see if it looks like words.

The reason OpenSSL is generally considered secure is because the key is ENOUGH LONG or difficult factorization, which makes the brute-force decryption too long time. But that kind of security never means more encryption methods accompanied with hiding the encryption mothod will not GREATLY enhance data security.

[Ken Takata]

Hi,

Perhaps I found a fix for this.

--- a/src/memfile.c
+++ b/src/memfile.c
@@ -1033,7 +1033,7 @@

#ifdef FEAT_CRYPT
/* Decrypt if 'key' is set and this is a data block. */
- if (*mfp->mf_buffer->b_p_key != NUL)
+ if (*mfp->mf_buffer->b_p_key != NUL || mfp->mf_old_key != NULL)
ml_decrypt_data(mfp, hp->bh_data, offset, size);
#endif

@@ -1136,7 +1136,7 @@

#ifdef FEAT_CRYPT
/* Encrypt if 'key' is set and this is a data block. */
- if (*mfp->mf_buffer->b_p_key != NUL)
+ if (*mfp->mf_buffer->b_p_key != NUL || mfp->mf_old_key != NULL)
{
data = ml_encrypt_data(mfp, data, offset, size);
if (data == NULL)


I checked this only on Windows (but not well).
I hope someone check this more deeply.
I wonder why the problem doesn't occur on Linux.

Regards,
Ken Takata

[Ben Fritz]

On Friday, December 18, 2015 at 8:01:37 AM UTC-6, Ken Takata wrote:
> > > However, if I encrypt as follows, I see the E381 error repeated a bunch of times, and Vim seems to hang if I quit that error and try the password again and write, presumably creating that giant file the OP mentioned:
> > >
> > > :r encrypt_in.txt
> > > ggdd
> > > :X
> > > password
> > > password
> > > :w encrypt_blowfish2.txt
> > >
>

> Perhaps I found a fix for this.
>
> --- a/src/memfile.c
> +++ b/src/memfile.c
> @@ -1033,7 +1033,7 @@
>

Unfortunately, I can no longer reproduce the issue: I was able to encrypt, save, quit, read, and decrypt in Vim 7.4.977 (still HUGE 64-bit on Windows 7) using the previously failing methods above.

So I'm not able to test whether your patch fixes it. :-(

[Christian Brabandt]

On Fr, 18 Dez 2015, Ken Takata wrote:

> Perhaps I found a fix for this.

Wasn't this fixes with 7.4.730?



Best,
Christian
--
Fußball ist wie Schach nur ohne Würfel.
-- Lukas Podolski

[Bram Moolenaar]

It's most likely related to the memory limit, causing Vim to use the
swap file.

> Perhaps I found a fix for this.

Thanks! Although I think only the first part matters, when writing the
old key isn't used.

--
If I tell you "you have a beautiful body", would you hold it against me?

[Ken Takata]

Hi Bram,

Ah, you are right!
I confirmed that the second part was not needed.

Regards,
Ken Takata

[Ken Takata]

Hi Christian,

2015/12/19 Sat 4:24:00 UTC+9 Christian Brabandt wrote:
> On Fr, 18 Dez 2015, Ken Takata wrote:
>
> > Perhaps I found a fix for this.
>
> Wasn't this fixes with 7.4.730?

Unfortunately, it wasn't.

Regards,
Ken Takata