[vim/vim] Test_writefile_fails_conversion failure on Solaris (#1872)

50 views
Skip to first unread message

Danek Duvall

unread,
Jul 24, 2017, 12:46:09 PM7/24/17
to vim/vim, Subscribed

Testing recent versions of vim on Solaris, I get the following error:

From test_writefile.vim:
Found errors in Test_writefile_fails_conversion():
function RunTheTest[24]..Test_writefile_fails_conversion line 10: command did not fail: write ++enc=cp932
function RunTheTest[24]..Test_writefile_fails_conversion line 11: Expected ['line one', 'line two'] but got ['first line', 'cannot convert ?', 'third line']

The test was introduced in 8.0.0685 (e6bf655). From our i18n experts, it appears that Linux and Solaris took different paths when implementing the POSIX iconv spec; namely, that

the results of encountering invalid characters in the input stream (either those that are not characters in the codeset of the input file or that have no corresponding character in the codeset of the output file) shall be specified in the system documentation

Thus the assertions are based on bad assumptions.

I don't know the problem well enough to suggest a more universal test, or to judge whether the fix itself works on Solaris (though I'm guessing it doesn't). Solaris does support //REPLACE_HEX and //RESTORE_HEX modifiers to the arguments to iconv_open(), which might prove useful here, and I've confirmed that the string in the test does get converted to and from a lossless hex representation appropriately when converted using the commandline iconv tool.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub

Bram Moolenaar

unread,
Jul 24, 2017, 5:06:34 PM7/24/17
to vim/vim, Subscribed

Danek Duvall wrote:

> Testing recent versions of vim on Solaris, I get the following error:
>
> ```
> >From test_writefile.vim:
> Found errors in Test_writefile_fails_conversion():
> function RunTheTest[24]..Test_writefile_fails_conversion line 10: command did not fail: write ++enc=cp932
> function RunTheTest[24]..Test_writefile_fails_conversion line 11: Expected ['line one', 'line two'] but got ['first line', 'cannot convert ?', 'third line']
> ```
>
> The test was introduced in 8.0.0685 (e6bf655bc4de1b7f4586e1f5c2fc4978141c3aa3). From our i18n experts, it appears that Linux and Solaris took different paths when implementing the POSIX iconv spec; namely, that

>
> > the results of encountering invalid characters in the input stream (either those that are not characters in the codeset of the input file or that have no corresponding character in the codeset of the output file) shall be specified in the system documentation
>
> Thus the assertions are based on bad assumptions.
>
> I don't know the problem well enough to suggest a more universal test, or to judge whether the fix itself works on Solaris (though I'm guessing it doesn't). Solaris does support `//REPLACE_HEX` and `//RESTORE_HEX` modifiers to the arguments to `iconv_open()`, which might prove useful here, and I've confirmed that the string in the test does get converted to and from a lossless hex representation appropriately when converted using the commandline `iconv` tool.

Perhaps we can first use the iconv() function to check whether the
conversion fails. With your Vim, try this:

echo iconv("cannot convert \u010b", 'utf-8', 'cp932')

I get "cannot convert ?".

--
hundred-and-one symptoms of being an internet addict:
237. You tattoo your email address on your forehead.

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

Danek Duvall

unread,
Jul 24, 2017, 5:27:46 PM7/24/17
to vim/vim, Subscribed

I get the same message.

Bram Moolenaar

unread,
Jul 25, 2017, 5:32:03 PM7/25/17
to vim/vim, Subscribed

> I get the same message.

Hmm, then perhaps we need to check for Solaris specifically.
What doest "uname" return?

Or perhaps this is a SysV common issue?


--
hundred-and-one symptoms of being an internet addict:
251. You've never seen your closest friends who usually live WAY too far away.


/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

Pavel Heimlich

unread,
Jul 31, 2017, 6:28:17 AM7/31/17
to vim/vim, Subscribed

On Solaris 'uname' returns:
SunOS

I don't have any other Unices around.

Danek Duvall

unread,
Jul 31, 2017, 12:01:56 PM7/31/17
to vim/vim, Subscribed

I've confirmed that this happens on 11.3 as well as the development builds of the next release. I don't have easy access to older versions.

It's easy enough to skip the test on Solaris, but I'd rather have a functioning test. I'm just not sure how to write one. And I suspect that the functionality is broken (on Solaris, or any OS with a similar iconv behavior) as well.

Bram Moolenaar

unread,
Jul 31, 2017, 5:22:27 PM7/31/17
to vim/vim, Subscribed

Pavel Heimlich wrote:

> On Solaris 'uname' returns:
> SunOS
>
> I don't have any other Unices around.

Thanks. I think it's not too bad to skip this test if "uname" returns
"SunOS".

--
PRINCE: He's come to rescue me, father.
LAUNCELOT: (embarrassed) Well, let's not jump to conclusions ...
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD


/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

Bram Moolenaar

unread,
Jul 31, 2017, 5:22:27 PM7/31/17
to vim/vim, Subscribed

Danek Duvall wrote:

> I've confirmed that this happens on 11.3 as well as the development
> builds of the next release. I don't have easy access to older
> versions.
>
> It's easy enough to skip the test on Solaris, but I'd rather have a
> functioning test. I'm just not sure how to write one. And I suspect
> that the functionality is broken (on Solaris, or any OS with a similar
> iconv behavior) as well.

The test checks that if the conversion fails, the file is not messed up.
In this situation the conversion does not fail. It's difficult to check
that.

--
A vacation is a period of travel during which you find that you
took twice as many clothes and half as much money as you needed.


/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

lacygoill

unread,
Feb 4, 2021, 4:35:45 PM2/4/21
to vim/vim, Subscribed

Fixed by 8.1.0347


You are receiving this because you are subscribed to this thread.

Reply to this email directly, view it on GitHub, or unsubscribe.

Christian Brabandt

unread,
Feb 5, 2021, 2:24:28 AM2/5/21
to vim/vim, Subscribed

Closed #1872.


You are receiving this because you are subscribed to this thread.

Reply to this email directly, view it on GitHub, or unsubscribe.

Reply all
Reply to author
Forward
0 new messages