Feature or bug? dw oddities

144 views
Skip to first unread message

Elmar Hinz

unread,
Feb 3, 2016, 5:45:08 AM2/3/16
to vim_use
Hello,

I am working on a Python clone of vim, which requires to inspect Vim behaviour in detail. Some behaviour looks odd and I like to know if it is a bug or what is the reasoning of that behaviour. This is my first observation:

Given, cursor on first x:

xxx
xxx
xxx

Now hitting 1dw deletes zero line breaks, while 2dw deletes two line breaks.

Feature or bug?

Regards

Elmar


https://github.com/elmar-hinz/Python.Vii

Random832

unread,
Feb 4, 2016, 12:31:03 AM2/4/16
to vim...@googlegroups.com
Elmar Hinz <t3e...@googlemail.com> writes:
> Hello,
>
> I am working on a Python clone of vim, which requires to inspect Vim
> behaviour in detail. Some behaviour looks odd and I like to know if it
> is a bug or what is the reasoning of that behaviour. This is my first
> observation:
>
> Given, cursor on first x:
>
> xxx
> xxx
> xxx
>
> Now hitting 1dw deletes zero line breaks, while 2dw deletes two line breaks.
>
> Feature or bug?

Some interesting information:

1. Other vi implementations differ on how they behave here:

- Elvis behaves the same way.

- Heirloom vi behaves the same way. But *crashes* on some operating
systems if asked to do this (it works on Ubuntu, but not Mac OSX,
for me).

- Most other vi implementations (nvi, vile, and Emacs' evil and viper
modes) do not behave this way.

2. 2cw does _not_ delete the second linebreak, in any tested editor.

3. The POSIX vi standard specifies some distinction between 2cw and 2dw:

- If there are <blank> characters or an end-of-line that precede the
countth bigword, and the associated command is c, the region of
text shall be up to and including the last character before the
preceding <blank> characters or end-of-line.

- If there are <blank> characters or an end-of-line that precede the
bigword, and the associated command is d or y, the region of text
shall be up to and including the last <blank> before the start of
the bigword or end-of-line.

(Reference to "bigwords" are because this is actually the description
of the W command.)

I _think_ that this is _not_ intended to imply vim's behavior, since
the end-of-line is, at least in context, not a <blank>.

Elmar Hinz

unread,
Feb 4, 2016, 7:30:26 AM2/4/16
to vim_use
On Thursday, February 4, 2016 at 6:31:03 AM UTC+1, Random832 wrote:

> Some interesting information:
>
> 1. Other vi implementations differ on how they behave here:
>
> - Elvis behaves the same way.
>
> - Heirloom vi behaves the same way. But *crashes* on some operating
> systems if asked to do this (it works on Ubuntu, but not Mac OSX,
> for me).
>
> - Most other vi implementations (nvi, vile, and Emacs' evil and viper
> modes) do not behave this way.
>
> 2. 2cw does _not_ delete the second linebreak, in any tested editor.
>
> 3. The POSIX vi standard specifies some distinction between 2cw and 2dw:
>
> - If there are <blank> characters or an end-of-line that precede the
> countth bigword, and the associated command is c, the region of
> text shall be up to and including the last character before the
> preceding <blank> characters or end-of-line.
>
> - If there are <blank> characters or an end-of-line that precede the
> bigword, and the associated command is d or y, the region of text
> shall be up to and including the last <blank> before the start of
> the bigword or end-of-line.
>
> (Reference to "bigwords" are because this is actually the description
> of the W command.)
>
> I _think_ that this is _not_ intended to imply vim's behavior, since
> the end-of-line is, at least in context, not a <blank>.


Thank you for the interesting input. I conclude you think the vim behaviour is at least disputable here. I keep this im mind for the future. For now I try to emulate Vim behaviour.

Obviously this is an example of

help: exclusive-linewise

That can be proved with, with cursor on X:

Xxx
xxx
xxx
xxx


With 2dw exclusive-likewise behaviour is used and the whitespace before X is deleted.
With 1dw neither whitespace nor line-break are deleted.

Elmar

Christian Brabandt

unread,
Feb 4, 2016, 8:56:28 AM2/4/16
to vim_use
Hi Elmar!
Where can I see your clone?

>
> Obviously this is an example of
>
> help: exclusive-linewise
>
> That can be proved with, with cursor on X:
>
> Xxx xxx xxx xxx
>
>
> With 2dw exclusive-likewise behaviour is used and the whitespace
> before X is deleted. With 1dw neither whitespace nor line-break are
> deleted.

I think, it is this part of the code from op_delete()

/*
* Imitate the strange Vi behaviour: If the delete spans more than one
* line and motion_type == MCHAR and the result is a blank line, make the
* delete linewise. Don't do this for the change command or Visual mode.
*/
if ( oap->motion_type == MCHAR
&& !oap->is_VIsual
&& !oap->block_mode
&& oap->line_count > 1
&& oap->motion_force == NUL
&& oap->op_type == OP_DELETE)
{
ptr = ml_get(oap->end.lnum) + oap->end.col;
if (*ptr != NUL)
ptr += oap->inclusive;
ptr = skipwhite(ptr);
if (*ptr == NUL && inindent(0))
oap->motion_type = MLINE;
}


So this seems to come from some old vi oddity.

Best,
Christian
--
Niemand ist ein ärgerer Feind des Christentums als das Christentum.
-- Hans-Hermann Kesten

Elmar Hinz

unread,
Feb 4, 2016, 9:58:19 AM2/4/16
to vim_use
On Thursday, February 4, 2016 at 2:56:28 PM UTC+1, Christian Brabandt wrote:
> Hi Elmar!

>
> Where can I see your clone?


Thank your for your interest im my Python clone:

https://github.com/elmar-hinz/Python.Vii


> > With 2dw exclusive-likewise behaviour is used and the whitespace
> > before X is deleted. With 1dw neither whitespace nor line-break are
> > deleted.
>
> I think, it is this part of the code from op_delete()
>
> /*
> * Imitate the strange Vi behaviour: If the delete spans more than one
> * line and motion_type == MCHAR and the result is a blank line, make the
> * delete linewise. Don't do this for the change command or Visual mode.
> */

>

> So this seems to come from some old vi oddity.
>


Thank you Christian for your input. So we find Vim itself thinks this behaviour disputable.

Elmar


Btw: I am still looking for the final name.

[ ] Vii
[ ] Vip


https://github.com/elmar-hinz/Python.Vii

Random832

unread,
Feb 4, 2016, 12:04:20 PM2/4/16
to vim...@googlegroups.com
On 2016-02-04, Christian Brabandt <cbl...@256bit.org> wrote:
> I think, it is this part of the code from op_delete()
>
> /*
> * Imitate the strange Vi behaviour: If the delete spans more than one
> * line and motion_type == MCHAR and the result is a blank line, make the
> * delete linewise. Don't do this for the change command or Visual mode.
> */
...
> So this seems to come from some old vi oddity.

This fits with the fact that heirloom vi also does it. It
contradicting the POSIX standard explains why no-one else does
it, except perhaps commercial Unixes/OpenSolaris with a vi
independently derived from 4BSD vi.

Note: Ubuntu's "elvis-tiny", which was the version of elvis I
actually tested, has far more simplistic and clearly incorrect
behavior - it _always_ deletes the newline when deleting the
last word of a line, and fails entirely on the last word of the
last line. I've since downloaded a proper version of elvis and
it behaves the same as all the other clones apart from vim.

Maybe there should be a compatibility option to control this
behavior.

Andy Wokula

unread,
Feb 4, 2016, 12:45:29 PM2/4/16
to vim...@googlegroups.com
Let me :help this for you.

:h w
`--> :h exclusive
:h cw

--
Andy

Elmar Hinz

unread,
Feb 4, 2016, 3:57:17 PM2/4/16
to vim_use

> last line. I've since downloaded a proper version of elvis and
> it behaves the same as all the other clones apart from vim.
>

Summarising this, you think vim is behaving differently than most other vi derivates?

Why is the comment quoted by Christian Brabandt telling, vim tries to behave like vi?


Elmar

Christian Brabandt

unread,
Feb 4, 2016, 4:58:26 PM2/4/16
to vim_use, vim-dev
Hi Elmar!

On Do, 04 Feb 2016, Elmar Hinz wrote:

>
Well, we can perhaps use this patch.

Mit freundlichen Grüßen
Christian
--
Es gibt mehr alte Trinker als alte Ärzte.
cpo-specialword

Matt Ackeret

unread,
Feb 4, 2016, 5:44:37 PM2/4/16
to vim...@googlegroups.com
On Wed, 3 Feb 2016, Random832 wrote:
>Some interesting information:
>
>1. Other vi implementations differ on how they behave here:
>
> - Elvis behaves the same way.
>
> - Heirloom vi behaves the same way. But *crashes* on some operating
> systems if asked to do this (it works on Ubuntu, but not Mac OSX,
> for me).

Is "heirloom vi" a different version of vi? So you're saying it crashes
on OS X? (Vim certainly isn't crashing for me on OS X..)

If the same program with the same version is crashing in some versions...
figure out why? (I suspect it's NOT an OS program, but it's not impossible.)

Christian Brabandt

unread,
Feb 4, 2016, 5:48:46 PM2/4/16
to vim...@googlegroups.com
Hi Matt!

On Do, 04 Feb 2016, Matt Ackeret wrote:

> Is "heirloom vi" a different version of vi? So you're saying it crashes
> on OS X? (Vim certainly isn't crashing for me on OS X..)

Some traditional vi clone. http://ex-vi.sourceforge.net/
I believe the project is dead, at least there hasn't been any activities
for more than 10 years.


Best,
Christian
--
Letzte Worte eines Kapitäns:
"Wo kommt all das verdammte Wasser her?"

Random832

unread,
Feb 5, 2016, 2:07:27 AM2/5/16
to vim...@googlegroups.com
Elmar Hinz <t3e...@googlemail.com> writes:
> Summarising this, you think vim is behaving differently than most
> other vi derivates?
>
> Why is the comment quoted by Christian Brabandt telling, vim tries to
> behave like vi?

"vi derivates" is a bit misleading term. All of the programs I
discussed, except heirloom vi, are actually vi *clones*. Looking at the
original vi itself (heirloom is a version for modern systems) on a
4.3BSD virtual machine I had set up, it does show the same behavior as
vim and heirloom vi here.

I assume the same behavior is also present on other (mainly commercial)
unix systems that have their own implementation of vi derived from 4BSD
vi. Does anyone happen to have e.g. an OpenSolaris machine to test on?

Elmar Hinz

unread,
Feb 5, 2016, 4:06:07 AM2/5/16
to vim_use
> > Summarising this, you think vim is behaving differently than most
> > other vi derivates?
> >
>
> "vi derivates" is a bit misleading term. All of the programs I
> discussed, except heirloom vi, are actually vi *clones*. Looking at the
> original vi itself (heirloom is a version for modern systems) on a
> 4.3BSD virtual machine I had set up, it does show the same behavior as
> vim and heirloom vi here.
>

Then my summary was wrong.

You found out original Vi and Vim behave the same way?

Elmar

Matt Ackeret

unread,
Feb 5, 2016, 7:33:17 PM2/5/16
to vim...@googlegroups.com
On Thu, 4 Feb 2016, Random832 wrote:
>"vi derivates" is a bit misleading term. All of the programs I
>discussed, except heirloom vi, are actually vi *clones*.

I'm nitpicking, but I think vi derivatives is more accurate.

If they were clones, they would behave IDENTICALLY to vi.

But these programs, or at the very least vim, is a derivative of vi,
at least from the user interface point of view. (I have no idea if any of
them took the original vi code and went on from there.)

Again, at least vim, is _almost_ a superset of vi... But since there are
some behaviors that are different (":help vi_diff.txt"), that's even more
ammunition that it's a derivative and not a clone.

Bram Moolenaar

unread,
Feb 6, 2016, 8:47:05 AM2/6/16
to Christian Brabandt, vim_use, vim-dev

Christian Brabandt wrote:

> > > last line. I've since downloaded a proper version of elvis and it
> > > behaves the same as all the other clones apart from vim.
> > >
> >
> > Summarising this, you think vim is behaving differently than most
> > other vi derivates?
> >
> > Why is the comment quoted by Christian Brabandt telling, vim tries to
> > behave like vi?
>
> Well, we can perhaps use this patch.

There are a few more places in the docs that need to point to this flag.
Also, plugin writers must be aware of the two different behaviors.

Is there a better character than "_" to use for this?


--
hundred-and-one symptoms of being an internet addict:
149. You find your computer sexier than your girlfriend

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

Christian Brabandt

unread,
Feb 6, 2016, 9:05:41 AM2/6/16
to vim_use, vim-dev
Hi Bram!

On Sa, 06 Feb 2016, Bram Moolenaar wrote:

>
> Christian Brabandt wrote:
>
> > > > last line. I've since downloaded a proper version of elvis and it
> > > > behaves the same as all the other clones apart from vim.
> > > >
> > >
> > > Summarising this, you think vim is behaving differently than most
> > > other vi derivates?
> > >
> > > Why is the comment quoted by Christian Brabandt telling, vim tries to
> > > behave like vi?
> >
> > Well, we can perhaps use this patch.
>
> There are a few more places in the docs that need to point to this flag.
> Also, plugin writers must be aware of the two different behaviors.

Alright, I'll post an update later.

> Is there a better character than "_" to use for this?

Hm, not sure. We are slowly running out of characters there. I'll check
that again.

Perhaps we should start to use letters from other alphabets. I hear
cyrillic has some beautiful letters ;)

Best,
Christian
--
Realität in der höchsten Nützlichkeit (Zweckmäßigkeit) wird
auch schön sein.
-- Goethe, Maximen und Reflektionen, Nr. 1196

Tony Mechelynck

unread,
Feb 6, 2016, 6:48:14 PM2/6/16
to vim...@googlegroups.com, vim-dev
On Sat, Feb 6, 2016 at 3:05 PM, Christian Brabandt <cbl...@256bit.org> wrote:
[…]
>> There are a few more places in the docs that need to point to this flag.
>> Also, plugin writers must be aware of the two different behaviors.
>
> Alright, I'll post an update later.
>
>> Is there a better character than "_" to use for this?
>
> Hm, not sure. We are slowly running out of characters there. I'll check
> that again.
>
> Perhaps we should start to use letters from other alphabets. I hear
> cyrillic has some beautiful letters ;)
>
> Best,
> Christian

Well, so it does:

Аа Бб Вв Гг Дд Ее Ёё Жж Зз Ии Йй Кк Лл Мм Нн Оо Пп Рр Сс Тт Уу Фф Хх
Цц Чч Шш Щщ Ъъ Ыы Ьь Ээ Юю Яя

(the above are from Russian Cyrillic; other varieties are each
slightly different) and so does Greek (even not counting accents,
breathings and diæresis):

Αα Ββ Γγ Δδ Εε Ζζ Ηη Θθ Ιι Κκ Λλ Μμ Νν Ξξ Οο Ππ Ρρ Σσς Ττ Υυ Φφ Χχ Ψψ Ωω

but not easy to type on a Western keyboard, not even Аа В Ее Ёё К М Н
О Рр Сс Т у Хх Α Β Ε Ζ Η Ι Κ Μ Ν Οο Ρ Τ Υ Χ
which look like Latin-alphabet letters but are different Unicode
codepoints (and I'm not even counting lowercase beta which looks
somewhat like eszett because the latter isn't found in us-ascii). On
my Belgian keyboard I have ²³é§èçൣù even without using the dead
keys, and a lot more with them, see
http://users.skynet.be/antoine.mechelynck/other/keybbe.htm — but I
don't know how one would get those non-ASCII letters on a
(non-international) QWERTY keyboard meant for en-US.

Another possibility :-P would be using CJK letters (with IME/XIM of
course) for some commands. Yes, they are beautiful, and there are
enough of them that you couldn't expect to use them all up, but how
many non-East-Asian Vimmers know how to use this feature which is
already built in Huge gvim?


Best regards,
Tony.

Elmar Hinz

unread,
Feb 7, 2016, 4:06:16 AM2/7/16
to vim_use, vim...@vim.org
> (the above are from Russian Cyrillic; other varieties are each
> slightly different) and so does Greek (even not counting accents,
> breathings and diæresis):
>
> Αα Ββ Γγ Δδ Εε Ζζ Ηη Θθ Ιι Κκ Λλ Μμ Νν Ξξ Ολ Ππ Ρρ Σσς Ττ Υυ Φφ Χχ Ψψ Ωω

In science the Greek would by the natural choice, then. CPO_WORD translates to λόγος.

https://en.wikipedia.org/wiki/Logos

Elmar

BPJ

unread,
Feb 7, 2016, 7:00:33 AM2/7/16
to vim...@googlegroups.com
Wouldn't some of the Latin-1 characters, at least £ and €, actually be rather widely supported. For some reason they are even marked out on my Swedish keyboard, and we use neither pounds nor euros.

Matt Ackeret

unread,
Feb 8, 2016, 1:50:26 PM2/8/16
to vim...@googlegroups.com

> On Feb 6, 2016, at 3:47 PM, Tony Mechelynck <antoine.m...@gmail.com> wrote:
>
> http://users.skynet.be/antoine.mechelynck/other/keybbe.htm — but I
> don't know how one would get those non-ASCII letters on a
> (non-international) QWERTY keyboard meant for en-US.
>

Use a different input method?

Bring up the Keyboard Viewer (or whatever similar thing your OS has)?

As much as I love/live in CLI programs/ASCII, there have been lots of
advancements in these situations over a long time..

The next text will be from the Russian input method:
АВСРОМИС

This wasn't typed in Terminal, but I just typed Russian characters into Terminal, and they showed up fine.

Christian Brabandt

unread,
Feb 8, 2016, 4:01:37 PM2/8/16
to vim_use, vim-dev
Hi Bram!

On Sa, 06 Feb 2016, Bram Moolenaar wrote:

> There are a few more places in the docs that need to point to this flag.
>
> Is there a better character than "_" to use for this?

Updated patch uses 'z'

Also added some more references to it in the documentation.

> Also, plugin writers must be aware of the two different behaviors.

I am not sure, what is meant with this statement. The current behaviour
does not change, only when removing the 'z' flag from 'cpo' it will
behave more consistent (as other vi clones behave).

Best,
Christian
--
Eine freie Seele, wie die seine, kommt in Gefahr, frech zu
werden, wenn nicht ein edles Wohlwollen das sittliche Gleichgewicht
herstellt.
-- Goethe, Maximen und Reflektionen, Nr. 190
cpo-specialword.diff

Bram Moolenaar

unread,
Feb 8, 2016, 4:37:50 PM2/8/16
to Christian Brabandt, vim_use, vim-dev

Christian Brabandt wrote:

> Hi Bram!
>
> On Sa, 06 Feb 2016, Bram Moolenaar wrote:
>
> > There are a few more places in the docs that need to point to this flag.
> >
> > Is there a better character than "_" to use for this?
>
> Updated patch uses 'z'
>
> Also added some more references to it in the documentation.

Thanks for the update.

> > Also, plugin writers must be aware of the two different behaviors.
>
> I am not sure, what is meant with this statement. The current behaviour
> does not change, only when removing the 'z' flag from 'cpo' it will
> behave more consistent (as other vi clones behave).

So most plugin writers will test with the current behavior, and then
when some user uses your change he may run into a plugin not handling
that exception. It would be a bit rare for plugins to notice the
difference though.


--
hundred-and-one symptoms of being an internet addict:
188. You purchase a laptop so you can surf while sitting on the can.

Erik Christiansen

unread,
Feb 9, 2016, 2:24:43 AM2/9/16
to vim...@googlegroups.com
On 08.02.16 10:50, Matt Ackeret wrote:
> The next text will be from the Russian input method:
> АВСРОМИС
>
> This wasn't typed in Terminal, but I just typed Russian characters
> into Terminal, and they showed up fine.

Viewing Cyrillic, Greek, or Russian characters doesn't work for me in
"xterm" on Debian 7.8.0, either in mutt or vim. Mutt just renders them
invisible, and vim displays the above line as:

-D0-90-D0-92-D0-A1-D0-A0-D0-9E-D0-9C-D0-98-D0-A1

( I did s/-/=/g , so as not to render as sent in your environment.)

If I use "xterm -lc", then mutt displays the above characters as
Russian, confirming a utf-8 locale AFAICT, but vim still gives the "=xx"
stuff, both within mutt, and invoked directly in the "xterm -lc". Hmmm,
is this relevant?:

$ echo $LOCALE

I.e. it's not set.

Alas, trying "xterm -en UTF-8" doesn't make any difference. Mutt still
works, and vim fails.

Does anyone know what it takes to get vim to display Cyrillic, Greek, or
Russian characters in a (utf-8 enabled) xterm? (Note: Vim encoding while
editing this post, in which the Russian won't display, is
"encoding=utf-8")

There's no problem with Danish: åæø
or German: ßöä

Erik

Tony Mechelynck

unread,
Feb 9, 2016, 3:49:45 AM2/9/16
to vim...@googlegroups.com
My xterm displays the Cyrillic characters above as Cyrillic, even in
Vim, so it is definitely possible. My only xterm startup argument is a
-geometry parameter (xterm -geometry 160x60) to give it a "reasonable"
size on my 1280x1024 screen.

I think the reason it works for me is related with my bash startup
scripts, one of which (I'm not sure which: see "man bash" for details)
sets the following environment variables:

export LANG=en_US.UTF-8
export LC_TIME=en_GB
export LC_PAPER=en_GB
export LC_NUMERIC=C

If you use (t)csh instead the syntax would of course be different.
$LANG is the fallback for any undefined locale variable, and $LC_ALL
(intentionally unset) would override them all. Since $LC_CTYPE (the
character set) is not set, it defaults to the value of $LANG, which
sets UTF-8. Then if once these settings are set you still don't see
non-ASCII characters, you might have to find out how to start xterm
with a different font. (The two "British" settings are so I get A4
paper and dd/mm/yyyy dates.)

In addition, for Vim you may need to make sure that it uses UTF-8
internally, see http://vim.wikia.com/wiki/Working_with_Unicode


Best regards,
Tony.

Erik Christiansen

unread,
Feb 9, 2016, 6:48:53 AM2/9/16
to vim...@googlegroups.com
On 09.02.16 09:49, Tony Mechelynck wrote:
> On Tue, Feb 9, 2016 at 8:24 AM, Erik Christiansen
> > Does anyone know what it takes to get vim to display Cyrillic, Greek, or
> > Russian characters in a (utf-8 enabled) xterm? (Note: Vim encoding while
> > editing this post, in which the Russian won't display, is
> > "encoding=utf-8")
> >
> > There's no problem with Danish: åæø
> > or German: ßöä
...
>
> I think the reason it works for me is related with my bash startup
> scripts, one of which (I'm not sure which: see "man bash" for details)
> sets the following environment variables:
>
> export LANG=en_US.UTF-8
> export LC_TIME=en_GB
> export LC_PAPER=en_GB
> export LC_NUMERIC=C

Both in the case of complete failure, and the case with mutt working but
vim not, I've had the default:

$ echo $LANG
en_AU.utf8

so that's not the impediment. (Trying "en_AU.UTF-8" instead makes no
difference) And trying "export LANG=en_US.UTF-8" before invoking mutt or
vim causes mutt to corrupt the Russian and the thread tree lines, and
vim to not even display the Danish, i.e. much worse.

...
> In addition, for Vim you may need to make sure that it uses UTF-8
> internally, see http://vim.wikia.com/wiki/Working_with_Unicode

Ah, that looks promising. Checking the settings used in the script
snippet presented there:

encoding=utf-8
termencoding=

Ahah! But adding a "let &termencoding = &encoding" to .vimrc did not
help vim to display the Russian charactors. They remain as "=xx" stuff.

fileencoding=utf-8
fileencodings=ucs-bom,utf-8,default,latin1

These also were already in line with the howto hints.

Thank you for the hints, Tony, but nothing tried so far is making any
difference. Vim still won't display Cyrillic, Greek, or Russian.

But is this a clue? My immediate prior post, containing åæø, came back
from the list server with those three characters now displaying in vim
as -C3-A5-C3-A6-C3-B8, despite the header: "X-NotAscii: charset=utf-8".
(Again, s/-/==/g) In vim, encoding, fileencoding, and termencoding are
all utf-8.


Is our list server not sending utf-8, despite the header claim?

Erik

Nikolay Aleksandrovich Pavlov

unread,
Feb 9, 2016, 7:19:43 AM2/9/16
to vim...@googlegroups.com
It is sending a proper UTF-8. You must use a client which correctly
decodes the message: you should see those equals signs if
Content-Transfer-Encoding is [quoted-printable][1], but message body
was not decoded. If there is no header Content-Transfer-Encoding with
the said value then you caught a bug somewhere, most likely somewhere
between google groups and your PC because I do not see any messages in
this thread which have X-NotAscii header. AFAIK GG does not bother
itself changing messages content or content* headers.

I do see some messages with `Content-Transfer-Encoding:
quoted-printable` in this thread, but in gmail they are displayed
correctly.

See also https://en.wikipedia.org/wiki/Unicode_and_email.

[1]: https://en.wikipedia.org/wiki/Quoted-printable

>
> Erik
>
> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Erik Christiansen

unread,
Feb 10, 2016, 5:41:01 AM2/10/16
to vim...@googlegroups.com
On 09.02.16 15:19, Nikolay Aleksandrovich Pavlov wrote:
> 2016-02-09 14:48 GMT+03:00 Erik Christiansen <dva...@internode.on.net>:
> > Thank you for the hints, Tony, but nothing tried so far is making any
> > difference. Vim still won't display Cyrillic, Greek, or Russian.
> >
> > But is this a clue? My immediate prior post, containing åæø, came back
> > from the list server with those three characters now displaying in vim
> > as -C3-A5-C3-A6-C3-B8, despite the header: "X-NotAscii: charset=utf-8".
> > (Again, s/-/==/g) In vim, encoding, fileencoding, and termencoding are
> > all utf-8.
> >
> >
> > Is our list server not sending utf-8, despite the header claim?
>
> It is sending a proper UTF-8. You must use a client which correctly
> decodes the message: you should see those equals signs if
> Content-Transfer-Encoding is [quoted-printable][1], but message body
> was not decoded.

OK, so after a post with e.g. Cyrillic characters is correctly displayed
by mutt in a utf-8_capable xterm, these high valued 8-bit characters
become unreadable when the post is opened in vim for composing a reply,
because the MUA only decodes for display. It does not alter the received
email.

...
Thank you for your advice and the links. Both have crystallised the
problem. I apparently now need to either have postfix or procmail invoke
a quoted-printable to utf-8 decoder on all incoming emails, or have mutt
invoke the same when passing the quoted text to vim for composing a
reply.

Erik

Tony Mechelynck

unread,
Feb 10, 2016, 6:13:45 AM2/10/16
to vim...@googlegroups.com
Ah, yes. Vim would presumably display a message in utf-8 with no
trouble if its Content-Transfer-Encoding were 7bit (us-ascii
compatible) or 8bit, but not quoted-printable (everything above
U+007F, even Latin1 accented characters, would be replaced by two or
more =xx groups, with xx being hex digits) and of course a message in
base64 would be even less understandable. You could try pasting from
the mail reader to Vim (using a different program or terminal if
necessary to remember the "* X selection, since IIUC the xterm, unlike
konsole, cannot access the "+ clipboard): that should keep everything
as you see it. But I suppose it wouldn't copy the headers.

Best regards,
Tony.

Erik Christiansen

unread,
Feb 10, 2016, 7:09:20 AM2/10/16
to vim...@googlegroups.com
Yes, I was about to try to employ qprint to translate, but at the last
minute found that a clean utf-8 xterm environment fixes it. Initially,
xterm was the problem, because it's not utf-8. So I tried uxterm, after
all, it's in the manpages. That allowed mutt to display the QP, but
upset vim. I then made the mistake of trying "xterm -en UTF-8" _from_ a
uxterm. That also failed. But zapping the uxterms from .bashrc, and
invoking "xterm -en UTF-8" from an ordinary xterm, I've joined the
civilised world. Queue angel choirs, parting clouds, and sunbeams!

Thanks.

Erik
Reply all
Reply to author
Forward
0 new messages