[dev] [st] Rendering only half of emoji in curses-based program

[Andrej Nabergoj]

Hello,

I am having an issue with wide characters not rendering correctly in my
curses-based program. This happens only when wide character (e.g. emoji)
is placed on third-to-last column of last row. This doesn't happen in
other terminals. I made sample program: github.com/anabergojzz/try
But I don't know how to further simplify it and find out what is source
of the problem. Can someone have a look?

st version: 0.9.2
libncurses-dev Version: 6.4-4

Kind regards,
Andrej

[Andrej Nabergoj]

The program is so small I don't need github, I apologise.
If you make st window 1 row high and 9 columns wide, so string
"😍😍😍😍x" should fill whole window, last emoji gets cut on half.

gcc st_bug.c -DNCURSES_WIDECHAR=1 -lncursesw

#include <curses.h>
#include <locale.h>
int main() {
setlocale(LC_ALL, "");
initscr();
printw("😍😍😍😍x");
getch();
endwin();
return 0;
}

For the same string without curses there is no problem, so probably is
not st problem, but in other terminals I don't get such behavior.

#include <stdio.h>
int main() {
printf("😍😍😍😍x");
int c = getchar();
return 0;
}

Regards,
Andrej

[Andrej Nabergoj]

wcwidth(L"😍") gives me 2 if you meant this. I tried on another laptop
(but it has debian installed too) and problem was the same.
If I print any character that has wcwidth() = 2 to the third last place
of terminal, only half of the character is rendered. This only happens
if using curses. I'v tried it on debian and on another fedora laptop.
```

#include <curses.h>
#include <locale.h>

int main() {

int last_row = 5, last_column = 25;
setlocale(LC_ALL, "");
initscr();
mvaddstr(last_row, last_column-2, "字");
getch();
endwin();
return 0;
}
```
Thank you for trying to reproduce it.

Andrej

On 30/03/25 05:33, Eric Pruitt wrote:

> On Sun, Mar 30, 2025 at 07:32:14PM +0200, Andrej Nabergoj wrote:
> > The program is so small I don't need github, I apologise.
> > If you make st window 1 row high and 9 columns wide, so string
> > "😍😍😍😍x" should fill whole window, last emoji gets cut on half.
>

> I can't reproduce this. What does wcwidth(3) return for the emoji?
>
> Eric
>
> PS: Sender BCC'd because I have delivery issues with the suckless
> mailing lists.

[Steffen Nurpmeso]

Andrej Nabergoj wrote in
<20250331180240....@debian.si>:

|wcwidth(L"😍") gives me 2 if you meant this. I tried on another laptop
|(but it has debian installed too) and problem was the same.
|If I print any character that has wcwidth() = 2 to the third last place
|of terminal, only half of the character is rendered. This only happens
|if using curses. I'v tried it on debian and on another fedora laptop.
|```
|#include <curses.h>
|#include <locale.h>
|
|int main() {
| int last_row = 5, last_column = 25;
| setlocale(LC_ALL, "");
| initscr();
| mvaddstr(last_row, last_column-2, "字");
| getch();
| endwin();
| return 0;
|}
|```
|Thank you for trying to reproduce it.

Note i have no idea and did not truly look into the issue, but
there is a problem with terminals and the lowermost, rightmost
cell. Ie i have, which names termcap/terminfo capabilities

mx_TERMCAP_QUERY_am, /* am/am, BOOL | auto_right_margin */
mx_TERMCAP_QUERY_sam, /* sam/YE, BOOL | semi_auto_right_margin */
mx_TERMCAP_QUERY_xenl, /* xenl/xn, BOOL | eat_newline_glitch */

But i am (the mailer i maintain, at least, does) not doing this
completely right, as can be done.
However, most programs either do not use the lowermost/rightmost
column to avoid cursor wrap if written there, or otherwise deal
with those capabilities.
I have zero idea on what st does though.

Hope that helps.

|Andrej
|
|On 30/03/25 05:33, Eric Pruitt wrote:
|> On Sun, Mar 30, 2025 at 07:32:14PM +0200, Andrej Nabergoj wrote:
|>> The program is so small I don't need github, I apologise.
|>> If you make st window 1 row high and 9 columns wide, so string
|>> "😍😍😍😍x" should fill whole window, last emoji gets cut on half.
|>
|> I can't reproduce this. What does wcwidth(3) return for the emoji?
|>
|> Eric
|>
|> PS: Sender BCC'd because I have delivery issues with the suckless
|> mailing lists.
|

--End of <20250331180240....@debian.si>

--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)

[Roberto E. Vargas Caballero]

Quoth Steffen Nurpmeso <ste...@sdaoden.eu>:

> Note i have no idea and did not truly look into the issue, but
> there is a problem with terminals and the lowermost, rightmost
> cell. Ie i have, which names termcap/terminfo capabilities

These capabilities are about when the terminal does wrapping. VT100
alike terminals do wrapping when you write past the last column.
It means that when you write the last column the cursor stays in the
last column and it does not moves to the next line. St behaves
like this.

All the logic about wide chars is in the function tputc() in st.c.
You can try run st from gdb, write in st until you arrive to that
column, then set a breakpoint in tputc() using gdb and see what
happens when you write that character. It is likely that some
miscalculation is done.

Regards,

[Andrej Nabergoj]

The problem is not with lowermost, rightmost cell but one that is
two positions before rightmost, lowermost cell.
I looked at tputc() function, but I don't know yet how to use these
debugging tools.
If I comment this two lines for example it renders whole character, but
of course this is not a solution:
if (width == 2) {
/* gp->mode |= ATTR_WIDE; */
if (term.c.x+1 < term.col) {
if (gp[1].mode == ATTR_WIDE && term.c.x+2 < term.col) {
gp[2].u = ' ';
gp[2].mode &= ~ATTR_WDUMMY;
}
gp[1].u = '\0';
/* gp[1].mode = ATTR_WDUMMY; */

}
}

#include <curses.h>
#include <locale.h>
int main() {

int row, col;
setlocale(LC_ALL, "");
initscr();
getmaxyx(stdscr, row, col);
mvprintw(row-1, col-3, "字");
getch();
endwin();
return 0;
}

I will try to learn how to use gdb and give it another try when I have
time.

Thank you to both.
Regrads,
Andrej

[Steffen Nurpmeso]

Andrej Nabergoj wrote in
<20250401210626....@debian.si>:

|The problem is not with lowermost, rightmost cell but one that is
|two positions before rightmost, lowermost cell.

Of course, if you write a wide character that takes multiple
cells, we end up exactly there again.

|I looked at tputc() function, but I don't know yet how to use these
|debugging tools.
|If I comment this two lines for example it renders whole character, but
|of course this is not a solution:

Hmmm, i never looked there, but i see quite a bit of
"term.c.x+width < term.col", but also "term.c.x+width > term.col",
which is a bit odd, shouldn't it be >= in the latter, then? From
the logical-only side of the road.
This is shortly before this code you code me thinks

|if (width == 2) {
| /* gp->mode |= ATTR_WIDE; */
| if (term.c.x+1 < term.col) {
| if (gp[1].mode == ATTR_WIDE && term.c.x+2 < term.col) {
| gp[2].u = ' ';
| gp[2].mode &= ~ATTR_WDUMMY;
|}
| gp[1].u = '\0';
| /* gp[1].mode = ATTR_WDUMMY; */
|}
|}

Ie i mean

if (term.c.x+width > term.col) {
if (IS_SET(MODE_WRAP))
tnewline(1);
else
tmoveto(term.col - width, term.c.y);
gp = &term.line[term.c.y][term.c.x];
}

What happens if you use ">= term.col"?

...

[Roberto E. Vargas Caballero]

Quoth Andrej Nabergoj <andrej....@siol.net>:

> For the same string without curses there is no problem, so probably is
> not st problem, but in other terminals I don't get such behavior.
>

Ok, this is an important point. It is possible that some terminfo
capability has some effect on this topic. There is the option -o
in st that allows you to get a full copy of the input stream and
then you can see if there is something else happening that you cannot
see.

Regards,

[Dave Blanchard]

My suggestion is to avoid the featureless, buggy, uncommented 'st' garbage code and use a real terminal emulator, like rxvt or xterm.

Dave

[Andrej Nabergoj]

Now I understand a little what terminal does and what programs are
responsible for. So if I use this example curses program and save output
to file (with this option -o) I get this additional escape sequences
after wide character (backspace, space,...) but not if I draw it on
positions other than (row-1, col-3).

Part of output: " [52;100H字 [1@ [C"

Example program:

#include <curses.h>
#include <locale.h>
int main() {

int row, col;
setlocale(LC_ALL, "");
initscr();
getmaxyx(stdscr, row, col);

mvprintw(row-1, col-3, "字");
getch();
endwin();
return 0;
}

Do you have any idea why? What should I look at now?
Thank you!

Regards,
Andrej

[Andrej Nabergoj]

ich capability was responsible for this behavior.
If I add if clause to do nothing if it is dummy char in tinsertblank() function
everything I tested works same as in other terminals.

if (!(term.line[term.c.y][term.c.x].mode & ATTR_WDUMMY)) {
memmove(&line[dst], &line[src], size * sizeof(Glyph));
tclearregion(src, term.c.y, dst - 1, term.c.y);
}

Is there any way to test if this is ok, I don't know how that works.