Re: several questions

[Nick Black]

[ cc to notcurses@ added ]

José Luis Cruz left as an exercise for the reader:
> 1) I see the only option to load a character directly to a cell on its
> creation is limited to 7 bits.
>
> Wouldn't be possible to provide some way to initialize a cell with an
> 8bit char, (or even 4 byte char) without having to give a plane for
> its egcpool?

there are no eight-bit chars in utf-8. any leading byte with a 1
in its MSB is either 2, 3, or 4 bytes. also, whether char is
unsigned or signed is implementation-defined in C, so you can't
safely encode more than CHAR_BITS (guaranteed to be at least 8)
minus one bits into a char.

hrmmm. there are a lot of invalid forms once you move past the
seven bits shared with ASCII-1968. we can just call that a user
error, though, and let them reap the rewards.

i suppose that a CELL_EGC_INITIALIZER accepting a u_int32_t (as
enforced by bare assignment rules, not typing, since it's a
macro) is reasonable enough. yeah, why don't you file a bug on
this? we can probably add it before 2.1.0.

> In Rust there's this 4byte utf-8 encoded char type that would be
> perfect for loading the cell with. Right now the only way is to use
> cell_load on an existing cell, providing an plane that wont be needed.

yep, i think it's a good idea.

> 2) Would it be possible to have a function like ncplane_putc_yx with
> the difference only advances the cursor one column? or nothing at all?
> for manually managing that part.

i'm not sure i understand. you want one that's capped at a
single column? but prints to a destination of arbitrary columns?
just making sure i understand. what exactly is the goal here?

> 3) What would be the equivalent of ncdirect_clear in full mode?
> calling plane_erase for each rendered plane?

you want to clear the actual rendered screen? i'd do:

n = ncplane_dup(notcurses_stdplane(nc));
ncplane_move_top(n);
ncplane_set_base(n, " ", 0, 0);
notcurses_render(nc);

or alternatively:

that puts a plane having the same size as the visual area
(guaranteed by properties of the standard plane, which we
duplicated) at the top of the pile, with all cells set to opaque
default color spaces, which is then rendered. this new plane
obstructs everything. remove it and get everything back. it is
worth noting that one ought supply a resizecb here to keep the
screen cleared despite treacherous geometry changes.

if you really just want to empty every plane, yeah, iterate over
the pile with notcurses_top() or ncpile_top() (the latter of
which i apparently need to write, erp) and ncplane_below(), or
_bottom() and _above() if you feel so urged.

alternatively, make a new pile and render that pile. no resizecb
necessary in this case, nor ncplane_move_top().

> 4) ncplane erase says it resets all colors to the default color, but
> it doesn't seem to change the default mark as reported by
> ncplane_bg_default_p & ncplane_fg_default_p

that's poorly worded on my part. it resets the color of every
*cell* in the plane's framebuffer. the plane's active attributes
are unchanged. i will amend this comment, thanks.

> 5) This is not very important, but I'm curious why you didn't call a
> cell nccell? Is it because its name starts with the letter `c`? I'm
> actually using the Nc prefix for each notcurses type alias in the Rust
> bindings, for the sake of consistency (except for the Notcurses
> object).

The Earth quakes and the heavens rattle; the beasts of nature
flock together and the nations of men flock apart; volcanoes
usher up heat while elsewhere water becomes ice and melts; and
then on other days it just rains. Indeed do many things come to
pass.

`cell` was, like, the first structure defined -- i think before
`struct notcurses` even -- and i had to take care of all those
users of 0.4.0 or whatever.

by the time i realized there were no such users but myself,
there were real users.

when i am king, `cell` will be first against the wall. it
authentically burns me up inside.

> hacking on :)

YASSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSs

https://www.youtube.com/watch?v=z2unV6W8D1s

--
nick black -=- https://www.nick-black.com
to make an apple pie from scratch,
you need first invent a universe.

[Nick Black]

Nick Black left as an exercise for the reader:

> `cell` was, like, the first structure defined -- i think before
> `struct notcurses` even -- and i had to take care of all those
> users of 0.4.0 or whatever.
>
> by the time i realized there were no such users but myself,
> there were real users.
>
> when i am king, `cell` will be first against the wall. it
> authentically burns me up inside.

i encourage you to use nccell, though your language benefits
from proper namespacing and thus it's only an issue of sanity,
not of being unable to compile against any other header from the
past fifty years with `cell`. goddamn i hate myself sometimes.

> > hacking on :)

but then i feel better! hack on, hack on!!!

[Nick Black]

Nick Black left as an exercise for the reader:

> that puts a plane having the same size as the visual area
> (guaranteed by properties of the standard plane, which we
> duplicated) at the top of the pile, with all cells set to opaque
> default color spaces, which is then rendered. this new plane
> obstructs everything. remove it and get everything back. it is
> worth noting that one ought supply a resizecb here to keep the
> screen cleared despite treacherous geometry changes.

actually, ncplane_dup() ought dup the resizecb as well, so you
don't even need to do this. standard plane already has a
resizecb that takes care of exactly this. or it should.
actually, i think that's inlined :(. i ought make it a resizecb,
prohibit you from changing the standard plane's resizecb, and
ensure resizecb is copied in ncplane_dup(). well, that's three
new bugs, huzzah!

[Nick Black]

Nick Black left as an exercise for the reader:

> i suppose that a CELL_EGC_INITIALIZER accepting a u_int32_t (as
> enforced by bare assignment rules, not typing, since it's a
> macro) is reasonable enough. yeah, why don't you file a bug on
> this? we can probably add it before 2.1.0.

oh *wait*, you meant an analogue to `cell_load_char()`, i got
it. ummmm yeah i think i can honestly just redefine it...though
that would technically be an ABI break, since we can't be sure
`char` gets passed at the same width as `uint32_t`. alright,
need to add cell_load_unicode(uint32_t ch). doing so now.

[Nick Black]

Nick Black left as an exercise for the reader:

> [ cc to notcurses@ added ]
>
> José Luis Cruz left as an exercise for the reader:
> > 1) I see the only option to load a character directly to a cell on its
> > creation is limited to 7 bits.
> >
> > Wouldn't be possible to provide some way to initialize a cell with an
> > 8bit char, (or even 4 byte char) without having to give a plane for
> > its egcpool?

oh nevermind you want one that doesn't take struct ncplane*. so
cell_load_char() is taking that ncplane to *release* anything
that might have been previously loaded. we could technically add
something like

cell_load_char_you_neednt_release_this_i_promise()

but i don't love the idea. so since this is C and a macro is
call-by-name, you can technically load a uint32_t via
CELL_CHAR_INITIALIZER() (it gets promoted for htole(), and
htole()'s return value is good for gcluster). so we don't
actually need a CELL_EGC32_INITIALIZER().

but i think adding cell_load_egc32() is still a good idea. it
won't absolve you of needing to pass an ncplane, though, because
otherwise you're having sex with every egcpool that cell has had
sex with, and egcpools are nasty, nasty things. take your dick
right off, an egcpool will.

[Nick Black]

Nick Black left as an exercise for the reader:

> actually, i think that's inlined :(. i ought make it a resizecb,
> prohibit you from changing the standard plane's resizecb, and
> ensure resizecb is copied in ncplane_dup(). well, that's three
> new bugs, huzzah!

i'm about to push a PR that does the latter two. the foremost is
addressed here: https://github.com/dankamongmen/notcurses/issues/1172,
and i will handle it before cutting 2.1.0.

[Nick Black]

José Luis Cruz left as an exercise for the reader:

> What I don't understand is, since the cell already knows whether it's
> using an egcpool, couldn't the cell just take care of cleaning after
> itself if necessary? In theory the plane shouldn't be a necessary
> intermediary for that... What am I missing?

cell has no link to the egcpool, just an offset into it. i only
have 25 bits at my disposal, hardly enough to encode an actual
64-bit address, even if we cheated and aligned everything at say
8 bytes (you'd still need 61 bits). so it can't release the
entry in the egcpool itself, because much like Alice it doesn't
know where it is.

[Nick Black]

Nick Black left as an exercise for the reader:

> cell has no link to the egcpool, just an offset into it. i only
> have 25 bits at my disposal, hardly enough to encode an actual
> 64-bit address, even if we cheated and aligned everything at say
> 8 bytes (you'd still need 61 bits). so it can't release the
> entry in the egcpool itself, because much like Alice it doesn't
> know where it is.

this might seem annoying from a wrappers perspective, but if you
think about how this would actually be used, i'm convinced
you've always got a handle to the ncplane at the ready. you're
presumably reusing a cell (or else just use the static
initializers), and if you used the cell once, presumably you
used it on an ncplane. so that ncplane ought be handy.

now what's no good is the fact that you can't cross planes with
this function, and we can't even reliably detect it as an error;
you just sometimes corrupt the new plane's egcpool, and always
leak from the old plane's. i consider that a major error in my
api design, but don't see how i could do differently without
embiggening the `cell` struct, which i avoid at all costs.

[Nick Black]

José Luis Cruz left as an exercise for the reader:

> My thinking is that I'd prefer access to the underlying cell-grid in
> the most pixel-buffer like fashion... And it can seem wasteful for the
> function that puts a cell at some row,col coordinates to also have to
> move the cursor to a place I've not chosen purposefully for certain
> symbols, so I have to check the return value and then correct,
> multiplying the number of operations by cell, instead of managing all
> of that myself.

well, you "need" check the return value for reasons other than
cursor positioning. you could be at an invalid location to start
with (as an example, print a full row without scrolling enabled;
further prints will fail before emitting anything). you could
fail to find room in the egcpool. you could provide illegally
encoded data. etc.

wcwidth() works pretty well. if you find places where it breaks
down (at least for single characters), imho we ought file those
as bugs upstream against glibc. EGCs make this question more
complex, of course.

moving the cursor is O(1), super cheap. it's literally just
updating two words in the ncplane struct (x and y). i wouldn't
worry about that cost at all.

if you're trying to work against the natural flow of text
layout by hyperexplicit cursor positioning, it's not going to do
what you want, i think. let's say for instance that you emit a
character at [0, 0] and explicitly place another one cell to the
right at [0, 1]. there are four possible outcomes:

* character is one column, and wcwidth() properly identifies it
as one column. everything is happy.
* character is two columns, and wcwidth() properly identifies it
as two columns. the second character will annihilate the first
character, because you can't overlay over half an EGC
(limitation of the terminal, not of us). run the "widestomp"
PoC to see this in action [0].
* character is N>1 columns, and wcwidth() improperly identifies
it as 1. your output will be messed up because two much width
will be emitted on the line, since notcurses thinks you're on
column 2, and really you're on N+1.
* character is 1 column, and wcwidth() improperly identifies it
as N>1. the same thing happens as #2, except it happens
unnecessarily.

> In my mind the new function would be a slightly faster alternative
> available for certain operations that don't need the default cursor
> management. Like being able to turn off a pseudo garbage collector.

there is no way that cursor management is ever going to show up
on a profile. it's just two writes to a memory region you've
already brought into cache. dereferencing and loading up the
next node of a linked list is going to take literally hundreds
of times as many cycles. i will of course reconsider if you can
bring perf data which disagrees with this conclusion, but i'm
just not seeing the win on either usability or perf concerns.

--nick

[0] why are the characters to the right of the 'r' turning
white in the last bit of "widestomp"? that looks like a
bug. a hax0r's work is never done!

[Nick Black]

José Luis Cruz left as an exercise for the reader:

> This makes me think it would be nice to have some kind of C little
> program, like a demo but very simple and focused just on benchmarking
> certain rendering key aspects. That could be easily replicated by the
> different language bindings in order to compare between their stats,
> and see if there are any noticeable slowdowns somewhere that could be
> taken care of.

completely agreed, and if someone writes them up, i'll merge
them. currently i just watch for changes in notcurses-demo runs.
i know what each one exercises, and can usually track down
regressions based off this. when i can't, they become issues
until resolved: https://github.com/dankamongmen/notcurses/issues/1012

> Thanks for the example. I suppose the terminal is also checking each
> character it prints with wcwidth() and fills the background of as
> many cells wcwidth() says.

no, from what i can tell this is *not* what happens, at least
not on all terminals. if it was, we'd be fine. the problem
emerges from the disconnect that we're looking at wcwidth(), and
it's looking at the result from the text rendering engine.

> At some point I know I'll want to play with custom glyphs, using
> custom fonts, displaying custom glyphs stored in the private blocks,
> where wcwidth() doesn't have a say nor can it recognize...
>
> For example this character as a POC:
> "𒐫"
> shows well in alacritty, and xfce4-terminal (not kitty) and if you
> print it with notcurses it overlays the next few characters you print
> next to it. That's an interesting effect that could be something worth
> exploring for games.

it's also its proper size in st, fwiw.

it's interesting in the way that being sawed in half starting at
the anus while being suspended upside down is interesting, for sure.

it's fundamentally true and totally shitty that wcwidth() is
insufficient. you can get further with wcwidth() plus
libunistring, which is where we are now.

i wish i had a better solution here, but i do not, and do not
believe one exists without either (a) asking the terminal for
our cursor location mid-rasterization (unacceptable for several
reasons) or (b) becoming intimate with the font engine (i have
found no acceptable way to do this).

into the valley of misalignment rode the three hundred!

[Nick Black]

José Luis Cruz left as an exercise for the reader:

> > completely agreed, and if someone writes them up, i'll merge
> > them. currently i just watch for changes in notcurses-demo runs.
> > i know what each one exercises, and can usually track down
> > regressions based off this. when i can't, they become issues
> > until resolved: https://github.com/dankamongmen/notcurses/issues/1012
>

> Good, I can do some in Rust. From the top of your mind, what would be
> the aspects to bench first?

[wild cheers]

if we're looking for performance changes from one commit to the
next, i'd say it's the innermost code that would be most useful
to start with:

* ncplane_putc_yx() (all cell-based output flows through this)
* ncplane_putegc_yx() (all direct-supplied output thus flows)
* ncplane_printf() (also exercises ncplane_putstr())
* ncpile_render() / ncpile_rasterize()
(these are very input-dependent, unfortunately)
* ncvisual_render()
(input- and blitter-dependent)

[José Luis Cruz]

> Nick Black left as an exercise for the reader:

> this might seem annoying from a wrappers perspective, but if you
> think about how this would actually be used, i'm convinced
> you've always got a handle to the ncplane at the ready. you're
> presumably reusing a cell (or else just use the static
> initializers), and if you used the cell once, presumably you
> used it on an ncplane. so that ncplane ought be handy.

Yeah, I believe you're right.

> now what's no good is the fact that you can't cross planes with
> this function, and we can't even reliably detect it as an error;
> you just sometimes corrupt the new plane's egcpool, and always
> leak from the old plane's. i consider that a major error in my
> api design, but don't see how i could do differently without
> embiggening the `cell` struct, which i avoid at all costs.

I'm sure I can work with the current state of affairs. And in case of
need, it's always possible to create a higher order abstraction
wrapper, in order to keep track of a bunch of cells associated with a
particular plane, which would be responsible for avoiding those kinds
of errors, among other things.

I'm just going step by step figuring out how all pieces fit together
into something I can imagine myself using painlessly...