Copy/Paste tab handling

[Andrew Athan]

The behavior I'm going to describe is not necessarily a bug or problem with iterm2 but it might be.

First, let me describe the activity:

(1) open an emacs buffer and edit some code that uses tabs for indentation

(2) copy/paste some of that code into another app, like slack, into a fixed-width-font area (e.g., a code block ```  ```)

Observe:

Sometimes, you end up with wrong indentation in the destination.

I believe the "wrong indentation" is because the paste uses spaces for the indentation of most lines, but certain lines contain actual tab characters.

Obviously, iterm2 can't know the contents of the file being displayed by emacs and must have a model of what's displayed on screen. If I were to "cat" a text file that contains tabs and then copy/paste the contents of the terminal, I might reasonably expect to get tabs copy/pasted. On the other hand, it is also reasonable to expect that I get one space for each displayed fixed-width blank area in the terminal region I'm copying from.

I also can't expect iterm2 to know how I expect it to deal with tabs vs spaces in its copy/paste buffer when pasting. It has an "Advanced Paste..." menu and a "Prompt to convert tabs to spaces" feature -- but these are useful only when pasting into iterm2.

It certainly would be nice to be able to affect the paste behavior when pulling data out of iterm2 as well. E.g., by telling it to prepare the paste buffer so that it reflects the physical layout by having represented all whitespace as spaces.

Separately, there's the question of why iterm2 only sometimes (vs consistently) thinks there's a tab in the indentation whitespace of a displayed emacs buffer. It must have to do with what emacs emits to draw the screen.

When I first load a buffer that is tabified, the copy/paste output from iterm2 seems to consistently NOT include tabs. *IF* you then switch to another buffer (e.g., using `electric-buffer-list`) and *IF* there is a blank line in the destination buffer then, when you switch back to the original buffer, and *IF* there is a tabified indented line in the original buffer that is at the same physical line location where the blank line was *THEN* iterm2 will end up with an actual physical tab in the paste buffer if you try to copy/paste that region.

I don't know if maybe emacs is emitting a tab vs a space to draw a blank on the previously empty line. It seems iterm2 believes there to be only one tab on the line, adjacent to the left border of the terminal. The rest of the whitespace on that line is copy/pasted as spaces.

So, it seems that something about the terminal sequences emitted by emacs to redraw the original buffer, filling in the blank line with the original tabified source, causes a single physical tab to exist in iterm2's model of what's on screen. The location of the cursor does not seem to be relevant.

Thoughts?

[George Nachman]

I suspect that emacs is using spaces when drawing instead of tabs to position the cursor. Information is often lost when an interactive app like emacs draws to the screen. You could make a session log to try to figure out if this is actually what’s going on.

It would certainly be possible to convert spaces to tabs when copying. You’d probably end up with more spaces than you want, though. If a space between two words happens exactly one cell before a tabstop, there’s no way to tell what it “should be”. Maybe a better algorithm would be to convert leading spaces to tabs?

--
You received this message because you are subscribed to the Google Groups "iterm2-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iterm2-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/iterm2-discuss/cea7fb5c-e0fa-446b-9d2a-8669b14aa39cn%40googlegroups.com.

[Andrew Athan]

First, I want to point out what could be an "inversion" of the problem in your mind. You stated "I suspect that emacs is using spaces when drawing instead of tabs to position the cursor."

The issue is that for most copy/pastes the paste yes does have spaces, but for some, after a particular kind of redraw, there is a leading tab instead of a leading space, prior to additional spaces, representing the whitespace indentation of certain lines.

In other words the issue is emacs using a tab *instead of a space* to clear some portions of the screen during certain redraws. At least, that's my suspicion.

Thinking about tab rendering in an app like emacs, where we could specify any tab-width: I'm assuming that it must be emitting cursor positioning commands to draw text at the configured tab stop locations (vs say emitting a tab and expecting the terminal to put the cursor in the correct final position). It's been a long time since I coded a terminal emulator, but I don't remember tab-width setting sequences. So, I assume emacs probably calculates the locations internally and then moves the cursor physically to the right spot (vs emitting the tab).

However, my further guess is that during certain redraws emacs ends up sending a tab to clear out a sequence of non-whitespace characters -- perhaps relying on "tab = 8 space standard" for example, because that's more efficient than emitting a bunch of spaces. Dunno. Anyway, the end result is some lines that have a "TAB SPACE SPACE SPACE" content...

If my assumption about how apps draw the screen is correct, then the physical layout of the text as displayed (with one space per on-screen blank area) would, in a fixed width font, correctly replicate the indentation. Whereas, using tabs would require knowing the tab width, and would further require that tabs-as-calculated-by-reading-out-the-line are correctly positioned.

That's why I suggested that an option to convert the paste to "spaces as laid out on screen" is probably the best you could do??

A.

[George Nachman]

> In other words the issue is emacs using a tab *instead of a space* to clear some portions of the screen during certain redraws. At least, that's my suspicion.

Emacs could use either tabs or spaces to position the cursor. I believe emacs uses ncurses, and that seems like a reasonable optimization for it to use.

> I don't remember tab-width setting sequences

There are control sequences to set tab stops. ESC H sets a tab stop at the cursor position. CSI g removes tab stops. Regardless of whether they change them, they could use tabs stops to slightly reduce the number of bytes needed to move the cursor.

> emacs ends up sending a tab to clear out a sequence of non-whitespace characters

Tabs move the cursor but won't erase any cells. It might tab over existing spaces, would would leave the spaces intact.

At any rate, I think it makes sense to convert tabs to spaces when copying when in alternate screen mode. Alternate screen mode is a signal that you're in an interactive app, and I suspect that you can almost never count on tabs to be meaningful then.

To view this discussion on the web visit https://groups.google.com/d/msgid/iterm2-discuss/a792bb96-0cdd-4af8-af4a-641678b35ce9n%40googlegroups.com.