[scintilla:bugs] #2497 Rectangular selection indent incorrect selection modes and positions

[Michael Heath]

---

[bugs:#2497] Rectangular selection indent incorrect selection modes and positions

Status: open
Group: Bug
Labels: scintilla
Created: Thu Jan 08, 2026 02:02 AM UTC by Michael Heath
Last Updated: Thu Jan 08, 2026 02:02 AM UTC
Owner: nobody
Attachments:

• scintillaRectIndentUpDown.zip (80.7 kB; application/x-zip-compressed)

• Windows 11

• ScITE: 5.5.8 Scintilla: 5.5.8 Lexilla: 5.4.6

The issue is based on:

BUG Incorrect tab behavior in column mode

caused by a change-set committed at

Remove unnecessary InvalidateStyleRedraw() for Message::SetUseTabs

Symptoms:

• Carets can move to incorrect positions.
• Undo changes selection mode so Redo is different.

---

Note: Preview of code converts tabs to spaces. I expect the results to remain the same.

1. Test start of rectangle at column 0:

-
    -

1. Rectangle select downwards and tab to indent result

    |-
    |   -

The pipe character represent the carets.
The change-set sets a thin selection.

1. Reset to step 1. Rectangle select upwards and tab to indent result.

    -|
        |-

The 1st caret is 1 position too far.

Another symptom from the change-set is undo reverts the selection mode to stream mode as the indenting was done with stream mode. So undo and redo does this with the select downwards continuing with step 2.

    |-
        |-

The undo seems to forget the rectangle mode and remembers the stream mode so the result is different.

I have attached possible fix. I tested the IDM_HELP code in SciTE and it seems to pass OK.

---

Sent from sourceforge.net because scintill...@googlegroups.com is subscribed to https://sourceforge.net/p/scintilla/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/scintilla/admin/bugs/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Neil Hodgson]

The patch appears to me to change behaviour for too many situations (any call to SetRectangularRange) to be safe.

A change restricted to Editor::Indent is less likely to cause new problems.

A branch inside Editor::Indent that specializes the rectangular case explicitly should allow tighter control. Operating on a copy of the initial selection could avoid interference from rectangular selection recalculations. This would likely duplicate the lengthy calculations for each selection although the specific issue (tab key) is just for Indent(true, false) which is a shorter subset of the code.

[Neil Hodgson]

The example shows the effect of trying to maintain selection rectangularity over the indent operation.

The current indent code is mostly designed to handle stream selection. It chooses the end of indentation as the best place to position the caret afterwards as it simplifies some common actions like adding spaces for fine control over indentation or adding to the line text. This choice is more advantageous when there are multiple selections.

In the example, the lines initially get empty selections after indentation: after one tab for line 1 and after two tabs for line 2. Then these end points are converted to a thin rectangular selection. Calculating rectangular ranges is asymmetric to match the direction of dragging when performed by mouse. For a thin selection, the anchor is chosen so a forward selection chooses the column after 1 tab and a reverse selection chooses the column after 2 tabs, leading to selecting after the '-' on line 1.

The purpose of the SelTypes::thin mode is to produce selections that contain no text so are suitable for following operations, The patch presented here may create non-empty selections so is not really upholding the SelTypes::thin mode.

As mentioned on the earlier issue, the best solution here is probably a separate rectangular code path that achieves more rectangular results, similar to the multi-line path.

A smaller change would be to first abandon rectangular mode, converting the selection to multiple stream selections before indenting each of these.

---

[bugs:#2497] Rectangular selection indent incorrect selection modes and positions

Status: open
Group: Bug
Labels: scintilla
Created: Thu Jan 08, 2026 02:02 AM UTC by Michael Heath

Last Updated: Thu Jan 22, 2026 01:16 AM UTC
Owner: nobody
Attachments:

[Zufu Liu]

• labels: scintilla --> scintilla, selection, indentation, rectangular

---

[bugs:#2497] Rectangular selection indent incorrect selection modes and positions

Status: open
Group: Bug
Labels: scintilla selection indentation rectangular

Created: Thu Jan 08, 2026 02:02 AM UTC by Michael Heath

Last Updated: Sat Jan 24, 2026 12:50 AM UTC
Owner: nobody
Attachments: