Discontiguous selection and virtual space

[Neil Hodgson]

As mentioned in the column/rectangular editing thread, I thought
the concept should be generalised to implement discontiguous
selection. This led to moving the selection state out into a new
Selection class. By mediating all selection operations through a
Selection class it appeared possible to implement virtual space which
has been on the to do list for a very long time. Virtual space is
being able to move the caret beyond the end of line without placing
actual space characters in the document. Some people want it for just
that but it also makes rectangular selection simpler since you can
mark an area that contains real content on longer lines and virtual
space on shorter lines.

To implement virtual space selections, each selection position
contains both a location between two bytes in the document and a
number of virtual spaces. Virtual spaces are only valid where the
location is at a line end. So, using a notation similar to imaginary
numbers, when a document's first line contains one character, the end
of the line is 1+0v and one virtual space beyond the line end is 1+1v.
A selection may be empty (just a caret) with the range (1+1v..1+1v) or
may be three characters wide with (1+1v..1+4v) and may contain both
some real and some virtual contents (0+0v..1+4v). The
SelectionPosition class (currently in PositionCache.h but soon to move
to a new Selection.h) contains a position and number of virtual
spaces. The SelectionRange class contains two SelectionPosition
objects for the anchor and caret of a range.

The Selection class builds on these by containing an ordered list
of SelectionRange objects. One range is the main range and this is
normally the one most recently defined although it can be changed by
code. The main range is the most important range used when deciding
which area of the document is to be displayed automatically to follow
the users actions. The main range is displayed with slightly more
emphasis (more alpha) than secondary ranges although the display
characteristics are likely to change and be user definable. Secondary
carets blink although that may be turned off. Additional selections
are made by dragging with the Ctrl key pressed. There is not yet any
way to move the caret using the keyboard without dropping the existing
selection so this feature can not be driven completely from the
keyboard.

When the user drags out a rectangular selection, this is converted
into a list of SelectionRange objects so it can be treated mostly with
the same code as discontiguous selections. The rectangular area and
state are remembered as some commands (such as Alt+Shift+Down) operate
by modifying the rectangular area.

When applications use the current API, most of the selection calls
map to the main range. Since applications were written assuming that
there is only a single selection with no virtual spaces, applications
must turn on discontiguous selection and virtual space before these
features will work. For virtual space there are two options - one for
allowing virtual space only in rectangular selections and one for
other cases. There is a new API (listed at end) so that applications
can fully examine and modify the new Selection class.

This is a really large change in terms of code modified since
selection is very central to Scintilla with a couple of thousand lines
or more added or modified. There is current code available from
http://www.scintilla.org/scite-vs.zip Source
http://www.scintilla.org/wscite-vs.zip Windows executable
This code does not yet compile on GTK+ or other platforms. For
GTK+, it would be great to work out what is stopping use of Alt+Drag
for rectangular selection so that the same keys can be used on Windows
and GTK+ even if it means users have to change their desktop
configurations. Otherwise it may need to use another modifier such as
the Windows key. SciTE was modified in the download so that the "Mark
All" command in the Find dialog selects all the matches. Thus you
could find and select all identifiers starting with 'sci_' and use the
Edit | Make Selection Uppercase command (also modified to work with
multiple selections) to capitalise them.

Since this changes so much code, there are bugs and some features
will not work well. Therefore, I want a good period with this code
available before a release that could freeze the API with mistakes
that would be hard to eliminate later. So there is going to be a
release 1.79 fairly soon with just the current code and then this
feature will be committed. Editor.cxx will also be frozen to other
changes until multiple selection is stable.

API:

## The 'MultiLine' here should really be something like 'MultiSelection'
# Set whether the caret will show on multiple lines for a rectangular selection
set void SetMultiLineCaret=2563(bool multiLine,)

# Whether the caret will show on multiple lines for a rectangular selection
get bool GetMultiLineCaret=2564(,)

# Set whether the multiline caret will blink
set void SetMultiLineCaretBlinks=2565(bool multiLineBlinks,)

# Whether the multiline caret will blink
get bool GetMultiLineCaretBlinks=2566(,)

# How many selections are there?
get int GetSelections=2570(,)

# Add a selection
fun void ClearSelections=2571(,)

# Set a simple selection
fun int SetSelection=2572(int currentPos,int anchor)

# Add a selection
fun int AddSelection=2573(int currentPos,int anchor)

# Set the main selection
set void SetMainSelection=2574(int selection,)

# Which selection is the main selection
get int GetMainSelection=2575(,)

set void SetSelectionNCaret=2576(int selection, position pos)
get position GetSelectionNCaret=2577(int selection,)
set void SetSelectionNAnchor=2578(int selection, position posAnchor)
get position GetSelectionNAnchor=2579(int selection,)
set void SetSelectionNCaretVirtualSpace=2580(int selection, int space)
get int GetSelectionNCaretVirtualSpace=2581(int selection,)
set void SetSelectionNAnchorVirtualSpace=2582(int selection, int space)
get int GetSelectionNAnchorVirtualSpace=2583(int selection,)

# Sets the position that starts the selection - this becomes the anchor.
set void SetSelectionNStart=2584(int selection, position pos)

# Returns the position at the start of the selection.
get position GetSelectionNStart=2585(,)

# Sets the position that ends the selection - this becomes the currentPosition.
set void SetSelectionNEnd=2586(position pos,)

# Returns the position at the end of the selection.
get position GetSelectionNEnd=2587(,)

set void SetRectangularSelectionCaret=2588(position pos,)
get position GetRectangularSelectionCaret=2589(,)
set void SetRectangularSelectionAnchor=2590(position posAnchor,)
get position GetRectangularSelectionAnchor=2591(,)
set void SetRectangularSelectionCaretVirtualSpace=2592(int space,)
get int GetRectangularSelectionCaretVirtualSpace=2593(,)
set void SetRectangularSelectionAnchorVirtualSpace=2594(int space,)
get int GetRectangularSelectionAnchorVirtualSpace=2595(,)

enu VirtualSpace=SCVS_
val SCVS_NONE=0
val SCVS_RECTANGULARSELECTION=1
val SCVS_USERACCESSIBLE=2

set void SetVirtualSpaceOptions=2596(int virtualSpace,)
get int GetVirtualSpaceOptions=2597(,)

Neil

[Neil Hodgson]

Committed the new selection code to CVS. Made work on GTK+. Use
the position of the caret for features like the Input Method Editor
(input for languages like Japanese) even when the caret is in virtual
space.

Available from CVS and from

http://www.scintilla.org/scite.zip Source
http://www.scintilla.org/wscite.zip Windows executable

Neil

[haimag ren]

Not take into account backspace operation at start of line.

2009/7/3 Neil Hodgson <nyama...@gmail.com>

[Neil Hodgson]

haimag ren:

> Not take into account backspace operation at start of line.

While the most commonly wanted behaviour is to not apply backspace
at line start, there are use cases for allowing it. Say I have a
directory listing and want to turn it into a list, the following set
of images show the sequence:

1) Initial listing
2) Select start of lines
3) Backspace removes line ends to make a single line
4) Type ", " to make a list

Unsure what to do here as it its not really something I'd want to
be a global default, more a per-action choice.

Neil

[Philippe Lhoste]

On 04/07/2009 02:00, Neil Hodgson wrote:
> 1) Initial listing
> 2) Select start of lines
> 3) Backspace removes line ends to make a single line
> 4) Type ", " to make a list

Very interesting use case, original and useful.
Funnily, it would answer exactly the question asked at:
http://stackoverflow.com/questions/1062423/extend-notepad

--
Philippe Lhoste
-- (near) Paris -- France
-- http://Phi.Lho.free.fr
-- -- -- -- -- -- -- -- -- -- -- -- -- --

[Neil Hodgson]

There have been quite a few additions to this code with some
discussion on the scite-interest group. Its now got almost all
expected functionally and works reasonably well although there are
still bugs and behaviour may not be as everyone wants.

New version available from CVS and from

http://www.scintilla.org/scite.zip Source
http://www.scintilla.org/wscite.zip Windows executable

New APIs:

# Set the foreground colour of additional selections.
# Must have previously called SetSelFore with non-zero first argument
for this to have an effect.
set void SetAdditionalSelFore=2600(colour fore,)

# Set the background colour of additional selections.
# Must have previously called SetSelBack with non-zero first argument
for this to have an effect.
set void SetAdditionalSelBack=2601(colour back,)

# Set the alpha of the selection.
set void SetAdditionalSelAlpha=2602(int alpha,)

# Get the alpha of the selection.
get int GetAdditionalSelAlpha=2603(,)

# Set the foreground colour of additional carets.
set void SetAdditionalCaretFore=2604(colour fore,)

# Get the foreground colour of additional carets.
get colour GetAdditionalCaretFore=2605(,)

Neil

[Neil Hodgson]

Another update to the selection code.

Backspace will not delete line starts when selection is rectangular
so it is safe to hold down Backspace without a chance of joining lines
together unexpectedly. Duplicate works on discontiguous selections by
duplicating each selection. Pastes can be made into virtual space on
Windows. An UndoGroup class simplifies grouping actions together in
the undo history.

  Available from CVS and from

http://www.scintilla.org/scite.zip Source
http://www.scintilla.org/wscite.zip Windows executable

Neil

[Neil Hodgson]

To allow access to both rectangular selection and discontiguous
selection on GTK+ where the Ctrl key has previously been used for
rectangular selection, a new setting allows a choice of rectangular
selection modifier key.

GTK+ is designed to cope with a wide range of keyboard layouts even
though it is most commonly used with a Windows keyboard now. The use
of modifier keys is complex with multiple layers of settings. GTK+
defines Shift (GDK_SHIFT_MASK), Caps Lock, and Ctrl modifiers before
giving up and having 5 more modifiers (GDK_MOD1_MASK ..
GDK_MOD5_MASK). These get names like <Super>, <Hyper>, and <Alt>
somewhere in the software stack but they aren't definitively
associated with any particular key. Alt is commonly associated with
GDK_MOD1_MASK so Scintilla uses this in keystroke definitions.

GDK_MOD4_MASK is <Super> on the Fedora and Ubuntu distributions I
checked with Mac and Windows keyboards. On the Mac keyboard it
corresponded with the Command key and on the Windows keyboard it was
the Left Windows (Start) key. I couldn't find other GDK_MOD* keys in
these distributions with default installations. The Hardware|Keyboard
control panel can be used to change the mapping to physical keys in
the Layout Options dialog in the Layouts pane.

The new API is SCI_SETRECTANGULARSELECTIONMODIFIER and it may be
called with SCMOD_CTRL(default), SCMOD_ALT, or SCMOD_SUPER where
SCMOD_SUPER maps to GDK_MOD4_MASK. This is the only place SCMOD_SUPER
currently works - it can not be used to define key commands. From
SciTE this is controlled by the property
rectangular.selection.modifier with values 2 (Ctrl), 4 (Alt) and 8
(Super).

The default mapping to Ctrl does not allow the user to perform
discontiguous selection. Choosing Super will work on many standard
distributions. My preference is for Alt since that matches Windows
behaviour but that requires configuring the window manager to not
intercept Alt + Mouse Drag. With the Metacity window manager used by
Ubuntu and Fedora, this can be changed using gconf-editor.
gconf-editor was already present on my Ubuntu installations but for
Fedora, it required: "yum install gconf-editor" first. Inside
gconf-editor use the tree and list to choose the
/apps/metacity/general/mouse_button_modifier property. This can be set
to <Super> (from <Alt>) to use the Super key for windows manager
operations, thus allowing Alt through to Scintilla.

Changed GTK+ code to allow pasting into virtual space.

Changed the additional selection style code so that more reasonable
values are chosen when an application only sets the corresponding main
selection values. Setting the main selection alpha also sets the
additional selection alpha. In SciTE, the additional selection alpha
defaults to half the main selection alpha. On GTK+, the primary
selection style is only applied to the main selection, not the
additional selections.

[Neil Hodgson]

Another update to the selection code. Fixed drawing problems with
line ends particularly when in wrapped mode. Added controls so that
the new features must be turned on to prevent failure of applications
that were designed without knowledge of the new features and also to
allow users to select which features they want. Some of these have
changed names from previous test versions because they now deal with
multiple selections rather than only with rectangular selections.

SCI_SETMULTIPLESELECTION
selection.multiple
Can multiple selections be made by using the Ctrl key when dragging the mouse?

SCI_SETADDITIONALSELECTIONTYPING
selection.additional.typing
Can multiple selections be typed into? Includes Backspace and Delete key use.

SCI_SETADDITIONALCARETSBLINK
caret.additional.blinks
Do additional caret blink?

Other code cleaned up and documentation written. Built on Windows,
GTK+, and Mac.

[Robert Roessler]

I was using 1.79, and noticed some seriously weird (unexpected) cursor-
positioning behavior - it looked like something the world calls
"virtual spaces" had been implemented, but ScintillaDoc.html had zero
to say on the subject. Since I knew that SciTE did not exhibit this
behavior, I checked SciTEDoc.html, and there it was! So I backtracked
into Scintilla, found the feature and how to use (i.e., disable) the
feature... ;)

BTW, the latest version of ScintillaDoc.html has an entry for the "set
virtual space options" capability, but it is wrong, insofar as the
stated default setting - as evidenced by my observed behavior...
further investigation revealed that Editor.cxx (still in the latest
CVS version) MAKES the documentation wrong:

virtualSpaceOptions is set twice in succession, first to SCVS_NONE,
and then to SCVS_USERACCESSIBLE - on the next line. ;)

Presumably the second assignment should be disappeared.

[Neil Hodgson]

Robert Roessler:

> I was using 1.79, and getting the weirdest cursor positioning
> behavior... it seemed like something that is usually called "virtual
> space" out in the world had been implemented, but I could find zero
> mention of it in ScintillaDoc.html. ;)

That is new code and should only be in CVS or development
downloads. There should be some documentation in the CVS version of
ScintillaDoc.html.

> ... reconnect, I thought I needed to use a gmail account - which
> apparently is not the case - how can I remedy this - I really want to
> see the group traffic on my non-gmail email account?), and saw all of
> this.

Replied off-list.

> BTW, the ScintillaDoc.html statement about the default condition for
> the "virtual space options" is incorrect (evidenced by the obviously
> virtual-space-enabled behavior I observed)... but it appears to be a
> code error in Editor.cxx which MAKES the documentation wrong. ;)

It was experimental code from before adding the property.

Neil

[Neil Hodgson]

When both Shift and Ctrl were down when the mouse was clicked, a
second empty selection was made. This has now been changed to
prioritize the Shift key so that this combination extends the current
selection which seems less confusing.

I am reasonably happy now with the design of these features. While
there may be bugs and there are many more commands that could be
implemented for multiple selections, the current techniques appear
sound. If anyone thinks this is not the case then it would be best to
comment now before this is included in a release.

I will be unfreezing core Scintilla soon so that other changes can
be committed. People sending in patches or changed files should use
the version now in CVS or the scite.zip download rather than 1.79 due
to the extensive changes.

[Mike Lischke]

Hey Neil,

>
> When both Shift and Ctrl were down when the mouse was clicked, a
> second empty selection was made. This has now been changed to
> prioritize the Shift key so that this combination extends the current
> selection which seems less confusing.

So how is the complete work flow regarding selections now? This is how
I implemented it on other occassions:

1) Click, no modifier: clear selection, set (primary) selection anchor
to clicked char cell.
2) Click + shift: extend main (first, primary...) selection part from
primary selection anchor to clicked cell, clear any other selection.
3) Click + control: set secondary selection anchor to clicked char cell.
4) Click + shift + control: extend secondary selection from current
secondary selection anchor to clicked cell.
5) Move caret via other means (e.g. keyboard input), no modifier:
clear selection, move primary selection anchor to caret pos.
6) Move + shift: like 2)
7) Move + control: no change to caret or selection (actually, I used
this for scrolling the content).
8) Move + shift + control: like 7).

Alternatively one could use the control modifier to move the secondary
anchor, instead scrolling content (if there is no other assigned
command for that input) and then use shift + control like in point 4),
though I have never seen such behavior in an application.

Mike

[Neil Hodgson]

Mike Lischke:

> 1) Click, no modifier: clear selection, set (primary) selection anchor
> to clicked char cell.

All selections are dropped. New main selection created with both
anchor and caret at clicked inter-character position.

> 2) Click + shift: extend main (first, primary...) selection part from
> primary selection anchor to clicked cell, clear any other selection.

Extend main selection so that caret is at click position.
Additional selections are left intact.

To me dropping other selections here would be surprising.

> 3) Click + control: set secondary selection anchor to clicked char cell.

Create new main selection with anchor and caret at position.
Previous main selection is now an additional selection.

> 4) Click + shift + control: extend secondary selection from current
> secondary selection anchor to clicked cell.

Same as Click + Shift.

> 5) Move caret via other means (e.g. keyboard input), no modifier:
> clear selection, move primary selection anchor to caret pos.

Same as 1: move both anchor and caret to new position.

> 6) Move + shift: like 2)

Same as 2.

> 7) Move + control: no change to caret or selection (actually, I used
> this for scrolling the content).

Ctrl + Up, Ctrl + Down bound to scroll. Ctrl + Left, Ctrl + Right
bound to word movement.

> 8) Move + shift + control: like 7).

Ctrl + Shift + Left, Ctrl + Shift + Left are word selection commands.

> Alternatively one could use the control modifier to move the secondary
> anchor, instead scrolling content (if there is no other assigned
> command for that input) and then use shift + control like in point 4),
> though I have never seen such behavior in an application.

Using the current API, it is possible to change the main selection
and to swap the caret and anchor of a selection so an application
could allow for all selections to be modified by binding [make next
selection main] and [swap main caret and anchor] commands. I'm
uncertain whether this is sufficiently useful to be bound to any keys
by default.

Neil

[Philippe Lhoste]

On 20/07/2009 00:48, Neil Hodgson wrote:
>> 2) Click + shift: extend main (first, primary...) selection part from
>> primary selection anchor to clicked cell, clear any other selection.
>
> Extend main selection so that caret is at click position.
> Additional selections are left intact.
>
> To me dropping other selections here would be surprising.

That's the behaviour of Windows' ListView, eg. in Windows Explorer.
Just FYI, both behaviour seems OK, even if I am more used to ListView
one (although Neil's one might indeed seem more natural).

Neil, the short list you made leaves me perplex. Is is possible to do
discontiguous selections with keyboard?

[Neil Hodgson]

Philippe Lhoste:

> Neil, the short list you made leaves me perplex. Is is possible to do
> discontiguous selections with keyboard?

It is not possible to create a new selection just with the keyboard
apart from the rectangular selection case.

A keyboard technique would need to move the caret without selecting
which means a mode or yet another modifier key or something similar.
The way I like currently is having a locked or sticky state for
selections so you press the lock-all-selections command then move the
caret and then shift move to select. Selections that are locked would
have a different visual state possibly boxed or underlined as there
are already three different selection states (on X, on Windows there
are only 2) normally indicated by background colour. Then you need to
work out what the duration of locking is because you don't want the
user stuck in a state. Separate lock and unlock commands make it
easier to be certain of what a command will do rather than a single
context-sensitive command. You may also want to unlock after any
command that uses the selection.

The current implementation doesn't allow selections to overlap. An
early version did and it was very confusing. So when moving the caret
separate from the selections, it should move over any other
selections.

I don't think that keyboard control will be in the initial release
- it may be an idea to watch what downstream projects do before adding
it to standard Scintilla.

Neil

[Jason Haslam]

I'm trying out discontiguous selection now and it looks great! The
only thing that didn't quite match my expectations was Ctrl + Alt +
Drag. I expected this combination to add an additional rectangular
selection. Instead it clears previous selections. Is there a switch
to turn this on somewhere?

Jason

[Neil Hodgson]

Jason Haslam:

Rectangular selection has some extra functionality over multiple
selection including copying and pasting as a rectangular block. If you
add a rectangular block to another selection then the set of
selections is not really rectangular. The discontiguous selection
feature could allow for a collection of rectangular and
non-rectangular pieces but that would be more complex and could not
reasonably enable the rectangular functionality.

An alternative would be to just add the lines of the rectangular
selection to the set of selection pieces without the is-rectangular
state. This doesn't fit into the current implementation well as the
rectangular selection completely defines the selection pieces and this
would have to be changed to preserve the pieces selected before
starting this rectangular selection.

Probably won't be done, at least for the initial release.

Neil

[Jason Haslam]

On Jul 21, 2009, at 7:53 PM, Neil Hodgson wrote:

> Rectangular selection has some extra functionality over multiple
> selection including copying and pasting as a rectangular block. If you
> add a rectangular block to another selection then the set of
> selections is not really rectangular. The discontiguous selection
> feature could allow for a collection of rectangular and
> non-rectangular pieces but that would be more complex and could not
> reasonably enable the rectangular functionality.
>
> An alternative would be to just add the lines of the rectangular
> selection to the set of selection pieces without the is-rectangular
> state. This doesn't fit into the current implementation well as the
> rectangular selection completely defines the selection pieces and this
> would have to be changed to preserve the pieces selected before
> starting this rectangular selection.
>
> Probably won't be done, at least for the initial release.

Yeah, I can see the difficulty with copy/paste. The most interesting
use case for me is actually to add another line of carets. Oh well,
just wanted to make sure I wasn't missing something. Thanks!

Jason

[Simon Steele]

Hi Neil,

2009/7/19 Neil Hodgson <nyama...@gmail.com>:

> I am reasonably happy now with the design of these features. While
> there may be bugs and there are many more commands that could be
> implemented for multiple selections, the current techniques appear
> sound. If anyone thinks this is not the case then it would be best to
> comment now before this is included in a release.

I've started playing with discontiguous selections, and the first
thing that tripped me up after enabling them is that combining
GetSelectionStart/GetSelectionEnd (to get the selection length) with
GetSelText isn't safe. The selection details that come back are for
the first selection of multiple, but GetSelText appears to provide the
text for all - or at least calling these in combination resulted in me
writing over the end of my buffer. I haven't investigated further yet,
as I worked around this case by avoiding doing GetSelText with
multiple selections, but it seemed to me that if the old functions
report just the first selection then GetSelText should return just the
first selection's text. If this isn't the case then converting code to
support the new features is a much more involved job.

Simon.

--
Programmer's Notepad - http://pnotepad.org/

[Neil Hodgson]

Simon Steele:

> I've started playing with discontiguous selections, and the first
> thing that tripped me up after enabling them is that combining
> GetSelectionStart/GetSelectionEnd (to get the selection length) with
> GetSelText isn't safe.

For rectangular selections, previous versions of Scintilla
GetSelText returned all of the selected text rather than the text
between SelectionStart and SelectionEnd. The multiple selection case
was modeled after the rectangular selection case. I think of
GetSelText as return the text that would be put on the clipboard by
Copy.

To find the length that will be returned for GetSelText, first call
it with a NULL pointer.

> If this isn't the case then converting code to
> support the new features is a much more involved job.

Some more accommodation may be needed for old applications being upgraded.

Neil