Bugs in folding and line state in 2.12

[Neil Hodgson]

A change was made in Scintilla 2.12 to make pressing Enter at the
start of line move markers on that line down. This was done by
checking if a line insertion started at the beginning of the line and
if so, inserting a new line (with default per-line data) before the
line rather than after the line.

This leads to problems with line state and folding since a default
value is set on this new line displacing the previously calculated
value. For line states this is 0 so it disrupts any multi-line
constructs. For example, there is a bug when entering into a PHP file
<?php<Enter><Enter>?> with the second Enter being at the start of a
line so a 0 is inserted into line state and the fact that this is a
PHP construct is lost. There have been several instances of this
problem in the HTML lexer and another which is probably the same issue
in the Lua lexer.

A more robust approach would be to store the state calculated for a
line on that line and for the next lexer run to pick up the state from
the previous line. Think of it like putting the line state at the end
of the line. The C++ folder works with both current line and next line
information in the fold level so that the next line information is
safe if the next line is removed or changed.

However, existing lexers work by placing this information on the
next line so the code needs to make them work. I think this can be
fixed by duplicating the current value of line state and fold level
from the line being inserted before, thus maintaining continuity of
information.

This is an important change and may not be correct so its going to
delay 2.20.

Code with change available from Hg and from
http://www.scintilla.org/scite.zip Source
http://www.scintilla.org/wscite.zip Windows executable

The patch looks like

=== (+4,-6) src/PerLine.cxx ===
@@ -245,10 +245,7 @@

void LineLevels::InsertLine(int line) {
if (levels.Length()) {
- int level = SC_FOLDLEVELBASE;
- if ((line > 0) && (line < levels.Length())) {
- level = levels[line-1] & ~SC_FOLDLEVELWHITEFLAG;
- }
+ int level = (line < levels.Length()) ? levels[line] : SC_FOLDLEVELBASE;
levels.InsertValue(line, 1, level);
}
}
@@ -306,7 +303,8 @@
void LineState::InsertLine(int line) {
if (lineStates.Length()) {
lineStates.EnsureLength(line);
- lineStates.Insert(line, 0);
+ int val = (line < lineStates.Length()) ? lineStates[line] : 0;
+ lineStates.Insert(line, val);
}
}

Neil

[Neil Hodgson]

Me:

>   This leads to problems with line state and folding since a default
> value is set on this new line displacing the previously calculated

> value. ... There have been several instances of this

> problem in the HTML lexer and another which is probably the same issue
> in the Lua lexer.
>
>   A more robust approach would be to store the state calculated for a
> line on that line and for the next lexer run to pick up the state from
> the previous line.

LexHTML is now changed to work this way: the state taking into
account the contents of a line is stored as the linestate of that line
rather than the next one. When the lexer starts processing another
block it picks up the line state from the previous line. This should
be more robust when lines are inserted and removed.

A similar change may help other lexers.

Neil