Code folding problems
Daniel Carrera
There is an issue with code folding in languages that permit multi-
line strings such as Perl, Python, Ruby, PHP, JavaScript and Lua. Let
me give an example with Perl:
1. Write a function.
----%<----
sub add {
my ($a, $b) = @_;
return $a+$b;
}
----%<----
2. Fold the function.
----%<----
+ sub ad { .. }
----%<----
3. Open a string before the function. Because the language permits
multi-line strings, Scintilla decides that the rest of the file is in
the string, so it unfolds the function:
----%<----
my $name = "Dan
sub add {
my ($a, $b) = @_;
return $a+$b;
}
----%<----
4. When you finish writing the string, all functions below the string
are now unfolded.
----%<----
my $name = "Daniel";
sub add {
my ($a, $b) = @_;
return $a+$b;
}
----%<----
In other words, every time you type a string you lose 100% of the code
folding below that point. This is an extremely frustrating issue. A
similar problem happen with languages that allow multi-line comments,
such as C, C++ and Java. Some other text editors don't have this
problem (e.g. NetBeans), therefore there must be a solution.
My proposed solution: When the user first types a quote character,
Scintilla should be able to figure out that the code is no longer well
formed. The program ends in a quoted string that is not closed. When
you see that happen, do not touch the fold points. Wait for the user
to type the closing quote (i.e. when the code is well-formed again)
before running the lexer. I suspect that this is more or less how
NetBeans avoids this problem.
Regards,
Daniel.
Neil Hodgson
> 3. Open a string before the function. Because the language permits
> multi-line strings, Scintilla decides that the rest of the file is in
> the string, so it unfolds the function:
This problem can be diminished by using single line strings in
languages that offer both and that have an unterminated string state
like Python.
> My proposed solution: When the user first types a quote character,
> Scintilla should be able to figure out that the code is no longer well
> formed.
Code is not well formed after many modifications but it should
still be displayed as well as possible. The change in appearance of
all the text after the new quote is a good indication to the user that
something needs fixing.
> The program ends in a quoted string that is not closed. When
> you see that happen, do not touch the fold points. Wait for the user
> to type the closing quote (i.e. when the code is well-formed again)
> before running the lexer.
The lexer is what determines that there is an end to the string. If
you don't run the lexer then how are you going to discover this?
Lexers are run incrementally to lex just as much as is required to
display the currently viewed code. If the end of the multi-line string
is off the page then it will not be processed.
Neil
Daniel Carrera
>> 3. Open a string before the function. Because the language permits
>> multi-line strings, Scintilla decides that the rest of the file is in
>> the string, so it unfolds the function:
>
> This problem can be diminished by using single line strings in
> languages that offer both and that have an unterminated string state
> like Python.
I'm not sure I understand this.
> The lexer is what determines that there is an end to the string. If
> you don't run the lexer then how are you going to discover this?
>
> Lexers are run incrementally to lex just as much as is required to
> display the currently viewed code. If the end of the multi-line string
> is off the page then it will not be processed.
Well, I don't know how Scintilla works, but I know that somehow
NetBeans doesn't have this problem. So obviously the problem Scintilla
has is not fundamental to how lexers work, but particular to
Scintilla. Do you have any hypothesis as to what NetBeans does that
Scintilla doesn't? I don't know how it works inside, but I can
describe the visual result:
1. Start with a function in NetBeans:
-----%<------
sub add {
my ($a,$b) = @_;
return $a+$b;
}
-----%<------
2. Fold it.
-----%<------
+ sub add { ... }
-----%<------
3. Start a new string above the function. The whole file changes
highlighting, reflecting the new quote character, but the function
remains folded.
-----%<------
my $name = "Daniel
+ sub add { ... }
-----%<------
At this point, if I close the string after 'Daniel', the function
remains folded and all is well:
-----%<------
my $name = "Daniel";
+ sub add { ... }
-----%<------
The function is only expanded if I close the quote after the function:
-----%<------
my $name = "Daniel
sub add {
my ($a,$b) = @_;
return $a+$b;
}
";
-----%<------
Daniel.
--
Intolerant people should be shot.
KHMan
> On Thu, Aug 5, 2010 at 1:36 AM, Neil Hodgson wrote:
>>> 3. Open a string before the function. Because the language permits
>>> multi-line strings, Scintilla decides that the rest of the file is in
>>> the string, so it unfolds the function:
>>
>> This problem can be diminished by using single line strings in
>> languages that offer both and that have an unterminated string state
>> like Python.
>
> I'm not sure I understand this.
>
>> The lexer is what determines that there is an end to the string. If
>> you don't run the lexer then how are you going to discover this?
>>
>> Lexers are run incrementally to lex just as much as is required to
>> display the currently viewed code. If the end of the multi-line string
>> is off the page then it will not be processed.
>
> Well, I don't know how Scintilla works, but I know that somehow
> NetBeans doesn't have this problem. So obviously the problem Scintilla
> has is not fundamental to how lexers work, but particular to
> Scintilla. Do you have any hypothesis as to what NetBeans does that
> Scintilla doesn't? I don't know how it works inside, but I can
> describe the visual result:
>
> 3. Start a new string above the function. The whole file changes
> highlighting, reflecting the new quote character, but the function
> remains folded.
>
> -----%<------
> my $name = "Daniel
>
> + sub add { ... }
> -----%<------
>
> At this point, if I close the string after 'Daniel', the function
> remains folded and all is well:
If you keep the string open, does NetBeans unfold the function
after a while? Is there a time delay or anything built in?
Perhaps a time delay might be useful in such cases. (But I don't
use folding, so I'm just shootin' the breeze here.)
--
Cheers,
Kein-Hong Man (esq.)
Kuala Lumpur, Malaysia
Daniel Carrera
> If you keep the string open, does NetBeans unfold the function after a
> while? Is there a time delay or anything built in?
>
> Perhaps a time delay might be useful in such cases. (But I don't use
> folding, so I'm just shootin' the breeze here.)
I just tested with PHP and it does not appear to use a time delay. I
opened a string and after 10 minutes the function remains unfolded. I
also note that the *colour* of the text changes instantly, denoting
that now the function is inside a string, and I even get a little red
marker indicating a syntax error.
A delay might be useful for Scintilla, but I don't think that's how
NetBeans work.
Neil Hodgson
>> This problem can be diminished by using single line strings in
>> languages that offer both and that have an unterminated string state
>> like Python.
>
> I'm not sure I understand this.
Here is a sequence of operations similar to your "Daniel" string
example but in Python:
http://www.scintilla.org/FoldString.png
> Do you have any hypothesis as to what NetBeans does that
> Scintilla doesn't? I don't know how it works inside, but I can
> describe the visual result:
NetBeans is open source so you could examine the implementation.
Neil
Daniel Carrera
>> Do you have any hypothesis as to what NetBeans does that
>> Scintilla doesn't? I don't know how it works inside, but I can
>> describe the visual result:
>
> NetBeans is open source so you could examine the implementation.
That is probably beyond my skill. I doubt that I could make any sense
out of the NetBeans source code.