user configurable explicit fold points

[dlchnr]

I now have implemented user configurable explicit fold points (like
jEdit's "Configurable Fold Handler") in the C++ and D Lexer.
The C++ Lexer can be configured to read explicit fold points for C
generated with Origami/WinF ( /*{{{ , /*}}} ), explicit
fold points for C++ generated with Code Browser ( //[of] , //[cf] )
and explicit fold points for Java generated with
Eclipse Plugin ( //[start] , //[end] ) or NetBeans ( //<editor-... , //
</edtior-... ).
If ported to the corresponding lexers, explicit fold points generate
by Delphi ( { $REGION } , { $ENDREGION } ) can be read too.

Some editors, like jedit, include brackets placed inside comments,
into the bracket matching algorithm. This can be used for
"comment fold matching"! Because { und } are used very often in C, C+
+, Java, ... and because matching {} brackets are spread
over the whole function, using //{ and //} as foldmarks cause problems
with comment fold matching, if } are missed in source code.
Because [ and ] are used more seldom or do match at least (matching[]
brackets are located in a small source code area), using
//[ and //] will lead to less problems with comment fold matching,
therefore I prefer these marks as explicit fold points - they
can be read too, of course.

I've followed the approach "one folding feature, one folding property"
and removed "fold.comment" and "fold.cpp.comment.explicit"
in these lexers, introduced "lexer.cpp/d.fold.multiline.comment" and
"lexer.cpp/d.fold.explicit.fold.points" instead. Having also
a property ("lexer.cpp/d.fold.sytax.based") for the sytax based
folding feature, pure explicit code folding can be configured.

I've changed also all properties to start with "lexer.cpp/d".

I now want to ask, if this patch can be added to the scintilla code
base? Should I change something, so it can be added?
I can deliver also a version compatible with current properties files,
which preserve the "fold.comment" property and the
names of the existing properties.

Of cource I would assist in implemnting these additional folding
features (and the OptionSet, which I need) to further lexers then.

http://dlchnr.dl.funpic.de/in/c_d_lex.zip

[Neil Hodgson]

dlchnr:

> I now have implemented user configurable explicit fold points (like
> jEdit's "Configurable Fold Handler") in the C++ and D Lexer.

That sounds useful.

> I've followed the approach "one folding feature, one folding property"
> and removed "fold.comment" and "fold.cpp.comment.explicit"
> in these lexers, introduced "lexer.cpp/d.fold.multiline.comment" and
> "lexer.cpp/d.fold.explicit.fold.points" instead. Having also
> a property ("lexer.cpp/d.fold.sytax.based") for the sytax based
> folding feature, pure explicit code folding can be configured.

There are many active projects downstream from Scintilla with
serious investment in current Scintilla features. By changing the
names of feature control properties in Scintilla these features will
stop working until their absence is noticed and a fix released. This
will churn downstream projects and damage Scintilla's reputation as a
stable, dependable component.

While it is possible to make incompatible changes to Scintilla,
there must be large corresponding benefits or a long deprecation and
migration process started. If at all possible, changes should be
additions to current behaviour where projects can decide to opt-in to
any changes.

Neil

[dlchnr]

> That sounds useful.
> :
> :
> :

> If at all possible, changes should be
> additions to current behaviour where projects can decide to opt-in to
> any changes.
>
>    Neil

Ok Neil, I've tuned my "compatible" version, so it should be 100%
compatible to current version.
(I've also preserved "fold.cpp.comment.explicit").

It creates the straightforward, "privat" fold property set of my
former version,
which is indepandant from the common fold properties, if "lexer.cpp/
d.fold" is set to 1.
It offers:
- user configurable explicit fold points (D hasn't had any explicit
fold points until now)
- one fold property for every fold feature, so "pure explicit code
folding" can be configured
- a systematic naming scheme, all fold property names start with
"lexer.cpp.fold." or "lexer.d.fold."

#lexer.cpp.fold=1
#
# fold properties evaluated, if lexer.cpp.fold=1
#--------------
lexer.cpp.fold.compact=1
lexer.cpp.fold.syntax.based=1
lexer.cpp.fold.multiline.comments=1
lexer.cpp.fold.explicit.fold.points=1
lexer.cpp.fold.start.string=//{
lexer.cpp.fold.end.string=//}
#lexer.cpp.fold.search.overall=1
lexer.cpp.fold.preprocessor=1
#lexer.cpp.fold.at.else=1
#~~~~~~~~~~~~~~

#lexer.d.fold=1
#
# fold properties evaluated, if lexer.d.fold=1
#--------------
lexer.d.fold.compact=1
lexer.d.fold.syntax.based=1
lexer.d.fold.multiline.comments=1
lexer.d.fold.explicit.fold.points=1
lexer.d.fold.start.string=//{
lexer.d.fold.end.string=//}
#lexer.d.fold.search.overall=1
#lexer.d.fold.at.else=1
#~~~~~~~~~~~~~~

If "lexer.cpp/d.fold=1" is commented out or if it is set to 0
(default), the "common" fold propeties
are evaluated, properties named "lexer.cpp/d.fold. ..." ignored, being
compatible with
current version of SciTE/Scintilla.

I hope, with this changes, this patch can be added to the scintilla
code base?

http://dlchnr.dl.funpic.de/in/cdlexer.zip

[Neil Hodgson]

dlchnr:

> Ok Neil, I've tuned my "compatible" version, so it should be 100%
> compatible to current version.
> (I've also preserved "fold.cpp.comment.explicit").

This adds second names for existing properties which just makes it
more complex to understand.

If you want to convert the D lexer to a lexer object, send in a
patch that does only that. Mixing changes in a single commit makes it
much harder to understand, to trace the cause of bugs, and apply
changes to externally maintained branches.

Neil

[dlchnr]

Neil:

>     This adds second names for existing properties which just makes it
> more complex to understand.

I can't agree - I haven't add second names for existing properties,
I've add privat, language specific properties for these languages
enabling independant configuration of these (and further) languages.
Nevertheless these privat, language specific properties could be
feed from one single, global property set by generating the
neccessary property names with string concatenation
("lexer.XYZ.fold." + "name.of.property").

And I don't think, having a property, normally enabling
folding multiline comments, but enabling additionally
folding explicit fold points inside C++ lexer, but only,
if it isn't declined by fold.cpp.comment.explit
or having something like handling fold.at.else in D
styler.GetPropertyInt("lexer.d.fold.at.else",
styler.GetPropertyInt("fold.at.else", 0)) != 0;
is more complex as having a privat, language specific property set
with a single option for every fold feature.

But I've implemented a further version -
a version, I hope you accept.

# Folding
#fold.cpp.not.syntax.based=1
fold.cpp.comment.multiline=1
fold.cpp.comment.explicit=1
#default: fold.cpp.start.string=//{ and fold.cpp.end.string=//}
#fold.cpp.start.string=//[
#fold.cpp.end.string=//]
#if fold strings are set to something like /*{{{ and /*}}} (Origami/
WinF style fold strings),
# enable fold.cpp.search.overall to search fold strings beyond line
comment start
#fold.cpp.search.overall=1
#fold.at.else=1

http://dlchnr.dl.funpic.de/in/LexCPP.zip

[dlchnr]

Of course I wanted to say:
And I would think so, having a property, normally enabling

folding multiline comments, but enabling additionally
folding explicit fold points inside C++ lexer, but only,
if it isn't declined by fold.cpp.comment.explit
or having something like handling fold.at.else in D
styler.GetPropertyInt("lexer.d.fold.at.else",
styler.GetPropertyInt("fold.at.else", 0)) != 0;
is more complex as having a privat, language specific property set
with a single option for every fold feature.

:-)

[Neil Hodgson]

dlchnr:

> I can't agree - I haven't add second names for existing properties,
> I've add privat,

"private" - I was confused when you used "privat" and "commen" in
an earlier version. I understand its not your first language but
"commen" could be either "comment" or "common".

> language specific properties for these languages
> enabling independant configuration of these (and further) languages.
> Nevertheless these privat, language specific properties could be
> feed from one single, global property set by generating the
> neccessary property names with string concatenation
> ("lexer.XYZ.fold." + "name.of.property").

Too much churn for insufficient benefit. Applications do not have
to use a global property set - that is just the way SciTE is
organised.

These appear inverted:

DefineProperty("fold.cpp.comment.multiline", &OptionsCPP::foldCommentMultiline,
"Set this property to 0 to disable folding explicit fold points when
fold.comment=1.");

DefineProperty("fold.cpp.comment.explicit", &OptionsCPP::foldCommentExplicit,
"Set this property to 0 to disable folding multi-line comments when
fold.comment=1.");

Options, like variables should be positively named rather than
include a negation like fold.cpp.not.syntax.based. Note that all
references inside Fold are negated again like:
if (!options.foldNotSyntaxBased

Options should not include data types like fold.cpp.start.string.
I'd use fold.cpp.explicit.start.

Chatty explanations ("your choice") are too long for the
information they are communicating:
"Set this property to the string of your choice to replace the
standard end string //} "
"used by the C++ lexer for explicit fold points, e.g. //].");
I'd use
"The string to use for explicit fold start points, replacing
the standard //{."
The use of "Set to x" in earlier explanations is because a
particular value (the only possible non-default) is being proposed.

The options object should be considered read-only inside the Fold
call - its the responsibility of LexerCPP::PropertySet to deal with
any modifications to options. I'd also like to avoid potentially
expensive Match calls for the defaults by retaining the current code.
Attached is my version of fold.cpp.explicit.*

Neil