C++17 intentions

296 views
Skip to first unread message

Neil Hodgson

unread,
Nov 22, 2016, 8:50:46 PM11/22/16
to Scintilla mailing list
In 2017, I would like to move to using C++17 features in Scintilla. New releases of the major compilers will be mostly C++17-compliant and, by April, these should be available on each platform. Making a big jump all the way to C++17 instead of just to C++14 or C++11 will avoid the extra work of more transitions.

Unlike the existing use of C++11 std::regex which is well isolated and can be turned off selectively for older compilers, there would be more pervasive use of features like move semantics and smart pointers, both unique and shared. This would make it difficult to maintain compatibility with old build systems.

Changes would occur over time - trying to modernise all the code at once would be a lot of work and would likely introduce bugs.

Patches received now often use C++1x features and these have to be changed before application so moving to C++17 will lessen the cost of accepting patches.

The precise feature set allowed in code would be whatever is in C++17 that is supported by the compilers that operate on that code. So, for macOS-specific code it would have to work in current Xcode and for Windows code with current MSVC. Cross-platform code would be constrained to the lowest common denominator, likely MSVC 2017.

If anyone is interested in performing the work, there could be a fork that maintains compatibility but backports fixes or even features.

Neil

Teodor Petrov

unread,
Nov 24, 2016, 3:07:20 PM11/24/16
to scintilla...@googlegroups.com
On 11/23/2016 03:50 AM, Neil Hodgson wrote:
> In 2017, I would like to move to using C++17 features in Scintilla. New releases of the major compilers will be mostly C++17-compliant and, by April, these should be available on each platform. Making a big jump all the way to C++17 instead of just to C++14 or C++11 will avoid the extra work of more transitions.
I don't see why you want to do this. The major changes come with C++11,
after that there are only
incremental changes to the language. Scintilla is not using too many
template tricks as far as I've seen,
so the benefits aren't that great.

For the Code::Blocks project this change will be very limiting. If you
do this change we won't be able to
use the latest scintilla after 2017. This is because we want to support
CentOS 6, which means we're
limited to GCC 4.8.x (c++11 I think). CentOS 7 is also using GCC 4.8.x.
And some of the other Linux
distros that has long-term-support variants are limited to GCC 4.8.x or
4.9.x. I'm not event sure how
many of the non-long-term-support Linux distros will have C++17 compiler
in April 2017. And we don't
have the man power to support and backport Scintilla and wxScintilla to
C++11.

I think C++11 is the maximum standard a component like Scintilla can
require for the next
2-3-4 years. I know that it is annoying that you cannot use the latest
and greatest parts of the standard,
but I don't see much benefit limiting users stuck on older platforms.

Best regards,
Teodor Petrov
Code::Blocks developer


Neil Hodgson

unread,
Nov 24, 2016, 4:07:53 PM11/24/16
to scintilla...@googlegroups.com
Teodor Petrov:

> I don't see why you want to do this. The major changes come with C++11, after that there are only
> incremental changes to the language.

Those incremental changes can also be useful and fix some issues with C++11.

> For the Code::Blocks project this change will be very limiting. If you do this change we won't be able to
> use the latest scintilla after 2017. This is because we want to support CentOS 6, which means we're
> limited to GCC 4.8.x (c++11 I think).

Newer compilers can be installed. Its common to install new compilers on Windows and macOS and it isn’t unreasonable to do this on Linux. Its not as if each user on CentOS has to compile Scintilla.

> And we don't
> have the man power to support and backport Scintilla and wxScintilla to C++11.

Then either use a newer compiler or stick with the current Scintilla or a fork if anyone wants to work on it.

> I think C++11 is the maximum standard a component like Scintilla can require for the next
> 2-3-4 years.

While I am not completely against some compromise, locking to C++11 for another 4 years is too much.

Neil

Lex Trotman

unread,
Nov 24, 2016, 6:56:05 PM11/24/16
to scintilla...@googlegroups.com
IIRC there was an ABI breakage around gcc 5 so using a newer compiler
on a system with all its libraries compiled with a 4.x version might
not work.
> --
> You received this message because you are subscribed to the Google Groups "scintilla-interest" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to scintilla-inter...@googlegroups.com.
> To post to this group, send email to scintilla...@googlegroups.com.
> Visit this group at https://groups.google.com/group/scintilla-interest.
> For more options, visit https://groups.google.com/d/optout.

Neil Hodgson

unread,
Nov 24, 2016, 9:38:39 PM11/24/16
to scintilla...@googlegroups.com
Lex Trotman:

> IIRC there was an ABI breakage around gcc 5 so using a newer compiler
> on a system with all its libraries compiled with a 4.x version might
> not work.

I think this is what you are referring to:
http://developers.redhat.com/blog/2015/02/05/gcc5-and-the-c11-abi/

The major issue appears to be with C++ library binaries compiled with a pre-5 compiler that are linked into an application compiled with g++ 5. I don’t think that is the case here where a library, Scintilla, will want the newer compiler. Both old and new ABIs are provided by the libstdc++ standard library.

Scintilla exposes a C API and there are no C++ standard library objects passed between Scintilla and client code. I’m not seeing any circumstances in which this will be a problem.

Neil

Matthew Brush

unread,
Nov 24, 2016, 11:14:05 PM11/24/16
to scintilla...@googlegroups.com
On 2016-11-24 01:07 PM, Neil Hodgson wrote:
> [...]
>
> Newer compilers can be installed. Its common to install new compilers
> on Windows and macOS and it isn’t unreasonable to do this on Linux.
> Its not as if each user on CentOS has to compile Scintilla.
>

Each project will have to ship own C++17-capable standard library on
these platforms as well (not allowed/reasonable on most distros), unless
Scintilla limits the C++17 features it uses to only language and not
library. Even more up-to-date distros like Ubuntu and Mint probably
won't have a fully c++17 capable standard library until a year or so
later, and I don't think that would be backported to versions from the
last few years. So most likely projects that want to support Ubuntu and
derivatives from 2015-2016 probably won't be able to upgrade their
Scintilla copy until 2018 or so.

Moreover, if Scintilla is going to depend on a bleeding edge stdlib, it
might as well drop support for all but the latest platform versions too
(GTK+, Cocoa, Qt, etc) since there's no point supporting those old
versions if none of the distros/OSes that use those versions will be
supported anymore.

Personally I don't mind that much as I use up-to-date distros and
maintaining support for older platforms is a pain, but I suspect it will
generate loads of complaints and bug reports.

Regards,
Matthew Brush

Lex Trotman

unread,
Nov 24, 2016, 11:46:09 PM11/24/16
to scintilla...@googlegroups.com
Yes, since even the C++ interfaces (ILexer and friends) don't use any
C++ STL objects its probably safe for the application to be compiled
with an earlier compiler than Scintilla, allowing the application to
still talk to existing C++ non-stdlibcxx libraries. So long as no
exceptions containing strings escape from Scintilla this should be ok,
apart from the build infrastructure issues of course.

Or both the application and Scintilla could be compiled with
-D_GLIBCXX_USE_CXX11_ABI=0 again allowing the application to talk to
existing libraries. Its not entirely clear what "standard
compatible" features would be lost but it appears to only be non-COW
std::strings and std::list length 0(1) performance and Scintilla
probably doesn't depend on either (see
https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html )
This would seem to be the preferred work around for applications that
need to access libraries that are part of their distro and compiled
with an older compiler.

Neil Hodgson

unread,
Nov 24, 2016, 11:54:40 PM11/24/16
to scintilla...@googlegroups.com
Matthew Brush:

> Even more up-to-date distros like Ubuntu and Mint probably won't have a fully c++17 capable standard library until a year or so later, and I don't think that would be backported to versions from the last few years.

Ubuntu 16.10 and Fedora 25 currently ship with g++ 6.2 which supports all of C++14 and about a third of C++17. They also have Clang 3.8 which is roughly equivalent to g++ 6. Clang 3.9 was released in September and has about 3/4 of C++17. Clang 3.9 is packaged in Ubuntu but the default install is 3.8.

Its possible g++ 7 won’t be ready for the next distribution releases but that will become clearer by then. If it isn’t then g++ 6 still provides much more of C++11, C++14, and C++17 than remaining with 4.8.

> Moreover, if Scintilla is going to depend on a bleeding edge stdlib, it might as well drop support for all but the latest platform versions too (GTK+, Cocoa, Qt, etc) since there's no point supporting those old versions if none of the distros/OSes that use those versions will be supported anymore.

GTK+ 2.x works well now and may even still be safer than GTK+ 3.x. Compilers support building for older systems so there is no need to drop support for old versions of Windows or macOS just because a newer compiler is used.

Neil

Matthew Brush

unread,
Nov 25, 2016, 12:23:37 AM11/25/16
to scintilla...@googlegroups.com
On 2016-11-24 08:54 PM, Neil Hodgson wrote:
> Matthew Brush:
>
>> Even more up-to-date distros like Ubuntu and Mint probably won't have a fully c++17 capable standard library until a year or so later, and I don't think that would be backported to versions from the last few years.
>
> Ubuntu 16.10 and Fedora 25 currently ship with g++ 6.2 which supports all of C++14 and about a third of C++17. They also have Clang 3.8 which is roughly equivalent to g++ 6. Clang 3.9 was released in September and has about 3/4 of C++17. Clang 3.9 is packaged in Ubuntu but the default install is 3.8.
>
> Its possible g++ 7 won’t be ready for the next distribution releases but that will become clearer by then. If it isn’t then g++ 6 still provides much more of C++11, C++14, and C++17 than remaining with 4.8.
>

So yeah, none of the Ubuntus before 17.10 (presumably, as the first
version after the standard is finalized and voted on) will be able to
use Scintilla if it starts using C++17 library features.

>> Moreover, if Scintilla is going to depend on a bleeding edge stdlib, it might as well drop support for all but the latest platform versions too (GTK+, Cocoa, Qt, etc) since there's no point supporting those old versions if none of the distros/OSes that use those versions will be supported anymore.
>
> GTK+ 2.x works well now and may even still be safer than GTK+ 3.x. Compilers support building for older systems so there is no need to drop support for old versions of Windows or macOS just because a newer compiler is used.
>

I'm only raising the issue of the standard C++ library, compilers are a
developer thing. I'm not knowledgeable on Win32 or OSX packaging, but I
assume it's frowned upon for every package to provide its own C++
standard library.

My point about dropping older stuff is that if the new bar is systems
that have a C++17 standard library, then it would be an opportune time
to get rid of all the legacy/compat code for old platforms/libraries.
For example there's no point supporting GTK+ < 3.2x (or whichever
recent-ish version) if no system that uses those older versions will
have a suitable C++ runtime library anyway. I assume the same would
apply for old OSX and Windows version, but like I said, not too
knowledgeable on their distribution models.

Regards,
Matthew Brush - Big C++17 fan :)

Mike Lischke

unread,
Nov 25, 2016, 4:00:05 AM11/25/16
to scintilla...@googlegroups.com
>
>> I think C++11 is the maximum standard a component like Scintilla can require for the next
>> 2-3-4 years.
>
> While I am not completely against some compromise, locking to C++11 for another 4 years is too much.


+1, there is nothing forcing upstream projects to immediately use the latest Scintilla version. Usually you don't update 3rd party libraries that often (maybe once a year for us).

Mike
--
www.soft-gems.net

Mike Lischke

unread,
Nov 25, 2016, 4:02:16 AM11/25/16
to scintilla...@googlegroups.com

Am 25.11.2016 um 06:23 schrieb Matthew Brush <matthe...@gmail.com>:

My point about dropping older stuff is that if the new bar is systems that have a C++17 standard library, then it would be an opportune time to get rid of all the legacy/compat code for old platforms/libraries. For example there's no point supporting GTK+ < 3.2x (or whichever recent-ish version) if no system that uses those older versions will have a suitable C++ runtime library anyway. I assume the same would apply for old OSX and Windows version, but like I said, not too knowledgeable on their distribution models.

IMO it would make a lot of sense to start a new major version with those changes, which can also have other legacy stuff removed (e.g. all the internal charset handling). This could go in parallel for a year or 2 and after that all upstream projects had enough time to upgrade their environment or stay on the previous major version forever.

Teodor Petrov

unread,
Nov 25, 2016, 4:04:14 PM11/25/16
to scintilla...@googlegroups.com
On 11/24/2016 11:07 PM, Neil Hodgson wrote:
> Newer compilers can be installed. Its common to install new compilers
> on Windows and macOS and it isn’t unreasonable to do this on Linux.
> Its not as if each user on CentOS has to compile Scintilla.

This is not 100% the case on Linux. One can install newer compiler, but
one needs to be more than
the average linux user (you must replace user with developer to be 100%
accurate) to manage it
without major problems.

Some facts about linux compilers:
1. GCC and libstdc++ are released at the same time
2. Newer GCCs cannot use older libstdc++
3. The default libstdc++ is the one from the main GCC. Most of the times
this is the old GCC.
4. The result of 3 is that building a c++ application with the newer
compiler that uses non-header only
parts of libstdc++ will result in an application that fails to
run. One needs to use rpath or modify the
LD_LIBRARY_PATH to tell the application where the correct
libstdc++ is.
5. One needs to rebuild all C++ libraries his/her application depends on
with the new compiler.
6. Static linking doesn't solve the problem because the symbols in
libstdc++ are deliberately made weak,
which means they cannot be hidden and they are part of the ABI of
the lib/exe.
7. Using clang and libstdc++ is possible (if you don't use features that
are missing from the library).
8. Using clang and libc++ is also possible, but probably harder to
accomplish.

A work-around can be found for most of the issues, but it is not really
trivial.
Most linux users cannot cope with the problems.

From my not non-scientific data there are too many people that are
using ubuntu 12 or ubuntu 14.

Best regards,
Teodor


Neil Hodgson

unread,
Nov 25, 2016, 6:03:04 PM11/25/16
to scintilla...@googlegroups.com
Teodor Petrov:

> 6. Static linking doesn't solve the problem because the symbols in libstdc++ are deliberately made weak,
> which means they cannot be hidden and they are part of the ABI of the lib/exe.

Doesn’t -static-libstdc++ work?

Neil

Neil Hodgson

unread,
Nov 25, 2016, 6:03:16 PM11/25/16
to scintilla...@googlegroups.com
Matthew Brush:

> I'm only raising the issue of the standard C++ library, compilers are a developer thing. I'm not knowledgeable on Win32 or OSX packaging, but I assume it's frowned upon for every package to provide its own C++ standard library.

Its quite common to distribute Microsoft’s C++ runtime DLL with applications. Its one of the reasons that Windows applications come with installers.

Another approach is to statically link the C++ runtime which is what the Win32 Scintilla DLLs do.

> My point about dropping older stuff is that if the new bar is systems that have a C++17 standard library, then it would be an opportune time to get rid of all the legacy/compat code for old platforms/libraries.

I see compiler choice as orthogonal to platform choice.

Neil

Neil Hodgson

unread,
Nov 25, 2016, 6:06:17 PM11/25/16
to scintilla...@googlegroups.com
Mike Lischke:

> IMO it would make a lot of sense to start a new major version with those changes, which can also have other legacy stuff removed (e.g. all the internal charset handling).

I know you don’t like the character set code but I don’t think its going away soon. Latin1, in particular, is still very widely used for source code.

There are also problems with transcoding at the file I/O edge. For example Shift-JIS (and one of the Chinese encodings) is not truly round-tripped through Unicode. Both NEC and IBM needed many extra characters and added them to their implementations of Shift-JIS with different values. There is a single Unicode representation of the characters so you can convert to Unicode but converting back you are left with the question of which value to use, which may be important to any processing of the file.

Maybe in 20 years time my successor will be able to rip out all the encoding code but its not likely in the near future.

To simplify the encoding code, I recently tried to reimplemented it in a cross-platform manner so it wouldn’t be needed in each platform layer. It requires around 160K of tables to support the 4 main CJK encodings and around 8 single byte encodings. While the translation to Unicode was tractable, conversion back is more difficult and commonly occurs differently on Windows and Unix with both making some effort to behave reasonably when there is no direct translation of a Unicode character.

> This could go in parallel for a year or 2 and after that all upstream projects had enough time to upgrade their environment or stay on the previous major version forever.

I did once try a major fork with SinkWorld but it was overly ambitious and mostly failed although some of the code has since been used in Scintilla.

Open source projects are often operating at the edge of available contributor time and its harder to find maintenance contributions than cool new feature contributions. If there is a fork then changes, both fixes and features, can be applied either to the fork or the base or to both. Extra effort will be incurred. Its similar to having a maintenance fork (although the emphasis is different) but as Teodor said, “we don’t have the man power to support and backport Scintilla and wxScintilla to C++11”.

Neil

Teodor Petrov

unread,
Nov 25, 2016, 7:12:29 PM11/25/16
to scintilla...@googlegroups.com
No, as far as I can see.
The symbols of STL are still weak.

Here is an example:
> cat test_static.cpp
#include <vector>
int main()
{
std::vector<int> v{1,2,3,4,5,6};
return 0;
}

> g++-6.2.0 -fvisibility=hidden -std=c++11 test_static.cpp -o test_default
> nm -C test_default |grep 'vector<int'
0000000000400a7c W void std::vector<int, std::allocator<int>
>::_M_range_initialize<int const*>(int const*, int const*,
std::forward_iterator_tag)
00000000004008f8 W std::vector<int, std::allocator<int>
>::vector(std::initializer_list<int>, std::allocator<int> const&)
00000000004008f8 W std::vector<int, std::allocator<int>
>::vector(std::initializer_list<int>, std::allocator<int> const&)
000000000040098e W std::vector<int, std::allocator<int> >::~vector()
000000000040098e W std::vector<int, std::allocator<int> >::~vector()

> g++-6.2.0 -fvisibility=hidden -std=c++11 test_static.cpp -o
test_static -static-libstdc++
> nm -C test_static |grep 'vector<int'
000000000041662c W void std::vector<int, std::allocator<int>
>::_M_range_initialize<int const*>(int const*, int const*,
std::forward_iterator_tag)
00000000004164a8 W std::vector<int, std::allocator<int>
>::vector(std::initializer_list<int>, std::allocator<int> const&)
00000000004164a8 W std::vector<int, std::allocator<int>
>::vector(std::initializer_list<int>, std::allocator<int> const&)
000000000041653e W std::vector<int, std::allocator<int> >::~vector()
000000000041653e W std::vector<int, std::allocator<int> >::~vector()

> ldd test_default
linux-vdso.so.1 (0x00007fff69349000)
libstdc++.so.6 =>
/usr/lib/gcc/x86_64-pc-linux-gnu/6.2.0/libstdc++.so.6 (0x00007fcf637f0000)
libm.so.6 => /lib64/libm.so.6 (0x00007fcf634eb000)
libgcc_s.so.1 =>
/usr/lib/gcc/x86_64-pc-linux-gnu/6.2.0/libgcc_s.so.1 (0x00007fcf632d4000)
libc.so.6 => /lib64/libc.so.6 (0x00007fcf62f3b000)
/lib64/ld-linux-x86-64.so.2 (0x00007fcf63be8000)

> ldd test_static
linux-vdso.so.1 (0x00007ffe8aac1000)
libm.so.6 => /lib64/libm.so.6 (0x00007f106875f000)
libgcc_s.so.1 =>
/usr/lib/gcc/x86_64-pc-linux-gnu/6.2.0/libgcc_s.so.1 (0x00007f1068548000)
libc.so.6 => /lib64/libc.so.6 (0x00007f10681af000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1068a64000)


Using -static-libstdc++ will prevent the need for rpath for libstdc++ or
the need to change the LD_LIBRARY_PATH.
It won't prevent the need to rebuild every C++ dependency of your
project with the new compiler and settings.
Failing to do rebuild them might result in random and hard to debug crashes.

/Teodor

Matthew Brush

unread,
Nov 25, 2016, 8:35:33 PM11/25/16
to scintilla...@googlegroups.com
On 2016-11-25 03:03 PM, Neil Hodgson wrote:
> Matthew Brush:
>
>> [...]
>
>> My point about dropping older stuff is that if the new bar is
>> systems that have a C++17 standard library, then it would be an
>> opportune time to get rid of all the legacy/compat code for old
>> platforms/libraries.
>
> I see compiler choice as orthogonal to platform choice.

If Scintilla drops support for distros which don't have a C++17 library
and never will, then why bother supporting the versions of various other
libraries only they use? For example if you had a block of code guarded
by `#if !GTK_CHECK_VERSION(3,18,0)` in Scintilla, but distros with GTK+
<= 3.18 will never have a C++17 standard library, then that block of
code will never be used, so it might as well be removed.

Regards,
Matthew Brush

Neil Hodgson

unread,
Nov 26, 2016, 1:54:18 AM11/26/16
to scintilla...@googlegroups.com
Teodor Petrov:

> No, as far as I can see.
> The symbols of STL are still weak.
> ...
> > g++-6.2.0 -fvisibility=hidden -std=c++11 test_static.cpp -o test_static -static-libstdc++
> > nm -C test_static |grep 'vector<int'
> 000000000041662c W void std::vector<int, std::allocator<int> >::_M_range_initialize<int const*>(int const*, int const*, std::forward_iterator_tag)
> 00000000004164a8 W std::vector<int, std::allocator<int> >::vector(std::initializer_list<int>, std::allocator<int> const&)
> 00000000004164a8 W std::vector<int, std::allocator<int> >::vector(std::initializer_list<int>, std::allocator<int> const&)
> 000000000041653e W std::vector<int, std::allocator<int> >::~vector()
> 000000000041653e W std::vector<int, std::allocator<int> >::~vector()

Does it matter that the symbols are weak when the library uses ABI tags to mark revised classes?

> Using -static-libstdc++ will prevent the need for rpath for libstdc++ or the need to change the LD_LIBRARY_PATH.
> It won't prevent the need to rebuild every C++ dependency of your project with the new compiler and settings.

Is the problem here that the application is using shared libraries from the distribution with APIs that return or consume standard C++ library objects? Or is there a plugin issue? What is to stop building “every C++ dependency of your project with the new compiler and settings”.

I’m used to the Windows world where symbols don’t leak out of DLLs unless you tell them to. Can a Scintilla .so be built that completely hides its use of a recent static libstdc++?

Neil

Neil Hodgson

unread,
Nov 26, 2016, 2:30:28 AM11/26/16
to scintilla...@googlegroups.com
Matthew Brush:

> If Scintilla drops support for distros which don't have a C++17 library and never will,

I don’t see this as dropping support although it makes things more difficult. Sufficiently motivated users can install new compilers.

> then why bother supporting the versions of various other libraries only they use? For example if you had a block of code guarded by `#if !GTK_CHECK_VERSION(3,18,0)` in Scintilla, but distros with GTK+ <= 3.18 will never have a C++17 standard library, then that block of code will never be used, so it might as well be removed.

If there is consensus for removing some 3.x versions then that would be OK but there aren’t that many places (around 12) where its an issue. Its not a significant simplification unless GTK+ 2.x is also removed.

Neil

Lex Trotman

unread,
Nov 26, 2016, 4:13:24 AM11/26/16
to scintilla...@googlegroups.com
[...]
> Is the problem here that the application is using shared libraries from the distribution with APIs that return or consume standard C++ library objects? Or is there a plugin issue? What is to stop building “every C++ dependency of your project with the new compiler and settings”.

In the linux world, if a distribution provides a library, it varies
from frowned on to forbidden, to compile versions of that library into
your application. Using the distro provided dynamic library means the
distro can provide fixes to the library simply by updating the .so,
not updating every application that uses it. This is of course
essential in crypto and network related components where security
fixes are continuous. But most libraries are therefore dynamically
linked.

So distributing an application that uses Scintilla is going to be a
problem if "every C++ dependency" has to be re-compiled and static
linked.

Also distros tend to build binaries on server farms and they may not
allow an application to install a newer compiler as part of the build.

>
> I’m used to the Windows world where symbols don’t leak out of DLLs unless you tell them to. Can a Scintilla .so be built that completely hides its use of a recent static libstdc++?

Also IIUC libstdc++ is configured for the system it is on, that will
be the compilation system, not the end-user system. So distributing
the application binary might not work, YMMV.

Essentially Neil using a new C++ standard immediately after its
release you risk causing significant problems for applications using
Scintilla if they support older stable systems. The *current* Debian
stable and Ubuntu stable, which is also the base of Mint, uses gcc5.4
which doesn't even have c++14 let alone c++17. So, at least for Linux
you are likely to get complaints and pushback from downstreams.

Applications staying on an older Scintilla is only an option if fixes
can be backported fairly easily, which will restrict where new code
features can be used.

Mike Lischke

unread,
Nov 26, 2016, 5:25:54 AM11/26/16
to scintilla...@googlegroups.com

>> IMO it would make a lot of sense to start a new major version with those changes, which can also have other legacy stuff removed (e.g. all the internal charset handling).
>
> I know you don’t like the character set code but I don’t think its going away soon. Latin1, in particular, is still very widely used for source code.
>
> ...
> Maybe in 20 years time my successor will be able to rip out all the encoding code but its not likely in the near future.

Hopefully it doesn't take that long. As you wrote, proper charset handling is difficult and error prone. These problems alone would make me switch to Unicode. It's a mystery to me why people still use charsets. It's not mythical, not historical, nor political. It's all about what codes to use to store text. On the other hand we all know most people like traditions.

>
> To simplify the encoding code, I recently tried to reimplemented it in a cross-platform manner so it wouldn’t be needed in each platform layer. It requires around 160K of tables to support the 4 main CJK encodings and around 8 single byte encodings. While the translation to Unicode was tractable, conversion back is more difficult and commonly occurs differently on Windows and Unix with both making some effort to behave reasonably when there is no direct translation of a Unicode character.

Your work is much valued, but isn't that crazy what you have to do to support charsets? Dropping them would also help you, to avoid wasting even more time in this area (which is my primary reason to suggest this from time to time :-) ).

>
>> This could go in parallel for a year or 2 and after that all upstream projects had enough time to upgrade their environment or stay on the previous major version forever.
>
> I did once try a major fork with SinkWorld but it was overly ambitious and mostly failed although some of the code has since been used in Scintilla.
>
> Open source projects are often operating at the edge of available contributor time and its harder to find maintenance contributions than cool new feature contributions. If there is a fork then changes, both fixes and features, can be applied either to the fork or the base or to both. Extra effort will be incurred. Its similar to having a maintenance fork (although the emphasis is different) but as Teodor said, “we don’t have the man power to support and backport Scintilla and wxScintilla to C++11”.

I completely hear you. Being myself an open source developer since 2 decades, I know all that ad nauseam. Still, I believe having 2 versions is the best solution. Teodor can use the older one and stay with C++11, but he cannot get new features then (which he may not even want). You can finally switch to C++17 and continue with your plans and if you only merge bug fixes to both branches (so they apply), you have not much extra work. Except for those bug fixes you wouldn't work in the older branch anyway. And this could even serve as a voting tool for you. From the downloads you can see if people rather want the new features or want to stay with old stuff, which in turn can help scheduling what to work on in your precious time. But I bet you know all that, so consider that just as encouraging you to your plans :-)

Mike
--
www.soft-gems.net

Neil Hodgson

unread,
Nov 26, 2016, 7:56:42 PM11/26/16
to Scintilla mailing list
Lex Trotman:

> Also distros tend to build binaries on server farms and they may not
> allow an application to install a newer compiler as part of the build.

I don’t see how the distro got involved in building the application here. Its the application developer or user that gets to build the application.

> Essentially Neil using a new C++ standard immediately after its
> release you risk causing significant problems for applications using
> Scintilla if they support older stable systems. The *current* Debian
> stable and Ubuntu stable, which is also the base of Mint, uses gcc5.4
> which doesn't even have c++14 let alone c++17. So, at least for Linux
> you are likely to get complaints and pushback from downstreams.

Not even C++14? There doesn’t appear to be an interesting intermediate position that will satisfy long term support Linux.

Neil

Lex Trotman

unread,
Nov 26, 2016, 8:49:23 PM11/26/16
to scintilla...@googlegroups.com
On 27 November 2016 at 10:56, Neil Hodgson <nyama...@me.com> wrote:
> Lex Trotman:
>
>> Also distros tend to build binaries on server farms and they may not
>> allow an application to install a newer compiler as part of the build.
>
> I don’t see how the distro got involved in building the application here. Its the application developer or user that gets to build the application.

Linux is not like Windows where you go to applications websites to
download software, each distribution keeps a central repository of
pre-built binaries compatible with each supported version of the
distro and the available versions of their dependencies. End users
usually load from the distro's software manager which handles
downloading any dependencies missing on the users system.
Distributions build the binaries they distribute, not developers. That
allows them to confirm compatibility for a more reliable user
experience, especially if they support a LTS version and a bleeding
edge version.

>
>> Essentially Neil using a new C++ standard immediately after its
>> release you risk causing significant problems for applications using
>> Scintilla if they support older stable systems. The *current* Debian
>> stable and Ubuntu stable, which is also the base of Mint, uses gcc5.4
>> which doesn't even have c++14 let alone c++17. So, at least for Linux
>> you are likely to get complaints and pushback from downstreams.
>
> Not even C++14? There doesn’t appear to be an interesting intermediate position that will satisfy long term support Linux.

I admit I was surprised, but according to the docs for gcc 5.4
https://gcc.gnu.org/onlinedocs/gcc-5.4.0/gcc/Standards.html#Standards
in section 2.2 there isn't even a -std=c++1y preliminary c++14 option.

Gcc 6 supports both c++14 and c++17
https://gcc.gnu.org/onlinedocs/gcc-6.2.0/gcc/Standards.html#Standards
but it was first released just too late for the last Ubuntu and Debian
LTS versions.

Gcc 6 isn't even offered in the LTS repositories, though 4.7, 4.8, 4.9
and 5 are available (maybe 6 isn't because of some other
incompatibilities, don't know), so anybody "installing" gcc 6 on those
systems would have to compile gcc and the required runtimes
themselves, not really something many people are willing to do, and
again not something likely to happen on the distros build servers.

Neil Hodgson

unread,
Nov 29, 2016, 5:06:29 PM11/29/16
to scintilla...@googlegroups.com
Thanks for all the feedback which I have considered quite a lot over the past few days.

It doesn’t look like it will be reasonable to use C++14/17 on long term support versions of Linux. So, there should be a long-term-support branch of Scintilla for older versions of Linux alongside the C++14/17 branch. Version numbers 3.x can be used for the LTS branch and 4.x for the C++14/17 branch.

The Windows model which encourages the use of new compilers when targeting old versions of the operating system appears better to me. The Linux model constrains the choice of tools in a way that I would like to avoid. Therefore, my involvement with the LTS branch will be small. I may cut a new release alongside a 4.x release when there has been progress on the LTS branch and may backport fixes for crashing and data-loss bugs. Other people with different priorities may be more interested in working on the LTS branch and they can have commit privileges for that branch if they wish.

One question is which compiler version should the LTS branch target?

Neil

Matthew Brush

unread,
Nov 29, 2016, 7:58:31 PM11/29/16
to scintilla...@googlegroups.com
On 2016-11-29 02:06 PM, Neil Hodgson wrote:
> [...]

> The Linux model constrains the choice of tools in a way that I would
> like to avoid. [...]
>

That's not the "Linux model", that's the "small projects supporting old
distros because megacorporations still support them model".

If you target current mainstream Linux distros, you have access to a
vast array of excellent, modern tools/libraries for free; you only have
to declare a dependency and your app can rely on them with no extra fuss.

> One question is which compiler version should the LTS branch target?
>
>

Not sure.

One data point; I'm using Ubuntu 16.04 LTS and it has GCC 5.4 and Clang
3.8 in the repos.

Regards,
Matthew Brush

Lex Trotman

unread,
Nov 29, 2016, 8:47:58 PM11/29/16
to scintilla...@googlegroups.com
On 30 November 2016 at 08:06, Neil Hodgson <nyama...@me.com> wrote:
> Thanks for all the feedback which I have considered quite a lot over the past few days.
>
> It doesn’t look like it will be reasonable to use C++14/17 on long term support versions of Linux. So, there should be a long-term-support branch of Scintilla for older versions of Linux alongside the C++14/17 branch. Version numbers 3.x can be used for the LTS branch and 4.x for the C++14/17 branch.

Thats probably reasonable, so long as bugfixes in the 4.x branch don't
wantonly use c++17 features :)

>
> The Windows model which encourages the use of new compilers when targeting old versions of the operating system appears better to me. The Linux model constrains the choice of tools in a way that I would like to avoid. Therefore, my involvement with the LTS branch will be small. I may cut a new release alongside a 4.x release when there has been progress on the LTS branch and may backport fixes for crashing and data-loss bugs. Other people with different priorities may be more interested in working on the LTS branch and they can have commit privileges for that branch if they wish.
>
> One question is which compiler version should the LTS branch target?

It appears that the situation may not be as bad as I thought, GCC 5.4
does in fact support c++14 and some experimental c++17, its just that
the "standards supported" part of the manual I quoted previously
appears to be out of date. See
https://gcc.gnu.org/onlinedocs/gcc-5.4.0/gcc/C-Dialect-Options.html#C-Dialect-Options
-std= option later in the same document.

So that sounds like the version to support for what Matthew called
"mainstream" LTS distros.

Commercial focused distros like Red Hat and Suse may provide support
for older versions for longer, but they are paid for that, so they can
do the backporting (IMHO).

Teodor Petrov

unread,
Nov 30, 2016, 2:01:39 PM11/30/16
to scintilla...@googlegroups.com
On 11/30/2016 12:06 AM, Neil Hodgson wrote:
> One question is which compiler version should the LTS branch target?

GCC 4.8.x if you ask me. So applications will be able to run on Centos 6
and 7.

/Teodor

Neil Hodgson

unread,
Feb 22, 2017, 1:48:56 AM2/22/17
to Scintilla mailing list
Microsoft Visual C++ 2017 will be released in around 2 weeks on March 7. I’ll be creating the 4.x branch after that.

* Long Term Windows Compiler

Earlier feedback suggested that GCC 4.8 should be the compiler supported for the 3.x LTS branch on Linux but no equivalent was proposed for the Microsoft Windows compiler. Since Visual C++ 2015 can still build back to XP I’d like this to be the LTS compiler but others may have other needs. Python 2.x requires MSVC 2008 and 3.4 requires MSVC 2010 but 3.5+ can use MSVC 2015 or later.

* Code modernization

Updating large amounts of code often causes regressions which should be avoided. So don’t send in any large automatically generated change sets. Running clang-tidy with all the modernizers on produces thousands of lines of changes. The largest set is for using “nullptr” instead of “NULL” which doesn’t appear to be worthwhile to me. However, nullptr should be used in new code.

Using “override” to specify that a method overrides a virtual method does look like it could prevent bugs and is unlikely to cause any problems. While the ILexer* interfaces are pure virtual so should produce warnings for missing methods, there has been some confused code where methods were implemented but the base class was wrong and this could have been detected with “override". Range-based for loops are good but each is a little different so should be changed piecemeal as that code is being worked on for other reasons.

Using auto instead of explicit types wherever possible is advocated by some but others, including me, are not completely convinced so this should not be converted now. I think new code should be permitted to use auto.

Another side of "don’t change too much” is that changing code before the branch will make it easier to move fixes between branches. A final release should be made before the branch and I’d like it to include changing lexers to use “override" which is available in both GCC 4.8 and MSVC 2015.

Neil

Mike Lischke

unread,
Feb 22, 2017, 5:18:52 AM2/22/17
to scintilla...@googlegroups.com
> * Code modernization
>
> Updating large amounts of code often causes regressions which should be avoided. So don’t send in any large automatically generated change sets. Running clang-tidy with all the modernizers on produces thousands of lines of changes. The largest set is for using “nullptr” instead of “NULL” which doesn’t appear to be worthwhile to me. However, nullptr should be used in new code.

On the other hand this is probably the best time ever to do bigger refactorings like that. It's probably also worth to run clang-format over all the code to get a consistent formatting, before anything else is changed. This will later make merges a lot easier. Also I would *much* appreciate if you could switch to consistent line endings (preferably \n).

>
> Using “override” to specify that a method overrides a virtual method does look like it could prevent bugs and is unlikely to cause any problems. While the ILexer* interfaces are pure virtual so should produce warnings for missing methods, there has been some confused code where methods were implemented but the base class was wrong and this could have been detected with “override".

+1, I've been bitten more than once by unintentional function overloading instead of overriding.

> Range-based for loops are good but each is a little different so should be changed piecemeal as that code is being worked on for other reasons.

+1, similar for initializer lists and other new language features

>
> Using auto instead of explicit types wherever possible is advocated by some but others, including me, are not completely convinced so this should not be converted now. I think new code should be permitted to use auto.

Also here +1, makes not much sense to change existing code to auto, but it makes certain things a lot easier to consume (e.g. when you replace lengthy iterator types with auto).

Mike
--
www.soft-gems.net

Neil Hodgson

unread,
Feb 22, 2017, 6:39:05 PM2/22/17
to scintilla...@googlegroups.com
Mike Lischke:

> It's probably also worth to run clang-format over all the code to get a consistent formatting, before anything else is changed.

Haven’t been able to make clang-format do as much as astyle and astyle still makes a few hand-formatted elements worse. Hey, there’s a new (December) version of astyle to try!

Formatters make a lot of no-real-effect changes so make it harder to trace back through source code control which has become quite an important tool for me.

> This will later make merges a lot easier. Also I would *much* appreciate if you could switch to consistent line endings (preferably \n).

Using ‘\n' on Windows is likely to lead to more problems as users normally use editors with native line endings leading to mixed line-ending files. They’ll also contribute messy patches.

> Also here +1, makes not much sense to change existing code to auto, but it makes certain things a lot easier to consume (e.g. when you replace lengthy iterator types with auto).

One of the reasons people advocate auto is to make code adapt to changes more easily such as when a method return is changed from 32-bit to 64-bit. An explicitly typed receiver may drop data but auto is always big enough.

Neil

Matthew Brush

unread,
Feb 22, 2017, 8:23:08 PM2/22/17
to scintilla...@googlegroups.com
On 2017-02-22 03:40 PM, Neil Hodgson wrote:
> Mike Lischke:
>
>> It's probably also worth to run clang-format over all the code to get a consistent formatting, before anything else is changed.
>
> Haven’t been able to make clang-format do as much as astyle and astyle still makes a few hand-formatted elements worse. Hey, there’s a new (December) version of astyle to try!
>

clang-format definitively doesn't have as many options as others but if
you tweak it to a style that matches the general style you want, IMO it
works much better and you basically never have to worry about formatting
again.

In case you weren't aware, you can use `// clang-format off` and `//
clang-format on` comments around specially formatted blocks (ex. data
tables and such). It's not a perfect solution, but it works well for the
odd case where it makes the code more readable to align stuff
differently than the configured style.

> Formatters make a lot of no-real-effect changes so make it harder to trace back through source code control which has become quite an important tool for me.
>

This is always a problem. I guess it's better to just do the whole thing
in one go rather than having a bunch of unrelated changes in various
commits, but it still mucks with the history. The nice thing about doing
it in one large commit is thereafter you can require code formatted
correctly, either by policy, commit hooks, or even just running it on
the diff before you merge it.

Regards,
Matthew Brush

Mike Lischke

unread,
Feb 23, 2017, 4:22:45 AM2/23/17
to scintilla...@googlegroups.com

>> It's probably also worth to run clang-format over all the code to get a consistent formatting, before anything else is changed.
>
> Haven’t been able to make clang-format do as much as astyle and astyle still makes a few hand-formatted elements worse. Hey, there’s a new (December) version of astyle to try!
>
> Formatters make a lot of no-real-effect changes so make it harder to trace back through source code control which has become quite an important tool for me.

Matthew brought up a few points I fully support. In my team we switched to the Google coding style recently and reformatted several hundred of thousands of lines of code with clang-format and most of that was exactly as we want it. Only lambdas/closures have a weird formatting, but then there is no established format yet for them.

>
>> This will later make merges a lot easier. Also I would *much* appreciate if you could switch to consistent line endings (preferably \n).
>
> Using ‘\n' on Windows is likely to lead to more problems as users normally use editors with native line endings leading to mixed line-ending files. They’ll also contribute messy patches.

I'm not sold on this argument. Most of the code is cross platform, hence there is no single line ending style for all of the platforms, anyway. So it makes a lot of sense to settle on one and keep using that consistently. All common IDEs (Visual Studio, VS code, XCode, Eclipse, Netbeans etc.) can cope with either style and also keep what is used in a file.

>
>> Also here +1, makes not much sense to change existing code to auto, but it makes certain things a lot easier to consume (e.g. when you replace lengthy iterator types with auto).
>
> One of the reasons people advocate auto is to make code adapt to changes more easily such as when a method return is changed from 32-bit to 64-bit. An explicitly typed receiver may drop data but auto is always big enough.


:-) I'm not sure if that is a positive argument or not. In a strongly typed environment like C++ we want to know data sizes at all times. But auto is certainly a very useful tool and we use it in many places.

Mike
--
www.soft-gems.net

Neil Hodgson

unread,
Feb 24, 2017, 1:36:21 AM2/24/17
to Scintilla mailing list
Mike Lischke:

> Matthew brought up a few points I fully support. In my team we switched to the Google coding style recently and reformatted several hundred of thousands of lines of code with clang-format and most of that was exactly as we want it. Only lambdas/closures have a weird formatting, but then there is no established format yet for them.

For complex calls and expressions that spill over multiple lines, both formatters try to align things and thus push code more to the right than I would. These are the settings I prefer for the formatters with astyle’s results being closer to how I like than clang-format:

.astylerc:
--style=attach
--indent=force-tab=8
--keep-one-line-blocks
--pad-header
--unpad-paren
--pad-comma
--indent-cases
--align-pointer=name

.clang-format:
BasedOnStyle: Google
IndentWidth: 8
UseTab: Always
AllowShortBlocksOnASingleLine: false
AllowShortFunctionsOnASingleLine: false
AllowShortIfStatementsOnASingleLine: false
AllowShortLoopsOnASingleLine: false
IndentCaseLabels: false
AlignAfterOpenBracket: false
AlignTrailingComments: false
ColumnLimit: 110
SortIncludes: false
AccessModifierOffset: -8
DerivePointerAlignment: false
PointerAlignment: Right

Lexers arrive from many contributors and there is a good chance that only the original contributor will ever make substantive changes. Its also unlikely that I or others that don’t use a language will see accidental regressions. Therefore, I like to leave lexers in their original style and that includes formatting.

> :-) I'm not sure if that is a positive argument or not. In a strongly typed environment like C++ we want to know data sizes at all times. But auto is certainly a very useful tool and we use it in many places.

Yeah, I’m uncertain too so don’t want to make changes that will be regretted.

Neil

Mike Lischke

unread,
Feb 24, 2017, 5:29:09 AM2/24/17
to scintilla...@googlegroups.com
>
> For complex calls and expressions that spill over multiple lines, both formatters try to align things and thus push code more to the right than I would.

Yeah, and so far there is no setting to prevent that. So we reverted changes to those lines and from now on format only parts of a document, rarely the entire file.
You prefer tabs? oh my... :-)

Our settings are more explicit, even though we often use the same value as is set by the BasedOnStyle template:

Language: Cpp
BasedOnStyle: Google
AccessModifierOffset: -2
AlignAfterOpenBracket: Align
AlignConsecutiveAssignments: false
AlignConsecutiveDeclarations: false
AlignEscapedNewlinesLeft: true
AlignOperands: true
AlignTrailingComments: true
AllowAllParametersOfDeclarationOnNextLine: true
AllowShortBlocksOnASingleLine: false
AllowShortCaseLabelsOnASingleLine: false
AllowShortFunctionsOnASingleLine: false
AllowShortIfStatementsOnASingleLine: false
AllowShortLoopsOnASingleLine: false
AlwaysBreakAfterDefinitionReturnType: None
AlwaysBreakAfterReturnType: None
AlwaysBreakBeforeMultilineStrings: true
AlwaysBreakTemplateDeclarations: true
BinPackArguments: true
BinPackParameters: true
BraceWrapping:
AfterClass: false
AfterControlStatement: false
AfterEnum: false
AfterFunction: false
AfterNamespace: false
AfterObjCDeclaration: false
AfterStruct: false
AfterUnion: false
BeforeCatch: false
BeforeElse: false
IndentBraces: false
BreakBeforeBinaryOperators: None
BreakBeforeBraces: Attach
BreakBeforeTernaryOperators: true
BreakConstructorInitializersBeforeComma: false
BreakAfterJavaFieldAnnotations: false
BreakStringLiterals: true
ColumnLimit: 120
CommentPragmas: '^ IWYU pragma:'
ConstructorInitializerAllOnOneLineOrOnePerLine: true
ConstructorInitializerIndentWidth: 2
ContinuationIndentWidth: 2
Cpp11BracedListStyle: true
DerivePointerAlignment: true
DisableFormat: false
ExperimentalAutoDetectBinPacking: false
ForEachMacros: [ foreach, Q_FOREACH, BOOST_FOREACH ]
IncludeCategories:
- Regex: '^<.*\.h>'
Priority: 1
- Regex: '^<.*'
Priority: 2
- Regex: '.*'
Priority: 3
IncludeIsMainRegex: '([-_](test|unittest))?$'
IndentCaseLabels: true
IndentWidth: 2
IndentWrappedFunctionNames: false
JavaScriptQuotes: Leave
JavaScriptWrapImports: true
KeepEmptyLinesAtTheStartOfBlocks: false
MacroBlockBegin: ''
MacroBlockEnd: ''
MaxEmptyLinesToKeep: 1
NamespaceIndentation: All
ObjCBlockIndentWidth: 2
ObjCSpaceAfterProperty: false
ObjCSpaceBeforeProtocolList: false
PenaltyBreakBeforeFirstCallParameter: 1
PenaltyBreakComment: 300
PenaltyBreakFirstLessLess: 120
PenaltyBreakString: 1000
PenaltyExcessCharacter: 1000000
PenaltyReturnTypeOnItsOwnLine: 200
PointerAlignment: Left
ReflowComments: true
SortIncludes: false
SpaceAfterCStyleCast: false
SpaceBeforeAssignmentOperators: true
SpaceBeforeParens: ControlStatements
SpaceInEmptyParentheses: false
SpacesBeforeTrailingComments: 1
SpacesInAngles: false
SpacesInContainerLiterals: true
SpacesInCStyleCastParentheses: false
SpacesInParentheses: false
SpacesInSquareBrackets: false
Standard: Auto
TabWidth: 8
UseTab: Never

and no tabs, phew :-D

I still need a setting to insert spaces in initializer lists (after opening curly brace and before closing one). Haven't found any yet.

Mike
--
www.soft-gems.net



Mike
--
www.soft-gems.net

Matthew Brush

unread,
Feb 24, 2017, 10:32:45 AM2/24/17
to scintilla...@googlegroups.com
On 2017-02-23 10:38 PM, Neil Hodgson wrote:
> Mike Lischke:
>
>> Matthew brought up a few points I fully support. In my team we switched to the Google coding style recently and reformatted several hundred of thousands of lines of code with clang-format and most of that was exactly as we want it. Only lambdas/closures have a weird formatting, but then there is no established format yet for them.
>
> For complex calls and expressions that spill over multiple lines, both formatters try to align things and thus push code more to the right than I would. These are the settings I prefer for the formatters with astyle’s results being closer to how I like than clang-format:
>
> [...]
> AlignAfterOpenBracket: false
>

Try `DontAlign` instead of `false`, which doesn't appear to be a valid
value[0].

Regards,
Matthew Brush

[0]: https://clang.llvm.org/docs/ClangFormatStyleOptions.html

Neil Hodgson

unread,
Feb 24, 2017, 5:30:51 PM2/24/17
to scintilla...@googlegroups.com
Matthew Brush:
>> AlignAfterOpenBracket: false
>>
>
> Try `DontAlign` instead of `false`, which doesn't appear to be a valid value[0].

OK, that helps although it seems to indent just 4 spaces for the additional lines rather than IndentWidth.

Attached is a patch with the override changes made by clang-tidy for files built for GTK+ - not qt, win32, or cocoa. The bulk is from lexers and they are all quite simple. I’ll apply these except for ScintillaGTK/Accessible.cxx as these files are being worked on for bug #1910.

override.patch

Neil Hodgson

unread,
Mar 4, 2017, 7:03:50 PM3/4/17
to Scintilla mailing list
The “override” changes have been committed along with similar changes for the Windows and Cocoa platform layers.
https://sourceforge.net/p/scintilla/code/ci/7efe018ffe6288d46c7ab4f2b1ef407a800a1f13/
https://sourceforge.net/p/scintilla/code/ci/d4f7d1439daa636ffe50cec516667627e04174a7/
https://sourceforge.net/p/scintilla/code/ci/a3894ee30cdba2c1ced617f80bc6eb900e5cbb42/

The http://en.cppreference.com/w/cpp/compiler_support page lists the (mostly compiler) features for C++11 (3rd table) along with the compilers that support these. There are also some library additions, notably tuple, regex, unordered_map, and smart pointers (unique_ptr, …).

Almost every C++11 feature that is useful is available from GCC 4.8, Visual C++ 2015, Clang 3.3, and Xcode 8.2 Clang (Xcode’s versions of Clang aren’t exactly the same as standard Clang versions). Most of the features are also available in GCC 4.6 and Visual C++ 2013. The only feature that appears completely non-useful is garbage collection support which no compilers currently implement.

To check that these really work, I implemented an example for each of the features I think most likely to be used in the attached patch. This patch worked completely in GCC 4.6, Clang 3.8, Xcode 8.2 (the current version), and Visual C++ 2013 (except for ‘noexcept’). I didn’t search for the minimum version of Clang since Clang often came with poor library support on older Linux distributions and it isn’t treated as the standard compiler on Linux. The features implemented with the patch or in current Scintilla are:

* auto
* defaulted and deleted functions
* lambda expressions
* local types as template parameters
* new character types
* nullptr
* r-value references
* static_assert
* strongly-typed enum
* range-for loop
* override (already committed)
* noexcept
* regex (already released)
* unique_ptr

The examples in the patch range in quality from changes that should be made now to less certain changes.

The changes that I think should be committed now are:
* The ‘move constructor’ for CharacterSet that uses an r-value reference which is a small optimisation that avoids some memory allocations in the C++ lexer.
* The unique_ptr used in Document for the regex. Over time, I’d like to move most applicable raw pointers to unique_ptr or shared_ptr.
* Deleted functions like the copy constructor and operator= for RGBAImage as this expresses that they must not be copied better than simply making them private.
* The ‘enum class’ for WrapScope. I’ve found enum class works well for true closed enumerations but less well for bit flags or where the enumeration lists only the extreme values and intermediate values are possible. Much casting can occur for these situations.
* The lambda expression used to sort selections in Editor as this avoids a named function that is only ever used in one place.

Committing these makes it more likely that people with incompatible compilers will complain now rather than after larger-scale changes are made, potentially allowing some negotiation over features used and compiler versions supported.

Neil
cxx11.patch

Mike Lischke

unread,
Mar 5, 2017, 5:23:37 AM3/5/17
to scintilla...@googlegroups.com
Am 05.03.2017 um 01:03 schrieb Neil Hodgson <nyama...@me.com>:

...

* The unique_ptr used in Document for the regex. Over time, I’d like to move most applicable raw pointers to unique_ptr or shared_ptr.

Automatic memory handling is certainly one of the most desired things here. Be careful with shared_ptr though, it can be slow, if you create many of them (perhaps in a loop) or when passing them around. Here's a discussion I had on SO about how to pass shared pointers around: http://stackoverflow.com/questions/37610494/passing-const-shared-ptrt-versus-just-shared-ptrt-as-parameter. Best is probably to use raw pointers everywhere unless you want to further control life time of that pointer.


Teodor Petrov

unread,
Mar 5, 2017, 10:32:26 AM3/5/17
to scintilla...@googlegroups.com
On 03/05/2017 02:03 AM, Neil Hodgson wrote:
> Committing these makes it more likely that people with incompatible compilers will complain now rather than after larger-scale changes are made, potentially allowing some negotiation over features used and compiler versions supported.

Does this mean that you've given up on your plan to switch everything to
c++17?
Or this discussion is only valid for the critical-backports-branch?

/Teodor

Neil Hodgson

unread,
Mar 5, 2017, 4:47:05 PM3/5/17
to scintilla...@googlegroups.com
Teodor Petrov:

> Does this mean that you've given up on your plan to switch everything to c++17?
> Or this discussion is only valid for the critical-backports-branch?

This is for changes before the branching so it applies to both the 4.x and long term branches.

Having C++11 features available in the long term branch will lead to less work and fewer problems in backporting.

Neil

Neil Hodgson

unread,
Mar 6, 2017, 7:18:47 PM3/6/17
to scintilla-interest
Mike:

> Be careful with shared_ptr though, it can be slow, if you create many of them (perhaps in a loop) or when passing them around.

Meyers in “Effective Modern C++” is quite useful on shared_ptr. I first read the book before using C++11 much and it seemed full of boring, arcane details. After working with the new features it has become much more interesting.

Neil

Neil Hodgson

unread,
Mar 7, 2017, 10:40:23 PM3/7/17
to scintilla-interest
Neil Hodgson:

> To check that these really work, I implemented an example for each of the features I think most likely to be used in the attached patch. This patch worked completely in GCC 4.6,

I was wrong here: GCC 4.6 does not support “override”. However, it will compile the current code by defining “override” to nothing with -Doverride= . Since override is an additional check, defining it out of existence works fine although you won’t see warnings in that case.

Neil

Mitchell

unread,
Mar 8, 2017, 10:10:16 PM3/8/17
to scintilla...@googlegroups.com
Hi Neil,
For what it's worth, I will not be able to benefit from your changes going
forward (even on LTS) because I am currently restricted to an ancient
Apple cross-compiler toolchain that originally targeted OSX 10.4-10.6 (GCC
4.2). I also have a build server with GCC 4.4 on it (both Linux and MinGW)
where trying to upgrade or install a newer version side-by-side with the
old is probably more trouble than it's worth.

I totally understand the need to make maintenance easier for you going
down the road, and I encourage you to do what you think is best. I am
probably an outlier, but I am a data point and I thought I should let you
know :)

If you'd like more information or any error messages I'm getting when
trying to build Scintilla on my setup, please let me know, but I don't
want to bother you with it seeing as you prefer to support at least GCC
4.6 for now.

Cheers,
Mitchell
Reply all
Reply to author
Forward
0 new messages