CMake support

[Robert Dailey]

Hey guys, I've created a build script for CMake that I can use to
build Scintilla with Visual Studio 2008. Additional work will need to
be done to make this work on mac/linux/other, and unfortunately I do
not have the environment to test these. However, this script certainly
works on Windows and I can help with getting it to work on other
platforms.

Would you guys find this useful? Can you look it over and let me know
if the way I've set up the library to build is incorrect? Note that I
currently have it setup to build as a shared library (which isn't
correct, since the code does not export symbols with
declspec(dllexport). I can easily change it to build as static, but I
kept it as shared for now so that I can resolve linker issues.

[Neil Hodgson]

Robert Dailey:

> Would you guys find this useful?

Some people may prefer CMake but its not something that I would want to distribute. There have been alternate build files distributed in the past and without maintenance their usefulness decayed.

A common change to Scintilla is to add a lexer and this will break your CMake scripts since they hard-code the list of lexer source code files. There are two techniques used to help with this in Scintilla. Build files that can use wild card file patterns use that to compile and link all the C++ files lexers/*.cxx -- for example, qt/ScintillaEditBase/ScintillaEditBase.pro. Other textual build files are modified by the src/LexGen.py script to refer to each of the lexer files.

Neil

[Robert Dailey]

You can use glob patterns to determine source files that CMake will
include, but this is generally not recommended because then CMake
can't tell if new source files are added (it does so by inspecting if
the file has changed, and it wouldn't with a glob). Honestly I don't
see a problem here -- if new source files are added, simply add them
to the CMake script. It's not difficult to do, and having automation
for this doesn't buy you much (unless you aren't telling me
everything).

Honestly I don't know why people wouldn't use CMake. It can generate
makefiles (so you essentially get exactly what you have now, with 1
extra step), but also allows developers to choose their toolset (code
blocks, visual studio, eclipse, etc) (which is especially valuable on
Windows). I think continuing to use make directly is unintuitive.

Anyway I did the work for you, to make it as easy as possible to adopt
it (and I'm also willing to make adjustments / improvements to the
script depending on needs of the community). I say at least give it a
shot before you completely reject someone's efforts (it's rather
offensive to me, since I took my free time to do this for the
project). I decided to do this because I wanted to work on the CMake
lexer and improve it.

> --
> You received this message because you are subscribed to the Google Groups "scintilla-interest" group.
> To post to this group, send email to scintilla...@googlegroups.com.
> To unsubscribe from this group, send email to scintilla-inter...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/scintilla-interest?hl=en.
>

[Dave Brotherstone]

On Mon, Sep 10, 2012 at 5:36 PM, Robert Dailey <rcdaile...@gmail.com> wrote:

 It can generate
makefiles (so you essentially get exactly what you have now, with 1
extra step), but also allows developers to choose their toolset (code
blocks, visual studio, eclipse, etc) (which is especially valuable on
Windows). 

I'd find this especially useful.  And, I don't see any particular reason why the current nmake file for windows couldn't be generated from cmake, and checked-in, so we don't lose anything.  I don't know the status of absolute / relative paths with the various generators for CMake, that might be an issue. 

Thus, assuming the paths remain relative, we lose nothing, and gain the massive advantage of being able to easily generate project files for each version of Visual Studio (for example).  This is awkward, especially when we upgrade Scintilla. We now "officially" use the nmake build, but that means we can't have the project open in VS, at least not without manually working out what files have been added etc.

If Neil is really against checking the CMake files in, what about a contrib directory?  Would seem like a reasonable compromise.  That would give the CMake build process time to mature, obviously it's going to have a few wrinkles on each platform at first.

Dave.

[Robert Dailey]

CMake converts all paths to relative when the projects are generated,
I'd have to verify the same is true for other generators (like make).

I'm happy with whatever needs to be done to trial / prototype the
build system until it's ready for consumption. However, I'd also like
to point out that CMake can reside side-by-side with the current build
system and it shouldn't interfere with anything (it'll just be an
optional choice). It'll just be 3 new files in the root directory of
the project (in fact, the find modules can be placed in a 'cmake'
subdirectory).

I chose a quick, straight-forward and naive script implementation for
now, and the single file is quite bloaty (could be simplified and
modularized into other script files). However, that was just a first
pass to demonstrate its functionality. There really is no limit to how
we can structure, modularize, and refactor the scripts. I'll be here
to help out if someone wants to look into other platforms with me. If
I have time later, I'll install a VM with linux on it and try it
there, but I'm not very knowledgeable with linux so I'll still need
some help.

[Robert Dailey]

Sorry I meant to say:

CMake converts all paths to *absolute* when the projects are generated

But I will verify this to make sure it works like this outside of
visual studio generators.

[Robert Dailey]

And here's the answer:
http://www.cmake.org/Wiki/CMake_FAQ#Why_does_CMake_use_full_paths.2C_or_can_I_copy_my_build_tree.3F

Full paths are used on all platforms it seems.

[Lex Trotman]

On 11 September 2012 01:36, Robert Dailey <rcdaile...@gmail.com> wrote:
> You can use glob patterns to determine source files that CMake will
> include, but this is generally not recommended because then CMake
> can't tell if new source files are added (it does so by inspecting if
> the file has changed, and it wouldn't with a glob). Honestly I don't
> see a problem here -- if new source files are added, simply add them
> to the CMake script. It's not difficult to do, and having automation
> for this doesn't buy you much (unless you aren't telling me
> everything).
>

Downstreams use differing variations of lexers, so they have to edit
the cmake each time they update scintilla to match the list they
provide.

> Honestly I don't know why people wouldn't use CMake. It can generate
> makefiles (so you essentially get exactly what you have now, with 1
> extra step), but also allows developers to choose their toolset (code
> blocks, visual studio, eclipse, etc) (which is especially valuable on
> Windows). I think continuing to use make directly is unintuitive.
>

Scintilla isn't usually built by itself, its built as part of building
the application, so it has to integrate into whatever build system the
particular app uses. And apps not using cmake would then have another
build dependency and tool to handle. Scintilla shouldn't enforce what
build tool the downstreams use.

> Anyway I did the work for you, to make it as easy as possible to adopt
> it (and I'm also willing to make adjustments / improvements to the
> script depending on needs of the community). I say at least give it a
> shot before you completely reject someone's efforts (it's rather
> offensive to me, since I took my free time to do this for the
> project). I decided to do this because I wanted to work on the CMake
> lexer and improve it.

Its probably always a good idea to ask before you expend effort,
nobody is trying to offend you, but differing opinions and points of
view mean that "good" ideas may not be well received because they
impact others negatively.

Cheers
Lex

[Robert Dailey]

On Mon, Sep 10, 2012 at 6:32 PM, Lex Trotman <ele...@gmail.com> wrote:
> Downstreams use differing variations of lexers, so they have to edit
> the cmake each time they update scintilla to match the list they
> provide.

Fair point, but there are multiple ways to solve this problem:
- Maintain a fixed list of source for the Scintilla (known) source,
and allow globbing expressions for downstream
- Continue to use fixed lists, and have the facilities in place for
downstream projects to pass in their own fixed lists, which would be
appended to the main list
- Use glob for everything (most extensible, but CMake won't be able to
detect if files were added, and thus won't automatically regenerate
its targets for you)

> Scintilla isn't usually built by itself, its built as part of building
> the application, so it has to integrate into whatever build system the
> particular app uses. And apps not using cmake would then have another
> build dependency and tool to handle. Scintilla shouldn't enforce what
> build tool the downstreams use.

As of right now, Scintilla already forces downstreams to use
make/nmake. The irony in what you are saying is that CMake actually
means more flexibility. One project won't be limited to one build
system, they can use a wide variety of build systems. IMHO, it's much
more difficult to mix makefiles + visual studio projects then it is to
mix cmake + visual studio (because, Scintilla can be setup to generate
visual studio projects, which can then be included in a downstream
project's existing solutions). Same applies to code blocks, and
especially makefiles. It's just 1 extra tool, 1 extra command to get
things going.

> Its probably always a good idea to ask before you expend effort,
> nobody is trying to offend you, but differing opinions and points of
> view mean that "good" ideas may not be well received because they
> impact others negatively.

Typically open source projects are more flexible, and are what they
are based on community contribution and ideas. I've not known them to
have an authoritative source (like a company) that would actually
reject reasonable contributions. Just seems backwards and
counter-intuitive to me, and discourages community involvement. Why
shouldn't I be able to pick up something that I think is a great idea
and just submit it, no questions asked? CMake won't interfere with
existing build pipelines, but only adds another option. As far as
maintainability is concerned, that doesn't have to be one person's
responsibility. Each would have its domain expert that maintains it -
seems easy to me.

Anyway, I couldn't work on Windows without it. I just can't deal with
Cygwin/NMake on Windows. I have some other code I want to modify to
improve the CMake lexer so this was an intermediate step for me. I
just offered it up just in case you guys would find some use in it
(but that's subjective, some people will, some won't).

[Neil Hodgson]

Robert Dailey:

> Honestly I don't
> see a problem here -- if new source files are added, simply add them
> to the CMake script. It's not difficult to do, and having automation
> for this doesn't buy you much (unless you aren't telling me
> everything).

Files not being updated with additional lexers in the past have caused problems.

> It can generate
> makefiles (so you essentially get exactly what you have now, with 1
> extra step), but also allows developers to choose their toolset (code
> blocks, visual studio, eclipse, etc) (which is especially valuable on
> Windows).

The make files that cmake creates are more complex than those written by hand making them more difficult to understand and debug.

> I say at least give it a
> shot before you completely reject someone's efforts (it's rather
> offensive to me, since I took my free time to do this for the
> project).

I think your attitude is very poor here: you sent in some code that was not asked for and will be 'offended' if no one wants to look at it.

All contributions have costs and if those costs are not justified then they will not be accepted.

> I decided to do this because I wanted to work on the CMake
> lexer and improve it.

Which is fine.

Neil

[Neil Hodgson]

Robert Dailey:

> Same applies to code blocks, and
> especially makefiles. It's just 1 extra tool, 1 extra command to get
> things going.

Its an additional tool that developers have to download, install, configure, understand and use. Scintilla tries to avoid additional tool (and library) dependencies: people can install a compiler and use the build tool that is packaged with that compiler.

> I've not known them to
> have an authoritative source (like a company) that would actually
> reject reasonable contributions.

Most open-source projects will reject contributions that do not provide sufficient benefits for their costs. Some develop quite complex defense mechanisms like Python's PEP process to ensure new features are viable in the long term.

> Just seems backwards and
> counter-intuitive to me, and discourages community involvement. Why
> shouldn't I be able to pick up something that I think is a great idea
> and just submit it, no questions asked?

Who is going to maintain this? Will you be here in 5 or 10 years?

> Anyway, I couldn't work on Windows without it. I just can't deal with
> Cygwin/NMake on Windows.

If you "just can't deal" with these then I can't see how you will be able to understand them well enough to accurately convert their functionality to cmake.

Neil

[Matthew Brush]

Just as a note, you could toss up a repository somewhere (ex. Github, Bitbucket, etc.) and keep it up-to-date there. I'd guess that Neil would add a link from the "related" page on Scintilla.org and perhaps even a note in one of the doc/readme files for CMake users to find it. If you "announce" it on this list with a link to the repo, anyone searching for "scintilla cmake" will more than likely be able to find it since this list is well indexed by (at least) Google.

I rarely use CMake myself, but my understanding is that you could just unzip/checkout the related files into Scintilla's source tree and run it like that. If it proves to be well maintained and a lot of people use it, it might make a better case for upstreaming it.

Just my 2 cents.

Cheers,
Matthew Brush

[Robert Dailey]

On Mon, Sep 10, 2012 at 7:38 PM, Neil Hodgson <nyama...@me.com> wrote:
> Robert Dailey:
>
>> Honestly I don't
>> see a problem here -- if new source files are added, simply add them
>> to the CMake script. It's not difficult to do, and having automation
>> for this doesn't buy you much (unless you aren't telling me
>> everything).
>
> Files not being updated with additional lexers in the past have caused problems.

Sounds like a programmer problem to me. Lack of discipline, maybe?

>
>> It can generate
>> makefiles (so you essentially get exactly what you have now, with 1
>> extra step), but also allows developers to choose their toolset (code
>> blocks, visual studio, eclipse, etc) (which is especially valuable on
>> Windows).
>
> The make files that cmake creates are more complex than those written by hand making them more difficult to understand and debug.

Why do you need to care? You should never be looking at Makefile code
generated by cmake, much less debug it. You work with CMake scripts
and that's it.

>
>> I say at least give it a
>> shot before you completely reject someone's efforts (it's rather
>> offensive to me, since I took my free time to do this for the
>> project).
>
> I think your attitude is very poor here

On the contrary, I think yours is.

> you sent in some code that was not asked for and will be 'offended' if no one wants to look at it.
>
> All contributions have costs and if those costs are not justified then they will not be accepted.

The problem I have with your response to my work is that you haven't
even tried it, nor will you even investigate it. Making a judgement
call based on experience / testing is one thing, but doing so without
any facts and just your personal "gut" feeling isn't acceptable, and
it's silly. You are primarily focused on your subjective view of what
I'm offering instead of considering a community-wide opinion. But hey,
I guess you're the omnipotent decision maker here, and no one should
challenge your opinions right?

I might end up just forking Scintilla and putting it on github, and
just keep my work to myself (you'd prefer that anyhow).

[Neil Hodgson]

Robert Dailey:

> Why do you need to care? You should never be looking at Makefile code
> generated by cmake, much less debug it. You work with CMake scripts
> and that's it.

When there are problems with a build you have to debug it at the makefile level because that is where you see any errors.

> The problem I have with your response to my work is that you haven't
> even tried it, nor will you even investigate it.

But I did investigate it. If I hadn't then I wouldn't have been able to point out the problem with the hard-coded lexer list.

> Making a judgement
> call based on experience / testing is one thing, but doing so without
> any facts and just your personal "gut" feeling isn't acceptable, and
> it's silly.

The facts that were presented were contained in your CMake scripts. That was the material on which I formed a judgment.

> You are primarily focused on your subjective view of what
> I'm offering instead of considering a community-wide opinion.

I can't see any opinion widely shared by the Scintilla community on this matter.

> I might end up just forking Scintilla and putting it on github, and
> just keep my work to myself (you'd prefer that anyhow).

Forks are good.

Neil

[na...@skydreams.org]

>> You are primarily focused on your subjective view of what
>> I'm offering instead of considering a community-wide opinion.
>
> I can't see any opinion widely shared by the Scintilla community on
> this matter.
>

I can of course only talk for myself here but when I started to review how
feasible it would be to port Scintilla to MorphOS a few years ago, I
immediately appreciated two things:

1) It didn't rely on an overly and messy toolchain like many *nix
projects. On MorphOS we have access to most of the typical GNU tools but
not always the latest version given that our resources are stretched. It's
also nice to understand what it going on in the build process and one more
tool/layer doesn't help in that respect.

2) Scintilla's source code is conservative in it's use of "advanced" c++
features.

I would hate to lose any of these.

I also don't see why the standard Scintilla tree should optionally include
build systems which are not used by the main developer. As already pointed
out, it will likely become dead and unmaintained sooner or later.

Best,

Nicholai