[vim/vim] Decouple C and C++ syntax files (Issue #18224)

[Amelia Clarke]

selenebun created an issue (vim/vim#18224)

Is your feature request about something that is currently impossible or hard to do? Please describe the problem.

Currently, the C and C++ syntax files are very tightly coupled, with the C++ syntax file relying heavily on syntax defined within the corresponding C file. As a result, the C syntax file has numerous special cases where it has been forced to check whether it was included from the C++ syntax file before defining syntax elements.

This adds a great deal of complexity, with many instances of deeply nested if-statements, and greatly hampers maintainability. This holds especially true when it comes to the handling of different language versions.

While C and C++ are very similar languages, C++ is no longer a superset of C, and the two have grown more divergent over time. I don't think the current organizational approach is particularly viable anymore, and expect it to only become more brittle with time.

Describe the solution you'd like

While this necessarily results in a lot of near-duplication, I believe a more maintainable approach would be to refactor the C++ syntax file so that it no longer depends on the C syntax file at all and is able to stand alone. This would result in significantly fewer moving parts and massively simplify both files overall.

Describe alternatives you've considered

A possible alternative would be to instead have a separate syntax file containing the common elements of both languages, which either can then safely include without being coupled to each other. However, I feel like this would likely not save as much work as it seems at first (most likely only block comments and shared keywords) as many other language elements — such as number or string literals — have changed quite a bit between language versions.

It might also be possible to refactor the C syntax file in such a way that it still expects to be included by the C++ syntax file, but contains far fewer nested conditionals and other special cases. However, I feel like this is probably easier said than done, though still worth consideration.

Additional context

I am entirely unfamiliar with Objective-C and Objective-C++, but both are strict supersets of C and C++, respectively (as far as I understand), and therefore should not need much in the way of special handling.

—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/issues/18224@github.com>

[Damien Lejay]

dlejay left a comment (vim/vim#18224)

Having a syntax file syntax/c90.vim that would be both included in syntax/c.vim and syntax/cpp.vim could indeed be a cleaner design.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/issues/18224/3263563338@github.com>

[Amelia Clarke]

selenebun left a comment (vim/vim#18224)

Sorry about that, I guess I never ended up seeing this message.

Anyways, I've been experimenting off-and-on with different ways to fully refactor the C syntax file into something a bit more sustainable to update, and I've come up with some prototypes I'm quite happy with. It's only recently that I've started tinkering with the C++ syntax, but I've changed my mind since the initial post, and am increasingly of the opinion that it doesn't make sense to have them wholly separate.

I think a shared syntax/c89.vim¹ could definitely be the way forward. There are some places where I think it's a bit neater to have regex patterns that are mutually exclusive, especially for things that have changed a lot across versions:

For example, here's how I handle integer constants:

if s:c23
  syn match cNumber '\v<[1-9]%(\'?\d)*%([Uu]%(LL?|ll?|WB|wb)?|%(LL?|ll?|WB|wb)[Uu]?)?>'
  syn match cNumber '\v<0%(\'?\o)*%([Uu]%(LL?|ll?|WB|wb)?|%(LL?|ll?|WB|wb)[Uu]?)?>'
  syn match cNumber '\v<0[Bb][01]%(\'?[01])*%([Uu]%(LL?|ll?|WB|wb)?|%(LL?|ll?|WB|wb)[Uu]?)?>'
  syn match cNumber '\v<0[Xx]\x%(\'?\x)*%([Uu]%(LL?|ll?|WB|wb)?|%(LL?|ll?|WB|wb)[Uu]?)?>'
elseif s:c99
  syn match cNumber '\v<[1-9]\d*%([Uu]%(LL?|ll?)?|%(LL?|ll?)[Uu]?)?>'
  syn match cNumber '\v<0\o*%([Uu]%(LL?|ll?)?|%(LL?|ll?)[Uu]?)?>'
  syn match cNumber '\v<0[Xx]\x+%([Uu]%(LL?|ll?)?|%(LL?|ll?)[Uu]?)?>'
else
  syn match cNumber '\v<[1-9]\d*%([Uu][Ll]?|[Ll][Uu]?)?>'
  syn match cNumber '\v<0\o*%([Uu][Ll]?|[Ll][Uu]?)?>'
  syn match cNumber '\v<0[Xx]\x+%([Uu][Ll]?|[Ll][Uu]?)?>'
endif

But for a lot of these, at least if the syntax is backwards compatible, you can just have the C89 version in the common file and rely on the new syntax groups that are defined later taking precedence.

Really, I think the biggest issue with the existing system right now is how it handles versioning. At present, it's all controlled with a bunch of variables like g:c_no_c99, g:c_no_c11, etc.² Aside from this many independent variables adding a lot more complexity to the syntax file, it also makes it much more tedious to use a specific version, since you have to set the variables to disable everything after that point. It also makes it possible to enter a "broken" state where you can disable stuff from a prior version, but still have syntax from a newer version active, which possibly relies on the existence of the earlier version to behave correctly.

Instead, I propose having a single variable, e.g. g:c_standard (or b:c_standard), which controls the version the user is targeting. Each part of the syntax file knows when it was introduced, and when it was deprecated or removed, and then can just test if the selected standard falls within that range. I've had a lot of success with this method, and it definitely lets you simplify a lot of things. It probably wouldn't be too hard to create a small "shim" layer that guesses which version the user wants based on the old variables if g:/b:c_standard doesn't exist. You'd also do a similar thing for C++.

Really, I think the tricky part is just figuring out which functionality to move out into the shared syntax file. While C and C++ have diverged a lot, some of the recent standards have borrowed from each other, and there are certain places like the recent #elifdef/#elifndef and other preprocessing directives that are basically identical. It would be nice to be able to move those parts to a common syntax file as well, but then it would need its own script-local version handling (or defining buffer-local version-related variables before including it and undefining them after).

Apologies for the long comment, I have done a lot of thinking about this, but am not really sure which method would work best for incorporating into the official Vim runtime, especially when it comes to potentially breaking backwards compatibility.

Footnotes

1. C89 tends to be the name people use most often in my experience, but the two are basically identical. ↩

2. This probably made more sense when the language was more conservative and there were only a couple versions you had to worry about, but C23 and C2y have been much more ambitious. ↩

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/issues/18224/5107941384@github.com>