[vim/vim] Modula-2 Support Broken Beyond Repair -- Replacement Offer (#6796)
Benjamin Kowarsch
The Bad News
Support for Modula-2 in Vim has been broken for many years in so many ways that it would take several pages to describe all the issues with it. So, I am only giving a very general overview here:
- many reserved words aren't recognised, for example THEN
- some predefined identifiers are misrecognised as reserved words
- depending on the content of comments, very weird things are happening[*]
- some reserved words are recognised in some context but not in others
- VIM associates weird file extensions with Modula-2 that are never actually used for Modula-2 source files, but it does not recognise those file extensions that are actually used
- overall inconsistent behaviour
Last but not least, there are three different dialects of Modula-2 which require separate treatment. That which Vim does when syntax is set to Modula-2 is not recognisable as any of these.
In other words, the current Modula-2 support in Vim ought to be completely removed altogether and replaced with proper support instead.
The Good News
I offer a complete replacement that has been in use at our project for many years.
It provides multi-dialect Modula-2 support with disambiguation between Modula-2 and the other formats that use the same file extensions, namely *.def and *.mod.
With this customisation, the Vim syntax menu shows a Modula-2 menu that brings up a pop-up window that shows the three main dialects to choose. For each dialect there is a separate syntax file.
There is a convention for Modula-2 by which the dialect can be determined from a dialect tag in a comment placed somewhere at the beginning of a source file. The customisation supports this convention, too.
DEFINITION MODULE String; (*!m2pim*) (* <-- dialect tag for PIM Modula-2 *)
Last but not least, users can set a variable in vimrc to choose a default dialect.
All necessary files and instructions how to patch Vim are on my Github repo:
https://github.com/trijezdci/M2-Syntax-Colour/tree/master/Vim
Back in 2016, one of our collaborators who said he knows Bram personally had tasked himself to contact Bram and offer the patches but since nothing has happened in all these years, I am not sure whether this contact was ever made or whether any such contact actually got through to Bram.
Recently, I had to move the above repo from Bitbucket to Github which reminded me of this issue and this time I am making contact directly myself. I have also sent mail to bu...@vim.org with a copy of this note.
[*] for example, when a quoted opening brace appears within a comment
(* next char is '{' *)all code that appears after the comment is also rendered as a comment until another comment appears which has a quoted closing brace within
(* next char is '}' *)This is just one recently observed example, there is plenty of weird behaviour of this kind.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Bram Moolenaar
Have you tried contacting the old maintainer, Peter Funk? I haven't heard from him for a long time, but it's common to at least try.
The THEN keyword is in the existing file. Not sure why it would not work for you.
Please add a license to your github project. Without it nobody is allowed to copy it.
Using the Vim license is preferred, we can include it with the distribution then.
Benjamin Kowarsch
Thank you for getting back so swiftly. I appreciate it.
As for the license, I am happy to use the Vim license as you suggested.
Do you want me to
(1) copy the license text into every file?
(2) simply add the Vim license file into the repo?
(3) copy the Vim license into the repo and then add a reference to the license to every file?
As for the author of the original Modula-2 support file, do you suggest we ought to ask him for permission to replace his work?
I have not contacted him simply because what I set out to do at the time went beyond the scope of the pre-existing Modula-2 support. It would have been akin to ask somebody else to do my homework. I would have needed to painstakingly investigate and document all the existing issues and then wait for him to fix them, and only after all those issues had been resolved would I have been able to extend the scope. The effort for both him and myself would have far exceeded the effort to simply start from scratch.
Also, this is not the first time I have contributed Modula-2 support to editors or syntax rendering software. In the early days, I merely fixed bugs in and then extended pre-existing support. At first, I extended the set of to be colourised reserved words and functions to the union of all dialects. This was then supposed to ensure that no matter for what dialect the source code to be rendered was written for, it would always colourise all the entities. But then there were people, who seemed to only know about Wirth's original work (the PIM dialect) and they then removed all the support for the ISO dialect that I had added.
I had also been participating in open source projects where similar situations led to a patch submission war, where one group of contributors had engaged in discussions and decided upon a design which was then documented and implemented, only to have some other folks come along and wipe it out to replace it with their own competing implementation and then there was a constant back and forth until one side would give up.
When it comes to supporting syntax colouring or rendering there is only one way to prevent any such situation from ever arising and that is multi-dialect support with a means for every user to set their preferred dialect.
It was for this very reason that I then contacted other Modula-2 compiler maintainers/implementors to discuss and agree a convention how to encode the dialect directly into the source files in form of a special comment that contains a dialect tag.
Whenever I implemented Modula-2 support in this way, there has never been anyone undoing support for a dialect, everybody can simply use their preferred dialect without having syntax entities colourised that do not belong to that dialect. It has also led to a certain standardisation of Modula-2 syntax rendering between different tools.
Gaius Mulley, the implementor of GNU Modula-2 has implemented the same solution for Emacs for example.
Before this background, I trust you will agree that it would not have been appropriate to ask somebody else to do this for me but to do it myself and then offer the patches as a replacement.
I hope this makes sense
Benjamin Kowarsch
Meanwhile I have sent an email to the email address listed in the pre-existing Modula-2 syntax file for Peter Funk to ask him if he is OK if we replace his support file.
The mail bounced with
DNS Error: 10643534 DNS type 'mx' lookup of artcom0.north.de responded with code NXDOMAIN Domain name not found: artcom0.north.de
Benjamin Kowarsch
I managed to find a new email address for Peter Funk, resent the mail and have promptly received a reply:
I am writing to you because you are listed as the maintainer of the
Modula-2 syntax file for Vim.I have implemented multi-dialect Modula-2 support for Vim which I have
offered to Bram Moolenaar as a replacement to your earlier work.https://github.com/trijezdci/M2-Syntax-Colour/tree/master/Vim
Out of courtesy, I would like to ask you if you have any objection.
...No. I have no objection and I will try yours as soon as my time permits.
Thank you very much for asking and for spending your time to do it all
over correctly.Kind regards or
as we write in German: Liebe Grüße,
Peter Funk
Bram Moolenaar
Benjamin Kowarsch wrote:
> Thank you for getting back so swiftly. I appreciate it.
>
> As for the license, I am happy to use the Vim license as you suggested.
>
> Do you want me to
>
> (1) copy the license text into every file?
> (2) simply add the Vim license file into the repo?
> (3) copy the Vim license into the repo and then add a reference to the
> license to every file?
Since using the files requires getting Vim it should be sufficient to
point to the license. You can link to the file in github, that should be
permanent enough.
> As for the author of the original Modula-2 support file, do you
> suggest we ought to ask him for permission to replace his work?
last change is long ago I don't expect any objection.
> I have not contacted him simply because what I set out to do at the
> time went beyond the scope of the pre-existing Modula-2 support. It
> would have been akin to ask somebody else to do my homework. I would
> have needed to painstakingly investigate and document all the existing
> issues and then wait for him to fix them, and only after all those
> issues had been resolved would I have been able to extend the scope.
> The effort for both him and myself would have far exceeded the effort
> to simply start from scratch.
more efficient than asking the existing maintainer to make the changes,
and a maintainer would usually just hand over maintenance.
> Also, this is not the first time I have contributed Modula-2 support
> to editors or syntax rendering software. In the early days, I merely
> fixed bugs in and then extended pre-existing support. At first, I
> extended the set of to be colourised reserved words and functions to
> the union of all dialects. This was then supposed to ensure that no
> matter for what dialect the source code to be rendered was written
> for, it would always colourise all the entities. But then there were
> people, who seemed to only know about Wirth's original work (the PIM
> dialect) and they then removed all the support for the ISO dialect
> that I had added.
>
> I had also been participating in open source projects where similar
> situations led to a patch submission war, where one group of
> contributors had engaged in discussions and decided upon a design
> which was then documented and implemented, only to have some other
> folks come along and wipe it out to replace it with their own
> competing implementation and then there was a constant back and forth
> until one side would give up.
> When it comes to supporting syntax colouring or rendering there is
> only one way to prevent any such situation from ever arising and that
> is multi-dialect support with a means for every user to set their
> preferred dialect.
>
> It was for this very reason that I then contacted other Modula-2
> compiler maintainers/implementors to discuss and agree a convention
> how to encode the dialect directly into the source files in form of a
> special comment that contains a dialect tag.
>
> Whenever I implemented Modula-2 support in this way, there has never
> been anyone undoing support for a dialect, everybody can simply use
> their preferred dialect without having syntax entities colourised that
> do not belong to that dialect. It has also led to a certain
> standardisation of Modula-2 syntax rendering between different tools.
>
> Gaius Mulley, the implementor of GNU Modula-2 has implemented the same
> solution for Emacs for example.
>
> Before this background, I trust you will agree that it would not have
> been appropriate to ask somebody else to do this for me but to do it
> myself and then offer the patches as a replacement.
>
> I hope this makes sense
to include support for whatever dialects people are using, with some way
to select the right one. How? Depends on what's more convenient.
Could be a different filetype name, but that only works well if the
filetype can be recognized easily. Otherwise the user has to indicate
what is desired.
--
GALAHAD turns back. We see from his POV the lovely ZOOT standing by him
smiling enchantingly and a number of equally delectable GIRLIES draped
around in the seductively poulticed room. They look at him smilingly and
wave.
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
Benjamin Kowarsch
Please add a short note in every file, in the header.
Since using the files requires getting Vim it should be sufficient to
point to the license. You can link to the file in github, that should be
permanent enough.
OK, done.
see trijezdci/M2-Syntax-Colour@0057dea
It's common courtesy to let the original maintainer know. Since the last change is long ago I don't expect any objection.
As mentioned further above, I managed to get hold of him and he did not have any objections.
Vim has always had the intention to support many users. Thus I prefer to include support for whatever dialects people are using, with some way to select the right one. How? Depends on what's more convenient. Could be a different filetype name, but that only works well if the filetype can be recognized easily. Otherwise the user has to indicate what is desired.
If somebody has Modula-2 code from the DOS era, the files are likely to have the *.def and *.mod file types that Modula-2 uses in all-uppercase *.DEF and *.MOD. Such code is almost certainly going to be in the PIM dialect. For this reason, the code I submitted sets the dialect to PIM if it finds all-uppercase *.DEF and *.MOD file types.
However, this alone would not cover all scenarios, it is merely a convenience. Hence the dialect tags within a comment within the source and the dialect variable that can be set within vimrc (in addition to the syntax menu).
regards
benjamin
Bram Moolenaar
If the dialect can be fairly well recognized by the file name, I propose to make a separate filetype for that dialect. It's easier than using another method, such as setting a global variable.
The files for that filetype can then source the common code, possibly by setting a buffer-local variable first to select the options.
You mention that the PIM dialect is old, perhaps we an call that Modula2pim ? Not sure if Modula users would recognize that name.
Can you make a pull request? I prefer that above me copying the files.
Doug Kearns
Hi Benjamin,
It would be great to have your files included in the Vim runtime. I was about to update the Modula-2 syntax file but fortunately remembered this outstanding issue.
If you're currently busy I'd be happy to prepare this as a PR for Bram in the next couple of days.
Thanks for your excellent work,
Doug
Benjamin Kowarsch
Thank you for your email.
I wasn't aware that there was an outstanding issue.
I have been using the modified files on my own copy of Vim for a very long
time and it all works just fine.
Can you tell me what appears to be holding up inclusion in the official Vim
release?
kind regards
benjamin
On Fri, 9 Apr 2021 at 20:16, Doug Kearns ***@***.***> wrote:
> Hi Benjamin,
>
> It would be great to have your files included in the Vim runtime. I was
> about to update the Modula-2 syntax file but fortunately remembered this
> outstanding issue.
>
> If you're currently busy I'd be happy to prepare this as a PR for Bram in
> the next couple of days.
>
> Thanks for your excellent work,
> Doug
>
> —
> Reply to this email directly, view it on GitHub
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AFHX3IWFK6YYXVTVXSXAWHDTH3OXXANCNFSM4QLR5KUQ>
Doug Kearns
Hi Benjamin,
Sorry, by "issue" I was just referring to this still open GitHub issue rather than any problem with the actual changes. It all works well for me too.
In his last message, from Sep 3, Bram requested a proper Pull Request which is a bit easier for him to handle rather than copying and patching files manually.
So basically someone just needs to apply the changes detailed in your README.md to a forked Vim repo so a PR can be generated.
Aside from swapping in the new syntax files for the old one, it appears to me that
- the contents of your project's
filetype-def.vimandfiletype-mod.vimfiles need to be appropriately distributed betweenruntime/filetype.vimandruntime/autoload/dist/ft.vim - the patches from
syntax-menu.vimneed to be applied to the menu files - some tests should be added to
src/testdir/test_filetype.vim(probably not a blocker)
The only other thing that caught my eye was the filetype naming scheme. If you're not attached to the shortened m2 prefix it might be better to keep using modula2? E.g., modula2pim, which is consistent with the other modula3 filetype.
Thanks,
Doug
Benjamin Kowarsch
I remember now.
Bram wanted to have it all presented as a set of diff files that he could
apply and have all the changes automated.
I did not feel comfortable with that because this may then fail to produce
a working end result if there are differences between my Vim and whatever
Bram is using to apply the patches.
I was thinking about what I should do and how I should respond and then got
busy with some stuff and then forgot to return to the request. Sorry about
that.
As for the naming of files, the convention in the Modula-2 universe has
always been to shorten Modula-2 to M2 or m2. That is what Modula-2 people
are used to. We have also used this convention when we agreed on a
nomenclature for dialect tags within special dialect tag comments for
automatic detection of the dialect a source file was intended for. We
agreed this between several authors/maintainers of M2 compilers and the
basic dialect tags are m2pim, m2iso and m2r10.
I think it would be better to be consistent with the conventions that exist
within the Modula-2 space even if that is not consistent with what Vim does
for Modula-3. Despite the similarity in name, Modula-2 and Modula-3 are
very different languages and I have only ever come across one person who
had used both Modula-2 and Modula-3, so even the user communities as far as
they still exist do not overlap. Tthere is far more overlap between Pascal,
Modula-2 and Oberon. Modula-3 shouldn't really be considered a close
relative and its conventions should have no bearing on conventions to be
used for Modula-2. Anyway, that's how I see it. But at the end of the day,
if Bram wants to change the names of things, Vim is his baby, so it is not
for me to make the call. He will have the last word of course.
I notice that he didn't like the idea of setting an environment variable to
set the default dialect, but that variable is set in the user's vimrc file,
and the default in that file is classical Modula-2, aka PIM, which no
Modula-2 user will object to, even if they aren't using it. They will not
have a problem to edit their vimrc file and change the default. Once they
have done that, they will not likely ever change it again. If they work
with multiple dialects, they will change the dialect in the menu. And this
is only needed if the source files do not contain a dialect tag anyway. If
somebody works on a project that supports multiple dialects, like our
compiler and preprocessor, then they will put those dialect tags in their
source code. So the changing of the default by variable is a fallback to a
fallback. I don't think Bram really understood that and I think he wouldn't
have otherwise taken an issue with it.
But anyway. How do we go from here? I'd really like to see this finally go
into the official release.
regards
benjamin
On Sat, 10 Apr 2021 at 00:06, Doug Kearns ***@***.***> wrote:
> Hi Benjamin,
>
> Sorry, by "issue" I was just referring to this still open GitHub issue
> rather than any problem with the actual changes. It all works well for me
> too.
>
> In his last message, from Sep 3, Bram requested a proper Pull Request
> which is a bit easier for him to handle rather than copying and patching
> files manually.
>
> So basically someone just needs to apply the changes detailed in your
> README.md to a forked Vim repo so a PR can be generated.
>
> Aside from swapping in the new syntax files for the old one, it appears to
> me that
>
> files need to be appropriately distributed between runtime/filetype.vim
> and runtime/autoload/dist/ft.vim
> - some tests should be added to src/testdir/test_filetype.vim
> (probably not a blocker)
>
> The only other thing that caught my eye was the filetype naming scheme. If
> you're not attached to the shortened m2 prefix it might be better to keep
> using modula2? E.g., modula2pim, which is consistent with the other
> modula3 filetype.
>
> Thanks,
> Doug
>
> —
> Reply to this email directly, view it on GitHub
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AFHX3IRVKD2GPNLSAHEGHRLTH4JZHANCNFSM4QLR5KUQ>
Bram Moolenaar
Benjamin Kowarsch wrote:
> I remember now.
>
> Bram wanted to have it all presented as a set of diff files that he could
> apply and have all the changes automated.
>
> I did not feel comfortable with that because this may then fail to produce
> a working end result if there are differences between my Vim and whatever
> Bram is using to apply the patches.
>
> I was thinking about what I should do and how I should respond and then got
> busy with some stuff and then forgot to return to the request. Sorry about
> that.
>
> As for the naming of files, the convention in the Modula-2 universe has
> always been to shorten Modula-2 to M2 or m2. That is what Modula-2 people
> are used to. We have also used this convention when we agreed on a
> nomenclature for dialect tags within special dialect tag comments for
> automatic detection of the dialect a source file was intended for. We
> agreed this between several authors/maintainers of M2 compilers and the
> basic dialect tags are m2pim, m2iso and m2r10.
has existed for a long time. I don't feel good about renaming it to
something obscure like "m2". You have to see this in the context of Vim
users, not just modula 2 users.
> I think it would be better to be consistent with the conventions that exist
> within the Modula-2 space even if that is not consistent with what Vim does
> for Modula-3. Despite the similarity in name, Modula-2 and Modula-3 are
> very different languages and I have only ever come across one person who
> had used both Modula-2 and Modula-3, so even the user communities as far as
> they still exist do not overlap. Tthere is far more overlap between Pascal,
> Modula-2 and Oberon. Modula-3 shouldn't really be considered a close
> relative and its conventions should have no bearing on conventions to be
> used for Modula-2. Anyway, that's how I see it. But at the end of the day,
> if Bram wants to change the names of things, Vim is his baby, so it is not
> for me to make the call. He will have the last word of course.
Also wikipedia uses these names.
> I notice that he didn't like the idea of setting an environment variable to
> set the default dialect, but that variable is set in the user's vimrc file,
> and the default in that file is classical Modula-2, aka PIM, which no
> Modula-2 user will object to, even if they aren't using it. They will not
> have a problem to edit their vimrc file and change the default. Once they
> have done that, they will not likely ever change it again. If they work
> with multiple dialects, they will change the dialect in the menu. And this
> is only needed if the source files do not contain a dialect tag anyway. If
> somebody works on a project that supports multiple dialects, like our
> compiler and preprocessor, then they will put those dialect tags in their
> source code. So the changing of the default by variable is a fallback to a
> fallback. I don't think Bram really understood that and I think he wouldn't
> have otherwise taken an issue with it.
desired language, no matter whether it is a dialect or not. We have had
a long history if the "sh" filetype used for various similar shell
dialects, it's far from ideal.
> But anyway. How do we go from here? I'd really like to see this finally go
> into the official release.
new files or a pull request both work. It would be good if a few people
can try it out and comment before I include it.
--
From "know your smileys":
O:-) Saint \\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
—
Christian Brabandt
Closed #6796 as completed via 68a8947.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.