interpreter/spad/aldor grammar for atom

[Bill Page]

I have been playing with defining a language package for atom to
provide syntax highlighting and other edit functions for spad. I have
a few simple things working. Is anyone else interested in using atom
this way? Does anyone have more experience with doing this sort of
thing in atom? I think it could eventually provide a good modern
alternative to emacs mode.

[Martin Baker]

I am curious, what are the pros and cons of using a hackable editor to
do this rather than an IDE as Peter Broadbery is doing:

https://github.com/pbroadbery/aldor-eclipse
https://github.com/pbroadbery/aldor-idea-plugin

I am guessing that an editor is easier to start with but an IDE would
ultimately be more powerful, is this guess correct?

Martin B

[Peter Broadbery]

I would be interested in seeing what atom can do (not really heard of it before now tbh).. From my experience with the intellij lexer/grammar, the easy part (relatively) is lexing and parsing.. the trickier part is recognising indented blocks.  Would be interested in seeing how your grammar works.

The intellij-idea plugin linked above is somewhat functional with some syntax highlighting (it's easy to do in intellij, but one shouldn't overdo it).  The eclipse one has foundered; I think I spent too much time trying to get building to work automatically.. the intellij version gives up on that and relies on makefiles doing the right thing.

The IDE approach can be very powerful when compared to a text editor; being able to see docs for symbols without having to switch contexts is great, as well as searching for functions by name.  Internally, the IDE has a full view of the abstract syntax for any file; this can be used for all sorts of things - basic semantic checks, some refactorings, documentation, basic type extraction.  Downside is that it can feel slower.  Some of that is inevitable due to having to reparse files more completely, of course.

As an aside, writing intellij extensions feels quite rewarding; one notices a feature in some other extension, and twisting it into a form for spad or aldor often turns out easier than expected.

To switch subjects slightly, I would like it to use some of the work that has gone into the  fricas.github.io docs, as the plugin shows raw text in docs at the moment.  There is a fair amount to investigate here, if anyone is interested.

Peter

--
You received this message because you are subscribed to the Google Groups "aldor-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aldor-devel+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ralf Hemmecke]

On 03/30/2017 10:59 PM, Peter Broadbery wrote:
> Sorry - somewhat unclear.. The plugin is fairly good for spad files and shows ++
> comments for domains, without any pre-processing.. In the java world the
> corresponding /** comments are turned into html with a bit more markup. This
> could potentially happen for spad, would "just" be a question of being able to
> generate the appropriate markup and link to it. As I understand it, the api
> docs on the .io site contain the right html.

Well, let me (from my memory) tell you how
http://fricas.github.io/api/BasicType works.

I get many of the information not from the .spad file, but from the
compiled FriCAS database of the constructors. In particular the ++
comments are taken as they are. The interlinking is pretty easy since I
know the URL of constructor and anchors of all the functions.

However, I actually let sphinx do the hard work. I only have to make
sure that I feed sphinx with rst.

Look into
https://github.com/hemmecke/fricas/blob/master-hemmecke/src/doc/api.spad
and there go from bottom to top. You'll find
ConstructorDocumentationSphinxTools and ConstructorDocumentationTeXTools
where I massage the ++ docstrings.
That's by far not perfect, but better than nothing.

The other code in api.spad first collects all available data from the
FriCAS DB for all constructors, then tries to simplify conditional
exports and finally formats everything into a .rst file.

Because you are thinking about a format for ++ docstrings... I also
don't like aldoc so much. In fact, the problem with TeX-like
documentation in source code files is that one cannot easily read them
nicely. ReStructuredText is much more pleasant to read also in
uncompiled raw form. So if (at least for Aldor) we could agree on saying
the the docstring format should be RST, that would simplify things quite
some bit. Yes, it would mean rewriting existing documentation, but if
then the docstrings would be more useful in an IDE, I would be quite
happy to help in this transformation. Since I've never worked with
Intellij, I don't know whether it can deal with RST documentation.

Ralf

[Bill Page]

On 30 March 2017 at 14:25, Peter Broadbery <p.bro...@gmail.com> wrote:
>
> I would be interested in seeing what atom can do (not really heard of
> it before now tbh).. From my experience with the intellij lexer/grammar,
> the easy part (relatively) is lexing and parsing.. the trickier part is
> recognising indented blocks. Would be interested in seeing how your
> grammar works.
>

"grammar" is the terminology used by atom but it is actually somewhat
of a misnomer. So it might be better to start at the end of the story
rather than the beginning. Atom is a text editor built on top of the
Electron platform which is essentially just a browser designed to be
used locally on the desktop instead of via the Internet.
https://electron.atom.io/ But Electron uses the same core technology
as Google's Chrome web browser - something called Chromium. Electron
is intended to be used for developing desktop applications that run
the same on Windows, Mac, and Linux so in some sense it is like the
"modern" reincarnation of gui tools like Qt with C++ replaced with
javascript, html, css, node.js and a lot of the other "modern" stuff
(re-)invented for the web. Electron and atom are part of the evolving
Google/GitHub ecosystem which includes IDE's like Nuclide
https://nuclide.io/ and a lot of other stuff:
https://electron.atom.io/apps/ .So the end of the story, or at least
the story so far is that after spending quite a lot of time working on
extending the OpenAxiom Qt-based front-end (which also in principle
works with FriCAS and Axiom), I now think that starting with Atom is a
much better option. But since I am not really well connected with the
Google "ecosystem" I only got here through the backdoor by trying to
learn more about how to highlight Spad syntax in Atom.

> The intellij-idea plugin linked above is somewhat functional with some
> syntax highlighting (it's easy to do in intellij, but one shouldn't overdo
> it). The eclipse one has foundered; I think I spent too much time trying
> to get building to work automatically.. the intellij version gives up on
> that and relies on makefiles doing the right thing.
>

The specifications for the Spad "grammar" in Atom amounts to a
relatively large set of regular expressions that define different
syntax elements.
https://www.sitepoint.com/how-to-write-a-syntax-highlighting-package-for-atom/
Atom packages in general however can be a lot more complex and a lot
more capable than this.

What I have so far is a package for Atom that installs, recognizes
Spad source code and highlights a few things. I will be glad to share
it once I get a bit more time to make it do something a bit more
interesting. But I was kind of disappointed that in Atom defining this
sort of package is not the route to being able to display and navigate
symbols in source nicely. https://github.com/atom/symbols-view That
actually requires an external tool called ctags
http://ctags.sourceforge.net/

> The IDE approach can be very powerful when compared to a text editor;
> being able to see docs for symbols without having to switch contexts
> is great, as well as searching for functions by name. Internally, the IDE
> has a full view of the abstract syntax for any file; this can be used for
> all sorts of things - basic semantic checks, some refactorings,
> documentation, basic type extraction. Downside is that it can feel
> slower. Some of that is inevitable due to having to reparse files more
> completely, of course.
>

Doing this sort of thing is perhaps the normal scope of an Atom
package but certainly not beyond what Electron is being used for. For
example I think Electron could make a great foundation for a
replacement for Hyperdoc.

> As an aside, writing intellij extensions feels quite rewarding; one notices
> a feature in some other extension, and twisting it into a form for spad or
> aldor often turns out easier than expected.
>

Yes, that was more or less what I was hoping to find in Atom. I think
I more or less found it. But of course it is still not that easy.

Bill Page.

[Bill Page]

On 30 March 2017 at 21:52, Bill Page <bill...@newsynthesis.org> wrote:
>
> What I have so far is a package for Atom that installs, recognizes
> Spad source code and highlights a few things. I will be glad to share
> it once I get a bit more time to make it do something a bit more
> interesting. But I was kind of disappointed that in Atom defining this
> sort of package is not the route to being able to display and navigate
> symbols in source nicely. https://github.com/atom/symbols-view That
> actually requires an external tool called ctags
> http://ctags.sourceforge.net/
>

Here is a very priliminary first version of my Spad syntax
highlighting package for Atom:

https://atom.io/packages/language-spad

Also the following preliminary .ctags configuration seems to work
pretty well to add symbols from in the FriCAS source tree.

--langdef=spad
--langmap=spad:.spad
--regex-spad=/^\)abbrev domain ([a-zA-Z_\?]+) ([a-zA-Z_\?]+)/\2/d,domain/
--regex-spad=/^\)abbrev package ([a-zA-Z_\?]+) ([a-zA-Z_\?]+)/\2/p,packages/
--regex-spad=/^\)abbrev category ([a-zA-Z_\?]+) ([a-zA-Z_\?]+)/\2/c,category/
--regex-spad=/\W([a-zA-Z_\?]+)[ ]*\:[^\:=].*\:=/\1/v,variable/
--regex-spad=/\W([a-zA-Z_\?]+)( *\(.*\) *)?==/\1/f,function/

I have successfully used "Go to Declaration" and the Atom
symbols-tree-view package for navigation.

In principle this configuration should also work with emacs
https://www.emacswiki.org/emacs/BuildTags . I tried it but I am not
sufficiently fluent in emacs to say whether it is really useful. If
anyone else has a better .ctags specification for FriCAS I would be
glad to here about it.

Let me know if you have any questions/suggestions.

Bill Page.