HAML and SCSS syntax highlighting

199 views
Skip to first unread message

Edoardo Tenani

unread,
Mar 21, 2011, 4:36:36 AM3/21/11
to scintilla-interest
Hello, is possible to add syntax highlighting for HAML and SCSS
languages?

I asked in Geany mailing list ( is the text editor that I'm currently
using, and uses Scintilla ) and they told me to ask here because for
add a new syntax to syntax highlighting it has to be added to
Scintilla.

Is already done or is in progress?
I don't understand so well Scintilla and how to achive this, but I
think I'm able to help ( if is in progress ) or to work on it, with a
little help!

Thanks,
Edoardo Tenani

Philippe Lhoste

unread,
Mar 21, 2011, 6:42:04 AM3/21/11
to scintilla...@googlegroups.com
On 21/03/2011 09:36, Edoardo Tenani wrote:
> Hello, is possible to add syntax highlighting for HAML and SCSS
> languages?
>
[...]

>
> Is already done or is in progress?
> I don't understand so well Scintilla and how to achive this, but I
> think I'm able to help ( if is in progress ) or to work on it, with a
> little help!

It is always possible... You just have to find somebody knowing Scintilla lexers,
interested/motivated by this goal... Or if you know C++ well enough (required level is
pretty low), you can do it yourself, by picking an existing lexer close of what you need,
and by hacking it until it fits your goal.
Of course, if you try yourself and have issues, just ask for help here, we will try and
assist you.

--
Philippe Lhoste
-- (near) Paris -- France
-- http://Phi.Lho.free.fr
-- -- -- -- -- -- -- -- -- -- -- -- -- --

Edoardo Tenani

unread,
Mar 21, 2011, 7:36:11 AM3/21/11
to scintilla...@googlegroups.com
It is always possible... You just have to find somebody knowing Scintilla lexers, interested/motivated by this goal... Or if you know C++ well enough (required level is pretty low), you can do it yourself, by picking an existing lexer close of what you need, and by hacking it until it fits your goal.
Of course, if you try yourself and have issues, just ask for help here, we will try and assist you.

Thank for your reply, Philippe.

I guess that I can manage to do this myself, but I have some questions:
  • first: what is a lexer? Is simply a text parser? The concept is not so clear to me.
  • then: where I can find some documentation about lexers and how to create them? I've searched around, I've looked at source code, but I did not find anything useful...
Thanks
Edoardo Tenani

KHMan

unread,
Mar 21, 2011, 7:54:13 AM3/21/11
to scintilla...@googlegroups.com
On 3/21/2011 7:36 PM, Edoardo Tenani wrote:
> It is always possible... You just have to find somebody
> knowing Scintilla lexers, interested/motivated by this goal...
> Or if you know C++ well enough (required level is pretty low),
> you can do it yourself, by picking an existing lexer close of
> what you need, and by hacking it until it fits your goal.
> Of course, if you try yourself and have issues, just ask for
> help here, we will try and assist you.
>
> Thank for your reply, Philippe.
>
> I guess that I can manage to do this myself, but I have some
> questions:
> * first: what is a lexer? Is simply a text parser? The concept

> is not so clear to me.

Lexical analyzer... yep, a text parser. For source code
highlighting, often classifying tokens is enough, and so avoids
actual syntax comprehension as far as possible.

> * then: where I can find some documentation about lexers and


> how to create them? I've searched around, I've looked at
> source code, but I did not find anything useful...

I took a peek at the HAML site, it seems to be a minimal syntax of
sorts like YAML -- there is a LexYAML.cxx in the Scintilla code
base that might be a good starting point.

--
Cheers,
Kein-Hong Man (esq.)
Kuala Lumpur, Malaysia

Philippe Lhoste

unread,
Mar 21, 2011, 8:12:02 AM3/21/11
to scintilla...@googlegroups.com
On 21/03/2011 12:36, Edoardo Tenani wrote:
> * first: what is a lexer? Is simply a text parser? The concept is not so clear to me.

Yes. It is a low level text parser, acting at char level, working like a state machine.
Most lexers are relatively simple (compared to the lexer of a compiler) as an identifier
is just an identifier (at best it is seen as a keyword), and it doesn't care (in general)
if it is a variable, a function, a call or a declaration, etc.

You can still do relatively sophisticated things, like handling nested comments, managing
here-docs, checking if a string doesn't terminate before line end (for languages not
allowing multiline strings) and so on.

> * then: where I can find some documentation about lexers and how to create them? I've


> searched around, I've looked at source code, but I did not find anything useful...

In the source distribution, curiously the lexer doc is in the scite/doc folder (probably
because it also need integration at the SciTE level).
Online, you can find it at: http://www.scintilla.org/SciTELexer.html
There are some new concepts (but you can stick with simpler, old style lexers) explained
at http://www.scintilla.org/nulex.html
And, of course, source code is your friend... (but not so much for information spread over
several files, which is explained in the first link).

Randy Kramer

unread,
Mar 21, 2011, 8:13:10 AM3/21/11
to scintilla...@googlegroups.com, Edoardo Tenani
Edoardo,

(Aside: I wasn't 100% sure you were subscribed to this list, so I sent
you a cc--once I'm sure you're subscribed, I won't send ccs to you,
just to the list.)

I'm a novice to writing lexers (and folders), so I'm not writing to
answer most of your questions. Their is documentation around, and I'm
sure someone will point you to it (maybe I will later today, after I
answer some other emails).

The reason I'm writing to you is to say that I'm writing my first
lexer /folder (and learning C/C++). The lexer / folder will be an
internal lexer (there are other possibilities, writing a lexer in the
container (the editor), writing an external lexer (and either of those
could be written in the scripting language Lua, or some other
language). There is also the Scintulua (sp?) project which is
specifically aimed at a version of scintilla with lexers written in
Lua.

(Hmm, thinking about it now, I may have been the source of some
misinformation on the Geany list--I wrote there, probably ~18 months
ago that lexers had to be written in / for scintilla, but you could
actually write a lexer in Geany (i.e., the "container"), iiuc. The
thing is, if you write it in / for scintilla, it can be easily applied
to any editor in the scintilla family (that is, editors that use
scintilla as the editing widget).)

So, I encourage you to write it for scintilla.

Like I say, I am writing mine as an internal lexer / folder for
scintilla. Older internal lexers were not written as objects, newer
ones are. I'd encourage you to write one as an object.

Finally, to my point. In my progress so far, what I've almost
accomplished (and could share with you) is an almost empty lexer in
object form. My approach to lexing seems so different than the other
examples I found that I was just totally confused by the examples, so
I've taken the YAML lexer in object form (LexYAML.cxx) and converted it
to a lexer in object form for RML (LexRML.cxx) by commenting out most
of the functionality.

(I plan to remove the commented out code soon, and it would be less
confusing to you (probably) if I did that, but I'm proceeding slowly to
understand what I'm stripping out before I do so. Maybe in a few days
(or a week) I'd have it actually stripped out.

The stripped out lexer is integrated into scintilla in the sense that
files with the appropriate extension invoke the RML lexer and it runs
and does some things, but the things it does now are print various
debug statements to help me understand what is going on with respect to
the lexer being called.

In a reply to another email which I hope to write today, you will see
more about what I've done.

Oh, my work has been done against scintilla 2.24, I also want to make a
patch from that and apply it to scintilla 2.25. (That would be my
first creation and application of a patch--I could offer that patch to
you (as opposed to a big zip or tarball of the entire source, but as
I'm keeping my local work under git source control, and I'd expect to
create the patch using git, I'm not sure that would be helpful to you
unless you're using git).

Randy Kramer

PS: A few more comments interspersed below.

On Monday 21 March 2011 07:36:11 am Edoardo Tenani wrote:
> > It is always possible... You just have to find somebody knowing
> > Scintilla lexers, interested/motivated by this goal... Or if you
> > know C++ well enough (required level is pretty low), you can do it
> > yourself, by picking an existing lexer close of what you need, and
> > by hacking it until it fits your goal.
> > Of course, if you try yourself and have issues, just ask for help
> > here, we will try and assist you.
>
> Thank for your reply, Philippe.
>
> I guess that I can manage to do this myself, but I have some
> questions:
>

> - first: what is a lexer? Is simply a text parser? The concept is


> not so clear to me.

I think it would be correct to say that it is a text parser. It looks
at the syntax of a document and decides what colors / styles should be
applied to each element to highlight the syntax.

> - then: where I can find some documentation about lexers and how


> to create them? I've searched around, I've looked at source code, but
> I did not find anything useful...

If no one else replies, I'll try to reply to this later today.


Lex Trotman

unread,
Mar 21, 2011, 8:26:03 AM3/21/11
to scintilla...@googlegroups.com, Randy Kramer, Edoardo Tenani
> (Hmm, thinking about it now, I may have been the source of some
> misinformation on the Geany list--I wrote there, probably ~18 months
> ago that lexers had to be written in / for scintilla, but you could
> actually write a lexer in Geany (i.e., the "container"), iiuc.  The

However Geany doesn't support either lexers in the container (ie
itself) or ATM external lexers, though I think there is some
consideration of it to get Scintilau..

Cheers
Lex

Randy Kramer

unread,
Mar 21, 2011, 8:36:45 AM3/21/11
to scintilla...@googlegroups.com, Lex Trotman, Edoardo Tenani
On Monday 21 March 2011 08:26:03 am Lex Trotman wrote:
> However Geany doesn't support either lexers in the container (ie
> itself) or ATM external lexers, though I think there is some
> consideration of it to get Scintilau..

Lex,

Thanks very much for clarifying that!

So it turns out I wasn't wrong, I just thought I was wrong. ;-)

(That's sort of an allusion to an old (American only?) joke about making
mistakes.)

Randy Kramer


Edoardo Tenani

unread,
Mar 21, 2011, 12:40:22 PM3/21/11
to scintilla...@googlegroups.com
Wow! So many replies! :)

Well, I'll try to reply in order ( trying to not forget nothing ):
  • I'm subscribed :)
  • I don't know lua, while I've used C++ to write a code for a videogame some years ago
  • my first thought was to develop something for geany, cause is the editor I use. But now, seeing that is Scintilla the editing component, I changed my mind and I'll write for Scintilla, so that my code will be available for everyone
  • thanks Randy for your offert, would be cool to check out your code!
  • I use git in other projects; if the code is hosted somewhere ( maybe on github?? ), I can clone the repo or I can apply patch easily
  • I'm also used to programm OOP style ( not so used to do it in C++, but I'm using Java and PHP, so I think I'll be ok )
  • thank Philippe for links to documentation, they will be usefuls

Randy, I'm looking forward to see your next mail

Thanks all guys!
Edoardo Tenani

Randy Kramer

unread,
Mar 21, 2011, 3:57:10 PM3/21/11
to scintilla...@googlegroups.com
On Monday 21 March 2011 12:40:22 pm Edoardo Tenani wrote:
> - I'm subscribed :)

Good!

> - I don't know lua, while I've used C++ to write a code for a
> videogame some years ago

Good--I might look to you for help, this is my first effort at writing C
or C++ for any useful purpose.

> - my first thought was to develop something for geany, cause is


> the editor I use. But now, seeing that is Scintilla the editing
> component, I changed my mind and I'll write for Scintilla, so that my
> code will be available for everyone

I like that approach!

> - thanks Randy for your offert, would be cool to check out your
> code! - I use git in other projects; if the code is hosted somewhere


> ( maybe on github?? ), I can clone the repo or I can apply patch
> easily

Ok, I think you understand that this is my private git repository, only
being used for me to manage my efforts (so if I go off the deep end I
can back out to someplace where I knew what I was doing).

And, I don't have an IP, so I will either have to make you a zip or
tarball of the entire code repository (which is my code plus what I
downloaded from the site as version 2.24 of SciTE / scintilla), or make
a (git) patch.

The other problem with a tarball or zip is that it will probably be
bigger than I can put in an email--hmm, or maybe not, I think it should
be 3 MB or less, and that should be OK (I think).

I sort of want to do the git patch approach, but that may be a few days
to do it. The reason I want to do the patch approach is that I plan to
use it anyway--I will shortly download version 2.25 of SciTE /
scintilla, then take a diff of my 2.24 changes and apply that as a
patch to the 2.25 version, and, hopefully, I'll be up to date.

> - I'm also used to programm OOP style ( not so used to do it

> in C++, but I'm using Java and PHP, so I think I'll be ok )

> - thank Philippe for links to documentation, they will be usefuls


>
>
> Randy, I'm looking forward to see your next mail

So, this is my next mail. Let me know if you want a zip or tarball
soon, or want to wait (longer) for a patch. ;-)

Randy Kramer


Philippe Lhoste

unread,
Mar 22, 2011, 6:29:06 AM3/22/11
to scintilla...@googlegroups.com
On 21/03/2011 17:40, Edoardo Tenani wrote:
> * my first thought was to develop something for geany, cause is the editor I use. But

> now, seeing that is Scintilla the editing component, I changed my mind and I'll
> write for Scintilla, so that my code will be available for everyone

Good decision. Neil confirmed that no language is too obscure to be included in Scintilla.
But he can reject lexers that are badly written, as he did for my LexAHK, which works (lot
of people use it) but has some odd code, because the language is quite irregular and hard
to lex: I am not happy myself with its code, but I just lack time, and now motivation, to
improve it further. (Sorry, a bit off topic.)

Anyway, HAML isn't that obscure, I have seen implementations in other languages (at least
in Scala, in the Scalate library IIRC [1]), so a general lexer is welcome.
I didn't know SCSS but it seems interesting too.

[1] I confirm, funnily I just have a tab opened on the right page:
http://scalate.fusesource.org/documentation/scaml-reference.html

Edoardo Tenani

unread,
Mar 22, 2011, 7:27:17 AM3/22/11
to scintilla...@googlegroups.com
In my ideas I'd like to write the best code I'm able ( obviously ), but I hope to  write something good enough to be inserted in Scintilla!



Anyway, HAML isn't that obscure, I have seen implementations in other languages (at least in Scala, in the Scalate library IIRC [1]), so a general lexer is welcome.

HAML has ruby ( obviously ), PHP, Scala, and I guess some other, implementations, so my idea is to create the lexer in the more general way ( the problem with this is that, for example, after the '-' or the '=' signs you'll have code in different languages, ruby, php, ..., and actually I really don't know how to implement this ).


I didn't know SCSS but it seems interesting too.
SCSS is a superset of CSS3, has the little modifications I needed so much in CSS ( think for example at the "functions", or variables you can use in SCSS ) , and more important lets you use Compass, which is a set of "functions" to perform some operations ( like border radius with all vendor prefixes in one line, and similar cool stuff )


@Randy: I'm more comfortable with zip or tarball actually, but it's your choice, I don't mind if you send me a patch!


Good--I might look to you for help, this is my first effort at writing C or C++ for any useful purpose.
If I can help you ( I'm not a genius in C++ ) let me know!


Regards,
Edoardo

Randy Kramer

unread,
Mar 22, 2011, 7:56:32 AM3/22/11
to scintilla...@googlegroups.com
On Tuesday 22 March 2011 07:27:17 am Edoardo Tenani wrote:
> @Randy: I'm more comfortable with zip or tarball actually, but it's
> your choice, I don't mind if you send me a patch!

Ok, but I have to offer you another choice ;-)

Some background: I created a patch (successful as far as I can tell)
from the 2.24 version of sciscint (my shorthand term for
Scite/scintilla), but it will not apply properly to 2.25--the point
being that I can't yet offer you a tarball of 2.25 with my changes. I
can only offer you 2.24 for now. Maybe soon (but maybe not so soon,
maybe as much as a week from now) I could offer you a 2.25 tarball with
my changes.

Let me know whether you'd like the 2.24 version now. I'll certainly
offer you the 2.25 version when I get it done.

> If I can help you ( I'm not a genius in C++ ) let me know!

Good, thanks!

Two other points I want to raise:

I'd suggest you answer each email separately. (If I'm not mistaken, you
sometimes combine answers from two or more different emails in one
response--it may happen that someone misses your response because of
that. I may be mistaken, I haven't looked carefully.

At least for the time being, I'm going to keep our emails on the list.
Perhaps they would help someone else, or someone else will chip in with
a response that helps (either of) us.

(The one exception will be when I send you the tarball (after your next
response)--I may send some notes to the list, but I will send the
tarball to just you (unless someone else asks for it).

Randy Kramer

Edoardo Tenani

unread,
Mar 22, 2011, 8:07:21 AM3/22/11
to scintilla...@googlegroups.com
Let me know whether you'd like the 2.24 version now. I'll certainly
offer you the 2.25 version when I get it done.

I'm ok with the 2.25 when you're done, I'm not in hurry!!!

 
I'd suggest you answer each email separately.  (If I'm not mistaken, you
sometimes combine answers from two or more different emails in one
response--it may happen that someone misses your response because of
that.  I may be mistaken, I haven't looked carefully.

ok, from now I'll reply separately!

(The one exception will be when I send you the tarball (after your next
response)--I may send some notes to the list, but I will send the
tarball to just you (unless someone else asks for it).

I'm ok with that!

Thanks!
Edoardo

Randy Kramer

unread,
Mar 22, 2011, 9:41:30 AM3/22/11
to scintilla...@googlegroups.com
On Tuesday 22 March 2011 08:07:21 am Edoardo Tenani wrote:
> I'm ok with the 2.25 when you're done, I'm not in hurry!!!

Ok, good. I should clarify though, I didn't really mean done as in done
with the lexer, I just meant done in terms of getting my current work
on 2.24 moved over to 2.25.

Randy Kramer

Edoardo Tenani

unread,
Mar 22, 2011, 12:05:17 PM3/22/11
to scintilla...@googlegroups.com
Ok that's no problem for me :)

thanks
Reply all
Reply to author
Forward
0 new messages