Re: [scite] Write a lexer: Confused about interface

[Andreas Tscharner]

On 03.05.2011 17:58, Philippe Lhoste wrote:
> On 03/05/2011 15:18, Andreas Tscharner wrote:

[snip]
> This is more a question for the Scintilla-interest list, but we will
> answer anyway.

Sorry, I didn't know there was another group.
> The static function is the old interface, still used in lot of lexers.
> It works fine and is quite simple for simple languages.
> The interface (lexer objects) is recent, with a demo implementation in
> LexCPP, and some lexers migrating to it. It is more powerful, allowing
> for example to handle C's pre-processor nesting and such.
>
Is there a tutorial or any other examples except LexCPP?

TIA and best regards
Andreas
--
Andreas Tscharner <sterne...@gmail.com>
----------------------------------------------------------------------
"Intruder on level one. All Aliens please proceed to level one."
-- Call in "Alien: Resurrection"

[Randy Kramer]

Andreas,

Note: I wasn't sure you were subscribed to the scintilla-interest group
so I included you as an addressee.

Depending on what you're trying to make a lexer for, I might be able to
help--I'm working on a lexer for a wiki markup language, and I've
learned quite a bit, but haven't finished my lexer.

OTOH, my advice may have scare off one or more people. ;-)

To be somewhat specific, there are a lot of lexers, and finding one or
more lexers that deal with a language something like the language
you're targeting will be more useful than one that is not like the
language you're targeting.

Randy Kramer

[Randy Kramer]

Andreas,

Thanks for the response! (I decided to keep a reply directly to you
even though you are now subscribed to the scintilla list--presumably
next time I won't. ;-)

On Thursday 05 May 2011 07:22:42 am Andreas Tscharner wrote:
> My lexer highlights strings, comments and numbers; that's good enough
> for the prototype. In case we choose scintilla for our product, I
> will gladly ask you...

I look forward to it. I would have to understand more about
highlighting strings--I presume you don't want to highlight all strings
and we'd have to understand how the lexer can distinguish which strings
to highlight and which not. (And maybe there is more than one type of
string, requiring more than one style of highlighting?)

Randy Kramer

[Andreas Tscharner]

On 04.05.2011 14:25, Randy Kramer wrote:
> Andreas,
>
> Note: I wasn't sure you were subscribed to the scintilla-interest group
> so I included you as an addressee.

I wasn't at first, but since yesterday afternoon I am...

>
> Depending on what you're trying to make a lexer for, I might be able to
> help--I'm working on a lexer for a wiki markup language, and I've
> learned quite a bit, but haven't finished my lexer.

My lexer highlights strings, comments and numbers; that's good enough

for the prototype. In case we choose scintilla for our product, I will
gladly ask you...
>

> OTOH, my advice may have scare off one or more people. ;-)
>
> To be somewhat specific, there are a lot of lexers, and finding one or
> more lexers that deal with a language something like the language
> you're targeting will be more useful than one that is not like the
> language you're targeting.

The final product has to lex DMIS, a language for CMMs
(http://en.wikipedia.org/wiki/Coordinate-measuring_machine). I haven't
found another language that has only a slight resemblance.
>

Thanks and best regards

[KHMan]

On 5/5/2011 7:22 PM, Andreas Tscharner wrote:
> On 04.05.2011 14:25, Randy Kramer wrote:

>>[snip snip snip]

>> To be somewhat specific, there are a lot of lexers, and finding
>> one or
>> more lexers that deal with a language something like the language
>> you're targeting will be more useful than one that is not like the
>> language you're targeting.
>
> The final product has to lex DMIS, a language for CMMs
> (http://en.wikipedia.org/wiki/Coordinate-measuring_machine). I
> haven't found another language that has only a slight resemblance.

I found the NISTIR 6012 PDF (The NIST DMIS Interpreter) on the
'net. Recognizing words, strings, numbers and operators is a good
baseline.

It looks line-based, so the lexer can rewind to a start-of-line
for easy processing. If words are unique, keeping some state will
enable highlighting of words based on their position in a statement.

A lexer can probably be cobbled together with the help of bits and
pieces from different lexers...

--
Cheers,
Kein-Hong Man (esq.)
Kuala Lumpur, Malaysia