LexAsm Improvements
Jason Oster
improvements, and it was decided to move the thread to the Scintilla ML,
where it is more appropriate.
The original post is from Manuel Mendez:
http://lists.uvena.de/geany/2008-October/003669.html where he explains a
problem with getting his comments highlighted by Scintilla. This brought
up some old ideas I came up with to extend the ASM lexer to support
enough configuration to make it roughly general purpose.
Some concerns brought up so far include:
* Comment characters are hard-coded into the lexer; not all common
comment characters are supported; some ASM dialects will use some
characters for comments, and other dialects will use the same characters
for other purposes (for example, pseudo-instructions and
pseudo-operands); C-style multi-line comments /*...*/ not supported
* Labels are not styled
* Number styling: Hexadecimal syntaxes not supported; negative numbers
not supported; some ASM dialects use $... to specify hex (while others
use $ to specify registers) and some use 0x...; some ASM dialects use
%... to specify binary (while others use % to specify registers)
Some comments in the lexer source suggest only MASM and NASM are
"supported" but I believe it would be within every [ASM programmer's]
interest to generalize the lexer enough that it can be customized to the
user's needs.
The following are some ideas to address the concerns above:
* Comment characters should be specified in either a lexer preference (a
white-space-delimited string, e.g. lexer.asm.chars.comment = "; # @ //")
or within a keyword list. Another [boolean] pref may be used to simply
enable and disable C-style multi-line comments /*...*/
* Labels should be parsed with a static algorithm. For example, the
following regex: /\n\w+[\n:]/ where \w matches a 'word' character,
usually [_a-zA-Z0-9] ~Word characters should also be a customizable
lexer pref. Some characters (. and ?) are currently hard-coded as word
characters, which I personally do not understand and would not use
myself; but someone obviously wanted these.
* Number characters should be a customizable lexer pref. For example:
lexer.asm.chars.number = "0123456789-+." The lexer would essentially
match this pref setting with an equivalent regex: /[0-9\-+.]+/ And to
support hex: "0123456789ABCDEFxh-+." would match /[0-9A-Fxh\-+.]+/
The number matching idea is not perfect; it will match valid numbers:
3.1415926
-100
0xDEADBEEF
But it will also match invalid numbers:
0..100
-+-+-
2+2
Unless rules can be specified in some way to define how these characters
may appear within a number, I would much prefer the lexer getting
confused over invalid syntax (and not so invalid: 2+2) than being unable
to customize how it shows me the numbers I write.
In MOS 6502/65816 (NES, C-64, SNES, Atari, etc) for example, we usually
write hexadecimal numbers as $BEEF ~I want this to show as a number, but
I don't want to hard-code a rule that says "treat /\$[0-9a-fA-F]+/ as a
number."
This is because in MIPS (N64, PSX, PS2, etc) we may use $ to specify
registers: "add $a0, $a1, $t0" is a valid instruction; $a0 and $a1 are
not numbers, even though they would match that rule.
Well again, any ideas or comments would be appreciated!
Jay
Neil Hodgson
> * Comment characters should be specified in either a lexer preference
>
> ~Word characters should also be a customizable
> lexer pref.
>
> * Number characters should be a customizable lexer pref.
Making all of these configurable can lead to conflicts and unclear
behaviour. Allowing multi-character comment starts may be particularly
messy.
Neil