My question is (for Neil or anybody) is: can I write a Lexer in Luddite and use it in, for example, Geany, Scite, or basically any Scintilla based editor?
> My question is (for Neil or anybody) is: can I write a Lexer in Luddite and
> use it in, for example, Geany, Scite, or basically any Scintilla based
> editor?
If, as the active state documentation states, it compiles into a
standard scintilla lexer then it "should" be able to be used in any
Scintilla editor. But only if the compiled lexer is fully self
contained and doesn't need any of Komodo code to run.
> --
> You received this message because you are subscribed to the Google Groups "scintilla-interest" group.
> To post to this group, send email to scintilla-interest@googlegroups.com.
> To unsubscribe from this group, send email to scintilla-interest+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/scintilla-interest?hl=en.
I've never tried it that way, but I don't see why not. The key is that
Luddite doesn't
generate a C++ lexer. It generates a lexres file, a pseudo-binary
file. The full path
to this file is placed in the first entry in the first keyword list
for the language you
want to lex (which is an instance of SCLEX_UDL), and ColouriseTemplate1Doc
will load it the first time it sees it and build essentially a state
machine that
drives the lexing process.
Unlike everything I said about keywords in lexers in the recent Visual Prolog
thread, keywords are defined in a .udl file, are compiled into the .lexres file,
and are managed by LexUDL.cxx, not a containing application.
On Tue, Jun 5, 2012 at 5:07 AM, Lex Trotman <ele...@gmail.com> wrote:
> On 5 June 2012 21:41, Randy Kramer <rhkra...@gmail.com> wrote:
>> In a comment to scintilla-Feature Requests-3531865, Neil (Hodgson) said this:
>> My question is (for Neil or anybody) is: can I write a Lexer in Luddite and
>> use it in, for example, Geany, Scite, or basically any Scintilla based
>> editor?
> If, as the active state documentation states, it compiles into a
> standard scintilla lexer then it "should" be able to be used in any
> Scintilla editor. But only if the compiled lexer is fully self
> contained and doesn't need any of Komodo code to run.
> Cheers
> Lex
>> Randy Kramer
>> --
>> You received this message because you are subscribed to the Google Groups "scintilla-interest" group.
>> To post to this group, send email to scintilla-interest@googlegroups.com.
>> To unsubscribe from this group, send email to scintilla-interest+unsubscribe@googlegroups.com.
>> For more options, visit this group at http://groups.google.com/group/scintilla-interest?hl=en.
> --
> You received this message because you are subscribed to the Google Groups "scintilla-interest" group.
> To post to this group, send email to scintilla-interest@googlegroups.com.
> To unsubscribe from this group, send email to scintilla-interest+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/scintilla-interest?hl=en.
> keywords are defined in a .udl file, are compiled into the
> .lexres file, and are managed by LexUDL.cxx, not a containing application.
So, as long as I "install" LexUDL.cxx and arrange to compile my (custom) .udf file into a .lexres file, I should be good to go.
BTW, I started to write a lexer (for TWiki markup language, plus some
modifications) in C++ but that effort is currently on hold (lack of time, frustration). When I get time and the ambition to restart that effort, I'll look more closely at Luddite. (C++ became very cumbersome with the things I wanted to do.)
Randy Kramer
On Tuesday 05 June 2012 01:13:25 pm Eric Promislow wrote:
> I've never tried it that way, but I don't see why not. The key is that
> Luddite doesn't
> generate a C++ lexer. It generates a lexres file, a pseudo-binary
> file. The full path
> to this file is placed in the first entry in the first keyword list
> for the language you
> want to lex (which is an instance of SCLEX_UDL), and ColouriseTemplate1Doc
> will load it the first time it sees it and build essentially a state
> machine that
> drives the lexing process.
> Unlike everything I said about keywords in lexers in the recent Visual
> Prolog thread, keywords are defined in a .udl file, are compiled into the
> .lexres file, and are managed by LexUDL.cxx, not a containing application.
> Hope this helps you get started. Let me know if you run into any problems.
> - Eric
> On Tue, Jun 5, 2012 at 5:07 AM, Lex Trotman <ele...@gmail.com> wrote:
> > On 5 June 2012 21:41, Randy Kramer <rhkra...@gmail.com> wrote:
> >> In a comment to scintilla-Feature Requests-3531865, Neil (Hodgson) said
> >> this:
> >> <quote>
> >> Message:
> >> Lexers are written in code for control and performance. There are other
> >> ways to write lexers. The most popular appear to be
> >> * Scintillua which uses Lua LPEG parsers http://foicica.com/scintillua/ > >> * Komodo's Luddite http://docs.activestate.com/komodo/4.4/luddite.html > >> </quote>
> >> My question is (for Neil or anybody) is: can I write a Lexer in Luddite
> >> and use it in, for example, Geany, Scite, or basically any Scintilla
> >> based editor?
> > If, as the active state documentation states, it compiles into a
> > standard scintilla lexer then it "should" be able to be used in any
> > Scintilla editor. But only if the compiled lexer is fully self
> > contained and doesn't need any of Komodo code to run.
Luddite isn't ideal for some of the reduced markup languages because
it works on one line at a time, and it provides limited state manipulation
facilities, mostly for programming-language constructs like
arbitrary string delimiters and here-documents.
But if it works for you, it's certainly faster than writing a
from-scratch C++ lexer.
On Thu, Jun 7, 2012 at 5:50 AM, Randy Kramer <rhkra...@gmail.com> wrote:
> Eric and Lex,
> Thanks for your responses!
> Now I understand, LexUDL.cxx is the key:
>> keywords are defined in a .udl file, are compiled into the
>> .lexres file, and are managed by LexUDL.cxx, not a containing application.
> So, as long as I "install" LexUDL.cxx and arrange to compile my (custom) .udf
> file into a .lexres file, I should be good to go.
> BTW, I started to write a lexer (for TWiki markup language, plus some
> modifications) in C++ but that effort is currently on hold (lack of time,
> frustration). When I get time and the ambition to restart that effort, I'll
> look more closely at Luddite. (C++ became very cumbersome with the things I
> wanted to do.)
> Randy Kramer
> On Tuesday 05 June 2012 01:13:25 pm Eric Promislow wrote:
>> I've never tried it that way, but I don't see why not. The key is that
>> Luddite doesn't
>> generate a C++ lexer. It generates a lexres file, a pseudo-binary
>> file. The full path
>> to this file is placed in the first entry in the first keyword list
>> for the language you
>> want to lex (which is an instance of SCLEX_UDL), and ColouriseTemplate1Doc
>> will load it the first time it sees it and build essentially a state
>> machine that
>> drives the lexing process.
>> Unlike everything I said about keywords in lexers in the recent Visual
>> Prolog thread, keywords are defined in a .udl file, are compiled into the
>> .lexres file, and are managed by LexUDL.cxx, not a containing application.
>> Hope this helps you get started. Let me know if you run into any problems.
>> - Eric
>> On Tue, Jun 5, 2012 at 5:07 AM, Lex Trotman <ele...@gmail.com> wrote:
>> > On 5 June 2012 21:41, Randy Kramer <rhkra...@gmail.com> wrote:
>> >> In a comment to scintilla-Feature Requests-3531865, Neil (Hodgson) said
>> >> this:
>> >> <quote>
>> >> Message:
>> >> Lexers are written in code for control and performance. There are other
>> >> ways to write lexers. The most popular appear to be
>> >> * Scintillua which uses Lua LPEG parsers http://foicica.com/scintillua/ >> >> * Komodo's Luddite http://docs.activestate.com/komodo/4.4/luddite.html >> >> </quote>
>> >> My question is (for Neil or anybody) is: can I write a Lexer in Luddite
>> >> and use it in, for example, Geany, Scite, or basically any Scintilla
>> >> based editor?
>> > If, as the active state documentation states, it compiles into a
>> > standard scintilla lexer then it "should" be able to be used in any
>> > Scintilla editor. But only if the compiled lexer is fully self
>> > contained and doesn't need any of Komodo code to run.
> --
> You received this message because you are subscribed to the Google Groups "scintilla-interest" group.
> To post to this group, send email to scintilla-interest@googlegroups.com.
> To unsubscribe from this group, send email to scintilla-interest+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/scintilla-interest?hl=en.
Thanks for the followup. The one line at a time might be the problem--one of my modifications to the TWiki markup is that I use the mbox From header to serve as a record separator.
If you're not familiar with the From header, it consists of an arbitrary number of lines, it can look something like this:
From "file index: askRhk04" Mon May 28 07:31:07 2007 Date: 05/28/07 07:31 am Subject: file index: askRhk04
(Normally the field after the From is the email address of the sender, but I've found in the software that I plan to use (including kmail, nail/mail) that I can put an arbitrary phrase in quotes and that works just fine.)
It can have any number of email header lines of the form:
<keyword>: <whatever>
and is finally ended by a blank line. For my purposes, I require that it have at least one Date: line and at least one Subject: line, but there may be an arbitrary number of such lines.
Does luddite have any ability to maintain state between lines? And then, for this case, after having recognized a valid mbox From header, go back to the From line and colorize and fold starting from there?
There are some other instances where I need to maintain state between lines, but this is the easiest example to give.
Thanks,
Randy Kramer
On Thursday 07 June 2012 01:17:43 pm Eric Promislow wrote:
> Luddite isn't ideal for some of the reduced markup languages because
> it works on one line at a time, and it provides limited state manipulation
> facilities, mostly for programming-language constructs like
> arbitrary string delimiters and here-documents.
> But if it works for you, it's certainly faster than writing a
> from-scratch C++ lexer.
> - Eric
> On Thu, Jun 7, 2012 at 5:50 AM, Randy Kramer <rhkra...@gmail.com> wrote:
> > Eric and Lex,
> > Thanks for your responses!
> > Now I understand, LexUDL.cxx is the key:
> >> keywords are defined in a .udl file, are compiled into the
> >> .lexres file, and are managed by LexUDL.cxx, not a containing
> >> application.
> > So, as long as I "install" LexUDL.cxx and arrange to compile my (custom)
> > .udf file into a .lexres file, I should be good to go.
> > BTW, I started to write a lexer (for TWiki markup language, plus some
> > modifications) in C++ but that effort is currently on hold (lack of time,
> > frustration). When I get time and the ambition to restart that effort,
> > I'll look more closely at Luddite. (C++ became very cumbersome with the
> > things I wanted to do.)
> > Randy Kramer
> > On Tuesday 05 June 2012 01:13:25 pm Eric Promislow wrote:
> >> I've never tried it that way, but I don't see why not. The key is that
> >> Luddite doesn't
> >> generate a C++ lexer. It generates a lexres file, a pseudo-binary
> >> file. The full path
> >> to this file is placed in the first entry in the first keyword list
> >> for the language you
> >> want to lex (which is an instance of SCLEX_UDL), and
> >> ColouriseTemplate1Doc will load it the first time it sees it and build
> >> essentially a state machine that
> >> drives the lexing process.
> >> Unlike everything I said about keywords in lexers in the recent Visual
> >> Prolog thread, keywords are defined in a .udl file, are compiled into
> >> the .lexres file, and are managed by LexUDL.cxx, not a containing
> >> application.
> >> Hope this helps you get started. Let me know if you run into any
> >> problems.
> >> - Eric
> >> On Tue, Jun 5, 2012 at 5:07 AM, Lex Trotman <ele...@gmail.com> wrote:
> >> > On 5 June 2012 21:41, Randy Kramer <rhkra...@gmail.com> wrote:
> >> >> In a comment to scintilla-Feature Requests-3531865, Neil (Hodgson)
> >> >> said this:
> >> >> <quote>
> >> >> Message:
> >> >> Lexers are written in code for control and performance. There are
> >> >> other ways to write lexers. The most popular appear to be
> >> >> * Scintillua which uses Lua LPEG parsers
> >> >> http://foicica.com/scintillua/ * Komodo's Luddite
> >> >> http://docs.activestate.com/komodo/4.4/luddite.html </quote>
> >> >> My question is (for Neil or anybody) is: can I write a Lexer in
> >> >> Luddite and use it in, for example, Geany, Scite, or basically any
> >> >> Scintilla based editor?
> >> > If, as the active state documentation states, it compiles into a
> >> > standard scintilla lexer then it "should" be able to be used in any
> >> > Scintilla editor. But only if the compiled lexer is fully self
> >> > contained and doesn't need any of Komodo code to run.
> > --
> > You received this message because you are subscribed to the Google Groups
> > "scintilla-interest" group. To post to this group, send email to
> > scintilla-interest@googlegroups.com. To unsubscribe from this group, send
> > email to scintilla-interest+unsubscribe@googlegroups.com. For more
> > options, visit this group at
> > http://groups.google.com/group/scintilla-interest?hl=en.
Oh, forgot to mention the most important / critical application I want to use--recoll, and the reason to use the mbox header is because recoll recognizes that as a record separator. (At one time that was the only record separator that recoll recognized, that may have changed by now. If it has, I probably still prefer to use that for the flexibility it gives me to view records in an email client.)
On Thursday 07 June 2012 04:24:35 pm Randy Kramer wrote:
> Thanks for the followup. The one line at a time might be the problem--one
> of my modifications to the TWiki markup is that I use the mbox From header
> to serve as a record separator.
> If you're not familiar with the From header, it consists of an arbitrary
> number of lines, it can look something like this:
> From "file index: askRhk04" Mon May 28 07:31:07 2007
> Date: 05/28/07 07:31 am
> Subject: file index: askRhk04
> (Normally the field after the From is the email address of the sender, but
> I've found in the software that I plan to use (including kmail, nail/mail)
> that I can put an arbitrary phrase in quotes and that works just fine.)
> It can have any number of email header lines of the form:
> <keyword>: <whatever>
> and is finally ended by a blank line. For my purposes, I require that it
> have at least one Date: line and at least one Subject: line, but there may
> be an arbitrary number of such lines.
> Does luddite have any ability to maintain state between lines? And then,
> for this case, after having recognized a valid mbox From header, go back to
> the From line and colorize and fold starting from there?
> There are some other instances where I need to maintain state between
> lines, but this is the easiest example to give.
> Thanks,
> Randy Kramer
> On Thursday 07 June 2012 01:17:43 pm Eric Promislow wrote:
> > Luddite isn't ideal for some of the reduced markup languages because
> > it works on one line at a time, and it provides limited state
> > manipulation facilities, mostly for programming-language constructs like
> > arbitrary string delimiters and here-documents.
> > But if it works for you, it's certainly faster than writing a
> > from-scratch C++ lexer.
On Thu, Jun 7, 2012 at 1:24 PM, Randy Kramer <rhkra...@gmail.com> wrote:
> Eric,
> Thanks for the followup. The one line at a time might be the problem--one of
> my modifications to the TWiki markup is that I use the mbox From header to
> serve as a record separator.
> If you're not familiar with the From header, it consists of an arbitrary
> number of lines, it can look something like this:
> From "file index: askRhk04" Mon May 28 07:31:07 2007
> Date: 05/28/07 07:31 am
> Subject: file index: askRhk04
> (Normally the field after the From is the email address of the sender, but
> I've found in the software that I plan to use (including kmail, nail/mail)
> that I can put an arbitrary phrase in quotes and that works just fine.)
> It can have any number of email header lines of the form:
> <keyword>: <whatever>
> and is finally ended by a blank line. For my purposes, I require that it have
> at least one Date: line and at least one Subject: line, but there may be an
> arbitrary number of such lines.
> Does luddite have any ability to maintain state between lines? And then, for
> this case, after having recognized a valid mbox From header, go back to the
> From line and colorize and fold starting from there?
> There are some other instances where I need to maintain state between lines,
> but this is the easiest example to give.
> Thanks,
> Randy Kramer
> On Thursday 07 June 2012 01:17:43 pm Eric Promislow wrote:
>> Luddite isn't ideal for some of the reduced markup languages because
>> it works on one line at a time, and it provides limited state manipulation
>> facilities, mostly for programming-language constructs like
>> arbitrary string delimiters and here-documents.
>> But if it works for you, it's certainly faster than writing a
>> from-scratch C++ lexer.
>> - Eric
>> On Thu, Jun 7, 2012 at 5:50 AM, Randy Kramer <rhkra...@gmail.com> wrote:
>> > Eric and Lex,
>> > Thanks for your responses!
>> > Now I understand, LexUDL.cxx is the key:
>> >> keywords are defined in a .udl file, are compiled into the
>> >> .lexres file, and are managed by LexUDL.cxx, not a containing
>> >> application.
>> > So, as long as I "install" LexUDL.cxx and arrange to compile my (custom)
>> > .udf file into a .lexres file, I should be good to go.
>> > BTW, I started to write a lexer (for TWiki markup language, plus some
>> > modifications) in C++ but that effort is currently on hold (lack of time,
>> > frustration). When I get time and the ambition to restart that effort,
>> > I'll look more closely at Luddite. (C++ became very cumbersome with the
>> > things I wanted to do.)
>> > Randy Kramer
>> > On Tuesday 05 June 2012 01:13:25 pm Eric Promislow wrote:
>> >> I've never tried it that way, but I don't see why not. The key is that
>> >> Luddite doesn't
>> >> generate a C++ lexer. It generates a lexres file, a pseudo-binary
>> >> file. The full path
>> >> to this file is placed in the first entry in the first keyword list
>> >> for the language you
>> >> want to lex (which is an instance of SCLEX_UDL), and
>> >> ColouriseTemplate1Doc will load it the first time it sees it and build
>> >> essentially a state machine that
>> >> drives the lexing process.
>> >> Unlike everything I said about keywords in lexers in the recent Visual
>> >> Prolog thread, keywords are defined in a .udl file, are compiled into
>> >> the .lexres file, and are managed by LexUDL.cxx, not a containing
>> >> application.
>> >> Hope this helps you get started. Let me know if you run into any
>> >> problems.
>> >> - Eric
>> >> On Tue, Jun 5, 2012 at 5:07 AM, Lex Trotman <ele...@gmail.com> wrote:
>> >> > On 5 June 2012 21:41, Randy Kramer <rhkra...@gmail.com> wrote:
>> >> >> In a comment to scintilla-Feature Requests-3531865, Neil (Hodgson)
>> >> >> said this:
>> >> >> <quote>
>> >> >> Message:
>> >> >> Lexers are written in code for control and performance. There are
>> >> >> other ways to write lexers. The most popular appear to be
>> >> >> * Scintillua which uses Lua LPEG parsers
>> >> >> http://foicica.com/scintillua/ * Komodo's Luddite
>> >> >> http://docs.activestate.com/komodo/4.4/luddite.html </quote>
>> >> >> My question is (for Neil or anybody) is: can I write a Lexer in
>> >> >> Luddite and use it in, for example, Geany, Scite, or basically any
>> >> >> Scintilla based editor?
>> >> > If, as the active state documentation states, it compiles into a
>> >> > standard scintilla lexer then it "should" be able to be used in any
>> >> > Scintilla editor. But only if the compiled lexer is fully self
>> >> > contained and doesn't need any of Komodo code to run.
>> > --
>> > You received this message because you are subscribed to the Google Groups
>> > "scintilla-interest" group. To post to this group, send email to
>> > scintilla-interest@googlegroups.com. To unsubscribe from this group, send
>> > email to scintilla-interest+unsubscribe@googlegroups.com. For more
>> > options, visit this group at
>> > http://groups.google.com/group/scintilla-interest?hl=en.
> --
> You received this message because you are subscribed to the Google Groups "scintilla-interest" group.
> To post to this group, send email to scintilla-interest@googlegroups.com.
> To unsubscribe from this group, send email to scintilla-interest+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/scintilla-interest?hl=en.
> Sure, this is similar to the way Luddite recognizes that it's transitioned
> from HTML markup to the JS inside a script tag, or CSS inside a style tag.
> Best to follow the links I gave. Start with html-mainlex.udl and follow
> the include's to get an idea of how it works.
