Loading 'built-in' lexers dynamically

[Nick Treleaven]

Hi,
I'm trying to load lexers distributed with Scintilla dynamically with
SCI_LOADLEXERLIBRARY but I've had no success. The idea was to reduce
binary size of our application and memory use by only loading the
lexers the user actually needs. The application is Geany (lightweight
IDE) and we want to support a wide range of languages.

A quick google found:
http://www.scintilla.org/SciTEExternalLexer.html

Only LexCaml.cxx and LexHaskell.cxx define the required symbols. So it
seems almost all the lexers distributed with Scintilla can't be loaded
dynamically?

Thanks,
Nick

[Robert Roessler]

I believe that would be correct... when I wrote the OCaml lexer, I was
still learning how a lexer interacted with its container - and it seemed
useful to explore this alternate packaging and use model.

While it hasn't been used in this mode for some time now, it worked fine
at one time. If you want to play with it (or adapt the code for use in
other lexers), [obviously] don't forget to define the symbol
BUILD_AS_EXTERNAL_LEXER. ;)

Robert

[Neil Hodgson]

Nick Treleaven:

> I'm trying to load lexers distributed with Scintilla dynamically with
> SCI_LOADLEXERLIBRARY but I've had no success. The idea was to reduce
> binary size of our application and memory use by only loading the
> lexers the user actually needs. The application is Geany (lightweight
> IDE) and we want to support a wide range of languages.

What do you think you will be saving here? Paging is generally much
more efficient than manual management of loading. There is also quite
a bit of overhead for having separate dynamic libraries considering
the small size of each lexer.

> A quick google found:
> http://www.scintilla.org/SciTEExternalLexer.html
>
> Only LexCaml.cxx and LexHaskell.cxx define the required symbols. So it
> seems almost all the lexers distributed with Scintilla can't be loaded
> dynamically?

Lexers must be coded to explicitly support being loaded
dynamically. The original use case was for extending an existing
application with separately developed lexers. It never looked
worthwhile to me but others wanted it.

Neil

[Nick Treleaven]

On Tue, 6 Oct 2009 11:19:51 +1100
Neil Hodgson <nyama...@gmail.com> wrote:

> > I'm trying to load lexers distributed with Scintilla dynamically with
> > SCI_LOADLEXERLIBRARY but I've had no success. The idea was to reduce
> > binary size of our application and memory use by only loading the
> > lexers the user actually needs. The application is Geany (lightweight
> > IDE) and we want to support a wide range of languages.
>
> What do you think you will be saving here? Paging is generally much
> more efficient than manual management of loading. There is also quite
> a bit of overhead for having separate dynamic libraries considering
> the small size of each lexer.

OK, perhaps it's not worth it for efficiency reasons then, I don't
really know about these things. I just don't like seeing the app binary
size creeping up each time we add a lexer.

> > A quick google found:
> > http://www.scintilla.org/SciTEExternalLexer.html
> >
> > Only LexCaml.cxx and LexHaskell.cxx define the required symbols. So it
> > seems almost all the lexers distributed with Scintilla can't be loaded
> > dynamically?
>
> Lexers must be coded to explicitly support being loaded
> dynamically. The original use case was for extending an existing
> application with separately developed lexers. It never looked
> worthwhile to me but others wanted it.

Thanks for explaining.

I still think this would be a useful feature so the user could add
some of the 'built-in' lexers we don't include in Geany if they need
them.

Regards,
Nick