Dynamic syntax highlighting

[Filipe Correia]

Hi,

I need is to make Ace highlight words that match the results of a
database query. So, the user will be entering text, and I need to
highlight the words dynamically, as they are entered. Any idea of how
I can approach this?

I've looked into creating a new mode, but it seems to be useful only
when we have a well defined set of tokenization rules.

I've also found this thread from some days ago, which seems to be
partially related to what I need, but couldn't figure out how to apply
it to this case:
https://groups.google.com/group/ace-discuss/browse_frm/thread/2c80a58b7293e86a#

Thanks!

Filipe

[Harutyun Amirjanyan]

Hi

basically you need to hook into session.bgTokenizer
and recompute tokens when you receive data from the server
xquery mode does something similar for semantic highlighting
information received from webWorker
mode can be useful to break string into words if you have a regexp for them

[Filipe Correia]

Thanks for the very quick reply Harutyun!

Something that I didn't say clearly and that i'd like to add: I don't
need anything fancy in terms of tokenization, as I will be searching
the database (and highlighting) single words. So, I assume each word
is a different token, at least for now.

What I envision is that when the user would type a word or a set of
words, I would issue a db query in the background, and then highlight
only the words that I found. Do you think this would work? Not sure
how to apply what you suggested, but maybe you were assuming a
different approach?

Some more comments below:

On Oct 30, 12:55 pm, Harutyun Amirjanyan <amirjan...@gmail.com> wrote:
> basically you need to hook into session.bgTokenizer
> and recompute tokens when you receive data from the server

You mean, I should create a new kind of Tokenizer and set the
bgTokenizer to it?

> xquery mode does something similar for semantic highlighting
> information received from webWorker

I found some code about the semantic highlighting that you mentioned
on worker-xquery.js
But I'm feeling a bit lost. The file is huuuuge :)

> mode can be useful to break string into words if you have a regexp for them

Right!


Thanks again!

Filipe

[Harutyun Amirjanyan]

i meant this one
https://github.com/ajaxorg/ace/blob/master/lib/ace/mode/xquery.js
the huge ones in lib\ace\mode\xquery folder are the equivalent of datebase

here is a very simple and dirty code doing something similar to what you want
showing which parts of ace you need to modify

var Tokenizer = require("ace/tokenizer").Tokenizer;
// words in datebase
var goodWords = Object.create(null)
// words checked to not be in datebase
// this can be omitted
// if it is ok to check for same words several times
var words = Object.create(null)
// cache for words pending resolution
var pending = []

var tokenizer = new Tokenizer({
"start": [
{token : function(val){
if (goodWords[val])
return "keyword"
if (words[val])
return "just-a-word"
if (pending.indexOf(val) == -1)
pending.push(val)
return "unknown"
}, regex : "\\w+"},
{token : "text", regex : "[^\\w]+"}
]
});


var queryTimeout = null
dirtySessions = []
attachToSession = function(session) {
var self = session.bgTokenizer
self.rehighlight = function() {
var states = this.states
var lines = this.lines
for (var i = states.length; i--;){
if (states[i] || !lines[i])
continue
states[i] = "start"
lines[i].forEach(function(t){
if (goodWords[t.value])
t.type = "keyword"
else if (words[t.value])
t.type = "just-a-word"
})
}
// this can be smarter and update only changed lines
this.fireUpdateEvent(0, states.length)
}
self.$tokenizeRow = function(row) {
// tokenize
var line = this.doc.getLine(row);
var data = this.tokenizer.getLineTokens(line, "start").tokens;
// if we found new words schedule server query
if (pending.length && !queryTimeout)
queryTimeout = setTimeout(queryServer, 700)
if (dirtySessions.indexOf(this) == -1)
dirtySessions.push(this)
// set state to null to indicate that it needs updating
this.states[row] = null

return this.lines[row] = data;
}
self.setTokenizer(tokenizer)
}
queryServer = function() {
fetchData(pending, function(serverWords){
// update goodWords and words based on serverWords
//...
// then
dirtySessions.forEach(function(x){
x.rehighlight()
})
dirtySessions = []
// some code to reset queryTimeout
// shedule again if there are more words in pending
// etc...
pending = []
})
}

/// use this instead of setMode
attachToSession(ace.session)

[Filipe Correia]

Great, got it working wonderfully now :)
Thanks a million!

Filipe

On Oct 30, 3:30 pm, Harutyun Amirjanyan <amirjan...@gmail.com> wrote:
> i meant this onehttps://github.com/ajaxorg/ace/blob/master/lib/ace/mode/xquery.js