Javascript choaking inside tiddly <script> tag

[Shankar Swamy]

This is probably a javascript/browser issue than tiddly related. But
I am starting here with the hope that others have run into this issue
and will help me out.

I have a tidler which has a javascript. The script takes an input
pattern interactively, and opens a tiddler - call that data string -
and does some processing on the text in the data string based on the
input pattern. So I have two javascript statements like this
somewhere in the middle of the script:

var ExactPattern = "TITLE:\\s*" + myTitle;
ExactPattern = myTmpString.match(ExactPattern);

"myTmpString" is the data string. I noticed that after the data
string reaches a certain length, the *.match(...) function never
returns!

I have not determined the exact limiting length of the string; but it
definitely is stalling with the data string that is ~12000 characters
long.

I am using Firefox browser.

I am wondering if someone knows what limit I am running into so that I
can find an appropriate work around.

Thanks for your time & help.

Cheers; 'best,
shankar swamy

[Shankar Swamy]

On further looking around on the Internet, it appears like the limit
is coming from the Javascript implementation.

Even though ECMAScript Programming Language Specification does not
seem to have a limitation on this, apparently various implementations
of Javascript do set a limit on this - or, that is what multiple
Javascript users seem to saying.

Now I am wondering if this also means that a statement like:
var myTmpString = store.getTiddlerText(...) ;
call might also fail at some point, when the tiddler text becomes too
large. But somehow that does not sound right to me - for one reason I
don't see anyone mention on the tiddly forums about this call failing
or issues with the length of a tiddler. (Or the group search is
failing me!)

This means, for the current use, I have to ensure that my tiddlers
don't exceed that length. :-(

If you have another way of doing it without putting a llimitation on
the size of the tiddler, please let me know.

Cheers; 'best,
shankar

[PMario]

On Apr 14, 5:08 am, Shankar Swamy <shankar.sw...@gmail.com> wrote:

> var ExactPattern = "TITLE:\\s*" + myTitle;
> ExactPattern = myTmpString.match(ExactPattern);

> I have not determined the exact limiting length of the string; but it
> definitely is stalling with the data string that is ~12000 characters
> long.

TW search term highlighting works with big tiddlers, so there
shouldn't be a limit.

Is it possible that you post a minimal test case at tiddlyspace or
tiddlyspot?

-m

[Eric Shulman]

On Apr 13, 8:08 pm, Shankar Swamy <shankar.sw...@gmail.com> wrote:

> var ExactPattern = "TITLE:\\s*" + myTitle;
> ExactPattern = myTmpString.match(ExactPattern);

The argument for .match(...) is supposed to be a regular expression.
Thus:

var pattern = "TITLE:\\s*" + myTitle;
var regexp = new RegExp(pattern);
var result = myTmpString.match(regexp);

It's possible that the browser's implementation of .match() allows for
passing a text string, which can then be converted to a RegExp object
internally. There may also be some static allocation of memory, as
some regexp implementations use globally-scoped 'state variables' to
track the 'parse position' in the text, so that iterative functions
can be applied to loop through all the matches.

I expect that the problem may be related to javascript-internal memory
allocation for locally vs. globally scoped variables. I note that
you've used the same variable name, ExactPattern, on both sides of the
assignment:
> ExactPattern = myTmpString.match(ExactPattern);
Potentially, this could trigger javascript to consume more memory when
the objects are pushed onto the 'call stack' during evaluation, or
when the resulting value is returned and then assigned.

In any case, I'm certain that regexp searches *can* operate on tiddler
content that is much larger than 12K, as I have several very large
plugins that are each >32K and they are searchable without problems.
Give my suggested code above a try and let me know if it makes any
difference.

-e

[Shankar Swamy]

Thanks PMario & Eric.

The crux was the issue was a substring "++" that was hiding somewhere
in the middle of the "myTmpString" that possibly sent the ::match(...)
into a tailspin. I caught that when I examined the characters where
the string search seemed to choke. To change that I had to insert the
following at an appropriate place:
myTmpString = myTmpString.replace(/[-[\]{}()*+?.,\\^$|#\s]/g, "\\$&");
and it started working without any limitations on the string lengths.

I also fixed the sloppy coding, as suggested by Eric, to read as:

var ExactPattern = "TITLE:\\s*" + myTitle;

ExactPattern = myTmpString.match(new RegExp(ExactPattern));
- thank you Eric.

Cheers; 'best,
shankar