text seach can eat memory

[jda...@gmail.com]

Searching a text widget for all instances of the empty string will lock up tclsh or wish, and apparently try to eat all memory. A very minimal example:

package require Tk
text .t
.t search -all "" 1.0

It also happens with nonempty text widgets, and ones that are displayed.
An error or empty list seems like a better response, or am I missing something?

I'm running tcl/tk 8.6.6 on slackware (linux 4.4.14)

Dave

[Francois Vogel]

Le 11/10/2018 à 17:31, jda...@gmail.com a écrit :
> Searching a text widget for all instances of the empty string will lock up tclsh or wish, and apparently try to eat all memory.

Many thanks for your report. I have opened a ticket to properly follow
this issue:

https://core.tcl-lang.org/tk/tktview/4b555aca34

Francois

[Francois Vogel]

I have proposed a fix of the search algorithm, and have added four test
cases in the test suite. See details in the ticket.

Francois

[Roderick]

On Thu, 11 Oct 2018, jda...@gmail.com wrote:

> Searching a text widget for all instances of the empty string [...]

> It also happens with nonempty text widgets, and ones that are displayed.

> An error or empty list seems like a better response, [...]

What is the better answer for 0/0 or 1/0 ? An error or 0? :)

There are infinitely many empty strings on a non empty widget. And
how many in an empty widget?

I just searched in google for an empty string, also there is the
result something special.

Rodrigo

[Roderick]

On Thu, 11 Oct 2018, Francois Vogel wrote:

> I have proposed a fix of the search algorithm, and have added four test cases
> in the test suite. See details in the ticket.

But what is the first character in the match, if the string has no
character?!

[Francois Vogel]

Regarding the expected result of '.t search -all "" 1.0', I proposed to
return the same result as '.t search "" 1.0'. This seemed more logical
to me than returning an error (ugly!) or the empty string (wrong!).
Indeed search results with the -all switch should contain search results
without this switch.

Actually the first match when searching for "" is the start index, and
this really is what '.t search "" $index' returned already. Now, when
the -all switch is added, I choose to return that same start index
rather than nothing or throwing and error out.

Now, in the special case when the text widget is empty there is no
difference: '.t search -all "" 1.0' returns the same result as '.t
search "" 1.0' (and this result is "1.0").

Francois

[heinrichmartin]

On Thursday, October 11, 2018 at 10:50:50 PM UTC+2, Francois Vogel wrote:
> Regarding the expected result of '.t search -all "" 1.0', I proposed to
> return the same result as '.t search "" 1.0'. This seemed more logical
> to me than returning an error (ugly!) or the empty string (wrong!).

Empty string is definitely wrong, it is an empty list if this was chosen for -all.

> Indeed search results with the -all switch should contain search results
> without this switch.

+1

> Now, when
> the -all switch is added, I choose to return that same start index
> rather than nothing or throwing and error out.

regexp, that is mentioned as reference in the documentation of search -all, returns *all* matches for the empty pattern. However, string first returns -1.

My guts feeling says, that there is no exact match for the empty string, but many matches for the empty regexp pattern. On the other hand, fixing this bug should not modify the behavior of search without -all.

Conclusion imho: return all indices (with and without -regexp).

> Now, in the special case when the text widget is empty there is no
> difference: '.t search -all "" 1.0' returns the same result as '.t
> search "" 1.0' (and this result is "1.0").

side note: when I init text as in your minimal example, it is not empty but contains a newline character {1 == [.t count -chars 1.0 end]}.

<nitpicking>
regexp matches the empty string at the start of the empty string, but not the empty string at the end of a non-empty string. This might also be debatable.
expect:~$ regexp -all -inline "" ""
{}
expect:~$ regexp -all -inline "" "a"
{}
expect:~$ regexp -all -inline "" "ab"
{} {}

[Roderick]

On Fri, 12 Oct 2018, heinrichmartin wrote:

> My guts feeling says, that there is no exact match for the empty string,

I think yes. It is a string of length 0 beginning at a certain (every)
index.

The solving of this problem thows many questions.

(1) it can introduce invisible inconsistencies that till now do not
appear because the command hangs. It may demand from programmers
checks and make programs more complex.

(2) it can introduce different behaviour of different commands as
you note (regexp, string first ...).

Rodrigo

[Francois Vogel]

Le 12/10/2018 à 11:11, Roderick a écrit :
>
> The solving of this problem thows many questions.
>
> (1) it can introduce invisible inconsistencies that till now do not
>     appear because the command hangs. It may demand from programmers
>     checks and make programs more complex.
>
> (2) it can introduce different behaviour of different commands as
>     you note (regexp, string first ...).

Well, NOT solving this ticket throws a much worse question: Tcl/Tk was
hanging and eating all user memory, which is simply not acceptable.

Re: (1): I'm not feeling ashamed about this, since, previsely,
previously the command hanged, and this was so much worse.

Re: (2): Perhaps. We can TIP unification and fix this. There are efforts
in this direction already. I think emergency fix for the problem was the
way to go.

Francois