How can I scroll 6,000 text widgets?

[Luc]

That's an eye catching topic title, isn't it?

I've created a monster.

I had this big text widget that didn't suit my purposes.

So I left it where it was and inserted hundreds of text boxes as children
of that one text widget. It looks like a big table or grid.

I know what you're thinking. You're thinking why in the red hot blazes of
this blue round world would anybody do that???

Well, they absolutely need to be separate from all others individually.
They even have their own number IDs each. It's a requirement. And I need
many text editing features including colors.

Everything works except that I can't scroll the monster. I can see the
first half a dozen sets of boxes and can't reach the others.

So I removed the big underlying text widget and used a frame instead.
Silly me, the scroll command won't even work with a frame. Back to the
drawing board.

So I tried a canvas. But it won't scroll either! That's weird. I should
be able to scroll a canvas.

What approach do you recommend in my case?

Additional problem: I can only have so many child text widgets, more or
less 4,200. With 5,000 the application won't load anymore, it crashes
after trying for a few seconds. I wonder if that can be fixed too.

Alright! It's a subtitle editor. One box for subtitle number, another one
for subtitle timestamp, then another one for the original subtitle, one
for the translated subtitle and another one for certain indications
(e.g. character count, errors etc.) A feature length movie can have 1,200
subtitles, that's 6,000 boxes.

--
Luc

[Rich]

Luc <l...@sep.invalid> wrote:
> That's an eye catching topic title, isn't it?

Yes, indeed, and unnecessary. :)

> Everything works except that I can't scroll the monster. I can see
> the first half a dozen sets of boxes and can't reach the others.

Again, demo code or else it is not happening.

This scrolls 6000 text widgets just fine:

#!/usr/bin/wish

text .t -yscrollcommand [list .vsb set]
scrollbar .vsb -orient vertical -command [list .t yview]

grid .t .vsb -sticky news

for {set i 0} {$i < 6000} {incr i} {
text .st$i -width 40 -height 5
.t window create end -window .st$i
.t insert end \n
}

So there must be something else interfering.

> So I tried a canvas. But it won't scroll either! That's weird. I should
> be able to scroll a canvas.

For a canvas you have to explicitly tell it the area to scroll,
otherwise it won't scroll at all. Read the manpage carefully.

> What approach do you recommend in my case?

Not creating 6000 text widgets.

> Additional problem: I can only have so many child text widgets, more or
> less 4,200. With 5,000 the application won't load anymore, it crashes
> after trying for a few seconds. I wonder if that can be fixed too.

My code above creates 6000, no crash.

Changing 6000 to 60000 takes a few minutes to run, and consumes 437MB
of memory (according to top). But once up, scrolling to the bottom
happens just fine.

> Alright! It's a subtitle editor. One box for subtitle number, another one
> for subtitle timestamp, then another one for the original subtitle, one
> for the translated subtitle and another one for certain indications
> (e.g. character count, errors etc.) A feature length movie can have 1,200
> subtitles, that's 6,000 boxes.

No, that's 6,000 datapoints, which does not need to equate to 6,000
boxes.

You need enough boxes to fit on screen without scrolling, and you move
the 6,000 datapoints through the smaller set of boxes that are
on screen. The screen boxes let you look through a screen sized
peephole at the 6,000 datapoints, and "scrolling" moves the data
through the peephole.

https://wiki.tcl-lang.org/page/Virtual%20Scrolling

[Christian Gollwitzer]

Am 27.01.23 um 04:29 schrieb Luc:

> That's an eye catching topic title, isn't it?
>
> I've created a monster.
>
> I had this big text widget that didn't suit my purposes.
>
> So I left it where it was and inserted hundreds of text boxes as children
> of that one text widget. It looks like a big table or grid.
>
> I know what you're thinking. You're thinking why in the red hot blazes of
> this blue round world would anybody do that???
>
> Well, they absolutely need to be separate from all others individually.
> They even have their own number IDs each. It's a requirement. And I need
> many text editing features including colors.

Use tablelist:

https://wiki.tcl-lang.org/page/tablelist


The first screenshot shows one demo with an editable list of widget
properties.

Christian

[Ralf Fassel]

* Rich <ri...@example.invalid>

| > Additional problem: I can only have so many child text widgets, more or
| > less 4,200. With 5,000 the application won't load anymore, it crashes
| > after trying for a few seconds. I wonder if that can be fixed too.
>
| My code above creates 6000, no crash.
>
| Changing 6000 to 60000 takes a few minutes to run, and consumes 437MB
| of memory (according to top). But once up, scrolling to the bottom
| happens just fine.

Probably the OP is on Windows, where you have additional constraints of
how many graphics handles (the exact term just won't come to mind) you
can create. The limit can be raised by fiddling with the registry, but
there is no guarantee. Only solution is the one suggested by you:
scroll the data, not the widgets.

R'

[Luc]

On Fri, 27 Jan 2023 10:48:48 +0100, Ralf Fassel wrote:

> Probably the OP is on Windows, where you have additional constraints of
> how many graphics handles (the exact term just won't come to mind) you
> can create.

**************************

No, I use Linux.

I think Rich's reply kind of solves everything. His code indeed works on my
machine, no crash, so I have to rebuild my GUI brick by brick until I sort
things out. I didn't submit any code of my own because it's a very old and
pretty big application split across four files/includes that I repurposed
for this new one.

I also like the idea of scrolling the data rather than the widget, that
will certainly make everything a lot lighter and less prone to crashing.
I looked it up on the wiki, multiple pages deal with that concept, but I
also have an idea of how to implement it in my own way.

Thank you all again.

--
Luc

[Luc]

This works:

--------------------
package require Tk

text .t -yscrollcommand [list .vsb set]
scrollbar .vsb -orient vertical -command [list .t yview]
grid .t .vsb -sticky news

focus .t

for {set i 1} {$i <= 600} {incr i} {
frame .st$i -width 40 -height 5

.t window create end -window .st$i

text .st$i.text$i -height 2 -width 50
pack .st$i.text$i
.t insert end \n
.st$i.text$i insert end "this is $i"
}
--------------------




This doesn't work:

--------------------
package require Tk

toplevel .scroller -background "#c0c0c0"

text .scroller.t -yscrollcommand [list .scroller.vsb set]
scrollbar .scroller.vsb -orient vertical -command [list .scroller.t yview]
grid .scroller.t .scroller.vsb -sticky news
focus .scroller.t

for {set i 1} {$i <= 600} {incr i} {
frame .st$i -width 40 -height 5
.scroller.t window create end -window .st$i
text .st$i.text$i -height 2 -width 50
pack .st$i.text$i
.scroller.t insert end \n
.st$i.text$i insert end "this is $i"
}
--------------------


Why?


--
Luc
>>
**************************

[Rich]

Luc <l...@sep.invalid> wrote:
> This doesn't work:
>
> --------------------
> package require Tk
>
> toplevel .scroller -background "#c0c0c0"
>
> text .scroller.t -yscrollcommand [list .scroller.vsb set]
> scrollbar .scroller.vsb -orient vertical -command [list .scroller.t yview]
> grid .scroller.t .scroller.vsb -sticky news
> focus .scroller.t
>
> for {set i 1} {$i <= 600} {incr i} {
> frame .st$i -width 40 -height 5
> .scroller.t window create end -window .st$i
> text .st$i.text$i -height 2 -width 50
> pack .st$i.text$i
> .scroller.t insert end \n
> .st$i.text$i insert end "this is $i"
> }
> --------------------
>
>
> Why?

Read the text manpage, particularly the section entitled "EMBEDDED
WINDOWS". Pay *very close attention* to the parenthetical clause at
the end of the first paragraph of that section.

[Luc]

Yes, I read that before I asked, and I still don't see the point.
It says:

(subject to the usual rules for geometry management, which require the
text window to be the parent of the embedded window or a descendant of
its parent).

I made the big parent .scroller then made the text widget .scroller.t
then made its big series of numbered children:

.scroller.t window create end -window .st$i

Yet, the error says:

can't embed .st1 in .scroller.t
while executing

".scroller.t window create end -window .st$i"

But .scroller.t is the parent of .st$i. So what is going on? I'm stumped.

--
Luc
>>

[Rich]

No, . is the parent of the .st$i windows.

[Christian Gollwitzer]

Am 29.01.23 um 23:19 schrieb Luc:

> Yet, the error says:
>
> can't embed .st1 in .scroller.t
> while executing
> ".scroller.t window create end -window .st$i"
>
> But .scroller.t is the parent of .st$i. So what is going on? I'm stumped.

I think you misunderstand what "parent"/"child" is in Tk. It is defined
by the name. A child of .scroller.t would be named, e.g., .scroller.t.st1

In your first example, you used a "sibling" of the text window, not a
child as well.

Christian

[Luc]

**************************

On Sun, 29 Jan 2023 23:20:54 -0000 (UTC), Rich wrote:

> No, . is the parent of the .st$i windows.

**************************

On Mon, 30 Jan 2023 07:45:10 +0100, Christian Gollwitzer wrote:

> I think you misunderstand what "parent"/"child" is in Tk. It is defined
> by the name. A child of .scroller.t would be named, e.g., .scroller.t.st1
>
> In your first example, you used a "sibling" of the text window, not a
> child as well.
>
> Christian
>

**************************


OK, I understand it now. It's fixed. Thank you.


--
Luc
>>

[Luc]

Hi, Rich. About the code that you shared, quoted below,
do you have idea of how I can jump to any specific box
and make it immediately visible?

I need something like the [see] command.

All the boxes are children of a text widget, but that
text widget doesn't have any lines of text so I can't
really use the [see] command here.



Less important: how do I scroll that big collection of
boxes without using the mouse? I mean, jump from one
box to the other and scroll accordingly?

--
Luc
>>
**************************
On Fri, 27 Jan 2023 05:45:53 -0000 (UTC), Rich wrote:

[Rich]

Luc <l...@sep.invalid> wrote:
> Hi, Rich. About the code that you shared, quoted below,
> do you have idea of how I can jump to any specific box
> and make it immediately visible?
>
> I need something like the [see] command.

The parent text widget holding all the embedded windows has a [see]
command. Use that.

> All the boxes are children of a text widget, but that
> text widget doesn't have any lines of text so I can't
> really use the [see] command here.

Ah, study the code more closely. You will see that each embedded
window is one line in the text widget.

Also study the text widget man page, specifically the index option
"pathName".

> Less important: how do I scroll that big collection of boxes without
> using the mouse? I mean, jump from one box to the other and scroll
> accordingly?

If keyboard scrolling is not already working using the standard Tk
bindings, then you'll need to setup your own custom bindings on
specific keys actuations to effect scrolling.

Do keep in mind that when keyboard focus is inside an embedded window
that this fact may cause the parent to not 'hear' some of the relevant
keyboard events.

Also, please don't put your quoted message underneath a line containing
just two hyphens and a space, that denotes a "signature" on Usenet, and
standard clients omit quoting the signature when replying.

[Luc]

**************************

Yeah, well, now I see you put a line break after every box.
I didn't. That's why I couldn't move around comfortably.

I guess that will change the game in scrolling too. I'm surprised it even
scrolled at all though only with the mouse.

Thank you again.

--
Luc
>>

[Rich]

Luc <l...@sep.invalid> wrote:
> **************************

>
>
> Yeah, well, now I see you put a line break after every box.
> I didn't. That's why I couldn't move around comfortably.

Ah, you ended up with a huge single long line with 'word wrap' giving
you the vertical aligment. That's why you could not scroll. Doing
that causes the text to consume a huge amount of CPU time computing
word wrapping.

> I guess that will change the game in scrolling too. I'm surprised it
> even scrolled at all though only with the mouse.

Yes, one widget per line means no overlong lines, and no O(N^2) (or
worse) complexity factor computing what should appear on screen after a
scroll.

But, as you'll likely more often than not have keyboard focus inside
the inner widgets, you will likely have to build your own keyboard
scrolling bindings.

An alternate would be for you to rethink your approach, back up, and
use something like tablelist (https://www.nemethi.de/) which has much
of what you'll need to recreate yourself already written and ready to
use (including editing within cells).

[Luc]

On Sun, 5 Feb 2023 05:07:03 -0000 (UTC), Rich wrote:

> Ah, you ended up with a huge single long line with 'word wrap' giving
> you the vertical aligment. That's why you could not scroll. Doing
> that causes the text to consume a huge amount of CPU time computing
> word wrapping.

Actually, the whole thing is taking a little longer to load/build now.
But it's a very small difference. It's definitely worth it as it fixes
the problems I was having before.

> But, as you'll likely more often than not have keyboard focus inside
> the inner widgets, you will likely have to build your own keyboard
> scrolling bindings.

Nope. As I Key-Tab from one box to another, that triggers a proc that
uses [see] to adjust the big text widget and make sure that the current
box is always in the ideal position. I am one happy camper!

> An alternate would be for you to rethink your approach, back up, and
> use something like tablelist (https://www.nemethi.de/) which has much
> of what you'll need to recreate yourself already written and ready to
> use (including editing within cells).

I will have to disagree. I investigated tablelist some time ago and
found it terribly complicated. I couldn't even make it work. You know
that I am a little thick, but I accomplish my goals with many things
including canvases or SQLite. The tablelist package is extraordinarily
difficult.

I know the author hangs around here. Sorry.

More importantly, I need the degree of familiarity and control that the
stock text widget gives me. My boxes are overloaded with multiple kinds
of witchcraft that let me type significantly faster. I can't afford the
learning curve of porting my warm and fuzzy witchcraft to a package
I am extremely not familiar with. Stock Tk does the job extremely well.
I just haven't mastered all the neat tricks it can play.

--
Luc
>>

[Emiliano Gavilan]

On Fri, 27 Jan 2023 00:29:14 -0300
Luc <l...@sep.invalid> wrote:

> That's an eye catching topic title, isn't it?
>
> I've created a monster.
>
> I had this big text widget that didn't suit my purposes.
>
> So I left it where it was and inserted hundreds of text boxes as children
> of that one text widget. It looks like a big table or grid.
>
> I know what you're thinking. You're thinking why in the red hot blazes of
> this blue round world would anybody do that???
>
> Well, they absolutely need to be separate from all others individually.
> They even have their own number IDs each. It's a requirement. And I need
> many text editing features including colors.
>
> Everything works except that I can't scroll the monster. I can see the
> first half a dozen sets of boxes and can't reach the others.
>
> So I removed the big underlying text widget and used a frame instead.
> Silly me, the scroll command won't even work with a frame. Back to the
> drawing board.
>
> So I tried a canvas. But it won't scroll either! That's weird. I should
> be able to scroll a canvas.
>
> What approach do you recommend in my case?

What's the point of creating such a high count of text widgets when you
can only use one at the time?

I would just create a single one, either permanent or on-demand, and map
or unmap it as desired on the different areas of text.

Using a combination of tags and bindings it's easy to do what you want.

>
> Additional problem: I can only have so many child text widgets, more or
> less 4,200. With 5,000 the application won't load anymore, it crashes
> after trying for a few seconds. I wonder if that can be fixed too.
>
> Alright! It's a subtitle editor. One box for subtitle number, another one
> for subtitle timestamp, then another one for the original subtitle, one
> for the translated subtitle and another one for certain indications
> (e.g. character count, errors etc.) A feature length movie can have 1,200
> subtitles, that's 6,000 boxes.
>
> --
> Luc
>

--
Emiliano Gavilan

[Luc]

On Mon, 6 Feb 2023 11:20:03 -0300, Emiliano Gavilan wrote:

> What's the point of creating such a high count of text widgets when you
> can only use one at the time?
>
> I would just create a single one, either permanent or on-demand, and map
> or unmap it as desired on the different areas of text.
>
> Using a combination of tags and bindings it's easy to do what you want.

You are probably right, but I was in a hurry and went with the first
design that occurred to me. Perfect or not, it's working very well. I'm
having two very productive weeks with it. I'm surprised I could get it
to work so well in so little time. I may redesign it in the future.
There are quite a few rough edges yet though so I'll probably work on
those first.


--
Luc
>>

[Rich]

In your text widget UI, do you have a grid layout of cells, i.e.
something like:

.entry1 | .entry2 | .entry3
.entry4 | .entry5 | .entry6

If so, then switching from a "long text with 6000 entry widgets" to
"just enough entries to fit on screen" and scrolling the data through
the widgets you might find the Tcllib 'matrix' data structure to be
useful.

Specifically it lets you extract a "rectangle" of cells, and that
'rectangle' (appropriately sized) is exactly the data you'd want to
display in the fixed set of widgets sized for the screen.

Scrolling then becomes a matter of keeping track of a row offset (or
row and column if you also have horizontal scrolling) in the matrix for
the top row of data presently in the display and a proc to do the
appropriate fill of the entries (plus another to map changes back into
the matrix for storage).

[saitology9]

On 2/7/2023 5:09 AM, Luc wrote:

> design that occurred to me. Perfect or not, it's working very well. I'm
> having two very productive weeks with it. I'm surprised I could get it
>

That is interesting. After all, this is what it all comes down to at the
end: whether something is useful to you or not. is this some kind of a
personal productivity app?

[Luc]

Yes, it's for my own work routine. It's helping me a lot.

--
Luc
>>