List of items with variable height

121 views
Skip to first unread message

Joe Betz

unread,
Oct 29, 2021, 5:04:13 PM10/29/21
to
Using Dolphin's UI framework, what is the correct way to generate a list of items of variable height with a vertical layout?

For example, a list of messages in a chat room. I tried ListView but that didn't work since all rows have to have the same height and will need to do a lot more customization besides. Also tried drawing each message on a canvas but that was too painful.

Is there an abstraction between these two that could solve the problem?

- Joe

john.a...@gmail.com

unread,
Oct 30, 2021, 6:22:38 AM10/30/21
to
Hi Joe - not something I've had to do myself but here's a few ideas:

1) Use a ListBox created with the LBS_OWNERDRAWVARIABLE style and override wmMeasureItem:wParam:lParam in ListBox to set the size of each item. You'll also need to draw the items yourself. See https://docs.microsoft.com/en-us/windows/win32/controls/wm-measureitem

2) Use a ContainerView with a FlowLayout layout manager and add each item as a subview. Example (modified from FramingLayout class>>example1):

(shell := View desktop addSubView: ShellView new)
backcolor: Color face3d;
layoutManager: (layout := FlowLayout new);
extent: 300 @ 200.
shell insets: (10 @ 10 corner: 10 @ 10).
layout
verticalGap: 10;
horizontalGap: SmallInteger maximum. "Force one item per row"
label := shell addSubView: StaticText new.
button := shell addSubView: PushButton new.
text1 := shell addSubView: MultilineTextEdit new.
text2 := shell addSubView: MultilineTextEdit new.
label text: 'Label'.
label rectangle: (10 @ 10 extent: 100 @ 20).
button
text: 'Button';
command: #yourself.
text1
text: 'Large multiline text';
wordWrap: true.
layout resize: text1 to: (10 @ 40 corner: 130 @ 190).
text2
text: 'Small multiline text';
wordWrap: true.
shell show

You'd lose the selection behavior compared to the ListBox approach but you could create a custom view for each item which provides any required functionality (e.g. reply to message).

3) Create a complete custom view to display the entire list and manage the required behavior. Probably the most work but also the most flexible. See the Dolphin MoenTree View package for an example of a custom view of a selectable collection of objects.

Hope this helps.

John Aspinall

danie...@gmail.com

unread,
Oct 30, 2021, 5:19:22 PM10/30/21
to
Expanding on John Aspinall's post—I've used the ContainerView-with-FlowLayout approach successfully myself. It helps to make a Presenter that displays a single row, with a view resource for the components needed to do so. Then a Presenter for the list that can use `<ItemPresenter> createIn: self` to add rows, `#model:` to update them, and `#remove:` to remove them. If the list is going to be longer than 3-5 items, avoid recreating everything every time the model changes—only create or remove as many sub-presenters as necessary to match the number of items in the model. It also helps to wrap that whole step in a #noRedrawDo: so you only repaint once the sub-presenters are back in sync with the model. You can of course wrap the whole thing in a ScrollingDecorator to get scrolling behavior.

Joe Betz

unread,
Nov 1, 2021, 12:38:55 AM11/1/21
to
Thanks! I went with John's second idea + Daniel's advice. Exactly what I needed.

The trickiest part was figuring out how to resize the MessagePresenter's based on the size of the message content. Ended up implementing `StaticText>>resizeToFitContent` with Blair's solution in https://groups.google.com/g/comp.lang.smalltalk.dolphin/c/uPXvG-V2DGo/m/G1uVuasFCAIJ.

StaticText>>resizeToFitContent
| contentSize |
contentSize := self canvas
font: self actualFont;
textExtent: self value
width: self width
alignment: self alignment.
self extent: (self calcExtentFromClientExtent: contentSize + 2)

Joe Betz

unread,
Nov 2, 2021, 12:55:11 AM11/2/21
to

> Then a Presenter for the list that can use `<ItemPresenter> createIn: self` to add rows, `#model:` to update them, and `#remove:` to remove them.

When the View is being generated for the first time, where should the `<ItemPresenter> create: self` messages be called to create presenters for the initial list of items?

Naively, I put it all in `model:`, but that broke when it was initialized with a nonempty list, presumably because the view for the container presenter is still a DeafObject at that point.

danie...@gmail.com

unread,
Nov 5, 2021, 11:29:34 AM11/5/21
to
> Naively, I put it all in `model:`, but that broke when it was initialized with a nonempty list, presumably because the view for the container presenter is still a DeafObject at that point.

What I did is have a method (e.g. `#updateListPresenters`) that brings the model and sub-presenters/views in sync, but bails if either the model or the view is missing (nil/deaf). Then I called it from both `#onViewAvailable` and `#model:`. That way it doesn't matter what order things happen in, we'll get everything in sync as soon as we have everything we need.

Joe Betz

unread,
Nov 7, 2021, 5:52:55 AM11/7/21
to
Okay, at this point I've tried both the ContainerView with FlowLayout and ListBox with custom draw method and neither worked out. In short, the former ended up being too slow and the former just ends up being all sorts of wrong on an implementation level.

> 2) Use a ContainerView with a FlowLayout layout manager and add each item as a subview. Example (modified from FramingLayout class>>example1):
ou'd lose the selection behavior compared to the ListBox approach but you could create a custom view for each item which provides any required functionality (e.g. reply to message).

The problem here was the messages took way too long to draw. For 50 messages it took 500ms rather than 0 or 1ms when I just shoved them into a ListView. My intuition was that it was an issue with the FlowLayout having to recompute each time a new message was added, but even after removing the layout it was still orders of magnitude too slow. The only thing that made any impact on the draw time was removing as many MVP triads as possible and render the entire message with a single TextPresenter. That got it down to 150ms, but that still isn't nearly good enough.

Running the profiler on the loop that calls `MessagePresenter createIn: self` for each message was interesting but not very enlightening. If interpreted it correctly, most of the time seemed to be spent in InputState, probably waiting for responses from Windows APIs. Another hotspot seemed to be code for replacing view proxies with actual views.

In the end, my conclusion is that there is significant performance overhead in using the MVP triad and that it doesn't scale well for non-standard controls.

> 1) Use a ListBox created with the LBS_OWNERDRAWVARIABLE style and override wmMeasureItem:wParam:lParam in ListBox to set the size of each item. You'll also need to draw the items yourself. See https://docs.microsoft.com/en-us/windows/win32/controls/wm-measureitem

It turns out that the draw and measure events don't get sent to the ListBox but rather their "owner", which ends up being whatever the parent view of ListBox is. E.g., if it's a ContainerView, then it needs to be subclassed and the wmDrawItem: and wmMeasureItem: methods need to be overriden in order to receive draw and measure events for the ListBox. Really annoying.

This is feasible for wmDrawItem: because the DRAWEVENTSTRUCT contains a reference to the ListBox view which you can then delegate the message to. It doesn't work so well with wmMeasureItem: since it contains no such reference, meaning that the ContainerView needed to hold either a reference to the ListBox or its model in order to even know what its measuring.

All in all it sortof works but seems to break the standard way of organizing views and would require a significant refactor to work in a sane way.

3) Create a complete custom view to display the entire list and manage the required behavior. Probably the most work but also the most flexible. See the Dolphin MoenTree View package for an example of a custom view of a selectable collection of objects.

So yeah, I'm left with this, which I strongly suspected was going to be the case from the beginning. The fact that the class hierarchy tree seems to be able to draw hundreds of objects nearly instantly is promising and what I'm aiming for.

Joe Betz

unread,
Nov 7, 2021, 5:54:18 AM11/7/21
to
**In short, the former ended up being too slow and the latter just ends up being all sorts of wrong on an implementation level.

Joe Betz

unread,
Nov 7, 2021, 5:55:30 AM11/7/21
to
> What I did is have a method (e.g. `#updateListPresenters`) that brings the model and sub-presenters/views in sync, but bails if either the model or the view is missing (nil/deaf). Then I called it from both `#onViewAvailable` and `#model:`. That way it doesn't matter what order things happen in, we'll get everything in sync as soon as we have everything we need.

Makes sense, thanks!

danie...@gmail.com

unread,
Nov 8, 2021, 11:23:35 PM11/8/21
to
Yeah, performance does start to suffer with massive numbers of controls on screen. The situations I've used this technique in have generally been limited to a dozen or so rows in the common case, with some performance degradation being expected and acceptable if it grows beyond that. You may well be right, in the end, that a fully custom control will be the only performant solution, but I have a few thoughts—if nothing else, I'm curious about what's really going on here:

1. When profiling, make sure you're really profiling everything—some things (painting in particular, but that in turn may be what triggers layout) may only happen when the main event loop returns to processing messages. As a start, however it is you measured the 500ms you refer to, make sure your profiler sample set actually contains that many samples, actually covers the whole time period you're interested in. If it doesn't...I'm not familiar enough with the guts of the profiler and the event loop to just tell you what you need to know offhand, but have a look at InputState>>loopWhile:, and at the way the profiler makes a distinction between a sample set being "active" or somesuch and the profiler actually _running_. You may not be able to use the simple Profiler class>>profile: API, but it should be possible to manually create a sample set, start the sampling process before opening the list, and stop it after all processing is done—a #postToInputQueue block (*not* the newer #postToMessageQueue, you want to wait until *all* Win32 messages are processed) might be a good place, or...I think there's a method #onEnterIdle somewhere, do a selector search for *idle* maybe. Conceptually this is more-or-less a hook for WM_IDLE, though that may not be the implementation.

2. When you say it takes 500ms, is that for the first render of 50 messages, or is that every time you add/remove/update one? Make sure your "sync" method, that gets the right number of sub-presenters in place, isn't doing extra work—only add or remove if the *number* of items changes, and only as much as you need to. Mine looked something like (pseudocode):

self view noRedrawDo: [
[self model size > self subPresenters size] whileTrue: [SubPresenter createIn: self].
[self subPresenters size > self model size] whileTrue: [self remove: self subPresenters last].
self subPresenters with: self model do: [:presenter :item | presenter model: item]].

Where only one, and perhaps neither, of the while loops will run. If you know things about what items may have changed, you can optimize which presenters have their model replaced as well.

3. You can compute the height of a row without actually instantiating a view for it—DrawText[Ex] with DT_CALCRECT, you can work backwards from there to figure out what Dolphin exposes it as. So, now you know the height the whole list *would* take up, and you can optimize the initial creation of sub-presenters to be lazy—only create as many as are actually visible on-screen, but size the container within the ScrollingDecorator to the full height it will eventually need. Then intercept scrolling and add new rows as needed—though this will probably force you to choose between laggy scrolling, if you do it in a synchronous/blocking way, or a flash of empty space before the new items fill in. It'll be...*interesting*...to get it to work with click-and-drag-the-scrollbar scrolling, too, since that involves mouse capture, but...well, actually you could fill the extra space with an ImageView displaying a tiled mockup of a row, so the user knows there will be *something* there but it only populates when they release the mouse. Or, you could create the visible ones on initial load, then the rest one-at-a-time in...oh, any number of approaches—WM_IDLE handler, fast-firing WM_TIMER, forked Process that calls back in with #postToInputQueue, etc—so that most likely they would all be created by the time the user interacts with them, but the event loop is only ever blocked for as long as it takes to create a single row.

4. You might experiment with hard-coding the creation of the row views rather than instantiating them from a resource. You would do something like:

newSubPresenter := self add: ItemPresenter new.
newView := ContainerView new.
newView name: (newView addSubView: StaticText new) as: 'message'.
...etc...
self view addSubView: newView.
newSubPresenter view: newView.

Not sure if this would be faster than the resource, but you said about switching from proxies to views being slow, so maybe? Also, I'm not sure whether creating the entire row view before adding it to the parent will work—some views have issues being re-parented, so if anything you do while configuring the children of the row causes it to be fully realized (i.e. #create'ed), it might not slot in to the outer container properly. If needed you can add it immediately when you create it, e.g. newView := self view addSubView: ContainerView new. Adding it later *might* have better performance, but it also might not, so check. In any case probably the most important thing is to wrap the whole thing in `self view noRedrawDo:`!

5. If you want to get *really* clever, you could do what the iOS and Android list components do, and recycle subviews when scrolling in order to *only* ever have as many as fit on screen at once. In your case you would reuse the Presenters as well, giving them a new model. Actually, now that I think about it, this might be quite doable—you would need to manually position the row views (still maintaining a container representing the overall size of the list, to get decent scrolling behavior out of the box), maintain a list of the order the item presenters are currently in and what index in the list is the first visible item, and cycle them around as you scroll, moving the first one to the end when scrolling down and vice-versa. You'd need to watch resize events in this case to add/remove sub-presenters.

----

Re: the owner-drawn ListBox—there are places elsewhere in Dolphin where a parent view automatically redirects such notifications to the appropriate child—I want to say all the NM_ messages work this way, or rather they are all *actually* a WM_NOTIFY to the parent but it re-dispatches to the appropriate child? You might be able to take a similar approach, and you might find you can get quite far with loose methods and additions to the message tables held in class variables, without needing to subclass ContainerView and without *technically* modifying core code at all.

For WM_MEASUREITEM specifically, I see something about "CtlID (Type: UINT): The identifier of the combo box or list box. This member is not used for a menu."—it's not a handle, but it might be enough information to identify the originating control in a generic way that you could be comfortable adding to ContainerView-in-general rather than a hack for your specific case. You'll have to search for what it actually means and how to use it to retrieve the actual control.

----

Okay, whew, wall of text. Hope that helps, let me know how it goes!

Joe Betz

unread,
Nov 21, 2021, 11:11:46 PM11/21/21
to
> Okay, whew, wall of text. Hope that helps, let me know how it goes!

At this point I've divorced from MVP and standard Windows controls completely and am now drawing everything using GDI+. Regressing to a lower level API was painful and it took some time to get rendering working correctly, but it's got both the performance and flexibility I was lacking with other options. With a double buffered view, it is able to handle redrawing a list of 100 messages for mouse over and mouse leave events with no visible flicker or delay (the message is outlined on focus).

I'm not sure what the implications are of using GDI+ (i.e., GdiplusGraphics) rather than the corresponding UserLibrary methods, but so far it seems to perform just as well with a better API.

> 1. When profiling, make sure you're really profiling everything—some things (painting in particular, but that in turn may be what triggers layout) may only happen when the main event loop returns to processing messages. As a start, however it is you measured the 500ms you refer to, make sure your profiler sample set actually contains that many samples, actually covers the whole time period you're interested in. If it doesn't...I'm not familiar enough with the guts of the profiler and the event loop to just tell you what you need to know offhand, but have a look at InputState>>loopWhile:, and at the way the profiler makes a distinction between a sample set being "active" or somesuch and the profiler actually _running_. You may not be able to use the simple Profiler class>>profile: API, but it should be possible to manually create a sample set, start the sampling process before opening the list, and stop it after all processing is done—a #postToInputQueue block (*not* the newer #postToMessageQueue, you want to wait until *all* Win32 messages are processed) might be a good place, or...I think there's a method #onEnterIdle somewhere, do a selector search for *idle* maybe. Conceptually this is more-or-less a hook for WM_IDLE, though that may not be the implementation.

Noted. I didn't dig any deeper with profiling the MVP solution, but good to know for the future profiling work.

> 2. When you say it takes 500ms, is that for the first render of 50 messages, or is that every time you add/remove/update one?
Just on the first render. It did well enough for add/remove/update events, aside from a flicker which I assume could have been fixed with a double buffered view.
> 3. You can compute the height of a row without actually instantiating a view for it—DrawText[Ex] with DT_CALCRECT, you can work backwards from there to figure out what Dolphin exposes it as. So, now you know the height the whole list *would* take up, and you can optimize the initial creation of sub-presenters to be lazy—only create as many as are actually visible on-screen, but size the container within the ScrollingDecorator to the full height it will eventually need.

Yup, makes sense. I think that's called virtual scrolling? It's something I want and might eventually need but am punting to the future now that the critical performance issues are resolved.

> Re: the owner-drawn ListBox—there are places elsewhere in Dolphin where a parent view automatically redirects such notifications to the appropriate child—I want to say all the NM_ messages work this way, or rather they are all *actually* a WM_NOTIFY to the parent but it re-dispatches to the appropriate child?

Huh, interesting. I'll check if ever I come back to standard controls, but I'm quite happy having ditched them so that might not ever happen. :D

danie...@gmail.com

unread,
Nov 24, 2021, 11:52:13 AM11/24/21
to
> At this point I've divorced from MVP and standard Windows controls completely and am now drawing everything using GDI+.

Yeah, that sounded like the direction you were going, I just got carried away since I've dealt with this stuff before.

> I think that's called virtual scrolling?

Yes, exactly—in fact this is already implemented in the native ListView/ListBox controls, and Dolphin ListViews use it by default (it's controlled by the #isVirtual aspect). It's actually probably *easier* to implement with a custom control, since you're just redrawing the whole thing and don't have to keep track of which subviews have which models etc, just ask for the scroll offset and figure out which item to start with and where.

Or, this *might* be a reason to explore the owner-drawn ListView option—tradeoff between figuring out the message redirection but getting virtual scrolling for free vs. no redirection (and perhaps more precise control of things like dividers, I realize) but having to implement virtual scrolling yourself.

Joe Betz

unread,
Dec 21, 2021, 10:24:01 AM12/21/21
to
I revisited the owner drawn ListBox approach just to get an idea of how it compared to my custom GDI+ list view. The message redirection ended up being less of an issue since I ended up subclassing ContainerView for other reasons.

Virtual scrolling definitely seems to be helping as there is definitely less flicker when scrolling, and hover effects look really nice, but it introduced a couple of issues of its own that made it worse overall.

The biggest issue is there doesn't seem to be a way to add a new list item without invalidating the entire view. Or I couldn't figure out how the hell to do it. Neither LB_INSERTSTRING (in ListBox>>basicAdd:atIndex) nor LB_SETITEMDATA seem to be intelligent enough to just send the minimal WM_DRAWITEM and WM_MEASUREITEM messages necessary to update the canvas. They have no immediate visible effect, which is particularly bad when the list is initially empty and gets populated dynamically. The only way I could get the new items to show up was by calling ListBox>>refreshContents, but each call to results in a nasty flicker effect that ruins the performance benefits. Even debouncing the updates so they're spaced out by at least 100ms was still an eyesore.

Another issue was that selecting an item adjusts the view so that the full item content is visible. Really not what I want for a list with variable heights. And the only reason I wanted to set the selection at all was to capture hover states since there doesn't seem to be a concept for a particular item being in focus. Perhaps a virtual focus could work, but then I'd lose the advantages of being able to use WM_DRAWITEM for precise updates which seems to be the biggest advantage of using ListBox thus far.

I'm still puzzled by the first issue since the inability to support dynamic list modifications makes it almost useless. I'm guessing it's a peculiarity of it being owner drawn, but eh, I've wasted too much time on it already. Maybe I'll come back to it later.

> For WM_MEASUREITEM specifically, I see something about "CtlID (Type: UINT): The identifier of the combo box or list box. This member is not used for a menu."—it's not a handle, but it might be enough information to identify the originating control in a generic way that you could be comfortable adding to ContainerView-in-general rather than a hack for your specific case. You'll have to search for what it actually means and how to use it to retrieve the actual control.

You're right, it is possible to use CtlID to get originating control. :)

Here's how:

ContainerView>>wmMeasureItem: message wParam: wParam lParam: lParam
| struct control itemExtent |
struct := MEASUREITEMSTRUCT fromAddress: lParam.
control := nil.
self subViewsDo:
[:subView |
| contrlId |
controlId := UserLibrary default getDlgCtrlID: subView handle.
controlId = struct CtlID ifTrue: [control := subView]].
control
ifNotNil:
[itemExtent := control measureItem: struct.
struct
itemWidth: itemExtent x;
itemHeight: itemExtent y].
^TRUE

UserLibrary>>getDlgCtrlID: aWindowHandle
"Retrieves the identifier of the specified control.

int GetDlgCtrlID(
[in] HWND hWnd
);"

<stdcall: sdword GetDlgCtrlID handle>
^self invalidCall: _failureCode
Reply all
Reply to author
Forward
0 new messages