Scintilla on Windows with a lot of text

[Mike Lischke]

Hi list,

currently I'm struggling with a problem when loading a big file into scintilla on Windows. The same file loads fine on OSX.

The problem on Windows is that an OutOfMemory exception is raised when setting the scrollbar after loading. The reason is that the .NET control that wraps Scintilla tries to get the window text (apparently for layout), which causes the usual native <-> managed crossing with all the additional memory allocations. The file is 130MB in size, the app uses around 750MB at this moment. The stack trace is:

Exception = System.OutOfMemoryException

Message = Insufficient memory to continue the execution of the program.

FullText = System.OutOfMemoryException: Insufficient memory to continue the execution of the program.

   at System.Runtime.InteropServices.Marshal.AllocCoTaskMem(Int32 cb)

   at System.Windows.Forms.UnsafeNativeMethods.GetWindowText(HandleRef hWnd, StringBuilder lpString, Int32 nMaxCount)

   at System.Windows.Forms.Control.get_WindowText()

   at System.Windows.Forms.Control.set_CacheTextInternal(Boolean value)

   at System.Windows.Forms.Control.PerformLayout(LayoutEventArgs args)

   at System.Windows.Forms.Control.System.Windows.Forms.Layout.IArrangedElement.PerformLayout(IArrangedElement affectedElement, String affectedProperty)

   at System.Windows.Forms.Control.OnResize(EventArgs e)

   at System.Windows.Forms.Control.OnSizeChanged(EventArgs e)

   at System.Windows.Forms.Control.UpdateBounds(Int32 x, Int32 y, Int32 width, Int32 height, Int32 clientWidth, Int32 clientHeight)

   at System.Windows.Forms.Control.UpdateBounds()

   at System.Windows.Forms.Control.WmWindowPosChanged(Message& m)

   at System.Windows.Forms.Control.WndProc(Message& m)

   at MySQL.Forms.ScintillaControl.WndProc(Message& m) in d:\work\mysql\workbench\5.2\First-chance exception at 0x7662b9bc in MySQLWorkbench.exe: Microsoft C++ exception: std::bad_alloc at memory location 0x0061b748..

Since Scintilla is an editor control the actual window text isn't interesting for this layout process, so to solve this problem one solution could be to return not the full text of the editor as window text but some dummy text. Though I don't see how this could be managed as GetWindowText still uses WM_GETTEXT, which is also used to get the editor content for normal processing. Another option would be to somehow override Control.get_WindowText to provide a simpler implementation. But unfortunately this is a private API. Yet another idea is to return dummy text for WM_GETTEXT while Scintilla is handling the WM_WINDOWPOSCHANGED message (tho that might have side effects and also sounds a bit hack-ish).

What else could we do to solve this? It would be a real pity to stop fixing a long standing problem we had just because the final SetScrollInfo (+ depending processing) fails, since everything else works now nicely. If I jump over SetScrollInfo no problem arises, except that the scrollbar isn't set properly of course (and it crashes when it is adjusted next time, say when collapsing a text part). But that's just to show how close we are.

Thankful for any helpful idea,

Mike
-- 
www.soft-gems.net

[Neil Hodgson]

Mike Lischke:

> Since Scintilla is an editor control the actual window text isn't interesting for this layout process, so to solve this problem one solution could be to return not the full text of the editor as window text but some dummy text. Though I don't see how this could be managed as GetWindowText still uses WM_GETTEXT, which is also used to get the editor content for normal processing.

Its WM_GETTEXTLENGTH that you want to override (and return 0) as it is the result from that that will be used to allocate the memory. The way to override a windows message is to subclass Scintilla and there is an example of this in SciTE: grep for "subclassed". .NET probably makes it even easier. When retrieving data with your own code, use SCI_GETTEXTLENGTH instead of WM_GETTEXTLENGTH.

Neil

[Mike Lischke]

Hey Neil,

thanks for the quick answer.

>> Since Scintilla is an editor control the actual window text isn't interesting for this layout process, so to solve this problem one solution could be to return not the full text of the editor as window text but some dummy text. Though I don't see how this could be managed as GetWindowText still uses WM_GETTEXT, which is also used to get the editor content for normal processing.
>
> Its WM_GETTEXTLENGTH that you want to override (and return 0) as it is the result from that that will be used to allocate the memory.

Hmm, interesting. I thought this would have a bad side effect for normal text retrieval.

> The way to override a windows message is to subclass Scintilla and there is an example of this in SciTE: grep for "subclassed". .NET probably makes it even easier.

Yes, I know.

> When retrieving data with your own code, use SCI_GETTEXTLENGTH instead of WM_GETTEXTLENGTH.

Great, thanks for such a simple but effective idea. It works like a charm. I love it. Scintilla is truly a gem.

Mike
--
www.soft-gems.net