Issue 669 in pdfium: Memory leak when rendering pages.

[matt.cur… via monorail]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 669 by matt.cur...@gmail.com: Memory leak when rendering pages.
https://bugs.chromium.org/p/pdfium/issues/detail?id=669

What steps will reproduce the problem?
1. Load the attached PDF into PDFium (Chrome is an easy way to test.)
2. Open Chrome's task manager and identify the starting memory level of the PDF process.
3. Scroll from the top of the PDF all the way to the bottom at a reasonable rate
4. Watch as memory usage increases and does not decrease, even after pages have fallen out of view. Memory is only released when the document is released, not before. This leak also only appears when rendering pages, simply loading a page via FPDF_LoadPage() does not cause the leak.

You can also test this in PDFium itself using the following code - it's in Objective-C, but the idea is pretty much the same in plain C:

----
FPDF_InitLibrary();

NSURL *pdfURL = [[NSBundle mainBundle] URLForResource:@"pdf_reference-1.7" withExtension:@"pdf"];

FPDF_DOCUMENT document = FPDF_LoadDocument(pdfURL.path.UTF8String, nil);

int pageCount = FPDF_GetPageCount(document);

for(int i = 0; i < pageCount; i++){
FPDF_PAGE page = FPDF_LoadPage(document, i);

FPDF_BITMAP bitmap = FPDFBitmap_Create(500, 500, 1);

FPDF_RenderPageBitmap(bitmap, page, 0, 0, 500, 500, 0, 0);

FPDFBitmap_Destroy(bitmap);

FPDF_ClosePage(page);
}
----

What is the expected output? What do you see instead?
Memory allocated by rendering should *not* remain allocated after the page is closed.

What version of the product are you using? On what operating system?
OSX 10.11.6, Chrome 56.0.2924.87. Also tested with latest PDFium commit 5f2d381028b2c09bd09a843368328155b1399c5e.

Please provide any additional information below.
...

Attachments:
pdf_reference_1-7.pdf 17.5 MB

--
You received this message because:
1. The project was configured to send all issue notifications to this address

You may adjust your notification preferences at:
https://bugs.chromium.org/hosting/settings

[matt.cur… via monorail]

Comment #1 on issue 669 by matt.cur...@gmail.com: Memory leak when rendering pages.
https://bugs.chromium.org/p/pdfium/issues/detail?id=669#c1

Side note: this happens with any PDF with as many pages as needed to make the memory leak noticeable, but happens with all I've tested.

[n… via monorail]

Comment #2 on issue 669 by n...@chromium.org: Memory leak when rendering pages.
https://bugs.chromium.org/p/pdfium/issues/detail?id=669#c2

I can't reproduce what you said in Chrome... the point of FPDFBitmap_Destroy is to destroy the bitmap used to render the page, so I'm confused about what problem you're encountering.

[matt.cur… via monorail]

Comment #3 on issue 669 by matt.cur...@gmail.com: Memory leak when rendering pages.
https://bugs.chromium.org/p/pdfium/issues/detail?id=669#c3

The problem I'm encountering is that I'm seeing that whenever I use FPDF_RenderPageBitmap/_Start, memory is allocated and not released when the bitmap is destroyed and the page closed.

I just tested this again with the attached PDF in Chrome. After the PDF finishes loading and I scroll from top to bottom at medium pace, memory used by the Chrome PDF Viewer process jumps from 80 MB at the start to ~110 MB by the end. I'm also seeing similar results with the code I posted above. What are you seeing?

[matt.cur… via monorail]

Comment #4 on issue 669 by matt.cur...@gmail.com: Memory leak when rendering pages.
https://bugs.chromium.org/p/pdfium/issues/detail?id=669#c4

Did some more testing. Even if you comment out the lines that do any rendering, and only load/'close' pages, memory usage still climbs and stays that way until the document is closed. You don't have to do any rendering to see this leak.

[n… via monorail]

Updates:
Status: Accepted

Comment #5 on issue 669 by n...@chromium.org: Memory leak when rendering pages.
https://bugs.chromium.org/p/pdfium/issues/detail?id=669#c5

Ok, it makes sense that the bitmap is not the problem. I tried again today, with more patience, and can reproduce this. I'll investigate, but one thing to note is that there are some objects in PDFs that may be used by multiple pages, so destroying absolutely everything when unloading the page may not be a great solution because it could slow down page loading.

[n… via monorail]

Comment #6 on issue 669 by n...@chromium.org: Memory leak when rendering pages.
https://bugs.chromium.org/p/pdfium/issues/detail?id=669#c6

It seems that the increasing memory usage is due to the document storing a bunch of dictionaries and streams that are processed the first time they are needed. Removing these from the document would mean having to load them every time they are used, and this would be bad in some cases (for example, consider fonts used in every page of the PDF). So it doesn't seem like there is much we can do at the moment.

[matt.cur… via monorail]

Comment #7 on issue 669 by matt.cur...@gmail.com: Memory leak when rendering pages.
https://bugs.chromium.org/p/pdfium/issues/detail?id=669#c7

How do you determine which dictionaries and streams need to stick around? At the least, would it be possible to provide an API by which to purge the cache?

[n… via monorail]

Comment #8 on issue 669 by n...@chromium.org: Memory leak when rendering pages.
https://bugs.chromium.org/p/pdfium/issues/detail?id=669#c8

I think that the ones that can be reused by pages are stored in the document. Why do you need this API? I would have to do an experiment to see if it is really worth it to have them stored in the document instead of the page.

[matt.cu...@gmail.com]

Because as things stand this bug means that having more than one document (or just one with many pages) open creates more and more of a memory issue over time, with no way to control the increase in memory use outside of closing the document. This makes it harder to use PDFium in a memory-restricted environment.

In my own application, I may want to keep a document open even if it is not immediately visible. If I want to keep memory usage low, I'd have to close and reopen documents and any pages, which seems a little unnecessary and potentially intensive. In many apps only one or two pages are ever open at any given time, but nonetheless the memory allocated to objects from other pages will stay in memory. I think it makes sense to give users a way to purge the cache if needed, or some other way to control/optimize the memory allocated.

[matt.cur… via monorail]

Comment #9 on issue 669 by matt.cur...@gmail.com: Memory leak when rendering pages.
https://bugs.chromium.org/p/pdfium/issues/detail?id=669#c9

[thes… via monorail]

Comment #10 on issue 669 by the...@chromium.org: Memory leak when rendering pages.
https://bugs.chromium.org/p/pdfium/issues/detail?id=669#c10

Can't you just call FPDF_CloseDocument() and reopen it again to simulate purging the cache?

[matt.cur… via monorail]

Comment #11 on issue 669 by matt.cur...@gmail.com: Memory leak when rendering pages.
https://bugs.chromium.org/p/pdfium/issues/detail?id=669#c11

I could, but that would mean having to close and reopen any other pages/text/data structures I have open, every time.

[matt.cur… via monorail]

Comment #12 on issue 669 by matt.cur...@gmail.com: Memory leak when rendering pages.
https://bugs.chromium.org/p/pdfium/issues/detail?id=669#c12

Granted, I could use a separate document... but it still feels clunky.