bad allocation

[Perry Manole]

I am using this function to convert PDF to XOD. Why am I getting a PDFNetException {"bad allocation"}  on line "intBytesRead = bjReader.Read(bytaBuffer) but only on some PDFs. Works fine on simple small PDFs. I am not running out of memory. This one file is 6Megs but has complex blue print markup.

 

    Public Function PdfToXod(bytaPdfBytes As Byte(), ByRef bytaXodBytes As Byte()) As Integer

        Try
            pdftron.PDFNet.Initialize()
            Using objPdfDoc As New pdftron.PDF.PDFDoc(bytaPdfBytes, bytaPdfBytes.Length)
                Dim objFilter As pdftron.Filters.Filter = pdftron.PDF.Convert.ToXod(objPdfDoc)
                Dim objReader As New pdftron.Filters.FilterReader(objFilter)

 

                Dim objXodBytes As New List(Of Byte)
                Dim bytaBuffer(30 * 1024) As Byte  'read 30 KB at a time
                Dim intBytesRead As Integer
                Dim blnAllBytesRead As Boolean = False
                Do
                    intBytesRead = objReader.Read(bytaBuffer)
                    Select Case True
                        Case intBytesRead = bytaBuffer.Length
                            'buffer was fully filled
                            objXodBytes.AddRange(bytaBuffer)
                        Case Else
                            If intBytesRead > 0 Then
                                'buffer was partialy filled
                                ReDim Preserve bytaBuffer(intBytesRead - 1)
                                objXodBytes.AddRange(bytaBuffer)
                            End If
                            blnAllBytesRead = True
                    End Select
                Loop Until blnAllBytesRead
                bytaXodBytes = objXodBytes.ToArray
            End Using

            Return 0
        Catch ex As Exception
            Return 1
        End Try

    End Function

[Matt Parizeau]

I'm having trouble reproducing this issue from your code with the latest PDFNet version.  Which PDFNet version are you using and is it for .NET, C++, etc; 32 or 64 bit?

If possible could you send me a PDF file that triggers the exception.  If you don't want to post it here then you can email it to mpar...@pdftron.com.

Matt Parizeau

Software Developer

PDFTron Systems Inc.

On Tuesday, March 11, 2014 12:47:38 AM UTC-7, Perry Manole wrote:

I am using this function to convert PDF to XOD. Why am I getting a PDFNetException {"bad allocation"}  on line "intBytesRead = bjReader.Read(bytaBuffer) but only on some PDFs. Works fine on simple small PDFs. I am not running out of memory. This one file is 6Megs but has complex blue print markup.

 

    Public Function PdfToXod(bytaPdfBytes As Byte(), ByRef bytaXodBytes As Byte()) As Integer

        Try
            pdftron.PDFNet.Initialize()
            Dim objPdfDoc As New pdftron.PDF.PDFDoc(bytaPdfBytes, bytaPdfBytes.Length)

            Dim objFilter As pdftron.Filters.Filter = pdftron.PDF.Convert.ToXod(objPdfDoc)
            Dim objReader As New pdftron.Filters.FilterReader(objFilter)

            Dim objXodBytes As New List(Of Byte)
            Dim bytaBuffer(30 * 1024) As Byte  'read 30 KB at a time
            Dim intBytesRead As Integer
            Dim blnAllBytesRead As Boolean = False
            Do
                intBytesRead = objReader.Read(bytaBuffer)
                Select Case True
                    Case intBytesRead = bytaBuffer.Length
                        'buffer was fully filled
                        objXodBytes.AddRange(bytaBuffer)
                    Case Else
                        If intBytesRead > 0 Then
                            'buffer was partialy filled
                            ReDim Preserve bytaBuffer(intBytesRead - 1)
                            objXodBytes.AddRange(bytaBuffer)
                        End If
                        blnAllBytesRead = True
                End Select
            Loop Until blnAllBytesRead
            bytaXodBytes = objXodBytes.ToArray

[Perry Manole]

I am using the 32bit for .NET 4.0 version. Did more testing and it looks like there is one PDF that is 61 Meg causing the exception during the conversion and then any following PDF conversion also throws the same exception except for very small PDFs. So I wonder if its got something to do with memory. Checked the server's memory and it still had plenty of headroom. Only if I restart the services on the web server doing the conversion does the error go away. I emailed you a link to download the problematic PDF file.

[Matt Parizeau]

I'm still having trouble reproducing this.  I used .NET 4.0, 32bit, Windows with PDFNet 6.1.2 downloaded from here: http://www.pdftron.com/pdfnet/downloads.html

Are you on Windows, and which version of PDFNet are you using?  Can you try it with the latest version to see if the issue still exists.

Matt Parizeau

Software Developer

PDFTron Systems Inc.

[Support]

FYI: Not sure if this is the problem, but you need to close the doc (e.g. objPdfDoc.Dispose()) when you are done with it. In C# you would use dispose pattern:

using (PDFDoc doc = new PDFDoc(....)) {

[Perry Manole]

Yes I am running the same version as you v6.1.2. I am running the code during a page request on IIS 7 in Win 7 32 bit. The page gets the PDF bytes from the database, converts the PDF bytes to XOD bytes, and returns the XOD bytes as a response to that page request. The exception occurs in line "intBytesRead = objReader.Read(bytaBuffer)". Did you try the large 6 Meg  PDF a sent you a link to? Did it convert to XOD for you?

[Perry Manole]

I have implemented the Using pattern for Disposing and still get the error. It occurs intermittently but after it occurs once it reoccurs over and over until I reset the server. What is the "pdftron.Common.PDFNetException" used for? The "bad allocation" is coming from within the PdfNet.dll so you have to look in there and see what condition can cause this error.

[Matt Parizeau]

I tried the 60 MB file that you sent and it converted fine.  The only difference in my code is that I get the bytes of the PDF from System.IO.File.ReadAllBytes.  I'll forward this on to someone who is more familiar with PDFNet Filters.

Matt Parizeau

Software Developer

PDFTron Systems Inc.

[Matt Parizeau]

I discussed this with another developer.  A bad allocation error usually means that the allocation failed because there wasn't available memory.  It's more likely that this will occur on a 32 bit system.

1) Are you running any other applications at the same time on your server?

2) Are you running multiple conversions simultaneously (multiple threads/processes)?

3) What's the peak memory usage while the conversion is happening?

Matt Parizeau

[Perry Manole]

Here is what's happening related to memory:

 

 

No I am not running other conversions at the same time. This was a single clean request with no other to the web server. Several observations:

 

1. Reading the PDF in memory takes relatively very little memory. The conversion to XOD takes slightly more but look at reading from the FilterReader? I don't understand why that process is so intensive if the conversion has already occurred? It's also the part that takes the longest. Three times longer than the conversion and that's without finishing.

 

2. Even if memory was an issue, which it does not look like it, shouldn't the swap file be engaged?

 

3. Why is the memory never reclaimed? Even with the recommended PDFDoc Dispose, every conversion moves the memory usage higher and higher.

 

[Matt Parizeau]

1. When you use ToXod to get a filter it will return the filter before the conversion has completely finished.  You can think of reading from the filter as reading from a stream and the conversion process will be happening at the same time as you are reading.

2. It can be more complex than that.  This article helped me understand things a bit better.

3. I ran some more tests running conversions repeatedly and found that with only disposing the PDFDoc the memory seemed to grow but if I also disposed the filter then the memory seemed to remain fairly stable.  One thing to note is that because of the garbage collector and system not necessarily releasing resources right away it may seem that the memory is growing for some time before eventually returning to a lower value.

Matt Parizeau

Software Developer

PDFTron Systems Inc.

[Perry Manole]

1. Is it normal through for a PDF to XOD conversion to take more than 10 minutes on a single 61Meg file? Seems very slow to me.

2. OK, I read the article. Still don't know what to do to solve this "bad allocation" error I am getting. If its a mapping issue more memory wont help. What will?

I have switched my dev box to 64bit hoping that will help but when I switched to the 64bit version of PDFNet is broke the entire site. After some searches in the forum I found out that for the 64bit to work I have to disable 32bit apps in the IIS app pool but I can't do that because I have other legacy APIs that can only run in 32bit. So now I am back to the 32bit PdfNet SDK with no solution to this error.

[Matt Parizeau]

1. For your document in particular the pages are large, there are special blend modes and there are lots of paths so it's not completely unexpected.  The DPI mentioned in the other thread was probably too large so if you can go lower than that (300 or less) then that would help the performance.

2. 64 bit would help because there is a much larger virtual address space (billions of times larger) so it's much more difficult to fragment.

One thing to note is that I just tried converting on WinXP 32bit VM and still wasn't able to reproduce the issue.

You could try disabling disk caching by calling PDFNet.SetDefaultDiskCachingEnabled(false);

You could also take a look at this article for how to increase process size to 4GB on 32 bit Windows: https://groups.google.com/forum/#!msg/pdfnet-sdk/PFgO038YjCw/f-VRnyc3H1AJ

Matt Parizeau

Software Developer

PDFTron Systems Inc.

[Perry Manole]

Will the fact that I switched to a 64bit OS help even if the IIS pool allows 32 bit apps (required by other APIs) and am forced to use the 32bit version of PDFNet.dll?

[Matt Parizeau]

I'm not entirely sure but I don't think it will be able to give your app more virtual address space.  You can give it a try and see if it makes a difference.  Have any of the changes helped with eliminating the issue?

Matt Parizeau

Software Developer

PDFTron Systems Inc.