[reportlab-users] Possible bug?

[Les]

  Hello,

I have found an error, and I created a minimal working example. The minimal working example starts with the very first example from Platypus user guide:

from reportlab.platypus import SimpleDocTemplate, Paragraph, Spacer
from reportlab.lib.styles import getSampleStyleSheet
from reportlab.lib.pagesizes import A4
from reportlab.lib.units import inch

PAGE_HEIGHT = A4[1]
PAGE_WIDTH = A4[0]
styles = getSampleStyleSheet()

Title = "Hello world"
pageinfo = "platypus example"


def myFirstPage(canvas, doc):
    canvas.saveState()
    canvas.setFont('Times-Bold', 16)
    canvas.drawCentredString(PAGE_WIDTH / 2.0, PAGE_HEIGHT - 108, Title)
    canvas.setFont('Times-Roman', 9)
    canvas.drawString(inch, 0.75 * inch, "First Page / %s" % pageinfo)
    canvas.restoreState()


def myLaterPages(canvas, doc):
    canvas.saveState()
    canvas.setFont('Times-Roman', 9)
    canvas.drawString(inch, 0.75 * inch, "Page %d %s" % (doc.page, pageinfo))
    canvas.restoreState()


def go():
    Story = [Spacer(1, 2 * inch)]
    style = styles["Normal"]
    for i in range(100):
        bogustext = ("This is Paragraph number %s. " % i) * 20
        p = Paragraph(bogustext, style)
        Story.append(p)
        Story.append(Spacer(1, 0.2 * inch))
    doc = SimpleDocTemplate("phello.pdf")
    doc.build(Story, onFirstPage=myFirstPage, onLaterPages=myLaterPages)


go()

If I change it to this (e.g. generate two identical files):

doc = SimpleDocTemplate("phello.pdf")
doc.build(Story, onFirstPage=myFirstPage, onLaterPages=myLaterPages)
doc = SimpleDocTemplate("phello2.pdf")
doc.build(Story, onFirstPage=myFirstPage, onLaterPages=myLaterPages)

then it builds phello.pdf correctly, but builds a totally empty phello2.pdf (960 bytes, a single white empty page).

It is hard to explain as it is, but something even more interesting happens if you try to make them totally independent, and create a copy of the story as well:

import copy
doc = SimpleDocTemplate("phello.pdf")
doc.build(copy.copy(Story), onFirstPage=myFirstPage, onLaterPages=myLaterPages)
doc = SimpleDocTemplate("phello2.pdf")
doc.build(copy.copy(Story), onFirstPage=myFirstPage, onLaterPages=myLaterPages)

This will render phello.pdf correctly, and it will throw this error when rendering phello2.pdf:

Traceback (most recent call last):
  File "C:\Projects\test\test2.py", line 48, in <module>
    go()
  File "C:\Projects\test\test2.py", line 45, in go
    doc.build(copy.copy(Story), onFirstPage=myFirstPage, onLaterPages=myLaterPages)
  File "C:\Users\nagyl\.virtualenvs\test-NC9-O-tN\lib\site-packages\reportlab\platypus\doctemplate.py", line 1314, in build
    BaseDocTemplate.build(self,flowables, canvasmaker=canvasmaker)
  File "C:\Users\nagyl\.virtualenvs\ test-NC9-O-tN\lib\site-packages\reportlab\platypus\doctemplate.py", line 1079, in build
    self.handle_flowable(flowables)
  File "C:\Users\nagyl\.virtualenvs\ test-NC9-O-tN\lib\site-packages\reportlab\platypus\doctemplate.py", line 958, in handle_flowable
    raise LayoutError(ident)
reportlab.platypus.doctemplate.LayoutError: Flowable <Paragraph at 0x148e102cb80 frame=normal>This is Paragraph number 6. This is Paragraph number 6. This(439.27559055118115 x 72) too large on page 1 in frame 'normal'(439.27559055118115 x 685.8897637795277) of template 'First'

And finally, here is the "solution" that solves all problems:

def go():
    def create_story():
        Story = [Spacer(1, 2 * inch)]
        style = styles["Normal"]
        for i in range(100):
            bogustext = ("This is Paragraph number %s. " % i) * 20
            p = Paragraph(bogustext, style)
            Story.append(p)
            Story.append(Spacer(1, 0.2 * inch))
        return Story

    doc = SimpleDocTemplate("phello.pdf")
    doc.build(create_story(), onFirstPage=myFirstPage, onLaterPages=myLaterPages)
    doc = SimpleDocTemplate("phello2.pdf")
    doc.build(create_story(), onFirstPage=myFirstPage, onLaterPages=myLaterPages)

This does not throw an error, and it renders both files correctly. It also works if I do copy.deepcopy instead of copy.copy

I was looking for an explanation in the user guide. I was looking for a note, telling me that a story can be used for document generation only once. But there is no such thing in the docs. Or maybe I just did not find it. Can somebody please explain what is happening here?

Environment details: Python 3.10.1 amd64 on Windows, reportlab 3.6.6

Note: it was also discussed here https://mail.python.org/pipermail/python-list/2022-March/905724.html due to a temporary problem with mailing list subscription.

Thanks,

    Laszlo

[Robin Becker]

On 15/03/2022 13:28, Les wrote:
> Hello,
>
> I have found an error, and I created a minimal working example. The minimal
> working example starts with the very first example from Platypus user guide:
>

.........

Hi Les,

glad you found the list. I don't believe this is a real bug. The DocTemplate class which handles the story processing
etc etc consumes the story which as it's a list means that the list becomes empty.

1) We need to be able to manipulate the list to support many processes with the story layout eg we might need to split
an element that cannot fit and the split elements are pushed back onto the list.

2) there is no guarantee that any specific flowable is immutable; to support repeated processing we make a shallow copy
and that works for many cases, but it's not guaranteed if the flowables are perverse. Shallow and deep copies may allow
duplicated usage, but obviously they consume resources.
--
Robin Becker
_______________________________________________
reportlab-users mailing list
reportl...@lists2.reportlab.com
https://pairlist2.pair.net/mailman/listinfo/reportlab-users

[Les]

2) there is no guarantee that any specific flowable is immutable; to support repeated processing we make a shallow copy
and that works for many cases, but it's not guaranteed if the flowables are perverse. Shallow and deep copies may allow
duplicated usage, but obviously they consume resources.

Hello Robin,

I understand the explanation. Just one more question. Nothing is perverse in my minimal working example, it only contains simple paragraphs. You also say that generally it is disallowed to use shallow copies for repeated document generation.  

However, BaseDocTemplate.multiBuild also uses a shallow copy:

https://github.com/Distrotech/reportlab/blob/master/src/reportlab/platypus/doctemplate.py#L1042

# work with a copy of the story, since it is consumed
tempStory = story[:]
self.build(tempStory, **buildKwds)

I'm curious - why is it allowed here, and not allowed elsewhere?

Regards,

   Laszlo

[Robin Becker]

>.......
2) explicitly says "if the flowables are perverse". Nobody is disallowing use of shallow copies, but they are not
guaranteed. If flowables lacked internal state things would work more easily. Python allows foot shooting :(

The life cycle of flowables ought to be more complex than just wrap & draw, but it's too late now.