[reportlab-users] Tried reportlab on python 3.4... failed

[Glenn Linderman]

Since reportlab was one of a very few reasons I have python 2.7 installed, I have watched with anticipation the development of reportlab for python 3. Today I decided to try it, because I tried to operate on a file with a non-ASCII character in it, and python 2.7 doesn't like those, so rather than rename the file, I installed reportlab in Python 3.4.1 via pip. The install seemed to go fine.

I tweaked the launching of my PDF generator to launch it via Python 3.4.1 instead of python 2.7. Since it has been a while since I looked at that code, I figured I would get a bunch of Python 2->3 sort of errors, but... not yet...

Instead, on the call to mkdoc(), I got a traceback, from mkdoc() down here it is:

 in mkdoc:
    doc.build( elements, onFirstPage=mkpage )
  File "c:\python34\lib\site-packages\reportlab\platypus\doctemplate.py", line 1127, in build
     BaseDocTemplate.build(self,flowables, canvasmaker=canvasmaker)
  File "c:\python34\lib\site-packages\reportlab\platypus\doctemplate.py", line 8 90, in build
    self.handle_flowable(flowables)
  File "c:\python34\lib\site-packages\reportlab\platypus\doctemplate.py", line 773, in handle_flowable
    if frame.add(f, canv, trySplit=self.allowSplitting):
  File "c:\python34\lib\site-packages\reportlab\platypus\frames.py", line 161, in _add
    w, h = flowable.wrap(aW, h)
  File "c:\python34\lib\site-packages\reportlab\platypus\paragraph.py", line 1035, in wrap
    blPara = self.breakLines([first_line_width, later_widths])
  File "c:\python34\lib\site-packages\reportlab\platypus\paragraph.py", line 1203, in breakLines
    splitLongWords = style.splitLongWords
AttributeError: 'MyParaSt' object has no attribute 'splitLongWords'


The definition for MyParaSt is as follows... works fine in reportlab 2.5 on python 2.7....

class MyParaSt(PropertySet):
    defaults = {
        'fontName': 'Times-New-Roman',
        'fontSize': defsize,
        'leading': deflead,
        'leftIndent': 0,
        'rightIndent': 0,
        'firstLineIndent': 0,
        'alignment': TA_LEFT,
        'spaceBefore': 0,
        'spaceAfter': 0,
        'bulletFontName': 'Times-New-Roman',
        'bulletFontSize': 10,
        'bulletIndent': 0,
        'textColor': black,
        'backColor': None,
        'wordWrap': None,
        'borderWidth': 0,
        'borderPadding': 0,
        'borderColor': None,
        'borderRadius': None,
        'allowWidows': 1,
        'allowOrphans': 0,
        'textTransform': None,
            #uppercase lowercase (captitalize not yet) or None or absent
        }

I'm not exactly sure why I have a MyParaSt, since it has been a long time since I wrote this, but I would guess it might have been to set the default font(s)?  Maybe there would have been a better way, like only changing the items I wanted to change, rather than creating my whole new own defaults hash? I probably cut-n-pasted this idea from somewhere, though, and maybe it wasn't the best idea?

Where should I start reading? I did do a search and it seems splitLongWords is relatively new...

[Glenn Linderman]

So I found the latest user guide, and in chapter six it shows something just like this:

class ParagraphStyle(PropertySet):
defaults = {
'fontName':'Times-Roman',
'fontSize':10,
'leading':12,

'leftIndent':0,
'rightIndent':0,
'firstLineIndent':0,
'alignment':TA_LEFT,
'spaceBefore':0,
'spaceAfter':0,

'bulletFontName':'Times-Roman',

'bulletFontSize':10,
'bulletIndent':0,
'textColor': black,
'backColor':None,
'wordWrap':None,
'borderWidth': 0,
'borderPadding': 0,
'borderColor': None,
'borderRadius': None,
'allowWidows': 1,
'allowOrphans': 0,
}

(indentation didn't survive the paste from the PDF file). And there is no sign of splitLongWords here, either... but I see that the things I might have changed were the fontSize & leading. Not sure where I got 'textTransform': None, or the uppercase comment, but... no doubt somewhere...

Anyway, in the spirit of "programming by guessing", I added

'splitLongWords': 0,

to my definition.  It got further, I guess, and complained similarly that endDots was not defined, so without additional research, I added

'endDots': 0,

and then the program ran. So it seems that these items should either be added to the example in the user guide, since they seem to be necessary, or perhaps their necessity should be removed by checking their existence before referencing them, and using some default if they do not exist.

[Andy Robinson]

Hi Glenn,

As far as we know the code behaves identically in Pythons 2.7, 3.3 and
3.4; certainly the user guide generates and all the tests run.

It looks like we managed to document a very, very old pattern in the
user guide and to give a completely misleading explanation of how to
set up the styles, and somehow it has survived about 13-14 years. My
apologies for this. We're warming up for a release in November and
need to make a pass through the whole user guide.

If we were redoing things now we would use a metaclass-like pattern
similar to Django's models or forms, but that didn't exist in Python
1.4 or whatever we were on at the time; reportlab/lib/styles has our
own handrolled version of this, designed for speed rather than
elegance. I'm not sure why we showed an example like this.

The intended use is for you to do this:

mystyle = ParagraphStyle(key1=value1, key2=value2, etc...)

...and just keep a style instance around. I don't think you would
want to create your own subclass for anything I can think of. We
support inheritance by allowing a parent attribute e.g.

styBase = ParagraphStyle(fontName="myfont", fontSize=10)
styBold = ParagraphStyle(parent=styBase, fontName="myBoldFont")

..

That way, all paragraph styles will be initialised with every
attribute currently supported.

- Andy

> _______________________________________________
> reportlab-users mailing list
> reportl...@lists2.reportlab.com
> https://pairlist2.pair.net/mailman/listinfo/reportlab-users
>



--
Andy Robinson
Managing Director
ReportLab Europe Ltd.
Thornton House, Thornton Road, Wimbledon, London SW19 4NG, UK
Tel +44-20-8405-6420
_______________________________________________
reportlab-users mailing list
reportl...@lists2.reportlab.com
https://pairlist2.pair.net/mailman/listinfo/reportlab-users

[Glenn Linderman]

On 10/25/2014 7:16 PM, Andy Robinson wrote:

Hi Glenn,

As far as we know the code behaves identically in Pythons 2.7, 3.3 and
3.4; certainly the user guide generates and all the tests run.

Thanks, Andy, your suggestions certainly look more practical than the user guide code, so I reworked  the code in the manner you suggest (although your styBase omits the name parameter, in case someone else uses this as a reference), and got it to work, no longer needing to deal with endDot or splitLongWords... or quite a few others.

However, in running the new code on an old file, it seems there may have been a change to the way image sizes are determined to fit or not, within a frame.

I don't recall how I figured out what image size (maybe from the error message I got then) was maximum when I set this up, but I was trying to fill the frame, and did... but now I get an error that the image is too big, but the image hasn't changed, nor has the frame size, unless this altered technique somehow changes it.

reportlab.platypus.doctemplate.LayoutError: Flowable <Image at 0x32f0b70 frame=normal filename=...\front.jpg>(197.76 x 197.76) too large on page 2 in frame 'normal'(186.0 x 186.0*) of template 'Later'

To obtain another data point, I used the new code, but the old python 2.7 and reportlab 2.5, and the image size was accepted. So it may be that the _current_ code behaves identically on Python 2.7, 3.3, and 3.4 (I don't have all the combinations installed), but the current reportlab code on Python 3.4 seems to behave differently than reportlab 2.5 on Python 2.7 regarding image and frame sizes.

The release notes on your site only go back to 2.7, so I can't say if 2.6 specifically called out changes in this area, but I would assume that "minor" fixes not completely described in the release notes that are on the site wouldn't include significant changes to the image and frame size calculations.

I do note the switch from PIL to PILLOW but have no idea if that would affect the sizes: since both PIL and PILLOW deal in bitmaps, I would doubt it.

Any clues?

Here are some more details: 

The actual image size is 412x412, and the resolution in the jpg file is 300dpi.
The page size (small booklet) is 3.25"x3.25", with .25" margins all around.

Here is the code the positions the image on the page:

        res = 150
        page.do( False )
        im = Image( img, theCovWid/res*inch, theCovHei/res*inch )
        im.hAlign = 'CENTER'
        im.vAlign = 'CENTER'
        elements.append( im )
        page.do( True )

theCovWid and theCovHei are both 412.

When I calculate 3.25" - 2*.25 * 150, I get 412.5 which is only slightly larger than 412, and it works in rl2.5/Py2.7.

I have no idea where 197.76 or 186.0 come from.  Maybe that is in mm?  Hmm. Nope. 2.75" == 69.85mm. Ah, maybe points... 2.75" = 198pt. Unless you use a point size other than 72/inch. Anyway this is pretty close. Assuming 186.0 is in the same units... that would be 2.58333"... the difference being about 12 points or 6 points a side if it is some sort of padding or margin, which seems likely, since it has affected both height and width calculations.

The documented defaults for all those sorts of things seem to be zero, rather than 6pts, however.

Glenn

[Marius Gedminas]

I think you tripped the Python 3 change to the / operator:

$ python2
>>> 412/150
2

$ python3
>>> 412/150
2.7466666666666666

If you want to round down like the code used to in Python 2, use the //
operator:

$ python3
>>> 412//150
2

i.e.
im = Image( img, theCovWid//res*inch, theCovHei//res*inch )

(which seems to be a bug in your old code, that went unnoticed for a
long time -- you're effectively using a resolution of 206 dpi instead of
150 dpi -- but if you use the right resolution, your image doesn't fit!)

> im.hAlign = 'CENTER'
> im.vAlign = 'CENTER'
> elements.append( im )
> page.do( True )
>
> theCovWid and theCovHei are both 412.
>
> When I calculate 3.25" - 2*.25 * 150, I get 412.5 which is only slightly
> larger than 412, and it works in rl2.5/Py2.7.
>
> I have no idea where 197.76 or 186.0 come from.

So assuming theCovWin == 412, res == 150, inch == 72

On Python 3 you get 412/150*72 == 197.76, which doesn't fit in the frame
that is 186 points wide.

On Python 2 you get 412/150*72 == 144, which fits in the 186-point
frame.

> Maybe that is in mm? Hmm.
> Nope. 2.75" == 69.85mm. Ah, maybe points... 2.75" = 198pt. Unless you use a
> point size other than 72/inch.

PDF points are 1/72 inch, yes.

> Anyway this is pretty close. Assuming 186.0
> is in the same units... that would be 2.58333"... the difference being about
> 12 points or 6 points a side if it is some sort of padding or margin, which
> seems likely, since it has affected both height and width calculations.
>
> The documented defaults for all those sorts of things seem to be zero,
> rather than 6pts, however.

HTH,
Marius Gedminas
--
Jim's Three Laws of Engineering:
1. F = ma
2. You can't solve a problem unless you know the answer
3. You can't push a rope

[Glenn Linderman]

Thanks, Marius, this explains what is actually happening, no doubt. Your numbers all add up. One remaining confusion on my part, though, is where does the 186 point frame come from?

( 3.25 - 2*.25 )in * 72pt/in = 198pt

So while I clearly didn't do the calculation correctly (in Py2 I should have done theCovWid*inch/res for more accuracy, that results in 197 on Py2 and 197.76 on Py3 due to the / change), either of those numbers seems to be smaller than, but closer to, the 198pt frame size I thought I was targeting (but I was thinking in inches and pixels rather than points). So is there some implicit padding or margin that produces the 186pt frame size mentioned in the error message? Sans my inaccurate calculation and failure to actually measure the results with a ruler, I would have asked about this when I wrote the Py2 version of my program! But that being my first reportlab program (and few others since, since it required the use of Py2, which is awkward for me, having chosen Py3 for my first forays into Python, I might use it more now that it works with Py3) I was pretty vague on how the calculations were done or what they meant, but the one I used fit my (apparently faulty, even still) mental model of how the sizes worked.

Your message, of course, leads me directly to a solution for producing the same output as I was before, but doesn't fully explain where the 186pt frame size comes from.

[Glenn Linderman]

On 10/27/2014 12:27 PM, Glenn Linderman wrote:

On 10/26/2014 11:18 AM, Marius Gedminas wrote:

On Sat, Oct 25, 2014 at 11:07:39PM -0700, Glenn Linderman wrote:

On 10/25/2014 7:16 PM, Andy Robinson wrote:

On Python 2 you get 412/150*72 == 144, which fits in the 186-point frame.

And 144pt == 2in, and the image I was actually printing came out 2in wide, which was not what I wanted, but I didn't measure it until now.

Maybe that is in mm?  Hmm.
Nope. 2.75" == 69.85mm. Ah, maybe points... 2.75" = 198pt. Unless you use a
point size other than 72/inch.

PDF points are 1/72 inch, yes.

Anyway this is pretty close. Assuming 186.0
is in the same units... that would be 2.58333"... the difference being about
12 points or 6 points a side if it is some sort of padding or margin, which
seems likely, since it has affected both height and width calculations.

OK, it seems that 6pt is the default padding for a Frame, and SimpleDocTemplate must create Frames with default padding... so that explains the 186pt limit reported in the error message.

There doesn't seem to be a way to override that... except by using BaseDocTemplate... or by making the margins 6pt smaller to compensate for the Frame padding.

OK, now my Python 3 version of the code produces what I really wanted in the first place, and I've learned more, but perhaps if the user guide is redone, some of these issues might be mentioned and explained, rather than being surprises or discoveries.