[reportlab-users] deprecation warning

[Jürgen Gmach]

Hi,

just updated to the latest reportalb version and now I get the following deprecation warning on Python 3.8

reportlab-3.5.65-py3.8-linux-x86_64.egg/reportlab/platypus/paraparser.py:54: DeprecationWarning: invalid escape sequence \s

    _re_unit = re.compile('^\s*(.*)(i|in|cm|mm|pt|pica)\s*$'),

Thanks for all your effort!

[Christoph Zwerschke]

On 19.03.2021 10:15, Jürgen Gmach wrote:
> just updated to the latest reportalb version and now I get the
> following deprecation warning on Python 3.8 >

> DeprecationWarning: invalid escape sequence \s
> _re_unit = re.compile('^\s*(.*)(i|in|cm|mm|pt|pica)\s*$'),

The warning is shown because this should be either a raw string or the
"\s" needs double escaping.

-- Christoph
_______________________________________________
reportlab-users mailing list
reportl...@lists2.reportlab.com
https://pairlist2.pair.net/mailman/listinfo/reportlab-users

[Robin Becker]

Thanks Jürgen,

I don't know how many tests I have to run, but I don't see this warning in 2.7.18, 3.7.5, 3.8.6, 3.9.2 & 3.10.0a6. It
doesn't show in the various github action / appveyor wheel builds.

Jürgen can you say what OS, exact version of python 3.8 and what command line options if any were used?

I will put this into the latest release.

--
Robin Becker

[Christoph Zwerschke]

On 19.03.2021 10:45, Robin Becker wrote:
> I don't know how many tests I have to run, but I don't see this
> warning in 2.7.18, 3.7.5, 3.8.6, 3.9.2 & 3.10.0a6. It doesn't show in
> the various github action / appveyor wheel builds.
>
> Jürgen can you say what OS, exact version of python 3.8 and what
> command line options if any were used?
>
> I will put this into the latest release.

Robin, this warning will certainly only show up since Python 3.8.

The docs for the re module say since Python 3.8: "Please note that any
invalid escape sequences in Python’s usage of the backslash in string
literals now generate a DeprecationWarning and in the future this will
become a SyntaxError. This behaviour will happen even if it is a valid
escape sequence for a regular expression."

So it's important to fix this.

If you run flake8 on the source, you should find all such instances of
invalid escape sequences (reported as warning W605).

[Robin Becker]

On 19/03/2021 10:21, Christoph Zwerschke wrote:
> On 19.03.2021 10:45, Robin Becker wrote:
> > I don't know how many tests I have to run, but I don't see this
> > warning in 2.7.18, 3.7.5, 3.8.6, 3.9.2 & 3.10.0a6. It doesn't show in
> > the various github action / appveyor wheel builds.
> >
> > Jürgen can you say what OS, exact version of python 3.8 and what
> > command line options if any were used?
> >
> > I will put this into the latest release.
>
> Robin, this warning will certainly only show up since Python 3.8.
>

How strange, I run all the tests in many pythons including 3.8.6 etc etc and don't see this warning; I'm just wondering
if I need to excite it somehow. I run on linux mostly, but the appveyor is windows and the github actions include OS X.

I've made the regex raw, but in my python 3.8 it worked as expected eg

> $ python
> Python 3.8.6 (default, Dec 4 2020, 10:19:20)
> [GCC 10.2.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import re

>>>> _re_unit = re.compile('^\s*(.*)(i|in|cm|mm|pt|pica)\s*$')

>>>> _re_unit

> re.compile('^\\s*(.*)(i|in|cm|mm|pt|pica)\\s*$')
>>>>

which is a bit odd since I don't see how the regex even sees the \s as double escaped; I assume some python magic is
delaying interpretation of the \s somehow. The python is compiled by me following the Arch Linux build process. The
standard arch linux python3 works the same eg
> $ /usr/bin/python3
> Python 3.9.2 (default, Feb 20 2021, 18:40:11)
> [GCC 10.2.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import re

>>>> _re_unit = re.compile('^\s*(.*)(i|in|cm|mm|pt|pica)\s*$')

>>>> _re_unit

> re.compile('^\\s*(.*)(i|in|cm|mm|pt|pica)\\s*$')
>>>>

> The docs for the re module say since Python 3.8: "Please note that any invalid escape sequences in Python’s usage of the
> backslash in string literals now generate a DeprecationWarning and in the future this will become a SyntaxError. This
> behaviour will happen even if it is a valid escape sequence for a regular expression."
>
> So it's important to fix this.
>
> If you run flake8 on the source, you should find all such instances of invalid escape sequences (reported as warning W605).

I just ran pylint and have 30000 + warnings and errors about 500 hours work should fix it :(
adding flake can't make the work any better :(
>
> -- Christoph

thanks anyway

--
Robin Becker

[Robin Becker]

Thanks  Jürgen credit in latest version 3.5.66 now uploading to pypi

https://pypi.org/project/reportlab/3.5.66/

[Frank Millman]

On 2021-03-19 1:05 PM, Robin Becker wrote:

How strange, I run all the tests in many pythons including 3.8.6 etc etc and don't see this warning; I'm just wondering if I need to excite it somehow. I run on linux mostly, but the appveyor is windows and the github actions include OS X.

From the docs - What’s New In Python 3.9

"""

You should check for DeprecationWarning in your code

When Python 2.7 was still supported, a lot of functionality in Python 3 was kept for backward compatibility with Python 2.7. With the end of Python 2 support, these backward compatibility layers have been removed, or will be removed soon. Most of them emitted a DeprecationWarning warning for several years. For example, using collections.Mapping instead of collections.abc.Mapping emits a DeprecationWarning since Python 3.3, released in 2012.

Test your application with the -W default command-line option to see DeprecationWarning and PendingDeprecationWarning, or even with -W error to treat them as errors. Warnings Filter can be used to ignore warnings from third-party code.

"""

Maybe they are running their program using 'python -W default'?

Frank Millman

[Christoph Zwerschke]

On 19.03.2021 12:24, Frank Millman wrote:
> Maybe they are running their program using 'python -W default'?

Or they are using a debug build of Python, or running in development
mode. In release builds the deprecation warnings are ignored in the
default filter.

The warnings should actually also show when you're running the main
module, see https://www.python.org/dev/peps/pep-0565/
Therefore I currently don't understand why they don't show when you
enter the command in the REPL without "-W default".

-- Christoph

[Jürgen Gmach]

Hi Robin,

sorry for not providing enough information.

I ran the test suite for a proprietary python app of my company when I saw the deprecation warning.

I am on Ubuntu 18.04 and I run Python 3.8.8 (via dead snakes ppa https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa ) and pytest 6.2.2

Maybe you have turned off warnings in your test suite?

I usually provide a reproducer when I report issues, but ... huh... I hope you do not mind me being honest, compared to other open source projects, this project does not make it very easy to contribute.

First, it took me a while to find the repository ( https://hg.reportlab.com/hg-public/reportlab/file/tip/src/reportlab/platypus/paraparser.py ), then.. uhm... no hg installed here 🙂

I then found the mirror https://github.com/MrBitBucket/reportlab-mirror !! 👍

Most open source projects I contribute to use tox - so running tests for different python versions is super easy.

Nevermind, I found how to run tests in the readme.. and all pass, no deprecation warning.

Running your test suite with pytest or with `python -m unittest discover` shows quite some warnings.

e.g.

❯ python -m unittest discover

......../home/jugmac00/Projects/reportlab-mirror/tests/test_graphics_barcode.py:306: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/jugmac00/Projects/reportlab-mirror/barcode-out/index.html' mode='w' encoding='UTF-8'>

  open(os.path.join(outDir,'index.html'),'w').write('\n'.join(html))

ResourceWarning: Enable tracemalloc to get the object allocation traceback

................./home/jugmac00/Projects/reportlab-mirror/src/reportlab/graphics/shapes.py:526: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()

  args, varargs, varkw, defaults = getargspec(self.__init__)

.........E.............../home/jugmac00/Projects/reportlab-mirror/tests/test_graphics_speed.py:57: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/jugmac00/Projects/reportlab-mirror/test_graphics_speed_test1.log' mode='w' encoding='UTF-8'>

  open(outputfile('test_graphics_speed_test%s.log' % (isFast+1)), 'w').write(result)

ResourceWarning: Enable tracemalloc to get the object allocation traceback

./home/jugmac00/Projects/reportlab-mirror/tests/test_graphics_speed.py:57: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/jugmac00/Projects/reportlab-mirror/test_graphics_speed_test2.log' mode='w' encoding='UTF-8'>

  open(outputfile('test_graphics_speed_test%s.log' % (isFast+1)), 'w').write(result)

ResourceWarning: Enable tracemalloc to get the object allocation traceback

.../usr/lib/python3.8/unittest/case.py:633: ResourceWarning: unclosed file <_io.BufferedReader name='/home/jugmac00/Projects/reportlab-mirror/tests/pythonpowered.gif'>

  method()

ResourceWarning: Enable tracemalloc to get the object allocation traceback

... tons more...

or

reportlab-mirror/tests on  master [!?] via 🐍 v3.8.8 (venv)❯ python -m pytest

==================================================== warnings summary ====================================================

tests/test_graphics_charts.py: 3493 warnings

tests/test_graphics_render.py: 444 warnings

  /home/jugmac00/Projects/reportlab-mirror/src/reportlab/graphics/shapes.py:526: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()

    args, varargs, varkw, defaults = getargspec(self.__init__)

tests/test_lib_utils.py::ImporterTestCase::test2

  /home/jugmac00/Projects/reportlab-mirror/tools/pythonpoint/stdparser.py:10: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

    import string, imp, sys, os, copy

Please note, that both pytest and unittest have some failures, because I think you use some path manipulation magic when doing a `python setup.py tests` run.

(Another note: `python setup.py tests` is deprecated - actually, python setup.py anything is deprecated -> quote from a python core dev, who refuses to write a blog post about it, which I could link to so often 🙂 )

My conclusion ... your current way of running tests suppresses warnings (and is deprecated).

pytest is super awesome, but it is an external dependency. unittest discover is pretty ok, too 🙂

Jürgen

---

Von: Robin Becker <ro...@reportlab.com>
Gesendet: Freitag, 19. März 2021 10:45
An: reportlab-users <reportl...@lists2.reportlab.com>; Christoph Zwerschke <ci...@online.de>; Jürgen Gmach <juerge...@apis.de>
Betreff: Re: [reportlab-users] deprecation warning

 

[Christoph Zwerschke]

On 19.03.2021 12:05, Robin Becker wrote:
>> If you run flake8 on the source, you should find all such instances
>> of invalid escape sequences (reported as warning W605). > I just ran pylint and have 30000 + warnings and errors about 500

> hours work should fix it adding flake can't make the work any better
The warnings should also show when you're using an IDE like PyCharm or
VS Studio Code. Of course it's difficult to completely cleanup a large
legacy code base. One strategy is to just cleanup the code you're
currently working on, following the boy-scout rule - always leaving the
code behind in a better state than you found it, or to concentrate on
one issue at a time, like W605 - you can always filter your flake8 or
pylint output for a special warning. You can also exclude less relevant
warnings in flake8 or pylint, and concentrate on the more important
ones, and then gradually add rules to the config. Another strategy to
avoid the formatting related warnings is to use automatic formatting
with "black".

-- Christoph

[Robin Becker]

On 19/03/2021 11:57, Christoph Zwerschke wrote:
........

>> hours work should fix it adding flake can't make the work any better
> The warnings should also show when you're using an IDE like PyCharm or VS Studio Code. Of course it's difficult to
> completely cleanup a large legacy code base. One strategy is to just cleanup the code you're currently working on,
> following the boy-scout rule - always leaving the code behind in a better state than you found it, or to concentrate on
> one issue at a time, like W605 - you can always filter your flake8 or pylint output for a special warning. You can also
> exclude less relevant warnings in flake8 or pylint, and concentrate on the more important ones, and then gradually add
> rules to the config. Another strategy to avoid the formatting related warnings is to use automatic formatting with "black".
>
> -- Christoph

every new tool adds its own problems I use vim relying on some tool to tell me what to do almost always ends in disaster.

reportlab is one of the longest running projects and has many users. Any changes likely mean breakages.

I think this topic has lasted too long :)

--
Robin Becker

[Robin Becker]

On 19/03/2021 12:27, Jürgen Gmach wrote:
>> Thanks Jürgen credit in latest version 3.5.66 now uploading to pypi
>> https://pypi.org/project/reportlab/3.5.66/
>

> Thank you!
>

FYI the master/main/original source code repository is

https://hg.reportlab.com/hg-public/reportlab

at some point I assume Andy Robinson will switch this into github to allow more foot shooting; the reportlab-mirror is
updated automatically and I am using it for action building of manylinux wheels.

[Andy Robinson]

Hi all.

I regret that we don't have the resources to endlessly rework our core
code and machinery because someone had a bright idea about coding
standards. We are NOT working on most of it, most of the time,
because "it works"; and any time we make a pass through changing
things, things tend to break - often downstream in a build process or
a corporate customer's project. So, our rule generally is to only
fix actual bugs. Several times in the past, we have had developers
use some linting or coding standards tool, which had subtle effects
later on with customers.

Regarding the move to Github: I concede it's inevitable, and Mercurial
(a far superior system in terms of docs and comprehensibility IMHO
which has served us well for 15 years) has lost; but I don't want to
promise a definite timeline. These changes always affect many other
downstream projects you cannot see, and cost us a huge amount of time;
getting all our customers off Python 2.7 was enough of an ordeal!

Best Regards

Andy Robinson
ReportLab

--
Andy Robinson
Managing Director, ReportLab
Wimbletech Zone 2
35 Wimbledon Hill Road
London SW19 7NB, UK
Tel +44-20-8191-7277