Markdown integration

[Francois]

Hello,

On a freshly installed windows/python I saw a regression in markdown integration.

With this program: 

''' hello.py

import markdown

import sys

from markdown.__meta__ import __version_info__

print("Python version  %s" % sys.version)

print("Version info    %s" % str(sys.version_info))

print("Markdown version %s" % str(__version_info__))

print(markdown.markdown('# hello'))

'''

The output is:

'''

Python version  3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]

Version info    sys.version_info(major=3, minor=9, micro=0, releaselevel='final', serial=0)

Markdown version (3, 3, 2, 'final', 0)

<h1>hello</h1>

'''

After packaging...

'''

$ pyinstaller.exe hello.py

35 INFO: PyInstaller: 4.0

35 INFO: Python: 3.9.0

35 INFO: Platform: Windows-10-10.0.18362-SP0

36 INFO: wrote C:\Dev\Home\fc\techno\python\fc\test\packaging\pyinstaller\hello.spec

37 INFO: UPX is not available.

38 INFO: Extending PYTHONPATH with paths

['C:\\Dev\\Home\\fc\\techno\\python\\fc\\test\\packaging\\pyinstaller',

 'C:\\Dev\\Home\\fc\\techno\\python\\fc\\test\\packaging\\pyinstaller']

42 INFO: checking Analysis

42 INFO: Building Analysis because Analysis-00.toc is non existent

42 INFO: Initializing module dependency graph...

43 INFO: Caching module graph hooks...

52 INFO: Analyzing base_library.zip ...

1839 INFO: Processing pre-find module path hook distutils from 'c:\\program files\\python39\\lib\\site-packages\\PyInstaller\\hooks\\pre_find_module_path\\hook-distutils.py'.

1839 INFO: distutils: retargeting to non-venv dir 'c:\\program files\\python39\\lib'

3195 INFO: Caching module dependency graph...

3263 INFO: running Analysis Analysis-00.toc

3264 INFO: Adding Microsoft.Windows.Common-Controls to dependent assemblies of final executable

  required by c:\program files\python39\python.exe

3292 WARNING: lib not found: api-ms-win-core-path-l1-1-0.dll dependency of c:\program files\python39\python39.dll

3306 INFO: Analyzing C:\Dev\Home\fc\techno\python\fc\test\packaging\pyinstaller\hello.py

3386 INFO: Processing module hooks...

3386 INFO: Loading module hook 'hook-markdown.py' from 'C:\\Users\\francois.connan\\AppData\\Roaming\\Python\\Python39\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...

3503 INFO: Loading module hook 'hook-distutils.py' from 'c:\\program files\\python39\\lib\\site-packages\\PyInstaller\\hooks'...

3503 INFO: Loading module hook 'hook-encodings.py' from 'c:\\program files\\python39\\lib\\site-packages\\PyInstaller\\hooks'...

3553 INFO: Loading module hook 'hook-lib2to3.py' from 'c:\\program files\\python39\\lib\\site-packages\\PyInstaller\\hooks'...

3575 INFO: Loading module hook 'hook-sysconfig.py' from 'c:\\program files\\python39\\lib\\site-packages\\PyInstaller\\hooks'...

3575 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from 'c:\\program files\\python39\\lib\\site-packages\\PyInstaller\\hooks'...

3576 INFO: Loading module hook 'hook-xml.py' from 'c:\\program files\\python39\\lib\\site-packages\\PyInstaller\\hooks'...

3605 INFO: Loading module hook 'hook-_tkinter.py' from 'c:\\program files\\python39\\lib\\site-packages\\PyInstaller\\hooks'...

3679 INFO: checking Tree

3679 INFO: Building Tree because Tree-00.toc is non existent

3679 INFO: Building Tree Tree-00.toc

3711 INFO: checking Tree

3711 INFO: Building Tree because Tree-01.toc is non existent

3711 INFO: Building Tree Tree-01.toc

3727 INFO: Looking for ctypes DLLs

3748 INFO: Analyzing run-time hooks ...

3750 INFO: Including run-time hook 'c:\\program files\\python39\\lib\\site-packages\\PyInstaller\\hooks\\rthooks\\pyi_rth__tkinter.py'

3751 INFO: Including run-time hook 'c:\\program files\\python39\\lib\\site-packages\\PyInstaller\\hooks\\rthooks\\pyi_rth_multiprocessing.py'

3756 INFO: Looking for dynamic libraries

3945 INFO: Looking for eggs

3945 INFO: Using Python library c:\program files\python39\python39.dll

3945 INFO: Found binding redirects:

[]

3960 INFO: Warnings written to C:\Dev\Home\fc\techno\python\fc\test\packaging\pyinstaller\build\hello\warn-hello.txt

3995 INFO: Graph cross-reference written to C:\Dev\Home\fc\techno\python\fc\test\packaging\pyinstaller\build\hello\xref-hello.html

4013 INFO: checking PYZ

4013 INFO: Building PYZ because PYZ-00.toc is non existent

4013 INFO: Building PYZ (ZlibArchive) C:\Dev\Home\fc\techno\python\fc\test\packaging\pyinstaller\build\hello\PYZ-00.pyz

4505 INFO: Building PYZ (ZlibArchive) C:\Dev\Home\fc\techno\python\fc\test\packaging\pyinstaller\build\hello\PYZ-00.pyz completed successfully.

4511 INFO: checking PKG

4511 INFO: Building PKG because PKG-00.toc is non existent

4511 INFO: Building PKG (CArchive) PKG-00.pkg

4519 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.

4519 INFO: Bootloader c:\program files\python39\lib\site-packages\PyInstaller\bootloader\Windows-64bit\run.exe

4519 INFO: checking EXE

4519 INFO: Building EXE because EXE-00.toc is non existent

4519 INFO: Building EXE from EXE-00.toc

4520 INFO: Appending archive to EXE C:\Dev\Home\fc\techno\python\fc\test\packaging\pyinstaller\build\hello\hello.exe

4521 INFO: Building EXE from EXE-00.toc completed successfully.

4523 INFO: checking COLLECT

4523 INFO: Building COLLECT because COLLECT-00.toc is non existent

4524 INFO: Building COLLECT COLLECT-00.toc

5137 INFO: Building COLLECT COLLECT-00.toc completed successfully.

'''

The binary fails on the 'import markdown' line.

'''

$ dist/hello/hello.exe

Traceback (most recent call last):

  File "hello.py", line 1, in <module>

    import markdown

  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load

  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked

  File "C:\Program Files\Python39\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 493, in exec_module

    exec(bytecode, module.__dict__)

  File "markdown\__init__.py", line 29, in <module>

  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load

  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked

  File "C:\Program Files\Python39\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 493, in exec_module

    exec(bytecode, module.__dict__)

  File "markdown\core.py", line 27, in <module>

  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load

  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked

  File "C:\Program Files\Python39\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 493, in exec_module

    exec(bytecode, module.__dict__)

  File "markdown\preprocessors.py", line 29, in <module>

  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load

  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked

  File "C:\Program Files\Python39\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 493, in exec_module

    exec(bytecode, module.__dict__)

  File "markdown\htmlparser.py", line 30, in <module>

  File "<frozen importlib._bootstrap>", line 562, in module_from_spec

AttributeError: 'NoneType' object has no attribute 'loader'

[9664] Failed to execute script hello

'''

I didn't see any reference on this issue on the web...

Do you have a workaround for this issue ?

Thanks,

Best Regards,

Francois

[bwoodsend]

Well the offending code is inside markdown/htmlparser.py which has been rather nastily monkey-patched.

# Import a copy of the html.parser lib as `htmlparser` so we can monkeypatch it.
# Users can still do `from html import parser` and get the default behavior.
spec = importlib.util.find_spec('html.parser')
htmlparser = importlib.util.module_from_spec(spec)
spec.loader.exec_module(htmlparser)
sys.modules['htmlparser'] = htmlparser

Currently there is no support for importlib.util.find_spec()in PyInstaller and nothing we have can force it to work. So no is the answer. You’ll have to stick to pip install markdown<3.3.0. Fell free to raise this as an issue on the hooks repo but as this would be a simple fix for markdown or a very difficult fix for us I doubt it’ll get much attention.

​

[Francois]

Thanks bwoodsend,

Your response point me to a workaround... It's sufficient to force "pyinstaller" to load "html.parser"...

Either in pyinstaller command line

'''

pyinstaller.exe -y --hidden-import html.parser hello.py

'''

or in the source code.

'''

import sys

import markdown

from markdown.__meta__ import __version_info__

from html import parser # unused but force pyinstaller to load "html.parser", needed by markdown 3.3.0

print("Python version  %s" % sys.version)

print("Version info    %s" % str(sys.version_info))

print("Markdown version %s" % str(__version_info__))

print(markdown.markdown('# hello'))

'''

'''

Python version  3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]

Version info    sys.version_info(major=3, minor=9, micro=0, releaselevel='final', serial=0)

Markdown version (3, 3, 3, 'final', 0)

<h1>hello</h1>

'''

Thanks,

Francois.

[bwoodsend]

Ah, well not only do I need to eat my words but it turns this has already been fixed. (I also was the one to merge said fix which speaks volumes on how poor my memory is…).

When it’s released you can pip install "pyinstaller-hooks-contrib>=2020.10" or before then pip install https://github.com/pyinstaller/pyinstaller-hooks-contrib/archive/master.zip.

​

[legorooj]

I've just released the October build (2020.10) of hooks contrib, which has the markdown fix included in it.

Legorooj

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Thursday, 29 October 2020 02:37, bwoodsend <bwoo...@gmail.com> wrote:

Ah, well not only do I need to eat my words but it turns this has already been fixed. (I also was the one to merge said fix which speaks volumes on how poor my memory is…).

When it’s released you can pip install "pyinstaller-hooks-contrib>=2020.10" or before then pip install https://github.com/pyinstaller/pyinstaller-hooks-contrib/archive/master.zip.

--

You received this message because you are subscribed to the Google Groups "PyInstaller" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pyinstaller...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pyinstaller/d13d8457-929e-46c1-acc8-a2f2318d6aa7n%40googlegroups.com.