Unable to validate using ArelleCmdLine on 1.2021.07.08 with multiple operating systems.

[Cody Gordon]

Hey Arelle group, 

We are unable to validate using your webserver function with the newest 1.2021.07.08 in Windows Server, Windows 10 or Centos 8. We are able to process a few with the GUI on windows 10 desktops using a workaround one of our engineers ran into, however straight out of the exe we throw the error described near the bottom of this email.  We have attempted to build this on multiple fresh vm’s in our GCP environment and locally without success. 

Rest kickback message same error for all 3 OS Types

2021-07-21 20:05:44,265 [Exception] Entry point loading Failed to complete request:
[Errno 2] No such file or directory: '/POSTupload.zip/_IXDS#?#/POSTupload.zip/jbht-20210331.htm'
[' File "/media/psf/arelle/hfdev/arelle/CntlrCmdLine.py", line 904, in run\n', ' File "/media/psf/arelle/hfdev/arelle/ModelManager.py", line 157, in load\n', ' File "/media/psf/arelle/hfdev/arelle/ModelXbrl.py", line 70, in load\n', ' File "/media/psf/arelle/hfdev/arelle/ModelDocument.py", line 129, in load\n', ' File "/media/psf/arelle/hfdev/arelle/ModelDocument.py", line 129, in <genexpr>\n', ' File "/etc/arelle/plugin/validate/EFM-htm/__init__.py", line 139, in isLoadableHtml\n with io.open(filepath, \'rt\', encoding=\'utf-8\') as f:\n'] –

 

Windows Server 2019 and Windows 10 Desktop version windows 64bit 2021.07.08 console error:

• Start command:

     .\arelleCmdLine.exe --webserver x.x.x.x:8080

       

• Console error:

Exception loading plug-in Validate GFM: No module named 'arelle.plugin.validate.EFM.Document'
[' File "Z:\\Users\\arelle\\hfdev\\arelle\\PluginManager.py", line 333, in loadModule\n', ' File "C:\\Users\\hermf\\AppData\\Local\\Programs\\Python\\Python39\\lib\\imp.py", line 244, in load_module\n', ' File "C:\\Users\\hermf\\AppData\\Local\\Programs\\Python\\Python39\\lib\\imp.py", line 216, in load_package\n', ' File "<frozen importlib._bootstrap>", line 711, in _load\n', ' File "<frozen importlib._bootstrap>", line 680, in _load_unlocked\n', ' File "<frozen importlib._bootstrap_external>", line 790, in exec_module\n', ' File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed\n', ' File "C:\\Program Files\\Arelle\\plugin\\validate\\GFM\\__init__.py", line 10, in <module>\n from arelle.plugin.validate.EFM.Document import checkDTSdocument\n']
Exception loading plug-in Validate GFM: No module named 'arelle.plugin.validate.EFM.Document'
[' File "Z:\\Users\\arelle\\hfdev\\arelle\\PluginManager.py", line 333, in loadModule\n', ' File "C:\\Users\\hermf\\AppData\\Local\\Programs\\Python\\Python39\\lib\\imp.py", line 244, in load_module\n', ' File "C:\\Users\\hermf\\AppData\\Local\\Programs\\Python\\Python39\\lib\\imp.py", line 216, in load_package\n', ' File "<frozen importlib._bootstrap>", line 711, in _load\n', ' File "<frozen importlib._bootstrap>", line 680, in _load_unlocked\n', ' File "<frozen importlib._bootstrap_external>", line 790, in exec_module\n', ' File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed\n', ' File "C:\\Program Files\\Arelle\\plugin\\validate\\GFM\\__init__.py", line 10, in <module>\n from arelle.plugin.validate.EFM.Document import checkDTSdocument\n']
10.128.0.4 - - [21/Jul/2021 17:49:12] "POST /rest/xbrl/validation?flavor=edgar%7Csec&plugins=EdgarRenderer%7Cvalidate%7CxbrlDB%7Cvalidate%2FDQC&media=json HTTP/1.1" 200 10509

 

• Centos 8 using redhat 2021-06-21
  • Start command:

./arelleCmdLine --webserver x.x.x.x:8002 --plugins 'validate/EFM|EdgarRenderer|EdgarRendererAllReports.py|transforms/SEC.py|xbrlDB'

 

• console error:

Exception loading plug-in Validate GFM: No module named 'arelle.plugin.validate.EFM.Document'

['  File "/media/psf/arelle/hfdev/arelle/PluginManager.py", line 333, in loadModule\n', '  File "/home/hermf/Python39/lib/python3.9/imp.py", line 244, in load_module\n', '  File "/home/hermf/Python39/lib/python3.9/imp.py", line 216, in load_package\n', '  File "<frozen importlib._bootstrap>", line 711, in _load\n', '  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked\n', '  File "<frozen importlib._bootstrap_external>", line 790, in exec_module\n', '  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed\n', '  File "/etc/arelle/plugin/validate/GFM/__init__.py", line 10, in <module>\n    from arelle.plugin.validate.EFM.Document import checkDTSdocument\n']

Exception loading plug-in Validate GFM: No module named 'arelle.plugin.validate.EFM.Document'

['  File "/media/psf/arelle/hfdev/arelle/PluginManager.py", line 333, in loadModule\n', '  File "/home/hermf/Python39/lib/python3.9/imp.py", line 244, in load_module\n', '  File "/home/hermf/Python39/lib/python3.9/imp.py", line 216, in load_package\n', '  File "<frozen importlib._bootstrap>", line 711, in _load\n', '  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked\n', '  File "<frozen importlib._bootstrap_external>", line 790, in exec_module\n', '  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed\n', '  File "/etc/arelle/plugin/validate/GFM/__init__.py", line 10, in <module>\n    from arelle.plugin.validate.EFM.Document import checkDTSdocument\n']

35.191.15.161 - - [21/Jul/2021 20:05:44] "POST /rest/xbrl/validation?flavor=edgar%7Csec&plugins=EdgarRenderer%7Cvalidate%7CxbrlDB%7Cvalidate%2FDQC&media=json HTTP/1.1" 200 10295

Exception loading plug-in Validate GFM35.191.14.11 - - [21/Jul/2021 20:08:25] "GET /rest/configure?plugins=show HTTP/1.1" 200 9888

 

Desktop error using a .zip when your don’t install win-x64-2020-08-12 first then uninstall leaving the 3 folder and install 2021-07-08 over top.

Exception loading C:\Users\cgordon\Desktop\alsk-20201231.zip\alsk-20201231.htm: [Errno 2] No such file or directory: 'C:\\Users\\cgordon\\Desktop\\alsk-20201231.zip\\alsk-20201231.htm', at ['  File "Z:\\Users\\arelle\\hfdev\\arelle\\CntlrWinMain.py", line 796, in backgroundLoadXbrl\n', '  File "Z:\\Users\\arelle\\hfdev\\arelle\\ModelManager.py", line 157, in load\n', '  File "Z:\\Users\\arelle\\hfdev\\arelle\\ModelXbrl.py", line 70, in load\n', '  File "Z:\\Users\\arelle\\hfdev\\arelle\\ModelDocument.py", line 129, in load\n', '  File "Z:\\Users\\arelle\\hfdev\\arelle\\ModelDocument.py", line 129, in <genexpr>\n', '  File "C:\\Program Files\\Arelle\\plugin\\validate\\EFM-htm\\__init__.py", line 139, in isLoadableHtml\n    with io.open(filepath, \'rt\', encoding=\'utf-8\') as f:\n']

[Kevin Tibbs]

Originally reported for 2021.07.08, but can confirm is still happening as of version 2021.09.13.

There seems to be some difference between running Arelle as a application and running Arelle in a webserver. All screenshots and snippets are the same Arelle installation on the same PC.

Here is the Desktop validation for an iXBRL submission. Validation messages show up and the rendering opens in the browser as expected.

When we switch to the webserver version, the story doesn't end the same. Starting it up in the typical way:

Then we submit the file for validation over CURL.

The command I'm using is

Curl -X POST "-HContent-type: application/zip" -o C:\Users\ [Redacted] \Desktop\test-output.html "http:// [redacted] :8080/rest/xbrl/validation?flavor=edgar|sec&media=html&plugins=EdgarRenderer|validate|xbrlDB|validate/DQC" -T C:\Users\ [Redacted] \Documents\ [Redacted] .zip

Which gives results like this:

and the server reports an error code very similar to Cody's post above.

or in Text form:

C:\Users\hermf\AppData\Local\Programs\Python\Python39\lib\site-packages\numpy\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:

C:\Program Files\Arelle\lib\numpy\.libs\libopenblas.JPIJNSWNNAN3CE6LLI5FWSPHUT2VXMTH.gfortran-win_amd64.dll

C:\Program Files\Arelle\lib\numpy\.libs\libopenblas.PYQHXLVVQ7VESDPUVUADXEVJOBGHJPAY.gfortran-win_amd64.dll

Exception loading plug-in Validate GFM: No module named 'arelle.plugin.validate.EFM.Document'

['  File "z:\\Users\\arelle\\hfdev\\arelle\\PluginManager.py", line 333, in loadModule\n', '  File "C:\\Users\\hermf\\AppData\\Local\\Programs\\Python\\Python39\\lib\\imp.py", line 244, in load_module\n', '  File "C:\\Users\\hermf\\AppData\\Local\\Programs\\Python\\Python39\\lib\\imp.py", line 216, in load_package\n', '  File "<frozen importlib._bootstrap>", line 711, in _load\n', '  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked\n', '  File "<frozen importlib._bootstrap_external>", line 790, in exec_module\n', '  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed\n', '  File "C:\\Program Files\\Arelle\\plugin\\validate\\GFM\\__init__.py", line 10, in <module>\n    from arelle.plugin.validate.EFM.Document import checkDTSdocument\n']

Exception loading plug-in Validate GFM: No module named 'arelle.plugin.validate.EFM.Document'

['  File "z:\\Users\\arelle\\hfdev\\arelle\\PluginManager.py", line 333, in loadModule\n', '  File "C:\\Users\\hermf\\AppData\\Local\\Programs\\Python\\Python39\\lib\\imp.py", line 244, in load_module\n', '  File "C:\\Users\\hermf\\AppData\\Local\\Programs\\Python\\Python39\\lib\\imp.py", line 216, in load_package\n', '  File "<frozen importlib._bootstrap>", line 711, in _load\n', '  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked\n', '  File "<frozen importlib._bootstrap_external>", line 790, in exec_module\n', '  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed\n', '  File "C:\\Program Files\\Arelle\\plugin\\validate\\GFM\\__init__.py", line 10, in <module>\n    from arelle.plugin.validate.EFM.Document import checkDTSdocument\n']

XXX.XXX.XXX.XXX - - [26/Oct/2021 15:10:10] "POST /rest/xbrl/validation?flavor=edgar|sec&media=html&plugins=EdgarRenderer|validate|xbrlDB|validate/DQC HTTP/1.1" 200 2147444

The response HTML I get from this validation has > 7,000 HTML:Syntax errors, one for each tag with the ix, link, ixbrl, ixbrldi, and us-gaap prefixes.

in text, that last error is:

2021-10-26 15:10:09,775 [IOerror] filename.htm: file error: [Errno 2] Archive \POSTupload.zip\_IXDS#?#\POSTupload.zip\filename.htm: '_IXDS#?#\\POSTupload.zip\\filename.htm' - filename.htm

Arelle works well in desktop application mode, but there is something quite wrong in webserver mode. Am I passing the wrong parameters in my CURL command? It worked previously with those flavor and plugin options. I'm at a loss.

Any advice would be much appreciated.

Kevin