Web2py Windows edition with 3.x Interpreter

[Jitun John]

"It requires Python 2.6 (no more supported), Python 2.7 (stable) or Python 3.5+ (recommended for new projects) already installed on your system. 

There are also binary packages for Windows and Mac OS X. They include the Python 2.7 interpreter so you do not need to have it pre-installed."

Sorry for my ignorance, I am assuming "Python 2.7 interpreter" as "python27.dll" and "pywintypes27.dll" files.

I recently moved my app from 2.x to 3.x using PyCharm IDE and the app works fine when using IDE on 3.7 python.

But when I try to compile it to .exe (using .iss file + hstart.exe), I cant use the existing web2py_win.zip file.

Gives me "<type 'exceptions.ValueError'> unsupported pickle protocol: 3"

As we are recommending new projects on 3.x, Is it possible for the next version to have a 3.x interpreter ?

or if we have a workaround, I am happy to try it.

Thanks.

[Jitun John]

Fixed the pickle issue by deleting "databases" folder in the app.

Wondering if we will ever (if it is possible) to get a 3.x python version of web2py windows  ("python37.dll" and "pywintypes37.dll" files.)

[elisha bere]

Hie, 

Is it possible to change from 2.7 to 3.x in my web2py app

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nico Zanferrari]

Hi,

this is a big problem also from my point of view and I've already opened issue 2027 . However it's not so easy to achieve this goal, because you've to switch to 3.x + go 64 bit + change the freezer program (py2exe and bbfreeze do not work with python >= 3.5). I'm playing with pyinstaller, but I'm not promising anything. And also a binary Mac version is needed (I only have a Win box), plus testing everything.

Cheers,

Nico 

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.

[Nico Zanferrari]

Good morning!

I'm proud to say that I've succeeded in building the Windows 64 bit binary with python 3.7.2 using pyinstaller !  Grab it from https://github.com/nicozanf/web2py-pyinstaller 

There are some little gothas, but it seems to work fine:

- in the console I've got many errors like 'ERROR:Rocket.Errors.Thread-2:Tried to send "500 Server Error" to client but received socket error'. They disappear as soon as I've disabled IPv6 and rebooted. There are users that report also to fix similar problems by adding the hostname on the hosts file

- psycopg2 is placed in a folder by itself, instead than in the root folder

- in the currently official binary build (with pyhton 2.7) there are also two binaries: web2py_no_console.exe and web2py_on_gevent.exe. They don't run correctly, and don't seem to be so important for me so I've decided to skip their generation.

- Macintosh compatibility is needed, but I cannot work on it

Please, help with testing it! I'll make a PR on the official web2py repository after some feedback.

Cheers,

Nico

[Massimo Di Pierro]

Great work. Thanks for this!

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscribe@googlegroups.com.

[Jitun John]

Apologies for the delay... 

I think my program has issues with new version of pydal 19.02, while it works fine on pydal 18.08

Thanks Nico for all the hard work.

I will test and let you know soon.

[Kevin Keller]

Whats the problem with pydal John?

[rāma]

This is wonderful. And, something I would love to be adopted to the official repo.

Thanks for this!

On Sunday, 3 March 2019 19:24:38 UTC+8, Nico Zanferrari wrote:

Good morning!

I'm proud to say that I've succeeded in building the Windows 64 bit binary with python 3.7.2 using pyinstaller !  Grab it from https://github.com/nicozanf/web2py-pyinstaller 

There are some little gothas, but it seems to work fine:

- in the console I've got many errors like 'ERROR:Rocket.Errors.Thread-2:Tried to send "500 Server Error" to client but received socket error'. They disappear as soon as I've disabled IPv6 and rebooted. There are users that report also to fix similar problems by adding the hostname on the hosts file

- psycopg2 is placed in a folder by itself, instead than in the root folder

- in the currently official binary build (with pyhton 2.7) there are also two binaries: web2py_no_console.exe and web2py_on_gevent.exe. They don't run correctly, and don't seem to be so important for me so I've decided to skip their generation.

- Macintosh compatibility is needed, but I cannot work on it

Please, help with testing it! I'll make a PR on the official web2py repository after some feedback.

Cheers,

Nico

Il giorno mer 27 feb 2019 alle ore 12:03 Nico Zanferrari <nico...@gmail.com> ha scritto:

Hi,

this is a big problem also from my point of view and I've already opened issue 2027 . However it's not so easy to achieve this goal, because you've to switch to 3.x + go 64 bit + change the freezer program (py2exe and bbfreeze do not work with python >= 3.5). I'm playing with pyinstaller, but I'm not promising anything. And also a binary Mac version is needed (I only have a Win box), plus testing everything.

Cheers,

Nico 

Il giorno mer 27 feb 2019 alle ore 11:45 elisha bere <elish...@gmail.com> ha scritto:

Hie, 

Is it possible to change from 2.7 to 3.x in my web2py app

On 17 Feb 2019 20:57, "Jitun John" <jitu...@gmail.com> wrote:

"It requires Python 2.6 (no more supported), Python 2.7 (stable) or Python 3.5+ (recommended for new projects) already installed on your system. 

There are also binary packages for Windows and Mac OS X. They include the Python 2.7 interpreter so you do not need to have it pre-installed."

Sorry for my ignorance, I am assuming "Python 2.7 interpreter" as "python27.dll" and "pywintypes27.dll" files.

I recently moved my app from 2.x to 3.x using PyCharm IDE and the app works fine when using IDE on 3.7 python.

But when I try to compile it to .exe (using .iss file + hstart.exe), I cant use the existing web2py_win.zip file.

Gives me "<type 'exceptions.ValueError'> unsupported pickle protocol: 3"

As we are recommending new projects on 3.x, Is it possible for the next version to have a 3.x interpreter ?

or if we have a workaround, I am happy to try it.

Thanks.

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to web...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to web...@googlegroups.com.

[rāma]

I tested this just now. I can't seem to import pandas.

raise ImportError("No module named %s" % modules_prefix)
ImportError: No module named applications.dqma.modules.pandas

I have a previous web2py from source and have all module in my site-packages which I copied over to this 64-bit version. I used a conda environment running of pycharm in my original version and made sure all the site-packages modules or not preinstalled in the conda environment.

Please help.

[Massimo Di Pierro]

Yes. The plan is to provide the following executable from the official web page:

OS web2py for py2.7, web2py for py3.7, web3py for py3.7 (WIP)

WINDOWS web2py for py2.7, web2py for py3.7, web3py for py3.7 (WIP)

[Nico Zanferrari]

Hi rāma,

you're right, the console in the new experimental binaries is not working fine - even the 'help' command gives errors ;-(

web2py -M -S my_app

web2py Web Framework

Created by Massimo Di Pierro, Copyright 2007-2019

Version 2.18.5-stable+timestamp.2019.04.08.04.22.03

Database drivers available: sqlite3, psycopg2, pyodbc, pymysql

application my_app does not exist, create (y/n)?y

WARNING:web2py:import IPython error; use default python shell

Python 2.7.14 (v2.7.14:84471935ed, Sep 16 2017, 20:25:58) [MSC v.1500 64 bit (AMD64)] on win32

Type "help", "copyright", "credits" or "license" for more information.

(InteractiveConsole)

>>> help

Traceback (most recent call last):

  File "<console>", line 1, in <module>

NameError: name 'help' is not defined

>>> import pandas

Traceback (most recent call last):

  File "<console>", line 1, in <module>

  File "C:\Users\nico\Desktop\web2py_win\web2py\gluon\custom_import.py", line 80, in custom_importer

    raise ImportError("No module named %s" % modules_prefix)

ImportError: No module named applications.my_app.modules.pandas

>>> quit()

Traceback (most recent call last):

  File "<console>", line 1, in <module>

NameError: name 'quit' is not defined

>>>

 

The official binaries are fine, and the sources, too.

Nico

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.

[rāma]

Any quick fix will be appreciated though.

[Nico Zanferrari]

Hi rāma, 

in fact we are both wrong and the console is working fine!

I've tested it with a simple module (the simplest one, helloword.py , see https://stackoverflow.com/questions/15746675/how-to-write-a-python-module-package)  and it can be regularly imported if it's placed on the main folder or on  the site-packages one. So, it seems that your problem is strictly related to copying the pandas module.

But indeed the interactive console is not so perfect. It's  somehow missing the help command (and "copyright", "credits" , "license"), and this happens with PyInstaller on PY2/PY3 with Windows and Mac even with the simple 2-lines program:

import code

code.interact() 

Nico

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.

[Nico Zanferrari]

I've found why the help command (and "copyright", "credits" , "license")  it's missing. It's due to the fact that PyInstaller replaces the original python module 'site'  with a fake one that contains this comment:

"This is a fake 'site' module available in default Python Library.

The real 'site' does some magic to find paths to other possible

Python modules. We do not want this behaviour for frozen applications.

Fake 'site' makes PyInstaller to work with distutils and to work inside

virtualenv environment."

It's possible to change this behaviour on gluon/shell.py by adding the missing code - I'm setting up a PR for this ;-)

Nico

[rāma]

Hi Nico,

Thank you for the insights. I am actually importing a number of modules within the controllers fyi, not only via shell.

Would patching shell.py fix the behaviour application wide?

Do you suggest I pip install into site-packages again instead of copying from my source version? I suspect that it's not going to fix it.

Folder structure:

web2py

   |_site-packages <- here's where my modules are

I am trying to package and bundle the whole web2py for work as an app solution and for test release soon.

Your experimental binaries would definitely help me to do early test with users while waiting for the official binaries. I could do some quick patches on web2py if required too, please advise how.

Best,

rama

On Monday, 29 April 2019 05:30:55 UTC+8, Nico Zanferrari wrote:

I've found why the help command (and "copyright", "credits" , "license")  it's missing. It's due to the fact that PyInstaller replaces the original python module 'site'  with a fake one that contains this comment:

"This is a fake 'site' module available in default Python Library.

The real 'site' does some magic to find paths to other possible

Python modules. We do not want this behaviour for frozen applications.

Fake 'site' makes PyInstaller to work with distutils and to work inside

virtualenv environment."

It's possible to change this behaviour on gluon/shell.py by adding the missing code - I'm setting up a PR for this ;-)

Nico

Il giorno dom 28 apr 2019 alle ore 19:45 Nico Zanferrari <nico...@gmail.com> ha scritto:

Hi rāma, 

in fact we are both wrong and the console is working fine!

I've tested it with a simple module (the simplest one, helloword.py , see https://stackoverflow.com/questions/15746675/how-to-write-a-python-module-package)  and it can be regularly imported if it's placed on the main folder or on  the site-packages one. So, it seems that your problem is strictly related to copying the pandas module.

But indeed the interactive console is not so perfect. It's  somehow missing the help command (and "copyright", "credits" , "license"), and this happens with PyInstaller on PY2/PY3 with Windows and Mac even with the simple 2-lines program:

import code

code.interact() 

Nico

[Nico Zanferrari]

Hi rama,

first of all, note that the shell.py fix will resolve only the 'cosmethic' problem of the help command (and "copyright", "credits" , "license" commands) that doesn't work in the interactive shell. But it will not change the import problem in any way.

According to the manual, if you run web2py from source you can import from it any python module that is in the PYTHONPATH (sys.path) - which is automatically changed by web2py in order to include also web2py/site-packages (globally) and web2py/application/yourapp/modules (inside yourapp). 

The last one is preferred if you are planning to distribute the app or you are experimenting with different versions of the same module. It also has higher priority and can even be accessed by other apps as web2py/application/anotherapp/modules.

If you run web2py from the compiled binary, any system-wide folder will be ignored - only what's down the web2py folder will be available. Modules can be placed on the folders: web2py, web2py/site-packages (globally for all the apps) and web2py/application/yourapp/modules (inside yourapp). You can check it by yourself with the interactive shell and the simple helloworld.py module.

What's different and problematic in the 'running from binary' is that you cannot use pip (or similar) for installing modules inside. You instead have to pip-install them on a full system (that has python of the same version of the binary one) and manually copy the related files inside the binary folders. For complex module like pandas, you also have to copy all the files for the related pre-requisites - and possibly test them one by one.

Unfortunately, this could fail - expecially if the modules have binary files. In this case, the last resort is to try to compile the binary version by yourself from a working web2py source - following the instructions on my repository and specifying your additional needed modules. Maybe PyInstaller will play the module dependency game better than you ;-)

I've also copied these advices on this howto page for future reference - feel free to correct me if I was wrong somewhere!

Cheers,

Nico 

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.

[rāma]

Hi Nico,

Thanks for your inputs.

I've actually run windows py2.7 binary with modules within site-packages, and pip can be leveraged by setting your target folder like this: 

pip install -t site-packages <package_name>

for the benefit of other readers. It's just that pandas didn't quite work for me in py2.7 and ported my code to python 3.5+.

It is convenient to distribute your app and whole web2py as a binary or as a client for desktop users. If we were to package with pynsist with webview: https://github.com/takluyver/pynsist/tree/master/examples/pywebview it can act like a standalone binary app on Windows or Mac masking the local web address. I also see a value in this as a way forward to have python based program running like a program locally on a client rather in the cloud; wherever necessary cloud sync can be employed. I have not been successful with pynsist though.

Best,

rama

On Monday, 29 April 2019 20:49:49 UTC+8, Nico Zanferrari wrote:

Hi rama,

first of all, note that the shell.py fix will resolve only the 'cosmethic' problem of the help command (and "copyright", "credits" , "license" commands) that doesn't work in the interactive shell. But it will not change the import problem in any way.

According to the manual, if you run web2py from source you can import from it any python module that is in the PYTHONPATH (sys.path) - which is automatically changed by web2py in order to include also web2py/site-packages (globally) and web2py/application/yourapp/modules (inside yourapp). 

The last one is preferred if you are planning to distribute the app or you are experimenting with different versions of the same module. It also has higher priority and can even be accessed by other apps as web2py/application/anotherapp/modules.

If you run web2py from the compiled binary, any system-wide folder will be ignored - only what's down the web2py folder will be available. Modules can be placed on the folders: web2py, web2py/site-packages (globally for all the apps) and web2py/application/yourapp/modules (inside yourapp). You can check it by yourself with the interactive shell and the simple helloworld.py module.

What's different and problematic in the 'running from binary' is that you cannot use pip (or similar) for installing modules inside. You instead have to pip-install them on a full system (that has python of the same version of the binary one) and manually copy the related files inside the binary folders. For complex module like pandas, you also have to copy all the files for the related pre-requisites - and possibly test them one by one.

Unfortunately, this could fail - expecially if the modules have binary files. In this case, the last resort is to try to compile the binary version by yourself from a working web2py source - following the instructions on my repository and specifying your additional needed modules. Maybe PyInstaller will play the module dependency game better than you ;-)

I've also copied these advices on this howto page for future reference - feel free to correct me if I was wrong somewhere!

Cheers,

Nico 

[Nico Zanferrari]

Hi rama, 

I've read that pip was not working with the binary package, but I've never actually tried it ;-) I'm happy to hear that it works fine - and that it has resolved your problem with the pandas module (at least on PY3). I'll test this a bit, and update my HOWTO page!

About pynsist, it seems to me like the NSIS Setup tool + PyInstaller (and it does not support Mac). I like the idea but I don't see any big advantage over PyInstaller itself.

Thank you,

Nico

Il giorno mar 30 apr 2019 alle ore 05:15 rāma <ranje...@gmail.com> ha scritto:

Hi Nico,

Thanks for your inputs.

I've actually run windows py2.7 binary with modules within site-packages, and pip can be leveraged by setting your target folder like this: pip install -t site-packages <package_name>. Just for the benefit of other readers. It's just that pandas didn't quite work for me in py2.7 and ported my code to python 3.5+.

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.

[Dave S]

On Monday, April 29, 2019 at 8:15:09 PM UTC-7, rāma wrote:

Hi Nico,

Thanks for your inputs.

I've actually run windows py2.7 binary with modules within site-packages, and pip can be leveraged by setting your target folder like this: 

pip install -t site-packages <package_name>

for the benefit of other readers. It's just that pandas didn't quite work for me in py2.7 and ported my code to python 3.5+.

 

Hi, rāma --

I'm interested in how you got pip installed with python as a DLL.  I want to play around with Pillow.

/dps

[Nico Zanferrari]

Hi Dave,

I think that rāma  meant that he used a Windows system with the same Python version as the one in a already frozen web2py program in order to install modules inside it. AFAIK he didn't use a pip program inside a frozen web2py.

See this howto page for details ;-)

Nico

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/59ead495-b06e-4dff-b644-7757849b63e5%40googlegroups.com.

[rāma]

Hi Dave,

Nico is right here. I used pip to pre-package modules for easier distribution. Not really a DLL but works. The only downside is that distributing the app can be really large.

[Dave S]

On Friday, June 14, 2019 at 9:00:05 AM UTC-7, rāma wrote:

Hi Dave,

Nico is right here. I used pip to pre-package modules for easier distribution. Not really a DLL but works. The only downside is that distributing the app can be really large.

Thanks, guys!

Dave S

/dps