Port web2py app to py4web app
646 views
Skip to first unread message
sandeep patel
Nov 18, 2020, 8:57:59 AM11/18/20
to py4web
Hello All,
I have a running application that is based on web2py. I want to port that app in py4web.
Is there any efficient way to port web2py app to py4web app?
Thanks
{Name: Sandeep}
Jim Steil
Nov 18, 2020, 10:28:53 AM11/18/20
to py4web
There is no direct migration for an app from web2py to py4web.
That being said, I believe moving to py4web is the right this to do for all web2py apps.
py4web has lots of the same concepts as web2py, but the implementation details are different for most.
Advantages of py4web over web2py
- significantly faster (by A LOT)
- a more pythonic approach
- designed for python 3
- much smaller code base
Things in common with web2py (although implementation may be different)
- pydal
- yatl (same templating) - py4web uses [[...]] instead of {{...}} by default
- built in auth (different from web2py)
- built in Form (different from web2py)
- built in Grid (significantly different than web2py)
- flash messages
Important things you have understand to use py4web
- in web2py, the entire app was executed on every request with variables in db.py being global - in py4web, app is executed once, at startup - each request then only runs the controller function that is called - you have to import everything you need in a controller function.
- action, action.uses, routes and factories
- use this google group for any questions - there are a number of us active on the list to help
I'm sure I'll think of more issues I've not touched on, but this list is a good start.
Check out the examples app that is provided if you clone from github. Lots of good examples in there on how to accomplish things.
The documentation is a work in progress. Please send suggestions on what needs to be improved. Any contributions are appreciated.
NOTE - I do not represent the py4web project. These are my thoughts on the subject.
-Jim
stifan kristi
Nov 18, 2020, 11:44:46 AM11/18/20
to py4web
think it depends on your web2py code, tried before it takes days to make it run smoothly port http://sugizo.pythonanywhere.com/music into py4web
objective
things to consider
- copy web2py models, controllers and modules files into py4web app folder
- copy web2py models, controllers and modules files into py4web app folder
better focus on models first after it work smooth on py4web then do the controllers and modules files
- add db.commit() in every define table
- in __init__.py
- in __init__.py
put all models define table files (so that can be access on _dashboard) and every python files that can be access from browser
controllers and modules part
- add db.commit() in every table activities : insert(), update(), delete()- put @action(), @action.uses() in every python function that can be access from web browser
- if use web2py response then you can search and replace it
- if use web2py request.vars, in py4web use request.json or request.query
and
web2py request.args in py4web the args is define on @action() decorator and python function name
web2py request.args in py4web the args is define on @action() decorator and python function name
- move web2py variables (mutable) inside the function that use it, never define it as global variables or will get unexpected weird result with no error traceback occured
- if use web2py SQLFORM() or SQLFORM.grid() in py4web use Form()
while web2py has FORM() please consider the signature (parameter that build it), not everything is same
even better define form in html yourself (IMO), so you cut about migration FORM(), SQLFORM() above
even better define form in html yourself (IMO), so you cut about migration FORM(), SQLFORM() above
- if use web2py that use module files, search module function and delete its module file name to make it work in py4web
- if use web2py modules files find current. and delete it in py4web appviews part
- in web2py views copy into py4web app templates
- in py4web templates folder
search {{ and replace it with [[
and
}} and replace it with ]]
- if use web2py ajax(), then you can consider replace the code with Q.ajax() in py4web or even better modify your code to use javascript on html files
there's many more, it's just experienced porting http://sugizo.pythonanywhere.com/music into py4web
objective
move define table files into models folder, accessed python function from browser into controllers folder and python function into modules folder, because put everythings in app folder make it a lot of files on it, better to make it organize into separate folders like web2py
question
is there any simple example how to make py4web execute python into separate subfolder ?
tried it before have face an error about variable define on common.py (db, T)
thanks and best regards,
stifan
AGRogers
Nov 18, 2020, 8:26:59 PM11/18/20
to stifan kristi, py4web
Thanks for the lists and details. They will be very handy in the future.
--
You received this message because you are subscribed to the Google Groups "py4web" group.
To unsubscribe from this group and stop receiving emails from it, send an email to py4web+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/py4web/6ecc5a32-df35-4786-a3d3-dd35e28d77ban%40googlegroups.com.
stifan kristi
Nov 18, 2020, 11:10:42 PM11/18/20
to py4web
in term of api not work (ATM), not sure in the future
- web2py service decorator (@service)
- web2py @request.restful()
- web2py jwt
- import gluon.contrib.simplejson
- response.json()
- XML() not work in python function on py4web, but XML() work in py4web templates
- validate_and_insert and validate_and_update not work, it's related with request.vars in web2py, tried request.query in py4web not work from cli
best regards,
stifan
sandeep patel
Nov 19, 2020, 10:39:01 AM11/19/20
to stifan kristi, py4web
Guys thanks for your valuable suggestions .I will definitely look into it while app migration.
--
You received this message because you are subscribed to the Google Groups "py4web" group.
To unsubscribe from this group and stop receiving emails from it, send an email to py4web+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/py4web/d1b302b1-0fec-4656-a582-969c9276d00fn%40googlegroups.com.
Michael Ellis
May 18, 2022, 3:34:20 PM5/18/22
to py4web
A client has asked for updates to a web2py app I wrote for them almost 10 years ago. I'm wondering how hard it would be to port it to py4web. The app controls emergency lighting systems and needs to run on a BeagleBone Black so the potential for a 10x speedup is very attractive as is the more standardized use of Python.
I've already ported the Python2.7 code to Python3.9 and verified that it runs on the latest web2py. The port went surprisingly smoothly; 2to3 did the work with only a few instances requiring manual intervention.
The web2py portion of app is not terribly complex. There are about 9 tables that describe lighting systems, circuits, and fixtures, test schedules and reports. The views are mostly pages with SQLFORMs to display data from the tables. There are also various hardware control programs that run as separate processes and exchange data through pipes. Those don't know anything about web2py and vice versa.
I've read through https://py4web.com/_documentation/static/en/chapter-15.html several times and think I follow the gist of it. I'm looking for a sensible step by step procedure. Is the procedure outlined in this thread by Stifan Kristi still a good approach?
Thanks,
Mike
Jim Steil
May 18, 2022, 4:08:17 PM5/18/22
to py4web
In my opinion it is worth the extra effort to migrate to py4web sooner than later. But, of course I'd say that.
I'm in the process of migrating 10 years worth of web apps (all wrapped in one web2py application) with over 300 tables to py4web. There is a learning curve, there is no doubt. While I'm moving to py4web I'm taking advantage of the move and rewriting new for py4web. Instead of 10 years worth of different interfaces, javascript/css frameworks, programming techniques, boiler plate templates, all code and interfaces are now consistent.
As for Stifan's list, I do things a bit differently.
- keep all model defs in one file, models.py, copying the model defs should work fine
- controllers - I'd rewrite everything following py4web protocols and using the included form and grid where you used Form and SQLFORM.grid in the past.
- I created a directory called 'lib' and there I keep the routines that were in modules in web2py. Make sure to import them all in __init__.py in your application root
- views/templates - I'm also moving from bootstrap to bulma and getting rid of jquery in my transition. So, I'm rewriting all templates. In addition I'm using htmx in place of 'loaded components'. I have almost zero lines of javascript in my application code.
In addition to the examples included in py4web you can see a sample I created at https://southbreeze.pythonbench.com. If uses py4web grid, htmx and bulma. There is a link to the github repo on the main page so you can see all the code.
Hope this helps.
Let us know if you run into any roadblocks. You can find help here or the discord channel.
-Jim
Michael Ellis
May 18, 2022, 4:44:47 PM5/18/22
to py4web
Thanks, Jim! I'll start as you recommended and see what happens...
Message has been deleted
Massimo
May 22, 2022, 6:52:23 PM5/22/22
to py4web
The most difficult part of this job is using the new Auth which has different APIs. All the DB logic, forms, and templates should be trivial. So try port auth first and then you move the rest of the logic.
Alexander Beskopilny
May 23, 2022, 2:39:14 AM5/23/22
to py4web
Hello!
The recommendation "Do not change global variables" in controllers.py is not very clear
py4web is a normal threaded program
What (and why) won't work if I use a global-threadsafe-list or global-threadsafe-counter in controllers.py
and I change these variables in the app-controllers ?
Thanks in advance for the explanation
# controllers.py
# 1------------------------------------
class SafeList:
def __init__(self):
self._list = list()
self._lock = threading.Lock()
def append(self, value):
with self._lock:
self._list.append(value)
def check(self, value):
with self._lock:
return value in self._list
def remove(self, value):
with self._lock:
# append the value
self._list.remove(value)
def pop(self):
with self._lock:
return self._list.pop()
def get(self, index):
# acquire the lock
with self._lock:
# read a value at the index
return self._list[index]
def __call__(self):
with self._lock:
return self._list
@property
def all(self):
with self._lock:
return self._list
def length(self):
# acquire the lock
with self._lock:
return len(self._list)
yield_id_list = SafeList()
# 2------------------------------------
class threadsafe_iter:
"""Takes an iterator/generator and makes it thread-safe by
serializing call to the `next` method of given iterator/generator.
"""
def __init__(self, it):
self.it = it
self.lock = threading.Lock()
def __iter__(self):
return self
def __next__(self):
with self.lock:
return self.it.__next__()
def threadsafe_generator(f):
"""A decorator that takes a generator function and makes it thread-safe."""
def g(*a, **kw):
return threadsafe_iter(f(*a, **kw))
return g
@threadsafe_generator
def id_generator(n):
result = 1
_lock = threading.Lock()
while True:
with _lock:
if result >= n:
result = 1
yield result
with _lock:
result += 1
id_gen = id_generator(100)
......
other code
The recommendation "Do not change global variables" in controllers.py is not very clear
py4web is a normal threaded program
What (and why) won't work if I use a global-threadsafe-list or global-threadsafe-counter in controllers.py
and I change these variables in the app-controllers ?
Thanks in advance for the explanation
# controllers.py
# 1------------------------------------
class SafeList:
def __init__(self):
self._list = list()
self._lock = threading.Lock()
def append(self, value):
with self._lock:
self._list.append(value)
def check(self, value):
with self._lock:
return value in self._list
def remove(self, value):
with self._lock:
# append the value
self._list.remove(value)
def pop(self):
with self._lock:
return self._list.pop()
def get(self, index):
# acquire the lock
with self._lock:
# read a value at the index
return self._list[index]
def __call__(self):
with self._lock:
return self._list
@property
def all(self):
with self._lock:
return self._list
def length(self):
# acquire the lock
with self._lock:
return len(self._list)
yield_id_list = SafeList()
# 2------------------------------------
class threadsafe_iter:
"""Takes an iterator/generator and makes it thread-safe by
serializing call to the `next` method of given iterator/generator.
"""
def __init__(self, it):
self.it = it
self.lock = threading.Lock()
def __iter__(self):
return self
def __next__(self):
with self.lock:
return self.it.__next__()
def threadsafe_generator(f):
"""A decorator that takes a generator function and makes it thread-safe."""
def g(*a, **kw):
return threadsafe_iter(f(*a, **kw))
return g
@threadsafe_generator
def id_generator(n):
result = 1
_lock = threading.Lock()
while True:
with _lock:
if result >= n:
result = 1
yield result
with _lock:
result += 1
id_gen = id_generator(100)
......
other code
Val K
May 23, 2022, 3:32:05 PM5/23/22
to py4web
About globals/threads:
if it would be so simple we won't need things like redis (sqlite in-memory is faster) The problem is that there may be more than one worker/process and each one will have its own set of globals and these sets should be interchangeable in terms of usage i.e. you cant store request dependent data (like a session or counter) in globals without synchronization which leads to use SSOT (file, db, redis, ...)
if it would be so simple we won't need things like redis (sqlite in-memory is faster) The problem is that there may be more than one worker/process and each one will have its own set of globals and these sets should be interchangeable in terms of usage i.e. you cant store request dependent data (like a session or counter) in globals without synchronization which leads to use SSOT (file, db, redis, ...)
понедельник, 23 мая 2022 г. в 09:39:14 UTC+3, ab9...@gmail.com:
Alexander Beskopilny
May 24, 2022, 9:33:20 AM5/24/22
to py4web
Luca and Val thank you for the answers!
if I understood correctly, then by monitoring synchronization between
threads and requests, it is quite possible to change global variables.
(I tried , it's not difficult in long-lived connections (polling or sse),
for example when using a generator inside a controller )
if I understood correctly, then by monitoring synchronization between
threads and requests, it is quite possible to change global variables.
(I tried , it's not difficult in long-lived connections (polling or sse),
for example when using a generator inside a controller )
Val K
May 24, 2022, 11:34:54 AM5/24/22
to py4web
@Alex
I mean sync globals between processes (workers), not threads. Just try to run gunicorn (or any other wsgi server) with more than one worker + concurrent requests load. 2 workers == 2 independent py4web instances (with all apps, fixtures, threads, etc) , which worker will process next request - managed by wsgi server
вторник, 24 мая 2022 г. в 16:33:20 UTC+3, ab9...@gmail.com:
Alexander Beskopilny
May 24, 2022, 2:05:23 PM5/24/22
to py4web
I'm using a small thread monitor
workers is threads
1 before running generators from browser (apps loaded)
========== worker_info
/home/w3p/set7-py39/py4web/apps/ssep4w/controllers.py
apps.ssep4w.controllers
-- called from: <module>
<module> was called from: _call_with_frames_removed --
name: MainThread
ident: 140680821565248
sys-ident: 8963
MainThread
2 after running 6 connections from firefox
(6 connections to one domain- is firefox limit)
========== worker_info
/home/w3p/set7-py39/py4web/apps/ssep4w/controllers.py
apps.ssep4w.controllers
-- called from: generate_time_data
generate_time_data was called from: __next__ --
name: ThreadPoolExecutor-0_5
ident: 140680361924352
linux-thread-ident: 9409
MainThread
ThreadPoolExecutor-0_0
ThreadPoolExecutor-0_1
ThreadPoolExecutor-0_2
ThreadPoolExecutor-0_3
ThreadPoolExecutor-0_4
ThreadPoolExecutor-0_5
--------------------------------------------
if you call the generator with the command line utility,
you can see several hundred workers-threads
source: https://github.com/ali96343/lvsio
workers is threads
1 before running generators from browser (apps loaded)
========== worker_info
/home/w3p/set7-py39/py4web/apps/ssep4w/controllers.py
apps.ssep4w.controllers
-- called from: <module>
<module> was called from: _call_with_frames_removed --
name: MainThread
ident: 140680821565248
sys-ident: 8963
MainThread
2 after running 6 connections from firefox
(6 connections to one domain- is firefox limit)
========== worker_info
/home/w3p/set7-py39/py4web/apps/ssep4w/controllers.py
apps.ssep4w.controllers
-- called from: generate_time_data
generate_time_data was called from: __next__ --
name: ThreadPoolExecutor-0_5
ident: 140680361924352
linux-thread-ident: 9409
MainThread
ThreadPoolExecutor-0_0
ThreadPoolExecutor-0_1
ThreadPoolExecutor-0_2
ThreadPoolExecutor-0_3
ThreadPoolExecutor-0_4
ThreadPoolExecutor-0_5
--------------------------------------------
if you call the generator with the command line utility,
you can see several hundred workers-threads
source: https://github.com/ali96343/lvsio
Alexander Beskopilny
May 25, 2022, 4:32:40 AM5/25/22
to py4web
@Val
I wrote incorrectly
worker != thread
worker == thread + loop-with-yield + connection
Thanks
I wrote incorrectly
worker != thread
worker == thread + loop-with-yield + connection
Thanks
Luca de Alfaro
May 26, 2022, 5:10:23 PM5/26/22
to Val K, py4web
In other words, global variables are good for storing things that are
global to a thread (for example, connections to the db that need to be
pooled). They are not good for storing global state of the web
server, for the reasons Val outlined. That is better kept in the db,
using db transactions to secure it against concurrent modifications.
Depending on how you host your app, you cannot even be sure that all
threads are on the same underlying host (eg. scaling web hosting).
Luca
global to a thread (for example, connections to the db that need to be
pooled). They are not good for storing global state of the web
server, for the reasons Val outlined. That is better kept in the db,
using db transactions to secure it against concurrent modifications.
Depending on how you host your app, you cannot even be sure that all
threads are on the same underlying host (eg. scaling web hosting).
Luca
> --
> You received this message because you are subscribed to the Google Groups "py4web" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to py4web+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/py4web/a94322b8-a7c7-47f2-aeb2-96107955d21cn%40googlegroups.com.
> You received this message because you are subscribed to the Google Groups "py4web" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to py4web+un...@googlegroups.com.
Luca de Alfaro
May 26, 2022, 5:10:30 PM5/26/22
to Alexander Beskopilny, py4web
Approximate answer:
In web2py, controllers.py is loaded at each request, so every value
you give to a variable is request-specific.
In py4web, threads are long-lived and reused for multiple requests, so
you cannot trust that something thread-specific is also
request-specific.
Luca
In web2py, controllers.py is loaded at each request, so every value
you give to a variable is request-specific.
In py4web, threads are long-lived and reused for multiple requests, so
you cannot trust that something thread-specific is also
request-specific.
Luca
> --
> You received this message because you are subscribed to the Google Groups "py4web" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to py4web+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/py4web/4e639c0e-d5f3-4bb2-a6ae-af05dd9d8f35n%40googlegroups.com.
> You received this message because you are subscribed to the Google Groups "py4web" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to py4web+un...@googlegroups.com.
Krzysztof Socha
Dec 20, 2023, 10:05:11 AM12/20/23
to py4web
I have been struggling with my aging web2py apps for a while now. Thanks Jim for an excellent suggestions supported by a working example! This is great!
Also - Massimo - maybe it would be useful to update the web2py page to at least _inform_ existing and potential new users of alternatives such as py4web? While I _really enjoyed_ creating applications with web2py, I feel that current lack of support and even _information_ is a bit disappointing. Maybe defining an end-of-life for web2py would make sense?
Also - Massimo - maybe it would be useful to update the web2py page to at least _inform_ existing and potential new users of alternatives such as py4web? While I _really enjoyed_ creating applications with web2py, I feel that current lack of support and even _information_ is a bit disappointing. Maybe defining an end-of-life for web2py would make sense?
Cheers,
Krzysztof.
Massimo
Dec 28, 2023, 3:03:46 AM12/28/23
to py4web
Let me give this some thought for a new days
Reply all
Reply to author
Forward
0 new messages