Nginx, uwsgi, on Windows

[Kiran Subbaraman]

Hello all,
I use Windows 10 based laptop for my development. By default, web2py serves its content via the rocket server.
My application makes use of the 'default' redis client () to connect to the local redis server.
I started noticing a stacktrace, in the cmd prompt, from within which web2py has started:

Created by Massimo Di Pierro, Copyright 2007-2016
Version 2.14.3-stable+timestamp.2016.03.26.17.54.43
Database drivers available: psycopg2, pymysql, imaplib, sqlite3, pg8000, pyodbc, mysqlconnector
please visit:
        http://127.0.0.1:8000/
starting browser...
please visit:
        http://127.0.0.1:8000/
starting browser...
2016-04-20 20:26:54,003 - Rocket.Errors.Port8000 - ERROR - Traceback (most recent call last):
  File "d:\code\git\web2py\gluon\rocket.py", line 587, in listen
    sock, addr = self.listener.accept()
  File "D:\programs\open\python27\lib\site-packages\gevent\socket.py", line 316, in accept
    self._wait(self._read_event)
  File "D:\programs\open\python27\lib\site-packages\gevent\socket.py", line 300, in _wait
    self.hub.wait(watcher)
  File "D:\programs\open\python27\lib\site-packages\gevent\hub.py", line 348, in wait
    result = waiter.get()
  File "D:\programs\open\python27\lib\site-packages\gevent\hub.py", line 575, in get
    return self.hub.switch()
  File "D:\programs\open\python27\lib\site-packages\gevent\hub.py", line 338, in switch
    return greenlet.switch(self)
LoopExit: This operation would block forever

The error in the error-ticket is:

<class 'gevent.hub.LoopExit'> This operation would block forever
Version
web2py™     Version 2.14.3-stable+timestamp.2016.03.26.17.54.43
Python     Python 2.7.10: D:\programs\open\python27\python.exe (prefix: D:\programs\open\python27)
Traceback


Traceback (most recent call last):
  File "d:\code\git\web2py\gluon\restricted.py", line 227, in restricted
    exec ccode in environment
  File "d:/code/git/web2py/applications/myapp/controllers/myservice.py", line 99, in <module>
  File "d:\code\git\web2py\gluon\globals.py", line 417, in <lambda>
    self._caller = lambda f: f()
  File "d:\code\git\web2py\gluon\cache.py", line 661, in wrapped_f
    rtn = func()
  File "d:/code/git/web2py/applications/myapp/controllers/myservice.py", line 68, in data
    data = api.get_data(url)
  File "applications\myapp\modules\data\api.py", line 145, in get_data
    return _get_data_for(url) if url else None
  File "applications\myapp\modules\data\api.py", line 164, in _get_data_for
    force_reload=rebuild
  File "applications\myapp\modules\myapp_constants.py", line 460, in get_from_cache
    cache_value = redis_cache.get(key)
  File "D:\programs\open\python27\lib\site-packages\redis\client.py", line 880, in get
    return self.execute_command('GET', name)
  File "D:\programs\open\python27\lib\site-packages\redis\client.py", line 573, in execute_command
    return self.parse_response(connection, command_name, **options)
  File "D:\programs\open\python27\lib\site-packages\redis\client.py", line 585, in parse_response
    response = connection.read_response()
  File "D:\programs\open\python27\lib\site-packages\redis\connection.py", line 577, in read_response
    response = self._parser.read_response()
  File "D:\programs\open\python27\lib\site-packages\redis\connection.py", line 238, in read_response
    response = self._buffer.readline()
  File "D:\programs\open\python27\lib\site-packages\redis\connection.py", line 168, in readline
    self._read_from_socket()
  File "D:\programs\open\python27\lib\site-packages\redis\connection.py", line 126, in _read_from_socket
    data = self._sock.recv(socket_read_size)
  File "D:\programs\open\python27\lib\site-packages\gevent\socket.py", line 394, in recv
    self._wait(self._read_event)
  File "D:\programs\open\python27\lib\site-packages\gevent\socket.py", line 300, in _wait
    self.hub.wait(watcher)
  File "D:\programs\open\python27\lib\site-packages\gevent\hub.py", line 348, in wait
    result = waiter.get()
  File "D:\programs\open\python27\lib\site-packages\gevent\hub.py", line 575, in get
    return self.hub.switch()
  File "D:\programs\open\python27\lib\site-packages\gevent\hub.py", line 338, in switch
    return greenlet.switch(self)
LoopExit: This operation would block forever

I do not see this issue on my ubuntu linux deployment.

I suspected, without any evidence, that it may have to do with process/threading between the rocket server to redis_client, and that it could do with the monkey_patching of socket in the gevent world. Again, I do not have any evidence, other than the stacktrace.

The knee-jerk reaction was to see if using uwsgi or nginx on windows alleviates this issue. The nearest discussion thread, that I came across, that talks about setting up such an environment (web2py without rocket, but uwsgi / nginx) is this one: https://groups.google.com/forum/#!searchin/web2py/nginx$20windows/web2py/NdV0CVX8zZs/wWiRxogLz4sJ

Any thoughts on what I could do or take a look at will help. Thanks.

In the meantime, I also intend to

• create a minimal app that is able to recreate this issue.
  
• take a look at the web2py's redis-cache.

[Niphlod]

forget about uwsgi on windows. never really landed as production-ready. fortunately iis can run python without any hiccup and it's the current recommended way to deploy web2py in production on windows.

that being said, I'm really eager to know how the hell you're getting gevent mixed up with web2py's internal webserver, which is not at all compatible with gevent.

[Kiran Subbaraman]

Thanks for the recommendation - IIS with web2py is a recommended deployment on Windows.

The gevent-mix-up-with-rocket, was based on the stack trace I pasted below. And like I mentioned, this connection was made without any evidence ... so yeah, I was writing this before I gave it too much thought or investigation.
In any case, I'll do some more investigation and try to track down this issue.

________________________________________
Kiran Subbaraman
http://subbaraman.wordpress.com/about/

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kiran Subbaraman]

Narrowed down the issue. Basically, the grequests modules that I was using seems to be the cause of the "LoopExit: This operation would block forever"

Create an application: redis_event

controller: default.py

import cache_mod

def index():
    return dict(message=cache_mod.get_value())

module: cache_mod.py

import redis
import grequests

redis_cache = redis.StrictRedis(host='localhost', port=6379, db=0)

def get_value(key='SOME_KEY'):
    return redis_cache.get(key)

view: index.html

<h2>{{=message}}</h2>

Once deployed on the server, access this url: http://127.0.0.1:8000/redis_event/default/index
Reload the page successively - repeatedly F5 after page load - and you will notice the error on the page.

Am moving away from grequests, and settling for just requests.

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscribe@googlegroups.com.

[Niphlod]

why would you use a library made to use gevent in an environment that doesn't support it ?

if you need to use gevent-coroutines-etc-etc-etc you need to have the COMPLETE env supporting it, not just a piece.

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.

[Kiran Subbaraman]

The module (which included grequests) was being used in a web2py and non-web2py context. The grequest capability was coming into play in the latter context.
Refactored by separating out this responsibility.

________________________________________
Kiran Subbaraman
http://subbaraman.wordpress.com/about/