pymysql gives Broken Pipe

[Fran]

I have been getting Broken Pipe issues with pymysql:

Traceback (most recent call last):

  File "/home/web2py/gluon/main.py", line 470, in wsgibase

    BaseAdapter.close_all_instances('commit')

  File "/home/web2py/gluon/dal.py", line 272, in close_all_instances

    getattr(instance,action)()

  File "/home/web2py/gluon/dal.py", line 1036, in commit

    return self.connection.commit()

  File "/home/web2py/gluon/contrib/pymysql/connections.py", line 562, in commit

    self.errorhandler(None, exc, value)

  File "/home/web2py/gluon/contrib/pymysql/connections.py", line 182, in defaulterrorhandler

    raise Error(errorclass, errorvalue)

Error: (<class 'socket.error'>, error(32, 'Broken pipe'))

Everything is fine if I refresh the page, which I guess means that we establish a fresh MySQL connection rather than trying to reuse an old one (This is a quiet site)....however tracebacks are horrible for users to face, so this really needs fixing.

I guess this has been happening ever since the move to pymysql, but I've not had production sites using it & generally my development is done on sqlite, so I hadn't noticed.

This is with pool_size = 30 & a default MySQL Debian Squeeze install.

I see someone else has experienced it occasionally:

https://groups.google.com/forum/?pli=1#!searchin/web2py/Broken$20pipe$20pymysql/web2py/W8xs1Vx38jo/Hz-BeQyMhfgJ

I suspect that this would best be fixed in pymysql, but we should at least be catching in web2py anyway.

I'm tempted by the same request as this user:

https://groups.google.com/d/msg/web2py/GXyyuBdrtyg/up0eZ1nkpQYJ

Portability of pymysql is kinda nice, but it's really no big deal to install the proven MySQLdb.

Many thanks,
Fran.

[Vasile Ermicioi]

I see no reason for you to use pool_size, 

I think you will avoid this error by not using it, 

and if you say that you have little traffic than I see almost no reason to use pool_size

[Fran]

Thanks - fine for this currently low-volume site now but we definitely want to take advantage of this feature for scalability (the site may suddenly start getting a lot of traffic):

http://www.web2py.com/book/default/chapter/06#Connection-Pooling

Also see:

http://www.web2py.com/book/default/chapter/06#Connection-Failures

The problem is known, but it seems web2py's retry mechanism isn't working any more...

F

[Massimo Di Pierro]

Here is a proposed solution in trunk

import mysqldb

from gluon.dal import MySQLAdapter

MySQLAdapter.adapter=mysqldb

before calling db=DAL('mysql://...')

[Fran Boon]

That seems very workable :)

Thanks a lot,
Fran.

[Michele Comitini]

I encountered the *broken pipe* error, when shutting down mysql server
and restarting it.
Is a pooled connection checked as active before being passed to the
DAL in the active process?

mic

2011/2/6 Fran Boon <franc...@gmail.com>:

[Fran Boon]

> On 6 February 2011 05:15, Massimo Di Pierro <massimo....@gmail.com> wrote:
>> Here is a proposed solution in trunk
>> import mysqldb
>> from gluon.dal import MySQLAdapter
>> MySQLAdapter.adapter=mysqldb
>> before calling db=DAL('mysql://...')

I'm not sure this is being read everywhere, since I just got a broken
pipe on a MySQLdb install:

Traceback (most recent call last):

File "/home/web2py/gluon/main.py", line 532, in wsgibase
BaseAdapter.close_all_instances('rollback')
File "/home/web2py/gluon/dal.py", line 285, in close_all_instances
getattr(instance,action)()
File "/home/web2py/gluon/dal.py", line 1058, in rollback
return self.connection.rollback()
File "/home/web2py/gluon/contrib/pymysql/connections.py", line 571,
in rollback

self.errorhandler(None, exc, value)
File "/home/web2py/gluon/contrib/pymysql/connections.py", line 182,
in defaulterrorhandler
raise Error(errorclass, errorvalue)
Error: (<class 'socket.error'>, error(32, 'Broken pipe'))

Cheers,
F

[Bilal Hasan]

Have there been any updates on this? 

I am in production environment using Web2Py 2.8.2 and this problem popped up randomly.

[Ben Lawrence]

I am seeing broken pipe errors like this on low volume site using pg8000 on web2py Version 2.11.2-stable+timestamp.2015.05.30.16.33.24

Should we use this http://stackoverflow.com/a/180922/3950334 

Error::

<class 'socket.error'> [Errno 32] Broken pipe

• File /home/www-data/web2py/gluon/contrib/pg8000/core.py in _close at line 1438 

  File /usr/lib/python2.7/socket.py in flush at line 303

  Function argument list

• (self=<socket._fileobject object>)

  Code listing

  298.
  299.
  300.
  301.
  302.
  303.
  
  304.
  305.
  306.
  307.
              data_size = len(data)
              write_offset = 0
              view = memoryview(data)
              try:
                  while write_offset < data_size:
                      self._sock.sendall(view[write_offset:write_offset+buffer_size])
  
                      write_offset += buffer_size
              finally:
                  if write_offset < data_size:
                      remainder = data[write_offset:]

[Derek]

are you on osx perchance?

[Ben Lawrence]

No, (Running on Apache/2.4.7 (Ubuntu), Python 2.7.6)