Scheduler workers die on python 3 + Ubuntu 18.04

[Antonio Salazar]

Edit: created github issue, ignore this

Using python 2 on the same system, the scheduler is rock-solid, but using python 3 the workers die after processing a task. The rest of the tasks are left in queue and the worker entry is deleted from the scheduler_worker table. The scheduler also works fine on Windows, on python 2 and 3, same database.

Running the scheduler interactively (either as root or unprivileged user) throws the following error, although the task is completed. Any queued task after that remains queued.

$ sudo python3 /var/www/web2py/web2py.py -K extranet
web2py Web Framework
Created by Massimo Di Pierro, Copyright 2007-2018
Version 2.17.1-stable+timestamp.2018.08.06.01.02.56
Database drivers available: sqlite3, psycopg2, imaplib, pymysql, pyodbc
starting single-scheduler for "extranet"...
ERROR:web2py.scheduler.vavextranetqa#7733:Error retrieving status
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/var/www/web2py/gluon/scheduler.py", line 1168, in send_heartbeat
    mybackedstatus = db(sw.worker_name == self.worker_name).select().first()
  File "/var/www/web2py/gluon/packages/dal/pydal/objects.py", line 2277, in select
    return adapter.select(self.query, fields, attributes)
  File "/var/www/web2py/gluon/packages/dal/pydal/adapters/base.py", line 763, in select
    return self._select_aux(sql, fields, attributes, colnames)
  File "/var/www/web2py/gluon/packages/dal/pydal/adapters/base.py", line 719, in _select_aux
    rows = self._select_aux_execute(sql)
  File "/var/www/web2py/gluon/packages/dal/pydal/adapters/base.py", line 713, in _select_aux_execute
    self.execute(sql)
  File "/var/www/web2py/gluon/packages/dal/pydal/adapters/__init__.py", line 67, in wrap
    return f(*args, **kwargs)
  File "/var/www/web2py/gluon/packages/dal/pydal/adapters/base.py", line 413, in execute
    rv = self.cursor.execute(command, *args[1:], **kwargs)
psycopg2.OperationalError: SSL connection has been closed unexpectedly


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/var/www/web2py/gluon/scheduler.py", line 637, in run
    self.send_heartbeat(counter)
  File "/var/www/web2py/gluon/scheduler.py", line 1239, in send_heartbeat
    db.rollback()
  File "/var/www/web2py/gluon/packages/dal/pydal/base.py", line 708, in rollback
    self._adapter.rollback()
  File "/var/www/web2py/gluon/packages/dal/pydal/adapters/__init__.py", line 56, in wrap
    return f(*args, **kwargs)
  File "/var/www/web2py/gluon/packages/dal/pydal/adapters/base.py", line 866, in rollback
    return self.connection.rollback()
psycopg2.InterfaceError: connection already closed

DEBUG:web2py.app.extranet:    new task report: COMPLETED

OS is Ubuntu 18.04 server

web2py is 2.17.1

postgresql is 10.5

python-psycopg2 and python3-psycopg2 both are 2.7.4-1.

The scheduler runs as a systemd service like this:

/usr/bin/sudo -u www-data <python> /var/www/web2py/web2py.py -K myapp,myapp:slow,myapp:slowest

where <python> is "python" or "python3"