clean up resources after interrupted page load

[Anatoliy Kushner]

With sample code below if subscriber interrupt page load somewhere between client.connect and client.close() then cherrypy will hang and will not clean up used resources like: 1) open tcp socket (from netstat -anpt on this server i see tcp connections to server that is established for some days, while time to executing any commands should finish in 10 sec) 2) from ps -p 'cherrypy pid' H , i see , that number of threads increased, and number of threads equals number of unclosed tcp connections

while debugging this i found that thread stop at "yield bytes(a, 'utf-8')" in while loop. So , is there any way to say cherrypy to kill threads after some timeout ?

P.S. response.timeout = 30 already set in config and this does not help

@cherrypy.expose

def test(self, *args, **kwargs):
 ...
 ...
 def getData(host, cmd):
   html = open("menuRp1.html").read()
   yield bytes(html, 'utf-8')
   client = paramiko.SSHClient()
   client.connect(hostname=host, username=user, password=pwd, port=22)
   stdin, stdout, stderr = client.exec_command(cmd)
   a=stdout.readline()
   while a:
     yield bytes(a, 'utf-8')
     a=stdout.readline()
   client.close()
   html = open("menuRp2.html").read()
   yield bytes(html, 'utf-8')
 return getData(h, c)
test._cp_config = {'response.stream': True} 

[Sviatoslav Sydorenko]

Timeout feature has been deprecated because it's not really possible to cancel threads. The solution would be to rewrite the engine to use async paradigm.

Try submitting an issue against Cheroot with a minimal reproducer. There's some WIP there which aims to improve the connection management.

--Sviatoslav.

Sent from my phone, please pardon any typos.

сб, 29 черв. 2019, 10:05 користувач Anatoliy Kushner <lama...@gmail.com> пише:

--
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cherrypy-user...@googlegroups.com.
To post to this group, send email to cherryp...@googlegroups.com.
Visit this group at https://groups.google.com/group/cherrypy-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/cherrypy-users/92d0f922-5e40-4a5c-b3d6-a99d10ff0830%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Robert Brewer]

In Python, you have to explicitly cancel threads inside the thread. So one thing you _can_ do is add a line inside your while loop: "if response.timed_out: raise cherrypy.TimeoutError()". It won't do anything if the thread blocks forever on readline(), but it will allow the thread to exit as long as readline is yielding.

To view this discussion on the web visit https://groups.google.com/d/msgid/cherrypy-users/CAFYONRBoAZFtBsPj1xs3GzLDac5iCvww5zd_wVbEPs4gVs_pGQ%40mail.gmail.com.

[Sviatoslav Sydorenko]

It seems to be the case that it blocks on socket interactions...

This will supposedly improve the situation https://github.com/cherrypy/cheroot/pull/176

--Sviatoslav.

Sent from my phone, please pardon any typos.

сб, 29 черв. 2019, 20:34 користувач Robert Brewer <amin...@gmail.com> пише:

To view this discussion on the web visit https://groups.google.com/d/msgid/cherrypy-users/CAJ4gti-fbZrzM-mfy7_OKDuTqBVLCJWWZ11LKOr9eB6d3tomDg%40mail.gmail.com.