KeyboardInterrupt catch does not shut down the socketserver

[Igor Katson]

I have problems in getting a SocketServer to shutdown. Why does this not
actually stop the application?

from SocketServer import UnixStreamServer, BaseRequestHandler

server = UnixStreamServer('/tmp/ss.sock', BaseRequestHandler)
try:
server.serve_forever()
except KeyboardInterrupt:
server.shutdown()

After that the server does not respond any more, but the application hangs.

What's the proper way to shutdown the socketserver and what is my mistake?

[Lawrence D'Oliveiro]

In message <mailman.183.1242371...@python.org>, Igor Katson
wrote:

> I have problems in getting a SocketServer to shutdown.

Do you want to do a shutdown or a close?

[Igor Katson]

I want the server close the socket, and the program to continue after
that (in this case, just to terminate).

[Lawrence D'Oliveiro]

In message <mailman.185.1242375...@python.org>, Igor Katson
wrote:

> Lawrence D'Oliveiro wrote:
>
>> In message <mailman.183.1242371...@python.org>, Igor
>> Katson wrote:
>>
>>> I have problems in getting a SocketServer to shutdown.
>>
>> Do you want to do a shutdown or a close?
>>

> I want the server close the socket ...

You want to do a close, do a close, not a shutdown
<http://docs.python.org/library/socket.html>.

[Igor Katson]

Lawrence D'Oliveiro wrote:
> In message <mailman.185.1242375...@python.org>, Igor Katson
> wrote:
>
>
>> Lawrence D'Oliveiro wrote:
>>
>>
>>> In message <mailman.183.1242371...@python.org>, Igor
>>> Katson wrote:
>>>
>>>
>>>> I have problems in getting a SocketServer to shutdown.
>>>>
>>> Do you want to do a shutdown or a close?
>>>
>>>
>> I want the server close the socket ...
>>
>
> You want to do a close, do a close, not a shutdown
> <http://docs.python.org/library/socket.html>.
>
>
>

Shutdown implies closing the listening socket, doesn't it?

[Gabriel Genellina]

En Fri, 15 May 2009 09:04:05 -0300, Igor Katson escribi�:

> Lawrence D'Oliveiro wrote:
>> In message <mailman.185.1242375...@python.org>, Igor
>> Katson wrote:
>>> Lawrence D'Oliveiro wrote:
>>>> In message <mailman.183.1242371...@python.org>, Igor
>>>> Katson wrote:
>>>>
>>>>> I have problems in getting a SocketServer to shutdown.
>>>>>
>>>> Do you want to do a shutdown or a close?
>>>>
>>> I want the server close the socket ...
>>>
>>
>> You want to do a close, do a close, not a shutdown
>> <http://docs.python.org/library/socket.html>.
>>

> Shutdown implies closing the listening socket, doesn't it?

No (perhaps it should, but that is another issue). There is a
documentation bug; BaseServer.shutdown is documented as "Tells the
serve_forever() loop to stop and waits until it does." [1]
The docstring is much more explicit: """Stops the serve_forever loop.
Blocks until the loop has finished. This must be called while
serve_forever() is running in another thread, or it will deadlock."""

So, if you have a single-threaded server, *don't* use shutdown(). And, to
orderly close the listening socket, use server_close() instead. Your code
would become:

from SocketServer import TCPServer, BaseRequestHandler

server = TCPServer(('localhost',1234), BaseRequestHandler)
try:
server.serve_forever()
except KeyboardInterrupt:
print "^C detected"
pass
finally:
print "server_close()"
server.server_close()
print "bye"

(I've opened http://bugs.python.org/issue6031 )

[1]
http://docs.python.org/dev/library/socketserver.html?highlight=baseserver#SocketServer.BaseServer.shutdown

--
Gabriel Genellina

[Igor Katson]

Gabriel Genellina wrote:
> En Fri, 15 May 2009 09:04:05 -0300, Igor Katson escribi�:

>> Lawrence D'Oliveiro wrote:
>>> In message <mailman.185.1242375...@python.org>,
>>> Igor Katson wrote:
>>>> Lawrence D'Oliveiro wrote:
>>>>> In message <mailman.183.1242371...@python.org>,
>>>>> Igor Katson wrote:
>>>>>
>>>>>> I have problems in getting a SocketServer to shutdown.
>>>>>>
>>>>> Do you want to do a shutdown or a close?
>>>>>
>>>> I want the server close the socket ...
>>>>
>>>
>>> You want to do a close, do a close, not a shutdown
>>> <http://docs.python.org/library/socket.html>.
>>>

>> Shutdown implies closing the listening socket, doesn't it?
>
> No (perhaps it should, but that is another issue). There is a
> documentation bug; BaseServer.shutdown is documented as "Tells the
> serve_forever() loop to stop and waits until it does." [1]
> The docstring is much more explicit: """Stops the serve_forever loop.
> Blocks until the loop has finished. This must be called while
> serve_forever() is running in another thread, or it will deadlock."""
>
> So, if you have a single-threaded server, *don't* use shutdown(). And, to
> orderly close the listening socket, use server_close() instead. Your code
> would become:
>
> from SocketServer import TCPServer, BaseRequestHandler
>

> server = TCPServer(('localhost',1234), BaseRequestHandler)
> try:
> server.serve_forever()
> except KeyboardInterrupt:

> print "^C detected"
> pass
> finally:
> print "server_close()"
> server.server_close()
> print "bye"
>
> (I've opened http://bugs.python.org/issue6031 )
>
> [1]
> http://docs.python.org/dev/library/socketserver.html?highlight=baseserver#SocketServer.BaseServer.shutdown
>
>

Thanks, Gabriel, that's exactly what I was waiting for.

[Igor Katson]

Gabriel Genellina wrote:
> En Fri, 15 May 2009 09:04:05 -0300, Igor Katson escribi�:

>> Lawrence D'Oliveiro wrote:
>>> In message <mailman.185.1242375...@python.org>,
>>> Igor Katson wrote:
>>>> Lawrence D'Oliveiro wrote:
>>>>> In message <mailman.183.1242371...@python.org>,
>>>>> Igor Katson wrote:
>>>>>
>>>>>> I have problems in getting a SocketServer to shutdown.
>>>>>>
>>>>> Do you want to do a shutdown or a close?
>>>>>
>>>> I want the server close the socket ...
>>>>
>>>
>>> You want to do a close, do a close, not a shutdown
>>> <http://docs.python.org/library/socket.html>.
>>>

>> Shutdown implies closing the listening socket, doesn't it?
>
> No (perhaps it should, but that is another issue). There is a
> documentation bug; BaseServer.shutdown is documented as "Tells the
> serve_forever() loop to stop and waits until it does." [1]
> The docstring is much more explicit: """Stops the serve_forever loop.
> Blocks until the loop has finished. This must be called while
> serve_forever() is running in another thread, or it will deadlock."""
>
> So, if you have a single-threaded server, *don't* use shutdown(). And, to
> orderly close the listening socket, use server_close() instead. Your code
> would become:
>
> from SocketServer import TCPServer, BaseRequestHandler
>

> server = TCPServer(('localhost',1234), BaseRequestHandler)
> try:
> server.serve_forever()
> except KeyboardInterrupt:

> print "^C detected"
> pass
> finally:
> print "server_close()"
> server.server_close()
> print "bye"
>
> (I've opened http://bugs.python.org/issue6031 )
>
> [1]
> http://docs.python.org/dev/library/socketserver.html?highlight=baseserver#SocketServer.BaseServer.shutdown
>
>

Hmm. Gabriel, could you please show the same for the threaded version?
This one deadlocks:

from SocketServer import TCPServer, BaseRequestHandler

from threading import Thread

server = TCPServer(('localhost',1234), BaseRequestHandler)
try:

run_thread = Thread(target=server.serve_forever)
run_thread.start()
run_thread.join()

except KeyboardInterrupt:
print "^C detected"
pass
finally:

print "server_shutdown()"
kill_thread = Thread(target=server.shutdown)
kill_thread.start()
print "bye"

[Gabriel Genellina]

En Sat, 16 May 2009 04:04:03 -0300, Igor Katson <desce...@gmail.com>
escribiￜ:
> Gabriel Genellina wrote:
>> En Fri, 15 May 2009 09:04:05 -0300, Igor Katson escribiￜ:

>>> Lawrence D'Oliveiro wrote:
>>>> In message <mailman.185.1242375...@python.org>, Igor
>>>> Katson wrote:
>>>>> Lawrence D'Oliveiro wrote:
>>>>>> In message <mailman.183.1242371...@python.org>,
>>>>>> Igor Katson wrote:
>>>>>>
>>>>>>> I have problems in getting a SocketServer to shutdown.

>>> Shutdown implies closing the listening socket, doesn't it?
>>
>> No (perhaps it should, but that is another issue). There is a
>> documentation bug; BaseServer.shutdown is documented as "Tells the
>> serve_forever() loop to stop and waits until it does." [1]
>> The docstring is much more explicit: """Stops the serve_forever loop.
>> Blocks until the loop has finished. This must be called while
>> serve_forever() is running in another thread, or it will deadlock."""
>>
>> So, if you have a single-threaded server, *don't* use shutdown(). And,
>> to orderly close the listening socket, use server_close() instead. Your

> Hmm. Gabriel, could you please show the same for the threaded version?
> This one deadlocks:
> [code removed]

The shutdown method should *only* be called while serve_forever is
running. If called after server_forever exited, shutdown() blocks forever.

<code>

from SocketServer import TCPServer, BaseRequestHandler
from threading import Thread

server = TCPServer(('localhost',1234), BaseRequestHandler)

run_thread = Thread(target=server.serve_forever)
run_thread.start()
try:
print "press ^C to exit"
import time; time.sleep(30)

except KeyboardInterrupt:
print "^C detected"
pass

server.shutdown()
run_thread.join()
print "bye"
</code>

But, what are you after, exactly? I think I'd use the above code only in a
GUI application with a background server.
There are other alternatives, like asyncore or Twisted.

--
Gabriel Genellina

[Igor Katson]

Gabriel Genellina wrote:
> En Sat, 16 May 2009 04:04:03 -0300, Igor Katson <desce...@gmail.com>
> escribiￜ:
>> Gabriel Genellina wrote:
>>> En Fri, 15 May 2009 09:04:05 -0300, Igor Katson escribiￜ:

>>>> Lawrence D'Oliveiro wrote:
>>>>> In message <mailman.185.1242375...@python.org>,
>>>>> Igor Katson wrote:
>>>>>> Lawrence D'Oliveiro wrote:
>>>>>>> In message <mailman.183.1242371...@python.org>,
>>>>>>> Igor Katson wrote:
>>>>>>>
>>>>>>>> I have problems in getting a SocketServer to shutdown.

>>>> Shutdown implies closing the listening socket, doesn't it?
>>>
>>> No (perhaps it should, but that is another issue). There is a
>>> documentation bug; BaseServer.shutdown is documented as "Tells the
>>> serve_forever() loop to stop and waits until it does." [1]
>>> The docstring is much more explicit: """Stops the serve_forever loop.
>>> Blocks until the loop has finished. This must be called while
>>> serve_forever() is running in another thread, or it will deadlock."""
>>>
>>> So, if you have a single-threaded server, *don't* use shutdown().
>>> And, to orderly close the listening socket, use server_close()
>>> instead. Your
>
>> Hmm. Gabriel, could you please show the same for the threaded
>> version? This one deadlocks:
>> [code removed]
>
> The shutdown method should *only* be called while serve_forever is
> running. If called after server_forever exited, shutdown() blocks
> forever.
>

> [code removed]

> But, what are you after, exactly? I think I'd use the above code only
> in a GUI application with a background server.
> There are other alternatives, like asyncore or Twisted.

For now, I am just using server.server_close() and it works. The server
itself is an external transaction manager for PostgreSQL, when a client
connects to it, serialized data interchange beetween the server and the
client starts, e.g. first the client sends data, then the server sends
data, then again the client, then the server and so on.
I haven't used asyncore or Twisted yet, and didn't know about their
possible usage while writing the project. I'll research in that direction.