OSError: [Errno 48] Address already in use

[T Berger]

Why am I getting this error? I'm not sure what additional information I need to supply, so please let me know.

[T Berger]

On Wednesday, June 27, 2018 at 11:49:18 AM UTC-4, T Berger wrote:
> Why am I getting this error? I'm not sure what additional information I need to supply, so please let me know.

I'm working on a Flask webapp. I know there is another google group specifically for Flask. Can you please provide the url.

Thanks

[Joaquin Henriquez]

>Subject: EXTERNAL: OSError: [Errno 48] Address already in use

The best way to help if got you to put the relevant code here.

The error you are experiencing means that the Port you are trying to bind is already taken by another running process.

[T Berger]

On Wednesday, June 27, 2018 at 12:17:28 PM UTC-4, Joaquin Henriquez wrote:
> >Subject: EXTERNAL: OSError: [Errno 48] Address already in use
>
> The best way to help if got you to put the relevant code here.

Last login: Wed Jun 27 12:45:08 on ttys000
192:~ TamaraB$ cd Desktop/Webapp/
192:Webapp TamaraB$ python3 vsearch4web.py
* Serving Flask app "vsearch4web" (lazy loading)
* Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
* Debug mode: on
Traceback (most recent call last):
File "vsearch4web.py", line 18, in <module>
app.run(debug=True)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/flask/app.py", line 943, in run
run_simple(host, port, self, **options)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/werkzeug/serving.py", line 795, in run_simple
s.bind(get_sockaddr(hostname, port, address_family))

OSError: [Errno 48] Address already in use

192:Webapp TamaraB$

> The error you are experiencing means that the Port you are trying to bind is already taken by another running process.

Why would another process be running?

Thanks,

Tamara

[Marko Rauhamaa]

Joaquin Henriquez <joaquin....@countercept.com>:

That error usually takes place when restarting a server. The server
exited with one or more TCP connections open, which left the connections
in a TIME-WAIT state for some minutes. When the server is restarted and
tries to bind its address again, the ghosts of the previous connections
prevent the socket from being bound.

The problem can be solved by turning on the SO_REUSEADDR flag of the
socket.


Marko

[T Berger]

Let me add this information to clarify the context in which I got this error 48. It doesn't make sense to me, and it might not to you.

This morning, I opened my webapp (vsearch4web.py in the terminal code above) and noticed a whole bunch of code I had not typed. I also noticed something weird about the file path in the title bar (it seemed to be in my downloads folder). So I created a completely fresh file, retyped everything, and replaced the older one in the folder with this one.

THEN, I ran the program at my terminal, and got this error 48 in response.

Does any of this make sense?

Thanks,

Tamara

[T Berger]

Hi Marko,

Can you please tell me how to "turn on the SO_REUSEADDR flag of the socket"?

Thanks,

Tamara

[Jim Lee]

On 06/27/18 08:49, T Berger wrote:
> Why am I getting this error? I'm not sure what additional information I need to supply, so please let me know.

You asked this question two weeks ago and got several answers.  Here is
one of them:


On 06/15/18 10:17, Percival John Hackworth wrote:

On 15-Jun-2018, T Berger wrote
(in article<3fe6b3e1-f582-415b...@googlegroups.com>):

> I’m trying to build a webapp with flask. I installed flask, created a
> webapp in IDLE, but when I tried testing it at my terminal, I got a huge
> error message. This is the app:
>
> [snip]
> line 467, in server_bind
> self.socket.bind(self.server_address)

> OSError: [Errno 48] Address already in use

> 192:Webapp TamaraB$
>
> What went wrong?

The last message in the error message "Address already in use" means that
there's a web server using the port already running on your machine. If
you're on a Mac, you already have a local web server running on your machine.
You may or may not have specifically set that up. I intentionally setup the
development version of my web site on my system for testing.

To find out what network ports are in use on your system, you can use
command-line tools to display what processes are listening to what ports. If
none of this makes sense to you, read more on Linux networking, web
programming, and system administration before you attempt to dive into
building a web application with python.

It's like you opened the hood of your car and pointed to a cap marked "710"
and are asking us "what's that?" It's actually where you pour the "OIL" when
you need to top it off.

Good luck.

[Cameron Simpson]

On 27Jun2018 10:44, Tamara Berger <brg...@gmail.com> wrote:
>On Wednesday, June 27, 2018 at 1:40:20 PM UTC-4, Marko Rauhamaa wrote:
>> Joaquin Henriquez <joaquin....@countercept.com>:
>>
>> >>Subject: EXTERNAL: OSError: [Errno 48] Address already in use
>> > The best way to help if got you to put the relevant code here.
>> >
>> > The error you are experiencing means that the Port you are trying to
>> > bind is already taken by another running process.
>>
>> That error usually takes place when restarting a server. The server
>> exited with one or more TCP connections open, which left the connections
>> in a TIME-WAIT state for some minutes. When the server is restarted and
>> tries to bind its address again, the ghosts of the previous connections
>> prevent the socket from being bound.
>>
>> The problem can be solved by turning on the SO_REUSEADDR flag of the
>> socket.
>

>Can you please tell me how to "turn on the SO_REUSEADDR flag of the socket"?

There's an example at the very bottom of the documentation for the "socket"
module:

https://docs.python.org/3/library/socket.html

However, Flash likely creates the server socket for you; I don't know how to
tell it to set the SO_REUSEADDR option, though the flask documentation might.

You can check the status of the system with the netstat(8) command:

netstat -an

lists all the system sockets and connections running on your machine. There
will be many, so if you know the port you're using you can narrow it does with
a grep:

netstat -an | grep 8000

if you were using port 8000.

This also points the way to a workaround: since you're running the script
yourself on an ad hoc basis, change ports! Maybe provide the listening port on
the command line, and just bump it by one when this happens.

Cheers,
Cameron Simpson <c...@cskk.id.au>

[Dan Stromberg]

Here's a small script that might serve as an example:
http://stromberg.dnsalias.org/svn/max-tcp-window/trunk/max-tcp-window

BTW, it's a security feature you're turning off. If you're on a multiuser
box, it prevents a second user from stealing lingering connections from a
first user on the same port.

[T Berger]

Hi Guys,

Thanks for all the suggestions. I found the problem. I had saved my program in IDLE and quit out of the shell, and then tried to run the program in Terminal. Previously, restarting the shell was enough to break the connection with the port. This time, I guess, it wasn't. So after thinking about your suggestions, I bypassed IDLE and went directly to Terminal, and the program worked.

Tamara

[Marko Rauhamaa]

Dan Stromberg <drsa...@gmail.com>:

>> > The problem can be solved by turning on the SO_REUSEADDR flag of
>> > the socket.

> BTW, it's a security feature you're turning off. If you're on a
> multiuser box, it prevents a second user from stealing lingering
> connections from a first user on the same port.

Can you provide a brief proof of concept?


Marko

[Dan Stromberg]

On Wed, Jun 27, 2018 at 8:49 AM, T Berger <brg...@gmail.com> wrote:

> Why am I getting this error? I'm not sure what additional information I
> need to supply, so please let me know.

AIUI, there are 2 possible causes. You either have some other process
listening on the requested port, or a process that Was using it no longer
is but not enough time has passed yet for the kernel to decide that the
port can safely be reused.

The former keeps two processes from trying to get data from the same port
at the same time, which could be confusing.

The latter is a security feature. It keeps person A from starting an imap
server on port tcp/5555, and having person B come along and start a fake,
password-stealing imap server on the same port shortly after person A
terminates theirs (for example). There would otherwise be a window of time
during which B's imap daemon could steal passwords intended for A's imap
daemon, because remote imap clients wouldn't know about the switcheroo.
Note that even if A come back and starts their imap daemon immediately
after a crash, the kernel doesn't know if that is a legitimate or
illegitemate imap daemon, so that is blocked for a while too.

You can check if something else is listening on that port with
http://stromberg.dnsalias.org/~strombrg/What-program-is-active-on-that-port.html
(Linux and Solaris - there will likely be similar tools for other OS's).

You can eliminate the waiting period with SO_REUSEADDR (but if you have
something else listening on that port, then don't!). Example:
http://stromberg.dnsalias.org/~strombrg/max-tcp-window.html

HTH.

[Dan Stromberg]

https://stackoverflow.com/questions/19960475/problems-related-to-so-reuseaddr

[Marko Rauhamaa]

Dan Stromberg <drsa...@gmail.com>:

> On Wed, Jun 27, 2018 at 10:31 PM, Marko Rauhamaa <ma...@pacujo.net> wrote:

> https://stackoverflow.com/questions/19960475/problems-related-to-so-reuseaddr

I'm sorry but I couldn't find a working example behind the link. Could
you demonstrate the problem with a few lines of Python.


Marko

[Steven D'Aprano]

On Thu, 28 Jun 2018 23:27:38 +0300, Marko Rauhamaa wrote:

> Dan Stromberg <drsa...@gmail.com>:
>> On Wed, Jun 27, 2018 at 10:31 PM, Marko Rauhamaa <ma...@pacujo.net>
>> wrote:

>> https://stackoverflow.com/questions/19960475/problems-related-to-so-

reuseaddr
>
> I'm sorry but I couldn't find a working example behind the link. Could
> you demonstrate the problem with a few lines of Python.

Do you think attackers are limited to a few lines of Python?

If you are asking from academic curiosity, limited by care factor ("I
care about this enough to read a few lines of Python but not 100 lines or
20 lines of C...") that's fair enough.

But if you're trying to express skepticism that this is a genuine
concern, then "a few lines of Python" is an unreasonable limitation.




--
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

[Dan Stromberg]

On Thu, Jun 28, 2018 at 1:27 PM, Marko Rauhamaa <ma...@pacujo.net> wrote:

> Dan Stromberg <drsa...@gmail.com>:
> > On Wed, Jun 27, 2018 at 10:31 PM, Marko Rauhamaa <ma...@pacujo.net>
> wrote:

> > https://stackoverflow.com/questions/19960475/problems-
> related-to-so-reuseaddr
>
> I'm sorry but I couldn't find a working example behind the link. Could
> you demonstrate the problem with a few lines of Python.

It's more practical to use English:

Start an echo server process P that listens on tcp/5555.

Initiate a connection from a client machine to process P at tcp/5555. It
works as expected.

Kill P.

Initiate a connection from a client machine to process P at tcp/5555. It
gives a connection refused as expected.

If someone else comes along soon after and starts a different echo server
process Q at tcp/5555 on the same server, it starts up immediately if P
used SO_REUSEADDR.

Then initiate a connection from the same (or different) client machine to
process P (which no longer exists). Q gets the data intended for P.

Naturally, for an echo server, we're just illustrating. But if it were a
daemon that supports password-based authentication, it could be a problem.

As security issues go, it's not the most severe one you'll ever see. It
pretty much assumes there are people on your server who you don't trust.

This is not Python-specific.

That's about all I have the patience for. I'm not sure I'm going to write
a few programs to demonstrate the issue. It's really not a "few lines of
python".

[Marko Rauhamaa]

Dan Stromberg <drsa...@gmail.com>:
> [on how SO_REUSEADDR is a security risk]

> Start an echo server process P that listens on tcp/5555.
>
> Initiate a connection from a client machine to process P at tcp/5555. It
> works as expected.
>
> Kill P.
>
> Initiate a connection from a client machine to process P at tcp/5555. It
> gives a connection refused as expected.
>
> If someone else comes along soon after and starts a different echo server
> process Q at tcp/5555 on the same server, it starts up immediately if P
> used SO_REUSEADDR.
>
> Then initiate a connection from the same (or different) client machine to
> process P (which no longer exists). Q gets the data intended for P.

Well, the same security issue can be demonstrated without SO_REUSEADDR:

DON'T start an echo server process P that listens on tcp/5555.

Initiate a connection from a client machine to process P at tcp/5555. It

gives a connection refused as expected.

If someone else comes along and starts an echo server process Q at
tcp/5555 on the same server, it starts up immediately.

Then initiate a connection from the same (or different) client machine to

process P (which never existed). Q gets the data intended for P.


The security issue can be real but is not directly related with
SO_REUSEADDR.


Marko

[T Berger]

I have to backtrack on my optimistic pronouncement. I know why I'm getting "address already in use" error messages, but I don't know how to avoid getting them. The problem is that I'm working with both IDLE and my Terminal at the same time. I have to close out of IDLE in some decisive way so that the connection with the port is cut. Otherwise I will get the error message. At first, just restarting the shell in IDLE worked. Then I actually had to close the app (which didn't seem like a flexible option to me). Now, even quitting out of IDLE doesn't work. So my revised question is: How do I work with both apps together so these error messages don't keep recurring?

Thanks,

Tamara

[T Berger]

Addendum

I'm using the flask test web address http://127.0.0.1:5000/ for both IDLE and Terminal.

[Jim Lee]

That's not a "web address" - it's the address of your own machine. In
other words, you're connecting to yourself.  You can't use it
simultaneously for both applications - you'll need to change the port
number to something other than 5000 in one of them.

-Jim

[Cameron Simpson]

On 29Jun2018 09:50, Jim Lee <jle...@gmail.com> wrote:
>On 06/29/18 08:05, T Berger wrote:
>>On Friday, June 29, 2018 at 10:45:10 AM UTC-4, T Berger wrote:
>>>On Thursday, June 28, 2018 at 1:21:32 AM UTC-4, T Berger wrote:
>>>I have to backtrack on my optimistic pronouncement. I know why I'm getting
>>>"address already in use" error messages, but I don't know how to avoid
>>>getting them. The problem is that I'm working with both IDLE and my
>>>Terminal at the same time. I have to close out of IDLE in some decisive way
>>>so that the connection with the port is cut. Otherwise I will get the error
>>>message. At first, just restarting the shell in IDLE worked. Then I actually
>>>had to close the app (which didn't seem like a flexible option to me). Now,
>>>even quitting out of IDLE doesn't work. So my revised question is: How do I
>>>work with both apps together so these error messages don't keep recurring?
>

>>Addendum
>>I'm using the flask test web address http://127.0.0.1:5000/ for both IDLE and
>>Terminal.
>
>That's not a "web address" - it's the address of your own machine. In
>other words, you're connecting to yourself.  You can't use it
>simultaneously for both applications - you'll need to change the port number
>to something other than 5000 in one of them.

The key point here from Jim is "simultaneously". Are you properly shutting down
the Flask instance in IDLE before running from Terminal, and vice versa?

Otherwise both will try to use the same local port and There Can Be Only One.

Do you need to run from both environments at the same time? I'd have thought
not, which also leads me to: why are you flicking from IDLE to Terminal? I
would have imagined using one or the other normally, not both. It isn't wrong
to use both, just surprising.

Cheers,
Cameron Simpson <c...@cskk.id.au>

[Dan Stromberg]

On Thu, Jun 28, 2018 at 10:30 PM, Marko Rauhamaa <ma...@pacujo.net> wrote:

> Well, the same security issue can be demonstrated without SO_REUSEADDR:
>

> The security issue can be real but is not directly related with
> SO_REUSEADDR.
>

Yes, it can. It just takes longer.

[Peter J. Holzer]

On 2018-06-28 18:04:16 -0700, Dan Stromberg wrote:
> On Thu, Jun 28, 2018 at 1:27 PM, Marko Rauhamaa <ma...@pacujo.net> wrote:
> > Dan Stromberg <drsa...@gmail.com>:

> > > On Wed, Jun 27, 2018 at 10:31 PM, Marko Rauhamaa <ma...@pacujo.net>
> > > wrote:
> > >> Dan Stromberg <drsa...@gmail.com>:
> > >> >> > The problem can be solved by turning on the SO_REUSEADDR flag of
> > >> >> > the socket.
> > >> > BTW, it's a security feature you're turning off. If you're on a
> > >> > multiuser box, it prevents a second user from stealing lingering
> > >> > connections from a first user on the same port.

[...]

> Start an echo server process P that listens on tcp/5555.
>
> Initiate a connection from a client machine to process P at tcp/5555. It
> works as expected.
>
> Kill P.
>
> Initiate a connection from a client machine to process P at tcp/5555. It
> gives a connection refused as expected.
>
> If someone else comes along soon after and starts a different echo server
> process Q at tcp/5555 on the same server, it starts up immediately if P
> used SO_REUSEADDR.
>
> Then initiate a connection from the same (or different) client machine to
> process P (which no longer exists). Q gets the data intended for P.

I don't think this is the issue TIME_WAIT is intended to prevent.
Firstly, why would this security issue stop to be a security issue after
two minutes? Secondly, it depends on the attacker not knowing about
SO_REUSEADDR, which seems overly optimistic.

What TIME_WAIT is IMHO intended to prevent is mixing up two TCP
connections:

You have just killed P, but some packets from a connection are still in
transit.

Now you start P again and one of the clients starts another connection.
While this connection is active, a packet from the previous connection
arrives. If its sequence number is in the right range, it will be
accepted as part of the current connection.

This can be prevented by not restarting the server until all packets for
old connections are guarantueed to have expired.

In practice this scenario is pretty unlikely: Not only has the client
have to get the same client port, it also needs to get the sequence
numbers (which are 32 bit numbers chosen at random at connection time)
just right.

OTOH, having to wait 2 minutes before you can restart your server is a
nuisance. Therefore, everybody uses SO_REUSEADDR.

hp

--
_ | Peter J. Holzer | we build much bigger, better disasters now
|_|_) | | because we have much more sophisticated
| | | h...@hjp.at | management tools.
__/ | http://www.hjp.at/ | -- Ross Anderson <https://www.edge.org/>

[Jim Lee]

On 06/28/18 18:04, Dan Stromberg wrote:
> [snip]

> Start an echo server process P that listens on tcp/5555.
>
> Initiate a connection from a client machine to process P at tcp/5555. It
> works as expected.
>
> Kill P.
>
> Initiate a connection from a client machine to process P at tcp/5555. It
> gives a connection refused as expected.
>
> If someone else comes along soon after and starts a different echo server
> process Q at tcp/5555 on the same server, it starts up immediately if P
> used SO_REUSEADDR.
>
> Then initiate a connection from the same (or different) client machine to
> process P (which no longer exists). Q gets the data intended for P.
>
>

There are all sorts of theoretical vulnerabilities that simply don't
manifest in real life.  I think this is one of them.

Me: "It hurts when I do this."  Doctor: "Well, don't do that."

-Jim

[Dan Stromberg]

On Sat, Jun 30, 2018 at 11:19 AM, Peter J. Holzer <hjp-p...@hjp.at> wrote:

> On 2018-06-28 18:04:16 -0700, Dan Stromberg wrote:
> > On Thu, Jun 28, 2018 at 1:27 PM, Marko Rauhamaa <ma...@pacujo.net>
> wrote:

> > Start an echo server process P that listens on tcp/5555.
> >
> > Initiate a connection from a client machine to process P at tcp/5555. It
> > works as expected.
> >
> > Kill P.
> >
> > Initiate a connection from a client machine to process P at tcp/5555. It
> > gives a connection refused as expected.
> >
> > If someone else comes along soon after and starts a different echo server
> > process Q at tcp/5555 on the same server, it starts up immediately if P
> > used SO_REUSEADDR.
> >
> > Then initiate a connection from the same (or different) client machine to
> > process P (which no longer exists). Q gets the data intended for P.
>

> I don't think this is the issue TIME_WAIT is intended to prevent.
>

It handles more than one thing.

Firstly, why would this security issue stop to be a security issue after
> two minutes?

It's a mitigation of both problems we've discussed.

> Secondly, it depends on the attacker not knowing about
> SO_REUSEADDR, which seems overly optimistic.
>

Not really, no. People on remote machines know little about process
bounces. And it's pretty easy to write a script that will check if a
process is up repeatedly, starting a daemon when its not. And you don't
have to know about SO_REUSEADDR to write that script.

What TIME_WAIT is IMHO intended to prevent is mixing up two TCP
> connections:
>

Yes, it does this too.

This can be prevented by not restarting the server until all packets for
> old connections are guarantueed to have expired.
>

Yes, and not using SO_REUSEADDR helps this happen.

> In practice this scenario is pretty unlikely: Not only has the client
> have to get the same client port, it also needs to get the sequence
> numbers (which are 32 bit numbers chosen at random at connection time)
> just right.
>

Right, it's a mitigation.

OTOH, having to wait 2 minutes before you can restart your server is a
> nuisance. Therefore, everybody uses SO_REUSEADDR.
>

During development, it's a good practice to setsockopt SO_REUSEADDR.

In production, different rules may or may not apply. On multiuser systems,
it's more important than on most of today's single-purpose virtual machines.

Some people feel that "security issues" must all be religiously 100%
avoided completely, or they'll offend the computer gods. This evidences a
lack of understanding about the practical aspects of security issues.
Security is very much a spectrum - multidimensional even.

[Peter J. Holzer]

On 2018-06-30 14:01:56 -0700, Dan Stromberg wrote:
> On Sat, Jun 30, 2018 at 11:19 AM, Peter J. Holzer <hjp-p...@hjp.at> wrote:
> > On 2018-06-28 18:04:16 -0700, Dan Stromberg wrote:
> > > If someone else comes along soon after and starts a different echo server

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

> > > process Q at tcp/5555 on the same server, it starts up immediately if P
> > > used SO_REUSEADDR.
> > >
> > > Then initiate a connection from the same (or different) client machine to
> > > process P (which no longer exists). Q gets the data intended for P.
> >
> > I don't think this is the issue TIME_WAIT is intended to prevent.

[...]

> > Secondly, it depends on the attacker not knowing about
> > SO_REUSEADDR, which seems overly optimistic.
> >
> Not really, no. People on remote machines know little about process
> bounces.

But the attack you described isn't remote. It is performed by another
user on the same machine.

> And it's pretty easy to write a script that will check if a process is
> up repeatedly, starting a daemon when its not.

Right. This is something the attacker can do (and I would assume they
do).

> And you don't have to know about SO_REUSEADDR to write that script.

No. But if the attacker does know about SO_REUSEADDR they can start
their own server as soon as the notice that the server is down.
Otherwise they have to wait for two minutes.

Actually, I overlooked something: If you don't use SO_REUSEADDR, but the
attacker does, you have to wait, but the attacker doesn't - so this
gives a huge advantage to the attacker. If you do use SO_REUSEADDR, at
least you are an equal footing.

> OTOH, having to wait 2 minutes before you can restart your server is a
> > nuisance. Therefore, everybody uses SO_REUSEADDR.
> >
> During development, it's a good practice to setsockopt SO_REUSEADDR.
>
> In production, different rules may or may not apply. On multiuser systems,

In production, being able to restart a failed server quickly is usually
more important than to prevent a very low probability mixup on the TCP
level. Especially as you can't rely on TCP for integrity and
confidentiality anyway and have to put an extra layer (typically TLS)
for that on top.

> it's more important than on most of today's single-purpose virtual machines.
>
> Some people feel that "security issues" must all be religiously 100%
> avoided completely, or they'll offend the computer gods. This evidences a
> lack of understanding about the practical aspects of security issues.
> Security is very much a spectrum - multidimensional even.

I agree fully. The old adage that "security is binary" is utter
balderdash. However, I do feel that we should only call something a
"security feature" if it actually increases security. Not using
SO_REUSEADDR doesn't - quite the opposite.

[T Berger]

On Friday, June 29, 2018 at 7:00:15 PM UTC-4, Cameron Simpson wrote:

> The key point here from Jim is "simultaneously". Are you properly shutting down
> the Flask instance in IDLE before running from Terminal, and vice versa?

Cameron, I try every option to quit either program, but they don't work. Or I should say, they mostly don't work. Once in a while they do. The one option which works (which is not a feasible option) is rebooting my Mac (actually that might not work either. I think I got the error message again this morning when I rebooted).

> Otherwise both will try to use the same local port and There Can Be Only One.
>
> Do you need to run from both environments at the same time? I'd have thought
> not, which also leads me to: why are you flicking from IDLE to Terminal? I
> would have imagined using one or the other normally, not both. It isn't wrong
> to use both, just surprising.

I'm working from a Python manual. I created the webapp in IDLE, and test it in Terminal, per the instructions in the manual. I use IDLE to edit my program, and then test it in terminal. When I go from one to the other, I get the error message. IDLE has a keyboard shortcut for quitting the shell—Cntl + C—but it doesn't work. Neither does restarting the shell. Neither does entering the kill command line, not in IDLE or terminal.

Do you have any other suggestions? I'm going to email the writer. He doesn't mention how to deal with problems that might arise from working with two programs at the same time. Maybe Cntl+C should do it, but in my case it doesn't.

Tamara

> Cheers,
> Cameron Simpson <c...@cskk.id.au>

[MRAB]

On 2018-06-30 23:01, T Berger wrote:
> On Friday, June 29, 2018 at 7:00:15 PM UTC-4, Cameron Simpson wrote:
>
>> The key point here from Jim is "simultaneously". Are you properly shutting down
>> the Flask instance in IDLE before running from Terminal, and vice versa?
>
> Cameron, I try every option to quit either program, but they don't work. Or I should say, they mostly don't work. Once in a while they do. The one option which works (which is not a feasible option) is rebooting my Mac (actually that might not work either. I think I got the error message again this morning when I rebooted).
>
>> Otherwise both will try to use the same local port and There Can Be Only One.
>>
>> Do you need to run from both environments at the same time? I'd have thought
>> not, which also leads me to: why are you flicking from IDLE to Terminal? I
>> would have imagined using one or the other normally, not both. It isn't wrong
>> to use both, just surprising.
>
> I'm working from a Python manual. I created the webapp in IDLE, and test it in Terminal, per the instructions in the manual. I use IDLE to edit my program, and then test it in terminal. When I go from one to the other, I get the error message. IDLE has a keyboard shortcut for quitting the shell—Cntl + C—but it doesn't work. Neither does restarting the shell. Neither does entering the kill command line, not in IDLE or terminal.

Ctrl+C is the shortcut for interrupting a Python program running on
IDLE's or Windows command line. In IDLE's editor, Ctrl+C is the shortcut
for copying text the clipboard.

The keyboard shortcut for quitting IDLE completely is Ctrl+Q.

[Gregory Ewing]

I don't see how the address-reuse timeout can be a security
measure, because the process trying to take over the address
can easily circumvent it by setting SO_REUSEADDR.

--
Greg

[Grant Edwards]

I've been thinking the same thing. One _might_ be able to argue
against the OS providing the SO_REUSEADDR option on security grounds.
But given that it _does_ exist, I don't see how using it can be
claimed to decrease security.

--
Grant

[Steven D'Aprano]

On Sat, 30 Jun 2018 23:49:41 +0200, Peter J. Holzer wrote:

> The old adage that "security is binary" is utter balderdash.

I think that "old adage" is one of those ones that only people denying it
actually use.

I've never seen anyone say "security is binary" except to disagree with
it, "don't make the mistake of thinking security is binary".

I mean, we lock our front doors, even though burglars can break in
through a window, right?

[T Berger]

[Peter J. Holzer]

On 2018-07-01 01:52:12 +0000, Steven D'Aprano wrote:
> On Sat, 30 Jun 2018 23:49:41 +0200, Peter J. Holzer wrote:
> > The old adage that "security is binary" is utter balderdash.
>
> I think that "old adage" is one of those ones that only people denying it
> actually use.
>
> I've never seen anyone say "security is binary"

That was almost a dogma in some (mostly German) usenet groups 15 to 20
years ago.

Some people apparently still believe it - it hasn't been that long
(maybe a year or so) since I've last had a discussion about this.

[T Berger]

Cameron, I'd like to add screenshots of my idle and terminal windows. Perhaps this would shed light on my this issue.

This is my last window in terminal:

[Restored Jul 2, 2018, 1:12:09 PM]
Last login: Mon Jul 2 10:58:54 on ttys000
Restored session: Mon Jul 2 11:04:19 EDT 2018
192:Webapp TamaraB$ exit
logout
Saving session...
...saving history...truncating history files...
...completed.

[Process completed]
~~~~
To me this looks like the program running in terminal has been killed.

Now this is my idle shell:

RESTART: /Users/TamaraB/Desktop/Webapp/vsearch4web.py
* Serving Flask app "vsearch4web" (lazy loading)
* Environment: production
[31m WARNING: Do not use the development server in a production environment. [0m
[2m Use a production WSGI server instead. [0m
* Debug mode: on
Traceback (most recent call last):
File "/Users/TamaraB/Desktop/Webapp/vsearch4web.py", line 26, in <module>
app.run(debug=True)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/flask/app.py", line 943, in run
run_simple(host, port, self, **options)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/werkzeug/serving.py", line 795, in run_simple
s.bind(get_sockaddr(hostname, port, address_family))

OSError: [Errno 48] Address already in use
>>>

Why should there be an error in idle? It's the only app running.

I emailed the writer of the manual I'm using, but who knows if, or when, he'll reply. So any help you provide would be great.

Thanks,

Tamara

[Marko Rauhamaa]

Gregory Ewing <greg....@canterbury.ac.nz>:

The address reuse timeout (TIME WAIT) is only used to deal with
"Byzantine generals" loophole in the TCP protocol. Its purpose is to
guarantee that both peers end up with a common understanding of the
final state of the connection.

Nevertheless, the later socket object cannot unilaterally take over a
socket using SO_REUSEADDR. The earlier socket object must have set the
same option previously.


Marko

[T Berger]

Addendum

I forgot to add that I get the error message even after I exit from idle. Exiting from idle is supposed to kill the running program. That is what the the confirming dialog box explicitly states.

[Gregory Ewing]

Marko Rauhamaa wrote:
> Nevertheless, the later socket object cannot unilaterally take over a
> socket using SO_REUSEADDR. The earlier socket object must have set the
> same option previously.

I just did an experiment that suggests that's not the case.
I created a socket without SO_REUSEADDR, made a connection to
it, and killed it. Then I created another socket within the
timeout period with SO_REUSEADDR, and it succeeded.

This was on MacOSX -- it's possible that other systems
behave differently.

--
Greg

[Marko Rauhamaa]

Gregory Ewing <greg....@canterbury.ac.nz>:

I tried it, too, (on Linux) and it failed.

It's crucial that the killed party is the server for the situation to
arise.

That's why polite clients close their end of the connection before
the server. Whoever closes first will suffer the TIME-WAIT state.


Marko

[Chris Angelico]

The same thing can happen with clients AND servers, but it's very
common with clients to allow an arbitrary port number to be chosen for
you. To trigger this problem from the client end, you would need to
use the same port. An attacker can do this easily.

ChrisA

[Marko Rauhamaa]

Chris Angelico <ros...@gmail.com>:

No, I mean that the TIME-WAIT state is triggered asymmetrically. See RFC
793 page 23. Whoever sends FIN first will end up in TIME-WAIT for 2*MSL.

A client that doesn't send FIN first will cause an accumulation of
connection objects (TCBs) at the server end. It can also prevent the
server from being restarted quickly.

The SO_REUSEADDR discussion has no direct relationship with computer or
network security. In fact, I don't see any reason why it's not always
on for TCP. Maybe it's done so UDP and TCP (and other protocols) would
work similarly.


Marko

[Grant Edwards]

On 2018-07-01, Marko Rauhamaa <ma...@pacujo.net> wrote:
> Gregory Ewing <greg....@canterbury.ac.nz>:
>

>> I don't see how the address-reuse timeout can be a security measure,
>> because the process trying to take over the address can easily
>> circumvent it by setting SO_REUSEADDR.
>

> [...]

>
> Nevertheless, the later socket object cannot unilaterally take over a
> socket using SO_REUSEADDR. The earlier socket object must have set the
> same option previously.

On what OS? In my experience, that's not true on Linux or BSD Unix.

--
Grant Edwards grant.b.edwards Yow! Yes, but will I
at see the EASTER BUNNY in
gmail.com skintight leather at an
IRON MAIDEN concert?

[Grant Edwards]

On 2018-07-03, Grant Edwards <grant.b...@gmail.com> wrote:
> On 2018-07-01, Marko Rauhamaa <ma...@pacujo.net> wrote:
>> Gregory Ewing <greg....@canterbury.ac.nz>:
>>

>>> I don't see how the address-reuse timeout can be a security measure,
>>> because the process trying to take over the address can easily
>>> circumvent it by setting SO_REUSEADDR.
>>

>> [...]

>>
>> Nevertheless, the later socket object cannot unilaterally take over a
>> socket using SO_REUSEADDR. The earlier socket object must have set the
>> same option previously.
>

> On what OS? In my experience, that's not true on Linux or BSD Unix.

I was wrong. I just did a quick test on Linux, and it works the way
Marko Hauhamaa describes. [Don't have handy access to a BSD system at
the moment.]

--
Grant Edwards grant.b.edwards Yow! I'm wearing PAMPERS!!
at
gmail.com

[Grant Edwards]

On 2018-07-03, Grant Edwards <grant.b...@gmail.com> wrote:
> On 2018-07-03, Grant Edwards <grant.b...@gmail.com> wrote:
>> On 2018-07-01, Marko Rauhamaa <ma...@pacujo.net> wrote:
>>> Gregory Ewing <greg....@canterbury.ac.nz>:
>>>

>>>> I don't see how the address-reuse timeout can be a security measure,
>>>> because the process trying to take over the address can easily
>>>> circumvent it by setting SO_REUSEADDR.
>>>

>>> [...]

>>>
>>> Nevertheless, the later socket object cannot unilaterally take over a
>>> socket using SO_REUSEADDR. The earlier socket object must have set the
>>> same option previously.
>>

>> On what OS? In my experience, that's not true on Linux or BSD Unix.
>
> I was wrong. I just did a quick test on Linux, and it works the way
> Marko Hauhamaa describes.

Um that was supposed to be Rauhamaa... sorry about that.

> [Don't have handy access to a BSD system at the moment.]

Found a NetBSD 7.1 system to test on...

After killing a server with an active connection, a new server can
bind to the same socket regardless of the SO_REUSEADDR setting in
either server. I don't know if that's some sort of system
configuration setting or what...

--
Grant Edwards grant.b.edwards Yow! Where's the Coke
at machine? Tell me a joke!!
gmail.com

[Marko Rauhamaa]

Grant Edwards <grant.b...@gmail.com>:

> Found a NetBSD 7.1 system to test on...
>
> After killing a server with an active connection, a new server can
> bind to the same socket regardless of the SO_REUSEADDR setting in
> either server. I don't know if that's some sort of system
> configuration setting or what...

The question is, why shouldn't you be able to bind to an unbound socket
that has stale connections in TIME-WAIT?

Maybe the NetBSD people found the answer: there's no reason.


Marko

[mm0fmf]

Tamara, read something on how TCP sockets work. Even if you close the
program that created the socket, it doesn't mean the port in use will
close immediately. It all depends on the state the socket was in. There
is a process of exchanging info that sockets go through when closing to
make sure both ends know the socket has closed. This can take time to
complete. ISTR the default is 120secs.

You can get past the wait by ensuring you close the socket when no
connections are pending. Then you can use that port immediately. If
there is something active on the socket, closing the socket puts you
into TIME_WAIT and you cannot rebind to that socket till the timeout is
over.

Or you can set SO_REUSEADDR on the socket after you create it but before
you use it. This allows another program to immediately use the port as
soon as you call close on the socket. Only 1 program can use the port
still, but this option allows you to bypass the wait state.

This info has already been given in this thread but maybe you missed it
or maybe didn't understand it.

Please read up some general background information on sockets
programming because it will make your programming life easier when you
know what is meant to happen.