Safe Outgoing Ports?

[Greg Keys]

I'm working on optimizing some security aspects of our servers, currently we are allowing all outgoing ports. 

Is there a safe range of outgoing ports that I can white-list or do we have to leave them all open for the outgoing socket connections?

My thinking is that some server exploits which are thwarted by simply closing the outgoing ports used by the exploit.

[Tobias Oberstein]

Hi Greg,

Am 21.03.2016 um 07:38 schrieb Greg Keys:
> I'm working on optimizing some security aspects of our servers,
> currently we are allowing all outgoing ports.
> Is there a safe range of outgoing ports that I can white-list or do we
> have to leave them all open for the outgoing socket connections?

Prenote: The criticial security measure is to restrict
incoming/listening ports - less so outgoing ports. Sure you are aware,
just wanted to note to make it explicit.

Now, outgoing ports.

Crossbar.io itself will not open any outgoing connection by default
(other than the optional upcoming Crossbar.io DevOps Center connection).

Of course, if you host an app component that in turn connects somewhere
else (not only a router worker running on the same host than the
component), then this will require open outgoing ports.

But actually, I am not sure I get what you are asking for / understand
your scenario ..

Cheers,
/Tobias

>
> My thinking is that some server exploits which are thwarted by simply
> closing the outgoing ports used by the exploit.
>

> --
> You received this message because you are subscribed to the Google
> Groups "Crossbar" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to crossbario+...@googlegroups.com
> <mailto:crossbario+...@googlegroups.com>.
> To post to this group, send email to cross...@googlegroups.com
> <mailto:cross...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/crossbario/bb66171b-b8ca-4092-a1a1-7e22b3a1c2f2%40googlegroups.com
> <https://groups.google.com/d/msgid/crossbario/bb66171b-b8ca-4092-a1a1-7e22b3a1c2f2%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

[Greg Keys]

Prenote: The criticial security measure is to restrict 
incoming/listening ports - less so outgoing ports. Sure you are aware, 
just wanted to note to make it explicit. 

Yes of course, here is a good read regarding securing both in and outgoing, while exploits requiring outgoing ports is less likely, they are still possible which is why we want to be as thorough as possible.

Crossbar.io itself will not open any outgoing connection by default 
(other than the optional upcoming Crossbar.io DevOps Center connection). 

It is entirely possible that I don't understand what is happening with the connection as well as I thought I did.

Let me explain what I have done to come to some of these conclusions.

When I lock down our incoming and outgoing ports to only 443 crossbar does not complete the connection; however when I open all outgoing ports in addition to the 443 incoming lock down; then crossbar connects.

Here is an example from wireshark:

You can see the ACK coming from 443 then going back out on different ports 58485 and 58743, these ports seem to increment per crossbar connection, so I'm assuming this is tied into the transport layer/protocol? 

while crossbar might not specifically require the outgoing ports opened, it appears that TCP does for the transmission control.

Im wondering if there is a way to know and or limit those outgoing ports, or if we need to keep them all open for maximum connections. 

This also leads me to another question, assuming we open a connection on ever single available ephemeral port does this limit the amount of websocket connections per server?

Found some answers while searching around:

This wiki page provides some interesting info about ephemeral ports, of note is that they are temporary and re-used. 

What this seems to mean is that we need to leave all the ephemeral ports of our OS open e.g. for linux it would be the port range of 32768 - 61000, or 1025-65535 to be more inclusive for various OS's.

Just found this which is another good read, sort of similar to what I am trying to accomplish for crossbar rather than nginx.

Here is a good link specific to Amazon AWS ephemeral port ranges and VPC subnet ACLS which is exactly what I was looking for when I originally asked this question.

My question is answered now, hopefully this information will help anyone else who might have similar questions.

> <mailto:crossbario+unsub...@googlegroups.com>.

[Tobias Oberstein]

Hi Greg,

> It is entirely possible that I don't understand what is happening with
> the connection as well as I thought I did.
>
> Let me explain what I have done to come to some of these conclusions.
> When I lock down our incoming and outgoing ports to only 443 crossbar
> does not complete the connection; however when I open all outgoing ports
> in addition to the 443 incoming lock down; then crossbar connects.
>
> Here is an example from wireshark:
>

> <https://lh3.googleusercontent.com/-Azv_ScYz7bQ/VvnKYojkBAI/AAAAAAAAA3U/e9vEOs6WJJUT9DBAoiBz29yvwtcvHccmA/s1600/wireshark_crossbar_ports.png>

>
> You can see the ACK coming from 443 then going back out on different
> ports 58485 and 58743, these ports seem to increment per crossbar
> connection, so I'm assuming this is tied into the transport layer/protocol?

No, these are ephemeral ports on the connecting client.

> while crossbar might not specifically require the outgoing ports opened,
> it appears that TCP does for the transmission control.

An outbound port X being closed does not mean there can't be IP packets
with a source port X!

It only means, that no _originating_ connection _to_ a port X is allowed.

On the other had, when an inbound port is open say 443, an incoming TCP
connection to that ports will be allowed, and the firewall then
remembers the source IP/port (the client IP and ephemeral port) and
subsequently will allow outgoing IP packets to that combination - but
only IP packets related to that establishing or established TCP connection!

This is called "stateful operation". Layer 4 firewalls (at TCP level)
usually work like this.

>
> <https://lh3.googleusercontent.com/-_nLUOssCUB8/VvnNl3LG7UI/AAAAAAAAA3g/RHixQy2x1tszp0qWtEHIFLh7Qo_HqktGg/s1600/wireshark_crossbar_ports2.png>

>
>
> Im wondering if there is a way to know and or limit those outgoing
> ports, or if we need to keep them all open for maximum connections.

You can disallow any outbound connection _originating_ from the
Crossbar.io host.

But you must open inbound ports 80/443 and operate a stateful firewall
at layer 4 which "knows about TCP".

>
> This also leads me to another question, assuming we open a connection on
> ever single available ephemeral port does this limit the amount of
> websocket connections per server?

There is a lot of confusion around ephemeral ports, the 64k limit and
maximum connection number of a server.

The 64k limit effects the maximum number of _outgoing_ connections _from
_a _single_ IP address (bound to an interface on the host where
connections originate).

Each such outgoing connection will need a locally assigned ephemeral
port, and there are only 64k _per IP_.

a) this does not effect the number of incoming connections a server can run
b) you can overcome the outgoing connections limit by using IP aliases
on your network interface

Conclusion: you can arbitrary (well, millions at least) numbers of both
incoming connections and outgoing connections on a single host.

Cheers,
/Tobias

>
>
> *Found some answers while searching around:*
> *
> *
> This wiki <https://en.wikipedia.org/wiki/Ephemeral_port>page provides

> some interesting info about ephemeral ports, of note is that they are
> temporary and re-used.
>
> What this seems to mean is that we need to leave all the ephemeral ports
> of our OS open e.g. for linux it would be the port range of 32768 -
> 61000, or 1025-65535 to be more inclusive for various OS's.
> Just found this which is another good read

> <https://serverfault.com/questions/627880/do-ephemeral-ports-need-to-be-open-on-the-network-firewall-in-order-for-nginx-an>,

> sort of similar to what I am trying to accomplish for crossbar rather
> than nginx.
>
> Here is a good link specific to Amazon AWS ephemeral port ranges

> <https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_ACLs.html#VPC_ACLs_Ephemeral_Ports> and
> VPC subnet ACLS
> <https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_ACLs.html> which

> > an email to crossbario+...@googlegroups.com <javascript:>
> > <mailto:crossbario+...@googlegroups.com <javascript:>>.

> > To post to this group, send email to cross...@googlegroups.com

> <javascript:>
> > <mailto:cross...@googlegroups.com <javascript:>>.

> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/crossbario/bb66171b-b8ca-4092-a1a1-7e22b3a1c2f2%40googlegroups.com
> <https://groups.google.com/d/msgid/crossbario/bb66171b-b8ca-4092-a1a1-7e22b3a1c2f2%40googlegroups.com>
>
> >

> <https://groups.google.com/d/msgid/crossbario/bb66171b-b8ca-4092-a1a1-7e22b3a1c2f2%40googlegroups.com?utm_medium=email&utm_source=footer

> <https://groups.google.com/d/msgid/crossbario/bb66171b-b8ca-4092-a1a1-7e22b3a1c2f2%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> > For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "Crossbar" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to crossbario+...@googlegroups.com

> <mailto:crossbario+...@googlegroups.com>.

> To post to this group, send email to cross...@googlegroups.com
> <mailto:cross...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/crossbario/59d8b369-b170-4992-a25a-c9e54bc16d22%40googlegroups.com
> <https://groups.google.com/d/msgid/crossbario/59d8b369-b170-4992-a25a-c9e54bc16d22%40googlegroups.com?utm_medium=email&utm_source=footer>.