Hi Greg,
> It is entirely possible that I don't understand what is happening with
> the connection as well as I thought I did.
>
> Let me explain what I have done to come to some of these conclusions.
> When I lock down our incoming and outgoing ports to only 443 crossbar
> does not complete the connection; however when I open all outgoing ports
> in addition to the 443 incoming lock down; then crossbar connects.
>
> Here is an example from wireshark:
>
> <
https://lh3.googleusercontent.com/-Azv_ScYz7bQ/VvnKYojkBAI/AAAAAAAAA3U/e9vEOs6WJJUT9DBAoiBz29yvwtcvHccmA/s1600/wireshark_crossbar_ports.png>
>
> You can see the ACK coming from 443 then going back out on different
> ports 58485 and 58743, these ports seem to increment per crossbar
> connection, so I'm assuming this is tied into the transport layer/protocol?
No, these are ephemeral ports on the connecting client.
> while crossbar might not specifically require the outgoing ports opened,
> it appears that TCP does for the transmission control.
An outbound port X being closed does not mean there can't be IP packets
with a source port X!
It only means, that no _originating_ connection _to_ a port X is allowed.
On the other had, when an inbound port is open say 443, an incoming TCP
connection to that ports will be allowed, and the firewall then
remembers the source IP/port (the client IP and ephemeral port) and
subsequently will allow outgoing IP packets to that combination - but
only IP packets related to that establishing or established TCP connection!
This is called "stateful operation". Layer 4 firewalls (at TCP level)
usually work like this.
>
> <
https://lh3.googleusercontent.com/-_nLUOssCUB8/VvnNl3LG7UI/AAAAAAAAA3g/RHixQy2x1tszp0qWtEHIFLh7Qo_HqktGg/s1600/wireshark_crossbar_ports2.png>
>
>
> Im wondering if there is a way to know and or limit those outgoing
> ports, or if we need to keep them all open for maximum connections.
You can disallow any outbound connection _originating_ from the
Crossbar.io host.
But you must open inbound ports 80/443 and operate a stateful firewall
at layer 4 which "knows about TCP".
>
> This also leads me to another question, assuming we open a connection on
> ever single available ephemeral port does this limit the amount of
> websocket connections per server?
There is a lot of confusion around ephemeral ports, the 64k limit and
maximum connection number of a server.
The 64k limit effects the maximum number of _outgoing_ connections _from
_a _single_ IP address (bound to an interface on the host where
connections originate).
Each such outgoing connection will need a locally assigned ephemeral
port, and there are only 64k _per IP_.
a) this does not effect the number of incoming connections a server can run
b) you can overcome the outgoing connections limit by using IP aliases
on your network interface
Conclusion: you can arbitrary (well, millions at least) numbers of both
incoming connections and outgoing connections on a single host.
Cheers,
/Tobias
>
>
> *Found some answers while searching around:*
> *
> *
> This wiki <
https://en.wikipedia.org/wiki/Ephemeral_port>page provides
> some interesting info about ephemeral ports, of note is that they are
> temporary and re-used.
>
> What this seems to mean is that we need to leave all the ephemeral ports
> of our OS open e.g. for linux it would be the port range of 32768 -
> 61000, or 1025-65535 to be more inclusive for various OS's.
> Just found this which is another good read
> <
https://serverfault.com/questions/627880/do-ephemeral-ports-need-to-be-open-on-the-network-firewall-in-order-for-nginx-an>,
> sort of similar to what I am trying to accomplish for crossbar rather
> than nginx.
>
> Here is a good link specific to Amazon AWS ephemeral port ranges
> <
https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_ACLs.html#VPC_ACLs_Ephemeral_Ports> and
> VPC subnet ACLS
> <
https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_ACLs.html> which
> > an email to
crossbario+...@googlegroups.com <javascript:>
> > <mailto:
crossbario+...@googlegroups.com <javascript:>>.
> <javascript:>
> > <mailto:
cross...@googlegroups.com <javascript:>>.
> <
https://groups.google.com/d/msgid/crossbario/bb66171b-b8ca-4092-a1a1-7e22b3a1c2f2%40googlegroups.com?utm_medium=email&utm_source=footer
> <
https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Crossbar" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to
crossbario+...@googlegroups.com
> <mailto:
crossbario+...@googlegroups.com>.
>
https://groups.google.com/d/msgid/crossbario/59d8b369-b170-4992-a25a-c9e54bc16d22%40googlegroups.com
> <
https://groups.google.com/d/msgid/crossbario/59d8b369-b170-4992-a25a-c9e54bc16d22%40googlegroups.com?utm_medium=email&utm_source=footer>.