UpdateApologies to the user who kindly placed a bounty on this question, and all those who have offered advice, but embarrassingly this is now working after a physical reboot of both server and client. I can't even say for sure which one fixed it - but since I believe I restarted all affected services on the server after making changes, I actually wonder if the Windows OpenVPN client may have needed a restart after the TAP driver installation so to anyone reading this, if you are having problems, try a different client eg OpenVPN Connect on Android. Again, apologies for not trying this sooner!
As stated in the nmap docs, openfiltered means that NMAP can not determine for whatever reason, but it doesn't think the port is explicitly closed. If this is the case, the best way to actually check would be to just try to use the port, and see if it works or not. So, in this case, connect to the OpenVPN host and see what happens.
Here nmap shows result Filtered which means that a firewall, filter, or other network obstacle is blocking the port so that Nmap cannot tell whether it is open or closed. In this case openvpn is using that port, that's why nmap is showing filtered. Use nc command.
I'm running an OpenVPN server and can use it normally; I know for a fact that the port is open. Running an Nmap scan on port 1194 (the one I'm using) says it is closed. What could be causing this incorrect response? Used this command:
I am able to connect to my OpenVPN server via port 1194, even though this port is not allowed (accepted) in my iptables config. I can confirm no exception is defined as this command gives no output: iptables -S grep 1194
Secondly, OpenVPN does not listen on loopback address (127.0.0.1) for client connections. And it will make no sense if it will listen on loopback address. Therefore nmap against localhost will always show UDP port 1194 as closed.
a situation i used to always run into in the past was: i needed to set up a new infoblox device or ha pair, and the grid master and grid master candidates were in different datacenters/locations and there were one or more firewalls between them. i would submit firewall rules, then hope they were implemented correctly, waiting for the stated time to try my grid join and hope it worked. then, if it didn't, trying to figure out for sure if it was a firewall issue or something else.
i always wished infoblox had an option somewhere to "test" grid communication in some way, just to verify the lines of communication were open without doing the actual join at that point. but they didn't. (and don't, that i'm aware of.)
at some point, i became aware of the expertmode option, and access to the new/different command line tools that mode provides. using that, i was able to figure out a way to test/verify connectivity to the grid master.
caveat: the device must not be already joined to a grid. if the new device is already joined to something, then the openvpn udp ports (at least 1194) will already be in use so this trick doesn't work.
so here's what i do now to verify the firewalls are opened properly, well in advance of my implementation date, so i can follow up with the firewall teams to get things fixed before the join date and time arrives...
(which interface you have to listen on depends on the infoblox device, but in general eth0 is mgmt, eth1 is lan1, eth2 is ha - in my experience. ha pairs do openvpn tunnels between each other on their lan1 ips, while devices talk to the grid master from their lan1 ip to the grid master's vip. if a grid master is an ha pair, the passive member talks to the active member from its lan1 ip to the active member's vip (instead of lan1 to lan1 like grid member ha pairs do with each other. so "eth2" above is the grid master's ha interface, and since you should be on the active member, the vip is riding on this interface. technically, vips are on eth2 or eth2 is empty on the passive member, and the ha ip for each member is on eth2:1. i know i used to have some devices that didn't ).
if there is nothing blocking the communication, then once the traceroute packets get through the other hops in the path to the grid master you'll see some packets show up in the tcpdump screen on the grid master. something like...
i just thought i'd mention this as an option, since i've found it helpful myself in making sure everything is ready to go before the actual night of a grid join and finding out the firewalls weren't actually opened correctly.
as a bonus: if you are doing the join and it says it's successful, then it never finishes joining (syncing, usually)...you can go into expert mode and look at a tcpdump for 2114 and 1194. if you see pretty consistently spaced 1194 packets of the same smaller size (a properly working openvpn tunnel will have a lot of 1194 traffic, of varying sizes, not just packets of the same size with a fairly consistent time gap between them), then the openvpn tunnel is having some problems. in my experience, going into the grid member settings and lowering the vpn tunnel mtu from 1450 down to 1000 or something has a decent chance of fixing this.
do i need to be a part of some secret society to know about it? i mean, if i have to wear a signet ring or know some secret handshake or something, i could probably be okay with that. having to get an infoblox tattoo or brand might be a deal breaker for me.
I wonder how long that command has been around. I've taken the advanced admin that gives bunch of the maintenance mode commands in the book and it wasn't included in 2008 or 2013 versions of the class materials. But my RFE-1737 for this this type of feature to test GMC's avaibilty(firewall rules) without actually failing to it was still open when I checked several years go. So hopefully its new-ish.
I'd messed with attempting to do this with tcpdump and dig generating traffic on the VPN ports at one time and never finished the script. I think this will be much easier to script a GMC firewall rule validatation without doing a fail over. It's nice to see it finally available \ made public.
while there are some interesting things in there, i've lost some of my excitement about my use of show network_connectivity. while it runs nmap, it's a wrapper, and it severely limits some of nmap's functionality that would provide what i need.
like i would assume in most environments, firewall rules where i've worked are punched very specifically, port-to-port. so if i can't specify both a source and destination port, it's not going to show connectivity - because the firewall will block random udp ports to the port specified with the network_connectivity wrapper. (by default, nmap will use a random udp port as source port.) so unless network_connectivity configures source port to match destination port, the nmap run will use a random udp source port. and the firewall will *only* allow 11941194 and 21142114, so the nmap test through network_connectivity will fail to show/prove anything. (and if any udp port can get to the 1194/2114 port on the other end, then it's not really as much of a firewall test as just a general connection test.)
so i guess right now i still have to stick with my abuse of traceroute and tcpdump to test that firewall rules have been properly entered. but my technique doesn't work if the system is already grid joined, since openvpn will be using 1194 and usually 2114 so i can't specify them as source ports.
It would depend on how well the firewall rules are written. With the default being "any" port for the source port for the vast majority of applications, it is generally a one off to lock down the source port on a rule. Well written firewall rules this would not test, but I would question how many customers took the time to actually lock down the source port on the rule as they should(could) have with this VPN connectivity.
On a related tangent, having the same source and destination port is a very good way to test NAT code. I've found several vendors NAT's that fail in specific ways when you have multiple connections through their NAT that have the same source and destination ports. (A GM on one side to multiple nodes on the other side) Specifically any kind of clustering HA hand off of connections for NAT redundancy. Somewhere in some likely reused NAT code someone made some bad assumptions on how likely that was to occur.
True.
In a scenario where your appliance/member is already grid joined and you are perhaps trying to check its connectivity to a grid master candidate on the grid (with potential master promotion in mind), I would recommend just using a random source port for the test and 1194/2114 as destination ports (and vice versa).
i guess i've just had the...fortune?...to work in shops where the firewall rules implemented for infoblox grid communication have always been locked down to one port on both the source and destination sides. i guess since the infoblox documentation lists that as the case, that's what we keep submitting. maybe i'll just have to make sure everyone on the ddi team starts relaxing the specifity of their firewall requests, so easier testing can ensue. : )
that's an interesting nat failure behaviour. i guess i can see why same source and destination port wouldn't be immediately thought of as a scenario, but you'd think it would have been noticed at some point and had to be dealt with. glad i haven't had to run across that one!
those are nmap responses. doing some google (or your search engine of choice) searches on nmap should give you a lot more detail about them. here's a bit from an nmap man page from the
nmap.org site:
The state is either open, filtered, closed, or unfiltered. Open means that an application on the target machine is listening for connections/packets on that port. Filtered means that a firewall, filter, or other network obstacle is blocking the port so that Nmap cannot tell whether it is open or closed. Closed ports have no application listening on them, though they could open up at any time. Ports are classified as unfiltered when they are responsive to Nmap's probes, but Nmap cannot determine whether they are open or closed. Nmap reports the state combinations openfiltered and closedfiltered when it cannot determine which of the two states describe a port.
3a8082e126