Using a Single IP Address on Multiple Devices

[Christian Paasche]

Hi,

I am very new to Netbox and love the product but am experiencing the normal bumps getting up to speed.

Our network has multiple locations using HA pairs of Palo Alto firewalls.  We are using the same VRFs on each HA firewall pair.

Documenting our HA pairs, I am assigning an IP address to Eth 1/1 of firewall HA1.

When I assign the same IP address to Eth 1/1 on firewall HA2, the IP address just appears on firewall HA2 and is gone from Eth 1/1 on HA1.

I have tried using the the CARP role but just adds a tag to the IP address.

Am I missing something?  I have read the documentation and search online for tutorials.  Nada.

How to assign the same IP to the same interface and VRF across HA devices?

Thanks.

[Erdem Kasnak]

may this help:

ENFORCE_GLOBAL_UNIQUE

Default: False

Enforcement of unique IP space can be toggled on a per-VRF basis. To enforce unique IP space within the global table (all prefixes and IP addresses not assigned to a VRF), set ENFORCE_GLOBAL_UNIQUE to True.

regards

[Brian Candler]

On Monday, 25 May 2020 17:40:56 UTC+1, Christian Paasche wrote:

When I assign the same IP address to Eth 1/1 on firewall HA2, the IP address just appears on firewall HA2 and is gone from Eth 1/1 on HA1.

The Netbox data model looks like this:

Device ---< Interface ---< IP Address

Interface to IP Address is a one-to-many relationship.  On the IP address object there is an "interface" field; hence a given IP address object can only belong to one interface.

You have a few options:

1. Create two different IP address objects with the same IP address, and assign them separately to each device.  I don't like doing this, because you have to turn off IP address uniqueness checking.

2. Create your firewall cluster as a "Virtual Chassis" (two linked devices).  Create a LAG which links A-Eth1/1 with B-Eth1/1, and assign the address to the LAG.

This is a bit icky because there's actually no LAG, and Virtual Chasses are a bit fiddly to created (you need to ensure the interfaces of the two devices are named differently before linking them)

3. Just assign the IP addresses to the interface on the Master, and none to its Failover partner.  Imperfect but simple.  This is what I do.

[Mahomed Hussein]

Personally, I think this is one of the biggest flaws/short-comings of Netbox. I believe someone mentioned elsewhere on the mailing lists that this one IP to one Interface is a true representation of real life (but I can't find the email now). I disagree, but I'll rant on that later in the message.

First, this is how I have worked around this, which is a variation on Brian's option 3.

1. On each device, create an interface of type "Other"

2. Name it with this naming convention (I guess you choose your own :-) ) [Physical-Int-Name]_V-IPs_[Pri|Sec] So in the case of my two screenshots below, my physical interface is ETH1 so I named them ETH1_V-IPs_Pri and ETH1_V-IPs_Sec

3. Create a cable link between the two interfaces. The advantage of this is that when you are looking at a device, you can get to the other device with one click.

4. I add the port description of "Virtual IPs for this Interface" (probably can be improved - let me know what you come up with)
5. Add all your IPs to the interface on your Primary device.
  Note: I use the term Primary loosely, but generally with HSRP, VRRP, CARP you have one device that has a higher priority (i.e. more important)

Screenshots below (apologies for any formatting issues. The Groups editor is a bit odd).

Primary router (p1)

Secondary router (p2)

<rant>

I believe that having to create multiple IP objects in Netbox in order to represent that IP on multiple devices is flawed (and is also why you then have to disable Unique IP address checking). After all, in real life, assigning the same IP to multiple devices does not create more instances of that IP in that block (and which is why you then get errors on your networking devices). Assigning the same IP to multiple devices should trigger a Duplicate IP warning unless it is of the type Anycast, VRRP, HSRP, CARP (and probably some other I am not familiar with personally) as these are specifically designed to be assigned to multiple devices (even though the devices then have mechanisms to only makes it live one device at a time). But if you looked at the config of these devices (e.g on Ciscos and pfSense) you would see that the IP is configured on the interface on each device. Ergo, when looking at an interface in Netbox, it's useful to see at a glance that there is another device that has this IP assigned too and you can click to it straight away. It also means that when you query a device for all its interfaces and IPs, you get all the IPs and Type from that one device and you can build your config from it, without having to do further processing of checking whether it is a secondary device, finding the primary then getting all the IPs again from that device.

Regarding this comment:

On the IP address object there is an "interface" field; hence a given IP address object can only belong to one interface.

I think the easy option is to remove this from the UI (at least) and only allow assigning of IPs from the interface that you are trying to assign it to. Generally this is how you do it reality (i.e. you get an available IP from a list, then go and set it up on the interface). I can see the possible case of going into your IPAM, assigning it to a device/interface and then Ansible/Saltstack etc. goes and programs it on the interface. But even in this case, the device and interface still has to be in Netbox, so why not just go to your device interface and assign the IP from there and still have Ansible then pick it up and go program your interface? 

If you search the mailing list, you can see that this issue has come up several times over the years and I think there was mussings of this being a case for a plugin. Not sure a plugin can work around this though as plugins cannot modify the internal structure of the database (understandably).

Anyway, maybe someone can enlighten me to the thinking behind this (or just ignore me :-) ) or maybe we can discuss it on slack. Either way, Netbox is great and thanks for making it open source.

</rant>

Mahomed

[Brian Candler]

> But if you looked at the config of these devices (e.g on Ciscos and pfSense) you would see that the IP is configured on the interface on each device. 

FWIW, Cisco (ASAs) are a bit different again.  You can configure them like this:

interface GigabitEthernet0/1.9
 vlan 9
 ip address 192.168.9.1 255.255.255.0 standby 192.168.9.2 

The devices are physically "primary" and "secondary",  but their roles are "active" and "standby".  In the above example, whichever device is currently "active" has the .1 address, and whichever is currently "standby" has the .2 address.  On failover, they swap roles and hence addresses.

By contrast, in pfSense, one device would always have .2, one would always have .3, and the .1 would VRRP between them.

Anyway, I can see two ways to achieve what you're asking for:

1. Change the uniqueness validation so that addresses with role HSRP, VRRP etc do not need to be unique.  You'd then create two different IP address objects with the same name and role, and assign them to the two interfaces.  It could be done without major changes I think.  When you search for the address, you see the two device interfaces it's associated with, so I think that does the job.

2. Change the data model so that interface<->IP address becomes a many-to-many relationship.  I really don't like this; it's almost always incorrect, so it opens up lots of scope for errors, just as duplicate IP addresses do; and it's covering a very minor use case.  (It could be constrained so that >1 links can only be made if the role is HSRP/VRRP etc)

3. Extend the "virtual chassis" model - or create a new model - to represent failover pairs. In that case, eth1 on device A and eth1 on device B are effectively the same interface; you'd assign the IP address to the master but connect cables to master and slave.

It would probably work, but I can think of cases which are problematic: (1) failover cables between ethX on device A and ethX on device B; (2) devices A and B have separate management interfaces and addresses, which don't fail over.

[Mahomed Hussein]

FWIW, Cisco (ASAs) are a bit different again.

ASAs should all be burned with fire! Technically they aren't using Virtual or Share IPs. They are just doing the equivalent of removing an IP and replacing it with another IP. Though I noticed that Netbox has a "Secondary" role for IPs. So I assume that could be used in this case where both ASAs either have both IPs on the same interface with one having the role of VIP (as there's not Primary role) and the other having the role of Secondary. If you were reading the details from Netbox to then fill in a template, I assume you'd read it something like this (disclaimer: I only used ASAs very briefly):

FW1:

192.168.9.1 - Role: Primary
192.168.9.2 - Role: Secondary

Actual config:

interface GigabitEthernet0/1.9
 vlan 9
 ip address 192.168.9.1 255.255.255.0 standby 192.168.9.2 

FW2:

192.168.9.2 - Role: Primary
192.168.9.1 - Role: Secondary

Actual config:

interface GigabitEthernet0/1.9
 vlan 9
 ip address 192.168.9.2 255.255.255.0 standby 192.168.9.1 

By contrast, in pfSense, one device would always have .2, one would always have .3, and the .1 would VRRP between them.

Yes, this is the more common setup and exactly how it works with HSRP, VRRP (including IOS/IOS XR on the Cisco 6500/ASR9K platforms), CARP, keepalived.

In regards to your suggestions, Option 2 with the caveat that it's limited to certain roles only is the real (and correct option IMHO). And with all due respect, I do not think this is a minor use case. Firstly, a search of the mailing list for VRRP/CARP/HSRP brings this question up several times over the years. Secondly, with our increasing "Always Up/Always online" lifestyles, load balancers and/or HA failover clusters are becoming more of a necessity in order to keep 24/7 operations going. I also worked at a Data Centre that provided ethernet circuits for every client and these were run to two separate routers with HSRP for the client's gateway (just as an example, because I am sure most people using Netbox in the DC environment must be running HA).

Thanks for your time and consideration.

[Daniel Sheppard]

> Change the uniqueness validation so that addresses with role HSRP, VRRP etc do not need to be unique.

This should already be the case, see here: https://master.netbox.dev/ipam/ip-addresses/262/

[Mahomed Hussein]

That is indeed the case Daniel, but it results in multiple IPs showing in the IP address list. As well as a duplicate IP warning. There probably are edge cases where this is desired, but I'd say in general, there really should not be any reason to see the same IP multiple times. Instead you should see that the same IP is assigned to multiple devices (if that's the case and it's one of the types of IPs that can be assigned to multiple devices i.e. it's not an error). Any other IP that get's assigned multiple times should actually bring up the duplicate IP warning (unless the IP is in a separate VRF) because this probably means someone has created an IP that they haven't checked is already created/assigned or it could've been a simple typo.

[Christian Paasche]

Hi,

Thanks for the replies.

I agree with Mahomed's point #2 that designating the IP address as CARP, etc should allow the IP to be assigned to multiple interfaces.  If not designated as CARP, etc, it should trigger a warning.

As it stands, you get the duplicate IP warning if you do not designate the IP as CARP, etc.  But essentially that allows the duplicate IP, makes the IP listing a mess and associates a tag with the interface.

Christian

[Daniel Sheppard]

That is indeed the case Daniel, but it results in multiple IPs showing in the IP address list. As well as a duplicate IP warning. There probably are edge cases where this is desired, but I'd say in general, there really should not be any reason to see the same IP multiple times. Instead you should see that the same IP is assigned to multiple devices (if that's the case and it's one of the types of IPs that can be assigned to multiple devices i.e. it's not an error). Any other IP that get's assigned multiple times should actually bring up the duplicate IP warning (unless the IP is in a separate VRF) because this probably means someone has created an IP that they haven't checked is already created/assigned or it could've been a simple typo.

However this isn't the real world data model.  You don't go into one switch and say "this IP is HSRP" and then it assign it to the other device.  You instead have to configure the IP on the second device as well.  There are cases where HSRP can break and there are actually two .x.y ip's that are duplicate on the network.  You essentially have two addresses on the network, it is just through the protocol they function as one.

The data model most likely won't change in the near (or far) future.

[Christian Paasche]

"However this isn't the real world data model."

I have two Palo Alto firewalls in HA mode.

The same VRFs and IPs are assigned to the same interfaces on each HA device.

The problem is that Netbox insists that I create duplicate IPs in order to assign the same IP to both HA instances.  This makes the IP listing messy.

From my viewpoint, the "real world data model" does not take into account the real world.

[Jeremy Stretch]

Folks, as Dan has pointed out, the data model for IP assignment in NetBox works is proven and isn't going to change. Adopting many-to-many IP assignment has been proposed in the past and declined for numerous reasons. I understand some people may not prefer NetBox's approach, in which case NetBox probably isn't the right tool for you.

--
You received this message because you are subscribed to the Google Groups "NetBox" group.
To unsubscribe from this group and stop receiving emails from it, send an email to netbox-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netbox-discuss/48592d55-e7ed-4ab0-91dd-a1bfb5e7eb5f%40googlegroups.com.

--

Jeremy Stretch
Sr. Network Automation Engineer
Network to Code, LLC

[Christian Paasche]

"Folks, as Dan has pointed out, the data model for IP assignment in NetBox works is proven and isn't going to change. Adopting many-to-many IP assignment has been proposed in the past and declined for numerous reasons. I understand some people may not prefer NetBox's approach, in which case NetBox probably isn't the right tool for you."

Could you please add this information to the documentation in the IP section?

It would save people considerable time when getting up to speed.

Other than this issue, Netbox is awesome so far.

[Rob Duffy]

Christian,

You need to create the IP twice in Netbox.  Assign copy one to Eth1/1 HA1 and copy two to Eth1/1 HA2.  

--
You received this message because you are subscribed to the Google Groups "NetBox" group.
To unsubscribe from this group and stop receiving emails from it, send an email to netbox-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/netbox-discuss/a284b3d4-faba-4a0d-879d-6ee809127feb%40googlegroups.com.

[Mahomed Hussein]

You don't go into one switch and say "this IP is HSRP" and then it assign it to the other device.  You instead have to configure the IP on the second device as well.

But the point is that you configure it as HSRP on both devices. Then they are treated as one IP. That's just how it works.

 You essentially have two addresses on the network, it is just through the protocol they function as one.

That's the whole point - they function as one. They should be represented as one.

There are cases where HSRP can break and there are actually two .x.y ip's that are duplicate on the network.

This is an extremely rare edge case that is, exactly as you say - a breakage. This is a fault and something that should be fixed and has nothing to do with record keeping. If an end user (with no access to Netbox) manually added a duplicate IP to a device on the network, you wouldn't go to Netbox to find out who did it or which machine it is. You would use other monitoring tools to do that. I am also curious how many times you've seen HSRP break because I've never seen it break in 12 years of using it in a data centre environment with separate customers all connecting to our routers running HSRP or VRRP for the gateway.

 

[Mahomed Hussein]

Jeremy, it's your software so I'll respect your decision (even though I very strongly disagree)  and this is the last I'll say on it. It would be really helpful if you could list these reasons (as Christian suggested, the documentation would be a good place) as I'm sure this issue will keep coming up.

Apart from that, Netbox is great and thank you (and all the contributors) for making it available.

To unsubscribe from this group and stop receiving emails from it, send an email to netbox-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/netbox-discuss/48592d55-e7ed-4ab0-91dd-a1bfb5e7eb5f%40googlegroups.com.

[Christian Paasche]

Hi Rob,

Thanks for your reply.

I've elected to just delete the IPs from the second HA device altogether because:

Duplicate IPs make the IP table messy and generates warnings.  I only want instance of an IP.

I only have a few HA locations and both devices have identical configurations. 

I am only using Netbox for documentation purposes at this point.

Christian

On Tuesday, May 26, 2020 at 2:22:19 PM UTC-7, Rob Duffy wrote:

Christian,

You need to create the IP twice in Netbox.  Assign copy one to Eth1/1 HA1 and copy two to Eth1/1 HA2.  

On Mon, 25 May 2020 at 17:40, Christian Paasche <cpaa...@gmail.com> wrote:

Hi,

I am very new to Netbox and love the product but am experiencing the normal bumps getting up to speed.

Our network has multiple locations using HA pairs of Palo Alto firewalls.  We are using the same VRFs on each HA firewall pair.

Documenting our HA pairs, I am assigning an IP address to Eth 1/1 of firewall HA1.

When I assign the same IP address to Eth 1/1 on firewall HA2, the IP address just appears on firewall HA2 and is gone from Eth 1/1 on HA1.

I have tried using the the CARP role but just adds a tag to the IP address.

Am I missing something?  I have read the documentation and search online for tutorials.  Nada.

How to assign the same IP to the same interface and VRF across HA devices?

Thanks.

--
You received this message because you are subscribed to the Google Groups "NetBox" group.

To unsubscribe from this group and stop receiving emails from it, send an email to netbox-...@googlegroups.com.

[Rob Duffy]

That makes sense.  It's probably the cleanest solution.  You might want to look into using reports to make sure all objects are configured as expected.  You can make one quite easily to check for duplicate IPs. 

To unsubscribe from this group and stop receiving emails from it, send an email to netbox-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netbox-discuss/893fee67-cd34-4bd8-bd60-2056c16f243b%40googlegroups.com.

[Christian Paasche]

Understood and thanks.  I am just starting our documentation process so I likely have other questions.

Netbox is what is exactly what we need so I am grateful for the application and the responses.

I did not mean to cause a fuss but it seems the question has come up before.   Documentation would likely save future aggravation.

Peace. Out.

To view this discussion on the web visit https://groups.google.com/d/msgid/netbox-discuss/893fee67-cd34-4bd8-bd60-2056c16f243b%40googlegroups.com.

[Daniel Sheppard]

It is possible in the future when the maintainers have time, we could look at removing the warnings for something like duplicate IP's if all the IP's are part of a "HA" role (HSRP, Anycast, etc)