Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

ifconfig: ioctl (SIOCAIFADDR): File exists - but not only with alias command

634 views
Skip to first unread message

Raimundo Santos

unread,
Oct 16, 2013, 2:17:23 PM10/16/13
to
Hello list,

I have a quad port Intel NIC and I am stucked on this problem: whenever I
try to modify the address of one port of this NIC, I see from ifconfig

ifconfig: ioctl (SIOCAIFADDR): File exists

Here is the status of one port:

# ifconfig igb1
igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
ether 00:1b:21:5a:93:31
inet6 fe80::21b:21ff:fe5a:9331%igb1 prefixlen 64 scopeid 0x2
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active

As you can see, no inet address. Try to put an address that was used before
in the same port of this NIC, and..


# ifconfig igb1 XX.XX.XX.20/28
ifconfig: ioctl (SIOCAIFADDR): File exists

I have routed running with this /etc/gateways:

ripv2
if=tun0 no_rip
if=tun1 no_rip
if=et0 no_rip
if=et1 no_rip
if=igb2 no_rip
if=igb3 no_rip
rdisc_interval=45
no_ag
no_super_ag
subnet=X.Y.Z.56/30,1
subnet=X.Y.Z.0/29,1
subnet=X.Y.Z.8/29,1
subnet=X.Y.Z.16/29,1
subnet=X.Y.Z.24/29,1
subnet=X.Y.Z.32/29,1
subnet=X.Y.Z.40/29,1
subnet=X.Y.Z.48/29,1
subnet=X.Y.Z.64/29,1
subnet=X.Y.Z.72/29,1
subnet=X.Y.Z.80/29,1
subnet=X.Y.Z.88/29,1
subnet=X.Y.Z.96/29,1
subnet=X.Y.Z.104/29,1
subnet=X.Y.Z.112/29,1
subnet=X.Y.Z.120/29,1

The problem occurs when configuring the primary address or an alias in any
port, after unconfigure some other address.

Using

# uname -a
FreeBSD XXX 9.1-RELEASE-p4 FreeBSD 9.1-RELEASE-p4 #10: Thu Aug 1 19:04:09
BRT 2013 XXX:/usr/obj/usr/src/sys/XXX_KERNEL amd64

A custom system with 0 thru 15 FIBs, FWD in ipfw, and VNET/VIMAGE all
enabled.

Thank you!
Raimundo Santos
_______________________________________________
freeb...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net...@freebsd.org"

Alan Somers

unread,
Oct 16, 2013, 2:54:31 PM10/16/13
to
I ran into the same problem, on a system with LAGGs, multiple FIBs and
multiple aliases per interface. I believe that the problem was due to
a race condition in the kernel. Two process tried to SIOCAIFADDR with
the same address simultaneously. The second one of them failed and
printed the error message that you see. But due to the race, the
first process reported success, yet the interface did not get the
address. Unfortunately, I was unable to locate the race. My solution
was to prevent two processes from trying to initialize the interface
at the same time. The first process was our custom management
software, which calls "service netif cloneup lagg0" and "service netif
start lagg0" in rapid succession. The second process was devd, which
was trying to initialize the newly attached lagg0 interface. I
commented out the following lines in /etc/devd.conf and the problem
went away. I don't know if this will solve your problem, but I
recommend that you use dtrace or some other method to determine
whether two processes are trying to SIOCAIFADDR at the same time.

notify 0 {
match "system" "IFNET";
match "type" "ATTACH";
action "/etc/pccard_ether $subsystem start";
};

-Alan

Raimundo Santos

unread,
Oct 16, 2013, 3:13:54 PM10/16/13
to
On 16 October 2013 15:54, Alan Somers <aso...@freebsd.org> wrote:

>
>
> I ran into the same problem, on a system with LAGGs, multiple FIBs and
> multiple aliases per interface. I believe that the problem was due to
> a race condition in the kernel. Two process tried to SIOCAIFADDR with
> the same address simultaneously. The second one of them failed and
> printed the error message that you see. But due to the race, the
> first process reported success, yet the interface did not get the
> address. Unfortunately, I was unable to locate the race. My solution
> was to prevent two processes from trying to initialize the interface
> at the same time. The first process was our custom management
> software, which calls "service netif cloneup lagg0" and "service netif
> start lagg0" in rapid succession. The second process was devd, which
> was trying to initialize the newly attached lagg0 interface. I
> commented out the following lines in /etc/devd.conf and the problem
> went away. I don't know if this will solve your problem, but I
> recommend that you use dtrace or some other method to determine
> whether two processes are trying to SIOCAIFADDR at the same time.
>
> notify 0 {
> match "system" "IFNET";
> match "type" "ATTACH";
> action "/etc/pccard_ether $subsystem start";
> };
>
>
Hey Alan!

Well, I don`t think it is the problem here. I just try to do a very simple
ifconfig, nothing more, there are no private applications running (I have
postgres siting around and munin gathering sysinfo to me). I was thinking
about some problem in the interaction ifconfig/adresses + routed, but this
is very strange.

In a less destructive test (the first one took me to reboot the machine), I
see a confusing behavior:

. put a alias into an igb1
. remove this alias (-alias)
. printing the main (#0) FIB, nothing is there related to the removed alias
. put the same alias again
. get the File exists result and no configuration of an alias over the igb1
. but now, listing the main routing table, there are info about that
alias!, but in another port of the NIC, igb2

How is this possible?

Alan Somers

unread,
Oct 16, 2013, 3:21:20 PM10/16/13
to
It's sad to say, but the fib code is pretty buggy. Without diving
into the source, one thing that you could try is the
net.add_addr_allfibs tunable. I think that it defaults to 1. If you
set it to 0, then adding an interface address in one fib will not
cause it to create routes in other fibs. Does that help?

Alexander V. Chernikov

unread,
Oct 17, 2013, 3:05:51 AM10/17/13
to
On 16.10.2013 22:17, Raimundo Santos wrote:
> Hello list,
Hello!

>
> I have a quad port Intel NIC and I am stucked on this problem: whenever I
> try to modify the address of one port of this NIC, I see from ifconfig
>
> ifconfig: ioctl (SIOCAIFADDR): File exists
>
> Here is the status of one port:
>
> # ifconfig igb1
> igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>
> options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> ether 00:1b:21:5a:93:31
> inet6 fe80::21b:21ff:fe5a:9331%igb1 prefixlen 64 scopeid 0x2
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> media: Ethernet autoselect (1000baseT <full-duplex>)
> status: active
>
> As you can see, no inet address. Try to put an address that was used before
> in the same port of this NIC, and..
>
>
> # ifconfig igb1 XX.XX.XX.20/28
> ifconfig: ioctl (SIOCAIFADDR): File exists
This seems rather strange. I'm using multiple fibs in production, and it
shouldn't behave that way.


Can you do the following and send me output?

1) issue `route -n monitor` and log all messages
2) add first address to interface
3) do route -n get network_address (e.g. XX.XX.XX.16/28) for added
prefix (in fib 0 and fib 1)
4) remove this address
5) repeat step 3
6) add this prefix to another port
7) repeat step 3
signature.asc

Raimundo Santos

unread,
Oct 17, 2013, 1:31:56 PM10/17/13
to
Oh my... I forgot to reply to list... an here we go again!


On 17 October 2013 13:46, Alexander V. Chernikov <meli...@freebsd.org>wrote:

> On 17.10.2013 18:25, Raimundo Santos wrote:
>
> On 17 October 2013 04:05, Alexander V. Chernikov <meli...@freebsd.org>wrote:
>
>> This seems rather strange. I'm using multiple fibs in production, and it
>> shouldn't behave that way.
>>
>>
> It occurs even with aliases.
>
>>
>> Can you do the following and send me output?
>>
>> 1) issue `route -n monitor` and log all messages
>> 2) add first address to interface
>> 3) do route -n get network_address (e.g. XX.XX.XX.16/28) for added
>> prefix (in fib 0 and fib 1)
>> 4) remove this address
>> 5) repeat step 3
>> 6) add this prefix to another port
>> 7) repeat step 3
>>
>>
> Hello, Alexander!
>
> The machine in question are under production, too. It hurts the results
> if I do these steps without removing all the addresses from the interface?
>
> By the way, this machine is a "border" router (not using BGP by now, but
> over the borders of my network and my providers network). All the addresses
> over this Intel NIC are routable, except in one port, where we have some
> equipment lying out the network (just don't ask why...) and all of them
> have private addresses. I saw the issue when trying to comunicate to these
> equipments adding and removing aliases in one of the ports. That said,
> there is an intersting output, after removing ALL the aliases from the
> refered NIC port:
>
> command issued:
>
> for i in `seq 0 14`; do \
> echo "fib $i" >> routing.fibs; \
> setfib $i netstat -rnfinet | grep '172\.31\.' >> routing.fibs; \
> echo "--------------------------------" >> routing.fibs; \
> done
>
> status of igb1 and 2:
>
> igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>
> options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> ether 00:1b:21:5a:93:31
> inet6 fe80::21b:21ff:fe5a:9331%igb1 prefixlen 64 scopeid 0x2
> inet (routable addr) netmask 0xfffffff0 broadcast (routable addr
> broadcast)
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> media: Ethernet autoselect (1000baseT <full-duplex>)
> status: active
>
> igb2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>
> options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> ether 00:1b:21:5a:93:34
> inet (routable addr) netmask 0xfffffff8 broadcast (routable
> broadcast)
> inet6 fe80::21b:21ff:fe5a:9334%igb2 prefixlen 64 scopeid 0x3
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> media: Ethernet autoselect (100baseTX <full-duplex>)
> status: active
>
> And the output of the commands:
>
> fib 0
> 172.31.19.0/24 172.31.19.190 U 0 0 igb1
> 172.31.20.0/24 172.31.20.190 U 0 0 igb2
>
> ^^^^^^^^^^
> This is probably installed by some routing daemon (issuing RTM_CHANGE
> message)
> (can be easily checked with `route -n monitor`)
> This is fixed in r248070 ( and merged to 9.2)
>
> --------------------------------
> fib 1
> 172.31.19.0/24 link#2 U 0 0 igb1
> --------------------------------
> fib 2
> 172.31.19.0/24 link#2 U 0 0 igb1
> --------------------------------
> fib 3
> 172.31.19.0/24 link#2 U 0 0 igb1
> --------------------------------
> fib 4
> 172.31.19.0/24 link#2 U 0 0 igb1
> --------------------------------
> fib 5
> 172.31.19.0/24 link#2 U 0 0 igb1
> --------------------------------
> fib 6
> 172.31.19.0/24 link#2 U 0 0 igb1
> --------------------------------
> fib 7
> 172.31.19.0/24 link#2 U 0 0 igb1
> --------------------------------
> fib 8
> 172.31.19.0/24 link#2 U 0 0 igb1
> --------------------------------
> fib 9
> 172.31.19.0/24 link#2 U 0 0 igb1
> --------------------------------
> fib 10
> 172.31.19.0/24 link#2 U 0 0 igb1
> --------------------------------
> fib 11
> 172.31.19.0/24 link#2 U 0 0 igb1
> --------------------------------
> fib 12
> 172.31.19.0/24 link#2 U 0 0 igb1
> --------------------------------
> fib 13
> 172.31.19.0/24 link#2 U 0 0 igb1
> --------------------------------
> fib 14
> 172.31.19.0/24 link#2 U 0 0 igb1
> --------------------------------
>
> I never set 172.31.whatever over igb2! Just over igb1.
>
> Just to remeber:
>
> kernel options
>
> options VIMAGE
>
> # routing options
> options ROUTETABLES=15
> # IPFW options
> options IPFIREWALL_FORWARD
> options IPFIREWALL_VERBOSE
> options IPFIREWALL_VERBOSE_LIMIT=300
> options IPFIREWALL_DEFAULT_TO_ACCEPT
>
> Under
>
> FreeBSD 9.1-RELEASE-p4
>
> Thank you for your time!
>
>
>
Okey, Alexander, I found the culprit: routed, running in this machine as
pid 99399. This is the result of 'route -n monitor' after doing a for loop
over all fibs and manualy removing 172.31.19/24 and 172.31.20/24. First
two, the deletes, last two, the adds. No RTM_CHANGE, so far, and the ADD
only apply to fib 0. (Thank you for the route monitor advice!)

got message of size 192 on Thu Oct 17 14:14:01 2013
RTM_DELETE: Delete Route: len 192, pid: 94802, seq 1, errno 0, flags:<DONE>
locks: inits:
sockaddrs: <DST,GATEWAY,NETMASK>
172.31.19.0 172.31.19.190 (255) ffff ffff ff

got message of size 192 on Thu Oct 17 14:14:07 2013
RTM_DELETE: Delete Route: len 192, pid: 94819, seq 1, errno 0, flags:<DONE>
locks: inits:
sockaddrs: <DST,GATEWAY,NETMASK>
172.31.20.0 172.31.20.190 (255) ffff ffff ff

got message of size 191 on Thu Oct 17 14:14:29 2013
RTM_ADD: Add Route: len 191, pid: 99399, seq 1142, errno 0, flags:<DONE>
locks: inits:
sockaddrs: <DST,GATEWAY,NETMASK>
172.31.19.0 172.31.19.190
pmsg_addrs: truncated route message, only 7 bytes left


got message of size 191 on Thu Oct 17 14:14:32 2013
RTM_ADD: Add Route: len 191, pid: 99399, seq 1143, errno 0, flags:<DONE>
locks: inits:
sockaddrs: <DST,GATEWAY,NETMASK>
172.31.20.0 172.31.20.190
pmsg_addrs: truncated route message, only 7 bytes left

From where routed take these addresses? When removing an fib entry, doesn't
it means that routed will never announce/add our whatever with the
non-existing entry?

When came to changing, adding or removing an interface address, will it be
good to restart routing and routed services?

And by upgrading to 9.2-RELEASE, may the issue goes away!

Best regards,

Raimundo Santos

unread,
Oct 18, 2013, 3:53:46 PM10/18/13
to
Upgraded from 9.1 to 9.2-RELEASE, problem solved.

Thank you all!
Raimundo Santos
0 new messages