Removal of elements from config_db.json is not reflected in running config after 'config load'

[Tore Anderson]

Hi,

I am currently looking at migrating my Edgecore/Broadcom-based network from Cumulus to SONiC. While attempting to migrate my Cumulus's /etc/network/interfaces templates to SONiC's config_db.json, I noticed that elements that are removed from config_db.json are not removed from the actual running configuration, as shown in the example below. This behaviour fundamentally breaks with a template-based automation workflow based on pushing the complete new configuration and reloading. Absence of an element in the pushed config means that element should not appear in the running configuration after the reload.

Is this the intentional behaviour, and if so, is there some other way to accomplish template-based automation short of having to reboot the switch completely after each configuration change?

The below example demonstrates the issue. It was performed on an Edgecore AS5812-54X running SONiC.Edgecore-SONiC_20210917_063104_ec202012_172. The starting point is a switch with the factory default configuration, without any VLANs configured. This config I save to config_db.json:

sonic$ show vlan brief

+-----------+--------------+---------+----------------+-----------------------+-------------+

| VLAN ID   | IP Address   | Ports   | Port Tagging   | DHCP Helper Address   | Proxy ARP   |

+===========+==============+=========+================+=======================+=============+

+-----------+--------------+---------+----------------+-----------------------+-------------+

sonic$ sudo config save -y

Running command: /usr/local/bin/sonic-cfggen -d --print-data > /etc/sonic/config_db.json

Next, I add a VLAN using the CLI. As expected, it shows up in the running configuration (which I do not save to config_db.json). So far so good:

sonic$ sudo config vlan add 1234

sonic$ show vlan brief

+-----------+--------------+---------+----------------+-----------------------+-------------+

|   VLAN ID | IP Address   | Ports   | Port Tagging   | DHCP Helper Address   | Proxy ARP   |

+===========+==============+=========+================+=======================+=============+

|      1234 |              |         |                |                       | disabled    |

+-----------+--------------+---------+----------------+-----------------------+-------------+

sonic$ diff -u /etc/sonic/config_db.json <(show runningconfig all)

--- /etc/sonic/config_db.json	2021-12-08 15:02:19.332205549 +0100

+++ /dev/fd/63	2021-12-08 15:02:37.169647768 +0100

@@ -897,6 +897,11 @@

             "VERSION": "version_2_0_1"

         }

     },

+    "VLAN": {

+        "Vlan1234": {

+            "vlanid": "1234"

+        }

+    },

     "ZTP": {

         "mode": {

             "inband": "true",

Next I attempt to roll back to the previous configuration using «config load» (remember that config_db.json does not contain any VLANs):

sonic$ sudo config load -y

Running command: /usr/local/bin/sonic-cfggen -j /etc/sonic/config_db.json --write-to-db

However, unexpectedly, VLAN 1234 remains present on the switch:

sonic$ show vlan brief

+-----------+--------------+---------+----------------+-----------------------+-------------+

|   VLAN ID | IP Address   | Ports   | Port Tagging   | DHCP Helper Address   | Proxy ARP   |

+===========+==============+=========+================+=======================+=============+

|      1234 |              |         |                |                       | disabled    |

+-----------+--------------+---------+----------------+-----------------------+-------------+

sonic$ diff -u /etc/sonic/config_db.json <(show runningconfig all)

--- /etc/sonic/config_db.json	2021-12-08 15:02:54.094344724 +0100

+++ /dev/fd/63	2021-12-08 15:02:58.881650902 +0100

@@ -897,6 +897,11 @@

             "VERSION": "version_2_0_1"

         }

     },

+    "VLAN": {

+        "Vlan1234": {

+            "vlanid": "1234"

+        }

+    },

     "ZTP": {

         "mode": {

             "inband": "true",

$ ip link show dev Vlan1234

66: Vlan1234@Bridge: <BROADCAST,MULTICAST> mtu 9100 qdisc noop state DOWN mode DEFAULT group default qlen 1000

    link/ether a8:2b:b5:28:db:08 brd ff:ff:ff:ff:ff:ff

This behaviour is not limited to VLANs, it seems to apply to many different kinds of elements/resources found in config_db.json (perhaps all).

Tore

[jyhe...@gmail.com]

Hi Tore,

Actually, the command "sudo config load -y" does not work like what you expect. Per my understanding, it simply applies the settings in '/etc/sonic/config_db.json' through writing the content into Redis DB.

Thus, it takes effect when you add the "VLAN 1234" setting but has no effect when you remove the "VLAN 1234".

One method to let the removal of "VLAN 1234" to take effect is to execute "sudo config reload -y". However, it will restart all of the sonic core services so it takes time to wait for the system ready.

Another method is to use the utility script 'configlet' (https://github.com/Azure/sonic-utilities/blob/acb5d84320dfc3428802ffcb023fd06316fb88ef/scripts/configlet). With this method, it will restart any sonic core services.

There is an example to tell you how to write the required file for 'configlet' to remove the field. (See https://github.com/Azure/sonic-utilities/blob/acb5d84320dfc3428802ffcb023fd06316fb88ef/scripts/configlet#L12)

Best regards,

Charlie Chen

to...@fud.no 在 2021年12月8日 星期三下午10:30:18 [UTC+8] 的信中寫道：

[Tore Anderson]

Hi Charlie,

Your understanding of «config load»  seems to match the behaviour I observe, indeed. I also tested «config reload», but it is very disruptive - interfaces unrelated to the difference between the running and the target configuration are brought down for about 90 seconds, so it is roughly equivalent to performing a full reboot.

Thank you for the «configlet» tip. I tried to play around with it a bit, but I was unable to make it work the way I need it to, as shown below:

Starting point is a running config with two VLANs and a configlet JSON file that contains only one of them. The desired behaviour is that the VLAN missing from the configlet file should be removed, while the one that is present in both the running config and the configlet file should be left alone and not disrupted in any way.

sonic$ show runningconfiguration all | jq .VLAN

{

  "Vlan1234": {

    "vlanid": "1234"

  },

  "Vlan880": {

    "vlanid": "880"

  }

}

sonic$ ./configlet --json /etc/sonic/VLAN.configlet.json --parse

Parsed:

[{'VLAN': {'Vlan880': {'vlanid': '880'}}}]

Attempting to apply the configlet in update mode makes no change to the running config whatsoever. Presumably it only attempts to add to the running config what is already present there (similar to «config load»):

sonic$ ./configlet --json /etc/sonic/VLAN.configlet.json --update

sonic$ show runningconfiguration all | jq .VLAN

{

  "Vlan1234": {

    "vlanid": "1234"

  },

  "Vlan880": {

    "vlanid": "880"

  }

}

Attempting to apply the configlet in delete mode, just removes the «vlanid» attribute from the VLAN that was supposed to be left alone (880), leaving the VLAN that was supposed to be removed alone (1234):

sonic$ ./configlet --json /etc/sonic/VLAN.configlet.json --delete

sonic$ show runningconfiguration all | jq .VLAN

{

  "Vlan1234": {

    "vlanid": "1234"

  },

  "Vlan880": {}

}

I was hoping there would be something similar to Cumulus Linux's «ifreload -a» or JunOS's «commit». These commands work on a target configuration file (/etc/network/interfaces and /config/juniper.conf.gz, respectively) and will do whatever is necessary to bring the running configuration in sync with the target configuration. This includes removing any elements found only in the running configuration (such as VLAN 1234 in my example) and adding any elements found only in the target configuration, while elements found in both the target and the running configuration are not disrupted in any way.

Tore

--
You received this message because you are subscribed to the Google Groups "sonicproject" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonicproject...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonicproject/91e77cf0-6f63-4bac-8d5b-07a0ec282fb4n%40googlegroups.com.