Issues with namespace and yang configuration

[chandan kumar]

Hello All,

I am using libnetconf to implement a netconf server over sshsubsystem. I took the example from netopeer server-sl code to write my own.

As of now "get" and "get-config" works well in my implementation, however, I can't get edit-config working. I started debugging libnetconf and it appears some namespace issue [I have limited knowledge about YANG/YIN. I used lnctool in dev_tools to convert the yang to yin file].

Basically it fails to generate the xmldiff. at

File: transapi.c
Function: transapi_running_changed:
Line: xmldiff_diff(&diff, old_doc, new_doc, ds->ext_model_tree) == XMLDIFF_ERR)

Inside xmldiff fails at this if statement.

if (old == NULL || new == NULL || diff == NULL || model == NULL) {
ERROR("%s: invalid parameter \"%s\".", __func__, !old ? "old" : !new ? "new" : !diff ? "diff" : "model");
return XMLDIFF_ERR;
}

I am pasting below first few lines of YIN and generated XML file for your reference. Let me know should I attach the whole YANG file.

YIN
--------------------------------------------------
<?xml version="1.0" encoding="UTF-8"?>
<module xmlns="urn:ietf:params:xml:ns:yang:yin:1" xmlns:of12-config="urn:onf:of12:config:yang" xmlns:yang="urn:ietf:params:xml:ns:yang:ietf-yang-types" xmlns:inet="urn:ietf:params:xml:ns:yang:ietf-inet-types" name="wd">
<namespace uri="urn:onf:of12:config:yang"/>
<prefix value="wd"/>
<import module="ietf-yang-types">
<prefix value="yang"/>
</import>
<import module="ietf-inet-types">
<prefix value="inet"/>
</import>
------------------------------------
XML generated based on YIN
-------------------------------------

<?xml version="1.0"?>
<datastores>
<startup lock=""/>
<candidate lock=""/>
<running lock="">
<capable-switch xmlns:wd="urn:onf:of12:config:yang">
<id>NoviCapableSwitch</id>
<configuration-points>
<configuration-point>
<id>root</id>
<uri>192.168.2.34</uri>
<protocol>ssh</protocol>
</configuration-point>
</configuration-points>
<resources>

-----------------------------


netconf> edit-config running

Type the edit configuration data (close editor by Ctrl-D):
<capable-switch xmlns:wd="urn:onf:of12:config:yang"><configuration-points><configuration-point><id>root</id><uri wd:operation="merge">192.168.2.34</uri></configuration-point></configuration-points>
</capable-switch>
NETCONF error: operation-failed (application) - Failed to apply configuration changes to device.

---------------

Thanks in advance for any help.

[Radek Krejčí]

Hi,

I believe the problem is in your edit-config data, resp. in the namespaces you are (not) using. While the operation attribute is supposed to be in the netconf namespace, the data must be placed in the namespace specified by the data model. So, in your case, the edit-config data from your example should look like this:

<capable-switch xmlns="urn:onf:of12:config:yang" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0">

<configuration-points>
<configuration-point>
<id>root</id>

<uri nc:operation="merge">192.168.2.34</uri>
</configuration-point>
</configuration-points>
</capable-switch>

And just a note - in this particular case, the operation attribute is actually useless, since merge is edit-config's default operation. For more details, see RFC 6241, section 7.2 (there are also examples).

Regards,
Radek


Dne 25.3.2015 v 16:58 chandan kumar napsal(a):

[chandan kumar]

I missed one point to mention [may be most important].

libnetconfi fails at much earlier stage while validating the namespace [I commented that part to move ahead].

It fails at

is_model_root() at line 5790:
File: datastore.c

Inside the function it could not find namespace of the root node [root->ns is null here]
if (root->ns == NULL || xmlStrcmp(root->ns->href, BAD_CAST (data_model->ns)) != 0) {

Not sure if name space is missing in the xml or yin file.

[chandan kumar]

Hi Radek,

Thanks for replying. Adding another namespace in the edit command helped. Now it passes from the is_root_model_root() function, but it still fails at xmldiff function.

Thanks
Chandan

[Radek Krejčí]

Hi,
yes, I see another problem - how did you created that "XML generated based on YIN"? It wasn't created by libnetconf, right? Because it contains invalid data - the capable-switch element has no namespace. The datastore file is not supposed for manual modification until you relly know what you are doing :) Remove the file and use edit-config or copy-config - remember to use namespaces correctly as I showed in the previous email.

Radek

Dne 25.3.2015 v 17:23 chandan kumar napsal(a):

--
Radek Krejci
mobile : +420 732 212 714
office : +420 234 680 256
e-mail : rkr...@cesnet.cz
LinkedIn: http://www.linkedin.com/in/radekkrejci

CESNET, Association of Legal Entities
Zikova 4
160 00 Praha 6
Czech Republic

[chandan kumar]

> e-mail : rkre...@cesnet.cz

> LinkedIn: http://www.linkedin.com/in/radekkrejci
>
> CESNET, Association of Legal Entities
> Zikova 4
> 160 00 Praha 6
> Czech Republic

Hi Radek,

It appears I have some issues in the generated XML file [owing to missing namespaces].

Now I added namespace into the xml file and it looks like..

---------------------------------------
<?xml version="1.0"?>
<datastores xmlns:wd="urn:ietf:params:xml:ns:netconf:base:1.0">

<startup lock=""/>
<candidate lock=""/>
<running lock="">

<of12:capable-switch xmlns:of12="urn:onf:of12:config:yang">
<of12:id>NoviCapableSwitch</of12:id>
<of12:configuration-points/>
<of12:configuration-point>
<of12:id>root</of12:id>
<of12:uri>192.168.2.34</of12:uri>
</of12:configuration-point>
</of12:configuration-points>
<of12:resources>
</of12:capable-switch>
</running>
</datastores>

-------------------

edit-config running

<of12:capable-switch>
<of12:configuration-points>
<of12:configuration-point operation="merge">
<of12:id>root</of12:id>
<of12:uri>192.168.2.32</of12:uri>
</of12:configuration-point>
</of12:configuration-points>
</of12:capable-switch>


-----------------

With this XML and edit-config messgae, the XML file is being updated[modified with uri=192.168.2.32 from 192.168.2.34], however, transapi call backs are not being called [owing to xmldiff failure as explained to you earlier].

Moreover, after libnetconf updates the XML it adds a name space xmlns:wd="urn:ietf:params:xml:ns:netconf:base:1.0" to capable-switch element.

Eg: <of12:capable-switch xmlns:of12="urn:onf:of12:config:yang" xmlns:wd="urn:ietf:params:xml:ns:netconf:base:1.0">


Not sure whether the xmldiff is failing because of namespace [I hope namespaces are correct now? :-)], or something else [It is driving me crazy :-)]

Appreciate your help in this regard.

[Radek Krejčí]

once again - stop the server, _remove the datastore file_ (using rm command in shell), and then use edit-config to create the datastore content. The current content of your datastore is not a valid XML! And study something about XML namespaces.You can also try to set libnetconf verbosity in your server usingnc_verbosity() function (and setting printing callback via nc_callback_print() if printing to stderr is not usefull) to get more information what libnetconf does in background.

To set up the initial datastore content, you can use e.g. something like this (note that it is not the same content as you posted):

netconf> edit-config running

<of12:capable-switch xmlns:of12="urn:onf:of12:config:yang">
<of12:id>NoviCapableSwitch</of12:id>
<of12:configuration-points/>
<of12:configuration-point>
<of12:id>root</of12:id>
<of12:uri>192.168.2.34</of12:uri>
</of12:configuration-point>
</of12:configuration-points>

</of12:capable-switch>

And then to modify it:

netconf> edit-config running

<of12:capable-switch xmlns:of12="urn:onf:of12:config:yang">

<of12:configuration-points>

<of12:configuration-point>
<of12:id>root</of12:id>

<of12:uri>192.168.2.32</of12:uri>
</of12:configuration-point>
</of12:configuration-points>
</of12:capable-switch>

Radek


Dne 25.3.2015 v 22:46 chandan kumar napsal(a):

[chandan kumar]

I was already debugging the library the way you suggested [verbosity  and adding extra prints inside the code].

Eventually now it is working. There were following issues.

1. Potential libnetconf bug, where it was not updating datastores external model tree for transAPI.

diff --git a/src/datastore.c b/src/datastore.c
index f611130..a752a6a 100644
--- a/src/datastore.c
+++ b/src/datastore.c
@@ -5235,6 +5235,10 @@ static nc_reply* ncds_apply_transapi(struct ncds_ds* ds, const struct nc_session
                /* announce error-option to the TransAPI module, if error-option not set, announce default stop-on-error */
                for (tapi_iter = ds->transapis; tapi_iter != NULL; tapi_iter = tapi_iter->next) {
                        *(tapi_iter->tapi->erropt) = (erropt != NC_EDIT_ERROPT_NOTSET) ? erropt : NC_EDIT_ERROPT_STOP;
+                       if ((ds->ext_model_tree = yinmodel_parse(ds->ext_model, tapi_iter->tapi->ns_mapping)) == NULL) {
+                               return EXIT_FAILURE;
+                       }
+
                }
 
                /* add default values */

2. libnetconf : replacing the namespace in the callback paths [callbacks[i].path] with "A" using a default NS pair structure.

struct ns_pair ext_ns_mapping[MAPPING_SIZE] = {
            {"A",NULL},{"B",NULL},{"C",NULL},{"D",NULL},{"E",NULL},{"F",NULL},
            {"G",NULL},{"H",NULL},{"I",NULL},{"J",NULL},{"K",NULL},{"L",NULL},{"M",NULL},
            {"N",NULL},{"O",NULL},{"P",NULL},{"Q",NULL},{"R",NULL},{"S",NULL},{"T",NULL},
            {"U",NULL},{"V",NULL},{"W",NULL},{"X",NULL},{"Y",NULL},{"Z",NULL},{NULL,NULL},};


3. Earlier the XML file I was generating had namespace issues, however, in the last example I sent, it was correct.


Hope this helps anyone using the lib.

Thanks for all your help.

Thanks
Chandan

[Radek Krejčí]

Hi Chandan,

Dne 26.3.2015 v 23:00 chandan kumar napsal(a):

I was already debugging the library the way you suggested [verbosity  and adding extra prints inside the code].

Eventually now it is working. There were following issues.

1. Potential libnetconf bug, where it was not updating datastores external model tree for transAPI.

diff --git a/src/datastore.c b/src/datastore.c
index f611130..a752a6a 100644
--- a/src/datastore.c
+++ b/src/datastore.c
@@ -5235,6 +5235,10 @@ static nc_reply* ncds_apply_transapi(struct ncds_ds* ds, const struct nc_session
                /* announce error-option to the TransAPI module, if error-option not set, announce default stop-on-error */
                for (tapi_iter = ds->transapis; tapi_iter != NULL; tapi_iter = tapi_iter->next) {
                        *(tapi_iter->tapi->erropt) = (erropt != NC_EDIT_ERROPT_NOTSET) ? erropt : NC_EDIT_ERROPT_STOP;
+                       if ((ds->ext_model_tree = yinmodel_parse(ds->ext_model, tapi_iter->tapi->ns_mapping)) == NULL) {
+                               return EXIT_FAILURE;
+                       }
+
                }
 
                /* add default values */

I'm not sure what are you trying to fix, but it is definitely wrong. You are rewriting datastore's ext_model_tree in _loop_. The ext_model_tree is supposed to be filled by calling ncds_consolidate from your server, so if your transAPI module has some callbacks, the ext_model_tree is already prepared. I believe, that your problem is that your transAPI module has no callback (libnetconf prints it as a verbose message) and libnetconf then fails when it tries to prepare xml diff. You are right that this should be better processed to avoid even try for creating xml diff since it is not usable in that case, but your fix does not solve it. I will try to prepare a patch. Anyway, your problem is probably that your transAPI module has no callback (or libnetconf does not know about them, plese check that you are calling ncds_consolidate() in your server) - can you share you transAPI module source code and complete verbose output of the libnetconf?

2. libnetconf : replacing the namespace in the callback paths [callbacks[i].path] with "A" using a default NS pair structure.

struct ns_pair ext_ns_mapping[MAPPING_SIZE] = {
            {"A",NULL},{"B",NULL},{"C",NULL},{"D",NULL},{"E",NULL},{"F",NULL},
            {"G",NULL},{"H",NULL},{"I",NULL},{"J",NULL},{"K",NULL},{"L",NULL},{"M",NULL},
            {"N",NULL},{"O",NULL},{"P",NULL},{"Q",NULL},{"R",NULL},{"S",NULL},{"T",NULL},
            {"U",NULL},{"V",NULL},{"W",NULL},{"X",NULL},{"Y",NULL},{"Z",NULL},{NULL,NULL},};

I don't know what is this issue about. What is the problem with unifying namespace prefixes internally? Actually, this task is necessary. Maybe it is connected with 3.

3. Earlier the XML file I was generating had namespace issues, however, in the last example I sent, it was correct.

No, it wasn't or you didn't provided enough data. Please compare your example data with my answer more carefully.

The datastore content:
- there is resources opening tag without closing tag. Therefore it is not a well-formed XML document

edit-config data:

- of12 prefix is not defined anywhere
- the 'operation' attribute in 'configuration' element has no namespace

Radek

--
You received this message because you are subscribed to the Google Groups "libnetconf" group.
To unsubscribe from this group and stop receiving emails from it, send an email to libnetconf+...@googlegroups.com.
Visit this group at http://groups.google.com/group/libnetconf.
For more options, visit https://groups.google.com/d/optout.

[Radek Krejčí]

Hi,

with commit 47b43162d6 (master) or e606a1ce19 (0.9.x) libnetconf allows
to use transAPI modules with no data callback (i.e. only with rpc
callbacks, or simply without any callback at all for any reason).

Regards,
Radek

Dne 27.3.2015 v 08:28 Radek Krejčí napsal(a):

[chandan kumar]

Hi Radek,

Please find the verbose logs as requested.

Yes, I am doing ncds_consolidate().

I have attached the log. This log contains connect, and edit-config using netopeer cli client application.

Thanks
Chandan

[Radek Krejčí]

Hi Chandan,
please, let me clarify what is your problem, ok?

- you have your own NETCONF server implementation using libnetconf (the 47b43162d6 commit from master or newer)
- you have your own transAPI module for of-config 1.2
- you have empty running datastore
- when you create some of-config data in running, it does nothing - no configuration data are created in running datastore nor the of-config transAPI callback is used

Am I right in all points? If you have some content in the running before performing the edit-config request (from the log), can you provide the content of the datastore (use get-config)?

Is the code of your server and the transAPI module open source and available somewhere?

Radek

Dne 1.4.2015 v 20:58 chandan kumar napsal(a):

--

You received this message because you are subscribed to the Google Groups "libnetconf" group.
To unsubscribe from this group and stop receiving emails from it, send an email to libnetconf+...@googlegroups.com.
Visit this group at http://groups.google.com/group/libnetconf.
For more options, visit https://groups.google.com/d/optout.

-- 
Radek Krejci
mobile  : +420 732 212 714
office  : +420 234 680 256

e-mail  : rkr...@cesnet.cz