Java API to interoperate with netopeer-server.

[hel...@ufersa.edu.br]

Hi, folks.

Can somebody tell me what is the most suitable Java API for development of NETCONF managers that (successfully) interoperate to netopeer-server? There are some APIs as the provided by Juniper and others like netconf4j and netconf4android. I'm trying to use the Juniper's API, because it seems the most well-documented and updated. However, an error occurs when I try to get the server capabilities - more accurately, the SSH channel is closed by netopeer-server.

My apologies for this off-topic issue.

Helcio.

[Radek Krejčí]

Hi Helcio,
isn't it the same problem that described Damian yesterday?

https://groups.google.com/forum/#!topic/netopeer/9ZRohOoqBLk

Can you post the <hello> message sent by the client? Maybe the steps for
debugging communication described in the thread mentioned above could help.

I don't have any experience with the NETCONF API for Java. libnetconf
now provides API for Python if it can help.

Best regards,
Radek


Dne 28.8.2014 02:43, hel...@ufersa.edu.br napsal(a):

[helcio wagner]

   Hi, Radek.

   I'm affraid it isn't. After I had lauched netopeer-server and netopeer-agent, I ran the Java client and received the same exception, wich tells that SSH2 channel was closed from a request sent by remote. The netopeer-server output was:

netopeer-server[3468]: message is a method-call

netopeer-server[3468]: message path: /org/liberouter/netopeer2/server

netopeer-server[3468]: message interface: org.liberouter.netopeer2.server

netopeer-server[3468]: message member: GetCapabilities

netopeer-server[3468]: message destination: org.liberouter.netopeer2.server

netopeer-server[3468]: Some message received

netopeer-server[3468]: Sending capabilities to agent.

   Everything looks fine. But the netopeer-agent output shows... nothing! That's it: there is no output provided by netopeer-agent, which is run separately as you recommeded.  Weird, don't you think?

   Again, thank you so much by your patience.

   Helcio

[Radek Krejčí]

Hi Helcio,

and does it work with netopeer-cli?

1) manually launched netopeer-agent will communicate only with you (not the client) via terminal.

2) However, it must print the hello message to its stdout. If not, there is some other problem and it should report it to the syslog. For example the netopeer-server is not running and after launching netopeer-agent I see the following lines in /var/log/messages:

2014-08-28T15:52:33.789623+02:00 krejci netopeer-agent[22215]: Connected with Netopeer server
2014-08-28T15:52:33.790461+02:00 krejci netopeer-agent[22215]: Cannot send message or get reply over DBus.
2014-08-28T15:52:33.791079+02:00 krejci netopeer-agent[22215]: Cannot get server capabilities!
2014-08-28T15:52:33.791365+02:00 krejci netopeer-agent[22215]: Cannot get server capabilities.

Do you have some messages from the netopeer-agent in the syslog?

3) Do you see (somewhere) what is that client sending to the server side?

4) can you provide me a link to the client (source code/download) you are using?

You are welcome, I'm also interested in this weird behavior.

Radek

Dne 28.8.2014 15:48, helcio wagner napsal(a):

-- 
Radek Krejci
mobile  : +420 732 212 714
office  : +420 234 680 256
e-mail  : rkr...@cesnet.cz
LinkedIn: http://www.linkedin.com/in/radekkrejci

CESNET
Association of Legal Entities
160 00 Praha 6, Zikova 4
Czech Republic

[helcio wagner]

   Hi, Radek.

 

   Well, I'm learning how I can use this Java API to establish a connection to netopeer-server and do some things that I did with the libnetconf API - i.e., send <create-subscription>, <edit-config> and generic <rpc> operations to netopeer-server and receive notifications from it. In order to do that with the Juniper, I must create a 'Device' object, According to the API's documentation, this class "defines the device on which the NETCONF server runs, and represents the SSHv2 connection and default NETCONF session with that device." The following lines have worked very well: 

   Device device = new Device("localhost", "root", "myRootPassword", null);

   device.connect();

   desvice.close();

   If my code is restricted to those lines, I'll see on netopeer-server output:

   netopeer-server[3482]: message is a method-call

   netopeer-server[3482]: message path: /org/liberouter/netopeer2/server

   netopeer-server[3482]: message interface: org.liberouter.netopeer2.server

   netopeer-server[3482]: message member: GetCapabilities

   netopeer-server[3482]: message destination: org.liberouter.netopeer2.server

   netopeer-server[3482]: Some message received

   netopeer-server[3482]: Sending capabilities to agent.

   For the record, netopeer-agent (which begun to run after netopeer-server had been launched) shows nothing. 

   OK, let's move to the problem itself, i.e.: an exception is thrown when I invoke the 'getRunningConfig' method, which "retrieve the whole running configuration". More accurately, the following line:

    XML capabilities = device.getRunningConfig();

   causes:

   Exception in thread "main" ch.ethz.ssh2.channel.ChannelClosedException: SSH channel is closed. (Close requested by remote)

   Again, netopeer-agent shows nothing.

   I'll try to load a module using that API and present the results. The Juniper API can be obtained from:

   http://www.juniper.net/techpubs/en_US/junos14.1/information-products/pathway-pages/netconf-java-toolkit/netconf-java-toolkit.html

   Bye, Radek.

   Helcio.

[Radek Krejčí]

Hi Helcio,

Dne 28.8.2014 18:26, helcio wagner napsal(a):

   Hi, Radek.

 

   Well, I'm learning how I can use this Java API to establish a connection to netopeer-server and do some things that I did with the libnetconf API - i.e., send <create-subscription>, <edit-config> and generic <rpc> operations to netopeer-server and receive notifications from it. In order to do that with the Juniper, I must create a 'Device' object, According to the API's documentation, this class "defines the device on which the NETCONF server runs, and represents the SSHv2 connection and default NETCONF session with that device." The following lines have worked very well: 

   Device device = new Device("localhost", "root", "myRootPassword", null);

   device.connect();

   desvice.close();

   If my code is restricted to those lines, I'll see on netopeer-server output:

   netopeer-server[3482]: message is a method-call

   netopeer-server[3482]: message path: /org/liberouter/netopeer2/server

   netopeer-server[3482]: message interface: org.liberouter.netopeer2.server

   netopeer-server[3482]: message member: GetCapabilities

   netopeer-server[3482]: message destination: org.liberouter.netopeer2.server

   netopeer-server[3482]: Some message received

   netopeer-server[3482]: Sending capabilities to agent.

I think that even in this case the session is not created. netopeer-agent is only launched by the SSH server and it asks for the server capabilities from the netopeer-server and sends them to the client. You should see something similar to this:

netopeer-server[27311]: New agent ID set to :1.4341.


Do you see, what the client is sending to the server?

   For the record, netopeer-agent (which begun to run after netopeer-server had been launched) shows nothing. 

   OK, let's move to the problem itself, i.e.: an exception is thrown when I invoke the 'getRunningConfig' method, which "retrieve the whole running configuration". More accurately, the following line:

    XML capabilities = device.getRunningConfig();

   causes:

   Exception in thread "main" ch.ethz.ssh2.channel.ChannelClosedException: SSH channel is closed. (Close requested by remote)

   Again, netopeer-agent shows nothing.

   I'll try to load a module using that API and present the results. The Juniper API can be obtained from:

   http://www.juniper.net/techpubs/en_US/junos14.1/information-products/pathway-pages/netconf-java-toolkit/netconf-java-toolkit.html

I'm going to check it.

Radek

--
You received this message because you are subscribed to the Google Groups "Netopeer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to netopeer+u...@googlegroups.com.
Visit this group at http://groups.google.com/group/netopeer.
For more options, visit https://groups.google.com/d/optout.

[Radek Krejčí]

Hi,

I found that the Juniper API doesn't send correct XML namespace in their NETCONF messages.

To correct the problem with the <hello> message, change Device.java:238 from

        helloRPC.append("<hello>\n");
to
        helloRPC.append("<hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">\n");

But I think that there will be problems also with <rpc> messages, since the messages in NetconfSession.java neither contain namespace. So there you should change these lines:

        rpc.append("<rpc>");
to
        rpc.append("<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">");


Another problem is, that the source code doesn't contain message-id string. The attribute with this name must be set to each NETCONF message (see RFC 6241 sec. 4.1).

So, I cannot recommend using the NETCONF Java Toolkit (at least in version 1.0.0) since it is not compliant with the NETCONF standard.

Regards,
Radek

Dne 1.9.2014 09:08, Radek Krejčí napsal(a):

[helcio wagner]

   Thank you so much, Radek. 

   How about Netcon4android? Have you any knowlegment about interoperability between that and netopeer?

   Best regards.

   Helcio.

[Radek Krejčí]

Hi,
I have no experience with it, but it is not maintained (last commit is from 2012) so I'm a little bit sceptic :(

Anyway, I think that it is not a bad idea to try it ;)

Regards,
Radek

Dne 1.9.2014 19:30, helcio wagner napsal(a):

[Damian Mrozowicz]

Hi Helcio,

There is a couple issues with netconf4j library:

1) Radek already mentioned there is wrong namespace ( "xmlns") declaration
2) there is also wrong capability that is advertised to the server, you need to change it in net.i2cat.netconf.rpc.Capability it should be without ":xml:ns" in the middle:
public static final Capability        BASE                    = new Capability("urn:ietf:params:xml:ns:netconf:base:1.0"); should be  changed to:
public static final Capability        BASE                    = new Capability("urn:ietf:params:netconf:base:1.0");

3) they assumed that "session-id" element may occur only in hello packet which is wrong, it may be a  problem when it occur in get/get-config content result
So  to avoid NullPointerException in the net.i2cat.netconf.transport.TransportContentParser in "endElement" method you have to add null check on hello variable:
        else if (localName.equalsIgnoreCase("session-id")) {
            insideSessionIdTag = false;
            if(hello != null) {
                hello.setSessionId(sessionIdTagContent.toString());
            }
            sessionIdTagContent = new StringBuffer();
        }

4) Radek rpc message-id is properly generated so it's ok, when you log it's null but then they are sending and increment message id just before sending message
5) The most critical change was that they were reading input stream using readLine method with is wrong, standard doesn't say that message has to end with newline. It's in

their SSHTransport class. I've refactored startParsing() it and I'm using java util.Scanner to read it using ]]>[[> as a delimiter.:
    private void startParsing() {

       
        parserThread = new Thread("Parser") {

            // private Log log = LogFactory.getLog(parserThread.class);
            @Override
            public void run() {
                log.debug("Parsing thread start");
                BufferedReader reader = null;
                Scanner scanner = null;
                while (!closed)
                {
                    try {

                        String message = null;
                        reader = new BufferedReader(new InputStreamReader(session.getStdout()));
                        scanner = new Scanner(reader);
                        scanner.useDelimiter(delimiter);
                        try {
                            message = scanner.next();
                            parser.parse(new InputSource(new StringReader(message)));
                        } catch (NoSuchElementException e) {
                            if(!closed) {
                                //TODO
                                //throw e;
                            }
                        }

                       
                       
                        /*
                         * // flag to log server response if (sessionContext.isLogRespXML()) { log.debug("Logging to " +
                         * sessionContext.getLogFileXML()); parser.parse(new InputSource(new BufferedReader(new InputStreamReader(new
                         * TeeInputStream(session.getStdout(), new FileOutputStream(sessionContext.getLogFileXML()), true))))); } else {
                         * parser.parse(new InputSource(new BufferedReader(new InputStreamReader(session.getStdout()) { public int read(char[] cbuf,
                         * int offset, int length) throws IOException { log.debug("char:" + new String(cbuf, offset, length)); return super.read(cbuf,
                         * offset, length); } }))); }
                         */
                    } catch (IOException e) {
                        log.error(e.getMessage());
                    } catch (SAXException e) {
                        if (e.getMessage().contentEquals("Content is not allowed in trailing section.")) {
                            // Using shitty non-xml delimiters forces us to detect
                            // end-of-frame by a SAX error.
                            // Do nothing will just restart the parser.
                            // Blame netconf
                        }
                        else {
                            log.error(e.getMessage());
                            disconnect();
                        }
                        log.info("End of parsing.");
                    } finally {
                        if (scanner != null) {
                            scanner.close();
                        }                       
                        if (reader != null) {
                            try {
                                reader.close();
                            } catch (IOException e) {
                                log.error(e.getMessage());
                            }
                        }
                       
                    }
                    log.debug("Looping");
                }
                log.debug("Parsing thread ended");
            }
        };
        parserThread.start();
    }
All these changes were made in netconf4j v0.8 since it was last stable (at least it's author's opinion ;)) version. But I agree with Radek this lib is very customized for Juniper API, doesn't follow RFCs and the quality is poor.

Thanks,
Damian

[Radek Krejčí]

Hi,
just a note - the delimiter should be ]]>]]>, not ]]>[[>

Damian, thanks for your notes about the netconf4j!

Regards,
Radek

Dne 4.9.2014 11:13, Damian Mrozowicz napsal(a):

[Damian Mrozowicz]

Thanks Radek for catching the typo :)

--
Pozdrawiam,
Damian Mrozowicz