Cluster Configuration in Owl (1.14) - io.atomix.utils.config.ConfigurationException: Unknown properties present in configuration: cluster.discovery.broadcastInterval

[scon...@gmail.com]

I'm following the instructions on the page Cluster Configuration in Owl (1.14) in the ONOS Wiki. I'm running on the master branch. I'm trying to form the Atomix cluster.

I've pulled the Atomix Agent Docker image 3.0.6 and I'm trying to use it with the Atomix configuration file you have given as atomix.conf (I've called mine atomix_sean.conf here) and I get

io.atomix.utils.config.ConfigurationException: Unknown properties present in configuration: cluster.discovery.broadcastInterval. See the whole stack below.

but there's no "broadcastInterval" in my conf file.

When I try running it against the Docker image of "latest" of Atomix I get

io.atomix.utils.config.ConfigurationException: Unknown properties present in configuration: cluster.name instead.

I'm guessing there's some version that doesn't have this problem - does anyone know what it is?

Also I'm finding this Wiki article is great, but it does leave me with a load of questions:

• Are the Atomix cluster nodes supposed to be another set of machines separate to the ONOS cluster nodes?
• If the Atomix cluster nodes can be co-located with the ONOS nodes (machines/VMs), do I have to have the Atomix cluster software up and running on these nodes separate to ONOS?
  
• Do the ONOS cluster nodes have to be up and running before the Atomix Cluster is formed?
• What is the minimum configuration I can have to run an ONOS cluster?
  

I've watched the TST meetings relating to Atomix and looked at discussion groups, but I can't find these answers. It would be great if these could be added to this Wiki page.

It would also be good if the Distributed ONOS Tutorial and the Running the published Docker ONOS images could be updated for this new Atomix architecture.

Sean

scondon@z420:~/onos$ docker run -v /home/scondon/onos:/mnt/onos atomix/atomix:3.0.6 -c /mnt/onos/atomix_sean.conf
Exception in thread "main" io.atomix.utils.config.ConfigurationException: Unknown properties present in configuration: cluster.discovery.broadcastInterval
    at io.atomix.core.utils.config.PolymorphicConfigMapper.checkRemainingProperties(PolymorphicConfigMapper.java:100)
    at io.atomix.utils.config.ConfigMapper.map(ConfigMapper.java:176)
    at io.atomix.utils.config.ConfigMapper.getValue(ConfigMapper.java:322)
    at io.atomix.utils.config.ConfigMapper.mapSetters(ConfigMapper.java:210)
    at io.atomix.utils.config.ConfigMapper.map(ConfigMapper.java:171)
    at io.atomix.utils.config.ConfigMapper.getValue(ConfigMapper.java:322)
    at io.atomix.utils.config.ConfigMapper.mapSetters(ConfigMapper.java:210)
    at io.atomix.utils.config.ConfigMapper.map(ConfigMapper.java:171)
    at io.atomix.utils.config.ConfigMapper.map(ConfigMapper.java:134)
    at io.atomix.utils.config.ConfigMapper.loadFiles(ConfigMapper.java:92)
    at io.atomix.core.Atomix.config(Atomix.java:314)
    at io.atomix.core.Atomix.config(Atomix.java:279)
    at io.atomix.core.Atomix.config(Atomix.java:265)
    at io.atomix.agent.AtomixAgent.createConfig(AtomixAgent.java:279)
    at io.atomix.agent.AtomixAgent.buildAtomix(AtomixAgent.java:328)
    at io.atomix.agent.AtomixAgent.main(AtomixAgent.java:60)
scondon@z420:~/onos$ docker run -v /home/scondon/onos:/mnt/onos atomix/atomix:latest -c /mnt/onos/atomix_sean.conf
Exception in thread "main" io.atomix.utils.config.ConfigurationException: Unknown properties present in configuration: cluster.name
    at io.atomix.core.utils.config.PolymorphicConfigMapper.checkRemainingProperties(PolymorphicConfigMapper.java:100)
    at io.atomix.utils.config.ConfigMapper.map(ConfigMapper.java:184)
    at io.atomix.utils.config.ConfigMapper.getValue(ConfigMapper.java:342)
    at io.atomix.utils.config.ConfigMapper.mapSetters(ConfigMapper.java:221)
    at io.atomix.utils.config.ConfigMapper.map(ConfigMapper.java:179)
    at io.atomix.utils.config.ConfigMapper.map(ConfigMapper.java:142)
    at io.atomix.utils.config.ConfigMapper.loadFiles(ConfigMapper.java:97)
    at io.atomix.core.Atomix.config(Atomix.java:320)
    at io.atomix.core.Atomix.config(Atomix.java:284)
    at io.atomix.core.Atomix.config(Atomix.java:270)
    at io.atomix.agent.AtomixAgent.createConfig(AtomixAgent.java:276)
    at io.atomix.agent.AtomixAgent.buildAtomix(AtomixAgent.java:325)
    at io.atomix.agent.AtomixAgent.main(AtomixAgent.java:60)
scondon@z420:~/onos$

[Moheed Ahmad]

Seems, like there is lot of work yet to be done for this new architecture. And it will stablize going forward.

I see the same on docker-image atomix:latest. I am answering few of your question theoretically, haven't checked all these by myself or details of current implementation in owl.

>>>>Are the Atomix cluster nodes supposed to be another set of machines separate to the ONOS cluster nodes?

Not necessarily. 

Actually, it should work even with m:n mapping of atomix and onos cluster. (eg 5 node atomix cluster and 3 node onos-cluster).

>>>>If the Atomix cluster nodes can be co-located with the ONOS nodes (machines/VMs), do I have to have the Atomix cluster software up and running on these nodes separate to ONOS?

Atomix cluster has to be up prior to onos-cluster.

>>>>>Do the ONOS cluster nodes have to be up and running before the Atomix Cluster is formed?

No

>>>>>What is the minimum configuration I can have to run an ONOS cluster?

Download onos-owl release and if you do 'onos-install <ips>' it should automatically form both atomix and onos clusters. 

Ideally, as a end-user, one should not be bothered about the cluster-configuration details. IMHO, what is missing is a 'simple-onos-user-config' file in releases with new-architecture.

All user need to do is mention bare-minimum, something like below:

atomix-node1: <ip>

atomix-node2:<ip>

:

:

atomix-node<n>: <ip>

onos-node1: <ip>

onos-node2: <ip>

:

:

onos-noden: <ip>

Given above, 'onos-install' should be able to validate and form appropriate clusters. (of course with other properties such as port etc  should be assumed default)

Cheers,

Moheed

[Jordan Halterman]

There’s an issue with the current Atomix Docker image that requires that you specify --ignore-resources as arguments when running the container. This is because the distribution packaged in the image is picking up a default configuration file even when an external configuration is provided. It’s already fixed in the snapshot, and I’m working on egging a new release out this week.

Aside from that, I’m just going to carefully update the wiki page with step-by-step instructions for configuring and running the distribution and the Docker image with a basic configuration as some things have changed since it was written. 

--
You received this message because you are subscribed to the Google Groups "ONOS Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to onos-dev+u...@onosproject.org.
To post to this group, send email to onos...@onosproject.org.
Visit this group at https://groups.google.com/a/onosproject.org/group/onos-dev/.
To view this discussion on the web visit https://groups.google.com/a/onosproject.org/d/msgid/onos-dev/2b0d20be-6c5a-4239-a60e-e8af7a3bdcfc%40onosproject.org.

[scon...@gmail.com]

Thanks for your responses guys.

I realize it's early days yet for this new external Atomix cluster, and it will take some time to get the docs in to shape. I think what's needed is a step-by-step tutorial of the basics - even a screen capture video would be a help

Thanks again, Sean

[Jordan Halterman]

Sean, try the 1.14 documentation again:

https://wiki.onosproject.org/pages/viewpage.action?pageId=28836788

It's certainly an improvement, but it doesn't quite feel procedural enough to me. I think it's a little challenging because there are a lot of concepts that warrant an explanation. Maybe we need a couple of documents in that case - step-by-step bare metal & Docker guides and a more thorough documentation of relevant configuration options.

Thoughts?

--
You received this message because you are subscribed to the Google Groups "ONOS Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to onos-dev+u...@onosproject.org.
To post to this group, send email to onos...@onosproject.org.
Visit this group at https://groups.google.com/a/onosproject.org/group/onos-dev/.

To view this discussion on the web visit https://groups.google.com/a/onosproject.org/d/msgid/onos-dev/5c1edf91-a42e-4138-962b-f38e17b68b29%40onosproject.org.

--

Jordan Halterman

Member of Technical Staff at ONF
Member of ONOS Technical Steering Team

[scon...@gmail.com]

Jordan - thanks for the update to the Wiki - yes it helps a lot.

I formed an Atomix cluster using Docker as soon as I figured out that I was meant to run the Docker instance for each node of the Atomix cluster, each with its own atomix.conf. Also I found I had to form my own Docker network to be able to prescribe IP addresses to the Atomix node Docker instances.

There is one more thing that I cannot find an answer to - how do I apply the cluster.json to my ONOS instance to tell it where the Atomix Cluster is? I thought this might have been through NetCfg REST interface on ONOS, but I am only getting an error back? Does it need to be placed somewhere on the file system of the ONOS deployment?

[Jordan Halterman]

We used to allow the cluster to be configured via the REST API, however, that was disabled since the behavior of the API changed significantly with the architecture change. It also proved to be pretty buggy since the Karaf container had to be rebooted after the configuration change. So, the only way to configure the cluster currently is to add cluster.json under the /config directory in the ONOS distro. I forgot to add this to the wiki and will update it.

--

You received this message because you are subscribed to the Google Groups "ONOS Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to onos-dev+u...@onosproject.org.
To post to this group, send email to onos...@onosproject.org.
Visit this group at https://groups.google.com/a/onosproject.org/group/onos-dev/.

To view this discussion on the web visit https://groups.google.com/a/onosproject.org/d/msgid/onos-dev/dba245ec-6df9-4d19-ac52-8ce1e3fa1eb4%40onosproject.org.

[Sean Condon]

I also found Thomas' tutorial file at onos/tools/tutorial/vm/createCluster in the code base which is a neat way of setting up a 3:3 cluster with Docker. This showed the copying of the file in to the config location.

Thanks again for all your help and great work, Sean

--

Sean Condon
scon...@gmail.com
087-7932229

[Jordan Halterman]

Ahh yes, that’s a good place to point to! I’ll have to add it to the wiki as an example. Thanks.

[Tasbiha Fatima]

Hi

Can you please tell me how to run these files? I mean proper commands. 

Thanks

On Thursday, November 8, 2018 at 2:05:34 AM UTC+5, Sean Condon wrote:

I also found Thomas' tutorial file at onos/tools/tutorial/vm/createCluster in the code base which is a neat way of setting up a 3:3 cluster with Docker. This showed the copying of the file in to the config location.

Thanks again for all your help and great work, Sean

On Wed, 7 Nov 2018 at 20:43, Jordan Halterman <jor...@opennetworking.org> wrote:

We used to allow the cluster to be configured via the REST API, however, that was disabled since the behavior of the API changed significantly with the architecture change. It also proved to be pretty buggy since the Karaf container had to be rebooted after the configuration change. So, the only way to configure the cluster currently is to add cluster.json under the /config directory in the ONOS distro. I forgot to add this to the wiki and will update it.

On Wed, Nov 7, 2018 at 3:58 AM <scon...@gmail.com> wrote:

Jordan - thanks for the update to the Wiki - yes it helps a lot.

I formed an Atomix cluster using Docker as soon as I figured out that I was meant to run the Docker instance for each node of the Atomix cluster, each with its own atomix.conf. Also I found I had to form my own Docker network to be able to prescribe IP addresses to the Atomix node Docker instances.

There is one more thing that I cannot find an answer to - how do I apply the cluster.json to my ONOS instance to tell it where the Atomix Cluster is? I thought this might have been through NetCfg REST interface on ONOS, but I am only getting an error back? Does it need to be placed somewhere on the file system of the ONOS deployment?

--
You received this message because you are subscribed to the Google Groups "ONOS Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos...@onosproject.org.

To post to this group, send email to onos...@onosproject.org.
Visit this group at https://groups.google.com/a/onosproject.org/group/onos-dev/.
To view this discussion on the web visit https://groups.google.com/a/onosproject.org/d/msgid/onos-dev/dba245ec-6df9-4d19-ac52-8ce1e3fa1eb4%40onosproject.org.

--

Jordan Halterman

Member of Technical Staff at ONF
Member of ONOS Technical Steering Team

[Tasbiha Fatima]

ubuntu@ubuntu-virtual-machine:~/onos/tools/tutorials/vm$ sudo ./destroyCluster
[sudo] password for ubuntu:
Destroying onos-1...
onos-1
Destroying onos-2...
onos-2
Destroying onos-3...
onos-3
Destroying atomix-1...
atomix-1
Destroying atomix-2...
atomix-2
Destroying atomix-3...
atomix-3
Deleted Containers:
9e3ef6446b922c26d4ddb1765b889341c0021265d498cff1d11c8308b727ccab
3b6353d18bf68a5958e3525df7d3be81db1283ada74991a130b43e11ad901887
4430863d3c2d74897eb3b750ca733c993a58bd7b86950bbf03f7dd378f17827a
edfca94a33df01c98ec65ac2df2752f0fc9e7fd28baf7a5647026fd200da5e71
78ae7defde6201b3025695bb94783e13bae9358437b96bb2e09fe6dcdf401d74
9c13b3a91db0ea78a25f9458fbc3bd91d192be873870489661e88da904fd212c

Total reclaimed space: 208.9MB
ubuntu@ubuntu-virtual-machine:~/onos/tools/tutorials/vm$ sudo ./createCluster
Setting up atomix-1...
037d8219b23f81471f7ac03fc09ec0f4c8dae7704a27c7f80da3875e5c1adf18
Setting up atomix-2...
8bfdf59e8ff473c824ce8637944b34115b79273f4fe44b8b02b895ec758f3dfc
Setting up atomix-3...
c52a923677b708e6ab2571b7dffba5209e652054d42c454bc3510c7439ff6566
Setting up onos-1...
84bb2b544f26ce7fb2320dabf4efc388286437c63d5bf413a9304a85ea903b69
Setting up onos-2...
13343eadc798c0cba4d0e3c5a58914d5b9ff0a74fc7fa9242e16b8b0afbd7f47
Setting up onos-3...
e15d117a0049f6a487e7d7e6a83cfe1cb393d7a8c9f38aa81419a1b97df6a294
Waiting for onos-1 startup...
Waiting for onos-2 startup...
Waiting for onos-3 startup...
Activating OpenFlow and ProxyARP applications...
./createCluster: line 47: onos: command not found

./createCluster: line 48: onos: command not found

This error is continiously shown please help.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-dev+u...@onosproject.org.
To view this discussion on the web visit https://groups.google.com/a/onosproject.org/d/msgid/onos-dev/1ecdb103-aff0-4c86-a19a-05cae6597f15%40onosproject.org.

[Tasbiha Fatima]

Hi 

When I followed the link below, my cluster was shown on GUI but now when I restarted my machine, it is continuously in the state as shown in screenshot. Do I need to do all the steps repeatedly every time for cluster? Kindly help 

https://wiki.onosproject.org/display/ONOS/Notes+on+cluster+formation+for+Docker+instances  

 

[qcorba]

By default docker containers are not restarted automatically. Please refer to docker docs if you want to set restart policy otherwise.

On the other hand, containers can be restarted using “docker start ...”. Please refer to docker docs for parameters required.

To view this discussion on the web visit https://groups.google.com/a/onosproject.org/d/msgid/onos-dev/CAC2X%2BZkUrNPJ%2BOQds4Vk4k3QY-TR_2-YYez0p%2BgEXCdXeiRQFg%40mail.gmail.com.

[Tasbiha Fatima]

Ok, thanks. 

[lye3...@gmail.com]

Hi,

When I follow the link below, and login ONOS Web UI, but the UI interface is the same as your screenshot. I do not know why it has to be like this, can you help me? Thank you.

在 2020年5月4日星期一 UTC+8下午4:45:22，Tasbiha Fatima写道：

To unsubscribe from this group and stop receiving emails from it, send an email to onos...@onosproject.org.