OpenStack Neat on OpenStack Juno -> Ubuntu 14.04 - Authentication Error

[Luigi Sambolino]

Hi Anton and OpenStack community.

I am trying to make OpenStack Neat work on a Openstack Juno - Ubuntu 14.04
Environment.
First I tried to search all the infos related to the Ubuntu integration found in this group (like here), but I got stucked somewhere still.

1. I installed all the dependencies (no problems so far).
2. I edited all the openstack-* files present in /etc/init.d/  replacing "/etc/rc.d/init.d/functions" with "/lib/lsb/init-functions" and "exec="/usr/bin/neat-$suffix" with exec="/usr/local/bin/neat-$suffix" as suggested here .

When I try to launch ./all-start.sh from the controller I encounter no error so far.

Still on the controller, I try to execute ./vm-placement but there I got stuck:

# ./vm-placement.py
Traceback (most recent call last):
  File "./vm-placement.py", line 51, in <module>
    hosts_to_vms = manager.vms_by_hosts(nova, hosts)
  File "<string>", line 2, in vms_by_hosts
  File "/usr/local/lib/python2.7/dist-packages/contracts/main.py", line 280, in contracts_checker
    result = function_(*args, **kwargs)
  File "/home/giallo/openstack-neat/neat/globals/manager.py", line 724, in vms_by_hosts
    for vm in nova.servers.list():
  File "/usr/local/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 600, in list
    return self._list("/servers%s%s" % (detail, query_string), "servers")
  File "/usr/local/lib/python2.7/dist-packages/novaclient/base.py", line 64, in _list
    _resp, body = self.api.client.get(url)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 451, in get
    return self._cs_request(url, 'GET', **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 407, in _cs_request
    self.authenticate()
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 560, in authenticate
    auth_url = self._v2_auth(auth_url)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 651, in _v2_auth
    return self._authenticate(url, body)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 664, in _authenticate
    **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 402, in _time_request
    resp, body = self.request(url, method, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 396, in request
    raise exceptions.from_response(resp, body, url, method)
novaclient.exceptions.Unauthorized: The request you have made requires authentication. (HTTP 401)

And "neat.conf" contains:

# Copyright 2012 Anton Beloglazov
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# This is the default configuration file for OpenStack Neat

[DEFAULT]

# The directory, where log files will be created by the Neat services
log_directory = /var/log/neat

# The level of emitted log messages:
# 0 -- no logging
# 1 -- errors and warnings
# 2 -- errors, warnings, and information messages
# 3 -- errors, warnings, information messages, and debug messages
log_level = 3

# The directory, where the VM instance data are stored
vm_instance_directory = /var/lib/nova/instances

# The host name and credentials for connecting to the MySQL database
# specified in the format supported by SQLAlchemy
sql_connection = mysql://neat:neatpassword@controller/neat

# The admin tenant name for authentication with Nova using Keystone
os_admin_tenant_name = admin

# The admin user name for authentication with Nova using Keystone
os_admin_user = neat

# The admin password for authentication with Nova using Keystone
os_admin_password = neatpassword

# The OpenStack authentication URL
os_auth_url = http://controller:5000/v2.0/

# A coma-separated list of compute host names
compute_hosts = compute1, amareno

# The name of the host running the global manager
global_manager_host = controller

# The port of the REST web service exposed by the global manager
global_manager_port = 60080

# The time interval between subsequent invocations of the database
# cleaner in seconds
db_cleaner_interval = 7200

# The directory used by the data collector to store the data on the
# resource usage by the VMs running on the host
local_data_directory = /var/lib/neat

# The time interval between subsequent invocations of the local
# manager in seconds
local_manager_interval = 300

# The time interval between subsequent invocations of the data
# collector in seconds
data_collector_interval = 300

# The number of the latest data values stored locally by the data
# collector and passed to the underload / overload detection and VM
# placement algorithms
data_collector_data_length = 100

# The threshold on the overall (all cores) utilization of the physical
# CPU of a host, above which the host is considered to be overloaded.
# This is used for logging host overloads into the database.
host_cpu_overload_threshold = 0.8

# The threshold on the overall (all cores) utilization of the physical
# CPU of a host that can be allocated to VMs.
host_cpu_usable_by_vms = 1.0

# The user name for connecting to the compute hosts to switch them
# into the sleep mode
compute_user = neat

# The password of the user account used for connecting to the compute
# hosts to switch them into the sleep mode
compute_password = neatpassword

# Whether to use block migration (includes disk migration)
block_migration = False

# The network bandwidth in MB/s available for VM migration
network_migration_bandwidth = 10

# A shell command used to switch a host into the sleep mode, the
# compute_user must have permissions to execute this command
sleep_command = pm-suspend

# The network interface to send a magic packet from using ether-wake
ether_wake_interface = eth0

# The fully qualified name of a Python factory function that returns a
# function implementing an underload detection algorithm
#algorithm_underload_detection_factory =
neat.locals.underload.trivial.threshold_factory
algorithm_underload_detection_factory = neat.locals.underload.trivial.
last_n_average_threshold_factory

# A JSON encoded parameters, which will be parsed and passed to the
# specified underload detection algorithm factory
#algorithm_underload_detection_parameters = {"threshold": 0.3}
algorithm_underload_detection_parameters = {"threshold": 0.5, "n": 2}

# The fully qualified name of a Python factory function that returns a
# function implementing an overload detection algorithm
#algorithm_overload_detection_factory =
neat.locals.overload.trivial.threshold_factory
algorithm_overload_detection_factory = neat.locals.overload.mhod.core.
mhod_factory
#algorithm_overload_detection_factory =
neat.locals.overload.trivial.last_n_average_threshold_factory
#algorithm_overload_detection_factory =
neat.locals.overload.statistics.loess_factory
#algorithm_overload_detection_factory = neat.locals.overload.otf.otf_factory

# A JSON encoded parameters, which will be parsed and passed to the
# specified overload detection algorithm factory
#algorithm_overload_detection_parameters = {"threshold": 0.9}
algorithm_overload_detection_parameters = {"state_config": [0.8], "otf": 0.1
, "history_size": 500, "window_sizes": [30, 40, 50, 60, 70, 80, 90, 100],
"bruteforce_step": 0.5, "learning_steps": 10}
#algorithm_overload_detection_parameters = {"threshold": 0.95, "n": 2}
#algorithm_overload_detection_parameters = {"threshold": 0.8, "param": 1.0,
"length": 30}
#algorithm_overload_detection_parameters = {"otf": 0.2, "threshold": 0.8,
"limit": 10}

# The fully qualified name of a Python factory function that returns a
# function implementing a VM selection algorithm
#algorithm_vm_selection_factory =
neat.locals.vm_selection.algorithms.minimum_migration_time_factory
algorithm_vm_selection_factory = neat.locals.vm_selection.algorithms.
minimum_migration_time_max_cpu_factory

# A JSON encoded parameters, which will be parsed and passed to the
# specified VM selection algorithm factory
#algorithm_vm_selection_parameters = {}
algorithm_vm_selection_parameters = {"last_n": 2}

# The fully qualified name of a Python factory function that returns a
# function implementing a VM placement algorithm
algorithm_vm_placement_factory = neat.globals.vm_placement.bin_packing.
best_fit_decreasing_factory

# A JSON encoded parameters, which will be parsed and passed to the
# specified VM placement algorithm factory
algorithm_vm_placement_parameters = {"cpu_threshold": 0.8, "ram_threshold":
0.95, "last_n_vm_cpu": 2}

What am I missing ? Any help would be extremely welcomed.
Thank you in advance

Luigi

[Anton Beloglazov]

Hi Luigi,

Looks like the OpenStack client fails to authenticate with the controller. Are you certain that the following settings correctly represent your OpenStack configuration?

# The admin tenant name for authentication with Nova using Keystone
os_admin_tenant_name = admin

# The admin user name for authentication with Nova using Keystone
os_admin_user = neat

# The admin password for authentication with Nova using Keystone
os_admin_password = neatpassword

# The OpenStack authentication URL
os_auth_url = http://controller:5000/v2.0/

Cheers,

Anton

--
You received this message because you are subscribed to the Google Groups "OpenStack Neat" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openstack-nea...@googlegroups.com.
Visit this group at http://groups.google.com/group/openstack-neat.
For more options, visit https://groups.google.com/d/optout.

[Luigi Sambolino]

Hi Anton,

Thank you for your support. My current settings (default Openstack Juno on Ubuntu 14.04 logins) are:

admin_tenant_name: service
admin_user: nova
admin_password: NOVA_PASS

So you were right, and after editing /etc/neat/neat.conf with the above login the error disappeared.
Nevertheless, another error appeared:

giallo@controller:~/openstack-neat$ sudo ./vm-placement.py

Traceback (most recent call last):

  File "./vm-placement.py", line 65, in <module>
    hosts_cpu_usage[host] = hosts_cpu_usage_hypervisor[host] + \
KeyError: 'compute1'

What am I doing wrong? Hostname seems correct to me. Thank you in advance.
Kind Regards,

Luigi

...

[Anton Beloglazov]

Luigi,

Please make sure that you start the data collector services on the compute hosts first so that they can submit data about the hosts to the DB.

Cheers,

Anton

--

[Luigi Sambolino]

Hello Anton,

On my setup I have multiple compute nodes, names are

compute1

and

amareno

They are both debian-based and nova-compute is running on both of them.

I tried to follow the given instructions on every node:

1. Cloned GIT folder
2. Modified neat.conf file with correct compute hostnames
3. Modified /etc/init.d with the above mentioned settings.
4. python setup.py install
   
5. ./all-start.sh
   

Now, running ./all-start.sh from the first compute node it says:

root@compute1:/home/compute1/openstack-neat# ./all-start.sh
+ from neat.config import LICENSE MANIFEST.in NOTICE README.md TODO all-clean-logs.sh all-start.sh all-stop.sh all-sync-time.sh all-update.sh build build-rpm.sh compute-clean-logs.py compute-clone-neat.py compute-copy-conf.py compute-data-collector-start.py compute-data-collector-status.py compute-data-collector-stop.py compute-install-deps.py compute-install-neat.py compute-local-manager-start.py compute-local-manager-status.py compute-local-manager-stop.py compute-sync-time.py compute-update.py dist distribute_setup.py distribute_setup.pyc doc init.d install-rpm.sh neat neat.conf nosetests.sh openstack_neat.egg-info setup setup.py start-data-collector.py start-global-manager.py start-local-manager.py stats.sh tests utils vm-placement.py
from: can't read /var/mail/neat.config
+ import neat.common as common
./compute-data-collector-start.py: 19: ./compute-data-collector-start.py: import: not found
./compute-data-collector-start.py: 21: ./compute-data-collector-start.py: Syntax error: "(" unexpected
$Starting openstack-neat-global-manager: /etc/init.d/openstack-neat-global-manager: 45: /etc/init.d/openstack-neat-global-manager: daemon: not found

$Starting openstack-neat-db-cleaner: /etc/init.d/openstack-neat-db-cleaner: 42: /etc/init.d/openstack-neat-db-cleaner: daemon: not found

Host: compute1
root@compute1's password:
$ service openstack-neat-local-manager start
$Starting openstack-neat-local-manager:
/etc/init.d/openstack-neat-local-manager: 47: /etc/init.d/openstack-neat-local-manager: daemon: not found

Host: amareno
root@amareno's password:
$ service openstack-neat-local-manager start
Job for openstack-neat-local-manager.service failed. See 'systemctl status openstack-neat-local-manager.service' and 'journalctl -xn' for details.

Now, since my linux knowledge is not very deep, I suspect these errors are caused by not having the needed installed daemons in the setup phase, due to the different system architecture of debian/ubuntu in comparison with centos/redhat, am I wrong?
If so, what's missing ?
Thank you again in advance.

Luigi

...

[Anton Beloglazov]

Hi Luigi,

These scripts were made for CentOS, the init scripts won't work on Debian. Please try to run the following commands on every compute node:

python2 start-data-collector.py

python2 start-local-manager.py

And then the following on your controller:

python2 start-global-manager.py

Please let me know how that works.

Cheers,

Anton

--

[Luigi Sambolino]

Hi Anton,

Now everything seems to work so far (I see stats from vm-placement.py) but I have to fix something in my openstack setup before proceeding further.
I will update this topic asap to show if correct vm migration happen.

As a side note, I think that inserting a couple of lines related to Ubuntu/debian installation in the readme file of the project would be extremely useful to who start to play with Neat.

Regards,

Luigi

...

[Anton Beloglazov]

Hi Luigi,

I'm glad it's working! Updating the readme is a good idea. Could you please write the sequence of steps you had to do to make it run on Ubuntu? I'll then add them to the readme.

Thanks,

Anton

--

[Luigi Sambolino]

Hi Anton,

I am not 100% sure everything is needed and correct, I'd like a double check on these.
Anyway:

OpenStack Neat - Install instructions for Ubuntu 14.04 LTS with OpenStack Juno

1. Clone the neat repository on every compute and controller node.

2. Adjust the configuration by modifying neat.conf file in the repo directory on every node.
In particular, you have to specify the names of your compute nodes in the field compute_hosts. Then, fix the following three fields:
admin_tenant_name, admin_user and admin_password according to your OpenStack Install. If you have default values, correct settings are:

admin_tenant_name: service
admin_user: nova
admin_password: NOVA_PASS

If you don't know or you don't remember your login, copy the login present in /etc/nova/nova.conf in the same node.


3. Edit all the openstack-* files present in /etc/init.d/  replacing /etc/rc.d/init.d/functions with /lib/lsb/init-functions and exec="/usr/bin/neat-$suffix" with exec="/usr/local/bin/neat-$suffix" on every neat node.

. In the project root folder, execute

python setup.py install

4. Just to be sure, install all the dependencies with:

apt-get install python-pip numpy scipy libvirt-python
sudo pip install --upgrade pyqcy PyContracts SQLAlchemy bottle requests Sphinx python-novaclient
sudo pip install mocktest

(not 100% sure you can find all, search the debian equivalent if you cannot find.)
While doing this, DO NOT remove any python related package previously installed by OpenStack, it will break your OpenStack install!!

5. run the following commands on every compute node:

python2 start-data-collector.py
python2 start-local-manager.py

And then the following on your controller:

python2 start-global-manager.py

6. You can monitor the current VM placement using the

./vm-placement.py

script.

[Anton Beloglazov]

Hi Luigi,

It took me ages, but I've finally updated the readme based on your description, thanks!

Cheers,

Anton