Neat with OpenStack Juno (CentOS 7.0)

[Raphael Ribeiro]

Hello Anton, I'm Computer Science Graduate Student and I want to work with neat in my research. So it is possible integrate Neat with OpenStack Juno?

I tried to install neat with 1 Controller and 2 Compute nodes running CentOS 7.0. I'm having problems starting the services, logs:

(controller node)

https://gist.github.com/raphapr/b7102a44c2388d496ea4

CentOS deps:

numpy (Version: 1.7.1)

scipy (Version: 0.12.1)
libvirt-python (Version : 1.1.1)
pyqcy (Version: 0.4.5)
PyContracts (Version: 1.7.1)
SQLAlchemy (Version: 0.9.8)
requests (Version: 2.5.3)
Sphinx (Version: 1.2.3)
python-novaclient (Version: 2.21.0)

mocktest (Version: 0.7)

Thanks!

[Anton Beloglazov]

Hi Raphael,

Thanks for your interest in the project! The following seems to be related to your problem: http://www.gossamer-threads.com/lists/openstack/dev/40429

Can you try to increase "max_connections" in your /etc/mysql/my.cnf file, and restart the mysql service?

Best regards,

Anton

--
You received this message because you are subscribed to the Google Groups "OpenStack Neat" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openstack-nea...@googlegroups.com.
Visit this group at http://groups.google.com/group/openstack-neat.
For more options, visit https://groups.google.com/d/optout.

[Wildanil Ghozi]

Hi Anton, previously I tried to use OpenStack neat on ubuntu, but I eventually gave up. now I use CentOS 7. we all know that CentOS 7 using systemd and no longer use init.d as in centos6. Is OpenStack neat can run well in centos 7? or any part of the script that should be changed?

Thanks..

[Anton Beloglazov]

Hi Wildanil,

The service configuration should definitely be updated to support systemd, but I haven't looked into how to configure that. Actually, if you just want to start the services, you can just run the start-*.py from the project root -- these scripts should start them up.

Cheers,

Anton

[Wildanil Ghozi]

Hi Anton,

Thanks for your explaination. Now I feel relived. My previous, I've failde to install openstack neat on ubuntu, I hope I'll success on CentOS 7.

cheers,
Wildanil

[Wildanil Ghozi]

Hi Anton,
I've tried to install openstack neat on CentOS 7. Openstack neat has been running without error log for one day. But I think there is something wrong. My machine is compute1 and compute2, compute1 run 2 VM and compute2 run 1 VM. VM should be consolidated into one machine, but it did not happen. I use a default algorithm that you have defined in neat.conf.

There are log report
global manager on controller 
2015-08-16 22:05:14,242 INFO     neat.globals.manager No VMs to migrate - completed the underload request
2015-08-16 22:06:56,878 DEBUG    neat.globals.manager Request parameters validated
2015-08-16 22:06:56,878 INFO     neat.globals.manager Received a request from 10.0.0.32: {'username': 'd033e22ae348aeb5660fc2140aec35850c4da997', 'host': 'compute2', 'password': 'b47208670e6be587a615a70d33dddbed6ca0cbf0', 'reason': 0, 'time': 1439737616.87}
2015-08-16 22:06:56,878 INFO     neat.globals.manager Processing an underload of a host compute2
2015-08-16 22:06:56,879 INFO     neat.globals.manager Started processing an underload request
2015-08-16 22:06:56,886 INFO     requests.packages.urllib3.connectionpool Starting new HTTP connection (1): controller
2015-08-16 22:06:56,983 DEBUG    requests.packages.urllib3.connectionpool "GET /v2/604dec3bd45748c4a7d022bce9307938/servers/detail HTTP/1.1" 200 15
2015-08-16 22:06:57,006 DEBUG    neat.globals.manager hosts_to_vms: {'compute1': [], 'compute2': []}
2015-08-16 22:06:57,006 DEBUG    neat.globals.manager Host CPU usage: {'compute1': 0, 'controller': 0, 'compute2': 0}
2015-08-16 22:06:57,006 DEBUG    neat.globals.manager Host total CPU usage: {}
2015-08-16 22:06:57,007 DEBUG    neat.globals.manager Excluded the underloaded host compute2
2015-08-16 22:06:57,007 DEBUG    neat.globals.manager Host CPU usage: {'compute1': 0, 'controller': 0, 'compute2': 0}
2015-08-16 22:06:57,007 DEBUG    neat.globals.manager Host total CPU usage: {}
2015-08-16 22:06:57,010 INFO     requests.packages.urllib3.connectionpool Starting new HTTP connection (1): controller
2015-08-16 22:06:57,094 DEBUG    requests.packages.urllib3.connectionpool "GET /v2/604dec3bd45748c4a7d022bce9307938/servers/detail HTTP/1.1" 200 15
2015-08-16 22:06:57,099 INFO     requests.packages.urllib3.connectionpool Starting new HTTP connection (1): controller
2015-08-16 22:06:57,150 DEBUG    requests.packages.urllib3.connectionpool "GET /v2/604dec3bd45748c4a7d022bce9307938/flavors/detail HTTP/1.1" 200 2069
2015-08-16 22:06:57,154 INFO     neat.globals.manager No VMs to migrate - completed the underload request

local manager on compute1
2015-08-16 22:02:29,469 INFO     neat.locals.manager Started an iteration
2015-08-16 22:02:29,478 DEBUG    neat.locals.manager The total physical CPU Mhz: 23408
2015-08-16 22:02:29,478 DEBUG    neat.locals.manager VM CPU MHz: {'b27a42a5-177d-4b86-bc27-99a30de66ba2': [0, 5, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1], '91b8f307-a93e-45c8-8045-f722f707613d': [74, 80, 75, 74, 80, 71, 74, 80, 75, 74, 80, 76, 76, 79, 75, 77, 79, 75, 76, 79, 76, 76, 77, 75, 77, 77, 75, 77, 77, 75, 77, 77, 72, 79, 76, 74, 79, 78, 76, 78, 77, 74, 79, 73, 76, 78, 75, 77, 79, 77, 75, 79, 75, 76, 80, 76, 76, 79, 75, 72, 78, 75, 76, 79, 75, 73, 79, 74, 76, 79, 76, 76, 80, 73, 75, 80, 76, 75, 80, 75, 76, 79, 75, 77, 80, 76, 74, 80, 75, 77, 79, 76, 75, 77, 76, 76, 80, 75, 78, 79]}
2015-08-16 22:02:29,478 DEBUG    neat.locals.manager Host CPU MHz: [867, 107, 846, 865, 92, 796, 897, 108, 900, 851, 117, 971, 739, 268, 839, 711, 270, 899, 648, 393, 885, 538, 411, 923, 478, 461, 899, 458, 438, 967, 413, 533, 866, 421, 547, 900, 374, 626, 899, 289, 671, 900, 245, 669, 863, 292, 613, 900, 300, 780, 839, 206, 773, 889, 159, 754, 843, 218, 745, 845, 224, 747, 919, 151, 834, 837, 148, 838, 858, 113, 825, 906, 86, 817, 934, 97, 880, 838, 113, 932, 774, 232, 823, 764, 191, 855, 768, 205, 928, 684, 265, 857, 688, 266, 848, 696, 279, 932, 606, 464]
2015-08-16 22:02:29,479 DEBUG    neat.locals.manager CPU utilization: [0.04019993164730007, 0.008202323991797676, 0.039345522898154475, 0.04011449077238551, 0.007390635680109364, 0.03703861927546138, 0.04148154477101845, 0.008074162679425838, 0.04165242652084757, 0.039516404647983595, 0.008415926179084075, 0.0447282980177717, 0.034817156527682844, 0.014823991797676008, 0.03904647983595352, 0.0336637047163363, 0.014909432672590567, 0.04160970608339029, 0.030929596719070404, 0.020206766917293232, 0.04105434039644566, 0.02627306903622693, 0.020890293916609706, 0.042634996582365005, 0.023709842788790157, 0.022983595352016405, 0.04160970608339029, 0.022855434039644566, 0.022001025290498976, 0.044514695830485305, 0.020933014354066987, 0.026102187286397814, 0.04007177033492823, 0.02136021872863978, 0.026614832535885168, 0.04160970608339029, 0.01935235816814764, 0.03007518796992481, 0.04165242652084757, 0.0156784005468216, 0.03195488721804511, 0.04160970608339029, 0.013841421736158578, 0.031741285030758716, 0.04011449077238551, 0.01580656185919344, 0.02939166097060834, 0.04173786739576213, 0.016191045796308954, 0.03661141490088859, 0.03904647983595352, 0.012175324675324676, 0.036226930963773066, 0.041225222146274776, 0.010210184552289815, 0.03550068352699932, 0.03926008202323992, 0.01268796992481203, 0.03503075871496924, 0.03917464114832536, 0.012901572112098428, 0.0351161995898838, 0.042506835269993166, 0.0098257006151743, 0.03883287764866712, 0.038875598086124404, 0.00974025974025974, 0.03900375939849624, 0.03990088858509911, 0.008202323991797676, 0.03849111414900889, 0.04195146958304853, 0.007134313055365687, 0.03802118933697881, 0.043104921394395077, 0.007604237867395762, 0.04084073820915926, 0.03900375939849624, 0.008245044429254956, 0.0430622009569378, 0.03631237183868763, 0.013286056049213945, 0.03836295283663705, 0.035927887901572114, 0.01161995898838004, 0.03977272727272727, 0.03597060833902939, 0.012218045112781954, 0.0428485987696514, 0.032510252904989746, 0.01469583048530417, 0.03985816814764183, 0.0325956937799043, 0.01465311004784689, 0.039473684210526314, 0.032980177717019825, 0.015336637047163364, 0.04301948051948052, 0.02922077922077922, 0.02323991797676008]
2015-08-16 22:02:29,479 INFO     neat.locals.manager Started underload detection
2015-08-16 22:02:29,480 INFO     neat.locals.manager Completed underload detection
2015-08-16 22:02:29,480 INFO     neat.locals.manager Started overload detection
2015-08-16 22:05:13,844 DEBUG    neat.locals.overload.mhod.core MHOD utilization:[0.04019993164730007, 0.008202323991797676, 0.039345522898154475, 0.04011449077238551, 0.007390635680109364, 0.03703861927546138, 0.04148154477101845, 0.008074162679425838, 0.04165242652084757, 0.039516404647983595, 0.008415926179084075, 0.0447282980177717, 0.034817156527682844, 0.014823991797676008, 0.03904647983595352, 0.0336637047163363, 0.014909432672590567, 0.04160970608339029, 0.030929596719070404, 0.020206766917293232, 0.04105434039644566, 0.02627306903622693, 0.020890293916609706, 0.042634996582365005, 0.023709842788790157, 0.022983595352016405, 0.04160970608339029, 0.022855434039644566, 0.022001025290498976, 0.044514695830485305, 0.020933014354066987, 0.026102187286397814, 0.04007177033492823, 0.02136021872863978, 0.026614832535885168, 0.04160970608339029, 0.01935235816814764, 0.03007518796992481, 0.04165242652084757, 0.0156784005468216, 0.03195488721804511, 0.04160970608339029, 0.013841421736158578, 0.031741285030758716, 0.04011449077238551, 0.01580656185919344, 0.02939166097060834, 0.04173786739576213, 0.016191045796308954, 0.03661141490088859, 0.03904647983595352, 0.012175324675324676, 0.036226930963773066, 0.041225222146274776, 0.010210184552289815, 0.03550068352699932, 0.03926008202323992, 0.01268796992481203, 0.03503075871496924, 0.03917464114832536, 0.012901572112098428, 0.0351161995898838, 0.042506835269993166, 0.0098257006151743, 0.03883287764866712, 0.038875598086124404, 0.00974025974025974, 0.03900375939849624, 0.03990088858509911, 0.008202323991797676, 0.03849111414900889, 0.04195146958304853, 0.007134313055365687, 0.03802118933697881, 0.043104921394395077, 0.007604237867395762, 0.04084073820915926, 0.03900375939849624, 0.008245044429254956, 0.0430622009569378, 0.03631237183868763, 0.013286056049213945, 0.03836295283663705, 0.035927887901572114, 0.01161995898838004, 0.03977272727272727, 0.03597060833902939, 0.012218045112781954, 0.0428485987696514, 0.032510252904989746, 0.01469583048530417, 0.03985816814764183, 0.0325956937799043, 0.01465311004784689, 0.039473684210526314, 0.032980177717019825, 0.015336637047163364, 0.04301948051948052, 0.02922077922077922, 0.02323991797676008]
2015-08-16 22:05:13,844 DEBUG    neat.locals.overload.mhod.core MHOD time_in_states:219
2015-08-16 22:05:13,844 DEBUG    neat.locals.overload.mhod.core MHOD time_in_state_n:0
2015-08-16 22:05:13,844 DEBUG    neat.locals.overload.mhod.core MHOD p:[[1.0, 0.0], [0.0, 0.0]]
2015-08-16 22:05:13,844 DEBUG    neat.locals.overload.mhod.core MHOD current_state:0
2015-08-16 22:05:13,844 DEBUG    neat.locals.overload.mhod.core MHOD p[current_state]:[1.0, 0.0]
2015-08-16 22:05:13,845 INFO     neat.locals.manager Completed overload detection
2015-08-16 22:05:13,845 INFO     neat.locals.manager Underload detected
2015-08-16 22:05:13,847 INFO     requests.packages.urllib3.connectionpool Starting new HTTP connection (1): controller
2015-08-16 22:05:14,243 DEBUG    requests.packages.urllib3.connectionpool "PUT / HTTP/1.1" 200 0
2015-08-16 22:05:14,244 INFO     neat.locals.manager Received response: [200]
2015-08-16 22:05:14,244 INFO     neat.locals.manager Completed an iteration

local-manager on compute2
2015-08-16 21:59:37,041 INFO     neat.locals.manager Completed an iteration
2015-08-16 22:04:37,141 INFO     neat.locals.manager Started an iteration
2015-08-16 22:04:37,148 DEBUG    neat.locals.manager The total physical CPU Mhz: 12768
2015-08-16 22:04:37,148 DEBUG    neat.locals.manager VM CPU MHz: {'27e3553c-f467-48d2-ba29-3ba7dd2daeea': [1, 1, 1, 2, 1, 1, 0, 51, 316, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 7, 11, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1]}
2015-08-16 22:04:37,148 DEBUG    neat.locals.manager Host CPU MHz: [806, 227, 652, 779, 398, 596, 803, 438, 453, 891, 481, 433, 732, 656, 340, 833, 661, 234, 768, 770, 122, 737, 815, 277, 600, 814, 413, 544, 778, 510, 329, 858, 638, 282, 839, 710, 207, 766, 783, 142, 770, 914, 151, 818, 797, 205, 630, 754, 437, 456, 872, 500, 403, 896, 532, 357, 766, 705, 176, 962, 693, 189, 865, 774, 116, 782, 775, 234, 604, 887, 333, 520, 779, 529, 335, 821, 671, 218, 783, 759, 147, 750, 797, 278, 629, 911, 287, 576, 858, 389, 584, 803, 438, 490, 801, 537, 353, 801, 677, 201]
2015-08-16 22:04:37,148 DEBUG    neat.locals.manager CPU utilization: [0.06320488721804511, 0.017857142857142856, 0.051143483709273185, 0.06116854636591479, 0.03125, 0.046757518796992484, 0.06289160401002507, 0.03829887218045113, 0.06022869674185464, 0.06986215538847118, 0.03775062656641604, 0.03399122807017544, 0.05733082706766917, 0.051456766917293235, 0.026707393483709273, 0.06524122807017543, 0.051848370927318296, 0.018405388471177946, 0.06015037593984962, 0.06038533834586466, 0.00963345864661654, 0.05772243107769424, 0.06383145363408521, 0.021773182957393483, 0.047070802005012534, 0.06383145363408521, 0.03242481203007519, 0.04268483709273183, 0.06101190476190476, 0.0400219298245614, 0.025845864661654134, 0.06719924812030076, 0.050046992481203006, 0.022164786967418547, 0.06578947368421052, 0.05568609022556391, 0.016290726817042606, 0.06007205513784461, 0.06140350877192982, 0.011199874686716791, 0.06038533834586466, 0.07166353383458647, 0.011904761904761904, 0.06414473684210527, 0.06242167919799499, 0.01613408521303258, 0.049342105263157895, 0.05905388471177945, 0.03430451127819549, 0.03579260651629073, 0.06837406015037593, 0.03923872180451128, 0.03164160401002506, 0.07025375939849623, 0.04174498746867168, 0.028038847117794485, 0.0599937343358396, 0.05521616541353384, 0.013862781954887217, 0.07534461152882206, 0.05482456140350877, 0.015664160401002505, 0.06782581453634086, 0.0606203007518797, 0.009163533834586466, 0.0612468671679198, 0.06069862155388471, 0.018405388471177946, 0.047384085213032584, 0.06954887218045112, 0.026159147869674184, 0.04080513784461153, 0.06101190476190476, 0.04151002506265664, 0.02631578947368421, 0.06430137844611529, 0.05263157894736842, 0.017152255639097745, 0.06132518796992481, 0.05952380952380952, 0.011591478696741854, 0.05881892230576441, 0.06242167919799499, 0.021851503759398497, 0.049342105263157895, 0.07142857142857142, 0.022556390977443608, 0.04519110275689223, 0.06727756892230577, 0.030545112781954886, 0.045817669172932333, 0.06289160401002507, 0.0343828320802005, 0.03845551378446115, 0.06273496240601503, 0.04213659147869674, 0.027725563909774435, 0.06281328320802006, 0.0531015037593985, 0.01582080200501253]
2015-08-16 22:04:37,148 INFO     neat.locals.manager Started underload detection
2015-08-16 22:04:37,150 INFO     neat.locals.manager Completed underload detection
2015-08-16 22:04:37,150 INFO     neat.locals.manager Started overload detection
2015-08-16 22:06:56,869 DEBUG    neat.locals.overload.mhod.core MHOD utilization:[0.06320488721804511, 0.017857142857142856, 0.051143483709273185, 0.06116854636591479, 0.03125, 0.046757518796992484, 0.06289160401002507, 0.03829887218045113, 0.06022869674185464, 0.06986215538847118, 0.03775062656641604, 0.03399122807017544, 0.05733082706766917, 0.051456766917293235, 0.026707393483709273, 0.06524122807017543, 0.051848370927318296, 0.018405388471177946, 0.06015037593984962, 0.06038533834586466, 0.00963345864661654, 0.05772243107769424, 0.06383145363408521, 0.021773182957393483, 0.047070802005012534, 0.06383145363408521, 0.03242481203007519, 0.04268483709273183, 0.06101190476190476, 0.0400219298245614, 0.025845864661654134, 0.06719924812030076, 0.050046992481203006, 0.022164786967418547, 0.06578947368421052, 0.05568609022556391, 0.016290726817042606, 0.06007205513784461, 0.06140350877192982, 0.011199874686716791, 0.06038533834586466, 0.07166353383458647, 0.011904761904761904, 0.06414473684210527, 0.06242167919799499, 0.01613408521303258, 0.049342105263157895, 0.05905388471177945, 0.03430451127819549, 0.03579260651629073, 0.06837406015037593, 0.03923872180451128, 0.03164160401002506, 0.07025375939849623, 0.04174498746867168, 0.028038847117794485, 0.0599937343358396, 0.05521616541353384, 0.013862781954887217, 0.07534461152882206, 0.05482456140350877, 0.015664160401002505, 0.06782581453634086, 0.0606203007518797, 0.009163533834586466, 0.0612468671679198, 0.06069862155388471, 0.018405388471177946, 0.047384085213032584, 0.06954887218045112, 0.026159147869674184, 0.04080513784461153, 0.06101190476190476, 0.04151002506265664, 0.02631578947368421, 0.06430137844611529, 0.05263157894736842, 0.017152255639097745, 0.06132518796992481, 0.05952380952380952, 0.011591478696741854, 0.05881892230576441, 0.06242167919799499, 0.021851503759398497, 0.049342105263157895, 0.07142857142857142, 0.022556390977443608, 0.04519110275689223, 0.06727756892230577, 0.030545112781954886, 0.045817669172932333, 0.06289160401002507, 0.0343828320802005, 0.03845551378446115, 0.06273496240601503, 0.04213659147869674, 0.027725563909774435, 0.06281328320802006, 0.0531015037593985, 0.01582080200501253]
2015-08-16 22:06:56,869 DEBUG    neat.locals.overload.mhod.core MHOD time_in_states:219
2015-08-16 22:06:56,870 DEBUG    neat.locals.overload.mhod.core MHOD time_in_state_n:0
2015-08-16 22:06:56,870 DEBUG    neat.locals.overload.mhod.core MHOD p:[[1.0, 0.0], [0.0, 0.0]]
2015-08-16 22:06:56,870 DEBUG    neat.locals.overload.mhod.core MHOD current_state:0
2015-08-16 22:06:56,870 DEBUG    neat.locals.overload.mhod.core MHOD p[current_state]:[1.0, 0.0]
2015-08-16 22:06:56,870 INFO     neat.locals.manager Completed overload detection
2015-08-16 22:06:56,870 INFO     neat.locals.manager Underload detected
2015-08-16 22:06:56,872 INFO     requests.packages.urllib3.connectionpool Starting new HTTP connection (1): controller
2015-08-16 22:06:57,153 DEBUG    requests.packages.urllib3.connectionpool "PUT / HTTP/1.1" 200 0
2015-08-16 22:06:57,154 INFO     neat.locals.manager Received response: [200]
2015-08-16 22:06:57,154 INFO     neat.locals.manager Completed an iteration

Local manager have detected underload condition and sent request to global manager. But global manager making dessicion "No VMs to migrate".

[Anton Beloglazov]

Hi Wildanil,

It looks like there is no data in the DB, this might happen if you haven't started the data collector service on the compute nodes. Could you please check that the data collector is running? Its log file should appear in the same directory.

Cheers,

Anton

[Wildanil Ghozi]

Hi Anton, I tried to generate workload for VM so that compute host state is overload. Local manager log reported overload detected and than started vm selection. But that is not continuation, it looks like local manager has stoped. I've checked that status using command systemctl status openstack-neat-local-manager.service and the results are shown above.
Please help me to understand and solve this problem.

Cheers,
Wildanil

[Anton Beloglazov]

Hi Wildanil,

It's hard to say what's happening from these logs. It seems was was some exception, but there is no stack trace. Did you check the earlier logs for exceptions? Did you have the data collector and global manager running as well?

Cheers,

Anton

[Wildanil Ghozi]

Hi Anton. Data Collector and Global Manager have running well without error log. I also checked database for neat and nothing wrong on there. Last log on Local Manager is  "Started VM Selection". No error log found there. previously Data collector also stopped and its status like OpenStack-neat-local-manager. But it is no longer after the service restarted.

Cheers,
Wildan