Scalr cloudstack network issue

[Aatxe Urrutia]

Hello I have an issue with scalr communication and cloudstack, I can bring up vm's fine i imported then via scalr and it saved them as snapshot when I start the farm it brings the vms up but then says 'Cloudstack handler failed: Argument missing: ipAddressId'


I have a setup in aws as well and what I notice that is apparently different is that my local scalr - cloudstack implementation brings up vm's with one network interface, so would this error come from a lack of a "public interface"?

[Aatxe Urrutia]

Here is an example of raw logs


Origionally from scalr this is the message I see

scalr
----

2014-03-04 00:00:05,946 - FATAL [tran53ddedede:Logger@264] - [FarmID: 2] Cloudstack handler failed: Argument missing: ipAddressId.

When I go into the cloudstack ui and attempt to add a network as interface2 i get this

Cloudstack

ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-45:job-276 = [ 6704521c-a18b-4607-997f-5659625ac9f8 ]) Unexpected exception while executing org.apache.cloudstack.api.command.user.vm.
AddNicToVMCmd
java.lang.NullPointerException
        at com.cloud.network.element.VirtualRouterElement.getRouters(VirtualRouterElement.java:950)
        at com.cloud.network.element.VirtualRouterElement.configDhcpSupportForSubnet(VirtualRouterElement.java:856)
        at com.cloud.network.NetworkManagerImpl.prepareElement(NetworkManagerImpl.java:2075)
        at com.cloud.network.NetworkManagerImpl.prepareNic(NetworkManagerImpl.java:2200)
        at com.cloud.network.NetworkManagerImpl.createNicForVm(NetworkManagerImpl.java:4606)
        at com.cloud.vm.VirtualMachineManagerImpl.addVmToNetwork(VirtualMachineManagerImpl.java:2979)
        at com.cloud.vm.UserVmManagerImpl.addNicToVirtualMachine(UserVmManagerImpl.java:976)
        at org.apache.cloudstack.api.command.user.vm.AddNicToVMCmd.execute(AddNicToVMCmd.java:109)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)

[Aatxe Urrutia]

Basically I have done a couple different redesigns and issue is still the same.

Issue is I am not seeing how to make the template image come up with a "remoteip and localip" and as a result am only getting a localip even though I have that set to the public interface in cloudstack, the image was tested to be able to come up with 2 networks however it is comming up with one in scalr

i.e.


2014-03-04 00:00:02,946 - INFO [tran5315178239fcb:Logger@46] - 'Starting SzrMessaging cronjob...'
2014-03-04 00:00:03,946 - INFO [tran5315178239fcb:Logger@29] - 'Current daemon process PID: '
2014-03-04 00:00:05,946 - INFO [tran5315178239fcb:Logger@68] - 'SzrMessaging daemon started. Memory usage: 5.75M'
2014-03-04 00:00:05,946 - INFO [tran5315178239fcb:Logger@246] - 'Event OnHostInit. Observer: DBEventObserver'
2014-03-04 00:00:05,946 - INFO [tran5315178239fcb:Logger@246] - 'Event OnHostInit. Observer: DNSEventObserver'
2014-03-04 00:00:05,946 - INFO [tran5315178239fcb:Logger@246] - 'Event OnHostInit. Observer: Modules_Platforms_Ec2_Observers_Ebs'
2014-03-04 00:00:05,946 - INFO [tran5315178239fcb:Logger@246] - 'Event OnHostInit. Observer: Modules_Platforms_Cloudstack_Observers_Cloudstack'
2014-03-04 00:00:05,946 - FATAL [tran5315178239fcb:Logger@264] - [FarmID: 2] Cloudstack handler failed: Argument missing: ipAddressId.
2014-03-04 00:00:05,946 - INFO [tran5315178239fcb:Logger@246] - 'Event OnHostInit. Observer: MessagingEventObserver'
2014-03-04 00:00:06,946 - WARN [tran5315178239fcb:Logger@1128] - [FarmID: 2] Cannot deliver message 'UpdateSshAuthorizedKeys' (message_id: 34634e80-e9cc-4be7-8e1d-34c8655d85a1) via REST to server '' (server_id: a9c45ee3-2536-47b0-971e-9fc8248d67a3). Error: 0 Could not parse URL (http://:8013/control)
2014-03-04 00:00:06,946 - WARN [tran5315178239fcb:Logger@1128] - [FarmID: 2] Cannot deliver message 'HostInitResponse' (message_id: 6b123a11-eb8a-487d-85c6-05897a6a5257) via REST to server '' (server_id: a9c45ee3-2536-47b0-971e-9fc8248d67a3). Error: 0 Could not parse URL (http://:8013/control)
2014-03-04 00:00:06,946 - INFO [tran5315178239fcb:Logger@246] - 'Event OnHostInit. Observer: ScalarizrEventObserver'

On Tuesday, March 4, 2014 1:45:39 PM UTC-8, Aatxe Urrutia wrote:

[Aatxe Urrutia]

ok digging into Code I believe I see why


This is hard coded to 10.x and 192.x networks, it is an assumption that these networks are meant to be private only


code is

        public function determineServerIps($client, $server)
        {
            $addr = $server->nic[0]->ipaddress;
            if (strpos($addr, "10.") === 0 || strpos($addr, "192.168") === 0)
                $localIp = $addr;
            else
                $remoteIp = $addr;

            if ($server->publicip)
                $remoteIp = $server->publicip;

            return array(
                'localIp'   => $localIp,
                'remoteIp'  => $remoteIp



Which makes sense however in our case we really didn't want to use public ip ranges in the private ( behind firewall space) its ok just have to design it a little more again might do a hack on my side as well

On Tuesday, March 4, 2014 1:45:39 PM UTC-8, Aatxe Urrutia wrote:

[Aatxe Urrutia]

OK after that I saw it did assign an ip as the remoteIp however as such since it did not work I understand now that basically what needs to happen is that the Router needs to hold an ip and forward all ports much like ec2's system would i.e. with its public and private ips


Will try and lookup how to do that

[Aatxe Urrutia]

Ok posting update, I did get my internal instance to use both amazon and cloudstack by modifying the scalr code I know this is not optimal but it worked for me for the setup I needed.

Basically I was having an issue where both amazon vpc instance and cloudstack instances were being required to have public ip when none where available.

Files modified for EC2
Modules/Platforms/Ec2/Ec2.php

 diff -urNp scalr/src/scalr/app/src/Modules/Platforms/Ec2/Ec2.php ec2/Ec2.php
--- scalr/src/scalr/app/src/Modules/Platforms/Ec2/Ec2.php       2014-01-30 12:05:12.000000000 -0800
+++ ec2/Ec2.php 2014-04-02 18:01:23.473999000 -0700
@@ -105,8 +105,7 @@ class Modules_Platforms_Ec2 extends Modu
                              ->get(0)->instancesSet->get(0);

         return array(
-            'localIp'  => $instance->privateIpAddress,
-            'remoteIp' => $instance->ipAddress
+            'localIp'  => $instance->privateIpAddress
         );
     }


class.DBServer.php
---

diff -urNp scalr/src/scalr/app/src/class.DBServer.php ec2/class.DBServer.php
--- scalr/src/scalr/app/src/class.DBServer.php  2014-01-30 12:05:12.000000000 -0800
+++ ec2/class.DBServer.php      2014-04-02 18:03:21.650424000 -0700
@@ -1,4 +1,5 @@
 <?php
+
 use Scalr\Server\Operations;

 class DBServer
@@ -1037,10 +1038,10 @@ class DBServer
             if (\Scalr::config('scalr.instances_connection_policy') == 'local')
                 $requestHost = "{$this->localIp}:{$ctrlPort}";
             elseif (\Scalr::config('scalr.instances_connection_policy') == 'public')
-                $requestHost = "{$this->remoteIp}:{$ctrlPort}";
+                $requestHost = "{$this->localIp}:{$ctrlPort}";
             elseif (\Scalr::config('scalr.instances_connection_policy') == 'auto') {
                 if ($this->remoteIp)
-                    $requestHost = "{$this->remoteIp}:{$ctrlPort}";
+                    $requestHost = "{$this->localIp}:{$ctrlPort}";
                 else
                     $requestHost = "{$this->localIp}:{$ctrlPort}";
             }

For Cloudstack
Observers/Cloudstack.php

 diff -urNp ./scalr/src/scalr/app/src/Modules/Platforms/Cloudstack/Observers/Cloudstack.php cloudstack/ObserversCloudstack.php
--- ./scalr/src/scalr/app/src/Modules/Platforms/Cloudstack/Observers/Cloudstack.php     2014-01-30 12:05:12.000000000 -0800
+++ cloudstack/ObserversCloudstack.php  2014-04-02 18:05:19.000948000 -0700
@@ -1,4 +1,5 @@
 <?php
+
     class Modules_Platforms_Cloudstack_Observers_Cloudstack extends EventObserver
     {
         public $ObserverName = 'Cloudstack';
@@ -117,9 +118,8 @@
                         $event->DBServer->GetCloudLocation(),
                         $event->DBServer->platform
                     );
-
-                    $ipInfo = $cs->listPublicIpAddresses(null, null, null, null, null, $event->DBServer->remoteIp);
-                    $info = $ipInfo->publicipaddress[0];
+                    $ipInfo = $cs->privateIpAddress(null, null, null, null, null, $event->DBServer->localIp);
+                    $info = $ipInfo->privateIpAddress[0];
                     if ($info->isstaticnat) {
                         if ($info->virtualmachineid == $event->DBServer->GetCloudServerID()) {
                             $this->Logger->warn(new FarmLogMessage($this->FarmID,
@@ -265,4 +265,4 @@
             }
         }
     }

To change the SSH user the key uses after bootstrap for the user I did

diff -urNp ./scalr/src/scalr/app/src/Scalr/UI/Controller/Servers.php cloudstack/Servers.php
--- ./scalr/src/scalr/app/src/Scalr/UI/Controller/Servers.php   2014-01-30 12:05:12.000000000 -0800
+++ cloudstack/Servers.php      2014-04-02 18:08:02.809705000 -0700
@@ -1,4 +1,5 @@
 <?php
+
 use Scalr\Acl\Acl;
 use Scalr\Server\Alerts;
 use Scalr\Service\Aws\Ec2\DataType\InstanceAttributeType;
@@ -327,7 +328,7 @@ class Scalr_UI_Controller_Servers extend
                 'farmId' => $dbServer->farmId,
                 'roleName' => $dbRole->name,
                 'port' => $sshPort,
-                'username' => $dbServer->platform == SERVER_PLATFORMS::GCE ? 'scalr' : 'root',
+                'username' => $dbServer->platform == SERVER_PLATFORMS::GCE ? 'scalr' : 'alloweduser',
                 "key" => base64_encode($sshKey->getPrivate())
             ));
         }


I know these changes are not ideal but I needed to get this done and had a deadline. I also briefly looked over the rest of the code and it appears it should work as expected however it seems that the unmodified version somehow when the variable for remoteIp is null does not unset it or disregard it and as a result of it being null & set attempts to use it which of course results in failure.

[Igor Savchenko]

You just need to configure scalr to use private ips:
https://scalr-wiki.atlassian.net/wiki/display/docs/Configure+Scalr

Setting name: instanes_connection_policy which need to be set to
'local' in your case.

Regards,
Igor

> --
> You received this message because you are subscribed to the Google Groups
> "scalr-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scalr-discus...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.