Scalr VPC Router not sending HostUp event call

74 views
Skip to first unread message

Aatxe Urrutia

unread,
Mar 25, 2014, 10:20:19 PM3/25/14
to scalr-...@googlegroups.com
Hello I have setup opensource scalr in both ec2 ( which is pretty straightforward) and also in my local location ( to avoid having an instance running in ec2 24/7). I have registered the system and gotten whitelisted and downloaded the templates but when I bring up the VPC router it goes through all the stages then gets terminated because it doesn't get to the "HostUp" stage of the calls.


I can login via the scalr key and see the connections ( & (I manually tested via telnet all the connections through the vpc to the local and public EIP)) in the logs such as

                "name": "SCALR_AVAIL_ZONE",
                "value": "us-west-2b"
            },
            {
                "name": "SCALR_EVENT_IMAGE_ID",
                "value": "ami-6d68f95d"
            },
            {
                "name": "SCALR_EVENT_EXTERNAL_IP",
                "value": "54.1x6.xx.xx"
            },
            {
                "name": "SCALR_EVENT_INTERNAL_IP",
                "value": "10.x.x.2xx"
            },
            {
                "name": "SCALR_EVENT_ROLE_NAME",
                "value": "router-ubuntu1204"
            },
            {
                "name": "SCALR_EVENT_ISDBMASTER",
                "value": ""
            },
            {
              

                "name": "SCALR_EVENT_INSTANCE_ID",
                "value": "i-81fx84xx"
            },
            {
                "name": "SCALR_EVENT_AMI_ID",
                "value": "ami-6d68f95d"
            },
            {
                "name": "SCALR_EVENT_REGION",
                "value": "us-west-2"
            },
            {
                "name": "SCALR_EVENT_AVAIL_ZONE",
                "value": "us-west-2b"
            },
            {
                "name": "SCALR_EVENT_NAME",
                "value": "HostInit"
            }
        ],
        "message_id": "898695xxxxxxxxxxxxx",
        "remote_ip": ""
    },
    "meta": {
        "server_id": "100xxxxxxxxxxxxxxxx",
        "scalr_version": "4.5.0"
    },
    "name": "HostInitResponse",
    "id": "xxxxxxxxxxxxxxxxxxxx"
}
2014-03-26 00:51:40,296 - DEBUG - scalarizr.messaging.p2p.consumer - Received message 'HostInitResponse' (message_id: 8xxxxxxxxxxxxxxxxxxxxxxxx, format: json)
2014-03-26 00:51:40,328 - DEBUG - scalarizr.messaging.p2p - Commiting put_ingoing
2014-03-26 00:51:40,328 - DEBUG - scalarizr.messaging.p2p - Commited put_ingoing
2014-03-26 00:51:40,337 - DEBUG - scalarizr.messaging.p2p.consumer - "POST /control HTTP/1.1" 201 -

all of the icons in scalr come on and I can use the ssh and other icons and I can login via scalr key but then it terminates via scalr ui eventually but not the actual node.

Is perhaps there anything that would stop the HostUp call say for example if the following scalr agent code is mistaking the scalr server ip for its own ec2 range? i.e.


def is_private_ip(ipaddr):
    return any(map(lambda x: ipaddr.startswith(x), ('10.', '172.', '192.168.')))


Scalr server ip is in a local dc at 10.1.x & amazon ips are at a 10.4.x


Or is there simply someother part I am not seeing about the  "HostUp" event call & is there a way for me to check this on the VP i.e. specific port and to which ip? i.e. ip of scalr server in its config or some other mechanism?

Aatxe Urrutia

unread,
Mar 26, 2014, 6:23:20 PM3/26/14
to scalr-...@googlegroups.com

Noticed as well although not posted in 1st message that I have no scripting.log at all

Aatxe Urrutia

unread,
Mar 26, 2014, 8:54:23 PM3/26/14
to scalr-...@googlegroups.com
OK so any clue how I change this for the router roles?

did not send 'hostInit' event in 900 seconds after launch (Try increasing timeouts in role settings). Considering it broken. Terminating instance. |     




On Tuesday, March 25, 2014 7:20:19 PM UTC-7, Aatxe Urrutia wrote:

Thomas Orozco

unread,
Mar 27, 2014, 5:43:30 AM3/27/14
to scalr-...@googlegroups.com
Hey there,

Can you provide the full scalarizr_debug.log from the VPC Router?  Most likely something's just going wrong there!

Cheers,

Aatxe Urrutia

unread,
Mar 27, 2014, 2:08:41 PM3/27/14
to scalr-...@googlegroups.com
Hello Thomas,

I have attached the log, thanks
scalarizr_debug.log

Aatxe Urrutia

unread,
Mar 27, 2014, 2:55:25 PM3/27/14
to scalr-...@googlegroups.com
wanted to also post some of the other logs from the scalr system as well just noticed these the uploaded log is from the last router instance fired last night it seems to have gotten farther but still timed out on HostUP logs did show however it responded to a HostInit as do these


| 478 |          | Farm Ec2_TestUtil, role router-ubuntu1204 scaling up. Starting new instance. ServerID = fb51d6a0-12fa-4ec3-97ec-7eb488785ed4.                                                                    |        2 | 1395906488 | FarmLog |      5 |
| 479 |          | Server 'fb51d6a0-12fa-4ec3-97ec-7eb488785ed4' did not send 'hostUp' event in 900 seconds after launch (Try increasing timeouts in role settings). Considering it broken. Terminating instance.   |        3 | 1395906907 | FarmLog |      5 |
| 480 |          | Server 'fb51d6a0-12fa-4ec3-97ec-7eb488785ed4' did not send 'hostUp' event in 900 seconds after launch (Try increasing timeouts in role settings). Considering it broken. Terminating instance.   |        3 | 1395906907 | FarmLog |      5 |

Aatxe Urrutia

unread,
Mar 27, 2014, 5:12:13 PM3/27/14
to scalr-...@googlegroups.com
Update!

I changed timeouts on the php scripts from the cron-ng dir and router came up and started spinning up a node will update again once I see status of regular server node and any changes I have to do.

Aatxe Urrutia

unread,
Mar 27, 2014, 6:18:14 PM3/27/14
to scalr-...@googlegroups.com
Ok so the nodes that came up terminated even though the router was up and running  when I went back into the farm I tried to change it to the same network and it wants to assign an EIP to the system, this is something I don't want to do, I am fine having it with a dynamic ip but dont want to fork $$ for each EIP. I thought the purpose of the router was to allow the nodes to nat through it for outbound access am I missing something?

To recap, router came up fine with a EIP now once I increased timeouts, node in farm spins up with no public ip and only a private ip then terminates.

Aatxe Urrutia

unread,
Mar 27, 2014, 10:00:54 PM3/27/14
to scalr-...@googlegroups.com
ok last item before I give up for the night, seems its pretty random if the router actually comes up, its a timeout issue & since I cant control ec2 prehaps its better to try this with an instance in the vpc, so I will try with that and see if it works better than the one I have locally ( ill still try w that as well)

Thomas Orozco

unread,
Mar 28, 2014, 12:07:41 PM3/28/14
to scalr-...@googlegroups.com
Hi there,

If the nodes try to get a public (Elastic IP) IP it may be due to your VPC configuration. VPC support (especially that router) is being worked on a lot in the next Scalr release, you may have to wait for that, unfortunately.

If increasing the timeouts helps, you can do that in the Farm Designer, where you can set a longer timeout for HostUp. No need to touch the cron jobs.

The developers are reading this group, so I'm sure that if they see something they'll help you out.

Cheers, 

--
You received this message because you are subscribed to a topic in the Google Groups "scalr-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/scalr-discuss/jSKZoAc9qf8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to scalr-discus...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages