Re: [boto-users] boto.ec2.autoscale.get_all_groups() returns max_size as None

[Mitchell Garnaat]

I'm seeing the same thing.  I'm investigating now.  Thanks for reporting it.

Mitch

On Thu, Aug 16, 2012 at 8:25 PM, ro <r...@jeez.org> wrote:

I'm using 2.5.2 and when I make call get_all_groups() the AutoScale group returned has a max_size of None.

It looks like the XML parsing may be broken, but I'm not sure where it's falling apart. If I modify connection.py to print out the body I see the correct MaxSize in the body of the response as returned in get_list():

         body = response.read()

<DescribeAutoScalingGroupsResponse xmlns="http://autoscaling.amazonaws.com/doc/2011-01-01/">

  <DescribeAutoScalingGroupsResult>

    <AutoScalingGroups>

      <member>

        <Tags>

          <member>

            <ResourceId>201208161118</ResourceId>

            <PropagateAtLaunch>true</PropagateAtLaunch>

            <Value>demo</Value>

            <Key>env</Key>

            <ResourceType>auto-scaling-group</ResourceType>

          </member>

          <member>

            <ResourceId>201208161118</ResourceId>

            <PropagateAtLaunch>true</PropagateAtLaunch>

            <Value>server</Value>

            <Key>nodeType</Key>

            <ResourceType>auto-scaling-group</ResourceType>

          </member>

          <member>

            <ResourceId>201208161118</ResourceId>

            <PropagateAtLaunch>true</PropagateAtLaunch>

            <Value>201208161118</Value>

            <Key>server</Key>

            <ResourceType>auto-scaling-group</ResourceType>

          </member>

        </Tags>

        <SuspendedProcesses/>

        <AutoScalingGroupName>201208161118</AutoScalingGroupName>

        <HealthCheckType>EC2</HealthCheckType>

        <CreatedTime>2012-08-16T15:18:38.049Z</CreatedTime>

        <EnabledMetrics/>

        <LaunchConfigurationName>201208161118</LaunchConfigurationName>

        <Instances>

          <member>

            <HealthStatus>Healthy</HealthStatus>

            <AvailabilityZone>us-east-1d</AvailabilityZone>

            <InstanceId>i-xxxxxxxx</InstanceId>

            <LaunchConfigurationName>201208161118</LaunchConfigurationName>

            <LifecycleState>InService</LifecycleState>

          </member>

          <member>

            <HealthStatus>Healthy</HealthStatus>

            <AvailabilityZone>us-east-1a</AvailabilityZone>

            <InstanceId>i-xxxxxxxx</InstanceId>

            <LaunchConfigurationName>201208161118</LaunchConfigurationName>

            <LifecycleState>InService</LifecycleState>

          </member>

        </Instances>

        <DesiredCapacity>2</DesiredCapacity>

        <AvailabilityZones>

          <member>us-east-1c</member>

          <member>us-east-1a</member>

          <member>us-east-1d</member>

        </AvailabilityZones>

        <LoadBalancerNames/>

        <MinSize>2</MinSize>

        <VPCZoneIdentifier/>

        <HealthCheckGracePeriod>0</HealthCheckGracePeriod>

        <DefaultCooldown>300</DefaultCooldown>

        <AutoScalingGroupARN>arn:aws:autoscaling:us-east-1:XXXXXXXXXXXX:autoScalingGroup:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:autoScalingGroupName/201208161118</AutoScalingGroupARN>

        <TerminationPolicies>

          <member>Default</member>

        </TerminationPolicies>

        <MaxSize>2</MaxSize>

-----snip-----

What's really confusing is that this was working fine earlier today.

Anyone have any ideas as to what may be going on?

--
You received this message because you are subscribed to the Google Groups "boto-users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/boto-users/-/Dc1j-LVDpl0J.
To post to this group, send email to boto-...@googlegroups.com.
To unsubscribe from this group, send email to boto-users+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/boto-users?hl=en.

[ro]

I noticed that someone posted issue https://github.com/boto/boto/issues/922: with the same problem.

I replied to that issue with a partial XML response from the AWS API endpoint with a MaxSize that actually has data.

So the API is responding with data, but could it be that the format is invalid so the XML parser isn't able to properly populate group.max_size?

Sorry, all of my AutoScale automation scripts no longer work properly and I'm really at a loss as to how to work around this.

[Mitchell Garnaat]

The problem seems to be that a new element has shown up in the response.  If you look near the end of the XML response, there is an element called TerminationPolicies.  That didn't used to be there and it doesn't appear in the documentation.  I'm not exactly sure why it's causing the SAX parser to skip the MaxSize element.  I'm not validating the XML against a schema or anything so I would have thought it would just ignore it but when I add an explicit handler for the TerminationPolicies element, it works again.

I'll run tests against these changes and, if everything checks out, I'll commit the changes to github this morning.

Mitch

To view this discussion on the web visit https://groups.google.com/d/msg/boto-users/-/rj548-QpaKIJ.

[ro]

I have a ticket open with AWS -- I'll update it with a note on their out of date docs.

Thanks for looking into this so quickly!

[Mitchell Garnaat]

I pushed up a change to github develop that should allow the response to be parsed correctly now.  Let me know if you have any problems with that.

Mitch

To view this discussion on the web visit https://groups.google.com/d/msg/boto-users/-/J8djEWVe_tsJ.

[ro]

Yes, it works for me now.

Did you intend to leave the extra print statements?

[ro]

I am seeing another error when trying to start an AutoScale group which I don't see in the version of boto I sourced from pip.

error_message: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

reason: Forbbiden

There's a lot more there, but I'm still looking through it to remove sensitive data.

[ro]

Here's the full failure:

 

Error creating launch config! Message: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
The Canonical String for this request should have been
'POST
/
Action=CreateLaunchConfiguration&ImageId=ami-xxxxxxxx&InstanceMonitoring.Enabled=false&InstanceType=m1.large&KeyName=xxxxxxxxxxxx&LaunchConfigurationName=xxxxxxxxxxxxxxxx&SecurityGroups.member.1=xxxxxxxxxxxxx&SecurityGroups.member.2=xxxxxxxxxxxxx&SecurityGroups.member.3=xxxxxxxxxxxx&SecurityGroups.member.4=xxxxxxxxxxxxxxxxxxx&SecurityGroups.member.5=xxxxxxxxxxxxxx&SecurityGroups.member.6=xxxxxxxxxxxxxt&UserData=COMPRESSEDUSERDATA&Version=2011-01-01
host:autoscaling.us-east-1.amazonaws.com
x-amz-date:20120817T150038Z
host;x-amz-date
HASHVALUE'
The String-to-Sign should have been
'AWS4-HMAC-SHA256
20120817T150038Z
20120817/us-east-1/autoscaling/aws4_request
DIFFERENTHASHVALUE'
 reason: Forbidden

The code calling this is:

asConn = boto.ec2.autoscale.connect_to_region(region_name=regionName)
lc = boto.ec2.autoscale.launchconfig.LaunchConfiguration(connection=asConn, name=name, image_id=ami,
                                                             key_name=keypairName,
                                                             security_groups=secGrps,
                                                             instance_type=instanceType,
                                                             user_data=userData)
try:
    asConn.create_launch_configuration(lc)
except Exception, e:
    print 'Error creating launch config! Message: ' + e.error_message + ' reason: ' + e.reason
    sys.exit(1)

[Mitchell Garnaat]

Could you enable full debug output (see https://gist.github.com/1539859) and either post the output or email the output to me?

Thanks,

Mitch

To view this discussion on the web visit https://groups.google.com/d/msg/boto-users/-/FaRzRb_uH04J.

[Mitchell Garnaat]

So far, I have been unable to reproduce this error.  Still trying.

Mitch

[ro]

Here's the output:

2012-08-17 11:59:49,533 boto.log [DEBUG]:Method: POST

2012-08-17 11:59:49,533 boto.log [DEBUG]:Path: /

2012-08-17 11:59:49,533 boto.log [DEBUG]:Data: 

2012-08-17 11:59:49,534 boto.log [DEBUG]:Headers: {}

2012-08-17 11:59:49,534 boto.log [DEBUG]:Host: autoscaling.us-east-1.amazonaws.com

2012-08-17 11:59:49,534 boto.log [DEBUG]:establishing HTTPS connection: host=autoscaling.us-east-1.amazonaws.com, kwargs={}

2012-08-17 11:59:49,534 boto.log [DEBUG]:Token: None

2012-08-17 11:59:49,535 boto.log [DEBUG]:CanonicalRequest:

POST

/

Action=CreateLaunchConfiguration&ImageId=ami-xxxxxxxx&InstanceMonitoring.Enabled=false&InstanceType=m1.large&KeyName=xxxxxxxxxxxx&LaunchConfigurationName=xxxxxxxxxxxxxxxxx&SecurityGroups.member.1=xxxxxxxx&SecurityGroups.member.2=xxxxxxxx&SecurityGroups.member.3=xxxxxxxx&SecurityGroups.member.4=xxxxxxxx&SecurityGroups.member.5=xxxxxxxx&SecurityGroups.member.6=xxxxxxxx&UserData=GZIPEDDATA&Version=2011-01-01

host:autoscaling.us-east-1.amazonaws.com

x-amz-date:20120817T155949Z

host;x-amz-date

HASH1

2012-08-17 11:59:49,535 boto.log [DEBUG]:StringToSign:

AWS4-HMAC-SHA256

20120817T155949Z

20120817/us-east-1/autoscaling/aws4_request

HASH2

2012-08-17 11:59:49,535 boto.log [DEBUG]:Signature:

HASHSIG1

send: 'POST /?Action=CreateLaunchConfiguration&ImageId=ami-xxxxxxxx&InstanceMonitoring.Enabled=false&InstanceType=m1.large&KeyName=xxxxxxxx&LaunchConfigurationName=xxxxxxxx&SecurityGroups.member.1=xxxxxxxx&SecurityGroups.member.2=xxxxxxxx&SecurityGroups.member.3=xxxxxxxx&SecurityGroups.member.4=xxxxxxxx&SecurityGroups.member.5=xxxxxxxx&SecurityGroups.member.6=xxxxxxxx&UserData=GZIPPEDDATA&Version=2011-01-01 HTTP/1.1\r\nHost: autoscaling.us-east-1.amazonaws.com\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nAuthorization: AWS4-HMAC-SHA256 Credential=xxxxxxxx/20120817/us-east-1/autoscaling/aws4_request,SignedHeaders=host;x-amz-date,Signature=HASHSIG1\r\nX-Amz-Date: 20120817T155949Z\r\nUser-Agent: Boto/2.5.2 (linux2)\r\n\r\n'

reply: 'HTTP/1.1 403 Forbidden\r\n'

header: x-amzn-RequestId: AWS-REQ-ID

header: Content-Type: text/xml

header: Content-Length: 5309

header: Date: Fri, 17 Aug 2012 15:59:49 GMT

2012-08-17 11:59:49,931 boto.log [DEBUG]:<ErrorResponse xmlns="http://autoscaling.amazonaws.com/doc/2011-01-01/">

  <Error>

    <Type>Sender</Type>

    <Code>SignatureDoesNotMatch</Code>

    <Message>The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been

'POST

/

Action=CreateLaunchConfiguration&amp;ImageId=ami-xxxxxxxx&amp;InstanceMonitoring.Enabled=false&amp;InstanceType=m1.large&amp;KeyName=xxxxxxxx&amp;LaunchConfigurationName=xxxxxxxx&amp;SecurityGroups.member.1=xxxxxxxx&amp;SecurityGroups.member.2=xxxxxxxx&amp;SecurityGroups.member.3=xxxxxxxx&amp;SecurityGroups.member.4=xxxxxxxx&amp;SecurityGroups.member.5=xxxxxxxx&amp;SecurityGroups.member.6=xxxxxxxx&amp;UserData=GZIPPEDDATA&amp;Version=2011-01-01

host:autoscaling.us-east-1.amazonaws.com

x-amz-date:20120817T155949Z

host;x-amz-date

HASH1'

The String-to-Sign should have been

'AWS4-HMAC-SHA256

20120817T155949Z

20120817/us-east-1/autoscaling/aws4_request

HASHSIG2'

</Message>

  </Error>

  <RequestId>AWS-REQ-ID</RequestId>

</ErrorResponse>

2012-08-17 11:59:49,931 boto.log [ERROR]:403 Forbidden

2012-08-17 11:59:49,931 boto.log [ERROR]:<ErrorResponse xmlns="http://autoscaling.amazonaws.com/doc/2011-01-01/">

  <Error>

    <Type>Sender</Type>

    <Code>SignatureDoesNotMatch</Code>

    <Message>The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been

'POST

/

Action=CreateLaunchConfiguration&amp;ImageId=ami-xxxxxxxx&amp;InstanceMonitoring.Enabled=false&amp;InstanceType=m1.large&amp;KeyName=xxxxxxxx&amp;LaunchConfigurationName=xxxxxxxx&amp;SecurityGroups.member.1=xxxxxxxx&amp;SecurityGroups.member.2=xxxxxxxx&amp;SecurityGroups.member.3=xxxxxxxx&amp;SecurityGroups.member.4=xxxxxxxx&amp;SecurityGroups.member.5=xxxxxxxx&amp;SecurityGroups.member.6=xxxxxxxx&amp;UserData=GZIPPEDDATA&amp;Version=2011-01-01

host:autoscaling.us-east-1.amazonaws.com

x-amz-date:20120817T155949Z

host;x-amz-date

HASH1

The String-to-Sign should have been

'AWS4-HMAC-SHA256

20120817T155949Z

20120817/us-east-1/autoscaling/aws4_request

HASHSIG2'

</Message>

  </Error>

  <RequestId>AWS-REQ-ID</RequestId>

</ErrorResponse>

Is that enough?

[ro]

I just applied your "Add handler for TerminationPolicies element in DescribeAutoScalingGroups response. Fixes 922" diff to my pip 2.5.2 version of boto and didn't see the same Signature errors as before.

It must be some other commit that is causing this problem.

[ro]

It appears that the signature problem is being caused by the fact that I'm passing Gzip compressed UserData to the CreateLaunchConfiguration call.

When I pass no or uncompressed UserData I do not run into this problem.

[Mitchell Garnaat]

Hmm.  Boto does a base64 encoding of the user_data prior to sending it to the service.  I would have thought that would be safe with any sort of data.

--

You received this message because you are subscribed to the Google Groups "boto-users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/boto-users/-/8u8JPRQ92jUJ.

[ro]

Even attempting to pass an empty compressed file returns a 403 from AWS.

[ro]

Here's some sample code to reproduce:

 

#!/usr/bin/env python

import boto

import gzip

import boto.ec2.autoscale

import sys

keyName = 'YOUR_KEYPAIR_NAME'

uncompressedFile = '/tmp/emptyFile.txt'

compressedFile = '/tmp/emptyFile.txt.gz'

# create an empty file

file = open(uncompressedFile, 'w')

file.write('blah')

file.close()

            

# compress it

uncompressed = open(uncompressedFile, 'rb')

compressed = gzip.open(compressedFile, 'wb')

compressed.writelines(uncompressed)

compressed.close()

uncompressed.close()

# read the compressed file

configGzip = open(compressedFile, 'r')

userData = configGzip.read()

configGzip.close        

# connect

asConn = boto.ec2.autoscale.connect_to_region(region_name='us-east-1', debug=2)

# set up the launch config

lc = boto.ec2.autoscale.launchconfig.LaunchConfiguration(connection=asConn, name='test', image_id='ami-aecd60c7',

                                                         key_name='prod-aws_installer',

                                                         security_groups=[],

                                                         instance_type='t1.micro',

                                                         user_data=userData)

# create the launch configuration

try:    

    asConn.create_launch_configuration(lc)

except Exception, e:

    print 'Error creating launch config! Message: ' + e.error_message + ' reason: ' + e.reason

    sys.exit(1)

It should print something like this:

me@hoath:~/src/$ ~/tmp/sampleCode.py 

Error creating launch config! Message: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been

'POST

/

Action=CreateLaunchConfiguration&ImageId=ami-aecd60c7&InstanceMonitoring.Enabled=false&InstanceType=t1.micro&KeyName=YOUR_KEYPAIR_NAME&LaunchConfigurationName=test&UserData=H4sICEmpLlAC%2F2VtcHR5RmlsZS50eHQAS8pJzAAAXGEpzgQAAAA%3D&Version=2011-01-01

host:autoscaling.us-east-1.amazonaws.com

x-amz-date:20120817T202753Z

host;x-amz-date

e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'

The String-to-Sign should have been

'AWS4-HMAC-SHA256

20120817T202753Z

20120817/us-east-1/autoscaling/aws4_request

4eec77ee851108eb3a2036677427bad5e26a24b0a3e06e31a2cb2e7c3c20f4f1'

 reason: Forbidden