Ansible 2.0 os_server 404 image not found

[V Benincosa]

Hi! 

Checking out the 2.0 release and having some problems launching machine.  When I run: 

ansible-playbook lab-machines.yml 

- name: Ensure lab machines are up

  connection: local

  hosts: localhost

  vars_files:

   - vars/metacloud_vars.yml

  tasks: 

  - name: Ensure lab machines are up

    os_server:

      state: present

#      auth: 

#        auth_url: "{{ lookup('env', 'OS_AUTH_URL') }}"

#        username: "{{ lookup('env', 'OS_USERNAME') }}"

#        password: "{{ lookup('env', 'OS_PASSWORD') }}"

#        project_name: "{{ lookup('env', 'OS_TENANT_NAME') }}"

      name: "{{ item }}"

      key_name: "{{ keypair }}"

      flavor: "{{ m1large }}"

      floating_ip_pools:

        - "{{ floating_ip_pool }}"

      security_groups: "{{ security_group }}"

      userdata:  "{{ lookup('file', 'files/coreos-python.sh') }}"

      image: CoreOS723

    with_items: 

      - lab01

I get the error:  

failed: [localhost] => (item=lab01) => {"extra_data": null, "failed": true, "item": "lab01", "msg": "Error fetching image list: 404 Not Found: The resource could not be found. (HTTP 404)"}

So you're probably thinking: I don't have that image named CoreOS723 defined.  I tried nova image-list and that shows the image: 

$ nova image-show CoreOS723

+-----------------------+--------------------------------------+

| Property              | Value                                |

+-----------------------+--------------------------------------+

| OS-EXT-IMG-SIZE:size  | 493813760                            |

| created               | 2015-08-11T23:46:41Z                 |

| id                    | c762b1d5-5177-421f-8870-aa77b7fe03a2 |

| metadata architecture | x86_64                               |

| metadata description  | CoreOS 723                           |

| minDisk               | 6                                    |

| minRam                | 1000                                 |

| name                  | CoreOS723                            |

| progress              | 100                                  |

| status                | ACTIVE                               |

| updated               | 2015-08-11T23:50:12Z                 |

+-----------------------+--------------------------------------+

But then I noticed that glance image-list does not: 

glance image-list

+----+------+

| ID | Name |

+----+------+

+----+------+

This worked with the older nova commands before but seems to not work with os_compute now.  I'm doing this on Cisco's MetaCloud so I don't have operator permissions to change things.  I'm using python-glanceclient 1.0.0

Thanks for any pointers. 

[V Benincosa]

I downgraded python-glanceclient to 0.19.0 and it seems to get past that error.  Now I get: 

failed: [localhost] => (item=lab01) => {"extra_data": null, "failed": true, "item": "lab01", "msg": "Error fetching image list: Expected endpoint"}

So it looks like I'll have to downgrade most of my python openstack clients one revision for this to work in my environment. 

[V Benincosa]

well, that was the wrong way to go.  Shade reinstalled version python-glanceclient 1.0.0 so I guess I need to figure out why glance doesn't return any images. Metacloud uses icehouse. 

[V Benincosa]

Running: export OS_IMAGE_API_VERSION=1 (and placing in .bash_profile) will now make glance image-list show something... but ansible still gets the same error.  

[Greg DeKoenigsberg]

So this is specifically in the os_ modules, right?

--g

> --
> You received this message because you are subscribed to the Google Groups
> "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ansible-proje...@googlegroups.com.
> To post to this group, send email to ansible...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ansible-project/629025da-885b-40cf-87df-1cf0261d0d56%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.



--
Greg DeKoenigsberg
Ansible Community Guy

Find out why SD Times named Ansible
their #1 Company to Watch in 2015:
http://sdtimes.com/companies-watch-2015/

[James Martin]

I've successfully launched servers with os_server as of today, with the latest version of shade, so I don't think the module is completely broken.  I do see that a 404 is being returned in your error, which is a "page not found" status code (https://en.wikipedia.org/wiki/HTTP_404).  My guess is that you're getting the initial token after keystone authentication, but a subsequent request is failing.  Since you don't have access to the openstack infrastructure to debug, I'd recommend using tcpdump and taking a look at what request is causing the 404.

- James

[Vallard]

Yes.

> You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/VwXk9GMJhhY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.

> To post to this group, send email to ansible...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAM1FbhHd1NCVkdfAT1s3GygXo8-ebUkDHT9jHEhkUVaORGou-g%40mail.gmail.com.

[Vallard]

The 404 I believe comes from ansible defaulting to the v2 for glance, in which case no images are found and thus the 404 error.  Our openstack only has v1 support. 

When I run glance image-list with the OS_IMAGE_API_VERSION=1 the command line tools show images.

This is the new behavior change in Python-glanceclient 1.0.0

Does Ansible  honor that flag?

--

You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/VwXk9GMJhhY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/6b47fe88-0512-4d7e-936a-202d4cb228e6%40googlegroups.com.

[V Benincosa]

I've done some more debugging on this and it looks like shade is putting an extra v1 in the URL so it looks like: 

http://some.client.metacloud.net:9292/v1/v1/images/detail?limit=20

I ended up changing the glanceclient/v1/images.py by removing all the v1s.  This solved the ansible problem, but now the command line doesn't work even though ansible does (e.g: glance image-list doesn't work).  I'll start digging into the shade implementation to see where this extra v1 is being added.  Would love any help.  

[Monty Taylor]

Sorry for your troubles. The problem is almost certianly in the v1/v2 version detection. I think we can do better for you here. However, in the mean time - would you mind trying something to test a hypothesis?

Make a ~/.config/openstack/clouds.yaml with:

clouds:
  metacloud:
    auth:
      username: your-user
      password: your-password
      project_name: your-project-name
      auth_url: your-auth-url
    image_api_version: '1'

And then put "cloud: metacloud" in your playbook and see if it works?

Figuring out v1 vs. v2 in glance is _hard_ and complex and very prone to breaking, which is why it's one of the tings we put in known-cloud profiles in os-client-config.

Now - if that works, awesome. It still sucks as a user experience for you - so I think we need to make sure we can plumb api version overrides through sanely in playbooks. Also - I'm going to go poke the metacloud folks about getting me a metacloud so that I can add a metacloud profile to occ.

Also - I'll see if I can reproduce and find another workaround/detection in case all of the above don't work.

[V Benincosa]

Thanks a bunch Monty for looking into that.  Using the clouds.yaml file didn't work for me.  Same error with 404.  I instead hacked shade to make it work.  

in def get_session_endpoint in the shade __init__ I added a sub to get rid of the v1:

if service_key == 'image':

        endpoint = re.sub('/v1', '', endpoint)

then passed that into the client create. I didn't understand why the endpoint was never passed into the client create around line 605.  Total hack, but I'm running now.  

Also, I didn't see in github a place to open issues or how you track this.  Thanks for any pointers.  I'm happy to help. 

[Robyn Bergeron]

Per a similar threat on twitter with the same folks involved :D ... (https://twitter.com/vallard/status/639862545738633216)

... I think we all agree this is a bit confusing. I believe the place where shade issues are currently supposed to be filed is in the openstack storyboard for shade [1] -- but I don't know that that's obvious to people having trouble with the associated modules or shade itself.

Any suggestions on what might be better? Hoping to find something logical that accommodates the overlapping workflows of two projects... :)

-robyn

[1] https://storyboard.openstack.org/#!/project/760

 

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.

To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/1f43ebe2-1246-488b-a098-fe55d26c6b1d%40googlegroups.com.