How to use new Azure modules

[Takeshi Kuramochi]

Hi All,

I would like to know details to use Azure modules ( azure_rm_xxxxxx )
. because it failed tasks for azure.

I already have own azure account.
I think that I must get some objects ( ad_user / password ) on my
azure portal before I write a playbook.
I read below.
https://azure.microsoft.com/en-us/documentation/articles/automation-credentials/

I set ~/.azure/credentials then I run playbook.

---
- name: VM image fact
azure_rm_virtualmachineimage_facts:
location: eastus
publisher: OpenLogic
offer: CentOS
sku: '7.1'
version: '7.1.20160308'


It failed.

TASK [azure : VM image fact] ***************************************************
task path: /home/vagrant/build_host/roles/azure/tasks/test.yml:1
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: vagrant
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo
$HOME/.ansible/tmp/ansible-tmp-1464532575.87-18674131270164 `" && echo
ansible-tmp-1464532575.87-18674131270164="` echo
$HOME/.ansible/tmp/ansible-tmp-1464532575.87-18674131270164 `" ) &&
sleep 0'
<localhost> PUT /tmp/tmpxJJVCb TO
/home/vagrant/.ansible/tmp/ansible-tmp-1464532575.87-18674131270164/azure_rm_virtualmachineimage_facts
<localhost> EXEC /bin/sh -c 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8
LC_MESSAGES=en_US.UTF-8 /usr/bin/python
/home/vagrant/.ansible/tmp/ansible-tmp-1464532575.87-18674131270164/azure_rm_virtualmachineimage_facts;
rm -rf "/home/vagrant/.ansible/tmp/ansible-tmp-1464532575.87-18674131270164/"
> /dev/null 2>&1 && sleep 0'
fatal: [localhost]: FAILED! => {"changed": false, "failed": true,
"invocation": {"module_name": "azure_rm_virtualmachineimage_facts"},
"module_stderr":
"/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:315:
SNIMissingWarning: An HTTPS request has been made, but the SNI
(Subject Name Indication) extension to TLS is not available on this
platform. This may cause the server to present an incorrect TLS
certificate, which can cause validation failures. For more
information, see
https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.\n
SNIMissingWarning\n/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:120:
InsecurePlatformWarning: A true SSLContext object is not available.
This prevents urllib3 from configuring SSL appropriately and may cause
certain SSL connections to fail. For more information, see
https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.\n
InsecurePlatformWarning\nTraceback (most recent call last):\n File
\"/tmp/ansible_pmEfA1/ansible_module_azure_rm_virtualmachineimage_facts.py\",
line 239, in <module>\n main()\n File
\"/tmp/ansible_pmEfA1/ansible_module_azure_rm_virtualmachineimage_facts.py\",
line 236, in main\n AzureRMVirtualMachineImageFacts()\n File
\"/tmp/ansible_pmEfA1/ansible_module_azure_rm_virtualmachineimage_facts.py\",
line 149, in __init__\n super(AzureRMVirtualMachineImageFacts,
self).__init__(self.module_arg_spec)\n File
\"/tmp/ansible_pmEfA1/ansible_modlib.zip/ansible/module_utils/azure_rm_common.py\",
line 169, in __init__\n File
\"/usr/local/lib/python2.7/dist-packages/msrestazure/azure_active_directory.py\",
line 261, in __init__\n self.set_token()\n File
\"/usr/local/lib/python2.7/dist-packages/msrestazure/azure_active_directory.py\",
line 296, in set_token\n raise_with_traceback(AuthenticationError,
\"\", err)\n File
\"/usr/local/lib/python2.7/dist-packages/msrest/exceptions.py\", line
50, in raise_with_traceback\n raise
error\nmsrest.exceptions.AuthenticationError: , InvalidGrantError:
(invalid_grant) AADSTS50034: To sign into this application the account
must be added to the management.core.windows.net directory.\r\nTrace
ID: cc08968a-517f-4e8b-ac2e-09423825472c\r\nCorrelation ID:
1bd9c2bf-8299-4455-88b1-de3eced3eb80\r\nTimestamp: 2016-05-29
14:36:19Z\n", "module_stdout": "", "msg": "MODULE FAILURE", "parsed":
false}

PLAY RECAP *********************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=1

Maybe ~/.azure/credentials has problems, but I did not understand.

~/.azure/credentials
subscription_id: <my subscription id>
ad_user: xxxxxxx
password: xxxxxxx



Any and all help is appreciated!

[Tuomas-Matti Soikkeli]

I have the same problem. 

I have azure_rm_virtualmachine task with defaults and credentials configured in ~/.azure/credentials. I can access the resource manager with azure cli and via portal.azure.com 

-vvvv output: 

TASK [Create VM with defaults] *************************************************

task path: /Users/tsoikkel/dev/mtg-rate-estimate-ui/ansible/rate-estimate-ui.yml:4

Using module file /Users/tsoikkel/.bin/ansible/lib/ansible/modules/core/cloud/azure/azure_rm_virtualmachine.py

<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: tsoikkel

<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1467294814.59-271701080215113 `" && echo ansible-tmp-1467294814.59-271701080215113="` echo $HOME/.ansible/tmp/ansible-tmp-1467294814.59-271701080215113 `" ) && sleep 0'

<127.0.0.1> PUT /var/folders/rl/bjksdpqn5w5_qk0fgv7p4z352df6b5/T/tmpD3jnAy TO /Users/tsoikkel/.ansible/tmp/ansible-tmp-1467294814.59-271701080215113/azure_rm_virtualmachine.py

<127.0.0.1> EXEC /bin/sh -c 'chmod -R u+x /Users/tsoikkel/.ansible/tmp/ansible-tmp-1467294814.59-271701080215113/ && sleep 0'

<127.0.0.1> EXEC /bin/sh -c '/usr/local/opt/python/bin/python2.7 /Users/tsoikkel/.ansible/tmp/ansible-tmp-1467294814.59-271701080215113/azure_rm_virtualmachine.py; rm -rf "/Users/tsoikkel/.ansible/tmp/ansible-tmp-1467294814.59-271701080215113/" > /dev/null 2>&1 && sleep 0'

An exception occurred during task execution. The full traceback is:

Traceback (most recent call last):

  File "/var/folders/rl/bjksdpqn5w5_qk0fgv7p4z352df6b5/T/ansible_5hoWa8/ansible_module_azure_rm_virtualmachine.py", line 1281, in <module>

    main()

  File "/var/folders/rl/bjksdpqn5w5_qk0fgv7p4z352df6b5/T/ansible_5hoWa8/ansible_module_azure_rm_virtualmachine.py", line 1278, in main

    AzureRMVirtualMachine()

  File "/var/folders/rl/bjksdpqn5w5_qk0fgv7p4z352df6b5/T/ansible_5hoWa8/ansible_module_azure_rm_virtualmachine.py", line 527, in __init__

    supports_check_mode=True)

  File "/var/folders/rl/bjksdpqn5w5_qk0fgv7p4z352df6b5/T/ansible_5hoWa8/ansible_modlib.zip/ansible/module_utils/azure_rm_common.py", line 164, in __init__

  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/msrestazure/azure_active_directory.py", line 336, in __init__

    self.set_token()

  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/msrestazure/azure_active_directory.py", line 371, in set_token

    raise_with_traceback(AuthenticationError, "", err)

  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/msrest/exceptions.py", line 50, in raise_with_traceback

    raise error

msrest.exceptions.AuthenticationError: , InvalidGrantError: (invalid_grant) AADSTS70002: Error validating credentials. AADSTS50126: Invalid username or password

Trace ID: e88d80a1-dbcb-4587-a61b-408e0803ea2b

Correlation ID: 6bab2577-18f9-4b2c-90d2-e6e09e3d00f7

Timestamp: 2016-06-30 13:53:35Z

fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_name": "azure_rm_virtualmachine"}, "module_stderr": "Traceback (most recent call last):\n  File \"/var/folders/rl/bjksdpqn5w5_qk0fgv7p4z352df6b5/T/ansible_5hoWa8/ansible_module_azure_rm_virtualmachine.py\", line 1281, in <module>\n    main()\n  File \"/var/folders/rl/bjksdpqn5w5_qk0fgv7p4z352df6b5/T/ansible_5hoWa8/ansible_module_azure_rm_virtualmachine.py\", line 1278, in main\n    AzureRMVirtualMachine()\n  File \"/var/folders/rl/bjksdpqn5w5_qk0fgv7p4z352df6b5/T/ansible_5hoWa8/ansible_module_azure_rm_virtualmachine.py\", line 527, in __init__\n    supports_check_mode=True)\n  File \"/var/folders/rl/bjksdpqn5w5_qk0fgv7p4z352df6b5/T/ansible_5hoWa8/ansible_modlib.zip/ansible/module_utils/azure_rm_common.py\", line 164, in __init__\n  File \"/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/msrestazure/azure_active_directory.py\", line 336, in __init__\n    self.set_token()\n  File \"/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/msrestazure/azure_active_directory.py\", line 371, in set_token\n    raise_with_traceback(AuthenticationError, \"\", err)\n  File \"/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/msrest/exceptions.py\", line 50, in raise_with_traceback\n    raise error\nmsrest.exceptions.AuthenticationError: , InvalidGrantError: (invalid_grant) AADSTS70002: Error validating credentials. AADSTS50126: Invalid username or password\r\nTrace ID: xxxxxxxxxxxx \r\nCorrelation ID: xxxxxxxxxxxxx \r\nTimestamp: 2016-06-30 13:53:35Z\n", "module_stdout": "", "msg": "MODULE FAILURE", "parsed": false}

[Matt Davis]

Have you read and followed http://docs.ansible.com/ansible/guide_azure.html about creating the service principals in AD?

[Tuomas-Matti Soikkeli]

On Thursday, 30 June 2016 20:03:00 UTC+3, Matt Davis wrote:

Have you read and followed http://docs.ansible.com/ansible/guide_azure.html about creating the service principals in AD?

Yes and no.

I do not have access to classic portal, it is denied by our enterprise policy. Authenticating with service principals require access to classic portal. So I need to get the AD authentication to work.

I currently have "contributor" privileges in azure, might that be the case that I don't have required permissions?

Service principal credentials shouldn't be required according to the documentation from https://docs.ansible.com/ansible/azure_rm_virtualmachine_module.html:

Note
Alternatively, credentials can be stored in ~/.azure/credentials. This is an ini file containing a [default] section and the following keys: subscription_id, client_id, secret and tenant or subscription_id, ad_user and password. It is also possible to add additional profiles. Specify the profile by passing profile or setting AZURE_PROFILE in the environment.

I've done some test to verify that previously mentioned has some issues.

1) Passing credentials as parameters doesn't work:

$ cat test.yml


- hosts: localhost
  tasks:
    - name: Create VM with defaults
      azure_rm_virtualmachine:
        resource_group: Testing
        name: testvm10
        ad_user: 'te...@user.com'
        password: 'password'
        subscription_id: 'test-sub'
        admin_username: root
        admin_password: pass
        image:
          offer: CentOS
          publisher: OpenLogic
          sku: '7.1'
          version: latest

$ ansible-playbook  test.yml

 [WARNING]: provided hosts list is empty, only localhost is available

PLAY [localhost] ***************************************************************


TASK [setup] *******************************************************************
ok: [localhost]

TASK [Create VM with defaults] *************************************************

fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Failed to get credentials. Either pass as parameters, set environment variables, or define a profile in ~/.azure/credentials."}


NO MORE HOSTS LEFT *************************************************************
 to retry, use: --limit @test.retry

PLAY RECAP *********************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1  

2) Providing env variables doesn't work:

$ cat test.env

AZURE_AD_USER='te...@test.com'
AZURE_PASSWORD='password'
AZURE_SUBSCRIPTION_ID='test-sub'

source test.env && ansible-playbook test.yml:

$ ansible-playbook  test.yml
 [WARNING]: provided hosts list is empty, only localhost is available

PLAY [localhost] ***************************************************************


TASK [setup] *******************************************************************
ok: [localhost]

TASK [Create VM with defaults] *************************************************

fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Failed to get credentials. Either pass as parameters, set environment variables, or define a profile in ~/.azure/credentials."}


NO MORE HOSTS LEFT *************************************************************
 to retry, use: --limit @test.retry

PLAY RECAP *********************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1  

3) credentials profile is working, but doesn't really. It seems that the module incorrectly excepts tenant id and fails when I have not provided it.

cat ~/.azure/credentials


[default]
ad_user='te...@test.com'
password='password'
subscription_id='sub-test'


ansible-playbook test.yml


t-m@ubuntubox:~/dev/mtg-rate-estimate-ui$ ansible-playbook test.yml
 [WARNING]: provided hosts list is empty, only localhost is available




PLAY [localhost] ***************************************************************


TASK [setup] *******************************************************************
ok: [localhost]

TASK [Create VM with defaults] *************************************************

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: Timestamp: 2016-06-30 19:43:52Z
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_AhQ2Oz/ansible_module_azure_rm_virtualmachine.py\", line 1281, in <module>\n    main()\n  File \"/tmp/ansible_AhQ2Oz/ansible_module_azure_rm_virtualmachine.py\", line 1278, in main\n    AzureRMVirtualMachine()\n  File \"/tmp/ansible_AhQ2Oz/ansible_module_azure_rm_virtualmachine.py\", line 527, in __init__\n    supports_check_mode=True)\n  File \"/tmp/ansible_AhQ2Oz/ansible_modlib.zip/ansible/module_utils/azure_rm_common.py\", line 164, in __init__\n  File \"/home/t-m/.local/lib/python2.7/site-packages/msrestazure/azure_active_directory.py\", line 261, in __init__\n    self.set_token()\n  File \"/home/t-m/.local/lib/python2.7/site-packages/msrestazure/azure_active_directory.py\", line 296, in set_token\n    raise_with_traceback(AuthenticationError, \"\", err)\n  File \"/home/t-m/.local/lib/python2.7/site-packages/msrest/exceptions.py\", line 50, in raise_with_traceback\n    raise error\nmsrest.exceptions.AuthenticationError: , InvalidClientIdError: (invalid_request) AADSTS90002: Requested tenant identifier '********.com'' is not valid.\r\nTrace ID: 3698ff8f-0117-4a0b-b4db-795d9d397d9e\r\nCorrelation ID: 46b00886-e1ed-43bd-9dfe-b1af1a59da45\r\nTimestamp: 2016-06-30 19:43:52Z\n", "module_stdout": "", "msg": "MODULE FAILURE", "parsed": false}


NO MORE HOSTS LEFT *************************************************************
 to retry, use: --limit @test.retry

PLAY RECAP *********************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1  

execution environment:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 15.10
Release: 15.10
Codename: wily


$ ansible --version
ansible 2.1.1.0 (stable-2.1 a598f26006) last updated 2016/06/30 22:57:28 (GMT +300)
  lib/ansible/modules/core: (detached HEAD 1d0d5db97a) last updated 2016/06/30 21:14:15 (GMT +300)
  lib/ansible/modules/extras: (detached HEAD 00b8b96906) last updated 2016/06/30 21:14:19 (GMT +300)
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides


$ pip list | grep azure
azure (2.0.0rc2)
azure-common (1.1.2)
azure-graphrbac (0.30.0rc2)
azure-mgmt (0.30.0rc2)
azure-mgmt-authorization (0.30.0rc2)
azure-mgmt-cdn (0.30.0rc2)
azure-mgmt-compute (0.30.0rc2)
azure-mgmt-logic (0.30.0rc2)
azure-mgmt-network (0.30.0rc2)
azure-mgmt-notificationhubs (0.30.0rc2)
azure-mgmt-nspkg (1.0.0)
azure-mgmt-redis (0.30.0rc2)
azure-mgmt-resource (0.30.0rc2)
azure-mgmt-scheduler (0.30.0rc2)
azure-mgmt-storage (0.30.0rc2)
azure-mgmt-web (0.30.0rc2)
azure-nspkg (1.0.0)
azure-servicebus (0.20.1)
azure-servicemanagement-legacy (0.20.2)
azure-storage (0.30.0)
msrestazure (0.2.1)

[Chris Houseknecht]

This was definitely a problem, and it has been fixed in the devel branch with PR #16559. To use this fix you will need to:

- Run from source 

- Install Azure Python SDK == 2.0.0rc5

--Chris