Different output from URI module on different ansible versions.

[ca...@canta.com.ar]

Hi all.

I'm having a problem with the URI module: in two different servers,
with different ansible versions, I get different output from the same
uri call. I'm taking some code from the uri module docs themselves,
like this:

```yaml
- name: create authentication cookie
uri:
url: "{{ non_public_host }}/stuff"
validate_certs: no
user: "{{ user }}"
password: "{{ pass }}"
method: GET
headers:
Accept: "application/stuff+json"
register: login

# and then we use later the login fact, this way:

- name: get stuff
uri:
url: "{{ non_public_host }}/stuff/list"
validate_certs: no
method: GET
return_content: yes
headers:
Cookie: "{{ login.set_cookie }}"
register: stuff_list

```

It's working so far, for about 2y. But when we installed a new server
recently, we found the output pasted below. There's a difference in
the cookies handling. v2.4 gives us the "cookies" property, while v2.3
returns "set_cookie".


```
# This is the output with ansible 2.4, with some strings edited for
privacy reasons:
TASK [roles/stuff : debugging login] ****************************************
ok: [127.0.0.1] => {
"msg": {
"cache_control": "max-age=0, private, must-revalidate",
"changed": false,
"connection": "close",
"content_type": "application/stuff+json; charset=utf-8",
"cookies": {
"JSESSIONID": "ui74vbyaobr68mj22nmr8ggl"
},
"date": "Tue, 15 May 2018 15:01:53 GMT",
"etag": "\"7a9c36bb30d647628294fb324a7c0e1f\"",
"failed": false,
"msg": "OK (unknown bytes)",
"redirected": false,
"status": 200,
"url": "https://non-public-url/stuff",
"vary": "Accept-Encoding, User-Agent",
"x_content_type_options": "nosniff",
"x_frame_options": "SAMEORIGIN",
"x_request_id": "62f27c01-aa91-4bd8-8979-05430a1c38c4",
"x_runtime": "0.020000",
"x_ua_compatible": "chrome=1",
"x_xss_protection": "1; mode=block"
}
}

# This is the output with ansible 2.3, using the same role with the same url:
TASK [roles/stuff : debugging login] ****************************************
ok: [localhost] => {
"msg": {
"cache_control": "max-age=0, private, must-revalidate",
"changed": false,
"connection": "close",
"content_type": "application/stuff+json; charset=utf-8",
"date": "Tue, 15 May 2018 15:05:17 GMT",
"etag": "\"5ce7ca165672c9ac77e47d0c3617abb2\"",
"expires": "Thu, 01 Jan 1970 00:00:00 GMT",
"msg": "OK (unknown bytes)",
"redirected": false,
"set_cookie":
"JSESSIONID=q5z2ud6gpvkh1cw3x6g5wf3ku;Path=/go;Expires=Tue,
29-May-2018 15:05:19 GMT;Secure;HttpOnly",
"status": 200,
"url": "https://non-public-url/stuff",
"vary": "Accept-Encoding, User-Agent",
"x_content_type_options": "nosniff",
"x_frame_options": "SAMEORIGIN",
"x_request_id": "105e228f-b356-4bc6-87dd-37c459dc008a",
"x_runtime": "0.025000",
"x_ua_compatible": "chrome=1",
"x_xss_protection": "1; mode=block"
}
}
```

I don't think this is a bug, but more like some kind of config
somewhere. But neither the docs or anywhere online seems to point to
any such direction like "config THIS like THIS in order to handle
cookies THIS way". I didn't found it at least.

I don't want to handle this with ad-hoc checks: I would like to force
always the same output from URI.
Any clue on how can I handle this?

Thanks in advance.

[ca...@canta.com.ar]

Transcript from a quick chat on Freenode ansible's IRC channel:


> [12:56] <Canta_> Any clue on how can i force the URI module output format?
> [12:56] <agaffney> can you be more specific?
> [12:57] <Canta_> Sure. I was saying a few minutes ago:
> [12:57] <Canta_> I do a call to a login service, this service gives
> me a cookie, and then I use the cookie for the rest of the calls.
> How to do this is stated in the URI module's docs.
> [12:57] <Canta_> But with a server with ansible 2.3, the URI output
> object has the "set_cookie" property, and with ansible 2.4 it
> returns the "cookies" property. Even when both of them do the same
> request to the same server.
> [12:58] <Canta_> So far, couldn't find how to config that. :(
> [12:58] <agaffney> config what? which field is used to contain the
> cookies? the change between 2.3 and 2.4 was either purposeful or
> unintentional, but it's probably not configurable
> [13:01] <Canta_> @agaffney : yep, i would like to force the property
> for the cookies in the URI module output.
> [13:01] <Canta_> @agaffney : If it's not configurable, then i guess
> there's an error in the module's docs.
> [13:01] <agaffney> the docs don't even show 'set_cookie' or
> 'cookies' as a valid return param for the 'uri' module, but I do see
> an example of accessing .set_cookie from a registered var
> [13:02] <Canta_> @agaffney : yeah. And that code in the example
> worked so far, until v2.4.
> [13:05] <Canta_> @agaffney : Thing is, i've poked in the module's
> code, and couldn't find anything related to those properties, in any
> version (2.3 nor 2.4).
> [13:05] <Canta_> @agaffney : So is kinda a mistery to me where does
> it comes from.
> [13:05] <agaffney> the properties probably come from fetch_url() in
> module_utils.basics
> [13:05] <agaffney> or thereabouts
> [13:06] <Canta_> Hmmm... that's useful. I'll check its code and see
> if there's any significant change there. Thanks agaffney! :D
> [13:08] <agaffney> Canta_:
> https://github.com/ansible/ansible/commit/df8fde4d786e32ed7e8313d63e9a559d5f6137d3#diff-3448ba8582852f79eb0d697763e046d8 looks
> relevant
> [13:09] <agaffney> that's where 'cookies' was added, but I have no
> idea where 'set_cookie' came from previously or where it went to
> [13:10] <agaffney> actually, that commit message says that
> 'set_cookie' should still be there
> [13:10] <agaffney> it's possible that it was removed in a later commit



So, I'm looking now for the origin of the "set_cookie" property. If it
was deleted from the code, then the docs should be changed. Otherwise,
there may be some other problem here.

[ca...@canta.com.ar]

Ended up creating an issue for this:
https://github.com/ansible/ansible/issues/40278

[Kai Stian Olstad]

On 16.05.2018 20:59, ca...@canta.com.ar wrote:
> Ended up creating an issue for this:
> https://github.com/ansible/ansible/issues/40278

My take on this is that is must be a bug.

The documentation in Ansible 2.3 says this is a stable interface, but
it's has clearly changed in 2.4.
No change of the module is mention in the porting guide for 2.4.

--
Kai Stian Olstad