get_url always return 401 Unauthorized

[Benny Kusman]

HI,

im running on ansible 2.4.2 and trying to get a file from sharepoint, but always gotten 401 Unauthorized error (even i confirmed that the password is correct)

fatal: [hostname -> localhost]: FAILED! => {

    "changed": false, 

    "dest": "filename", 

    "invocation": {

        "module_args": {

            "attributes": null, 

            "backup": null, 

            "checksum": "", 

            "client_cert": null, 

            "client_key": null, 

            "content": null, 

            "delimiter": null, 

            "dest": "filename", 

            "directory_mode": null, 

            "follow": false, 

            "force": true, 

            "force_basic_auth": true, 

            "group": null, 

            "headers": null, 

            "http_agent": "ansible-httpget", 

            "mode": null, 

            "owner": null, 

            "regexp": null, 

            "remote_src": null, 

            "selevel": null, 

            "serole": null, 

            "setype": null, 

            "seuser": null, 

            "sha256sum": "", 

            "src": null, 

            "timeout": 10, 

            "tmp_dest": null, 

            "unsafe_writes": null, 

            "url": "http://someurlhere/filename.txt", 

            "url_password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", 

            "url_username": "benny", 

            "use_proxy": true, 

            "validate_certs": false

        }

    }, 

    "msg": "Request failed", 

    "response": "HTTP Error 401: Unauthorized", 

    "state": "absent", 

    "status_code": 401, 

     "url": "http://someurlhere/filename.txt", 

}

My playbook is rather simple:

    - name: get the lockdown script

      get_url:                                        

        url: "http://someurlhere/filename.txt", 

        url_username: "{{ url_username }}"

        url_password: "{{ url_password }}"

        dest: filename

        force_basic_auth: yes

        force: yes

        validate_certs: no

      delegate_to: localhost

i can open the URL link from browser, and also tried using curl and it works (curl -u benny:mypassword -o /tmp/a.sh http://someurlhere/filename.txt)

Anyone has any idea what is the cause  ? 

beside, my password contains special character "$", i have tried to put them under double-quote in the url_password field as well, but didnt work.

Thank you

[Dick Visser]

I've seen instances where Sharepoint treated different clients
slightly different based on the user-agent string.
If you use "curl -A ansible-httpget .." to quickly rule that out.

Dick

> --
> You received this message because you are subscribed to the Google Groups
> "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ansible-proje...@googlegroups.com.
> To post to this group, send email to ansible...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ansible-project/06e36adb-f13c-4a76-847a-45bba63dee51%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Dick Visser
Trust & Identity Service Operations Manager
GÉANT

GÉANT Vereniging (Association) is registered with the Chamber of
Commerce in Amsterdam with registration number 40535155 and operates
in the UK as a branch of GÉANT Vereniging. Registered office:
Hoekenrode 3, 1102BR Amsterdam, The Netherlands. UK branch address:
City House, 126-130 Hills Road, Cambridge CB2 1PQ, UK.


Want to join us? We're hiring: https://www.geant.org/jobs

[Sosys]

Hi Dick,

thanks for the response.

adding  curl -A ansible-httpget will result in below:

 STATE: DO_DONE => WAITPERFORM handle 0x600057980; line 1795 (connection #0)

* STATE: WAITPERFORM => PERFORM handle 0x600057980; line 1811 (connection #0)

* HTTP 1.1 or later with persistent connection, pipelining supported

< HTTP/1.1 401 Unauthorized

* Server Microsoft-IIS/7.5 is not blacklisted

< Server: Microsoft-IIS/7.5

< SPRequestGuid: 024a709e-b9df-b0eb-e894-fb84e122f805

< request-id: 024a709e-b9df-b0eb-e894-fb84e122f805

< X-FRAME-OPTIONS: SAMEORIGIN

< SPRequestDuration: 10

< SPIisLatency: 0

< WWW-Authenticate: Negotiate

* NTLM handshake rejected

* Authentication problem. Ignoring this.

< WWW-Authenticate: NTLM

< X-Powered-By: ASP.NET

< MicrosoftSharePointTeamServices: 15.0.0.4569

< X-Content-Type-Options: nosniff

< X-MS-InvokeApp: 1; RequireReadOnly

< Date: Tue, 12 Jun 2018 05:23:54 GMT

< Content-Length: 0

Any way to bypass this in ansible get_url ?

Thank you

> email to ansible-project+unsubscribe@googlegroups.com.
> To post to this group, send email to ansible-project@googlegroups.com.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ansible-project/06e36adb-f13c-4a76-847a-45bba63dee51%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
Dick Visser
Trust & Identity Service Operations Manager
GÉANT

GÉANT Vereniging (Association) is registered with the Chamber of
Commerce in Amsterdam with registration number 40535155 and operates
in the UK as a branch of GÉANT Vereniging. Registered office:
Hoekenrode 3, 1102BR Amsterdam, The Netherlands. UK branch address:
City House, 126-130 Hills Road, Cambridge CB2 1PQ, UK.


Want to join us? We're hiring: https://www.geant.org/jobs

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscribe@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAL8fbwNaS8-9f0r1%2BGiAeLwB%3DTiyeMbeTeVbTD0yEGDLyhMvOg%40mail.gmail.com.

[Tony Chia]

Have you tried adding 
"force_basic_auth : yes" to your get_url task? 

[Sosys]

Yes, i tried to force:yes and force_basic_auth:yes

i guess the header agent was the issue, but not sure on how to overwrite it

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscribe@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/f34c4a31-e236-45a9-90e2-a29bfb94a7fb%40googlegroups.com.

[Kai Stian Olstad]

On 14.06.2018 05:03, Sosys wrote:
> Yes, i tried to force:yes and force_basic_auth:yes
> i guess the header agent was the issue, but not sure on how to
> overwrite it

You have two options.

User-Agent is request header information so could use
headers: "User-Agent:<a User-Agent string>"

According to the source code you can also use
http_agent: <a User-Agent string>

--
Kai Stian Olstad

[Sosys]

Hi Kai,

yes i have tried both that but no success

get_url:

  ....       

   http_agent: "Mozilla/5.0"

   headers: "User-Agent:Mozilla/5.0"

  ....

on the debug mode, i can see that http_agent has changed as per what i have key in, but still gotten 401 Unauthorized.

i tried to use both parameter OR either one, but none succeed.

i guess for now, curl would be the straight forward

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscribe@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/a59fa1e27f93b6ed2e986b96d39bfdbf%40olstad.com.

[Kai Stian Olstad]

On 14.06.2018 09:08, Sosys wrote:
> yes i have tried both that but no success
>
>
> get_url:
> ....
> http_agent: "Mozilla/5.0"
> headers: "User-Agent:Mozilla/5.0"
>
>

> on the debug mode, i can see that http_agent has changed as per what i
> have
> key in, but still gotten 401 Unauthorized.
> i tried to use both parameter OR either one, but none succeed.
>
>
> i guess for now, curl would be the straight forward

I can't help with Sharepoint since that is a product I don't use.

But if it's anal about the user-agent string you probably should use a
valid one(one that exist in a agent).
To my knowledge no agent is using just "Mozilla/5.0".

Why don't you try with the user-agent string that curl is using.

--
Kai Stian Olstad

[Sosys]

Hi Kai,

Thanks for the quick response.

i tried use the curl user-agent, which is curl/7.59.0

both also return 401 unauthorized.. .. flusterring...

--
Kai Stian Olstad

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscribe@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/ec79e25ee4cb48cabb7eecd1769ed961%40olstad.com.