Creating session using cURL while proxing to ZAP port

[skzaproxy]

Hi,

I have to test website on demand.

So I did following:

Case I:

1. I started zap daemon listening on port 8090

2. Accepted the login URL for the first website under test and created a login session using username and password while using proxy in curl as 8090 port.

3. Now performed spidering using ZAP API.

4. ZAP spider results only few login URL and it does not go beyond that though the session is active.

5. I want to have all URL for a particular website and pass same for active scan.

Please let me know what I am missing out and how to identify all URLs.

Case II: If I use ZAP GUI and configure IE to listen on port 8090 and then login to application, ZAP automatically identifies all the URLs.

Please let me know how to make case I work like case II. Why case II identifies more URL than case I.

Thanks in advance.

Regards

Supriya

[thc...@gmail.com]

Hi.

How are you starting the step 3? Which URL are you using as seed?

How are you setting the session as active?

Best regards.

On 24/08/16 13:33, skzaproxy wrote:
> Hi,
>
> I have to test website on demand.
> So I did following:

> *Case I*:

> 1. I started zap daemon listening on port 8090
> 2. Accepted the login URL for the first website under test and created a
> login session using username and password while using proxy in curl as
> 8090 port.
> 3. Now performed spidering using ZAP API.
> 4. ZAP spider results only few login URL and it does not go beyond that
> though the session is active.
> 5. I want to have all URL for a particular website and pass same for
> active scan.
>
> Please let me know what I am missing out and how to identify all URLs.
>

> *Case II*: If I use ZAP GUI and configure IE to listen on port 8090 and

> then login to application, ZAP automatically identifies all the URLs.
>

> Please let me know how to make *case I* work like *case II. Why case II
> identifies more URL than case I.*

>
>
> Thanks in advance.
>
> Regards
> Supriya
>

> --
> You received this message because you are subscribed to the Google
> Groups "OWASP ZAP User Group" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to zaproxy-user...@googlegroups.com
> <mailto:zaproxy-user...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/zaproxy-users/084c907d-2f6f-49f4-abea-148776d2f9f9%40googlegroups.com
> <https://groups.google.com/d/msgid/zaproxy-users/084c907d-2f6f-49f4-abea-148776d2f9f9%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

[skzaproxy]

My curl command for a single website is :

curl -k -v -X "POST" --data-binary {\"method\":\"login\",\"user_login\":\"admin\",\"password\":\"admin456\"} "https://1x.1xx.1xx.2x/json/login" --proxy "http://localhost:8091"

This command returns me output like this:

Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying ::1...
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8091 (#0)
* Establish HTTP proxy tunnel to 15.154.125.3:443
> CONNECT 1x.1xx.1xx.2x:443 HTTP/1.1
> Host: 1x.1xx.1xx.2x:443
> User-Agent: curl/7.46.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
< Proxy-connection: Keep-alive
<
* Proxy replied OK to CONNECT request
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*        subject: CN=15.154.125.3; OU=Zed Attack Proxy Project; O=OWASP; C=xx; e
mailAddress=owasp-zed-a...@lists.owasp.org
*        start date: Jul 25 14:12:08 2016 GMT
*        expire date: Nov 10 14:12:08 2024 GMT
*        issuer: CN=OWASP Zed Attack Proxy Root CA; L=36edd771394cd10a; O=OWASP
Root CA; OU=OWASP ZAP Root CA; C=xx
*        SSL certificate verify result: self signed certificate in certificate c
hain (19), continuing anyway.
> POST /json/login HTTP/1.1
> Host: 1x.1xx.1xx.2x
> User-Agent: curl/7.46.0
> Accept: */*
> Content-Length: 61
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 61 out of 61 bytes
< HTTP/1.1 200 OK
< Content-Type: application/x-javascript
< Content-Length: 299
< Cache-Control: no-cache
< Date: Wed, 24 Aug 2016 09:06:46 GMT
< Server: ABC
< X-Frame-Options: sameorigin
< Set-Cookie: sessionKey=db5583960e94c556bb1c8ed538d320a9; path=/; secure
<
{"session_key":"db5583960e94c556bb1c8ed538d320a9","user_name":"admin","user_acco
unt":"admin","user_dn":"","user_type":"Local","user_ip":"1x.1xx.1xx.1x","user_e
xpires":"Wed Aug 24 09:11:46 2016","login_priv":1,"virtual_
mediapriv":2,"resetpriv":3}* Connection #0 to ho
st localhost left intact

After this when I do spidering using https://1x.1xx.1xx.2x.* included in default context I get

Only URLs that are browsed are just few like https://https://1x.1xx.1xx.2x/json/login , https://1x.1xx.1xx.2x/js, https://1x.1xx.1xx.2x/css". etc

[skzaproxy]

Number of links identified using ZAP are very less when user logs in to the product/website under test via an interface other than browser?

Hi All,

I have following test scenario:

1. Type the website IP on Chrome browser (browser proxying to port 8091 and ZAP listening on port 8091) and perform login to website using username and password:

ZAP shows following links under json automatically without any interaction:
737 Mon Aug 29 16:46:38 IST 2016 POST https://1y.1yy.1zz.1a/json/Login 200 OK 110 298 Low false SetCookie
740 Mon Aug 29 16:46:38 IST 2016 GET https://1y.1yy.1zz.1a/sse/gui 200 OK 29 0 Low false 
742 Mon Aug 29 16:46:38 IST 2016 GET https://1y.1yy.1zz.1a/html/application.html 200 OK 87 30429 Low false Script, Comment
744 Mon Aug 29 16:46:38 IST 2016 GET https://1y.1yy.1zz.1a/css/jquery-gui.css 200 OK 77 33620 Low false Comment
746 Mon Aug 29 16:46:38 IST 2016 GET https://1y.1yy.1zz.1a/css/eov.css 200 OK 135 38643 Low false Password, Upload
747 Mon Aug 29 16:46:38 IST 2016 GET https://1y.1yy.1zz.1a/css/application.css 200 OK 134 5316 Low false 
748 Mon Aug 29 16:46:38 IST 2016 GET https://1y.1yy.1zz.1a/alt/css/style.css 200 OK 251 145636 Low false Password, Upload, Comment
751 Mon Aug 29 16:46:38 IST 2016 GET https://1y.1yy.1zz.1a/js/json2.js 200 OK 267 4079 Low false 
753 Mon Aug 29 16:46:38 IST 2016 GET https://1y.1yy.1zz.1a/js/jquery.js 200 OK 220 299 Low false Script
754 Mon Aug 29 16:46:38 IST 2016 GET https://1y.1yy.1zz.1a/js/jquery-gui.js 200 OK 622 235096 Low false Comment
755 Mon Aug 29 16:46:39 IST 2016 GET https://1y.1yy.1zz.1a/js/jquery-2.1.4.js 200 OK 666 162920 Low false Script, Comment
756 Mon Aug 29 16:46:39 IST 2016 GET https://1y.1yy.1zz.1a/js/product.js 200 OK 1074 99956 Low false Comment
759 Mon Aug 29 16:46:39 IST 2016 GET https://1y.1yy.1zz.1a/js/hostPwr.js 200 OK 1144 1774 Low false 
760 Mon Aug 29 16:46:40 IST 2016 GET https://1y.1yy.1zz.1a/html/masthead.html 200 OK 27 3944 Low false Script, Comment
762 Mon Aug 29 16:46:40 IST 2016 GET https://1y.1yy.1zz.1a/json/host_power?_=1472469399961 200 OK 98 68 Low false 
763 Mon Aug 29 16:46:40 IST 2016 GET https://1y.1yy.1zz.1a/css/jquery-gui.css 304 Not Modified 80 0 Low false 
764 Mon Aug 29 16:46:40 IST 2016 GET https://1y.1yy.1zz.1a/css/eov.css 304 Not Modified 59 0 Low false 
765 Mon Aug 29 16:46:40 IST 2016 GET https://1y.1yy.1zz.1a/js/json2.js 304 Not Modified 179 0 Low false 
766 Mon Aug 29 16:46:40 IST 2016 GET https://1y.1yy.1zz.1a/alt/css/style.css 304 Not Modified 205 0 Low false 
767 Mon Aug 29 16:46:40 IST 2016 GET https://1y.1yy.1zz.1a/js/jquery.js 304 Not Modified 21 0 Low false 
768 Mon Aug 29 16:46:40 IST 2016 GET https://1y.1yy.1zz.1a/js/jquery-gui.js 304 Not Modified 46 0 Low false 
769 Mon Aug 29 16:46:40 IST 2016 GET https://1y.1yy.1zz.1a/js/product.js 304 Not Modified 46 0 Low false 
770 Mon Aug 29 16:46:40 IST 2016 GET https://1y.1yy.1zz.1a/js/jquery-2.1.4.js 304 Not Modified 28 0 Low false 
771 Mon Aug 29 16:46:40 IST 2016 GET https://1y.1yy.1zz.1a/js/global.js 200 OK 119 29891 Low false 
774 Mon Aug 29 16:46:40 IST 2016 GET https://1y.1yy.1zz.1a/json/masthead?_=1472469400619 200 OK 117 521 Low false 
775 Mon Aug 29 16:46:40 IST 2016 GET https://1y.1yy.1zz.1a/json/session_info?_=1472469399962 200 OK 209 249 Low false 
780 Mon Aug 29 16:46:40 IST 2016 GET https://1y.1yy.1zz.1a/json/login_session?_=1472469399963 200 OK 259 1027 Low false 
781 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/json/health_summary?_=1472469399965 200 OK 234 488 Low false 
782 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/json/host_power?_=1472469399964 200 OK 282 68 Low false 
783 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/lang/en/strings.js?_=1472469399966 200 OK 314 163203 High false 
787 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/html/content.html 200 OK 44 7611 Low false Script, Comment
788 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/html/navigation.html 200 OK 68 3126 Low false Script, Comment
791 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/css/jquery-gui.css 304 Not Modified 65 0 Low false 
793 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/css/eov.css 304 Not Modified 127 0 Low false 
794 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/alt/css/style.css 304 Not Modified 120 0 Low false 
795 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/js/json2.js 304 Not Modified 90 0 Low false 
796 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/json/guid_status?_=1472469399967 200 OK 236 20 Low false 
797 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/js/jquery.js 304 Not Modified 54 0 Low false 
798 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/js/tree.js 200 OK 39 7794 Low false 
799 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/js/jquery-gui.js 304 Not Modified 93 0 Low false 
800 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/js/product.js 304 Not Modified 95 0 Low false 
801 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/js/jquery-2.1.4.js 304 Not Modified 49 0 Low false 
802 Mon Aug 29 16:46:41 IST 2016 GET https://1y.1yy.1zz.1a/css/hp-piano.css 200 OK 145 15002 Low false 
804 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/html/summary.html 200 OK 56 20990 Low false Script, Comment
805 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/js/jquery-2.1.4.js 304 Not Modified 57 0 Low false 
806 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/js/nav.json?_=1472469402198 200 OK 34 8228 Low false JSON
807 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/js/jquery-gui.js 304 Not Modified 18 0 Low false 
808 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/js/json2.js 304 Not Modified 47 0 Low false 
809 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/css/eov.css 304 Not Modified 78 0 Low false 
810 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/js/product.js 304 Not Modified 67 0 Low false 
811 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/js/jquery.js 304 Not Modified 118 0 Low false 
812 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/js/jquery.tablesorter.min2.js 200 OK 132 34552 Low false Comment
813 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/js/hostPwr.js 304 Not Modified 129 0 Low false 
814 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/html/summary.html 304 Not Modified 110 0 Low false 
815 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/css/jquery-gui.css 304 Not Modified 247 0 Low false 
816 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/alt/css/style.css 304 Not Modified 256 0 Low false 
819 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/alt/css/style.css 200 OK 265 145636 Low false Password, Upload, Comment
820 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/css/jquery-gui.css 304 Not Modified 282 0 Low false 
821 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/js/jquery-2.1.4.js 304 Not Modified 21 0 Low false 
822 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/js/FileSaver.min.js?_=1472469402881 200 OK 31 3249 Low false Comment
823 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/html/jnlp_template.html?_=1472469402884 200 OK 54 1041 Low false Script
824 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/json/host_power?_=1472469402880 200 OK 107 68 Low false 
825 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/json/active_sessions?_=1472469402883 200 OK 132 101 Low false 
826 Mon Aug 29 16:46:42 IST 2016 GET https://1y.1yy.1zz.1a/json/overview?_=1472469402882 200 OK 230 827 Low false 
829 Mon Aug 29 16:46:43 IST 2016 GET https://1y.1yy.1zz.1a/sse/gui 200 OK 29 0 Low false 
830 Mon Aug 29 16:46:49 IST 2016 GET https://1y.1yy.1zz.1a/sse/gui 200 OK 44 0 Low false 
831 Mon Aug 29 16:46:54 IST 2016 GET https://1y.1yy.1zz.1a/sse/gui 200 OK 42 0 Low false 
832 Mon Aug 29 16:46:59 IST 2016 GET https://1y.1yy.1zz.1a/sse/gui 200 OK 43 0 Low false 
834 Mon Aug 29 16:47:04 IST 2016 GET https://1y.1yy.1zz.1a/sse/gui 200 OK 43 0 Low false 
835 Mon Aug 29 16:47:10 IST 2016 GET https://1y.1yy.1zz.1a/sse/gui 200 OK 43 0 Low false 
836 Mon Aug 29 16:47:15 IST 2016 GET https://1y.1yy.1zz.1a/sse/gui 200 OK 365 0 Low false 
837 Mon Aug 29 16:47:20 IST 2016 GET https://1y.1yy.1zz.1a/sse/gui 200 OK 41 0 Low false 
838 Mon Aug 29 16:47:26 IST 2016 GET https://1y.1yy.1zz.1a/sse/gui 200 OK 41 0 Low false 
839 Mon Aug 29 16:47:31 IST 2016 GET https://1y.1yy.1zz.1a/sse/gui 200 OK 52 0 Low false 
840 Mon Aug 29 16:47:36 IST 2016 GET https://1y.1yy.1zz.1a/sse/gui 200 OK 40 0 Low false 
843 Mon Aug 29 16:47:42 IST 2016 GET https://1y.1yy.1zz.1a/sse/gui 200 OK 48 0 Low false 
844 Mon Aug 29 16:47:47 IST 2016 GET https://1y.1yy.1zz.1a/sse/gui 200 OK 48 0 Low false 

2. I type following command in curl and using Phantom js, but it shows only 1 link under json in ZAP.
ZAP does not identify all links by itself as it does when I login using browser.

Curl command:
C:\Program Files\cURL\bin>curl --insecure -v "https://1y.1yy.1zz.1a/json/Login" -H "Cookie: sessionUrl=https"%"253A"%"2F"%"2F15.154.125.3"%"2F; sessionLa
ng=en" -H "Origin: https://1y.1yy.1zz.1a" -H "Accept-Encoding: gzip, deflate, br"
 -H "Accept-Language: en-US,en;q=0.8" -H "User-Agent: Mozilla/5.0 (Windows NT 6.
1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537
.36" -H "Content-Type: application/json; charset=UTF-8" -H "Accept: application/
json, text/javascript, */*; q=0.01" -H "Referer: https://1y.1yy.1zz.1a/html/login
.html" -H "X-Requested-With: XMLHttpRequest" -H "Connection: keep-alive" --data-
binary "{""method"":""login"",""user_login"":""admin"",""password"":""admin123""
}" --compressed -proxy --proxy "http://localhost:8091"

ZAP Output:
POST https://1y.1yy.1zz.1a/json/Login 200 OK

Please help me how to simulate the way ZAP shows when I login to a website using browser and when I login using Phantom js or curl.
Or is there any other way to simulate browser behavior from command prompt or API or any other command line webkit.

Since I have to itegrate ZAP to do security testing by just taking any website IP and username and password, I cannot ask user to login to their product manually as par of business requirement.

 

On Wednesday, August 24, 2016 at 6:03:01 PM UTC+5:30, skzaproxy wrote:

[kingthorin+owaspzap]

cURL did exactly and precisely what you asked it to do.

[skzaproxy]

Hi,

Thanks for you reply.  I understand, curl performed as expected, but when same login happens via browser than zap identifies more link automatically but same login happens via curl for phantom js, ZAP identifies lesser link, so just wanted to understand, what I am missing that number of links identified by zap for same website and for same login link are different.

On Wednesday, August 24, 2016 at 6:03:01 PM UTC+5:30, skzaproxy wrote:

[kingthorin+owaspzap]

cURL is not a browser,  it doesn't execute javascript and doesn't download all resources needed for rendering/interacting with a response the way a browser would.

I can't speak for PhantomJS based on the provided details, however my guess is that it's not doing all the same things your browser does.

Comparing a browser to not a browser is very much like comparing apples and oranges.

[skzaproxy]

Thanks again.

In case you can suggest any other way to identify more link via zap.

Due to the constraint from my end I cannot use browser GUI, so it there any other way I can launch my website and still zap discovers all the required links for testing using ZAP API.

On Wednesday, August 24, 2016 at 6:03:01 PM UTC+5:30, skzaproxy wrote:

[kingthorin+owaspzap]

Well until https://github.com/zaproxy/zaproxy/issues/2439 is fixed you won't be able to authenticate with a JSON object. Not simply anyway.

You could probably setup Scripted authentication:

https://github.com/zaproxy/zap-core-help/wiki/HelpStartConceptsAuthentication#script-based-authentication

https://github.com/zaproxy/community-scripts/tree/master/authentication

https://www.coveros.com/scripting-authenticated-login-within-zap-vulnerability-scanner/

If you get script authentication setup and working properly you can then do authenticated spidering with the traditional spider, and if you use forced user mode the ajax spider as well.