ZAP and Angular

[ryerson...@gmail.com]

I have not jumped into this yet, I noticed this issue: https://groups.google.com/forum/#!topic/zaproxy-develop/34S3dOpuXXQ

and i was wondering if ZAP was capable of scanning websites based off of the angular js framework.

I am currently going over: http://www.slideshare.net/kevinhakanson/ng-owasp-ndc

I was wondering if there is a demo available or a more detailed guide?

Cheers and thank you!

[kingthorin+owaspzap]

You'll have to be more specific. We are not aware of anything that would prevent scanning of angular framework based sites.

Does ZAP have specific checks for things related to angular? Not that I know of....
The wappalyzer extension will recognize the technology and may alert on out-dated angular use.
ZAP does support single page apps.
etc.

[ryerson...@gmail.com]

Well, i was curious because these apps are heavily reliant on ajax, etc. The issue then becomes: how effective can zap discover and test each api. These api's are restful and exchange data in json. As far as i understand it, reflection attacks become impossible to detect using standard methods.

I am also wondering if there is any special setup that i need to do to setup ZAP for an angular app?

I'll try it out and get back on this if any issues arise :) If you have any guides for running ZAP on angular based applications, that would be a huge help.

[Simon Bennetts]

The more ZAP (or any other scanner) understands about your app the better.
I'd start by proxying manually through ZAP and then making sure its configured correctly for single page apps etc.
Then I'd recommend proxying regression tests through ZAP, followed by using both the traditional and Ajax spiders.
And try using the DOM XSS scanner in addition to the release and beta rules.

If you dont have comprehensive regression tests do you have any definition of your API?
At the moment we can only import WSDL, but it shouldnt be too hard to write a script to import WADL or OpenAPI...

And definitely let us know how you get on :)

Cheers,

Simon

[ryerson...@gmail.com]

Hi Simon,

We have 100s of functions. wsdl/wadl too complicated to be generated for our 100s of functions sadly.

I need a way to manually tell ZAP about REST services. Also, these are not fully REST compliant services. They only work on GET and POST methods. 

These take JSON and return JSON.  Need a way to take the output of a service and inspect it.

Cheers. Goran.

p.s. i can't find the DOM XSS scanner.

[Simon Bennetts]

Do you have any (functional) tests for these end points?
If so, proxy them through ZAP.
If not .. you might need to write some ;)

Once you have proxied them through ZAP then ZAP will be able to attack them - it understands JSON. I'd also enable 'URL Path' and 'HTTP Headers' active scan input vectors.

The DOM XSS scanner is on the ZAP Marketplace, its called "DOM XSS Active Scanner Rule" ;)
Its an alpha quality rule, so you might need to scroll down a bit.
Help page: https://github.com/zaproxy/zap-extensions/wiki/HelpAddonsDomxssDomxss

Cheers,

Simon

[ryerson...@gmail.com]

Great thank you! I'll have some time tomorrow to work on this, i'll let you know of the results :)

[ryerson...@gmail.com]

Definitely something wrong with the spider. I had to forcibly kill it after it only found 322 URIs after 12 hours. It found those 322 in minutes and spent the last hours and minutes stuck at 96 and 99% respectively.

The log shows:

96 to 99%

2016-02-12 13:32:01,107 [pool-2-thread-1] DEBUG header - << "[\r][\n]"

2016-02-12 16:58:14,194 [pool-2-thread-2] DEBUG header - >> "GET /url.css HTTP/1.1[\r][\n]"

stuck at 99% till i manually killed it.

2016-02-12 16:58:16,269 [pool-2-thread-2] DEBUG header - << "[\r][\n]"

2016-02-13 00:03:49,970 [AWT-EventQueue-0] INFO  Spider - Stopping spidering process by request.

a lot of hours of no activity. Going to try the active scan and ajax spider and give more feedback :) There was no error recorded in the log btw.

[ryerson...@gmail.com]

So , some feedback. I spent all weekend on this. The spider is not able to get past 99% after 23 hours. The active scan took 17 hours to complete!

The results are not promising at all. Not only that but it looks as though it failed to apply the DOM XSS scanner even though it is installed and set.

To give a more detailed description and to clarify some points.

Normally we have

1) a form action (url), method(post)

2) the actual form with all the fields, text box, etc and a submit button

3) then we have the returning page

From my understanding, this is how ZAP analyzes for vulnerabilities?

---

I might be wrong on this but it looks to me as if ZAP won't know what the submitted payload is. It doesn't know the members. All we get from a rest service is a confirmation of a success or not. For ZAP to see if say java script was injected successfully it would need to call all the api's (angular ajax service calls) and they would have to be in various configurations.

As an example: you'll have to call a certain api with a page 2 parameter. From ZAP's perspective, you don't know what page 2 is. It is all angular based but how do you know page 2 is even required?

This isn't just a simple structural modifier, i don't think but it's not just a page parameter but many different parameters that are used under certain conditions.

P.S. It would be a huge help if there was a guide or tutorial on how to scan an angular based website using rest services.

[Simon Bennetts]

OK, so thats not so great :/
I'd recommend looking at these things one at a time otherwise it gets a bit overwhelming, but I'll start by asking some general questions...

How large is your application? Eg rough number of unique urls, rough number of forms?
And how much of that is 'data driven' ie pages generated based on different data rather than different code?
Have you tried using the Ajax Spider?
Do you get any errors related to the DOM XSS scanner?
What happens if you try to run just the DOM XSS scanner on a singe page?
Which rules are taking the longest amount of time?

Cheers,

Simon

[ryerson...@gmail.com]

Hi Simon,

How large is your application? Eg rough number of unique urls, rough number of forms?

25-50 unique url's

And how much of that is 'data driven' ie pages generated based on different data rather than different code?

From our perspective, they are all different pages and zap should traverse each. It does not matter what is changing, to ZAP it should be unique and analyze the json out respone.

Have you tried using the Ajax Spider?

I am getting an java error, will update with more details. Waiting on the DOM XSS scan to finish, ZAP is lagging.

Do you get any errors related to the DOM XSS scanner?

Under policy > client browser > Cross site scripting (dom based), i assume this is it? No error's in the log, might not be properly documented. Not sure.

What happens if you try to run just the DOM XSS scanner on a singe page?

I am running this on it's own right now. But like when i ran the full scan, a lot of the requests in the active scan tab are coming back as 403 forbidden. Will update on this once the scan is complete on it's own.

Which rules are taking the longest amount of time?

Generally all the rules took 10 - 40 minutes each.

Cross Site Scripting (Persistent) - Spider took 120 minutes

Source code disclosure - SVN took 70 minutes

Script Active Scan Rules - took 10 minutes but cancelled itself.

Backup File Disclosure - took 101 minutes and i cancelled it.

Possible username enumeration - 0 minutes, cancelled itself

Then there are some rules who only took 1 second, making me wonder if they ran at all such as SOAP.

1160 minutes total for this scan.Roughly 40-50 rules in the active scan progress list.

[ryerson...@gmail.com]

Running the above mentioned rule, firefox opens a LOT but all of them result in:

Secure Connection Failed

2016-02-17 12:46:13,278 [ZAP-ActiveScanner-0] ERROR TestDomXSS - Error communicating with the remote browser. It may have died.

Build info: version: 'unknown', revision: 'unknown', time: 'unknown'

System info: host: 'mySystem', ip: 'xxx.xxx.xx.xx', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_71'

Driver info: driver.version: RemoteWebDriver

org.openqa.selenium.remote.UnreachableBrowserException: Error communicating with the remote browser. It may have died.

Build info: version: 'unknown', revision: 'unknown', time: 'unknown'

System info: host: 'mySystem', ip: 'xxx.xxx.xx.xx', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_71'

Driver info: driver.version: RemoteWebDriver

at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:641)

at org.openqa.selenium.remote.RemoteWebDriver$RemoteWebDriverOptions$RemoteTimeouts.pageLoadTimeout(RemoteWebDriver.java:841)

at org.zaproxy.zap.extension.domxss.TestDomXSS.getNewFirefoxDriver(TestDomXSS.java:207)

at org.zaproxy.zap.extension.domxss.TestDomXSS.getFirefoxDriver(TestDomXSS.java:226)

at org.zaproxy.zap.extension.domxss.TestDomXSS.scan(TestDomXSS.java:511)

at org.parosproxy.paros.core.scanner.AbstractPlugin.run(AbstractPlugin.java:282)

at java.lang.Thread.run(Unknown Source)

Caused by: org.apache.http.conn.HttpHostConnectException: Connect to 127.0.0.1:7056 [/127.0.0.1] failed: Connection refused: connect

at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)

at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)

at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)

at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)

at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)

at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)

at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)

at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)

at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:71)

at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)

at org.openqa.selenium.remote.internal.ApacheHttpClient.fallBackExecute(ApacheHttpClient.java:161)

at org.openqa.selenium.remote.internal.ApacheHttpClient.execute(ApacheHttpClient.java:89)

at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:142)

at org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.execute(NewProfileExtensionConnection.java:170)

at org.openqa.selenium.firefox.FirefoxDriver$LazyCommandExecutor.execute(FirefoxDriver.java:380)

at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:620)

... 6 more

Caused by: java.net.ConnectException: Connection refused: connect

at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)

at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)

at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)

at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)

at java.net.AbstractPlainSocketImpl.connect(Unknown Source)

at java.net.PlainSocketImpl.connect(Unknown Source)

at java.net.SocksSocketImpl.connect(Unknown Source)

at java.net.Socket.connect(Unknown Source)

at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74)

at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)

... 21 more

[ryerson...@gmail.com]

I watched this video related to the AJAX Spider: https://vimeo.com/101953742

My current issue is that the spider CANNOT progress to the next page of the form, it is stuck behind constraints because it has no idea how to proceed. Is there any way i can tell the ajax spider exactly what to enter as valid input for it to be able to continue? Or am i going about this the wrong way?

Cheers, Goran.

[ryerson...@gmail.com]

Single page dom scan:

2016-02-17 15:03:18,464 [Thread-451] INFO  HostProcess - Scanning 1 node(s) from https://url.ca

2016-02-17 15:03:18,464 [Thread-451] INFO  HostProcess - start host https://url.ca | TestDomXSS strength MEDIUM threshold MEDIUM

2016-02-17 15:03:18,464 [Thread-451] DEBUG HostProcess - traverse: plugin=Cross Site Scripting (DOM Based) url=https://url.ca/myapp/studentapplication

2016-02-17 15:03:18,464 [Thread-451] DEBUG HostProcess - scanSingleNode node plugin=Cross Site Scripting (DOM Based) node=https://url.ca/myapp/studentapplication

2016-02-17 15:04:16,531 [Thread-453] INFO  TestDomXSS - Reaper thread starting

2016-02-17 15:04:59,719 [Thread-451] INFO  HostProcess - completed host/plugin https://url.ca | TestDomXSS in 101.255s

2016-02-17 15:04:59,922 [Thread-451] INFO  HostProcess - completed host https://url.ca in 101.458s

2016-02-17 15:04:59,922 [Thread-450] INFO  Scanner - scanner completed in 101.475s

but firefox shows Secure Connection Failed and the active scan shows 0 requests.

[ryerson...@gmail.com]

So i have two websites to play with.

Website 1: very restrictive, page 1 must be completed before being able to progress to page 2 and you cannot go back.

Ajax spider is struggling with this one.

Website 2: Very open, you can do anything as long as the data is correct input, but pages are not restricted to be completed.

Testing website 2 with the ajax spider results in.

http://prntscr.com/a4llzo

If you look at the screenshot, location is an ajax page/tab. Where you can add locations to the list. There are other similar tabs and i tried to create a data driven node in this case with no success.

So what i assumed at this point is that i need to define a data driven node.

ZAP generated for https://mywebsite.ca/myapp/admin/)(.+?)(/.*)

but that group's EVERYTHING and i needed to be more specific for each page, so i tried the following:

https://mywebsite.ca/(myapp/admin/)(language)(/.*)

and got this:

http://prntscr.com/a4m29v

It looks like it recognizes it, but there are still 5 nodes in the tree, is this right?

[Simon Bennetts]

Replies inline - I'm reply to your later messages too, but one at a time ;)

On Wednesday, 17 February 2016 17:37:48 UTC, ryerson...@gmail.com wrote:

Hi Simon,

How large is your application? Eg rough number of unique urls, rough number of forms?

25-50 unique url's

Its really surprising that ZAP takes so long to spider this app, so somethings up.
I'm also having problems with the spider taking too long, but in my case it find well over 65,000 urls!
Low hundreds should really not be a problem.
 

And how much of that is 'data driven' ie pages generated based on different data rather than different code?

From our perspective, they are all different pages and zap should traverse each. It does not matter what is changing, to ZAP it should be unique and analyze the json out respone.

That makes things easier.
 

Have you tried using the Ajax Spider?

I am getting an java error, will update with more details. Waiting on the DOM XSS scan to finish, ZAP is lagging.

If the Ajax spider doesnt work then the DOM scan probably wont work either.
Both use Selenium, and it looks like takes failing somewhere (no idea where yet, could be Selenium code, our code, configs etc etc)
 

Do you get any errors related to the DOM XSS scanner?

Under policy > client browser > Cross site scripting (dom based), i assume this is it? No error's in the log, might not be properly documented. Not sure.

What happens if you try to run just the DOM XSS scanner on a singe page?

I am running this on it's own right now. But like when i ran the full scan, a lot of the requests in the active scan tab are coming back as 403 forbidden. Will update on this once the scan is complete on it's own.

Which rules are taking the longest amount of time?

Generally all the rules took 10 - 40 minutes each.

Cross Site Scripting (Persistent) - Spider took 120 minutes

Source code disclosure - SVN took 70 minutes

Script Active Scan Rules - took 10 minutes but cancelled itself.

Backup File Disclosure - took 101 minutes and i cancelled it.

Possible username enumeration - 0 minutes, cancelled itself

Then there are some rules who only took 1 second, making me wonder if they ran at all such as SOAP.

1160 minutes total for this scan.Roughly 40-50 rules in the active scan progress list.

These are really long times for such a small app :/
The time a scan takes is based on:

[Number pages] x [number parameters] x [number attacks] x [how long a request takes] / [number of threads]

You dont have many pages.
Do you have a large number of parameters? You can see these in the Params tab.
The number of attacks shouldnt be unusual.
So my guess is that it could well be due to the time a single request is taking...

When you're browsing the site does it respond quickly?
When you start the scan, how quickly do new requests appear in the Active Scan tab?
How many requests / second are you getting? You can work this out from the Scan Progress Dialog.
Could either the machine running the app or the one running ZAP be underpowered?
Can you try browsing the site while the ZAP scan is running - does it respond at the same rate?
Could there be something on the network rate limiting ZAP?
Have you changed any of the scan configs, in particular the 'Delay when scanning in milliseconds'?

Many thanks,

Simon

[ryerson...@gmail.com]

Do you have a large number of parameters? You can see these in the Params tab.

There are 11 text fields on the admin tab 1, the other's have maybe 3-4 text fields that just add to a list. The params tab only shows me a cookie param and 9 url param's.

When you're browsing the site does it respond quickly?

Instantaneous, from a user standpoint there is no delay. The entire website is rendered on the user side.

When you start the scan, how quickly do new requests appear in the Active Scan tab?

every second, there is a new request, the log shows multiple records at the same time as well. milliseconds apart. Responses are 1 second after the request.

How many requests / second are you getting? You can work this out from the Scan Progress Dialog.

Will work this out once i complete a scan again, i loaded a session and lost this data. So i won't have this till tomorrow.

Could either the machine running the app or the one running ZAP be underpowered?

i7 4790 @ 3.6GHz each, 16GB RAM with an SSD. Made this machine specifically for mobile automation and security automation. Have not been running mobile testing while using zap.

Can you try browsing the site while the ZAP scan is running - does it respond at the same rate?

Yes, no issues at all :)

Could there be something on the network rate limiting ZAP?

I don't think so, the other non angular applications (ones a lot larger than this) have no issues and they are on the same network.

Have you changed any of the scan configs, in particular the 'Delay when scanning in milliseconds'?

Only the strength and threshold, those are the two parameter's I've played with but for the scan in question, i had left everything on default.

[ryerson...@gmail.com]

I split off the question about data driven nodes to another post: https://groups.google.com/forum/#!topic/zaproxy-users/wRbXkXWOk1Y

I felt it deserve'd it's own attention because of how important DDN and structural modifiers are :)