ZAP Authentication in jenkins plugin

[Thilina Madhusanka]

Hi

I have modified the existing [1]  zaproxy jenkins plugin to use authentication .

any one want to test it works and do any suggestions or add methods or do modification can get it [2]. 

this plugin works with the new zap weekly relese [3] (recommend to use this, im using it to work with this)

[1] - https://github.com/jenkinsci/zaproxy-plugin

[2] - https://github.com/thilina27/zapauthentication

[3] - https://github.com/zaproxy/zaproxy/wiki/Downloads#zap-weekly

[thc...@gmail.com]

Hi.

Thanks for sharing!

CC'ing dev mailing list in case someone interested is not on the users list.

Best regards.

> --
> You received this message because you are subscribed to the Google
> Groups "OWASP ZAP User Group" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to zaproxy-user...@googlegroups.com
> <mailto:zaproxy-user...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

[ryerson...@gmail.com]

I downloaded the files off of github but i cannot find the .hpi file, is there a particular install method that i need to do? Thank you!

[Thilina Madhusanka]

Hi

You have to buid it to get the .hpi file. 

New version of the plugin [1] will be soon available to download. 

Ill guide you through the process. 

Download source code in [1] and extract it to some directory 

Then navigate to that dir using terminal

then typ mvn install 

it will take like 2 mins and then in that dir there will be a directory named target. in side you will find the .hpi file. 

you need to have maven and maven path must be set. let me know if you need any help. 

cheers

[Thilina Madhusanka]

Hi 

forget to add the link. 

[1] - https://github.com/jenkinsci/zaproxy-plugin

[ryerson...@gmail.com]

Greatly appreciated, thank you! Going to try it right now.

[ryerson...@gmail.com]

I got the .hpi file. A couple questions if you don't mind.

In Jenkins, Manage Jenkins > Configure System

1) Do i need to have anything for the Maven Configurations or Maven Installations setup?
2) When adding Custom tool installation for Zap Proxy Plugin, would you mind posting a screenshot of your config settings please. The server is rejecting the connection to the tar.gz file because it no longer exists at the specified url.
3) Where do i enter the username and password for the site as well as it's method of authentication?

Really, thank you for taking the time to help, teach. I have been looking for the last couple days and couldn't make much progress.

Cheers.

[Thilina Madhusanka Perera]

Hi

Im same Thilina. 

Now the new plugin has released so you can directly get it from Jenkins.

Here [1] i have also added the configuration details 

Sorry about the late reply. If you need any help ill be happy to provide it

[1] - https://wiki.jenkins-ci.org/display/JENKINS/ZAProxy+Plugin

Cheers

[ryerson...@gmail.com]

Don't apologize, you're helping me! Thank you. I followed the wiki setup page and i still get errors. I do not have my pc in front of me but I'll post some of the errors if you don't mind and hopefully you'll be able to shed some light on the issue.

P.S. The .tar.gz file in the links under Custom Tools is a dead link (file does not exist).

[ryerson...@gmail.com]

“zaproxy-plugin” configuration in jobs: ZAPROXY_HOME, where is that set and what is it actually pointing to?

[Thilina Madhusanka]

HI

I have set up ZAPROXY_HOME="path to may zap directory" , in  /etc/ environment file witch has path variable setup

Put ZAPROXY_HOME in your etc/environment file like as ZAPROXY_HOME="YOur zap directory path" 

cheers

[ryerson...@gmail.com]

Deleted my previous posts, i figured those issues out.  This is a piece of the console log, i can post the full log if needed but do you have nay idea how to fix these issues. Or are they even issues at all?

Status spider = running
Alerts number =         ApiResponseElement numberOfAlerts = 210

92344 [pool-2-thread-2] INFO com.crawljax.core.state.StateMachine  - State state16 added to the StateMachine.
92344 [pool-2-thread-2] INFO com.crawljax.core.Crawler  - New DOM is a new state! crawl depth is now 1
92390 [pool-2-thread-2] INFO com.crawljax.browser.WebDriverBackedEmbeddedBrowser  - Closing the browser...
92390 [Thread-8] INFO com.crawljax.core.CrawlController  - Received shutdown notice. Reason is Exausted
92529 [pool-2-thread-2] WARN org.openqa.selenium.os.ProcessUtils  - Process refused to die after 10 seconds, and couldn't taskkill it
org.openqa.selenium.WebDriverException: java.lang.InterruptedException: sleep interrupted
Build info: version: 'unknown', revision: 'unknown', time: 'unknown'
System info: host: 'testMachine', ip: 'xxx.xxx.xx.xx', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_45'
Driver info: driver.version: RemoteWebDriver
    at org.openqa.selenium.os.CommandLine.waitFor(CommandLine.java:123)
    at org.openqa.selenium.os.CommandLine.execute(CommandLine.java:116)
    at org.openqa.selenium.os.WindowsUtils.killPID(WindowsUtils.java:173)
    at org.openqa.selenium.os.ProcessUtils.killWinProcess(ProcessUtils.java:138)
    at org.openqa.selenium.os.ProcessUtils.killProcess(ProcessUtils.java:81)
    at org.openqa.selenium.os.UnixProcess$SeleniumWatchDog.destroyHarder(UnixProcess.java:247)
    at org.openqa.selenium.os.UnixProcess$SeleniumWatchDog.access$2(UnixProcess.java:246)
    at org.openqa.selenium.os.UnixProcess.destroy(UnixProcess.java:125)
    at org.openqa.selenium.os.CommandLine.destroy(CommandLine.java:153)
    at org.openqa.selenium.firefox.FirefoxBinary.quit(FirefoxBinary.java:259)
    at org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.quit(NewProfileExtensionConnection.java:204)
    at org.openqa.selenium.firefox.FirefoxDriver$LazyCommandExecutor.quit(FirefoxDriver.java:377)
    at org.openqa.selenium.firefox.FirefoxDriver.stopClient(FirefoxDriver.java:323)
    at org.openqa.selenium.remote.RemoteWebDriver.quit(RemoteWebDriver.java:467)
    at com.crawljax.browser.WebDriverBackedEmbeddedBrowser.close(WebDriverBackedEmbeddedBrowser.java:328)
    at com.crawljax.core.Crawler.close(Crawler.java:94)
    at com.crawljax.core.CrawlTaskConsumer.call(CrawlTaskConsumer.java:50)
    at com.crawljax.core.CrawlTaskConsumer.call(CrawlTaskConsumer.java:16)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.InterruptedException: sleep interrupted
    at java.lang.Thread.sleep(Native Method)
    at org.apache.commons.exec.DefaultExecuteResultHandler.waitFor(DefaultExecuteResultHandler.java:121)
    at org.openqa.selenium.os.UnixProcess.waitFor(UnixProcess.java:139)
    at org.openqa.selenium.os.CommandLine.waitFor(CommandLine.java:121)
    ... 21 more
92545 [pool-2-thread-2] ERROR com.crawljax.core.CrawlTaskConsumer  - Unexpected error Process refused to die after 10 seconds, and couldn't taskkill it: java.lang.InterruptedException: sleep interrupted
Build info: version: 'unknown', revision: 'unknown', time: 'unknown'
System info: host: 'testMachine', ip: 'xxx.xxx.xx.xx', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_45'
Driver info: driver.version: RemoteWebDriver
java.lang.RuntimeException: Process refused to die after 10 seconds, and couldn't taskkill it: java.lang.InterruptedException: sleep interrupted
Build info: version: 'unknown', revision: 'unknown', time: 'unknown'
System info: host: 'testMachine', ip: 'xxx.xxx.xx.xx', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_45'
Driver info: driver.version: RemoteWebDriver
    at org.openqa.selenium.os.ProcessUtils.killWinProcess(ProcessUtils.java:142)
    at org.openqa.selenium.os.ProcessUtils.killProcess(ProcessUtils.java:81)
    at org.openqa.selenium.os.UnixProcess$SeleniumWatchDog.destroyHarder(UnixProcess.java:247)
    at org.openqa.selenium.os.UnixProcess$SeleniumWatchDog.access$2(UnixProcess.java:246)
    at org.openqa.selenium.os.UnixProcess.destroy(UnixProcess.java:125)
    at org.openqa.selenium.os.CommandLine.destroy(CommandLine.java:153)
    at org.openqa.selenium.firefox.FirefoxBinary.quit(FirefoxBinary.java:259)
    at org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.quit(NewProfileExtensionConnection.java:204)
    at org.openqa.selenium.firefox.FirefoxDriver$LazyCommandExecutor.quit(FirefoxDriver.java:377)
    at org.openqa.selenium.firefox.FirefoxDriver.stopClient(FirefoxDriver.java:323)
    at org.openqa.selenium.remote.RemoteWebDriver.quit(RemoteWebDriver.java:467)
    at com.crawljax.browser.WebDriverBackedEmbeddedBrowser.close(WebDriverBackedEmbeddedBrowser.java:328)
    at com.crawljax.core.Crawler.close(Crawler.java:94)
    at com.crawljax.core.CrawlTaskConsumer.call(CrawlTaskConsumer.java:50)
    at com.crawljax.core.CrawlTaskConsumer.call(CrawlTaskConsumer.java:16)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: org.openqa.selenium.WebDriverException: java.lang.InterruptedException: sleep interrupted
Build info: version: 'unknown', revision: 'unknown', time: 'unknown'
System info: host: 'testMachine', ip: 'xxx.xxx.xx.xx', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_45'
Driver info: driver.version: RemoteWebDriver
    at org.openqa.selenium.os.CommandLine.waitFor(CommandLine.java:123)
    at org.openqa.selenium.os.CommandLine.execute(CommandLine.java:116)
    at org.openqa.selenium.os.WindowsUtils.killPID(WindowsUtils.java:173)
    at org.openqa.selenium.os.ProcessUtils.killWinProcess(ProcessUtils.java:138)
    ... 18 more
Caused by: java.lang.InterruptedException: sleep interrupted
    at java.lang.Thread.sleep(Native Method)
    at org.apache.commons.exec.DefaultExecuteResultHandler.waitFor(DefaultExecuteResultHandler.java:121)
    at org.openqa.selenium.os.UnixProcess.waitFor(UnixProcess.java:139)
    at org.openqa.selenium.os.CommandLine.waitFor(CommandLine.java:121)
    ... 21 more
92547 [Thread-8] INFO com.crawljax.core.CrawlController  - Shutdown process complete
92547 [Thread-8] INFO org.zaproxy.zap.extension.spiderAjax.SpiderThread  - Stopping proxy...
92647 [Thread-8] INFO org.zaproxy.zap.extension.spiderAjax.SpiderThread  - Proxy stopped.

[Thilina Madhusanka]

hi

I dont think these issue are from zaproxy plugin. 

Your using newest version of plugin and the ZAP application ? 

I think you should ask this one also from some one in ZAP. i cant seem to figure out the problem.

Is the report generate successfully ?

  

Cheers. 

[thc...@gmail.com]

Hi.

Indeed, that's not related to the plugin.

The AJAX Spider seems to be failing to close the browser windows. If the
windows are not being closed you would have to close them manually
(otherwise you would end up with a lot of them).

Which Firefox version are you using? That shouldn't be happening with
the latest versions.

Best regards.

> <http://os.name>: 'Windows 7', os.arch: 'amd64', os.version: '6.1',

> <http://os.name>: 'Windows 7', os.arch: 'amd64', os.version: '6.1',

> java.version: '1.8.0_45'
> Driver info: driver.version: RemoteWebDriver
> java.lang.RuntimeException: Process refused to die after 10 seconds,
> and couldn't taskkill it: java.lang.InterruptedException: sleep
> interrupted
> Build info: version: 'unknown', revision: 'unknown', time: 'unknown'
> System info: host: 'testMachine', ip: 'xxx.xxx.xx.xx', os.name

> <http://os.name>: 'Windows 7', os.arch: 'amd64', os.version: '6.1',

> <http://os.name>: 'Windows 7', os.arch: 'amd64', os.version: '6.1',

[ryerson...@gmail.com]

Amazing! Thank you both so much :)

Updated: Plugin is 1.1.7 that i got off of the jenkins list.
Updated: ZAP was 2.4.2 (2015-09-07) but updated just now to the 2.4.2 weekly release (2015-09-21) which is giving me a different error.

Updated: Firefox was at 39 but updated now to 41 which seems to have solved the issue, the report is being generated and it is a thing of beauty.

The Spider URL As User is to have authentication for the active scan as well? My website uses a randomly generated unique token on page load to prevent cross site scripting (it is posted along with other parameters and not just the username and password), is there any way to get that token and post it as a parameter?

[ryerson...@gmail.com]

Just to elaborate a bit more and step back a little bit from the token

Target URL: https://dev.test.cf.mytestsite.ca/humanrights/
but then my login URL would be the same as my target URL?

because https://dev.test.cf.mytestsite.ca/ can have multiple pages under it such as

https://dev.test.cf.mytestsite.ca/abcpage

https://dev.test.cf.mytestsite.ca/defpafe

which i don't want to be checked.

My logout button is

<div style="text-align:right">
	<span style="..."><a href="/humanrights/?logout=UNIQUE_TOKEN"><img style="..." src="/humanrights/icons/log_out.png" />Logout</a></span>
</div>

I was looking at \\Q>ahref=\"../admin/logout_action.jsp\">Sign-out</a>\\E

What are the Q and E, regex?

[ryerson...@gmail.com]

I've been playing around with it and i'm trying to figure out if it logged me in or not, not sure how to tell. Also the report's the plugin is generating is different than when i do a manual zap spider + active scan. The plugin reports less issues for some reason.

[ryerson...@gmail.com]

SQl Injection Errors are the cause of the discrepancy. I am unsure of why they are being caused thought. I have everything at the most recent version.

http://pastebin.com/cMsWcnBu

[Thilina Madhusanka]

Hi

dose reports mismatch a lot? 

All the URL's in the ZAP report are your website?

What kind of test result are missing? 

I think You should seek help from some one in ZAP  about the SQL error. 

Cheers.  

[kingthorin+owaspzap]

java.net.SocketException: Connection reset

Can be caused by a lot of things. Back timeout, firewall, IPS, overlong URL, etc. In order to troubleshoot we'd need more detail.

[kingthorin+owaspzap]

That should have been "bad timeout" not "back".....

[ryerson...@gmail.com]

Thank you for the help, I will post both the ZAP plugin and the tool reports later today after they finish running.

[ryerson...@gmail.com]

What details can i contribute?

[Simon Bennetts]

Yes, there are a load of reasons why connections can fail, with application failure and IDSs being pretty common :)
I wouldnt worry about these sort of errors too much unless you see loads of them and/or your application stops responding.
I'm actually wondering if maybe we shouldnt log these as errors but instead show a count of failed requests that is shown in the Active Scan Progress Dialog.
That could be more useful..

Cheers,

Simon

[ryerson...@gmail.com]

I just found out that my manual test is not going through the entire site, will update this thread with more details and full reports once i am able to generate the full security report.

[Albert]

I am exactly in the same stage. I got zap poxy plugin running and being able to scan a local Webgoat running site. 

I am trying to figure out if the Spider As User is actually login in. The result in the log seems to show a positive login but the spider ends at 0%.

Did you get any further on how to debug if login is happening? Or what actually needs to be set in the jerkins admin page? Attached my current configuration screenshot.

This my my current log:

13164 [Thread-5] INFO org.zaproxy.zap.spider.Spider  - Starting spider...
13190 [Thread-5] INFO org.zaproxy.zap.spider.Spider  - Adding seed for spider: http://localhost:8080/WebGoat/login.mvc
13216 [Thread-5] INFO org.zaproxy.zap.spider.Spider  - Adding seed for spider: http://localhost:8080/robots.txt
13223 [Thread-5] INFO org.zaproxy.zap.spider.Spider  - Adding seed for spider: http://localhost:8080/sitemap.xml
Status spider = 66%
Alerts number = 		ApiResponseElement numberOfAlerts = 0

15508 [pool-1-thread-2] WARN org.zaproxy.zap.spider.URLCanonicalizer  - Error while Processing URL in the spidering process (on base ): Host could not be reliably evaluated from: http://getbootstrap.com)
Status spider = 45%
Alerts number = 		ApiResponseElement numberOfAlerts = 0

16120 [pool-1-thread-1] INFO org.zaproxy.zap.spider.Spider  - Spidering process is complete. Shutting down...
16129 [Thread-7] INFO org.zaproxy.zap.extension.spider.SpiderThread  - Spider scanning complete: true
Skip Ajax spidering the site [http://localhost:8080/WebGoat/login.mvc]
Setting up Authentication
URL http://localhost:8080/WebGoat/login.mvc added to Context [2]
Form Based Authentication added to context
Logged in indicator <a\s+(?:[^>]*?\s+)?href="([^"]*)" added to context 
New user added. username :guest
User : guest is now Enabled
Spider the site [http://localhost:8080/WebGoat/login.mvc] as user [guest]
18138 [Thread-8] INFO org.zaproxy.zap.extension.spider.SpiderThread  - Starting spidering scan on SpiderApi-1 at Fri Dec 04 13:28:09 CET 2015
18144 [Thread-8] INFO org.zaproxy.zap.spider.Spider  - Spider initializing...
18145 [Thread-8] INFO org.zaproxy.zap.spider.Spider  - Starting spider...
18145 [Thread-8] INFO org.zaproxy.zap.spider.Spider  - Scan will be performed from the point of view of User: guest
18145 [Thread-8] INFO org.zaproxy.zap.spider.Spider  - Adding seed for spider: http://localhost:8080/WebGoat/login.mvc
18195 [Thread-8] INFO org.zaproxy.zap.spider.Spider  - Adding seed for spider: http://localhost:8080/robots.txt
18197 [Thread-8] INFO org.zaproxy.zap.spider.Spider  - Adding seed for spider: http://localhost:8080/sitemap.xml
18199 [pool-2-thread-1] INFO org.zaproxy.zap.users.User  - Authenticating user: guest
Status spider = 0%
Alerts number = 		ApiResponseElement numberOfAlerts = 0

19111 [pool-2-thread-2] INFO org.zaproxy.zap.users.User  - Authenticating user: guest
19713 [pool-2-thread-2] INFO org.zaproxy.zap.spider.Spider  - Spidering process is complete. Shutting down...
19768 [Thread-9] INFO org.zaproxy.zap.extension.spider.SpiderThread  - Spider scanning complete: true
Scan the site [http://localhost:8080/WebGoat/login.mvc]
Scan url [http://localhost:8080/WebGoat/login.mvc] with the policy by default
20812 [ZAP-ProxyThread-25] INFO org.parosproxy.paros.core.scanner.PluginFactory  - loaded plugin Path Traversal

[wade.sc...@gmail.com]

I'm having the same problem with my set up. Did you ever find the solution to this?