Connection Refused Error in Jenkins when trying to run ant build script which tries to run ZAP

[Usman Waheed]

Hi,

I have a demo setup on my local workstation where i have installed Jenkins and have integrated it with GIT. 

Basically i am trying to run the ZAP spider and scanner against a local website on my localhost whenever a code submit happens.

I have the GIT part and Jenkins working and i use the build-zap.xml noted below:

<project name="zap" default="wave-test" basedir=".">

        <description>Build ZAP.</description>

        <!-- set global properties for this build -->

        <property name="src" location="/home/usmanw/ZAP_2.1.0" />

        <property name="dist" location="/home/usmanw/ZAP_2.1.0" />

        <!--

        In Eclipse you can get these taskdefs to work by adding the following jars to the 'Ant Home Entries'.

        To get there: Windows / Preferences / Ant / Runtime

        TODO: Work out how this should be done properly and document ;)

        lib/json-lib-2.4-jdk15.jar

        lib/commons-lang-2.6.jar

        lib/commons-logging-1.1.1.jar

        lib/commons-collections-3.2.1.jar

        lib/commons-beanutils-1.8.3.jar

        lib/ezmorph-1.0.6.jar

        build/zap/zap-api.jar (after you've run the dist target in build.xml)

        -->

        <taskdef name="accessUrlTask" classname="org.zaproxy.clientapi.ant.AccessUrlTask"/>

        <taskdef name="activeScanUrlTask" classname="org.zaproxy.clientapi.ant.ActiveScanUrlTask"/>

        <taskdef name="activeScanSiteTask" classname="org.zaproxy.clientapi.ant.ActiveScanSiteTask"/>

        <taskdef name="alertCheckTask" classname="org.zaproxy.clientapi.ant.AlertCheckTask"/>

        <taskdef name="loadSessionTask" classname="org.zaproxy.clientapi.ant.LoadSessionTask"/>

        <taskdef name="newSessionTask" classname="org.zaproxy.clientapi.ant.NewSessionTask"/>

        <taskdef name="saveSessionTask" classname="org.zaproxy.clientapi.ant.SaveSessionTask"/>

        <taskdef name="spiderUrlTask" classname="org.zaproxy.clientapi.ant.SpiderUrlTask"/>

        <taskdef name="stopZapTask" classname="org.zaproxy.clientapi.ant.StopZapTask"/>

        <taskdef name="alert" classname="org.zaproxy.clientapi.ant.AlertTask"/>

        <!--

        These settings are for ZAP installed in the standard location on a Windows machine

        listenning on localhost:8090 and

        testing the ZAP Web App Vulnerability Examples running on localhost:8080

        -->

        <property name="targetHost" value="http://localhost" />

        <property name="targetApp" value="${targetHost}"/>

        <property name="zapaddr" value="localhost" />

        <property name="zapport" value="8090" />

        <property name="zapdir" value="/home/usmanw/ZAP_2.1.0" />

<target name="startZap">

                <java classname="org.zaproxy.zap.ZAP" fork="true" spawn="true" dir="${zapdir}">

                        <classpath>

                                <pathelement location="${zapdir}/zap.jar"/>

                        </classpath>

                </java>

                <!-- Give ZAP a chance to start -->

                <sleep seconds="20"/>

        </target>

        <target name="startZapDaemon">

                <java classname="org.zaproxy.zap.ZAP" fork="true" spawn="true" dir="${zapdir}">

                        <arg value="-daemon"/>

                        <classpath>

                                <pathelement location="${zapdir}/zap.jar"/>

                        </classpath>

                </java>

                <!-- Give ZAP a chance to start -->

                <sleep seconds="20"/>

        </target>

        <target name="stopZap">

            <stopZapTask zapAddress="${zapaddr}" zapPort="${zapport}" debug="true"/>

        </target>

        <target name="spider">

            <spiderUrlTask zapAddress="${zapaddr}" zapPort="${zapport}" url="${targetHost}"/>

        </target>

        <target name="activescanSite">

            <activeScanSiteTask zapAddress="${zapaddr}" zapPort="${zapport}" url="${targetHost}"/>

        </target>

    <target name="activescanUrl">

        <activeScanUrlTask zapAddress="${zapaddr}" zapPort="${zapport}" url="${targetHost}"/>

    </target>

        <target name="wave-test">

                <!-- Start ZAP -->

                <antcall target="startZapDaemon" />

                <!-- Access the top URL -->

            <accessUrlTask zapAddress="${zapaddr}" zapPort="${zapport}" url="${targetApp}" debug="true"/>

                <!-- Spider the whole site -->

            <spiderUrlTask zapAddress="${zapaddr}" zapPort="${zapport}" url="${targetHost}" debug="true"/>

                <sleep seconds="5"/>

                <!-- Need to Spider again - and need to investigate why this is required ;) -->

            <spiderUrlTask zapAddress="${zapaddr}" zapPort="${zapport}" url="${targetHost}" debug="true"/>

                <sleep seconds="5"/>

                <!-- Active scan the whole site -->

            <activeScanUrlTask zapAddress="${zapaddr}" zapPort="${zapport}" url="${targetHost}" debug="true"/>

                <sleep seconds="5"/>

                <!-- Save to a timestamped session file -->

                <tstamp>

                        <format property="timestamp" pattern="MM-dd-yyyy-HH-mm-ss"/>

                </tstamp>

            <saveSessionTask zapAddress="${zapaddr}" zapPort="${zapport}" name="usmantest-${timestamp}" debug="true"/>

                <!-- Give the passive scanner thread a chance to catch up -->

                <sleep seconds="20"/>

                <!-- Stop ZAP -->

            <stopZapTask zapAddress="${zapaddr}" zapPort="${zapport}"/>

        </target>

</project>

When i run the build-zap.xml file using ant from the command line everything works fine. I get an ERROR message when it runs within Jenkins.

The error message is noted below:

Started by user usman waheed

Building in workspace /var/lib/jenkins/workspace/Test1

Checkout:Test1 / /var/lib/jenkins/workspace/Test1 - hudson.remoting.LocalChannel@6b29745b

Using strategy: Default

Last Built Revision: Revision 9313c9d48c8eb33feda6f49cac16659b391ff16c (origin/master, origin/HEAD)

Fetching changes from 1 remote Git repository

Fetching upstream changes from origin

Seen branch in repository origin/HEAD

Seen branch in repository origin/master

Seen 2 remote branches

Commencing build of Revision 9313c9d48c8eb33feda6f49cac16659b391ff16c (origin/master, origin/HEAD)

Checking out Revision 9313c9d48c8eb33feda6f49cac16659b391ff16c (origin/master, origin/HEAD)

Warning : There are multiple branch changesets here

[tools] $ ant -file build-zap.xml -lib /home/usmanw/ZAP_2.1.0/zap.jar -lib /home/usmanw/ZAP_2.1.0/zap-api-v2-2.jar

Buildfile: /home/usmanw/tools/build-zap.xml

wave-test:

startZapDaemon:

[accessUrlTask] Open URL: http://localhost

BUILD FAILED

/home/usmanw/tools/build-zap.xml:93: org.zaproxy.clientapi.core.ClientApiException: java.net.ConnectException: Connection refused

at org.zaproxy.clientapi.core.ClientApi.accessUrlViaProxy(Unknown Source)

at org.zaproxy.clientapi.core.ClientApi.accessUrl(Unknown Source)

at org.zaproxy.clientapi.ant.AccessUrlTask.execute(Unknown Source)

at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)

at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:601)

at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)

at org.apache.tools.ant.Task.perform(Task.java:348)

at org.apache.tools.ant.Target.execute(Target.java:392)

at org.apache.tools.ant.Target.performTasks(Target.java:413)

at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)

at org.apache.tools.ant.Project.executeTarget(Project.java:1368)

at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)

at org.apache.tools.ant.Project.executeTargets(Project.java:1251)

at org.apache.tools.ant.Main.runBuild(Main.java:811)

at org.apache.tools.ant.Main.startAnt(Main.java:217)

at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)

at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)

Caused by: java.net.ConnectException: Connection refused

at java.net.PlainSocketImpl.socketConnect(Native Method)

at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)

at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)

at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)

at java.net.Socket.connect(Socket.java:579)

at java.net.Socket.connect(Socket.java:528)

at sun.net.NetworkClient.doConnect(NetworkClient.java:180)

at sun.net.www.http.HttpClient.openServer(HttpClient.java:378)

at sun.net.www.http.HttpClient$1.run(HttpClient.java:430)

at sun.net.www.http.HttpClient$1.run(HttpClient.java:428)

at java.security.AccessController.doPrivileged(Native Method)

at sun.net.www.http.HttpClient.privilegedOpenServer(HttpClient.java:427)

at sun.net.www.http.HttpClient.openServer(HttpClient.java:468)

at sun.net.www.http.HttpClient.<init>(HttpClient.java:203)

at sun.net.www.http.HttpClient.New(HttpClient.java:290)

at sun.net.www.http.HttpClient.New(HttpClient.java:306)

at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:995)

at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:974)

at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:849)

... 19 more

Total time: 20 seconds

Build step 'Invoke Ant' marked build as failure

Finished: FAILURE

I though this might have to do with permissions but that does not seem to be the case. Has anyone using Jenkins run into this error and if clues would be appreciated. 

If you have more questions please let me know and i can share more info on what i have done so far.

Thanks and Regards,

Usman

 

[thc202]

Hi.

That error means that the client API is not able to connect to ZAP (either ZAP was not able to start or it's not listening on the port set in the "zapport" property).

It might be better to add the argument "-port" to the targets that start ZAP to ensure that the port that the tasks are connecting to is the same that ZAP is listening:
<arg line="-port ${zapport}" />


Would you mind check the ZAP log file to see if there's any error when starting with Jenkins? (file zap.log located in ZAP's default directory [1]).


[1] https://code.google.com/p/zaproxy/wiki/FAQconfig

Best regards.

[Usman Waheed]

Yup ZAP is starting as a process when run with userid: jenkins but i dont see it listening on port 8090. I tried adding an additional <arg line="-port ${zapport}"/> which seems to pass the port number to the command line but for some reason netstat -nple | grep java does not show a java process running on port 8090. When i do the same with my userid: usmanw then it works.

Something to do with the userid: jenkins i guess.

I'll keep invesigating but when the ant build runs there is no logging in the ~/.ZAP/zap.log and the ant log does not show any errors either.

Thanks,

Usman

[Usman Waheed]

Hi Thc202,

Thanks for the pointer(s). The problem had to do with the user: jenkins. I changed the build jobs to run with my userid: usmanw and everything works now.

Started by user usman waheed

Building on master in workspace /var/lib/jenkins/workspace/Test1
Checkout:Test1 / /var/lib/jenkins/workspace/Test1 - hudson.remoting.LocalChannel@5faf3306

Using strategy: Default
Last Built Revision: Revision 9313c9d48c8eb33feda6f49cac16659b391ff16c (origin/master, origin/HEAD)
Fetching changes from 1 remote Git repository
Fetching upstream changes from origin
Seen branch in repository origin/HEAD
Seen branch in repository origin/master
Seen 2 remote branches
Commencing build of Revision 9313c9d48c8eb33feda6f49cac16659b391ff16c (origin/master, origin/HEAD)
Checking out Revision 9313c9d48c8eb33feda6f49cac16659b391ff16c (origin/master, origin/HEAD)
Warning : There are multiple branch changesets here
[tools] $ ant -file build-zap.xml -lib /home/usmanw/ZAP_2.1.0/zap.jar

Buildfile: /home/usmanw/tools/build-zap.xml

wave-test:

startZapDaemon:
[accessUrlTask] Open URL: http://localhost:80/
[accessUrlTask] <html><body><h1>Welcome!!!</h1></body></html>
[spiderUrlTask] Open URL: http://zap/xml/spider/action/scan/?url=http://localhost:80/&
[spiderUrlTask] Open URL: http://zap/xml/spider/view/status/
[spiderUrlTask] Open URL: http://zap/xml/spider/view/status/
[spiderUrlTask] Open URL: http://zap/xml/spider/action/scan/?url=http://localhost:80/&
[spiderUrlTask] Open URL: http://zap/xml/spider/view/status/
[spiderUrlTask] Open URL: http://zap/xml/spider/view/status/
[activeScanUrlTask] Open URL: http://zap/xml/ascan/action/scan/?inScopeOnly=false&recurse=true&url=http://localhost:80/&
[activeScanUrlTask] Open URL: http://zap/xml/ascan/view/status/
[activeScanUrlTask] Open URL: http://zap/xml/ascan/view/status/
[saveSessionTask] Open URL: http://zap/xml/core/action/saveSession/?name=zaptest-06-12-2013-15-34-28&

BUILD SUCCESSFUL
Total time: 1 minute 0 seconds
Finished: SUCCESS

Now that i have some scan results saved in the zaptest* file i need to investigate how i can access these results, format them and email them to myself.

Will be looking how to use the ZAP api more down the road. Not sure if i need to integrate this with Selenium at this point.

Thanks for your help and Regards,

Usman

Please note: If anyone has any questions on what i have done here please feel free to drop a line and i will be more than happy to share what i have so far.

[thc202]

Hi.

That's great!


As you were running with jenkins user the ZAP log (and .ZAP directory) would be located in the jenkins home directory [1] ("/var/lib/jenkins" ?).

Did you run the command "netstat -nple | grep java" with enough privileges? as it's the jenkins user that's creating the processes you might not be able to see it listed.

If it's not asking too much, would you mind to check if there's any error in the ZAP log?


Regarding the handling of the scan results that can be done with another (new) ant task.


[1] https://wiki.jenkins-ci.org/display/JENKINS/Administering+Jenkins#AdministeringJenkins-JENKINSHOMEdirectory

Best regards.

[Usman Waheed]

Hi,

I checked the zap.log for the user: jenkins and found no errors in there. The spider + scan ran fine, no issues encountered. 

Next i am going to look/work on a second ant task that can handle the scan results.

Will keep posting on here with questions and progress.

Thanks for your help,

Usman