How to drive automated scans through the command line (docker)

[Michael Peters]

I'm trying to automate scans of an API (unfortunately it's not OpenAPI or SOAP so I can't use the handy built-ins for that). I have several curl/postman requests that I've used to do things from the UI to make sure it works and that's fine.

But I need to automate this with the ZAP docker image (the application in development is a docker container itself). If this was a webapp with a single URL to start spidering this would be pretty trivial, but it's not.

So my thinking was to start the container, then run a bunch of curl commands running through the proxy so they get recorded and then run the active scan/tests when those are done. But how do I tell ZAP I'm done adding URLs and it should start the scan? I'd prefer to use the REST API built-in but everytime I call the API from the command line (using curl) it just gives either an EOF or empty response. So I feel I'm missing something. For instance:

docker run -u zap -p 8080:8080 -d -i owasp/zap2docker-stable zap.sh -daemon -host 0.0.0.0 -port 8080 -config api.disablekey=true

curl -k -X GET https://${DOCKER_HOST_IP}:9023/ccm/info  -H 'cache-control: no-cache'  -H 'content-type: application/json'

# lots more curl commands, some with POST bodies, etc

...

# now let's see what info ZAP has about the URLs we've proxied through it

curl http://localhost:8080/JSON/core/view/sites/

curl: (52) Empty reply from server

So what am I missing?

[Michael Peters]

On Monday, September 18, 2017 at 12:17:42 PM UTC-4, Michael Peters wrote:

curl http://localhost:8080/JSON/core/view/sites/

curl: (52) Empty reply from server

I also tried this:

$ HTTP_PROXY=http://127.0.0.1:8080 curl http://zap/JSON/core/view/sites/

curl: (7) Failed connect to zap:80; Connection refused

 

[Simon Bennetts]

But we do have a handy built in script :)
https://github.com/zaproxy/zaproxy/blob/develop/build/docker/zap-full-scan.py
Use the -D option to delay while you request your URLs.

Its included in the weekly docker releases, so you can use it right now.

Note that uou'll probably have to allow connections from the host container - see https://github.com/zaproxy/zaproxy/wiki/FAQapikey

Cheers,

Simon

[Michael Peters]

I'm also trying via the Python API and a simple Python script

$ docker run -u zap -p 8080:8080 -d -i owasp/zap2docker-stable zap.sh -daemon -host 0.0.0.0 -port 8080 -config api.disablekey=true

$ pip install python-owasp-zap-v2.4

$ $ python simple.py

Accessing target https://10.81.120.222:9023/ccm/info

Spidering target https://10.81.120.222:9023/ccm/info

Traceback (most recent call last):

  File "simple.py", line 18, in <module>

    scanid = zap.spider.scan(target)

  File "/home/mipeter3/zap-automation/python/env/lib/python2.7/site-packages/zapv2/spider.py", line 189, in scan

    return six.next(six.itervalues(self.zap._request(self.zap.base + 'spider/action/scan/', params)))

  File "/home/mipeter3/zap-automation/python/env/lib/python2.7/site-packages/zapv2/__init__.py", line 157, in _request

    data = self._request_api(url, get)

  File "/home/mipeter3/zap-automation/python/env/lib/python2.7/site-packages/zapv2/__init__.py", line 147, in _request_api

    return self.session.get(url, params=query, proxies=self.__proxies, verify=False)

  File "/home/mipeter3/zap-automation/python/env/lib/python2.7/site-packages/requests/sessions.py", line 521, in get

    return self.request('GET', url, **kwargs)

  File "/home/mipeter3/zap-automation/python/env/lib/python2.7/site-packages/requests/sessions.py", line 508, in request

    resp = self.send(prep, **send_kwargs)

  File "/home/mipeter3/zap-automation/python/env/lib/python2.7/site-packages/requests/sessions.py", line 618, in send

    r = adapter.send(request, **kwargs)

  File "/home/mipeter3/zap-automation/python/env/lib/python2.7/site-packages/requests/adapters.py", line 490, in send

    raise ConnectionError(err, request=request)

requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",))

And then it fails with an exception about bad status lines.

Here's the simple.py for reference:

$ cat simple.py

#!/usr/bin/env python

import time

from pprint import pprint

from zapv2 import ZAPv2

target = 'https://10.81.120.222:9023/ccm/info'

zap = ZAPv2()

# do stuff

print 'Accessing target %s' % target

# try have a unique enough session...

zap.urlopen(target)

# Give the sites tree a chance to get updated

time.sleep(2)

print 'Spidering target %s' % target

scanid = zap.spider.scan(target)

[Michael Peters]

I'll look into zap-full-scan. Btw, the link in the script's --help points to https://github.com/zaproxy/zaproxy/wiki/ZAP-Full-Scan which doesn't work (redirects to https://github.com/zaproxy/zaproxy/wiki).

--
You received this message because you are subscribed to a topic in the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/zaproxy-users/tuKgWzt5Sw4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to zaproxy-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/ebfb9edf-0bda-4ced-95f9-62799d55a9ad%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Peters]

Simon,

Thanks for your reply, but it seems there are other problems with this. It's trying to reach out to the external network to check the client API version number (https://pypi.python.org/pypi/python-owasp-zap-v2.4/json). This seems like a design flaw to me. I'm using a self-contained docker container with ZAP and the Python API already installed and they should be matching versions. Why does it need to verify this? And why does it need to try to hit an external URL to do it?

I'm running this inside of an isolated network that can't reach outside and the system under test is another docker container on the same host, so why should I be required to enable external network access to use it?

Thanks,

On Mon, Sep 18, 2017 at 12:42 PM, Simon Bennetts <psi...@gmail.com> wrote:

--

You received this message because you are subscribed to a topic in the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/zaproxy-users/tuKgWzt5Sw4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to zaproxy-users+unsubscribe@googlegroups.com.

[Michael Peters]

I'd be happy to submit a patch for you on this if you'd accept it. Basically don't run the version check if we are running in docker because we should rightly assume that the python API matches the version of ZAP running in the same container.

[Simon Bennetts]

The packaged scripts should all work offline, so this is definitely a bug.
PRs always appreciated :)
For info here is the PR that added the check - its got some notes which might be relevant: https://github.com/zaproxy/zaproxy/pull/3746

And yes, I do need to document it on the wiki....

Cheers,

Simon

[jagriti priya]

Hi Michael,

i want your help, doing the same automated scans through zap proxy server.

To unsubscribe from this group and all its topics, send an email to zaprox...@googlegroups.com.

[thc...@gmail.com]

Hi.

This thread is very old and now outdated, better start a new one.

Best regards.

>>>> zaprox...@googlegroups.com <javascript:>.

>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/zaproxy-users/ebfb9edf-0bda-4ced-95f9-62799d55a9ad%40googlegroups.com

>>>> <https://groups.google.com/d/msgid/zaproxy-users/ebfb9edf-0bda-4ced-95f9-62799d55a9ad%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .