Scanning Multiple URLs with HAR Files

[Angus Grahame]

Hi Folks,

I have lots of URLs and HAR format files generated by a crawler.

URL list
--------
a.com/a-path
b.com/a-path?with_query=string
subdomain.c.com/another/url/a.php
....

HAR files
---------------
har_file_for_a.com_a_path
har_file_for_b.com_a_path
har_file_for_subdomain.c.com_another_url_a_php

I'm using python for my projects with docker environment.

My crawler generating a few hundred URLs and more HAR files (some URLs have multiple input) per day.

I'm trying to scan URLs in the list with giving HAR files for only a few vulnerabilities. (har files can contain get or post data)

I researched zaproxy and found out with three options:

1. Using zap-cli (https://github.com/Grunny/zap-cli) : It looks like quick-scan is ok, but there is no option to give har file.

2. Using zap-api-python (https://github.com/zaproxy/zap-api-python), I tested this project. But this it has the same problem as zap-cli, no har file option.

3. Using raw zap-api:  I have no idea to start with.

Could you please help me to come up with this problem?

1. What is the best method to achieve this?
2. Am I heading in the right direction?
3. How should I use zap-api? (which requests should I make?

Thank you,

Angus

[Angus Grahame]

PS: All solutions must be from a cli interface to easy integrate my other projects. 

[Simon Bennetts]

The full set of ZAP automation options are given on https://www.zaproxy.org/docs/automate/

In your case I think you'll have to go down the API route.

Although the ZAP python API does not currently support the har file option it is actually very extensible and should be able to cope with any ZAP API call.

Har file import is provided by the Exim add-on: https://www.zaproxy.org/docs/desktop/addons/import-export/

That page shows that the API call is

• /exim/importHar (filePath*)

This can be called using the existing API using:

from zapv2 import ZAPv2
import six
zap = ZAPv2(proxies={'http': 'http://127.0.0.1:8090', 'https': 'http://127.0.0.1:8090'})
print(six.next(six.itervalues(zap._request(zap.base + 'exim/action/importHar/', {'filePath': '/full/path/to/file.har'}))))

The exim add-on is actually a prime candidate for an Automation Framework job - I'll see if we can do something about that asap.

Cheers,

Simon

[Angus Grahame]

Thank you for your response.

I created (mostly copied from zap-API-python) simple script like that:

https://gist.github.com/angusgr/2119be2a81d566498f67f5877a162f76

My har file:
https://gist.github.com/angusgr/15a92fbbe4dde2b9472eebcd5f98dfe0

I can not set a config something like 'use har file as target.'

Also, it looks like har file must be on the Docker, not the host machine, to upload correctly. It seems a little bit weird to me, but I couldn't figure out where I did wrong. 

On Tuesday, February 8, 2022 at 8:40:44 PM UTC+3 psi...@gmail.com wrote:

The full set of ZAP automation options are given on https://www.zaproxy.org/docs/automate/

In your case I think you'll have to go down the API route.

Although the ZAP python API does not currently support the har file option it is actually very extensible and should be able to cope with any ZAP API call.

Har 

[Simon Bennetts]

So is it working for you or not?

In the example I gave the har file must be available to the script.

If the script runs in docker then the script must be in docker.

Thats not a ZAP restriction, thats a docker one :)

Cheers,

Simon

[Simon Bennetts]

FYI I've raised an issue for adding an Automation Framework job for importing files (including har ones): https://github.com/zaproxy/zaproxy/issues/7078

This should take too much work so I'm hoping we can encourage someone to work on this soon...

[Angus Grahame]

I couldn't find a setting to use HAR file as target (as input). 

Even if i uploaded HAR file correctly, ZAP still scanning the target variable not the target in the HAR file.

I want to scan target in the HAR file for scanID 40018

TY for issues

[kingthorin+owaspzap]

The HAR you import isn't the target, the target is the request the HAR pertains to.