Active SCAN detailed progress in the command prompt

[Rom ain]

Hello, I usually launch full scans with the automation framework using the command line, and I would like to know if it's possible to get information about the ongoing active scan in the command prompt (progress percentage, steps)?

[psiinon]

That is not a built in option, but you will be able to make it happen via scripting.

You'll probably need to start a timer via an extender script https://github.com/zaproxy/community-scripts/tree/main/extender

read the scan progress and then write it to stdout.

Cheers,

Simon

On Tue, Jun 13, 2023 at 10:45 AM Rom ain <romt...@gmail.com> wrote:

Hello, I usually launch full scans with the automation framework using the command line, and I would like to know if it's possible to get information about the ongoing active scan in the command prompt (progress percentage, steps)?

--
You received this message because you are subscribed to the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/0e36d7cc-6265-4737-8079-714fbbc01c6an%40googlegroups.com.

--

OWASP ZAP Project leader

[psiinon]

You could also register for a SCAN_STARTED_EVENT and then add a ScannerListener to the scan

https://github.com/zaproxy/community-scripts/blob/main/extender/ZAP%20onEvent%20Handler.js

[psiinon]

The events available are all listed on https://www.zaproxy.org/docs/internal-events/

[Rom ain]

Thank you for your response,

I have adapted the default script to make it work with the ActiveScanEventPublisher event.
--------------------------

var consumer;

function install(helper) {
var ConsumerClass = Java.extend(Java.type("org.zaproxy.zap.eventBus.EventConsumer"));
consumer = new ConsumerClass({
eventReceived: function(event) {
var target = '---';
if (event.getTarget()) {
target = event.getTarget().getDisplayName();
}
print(
'Event received: \n' +
' Publisher: ' + event.getPublisher().getPublisherName() + '\n' +
' Type: ' + event.getEventType() + '\n' +
' Target: ' + target + '\n' +
' Params: ' + event.getParameters());
}
});

var ActiveScanEventPublisher = Java.type("org.zaproxy.zap.extension.ascan.ActiveScanEventPublisher");
org.zaproxy.zap.ZAP.getEventBus().registerConsumer(consumer, ActiveScanEventPublisher.getPublisherName());
}

function uninstall(helper) {
org.zaproxy.zap.ZAP.getEventBus().unregisterConsumer(consumer);
}

--------------------------


I have integrated the script into my automation plan:

parameters:
action: "add"
type: "active"
engine: "ECMAScript : Graal.js"
name: "ActiveScanEvent"
file: "C:\Users\****\Documents\\\AUTOMATION\V3\listener_activescan.js"
target: ""
inline: ""
name: "script"
type: "script"
My script is not working. Do you know what could be causing this?

Thank you very much!

[thc...@gmail.com]

The type of the script should be "extender" not "active".

Best regards.

>>>>> <https://groups.google.com/d/msgid/zaproxy-users/0e36d7cc-6265-4737-8079-714fbbc01c6an%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>>
>>>>
>>>>
>>>> --
>>>> OWASP ZAP <https://www.zaproxy.org/> Project leader
>>>>
>>>
>>>
>>> --
>>> OWASP ZAP <https://www.zaproxy.org/> Project leader
>>>
>>
>>
>> --
>> OWASP ZAP <https://www.zaproxy.org/> Project leader
>>
>

[Rom ain]

Thank you very much!

[Illia]

Hi there,

I have included the similar extender script in my automation plan and run the script before the "ActiveScan" job.

But I am still getting this output in console:

When I run the plan via ZAP GUI, I saw the following error message: 

ActiveScanEventPublisher.getPublisherName is not a function

However, when I checked the source code for this event, there is a function called .getPublisherName()

How could I make it work so that I can see the active scan progress in console while the scan is running via automation plan in cmd?

[Illia]

Just in case the image is not viewable, I included as attachment too.

[thc...@gmail.com]

Hi.

You should use:
ActiveScanEventPublisher.getPublisher().getPublisherName()

because `getPublisherName()` is an instance method not a class method.


Regarding the WebDriver output, that's unrelated to the script and
that's expected.

Best regards.

On 28/08/2023 09:14, Illia wrote:
> Just in case the image is not viewable, I included as attachment too.
>
> On Monday, 28 August 2023 at 16:10:13 UTC+8 Illia wrote:
>
>> Hi there,
>>
>> I have included the similar extender script in my automation plan and run
>> the script before the "ActiveScan" job.
>> But I am still getting this output in console:
>> [image: image]
>>
>> When I run the plan via ZAP GUI, I saw the following error message:

>> *ActiveScanEventPublisher.getPublisherName is not a function*

[Illia]

Hi there,

Thanks for the reply! I'll modify the script and run it again.

Regarding the WebDriver output, you said it is expected. 

Do you mean that even by using this script, I can't see the scan progress eg. the URLs that it is currently scanning etc. ?

It would be really helpful to be able to monitor the scan's progress via console as I'm planning to integrate it in CICD pipeline.

[thc...@gmail.com]

No, I mean that the WebDriver output will happen regardless of the
extender script. If the script is working fine it will show the active
scan events.

Best regards.

[Illia]

Got it!

Thanks to your help, I have modified the script and attached the output:

Regarding the output, I have two questions:

1. I noticed there is a javax.script.ScriptException: Failed to find script engine: Oracle NashornEvent received message after the activeScan job started. I am using Java 20 and the ZAP v2.13.0 as I figured it is best to use the latest version. 
   
   Is the message related to the JAVA version? If yes, what version should I use instead?
   
   
2. I only see the userName=support parameter in the first "event output". Then, the rest of the active scan events are no longer showing this parameter and the target is also not using the context anymore. May I know why and how can I solve this issue?

[thc...@gmail.com]

1. Yes, using Java 11 should address that.

2. Because that information is unnecessary for the progress events. The
scan ID indicates to which active scan those events pertain to.

Best regards.

On 28/08/2023 10:59, Illia wrote:
> Got it!
>
> Thanks to your help, I have modified the script and attached the output:
>
> Regarding the output, I have two questions:
>

> 1. I noticed there is a *javax.script.ScriptException: Failed to find
> script engine: Oracle NashornEvent received* message after the

> activeScan job started. I am using Java 20 and the ZAP v2.13.0 as I figured
> it is best to use the latest version.
>
> Is the message related to the JAVA version? If yes, what version should
> I use instead?
>

> 2. I only see the *userName=support* parameter in the first "event

> output". Then, the rest of the active scan events are no longer showing

> this parameter and the *target* is also not using the context anymore.

[Illia]

1. Alright, noted on that.

2. Can I take it that ZAP has authenticate successfully using the credentials that I have set? In the Output tab of ZAP GUI, I did saw an "Authentication Successful" message appear.

    But what seems weird to me is that in my company site's audit log, i only saw certain modules being accessed over and over and other modules is not being accessed at all hence why I thought I need to monitor the Active Scan progress and see whether ZAP has authenticate correctly. 

Do you have any idea on what is causing this issue? 

[psiinon]

You can check how ZAP thinks it is doing via the internal stats as per https://www.zaproxy.org/blog/2023-05-02-authentication-auto-detection/#test-request

They do depend on you using the right verification strategy of course - that has to be able to work out if you are logged in or not correctly.

Debugging authentication in an active scan is really hard - I recommend trying to debug simpler setups.

To start with - you say that ZAP is not accessing specific modules when active scanning.

Have you checked that ZAP is actually finding them via the spider(s) or whichever tools you are using to explore your app?

Cheers,

Simon

You received this message because you are subscribed to the Google Groups "ZAP User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/b1e31eca-04d5-418f-886a-9f8e7f6a2e4cn%40googlegroups.com.

--

ZAP Project leader

[Illia]

Hi Simon,

Thanks for the reply!

I actually use the "Import URLs" function which allows me to import a set of URL in a .txt file.

After discussion with my co-workers, we performed the below steps in ZAP GUI and in our audit log, ZAP is accessing all modules:

1) Manually login and logout from our website via Manual Explore tab

2) After the URL appear in the Sites , include it in a new context

3) Set my login URL as the Form-Based Auth Login Request 

4) Configured the username, password, regex pattern used to identify Logged Out messages and added user in the Users tab

5) Excluded the logout URL from the context

6) Go to Params tab and set the below 3 as Flag as Session Token

• AntiXsrfToken
• ASP.NET_SessionId
• MySession
  

7) Manually log in and log out from the website via Manual Explore tab again

8) In HTTP Session tab, set the new session as Active

9) Right-click the context and perform Active Scan

I already know how to perform step 1 to step 5 and step 9 via automation framework. 

How do I perform step 6 to step 8 via automation framework too?

Any helpful reference/guide is deeply appreciated.

[psiinon]

You are partly setting up 2 different ZAP features which are not designed to work together.

The HTTP Sessions tab and Active Session are only really supported for manual testing: https://www.zaproxy.org/docs/authentication/manual-auth/

I do not recommend using them for automation, and definitely not with ZAP handling the authentication.

I recommend having a look at all of the links on https://www.zaproxy.org/docs/authentication/ especially the "Auto-Detection" one - if that works then your life will be much easier.

Cheers,

Simon

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/a20cdccc-0e57-4565-978d-0c7f7cb14905n%40googlegroups.com.

--

ZAP Project leader

[Illia]

Hi Simon,

Thanks for the reply!

I took a look at the link that you provide, especially the Auto-Detection one.

I tried using Authentication Tester tool and entered the login URL, valid username and password.

But when ZAP launches the browser, I saw that it skipped the Username field and only entered the password in the Password field.

It then could not log into our website as the Username field is empty hence it could not find the "Verification URL"

Attached is the Result screenshot

Regarding this, I have two questions:

1. Do you know why ZAP is skipping the Username field? 

2. I saw you mention in other conversation that the Authentication Tester is meant for Browser-based Authentication.

    Since I was using Form-based Authentication from the very beginning, does this mean that it is useless for me to keep trying to use          Authentication Tester tool?

[Illia]

Hi Simon,

Sorry I just realised that I did not include the diagnostics in my previous message. 

Here it is:

>>>>>
POST https://example0/ListAccounts
content-type: application/x-www-form-urlencoded
<<<
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8

["token0",[]]
>>>>>
GET https://example1/login.aspx
<<<
HTTP/1.1 200 OK
content-type: text/html; charset=utf-8
cookie: __AntiXsrfToken=3f880640900f46c7827b5c4a7bc7412b
cookie: mySession=e801d756-1b3d-4070-a0fc-e52a615d14f1
cookie: ASP.NET_SessionId=4cctefwevijtaegrgx4ptkik
>>>>>
GET https://example1/
set-cookie: ASP.NET_SessionId="token3"
set-cookie: __AntiXsrfToken="token1"
set-cookie: mySession="token2"
<<<
HTTP/1.1 403 Forbidden
content-type: text/html
>>>>>
GET https://example2/ChVDaHJvbWUvMTE2LjAuNTg0NS4xMTISHgl15CH0kksUXBIFDShkG0ISBQ0gscLVEgUNlY0OQg==
<<<
HTTP/1.1 200 OK
content-type: text/plain
>>>>>
GET https://example2/ChVDaHJvbWUvMTE2LjAuNTg0NS4xMTISFwkWyxl4D_erFRIFDTQ30ysSBQ3c5MosEhcJbpxO9YafN7gSBQ00N9MrEgUN3OTKLA==
<<<
HTTP/1.1 200 OK
content-type: text/plain
>>>>>
GET https://example2/ChVDaHJvbWUvMTE2LjAuNTg0NS4xMTISVgmhafCFryYCZRIFDTQ30ysSBQ3c5MosEgUNBu27_xIFDQbtu_8SBQ0G7bv_EgUNBu27_xIFDQbtu_8SBQ0G7bv_EgUNBu27_xIFDQbtu_8SBQ0G7bv_EhcJbpxO9YafN7gSBQ00N9MrEgUN3OTKLBJICbTmrF5B_mW2EgUNBu27_xIFDQbtu_8SBQ0G7bv_EgUNBu27_xIFDQbtu_8SBQ0G7bv_EgUNBu27_xIFDQbtu_8SBQ0G7bv_
<<<
HTTP/1.1 200 OK
content-type: text/plain
>>>>>
POST https://example3/v1:GetModels
content-type: application/x-protobuf
<<<
HTTP/1.1 200 OK
content-type: application/x-protobuf
>>>>>
GET https://example2/ChVDaHJvbWUvMTE2LjAuNTg0NS4xMTISSAlm3WMD0GRaMRIFDQbtu_8SBQ0G7bv_EgUNBu27_xIFDQbtu_8SBQ0G7bv_EgUNBu27_xIFDQbtu_8SBQ0G7bv_EgUNBu27_xJICbTmrF5B_mW2EgUNBu27_xIFDQbtu_8SBQ0G7bv_EgUNBu27_xIFDQbtu_8SBQ0G7bv_EgUNBu27_xIFDQbtu_8SBQ0G7bv_
<<<
HTTP/1.1 200 OK
content-type: text/plain
>>>>>
GET https://example3/downloads
<<<
HTTP/1.1 200 OK
content-type: text/html; charset=UTF-8
>>>>>
GET https://example3/downloads
<<<
HTTP/1.1 200 OK
content-type: text/html; charset=UTF-8
>>>>>
GET https://example3/downloads
<<<
HTTP/1.1 200 OK
content-type: text/html; charset=UTF-8
>>>>>
GET https://example1/
<<<
HTTP/1.1 403 Forbidden
content-type: text/html
>>>>>
GET https://example1/
<<<
HTTP/1.1 403 Forbidden
content-type: text/html
>>>>>
GET https://example1/
<<<
HTTP/1.1 403 Forbidden
content-type: text/html
>>>>>
GET https://example1/
<<<
HTTP/1.1 403 Forbidden
content-type: text/html
>>>>>
GET https://example1/
<<<
HTTP/1.1 403 Forbidden
content-type: text/html
>>>>>
GET https://example1/
<<<
HTTP/1.1 403 Forbidden
content-type: text/html
>>>>>
GET https://example1/
<<<
HTTP/1.1 403 Forbidden
content-type: text/html
>>>>>
GET https://example1/
<<<
HTTP/1.1 403 Forbidden
content-type: text/html
>>>>>
GET https://example1/
<<<
HTTP/1.1 403 Forbidden
content-type: text/html
>>>>>
GET https://example1/
<<<
HTTP/1.1 403 Forbidden
content-type: text/html
>>>>>
GET https://example1/
<<<
HTTP/1.1 403 Forbidden
content-type: text/html
>>>>>
GET https://example1/
<<<
HTTP/1.1 403 Forbidden
content-type: text/html
>>>>>
GET https://example3/downloads
<<<
HTTP/1.1 200 OK
content-type: text/html; charset=UTF-8

[Simon Bennetts]

ZAP _thinks_ it has found the username field, but from what you have seen it looks like it got it wrong.

Can you share the HTML for all of the form fields?

Feel free to obfuscate anything you dont want to share.

Cheers,

Simon

[Illia]

Hi Simon,

In another thread, I have replied that I changed my command line usage and is now able to run the scan authenticated.

Thanks for your help Simon and thc202!

[Simon Bennetts]

Thanks for letting us know!