Issues with Juice Shop + Ajax Spider w/ Standard Auth
435 views
Skip to first unread message
Kyle Orlando
Feb 20, 2021, 9:14:29 AM2/20/21
to OWASP ZAP User Group
Hi,
I was following along with the ZAP in Ten: ADDO Workshop
Section 6 - Standard Auth with JuiceShop video, and it was going well until I tried the Ajax spider after having setup the context and scripts. The Selenium Juice Shop script seems unable to retrieve the URL when ssutils.waitForURL(5000) is called. The value returned is always about:blank (for Firefox) or default; (for Chrome). Increasing the wait time does not help; it merely causes the browser to sit at one of the aforementioned blank pages for that additional amount of time. In contrast, if I launch the browser manually (from within Zap) with force user mode on, I have no issues.The only modification I had to make was to the Selenium Juice Shop script, and that was to change the jsUrl (changed localhost to kyles-juice-shop) since I am running Juice Shop on a different (virtual) machine.
Taking a look at the log file, it's not exactly clear to me what's causing this to happen. I've bolded a couple of lines that I think look suspicious. I've included the relevant excerpt below:
2021-02-19 21:40:42,073 [AWT-EventQueue-0] DEBUG ZestZapUtils - getElement My Selenium Juice Shop Unrecognised class: org.zaproxy.zap.extension.script.ScriptWrapper
2021-02-19 21:40:44,736 [AWT-EventQueue-0] DEBUG PopupMenuItemHttpMessageContainer - actionPerformed SITES_PANEL AJAX Spider...
2021-02-19 21:40:44,736 [AWT-EventQueue-0] DEBUG AjaxSpiderDialog - init org.zaproxy.zap.model.Target@2ca9de26
2021-02-19 21:40:52,528 [ZAP-AjaxSpider] INFO SpiderThread - Running Crawljax (with chrome): Context: Default Context
2021-02-19 21:40:52,529 [ZAP-AjaxSpider] DEBUG SiteMap - findChild Sites / http://kyles-juice-shop:3000
2021-02-19 21:40:52,529 [ZAP-AjaxSpider] DEBUG SimpleEventBus - publishSyncEvent scan.started from org.zaproxy.zap.extension.spiderAjax.SpiderEventPublisher
2021-02-19 21:40:52,529 [ZAP-AjaxSpider] INFO SpiderThread - Starting proxy...
2021-02-19 21:40:52,530 [ZAP-AjaxSpider] INFO SpiderThread - Proxy started, listening at port [36481].
2021-02-19 21:40:52,538 [ZAP-AjaxSpider] INFO Plugins - Loaded org.zaproxy.zap.extension.spiderAjax.SpiderThread$DummyPlugin@71364522 as a OnBrowserCreatedPlugin
2021-02-19 21:40:52,550 [ZAP-AjaxSpider] DEBUG SpiderThread - Setting up a Browser
2021-02-19 21:40:52,552 [UrlChecker-9] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/status [DIRECT]
2021-02-19 21:40:52,563 [UrlChecker-9] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/status [DIRECT]
2021-02-19 21:40:52,583 [UrlChecker-9] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/status [DIRECT]
2021-02-19 21:40:52,624 [UrlChecker-9] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/status [DIRECT]
2021-02-19 21:40:52,632 [Forwarding newSession on session null to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:52,632 [Forwarding newSession on session null to remote] DEBUG ZapProxySelector - Selected proxies for socket://localhost:20020 [DIRECT]
2021-02-19 21:40:53,896 [ZAP-ProxyThread-729] DEBUG SpiderThread - Excluding request [https://accounts.google.com/ListAccounts?gpsia=1&source=ChromiumBrowser&json=standard] not in specified context.
2021-02-19 21:40:54,045 [Forwarding newSession on session null to remote] INFO ProtocolHandshake - Detected dialect: W3C
2021-02-19 21:40:54,046 [ZAP-AjaxSpider] DEBUG ExtensionScript - invokeScript My Selenium Juice Shop
2021-02-19 21:40:54,074 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:54,290 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:54,509 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:54,721 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:54,723 [ZAP-ProxyThread-729] DEBUG SpiderThread - Excluding request [https://accounts.google.com/ListAccounts?gpsia=1&source=ChromiumBrowser&json=standard] not in specified context.
2021-02-19 21:40:54,926 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:55,135 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:55,342 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:55,549 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:55,755 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:55,960 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:56,170 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:56,384 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:56,595 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:56,804 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:57,014 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:57,221 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:57,226 [ZAP-ProxyThread-729] DEBUG ProxyThread - Timed out while reading a new HTTP request.
2021-02-19 21:40:57,344 [ZAP-ProxyThread-730] DEBUG SpiderThread - Excluding request [https://accounts.google.com/ListAccounts?gpsia=1&source=ChromiumBrowser&json=standard] not in specified context.
2021-02-19 21:40:57,425 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:57,554 [ZAP-PassiveScanner] DEBUG PassiveScanData - No Context found for: https://accounts.google.com/ListAccounts?gpsia=1&source=ChromiumBrowser&json=standard
2021-02-19 21:40:57,554 [ZAP-PassiveScanner] DEBUG PassiveScanData - No Context found for: https://accounts.google.com/ListAccounts?gpsia=1&source=ChromiumBrowser&json=standard
2021-02-19 21:40:57,555 [ZAP-PassiveScanner] DEBUG PassiveScanData - No Context found for: https://accounts.google.com/ListAccounts?gpsia=1&source=ChromiumBrowser&json=standard
2021-02-19 21:40:57,630 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:57,839 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:58,043 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:58,250 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:58,458 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:58,671 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:58,883 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:59,088 [Forwarding getCurrentUrl on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:59,301 [Forwarding get on session 5e2ee4f9dddcabeb8ce9e18de40d3cc0 to remote] DEBUG ZapProxySelector - Selected proxies for http://localhost:20020/ [DIRECT]
2021-02-19 21:40:59,328 [ZAP-ProxyThread-731] DEBUG HttpSender - sendAndReceive GET http://kyles-juice-shop:3000/ start
2021-02-19 21:40:59,330 [ZAP-ProxyThread-731] DEBUG HttpSessionsSite - No session tokens for: kyles-juice-shop:3000
2021-02-19 21:40:59,330 [ZAP-ProxyThread-731] DEBUG ExtensionScript - invokeScript My Juice Shop Session Management
2021-02-19 21:40:59,368 [ZAP-ProxyThread-731] DEBUG HttpSender - Sending message to: http://kyles-juice-shop:3000/
2021-02-19 21:40:59,368 [ZAP-ProxyThread-731] DEBUG ZapProxySelector - Selected proxies for socket://kyles-juice-shop:3000 [DIRECT]
2021-02-19 21:40:59,371 [ZAP-ProxyThread-731] DEBUG HttpSender - SUCCESSFUL
2021-02-19 21:40:59,371 [ZAP-ProxyThread-731] DEBUG HttpSender - sendAndReceive GET http://kyles-juice-shop:3000/ took 41
2021-02-19 21:40:59,371 [AWT-EventQueue-0] DEBUG SiteMap - addPath http://kyles-juice-shop:3000/
2021-02-19 21:40:59,371 [AWT-EventQueue-0] DEBUG SiteMap - findAndAddChild Sites / http://kyles-juice-shop:3000
2021-02-19 21:40:59,371 [AWT-EventQueue-0] DEBUG SiteMap - findChild Sites / http://kyles-juice-shop:3000
2021-02-19 21:40:59,371 [AWT-EventQueue-0] DEBUG SiteMap - findAndAddLeaf http://kyles-juice-shop:3000 / /
2021-02-19 21:40:59,372 [AWT-EventQueue-0] DEBUG SiteMap - findChild http://kyles-juice-shop:3000 / GET:/
2021-02-19 21:40:59,409 [ZAP-ProxyThread-731] DEBUG HttpSender - sendAndReceive GET http://kyles-juice-shop:3000/styles.css start
2021-02-19 21:40:59,410 [ZAP-ProxyThread-731] DEBUG HttpSessionsSite - No session tokens for: kyles-juice-shop:3000
2021-02-19 21:40:59,410 [ZAP-ProxyThread-731] DEBUG ExtensionScript - invokeScript My Juice Shop Session Management
Here's my setup:
- Linux kali 5.9.0-kali1-amd64 #1 SMP Debian 5.9.1-1kali2 (2020-10-29) x86_64 GNU/Linux running on Virtual Box VM
- Virtual Box Version 6.1.16 r140961 (Qt5.6.2)
- Base memory: 3GB
- ZAP 2.10.0
- Ajax Spider 23.2.0
- Selenium 15.3.0
- Google Chrome
- Browser 88.0.4324.182 (64-bit)
- ChromeDriver 88.0.4324.96 (64-bit)
- Mozilla Firefox
- Browser 78.3.0esr (64-bit)
- geckodriver 0.29.0 (64-bit)
Does anyone know what's going on and/or have some suggestions for me to try?
Thanks,
Kyle
Simon Bennetts
Feb 22, 2021, 4:46:10 AM2/22/21
to OWASP ZAP User Group
Have you changed all instances of localhost:20020 in the scripts to your new URL?
Based on the log it looks like you might now, but I dont know if that includes log records from when you were using the old one.
Cheers,
Simon
Kyle Orlando
Feb 22, 2021, 9:14:24 PM2/22/21
to zaprox...@googlegroups.com
Hi Simon,
I actually don't know where http://localhost:20020/ is coming from. It's not something I have specified or changed, nor do I see a reference to it in the scripts I'm using.
The only thing I changed was in (my copy of) Selenium Juice Shop.js. The default:
var jsUrl = 'http://localhost:3000';
has been changed to:
var jsUrl = 'http://kyles-juice-shop:3000';
At some point I had also added some additional logging, which is why I knew that the url I was getting was about:blank or default;.
--
You received this message because you are subscribed to a topic in the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/zaproxy-users/g_-oS4W7vt8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to zaproxy-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/209dad47-4f2b-43e7-8297-4e1d03bae76cn%40googlegroups.com.
Kyle Orlando
Feb 24, 2021, 9:02:35 AM2/24/21
to OWASP ZAP User Group
It seems that the target URL for Ajax Spider isn't being retrieved until after browserLaunched() finishes executing. If I manually instruct the web driver in my selenium script to retrieve the URL, it functions with the Ajax Spider and browser as expected. Here's what my Selenium script looks like now (my changes are in bold).
var ScriptVars = Java.type('org.zaproxy.zap.extension.script.ScriptVars');
//Change the jsUrl var if the instance of Juice Shop you are using is not listening on http://localhost:3000
var jsUrl = 'http://kyles-juice-shop:3000';
function browserLaunched(ssutils) {
var token = ScriptVars.getGlobalVar("juiceshop.token");
if (token != null) {
logger('browserLaunched ' + ssutils.getBrowserId());
var wd = ssutils.getWebDriver();
wd.get(jsUrl);
var url = ssutils.waitForURL(5000);
if (url.startsWith(jsUrl)) {
logger('url: ' + url + ' setting token ' + token);
var script = 'document.cookie = \'token=' + token + '\';\n' +
'window.localStorage.setItem(\'token\', \'' + token + '\');';
wd.executeScript(script);
}
} else {
logger('no token defined');
}
}
// Logging with the script name is super helpful!
function logger() {
print('[' + this['zap.script.name'] + '] ' + arguments[0]);
}
I'm not sure if this indicates an underlying issue with ZAP that needs to be fixed, or if the Selenium script templates need to be updated. I'm curious if this behavior is different in earlier versions of ZAP.
Kyle
Message has been deleted
Kyle Orlando
Mar 10, 2022, 2:43:58 PM3/10/22
to zaprox...@googlegroups.com
Hi Henri,
Did you try making the change I suggested? It's been a while, but I believe that fixed my issue.
On Thu, Mar 10, 2022, 02:21 Henri H <henri.h...@lahtiprecision.com> wrote:
Hey, did you (or anyone else) find a solution for this issue?
I followed those same tutorials that you mentioned on your post, and I am facing the same issue. AJAX spider does not start until "browserLaunched" function has finished executing. I am using ZAP version 2.11.1.
To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/43a43e70-aa46-471c-b481-c0903664b898n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages