Ajax Spider fails to open the URL in Firefox

[kr]

Ajax Spider fails to open the URL in Firefox

What steps will reproduce the problem?
1. Proxy a https site, set current HTTPSession to Active.
2. Right click Site URL > Include in Context  & Attack > Ajax Spider in Scope
3. Firefox Browser opens with empty URL & the process stops

What is the expected output? What do you see instead?
Firefox Browser should have opened the URL but the browser opens with empty URL & the process stops.

What version of the product are you using? On what operating system?
ZAP 2.3.1, Ajax Spider & Mozilla Firefox version 33.0

Please provide any additional information below.
1) Also, the Proxy port in Firefox window that is opened is set to any random value like 52959
2) Following is printed in zap.log:
Driver info: driver.version: FirefoxDriver
  at com.crawljax.di.ConfigurationModule.configure(ConfigurationModule.java:47)
  while locating com.crawljax.browser.EmbeddedBrowser
    for parameter 0 at com.crawljax.core.CrawlerContext.<init>

(CrawlerContext.java:32)
  while locating com.crawljax.core.CrawlerContext
    for parameter 0 at com.crawljax.core.Crawler.<init>(Crawler.java:72)
  while locating com.crawljax.core.Crawler
    for parameter 2 at com.crawljax.core.CrawlTaskConsumer.<init>
(CrawlTaskConsumer.java:30)
  while locating com.crawljax.core.CrawlTaskConsumer

1 error
    at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:987)
    at com.crawljax.core.CrawlController.call(CrawlController.java:65)
    at com.crawljax.core.CrawljaxRunner.call(CrawljaxRunner.java:37)
    at org.zaproxy.zap.extension.spiderAjax.SpiderThread.run(SpiderThread.java:207)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.openqa.selenium.WebDriverException: Failed to connect to binary FirefoxBinary(C:\Program Files (x86)\Mozilla Firefox\firefox.exe) on port 7055;

process output follows:
85-3208198ce6fd} visible
1411399118305    addons.xpi    DEBUG    checkForChanges
1411399118321    addons.xpi    DEBUG    No changes found
*** Blocklist::_preloadBlocklistFile: blocklist is disabled
    at org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.start(NewProfileExtensionConnection.java:118)
    ... 30 more

Any pointers in this regard will be highly appreciated.

Thanks!
KR

[thc...@gmail.com]

Hi.

There's a new version of "Ajax Spider" add-on (version 11) in the marketplace [1]. It adds support for version 32 of Firefox.
I don't know if it will also work with version 33. Could you give it a try?

1) Also, the Proxy port in Firefox window that is opened is set to any random value like 52959

That's the expected behaviour since "Ajax Spider" add-on uses another proxy instance to intercept the messages of "its" browsers.


[1] https://code.google.com/p/zaproxy/wiki/HelpUiDialogsManageaddons

Best regards.

--
You received this message because you are subscribed to the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[kr]

Hi,

Thank you for your reply.

This works fine with the updated version of "Ajax Spider".

But it does not seem to enter valid credentials on the login page even after configuring all Session Properties as demonstrated by Mr. Cosmin Stefan's Youtube video. It tries with a set of random credentials before it stops the process. Could you please shed some light if am missing anything?

Thanks,
KR

[kr]

Hi,

We are using HTTPS for this scenario. Attached is zap.log with more details.

Even after setting the current session to 'Active' & enabling 'Session Tracking (Cookie)' option, it is unable to pass through the login page. The process stops after ZAP tries random user names.

We are using JBoss4.2.3 as the application server. Does it expect any additional ports to be opened on the application side or is it some known issue which needs a patch or upgrade?

Any pointers will be highly appreciated.

Thanks!

[thc...@gmail.com]

Hi.

I'm glad that it works with Firefox version 33.

The "Ajax Spider" does not introduce the authentication credentials (set
in ZAP) in the browser (although that could be done).

When using the "Forced User" mode the authentication is done by ZAP
behind the scenes. When ZAP detects that a, supposedly authenticated,
request was not successful because it was not authenticated (by checking
the "Logged In"/"Logged Out" regexes in the response), it authenticates
(i.e. obtains the authenticating data. e.g. Cookie(s), Authorization
header, ...) and resends the (original) request again with the
authenticating data set.

The problem in this case might be with the seed used for spidering.
The login page might not be the best (not saying that this can't be
improved ;) since ZAP would send an authenticated login attempt, which
probably is not accepted by the target application. What happens if an
authenticated user tries to authenticate again?

Could you try using another seed (e.g. main page) for the spider?

When using "Http Sessions" active session, you also need to use as seed
a "main" page and exclude the login/logout pages from the spider.

Regarding the options/components "Http Sessions", "Forced User" and
"Enable Session Tracking (Cookie)", those should not be used at the same
time. Otherwise ZAP might end up sending more cookies than expected (or
not sending the expected cookies) by the target application and fail to
authenticate.


Best regards.

>> *What steps will reproduce the problem?*

>> 1. Proxy a https site, set current HTTPSession to Active.
>> 2. Right click Site URL > Include in Context & Attack > Ajax
>> Spider in Scope
>> 3. Firefox Browser opens with empty URL & the process stops
>>

>> *What is the expected output? What do you see instead?*

>> Firefox Browser should have opened the URL but the browser
>> opens with empty URL & the process stops.
>>

>> *What version of the product are you using? On what operating
>> system?*

>> ZAP 2.3.1, Ajax Spider & Mozilla Firefox version 33.0
>>

>> *Please provide any additional information below.*

>> <https://groups.google.com/d/optout>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "OWASP ZAP User Group" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to zaproxy-user...@googlegroups.com

> <mailto:zaproxy-user...@googlegroups.com>.

[kr]

Hi Simon,

Using another seed for Ajax spider works for one of our Products. Apparently, all the requests for this Product are POSTS & no parameter is being passed in query string.

However, we are still facing the same issue with another Product, following are the findings:
1) With HTTP Session active, login & logout requests excluded & irrespective of any page selected as seed, Ajax Spider somehow always tries to open login.jsp, but does not proceed ahead.

2) With HTTP Session as off & Forced User Mode enabled, Ajax Spider commences from the desired seed page, but the target page does not load.

3) Resending those requests individually returns a success, but not so with Ajax Spider. Note that the POST actions in this product also pass a couple of parameters in GET URL. Do you think this is causing an issue? Could you please explain on what basis ZAP identifies a GET vs a POST request?

4) Does ZAP has any settings for 302 direction.

As for your query - the product allows an already authenticated user to get authenticated again.

This product makes numerous Ajax requests, hence we are specifically interested in Ajax Spidering the setup.

Being a newbie to ZAP, pl. bear with me.

Thanks,
KR

[kr]

Also, for the POST forms we do pass a couple of params in GET request. Not sure if that is causing the issue.

[thc...@gmail.com]

Hi.

> 1) With HTTP Session active, login & logout requests excluded & irrespective of any page selected as seed, Ajax Spider somehow always tries to open login.jsp, but does not proceed ahead.

Might be that it's being redirected to login.jsp? Maybe the session set
as active was not authenticated?

> 2) With HTTP Session as off & Forced User Mode enabled, Ajax Spider commences from the desired seed page, but the target page does not load.

Could you check if there's any error logged? That shouldn't happen...

> 3) Resending those requests individually returns a success, but not so with Ajax Spider. Note that the POST actions in this product also pass a couple of parameters in GET URL. Do you think this is causing an issue?

When submitting the forms? It shouldn't cause any issue.
When using as seed? Well the pages will be, always, fetched using the
GET method without any query parameters. Maybe that's causing the problems?

> Could you please explain on what basis ZAP identifies a GET vs a POST request?

By looking at the request method of the message, but I think that's not
the answer that you are looking for.

> 4) Does ZAP has any settings for 302 direction.

A global setting? No. When resending a request ("Resend"/"Manual Request
Editor" dialogues) you can set to follow or not the redirections.
When spidering with the Ajax Spider the redirections are handled by the
browser.
When automatically authenticating (e.g. forced user mode) the
redirections are not followed.


Regarding the last message, it shouldn't cause any issue, the browser
should be handling/sending those parameters as well.

Best regards.

> *What steps will reproduce the problem?*

> 1. Proxy a https site, set current HTTPSession to Active.
> 2. Right click Site URL > Include in Context & Attack > Ajax
> Spider in Scope
> 3. Firefox Browser opens with empty URL & the process stops
>

> *What is the expected output? What do you see instead?*

> Firefox Browser should have opened the URL but the browser opens
> with empty URL & the process stops.
>

> *What version of the product are you using? On what operating
> system?*

> ZAP 2.3.1, Ajax Spider & Mozilla Firefox version 33.0
>

> *Please provide any additional information below.*