call graph and quick scan!

[m.pen...@gmail.com]

Hi all,
when i use quick start to scan an url, the requests that send to the server don't contains referrer field,
in the other hand, call graph is based on this field!
is there any way for using call graph in quick start?

best regards

[Colm O'Flaherty]

I don't believe so.  If I'm not mistakn, the Spider does not currently set the referer field when spidering the application, so the call graph does not have this information available to it. until the application is manually browsed. 

If the Ajax spider is used with a web browser such as Firefox or Chrome rather than HtMUnit, the browser should set the referer field in the normal way.  You could try that and see if it solves your problem to any degree.

Colm

--
You received this message because you are subscribed to the Google Groups "OWASP ZAP Developer Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-devel...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[psiinon]

That would be a nice enhancement - an option to set the referer :)

To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-develop+unsubscribe@googlegroups.com.

[m.pen...@gmail.com]

we can set the referrer field with parent of each link,
i think the place that it must be set is HttpRequestHeader class,
is this right?!

an other problem is that i can't load call graph in debug process, when i load that with the menu of file->load add-on file... , nothing change in the program and i get this errors in log console:
4854382 [AWT-EventQueue-0] ERROR org.zaproxy.zap.control.AddOnLoader  - C:\Users\mahin\OWASP ZAP_D\plugin\callgraph-alpha-1.zap (The system cannot find the file specified)
java.io.FileNotFoundException: C:\Users\mahin\OWASP ZAP_D\plugin\callgraph-alpha-1.zap (The system cannot find the file specified)
    at java.util.zip.ZipFile.open(Native Method)
    at java.util.zip.ZipFile.<init>(Unknown Source)
    at java.util.zip.ZipFile.<init>(Unknown Source)
    at java.util.jar.JarFile.<init>(Unknown Source)
    at java.util.jar.JarFile.<init>(Unknown Source)
    at org.zaproxy.zap.control.AddOnLoader.getJarClassNames(AddOnLoader.java:577)
    at org.zaproxy.zap.control.AddOnLoader.getImplementors(AddOnLoader.java:481)
    at org.zaproxy.zap.control.ExtensionFactory.loadAddOnExtensions(ExtensionFactory.java:155)
    at org.zaproxy.zap.control.AddOnLoader.addAddon(AddOnLoader.java:220)
    at org.zaproxy.zap.extension.autoupdate.ExtensionAutoUpdate.install(ExtensionAutoUpdate.java:757)
    at org.zaproxy.zap.extension.autoupdate.ExtensionAutoUpdate.installLocalAddOn(ExtensionAutoUpdate.java:210)
    at org.zaproxy.zap.extension.autoupdate.ExtensionAutoUpdate.access$1(ExtensionAutoUpdate.java:183)
    at org.zaproxy.zap.extension.autoupdate.ExtensionAutoUpdate$2.actionPerformed(ExtensionAutoUpdate.java:172)
    at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
    at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
    at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
    at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
    at javax.swing.AbstractButton.doClick(Unknown Source)
    at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source)
    at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown Source)
    at java.awt.Component.processMouseEvent(Unknown Source)
    at javax.swing.JComponent.processMouseEvent(Unknown Source)
    at java.awt.Component.processEvent(Unknown Source)
    at java.awt.Container.processEvent(Unknown Source)
    at java.awt.Component.dispatchEventImpl(Unknown Source)
    at java.awt.Container.dispatchEventImpl(Unknown Source)
    at java.awt.Component.dispatchEvent(Unknown Source)
    at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
    at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
    at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
    at java.awt.Container.dispatchEventImpl(Unknown Source)
    at java.awt.Window.dispatchEventImpl(Unknown Source)
    at java.awt.Component.dispatchEvent(Unknown Source)
    at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
    at java.awt.EventQueue.access$200(Unknown Source)
    at java.awt.EventQueue$3.run(Unknown Source)
    at java.awt.EventQueue$3.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
    at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
    at java.awt.EventQueue$4.run(Unknown Source)
    at java.awt.EventQueue$4.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
    at java.awt.EventQueue.dispatchEvent(Unknown Source)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
    at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
    at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
    at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
    at java.awt.EventDispatchThread.run(Unknown Source)
4854400 [AWT-EventQueue-0] ERROR org.zaproxy.zap.control.AddOnLoader  - C:\Users\mahin\OWASP ZAP_D\plugin\callgraph-alpha-1.zap (The system cannot find the file specified)
java.io.FileNotFoundException: C:\Users\mahin\OWASP ZAP_D\plugin\callgraph-alpha-1.zap (The system cannot find the file specified)
    at java.util.zip.ZipFile.open(Native Method)
    at java.util.zip.ZipFile.<init>(Unknown Source)
    at java.util.zip.ZipFile.<init>(Unknown Source)
    at java.util.jar.JarFile.<init>(Unknown Source)
    at java.util.jar.JarFile.<init>(Unknown Source)
    at org.zaproxy.zap.control.AddOnLoader.getJarClassNames(AddOnLoader.java:577)
    at org.zaproxy.zap.control.AddOnLoader.getImplementors(AddOnLoader.java:481)
    at org.zaproxy.zap.control.ExtensionFactory.loadAddOnExtensions(ExtensionFactory.java:156)
    at org.zaproxy.zap.control.AddOnLoader.addAddon(AddOnLoader.java:220)
    at org.zaproxy.zap.extension.autoupdate.ExtensionAutoUpdate.install(ExtensionAutoUpdate.java:757)
    at org.zaproxy.zap.extension.autoupdate.ExtensionAutoUpdate.installLocalAddOn(ExtensionAutoUpdate.java:210)
    at org.zaproxy.zap.extension.autoupdate.ExtensionAutoUpdate.access$1(ExtensionAutoUpdate.java:183)
    at org.zaproxy.zap.extension.autoupdate.ExtensionAutoUpdate$2.actionPerformed(ExtensionAutoUpdate.java:172)
    at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
    at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
    at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
    at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
    at javax.swing.AbstractButton.doClick(Unknown Source)
    at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source)
    at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown Source)
    at java.awt.Component.processMouseEvent(Unknown Source)
    at javax.swing.JComponent.processMouseEvent(Unknown Source)
    at java.awt.Component.processEvent(Unknown Source)
    at java.awt.Container.processEvent(Unknown Source)
    at java.awt.Component.dispatchEventImpl(Unknown Source)
    at java.awt.Container.dispatchEventImpl(Unknown Source)
    at java.awt.Component.dispatchEvent(Unknown Source)
    at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
    at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
    at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
    at java.awt.Container.dispatchEventImpl(Unknown Source)
    at java.awt.Window.dispatchEventImpl(Unknown Source)
    at java.awt.Component.dispatchEvent(Unknown Source)
    at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
    at java.awt.EventQueue.access$200(Unknown Source)
    at java.awt.EventQueue$3.run(Unknown Source)
    at java.awt.EventQueue$3.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
    at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
    at java.awt.EventQueue$4.run(Unknown Source)
    at java.awt.EventQueue$4.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
    at java.awt.EventQueue.dispatchEvent(Unknown Source)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
    at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
    at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
    at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
    at java.awt.EventDispatchThread.run(Unknown Source)

To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-devel...@googlegroups.com.

[Colm O'Flaherty]

I think it's primarily an issue in the Spider framework, which currently does not support adding the URL where each page was found. There may also be changes required in other classes to support it, like in HttpRequestHeader.

I'm presuming you have verified that the file exists, since that's the exception you're seeing?

[thc...@gmail.com]

Hi.

Note that the graph shows only the proxied requests, requests with history type = 1. The Ajax and "normal" spiders use other history types for its requests (i.e. history types 10 and 2, respectively) so even if they set the "Referer" header the requests will not be taken into account when building the graph.

P.S. The browser HtmlUnit also sets the "referer" header ;)

Best regards.

[Colm O'Flaherty]

This is true..

Would it be worth including requests with history types 2 and 10 when building up the graph? That should be a straightforward change. A possible downside is that the resulting graph could be a lot "noisier" than it currently is as a result, and it might necessitate other changes to better manage the additional volume of requests (nodes+edges in the graph).

[kingthorin+owaspzap]

Could we collect all of the info and then have toggles to turn the various types on or off, it would be very cool to see things grow/morph in realtime based on selection.

Does call graph only work from live data or does it also work from the history table? i.e.: If I open a saved session is call graph able to build up a graph?

[Colm O'Flaherty]

It pulls from the history table, so yes, you should be able to pull up a historic session (as long as the format is recent), and get a call graph out of it (for proxied requests only).

I'm no good at GUI stuff, but if someone else wants to have a go at making the Call Graph more flexible / integrated (as an alternative to the Sites node), then please be my guest!

Colm

[thc...@gmail.com]

Hi.

Would it be worth including requests with history types 2 and 10 when building up the graph?

Yes, as you say that's a straightforward change and would make use of the Ajax Spider requests. I don't think we need to add any other changes, at least for now.

Are you OK doing the changes for a new release?

Does anyone disagrees with doing the changes and release?

Best regards.

[Colm O'Flaherty]

No problem. Will do.

Colm

[thc...@gmail.com]

Thanks!

Best regards.

[Colm O'Flaherty]

I've tested the change locally, and it works fine for me with the Ajax Spider, for both Firefox and HtMLUnit (not tested under Chrome).  The "traditional" Spider does not set the Referer header, however, so any app that has been spidered using just that technique will not display a call graph, however.

Cosmin, how much work would it be to set the Referer header in requests originating in the Spider?

Colm

[m.pen...@gmail.com]

Hi Colm,
how can I access to the improved code?

Best regards

[Colm O'Flaherty]

Hi,

You can build the latest extension directly using the Zap trunk, or it should be in the latest weekly release.  I always build and run from the trunk, so I haven't verified the weekly build, however.

The latest version has the following features

- root nodes are labelled with the full URL (minus the query), and are colour coded, to aid the eye

- non-root nodes are labelled with just the path, since the schema and host can be determines by looking at the root node (using the colour coding to help you)

- microhelp is now in place for nodes and vertices, to help you follow complex graphs.

Colm

--

[m mm]

is the code public now? can I use that?

if yes, would you please send me a link to use that?

best regards :)

--
You received this message because you are subscribed to a topic in the Google Groups "OWASP ZAP Developer Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/zaproxy-develop/bEdcPMeYzyk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to zaproxy-devel...@googlegroups.com.

[Colm O'Flaherty]

Its in the trunk (/zap-extensions-alpha/src/org/zaproxy/zap/extension/callgraph), so you can build it directly yourself, or you can use the attached file.

Colm

[m mm]

thanks :)

i used that with installed ZAP on windows, how can i use this in develop mode?

[Colm O'Flaherty]

Just copy the file into ~/.ZAP_D/plugin/.

Once the dev version starts up, it looks in there for plugins in the same way that the non-dev version looks in ~/.ZAP/plugin/.

Colm

[thc...@gmail.com]

Hi.

It's also possible to manually install/update the add-on from within ZAP, using "File" > "Load Add-on file..." (default: Ctrl + L).

Best regards.

[Colm O'Flaherty]

Sorry, I should have mentioned that, since that's the more user friendly, and hence the preferred way to go, rather than copying files on the filesystem.

[m.pen...@gmail.com]

thanks :)

On Wednesday, July 16, 2014 12:21:39 AM UTC+4:30, m.pen...@gmail.com wrote: