ZAP Error [java.net.SocketException]: Socket closed

[Brian Fallik]

Hi,

I'm trying to use zaproxy in browser proxy mode to test an internal application but all attempts to use the proxy result in an error and stack trace:

ZAP Error [java.net.SocketException]: Socket closed

Stack Trace:
java.net.SocketException: Socket closed
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at java.net.Socket.connect(Socket.java:538)
	at java.net.Socket.<init>(Socket.java:434)
	at java.net.Socket.<init>(Socket.java:286)
	at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
	at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
	at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:727)
	at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361)
	at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:447)
	at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:199)
	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
	at org.parosproxy.paros.network.HttpSender.executeMethod(HttpSender.java:333)
	at org.parosproxy.paros.network.HttpSender.runMethod(HttpSender.java:564)
	at org.parosproxy.paros.network.HttpSender.send(HttpSender.java:523)
	at org.parosproxy.paros.network.HttpSender.sendAuthenticated(HttpSender.java:501)
	at org.parosproxy.paros.network.HttpSender.sendAuthenticated(HttpSender.java:490)
	at org.parosproxy.paros.network.HttpSender.sendAndReceive(HttpSender.java:405)
	at org.parosproxy.paros.network.HttpSender.sendAndReceive(HttpSender.java:362)
	at org.parosproxy.paros.core.proxy.ProxyThread.processHttp(ProxyThread.java:509)
	at org.parosproxy.paros.core.proxy.ProxyThread.run(ProxyThread.java:317)
	at java.lang.Thread.run(Thread.java:748)

I'm browsing from chrome on macOS 10.14. Within chrome I'm using SwitchyOmega to configure the proxy for specific hosts. I'm fairly sure that this host matching is working. Even when I try to proxy an external website Zap spits out the error above for all requested resources.

The usual searches on Google and within this email group didn't turn anything interesting up. The error message is generic enough that I don't have many clues to debug this further. Any help would be appreciated.

Thanks,

brian

[hauschu...@gmail.com]

What if you point your browser to ZAP using the regular browser proxy settings? Does that work?

[Brian Fallik]

Same result.

I modified the default setting in Firefox to use 127.0.0.1:8080 to proxy all traffic and browsed to www.google.com. I see a series of 504 responses in the Zap history tab. That's what I saw from Chrome too, a detail I forgot to mention in my earlier email.

One difference is that Firefox displays an error in the browser too:

    Failed to read http://google.com/ within 20 seconds, check to see if the site is available and if so consider adjusting ZAP's read time out in the Connection options panel.

It seems like both browsers are successfully connecting to Zap but after that initial connection Zap seems unable to fulfill the proxy request.

brian

On Thu, Apr 4, 2019 at 2:18 AM <hauschu...@gmail.com> wrote:

What if you point your browser to ZAP using the regular browser proxy settings? Does that work?

--
You received this message because you are subscribed to the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/8a460a1c-e551-497b-a1c1-f72aa4abd546%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[hauschu...@gmail.com]

What are the details in the 504 error in ZAP's history tab?

Please include a screenshot if possible!

[Brian Fallik]

Hi,

It's interesting but I'm seeing both 502s and 504s depending on which browser I'm using.

From Chrome I browsed to http://cornell.edu and Zap reports 502 "Bad Gateway". From Firefox I browsed to http://google.com and Zap reports 504 "Gateway Timeout". See screenshot showing both errors:

Thanks,

brian

On Thu, Apr 4, 2019 at 7:53 AM <hauschu...@gmail.com> wrote:

What are the details in the 504 error in ZAP's history tab?

Please include a screenshot if possible!

--

You received this message because you are subscribed to the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/c29f4235-bd7b-4270-8ff8-ed0b2445c3a6%40googlegroups.com.

[hauschu...@gmail.com]

Sorry, could you highlight one of each of the responses so that we can see the full response headers and body in the screenshot?

Are you working behind a corporate proxy?

[Brian Fallik]

OK. Hopefully the copied text is as useful as a screenshot.

The request headers are:

  GET http://download.mozilla.org/?product=firefox-57.0.4-complete-bz2&os=osx&lang=en-US HTTP/1.1

  User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:39.0) Gecko/20100101 Firefox/39.0

  Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

  Accept-Language: en-US,en;q=0.5

  Range: bytes=600000-899999

  Cookie:  optimizelySegments=%7B%22245617832%22%3A%22none%22%2C%22245875585%22%3A%22direct%22%2C%22245677587%22%3A%22ff%22%2C%22246048108%22%3A%22false%22%2C%22869421433%22%3A%22true%22%7D; optimizelyEndUserId=oeu1400003814859r0.025868374937178862; optimizelyBuckets=%7B%7D

  Connection: keep-alive

  Proxy-Connection: Keep-Alive

  Host: download.mozilla.org

with an empty body.

The response headers are:

  HTTP/1.1 504 Gateway Timeout

  Content-Type: text/plain; charset=UTF-8

  Content-Length: 236

with body:

  Failed to read http://download.mozilla.org/?product=firefox-57.0.4-complete-bz2&os=osx&lang=en-US within 20 seconds, check to see if the site is available and if so consider adjusting ZAP's read time out in the Connection options panel.

Thx,

brian

On Thu, Apr 4, 2019 at 8:06 AM <hauschu...@gmail.com> wrote:

Sorry, could you highlight one of each of the responses so that we can see the full response headers and body in the screenshot?

Are you working behind a corporate proxy?

--

You received this message because you are subscribed to the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/7027e9ed-2f9e-4145-a796-5d4ca091dcdd%40googlegroups.com.

[hauschu...@gmail.com]

Are you working behind a corporate/system proxy?

Can you access the internet at all?

[Brian Fallik]

Hi,

No corporate proxy but I am behind a NAT firewall. Internet works fine and I can browse anywhere if I disable the browser proxy.

Brian

On Thu, Apr 4, 2019, 8:27 AM <hauschu...@gmail.com> wrote:

Are you working behind a corporate/system proxy?

Can you access the internet at all?

--

You received this message because you are subscribed to the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/ae2ea03e-71a2-4b05-8075-f47f0dd20cdc%40googlegroups.com.

[hauschu...@gmail.com]

It definitely sounds like a network connectivity issue, so for the sake of troubleshooting, if you turn off the firewall, or go 'full open' (both directions), does that resolve the issue?

[hauschu...@gmail.com]

Obviously this isn't a solution, just part of diagnosis :)

[Brian Fallik]

Yup, thank for your help diagnosing this. ;-)

Unfortunately I can't turn off the firewall. This is a corporate environment with a shared network. But I do help manage the network and I'm reasonably confident that the firewall isn't causing this issue. For one our firewall settings are very basic - NAT translation, outbound connections are all open and new inbound connections are all blocked. Also our network supports usage from multiple software development companies and we haven't heard of any reports of any networking or firewall issues. Given the usage I'd expect to hear about it if our network somehow blocked proxied requests.

It's more likely that if this is a networking issue then the problem is on my laptop. Are there any special settings needed to support proxying on macOS? Does Zap need special permissions or custom configuration?

brian

On Thu, Apr 4, 2019 at 8:42 AM <hauschu...@gmail.com> wrote:

Obviously this isn't a solution, just part of diagnosis :)

--

You received this message because you are subscribed to the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/1c1076d1-ad95-4a42-98f1-8b444d1e0fad%40googlegroups.com.

[hauschu...@gmail.com]

generally I'm  not aware of any special permissions needed....though I'm a little surprised that you're on a corporate environment without a proxy.

In any case, the only real configuration needed is something like this:

client (points to zap host:port) ----> ZAP (listens on host:port) ------> routes traffic (preflight CONNECTs won't show up in history tab but will be passed along)

here you would normally have ZAP on 127.0.0.1:8081 and your other local application on 127.0.0.1:8090 (or whatever)

for instance, I am able to successfully view my local jenkins instance through firefox proxying through ZAP (and Fiddler)

however MacOS is the area I'm least familiar with, so there may be some special proprietary thing I'm unaware of....

I'm actually a little stumped at the moment, it's definitely a basic connectivity issue but I can't think of what would do that.....do you have 'bypass proxy for local addresses' UNchecked? (it should be unchecked to force all traffic through ZAP)

obviously you need certificates installed and all that, but if they weren't you'd be seeing a different error.....

[hauschu...@gmail.com]

Not everyone's favorite method, but at this point I would try to slap Wireshark in there and see if you can trace back the root cause of these 502 and 504s....like where were they addressed to/from, who sent the grumpy reply, etc, with an emphasis on outgoing traffic from ZAP (since it looks like your browser to ZAP portion is OK)

[Brian Fallik]

That's a good call. It's been years since I've used wireshark but I do still have it installed on this laptop. I'll fire that up and see if it uncovers anything interesting.

brian

On Thu, Apr 4, 2019 at 9:40 AM <hauschu...@gmail.com> wrote:

Not everyone's favorite method, but at this point I would try to slap Wireshark in there and see if you can trace back the root cause of these 502 and 504s....like where were they addressed to/from, who sent the grumpy reply, etc, with an emphasis on outgoing traffic from ZAP (since it looks like your browser to ZAP portion is OK)

--

You received this message because you are subscribed to the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/a96ebdaf-c1ee-443a-9e77-8c338c8e75cd%40googlegroups.com.

[hauschu...@gmail.com]

i realize one part of my earlier description could be improved, it should look more like this:

client -----------------------------------> ZAP (listens on zapHost:port2, directs request to url) ----------------> application (listens on applicationHost:port)

-url: applicationHost:port

-proxy set: zapHost:port2

Also, can you confirm that your internal application is actively listening on the port you think it is?

one of these things or fuser, or whatever the preference is 

sudo lsof -i -P -n | grep LISTEN 

sudo netstat -tulpn | grep LISTEN
sudo nmap -sTU -O IP-address-Here

[Brian Fallik]

Thanks for the tip on using wireshark. That let me to two interesting observations:

1. for some reason I was seeing the same HTTP request repeated numerous times in the capture

2. I didn't see any packets leave Zap en route to the destination

This led me to poke around the Zap config where I realized I had "Use an outgoing proxy server" selected. And what was the proxy server configured to use? localhost:8080, the same as Zap was listening for inbound proxy requests. Once I deselected that option everything started working again.

Obviously this was a config error on my part but I suspect the debugging was more difficult because Zap was able to be configured to use the same proxy address for inbound (under "local proxies" section) and outbound connections (under "connection" section). It might be worth disallowing this configuration since I can't imagine that ever working correctly. As a test I changed the outbound proxy to use 127.0.0.1:8081 and I immediately saw a more helpful "connection refused" error.

Thanks for your help.

brian

--
You received this message because you are subscribed to the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/24231c84-34de-4dad-85c2-31c6e1bdf010%40googlegroups.com.

[kingthorin+owaspzap]

Thanks for following up!

I opened a new issue for this: https://github.com/zaproxy/zaproxy/issues/5308

[Brian Fallik]

Thanks for the help and for logging that ticket.

brian

On Thu, Apr 4, 2019 at 3:18 PM kingthorin+owaspzap <kingt...@gmail.com> wrote:

Thanks for following up!

I opened a new issue for this: https://github.com/zaproxy/zaproxy/issues/5308

--

You received this message because you are subscribed to the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/52ed15b0-a19f-45a8-b93b-40c9ad42a770%40googlegroups.com.

[hauschu...@gmail.com]

Haha, nice! 

Well, it was a good excuse to go dust off the wireshark at least! :)