[groovy-user] Reading from a socket
Anders Viklund
The code found here is writing to a socket.
s = new Socket("localhost", 8167);
s << "Why don't you call me anymore?\n"
s.close()
How can I read the response from the socket?
Thanks!
Hotmail: Trusted email with Microsoft’s powerful SPAM protection. Sign up now.
John Hartnup
> Hi,
>
> The code found here is writing to a socket.
>
> s = new Socket("localhost", 8167);
> s << "Why don't you call me anymore?\n"
> s.close()
s.withStreams { inStream, outStream ->
def reader = inStream.newReader()
def name = reader.readLine()
outStream << "Hello ${name}"
}
... one of my favourite examples of closures leading to neat idioms.
--
"There is no way to peace; peace is the way"
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
Anders Viklund
server = new ServerSocket(8182)
while(true) {
server.accept() { socket ->
socket.withStreams { input, output ->
def s = new Socket("localhost", 9090)
s << input // < -- program hangs here, why?
Thanks for any ideas how to solve this!
> Date: Fri, 12 Mar 2010 10:10:37 +0000
> From: john.h...@gmail.com
> To: us...@groovy.codehaus.org
> Subject: Re: [groovy-user] Reading from a socket
Hotmail: Free, trusted and rich email service. Get it now.
John Hartnup
> I am trying to pipe an input stream into the socket, but it never closes.
>
> server = new ServerSocket(8182)
> while(true) {
> server.accept() { socket ->
> socket.withStreams { input, output ->
> def s = new Socket("localhost", 9090)
> s << input // < -- program hangs here, why?
Basically, I don't think the Socket.leftShift() function does what you
want, with an inputStream as an argument.
Instead I think you need a loop that reads from input and writes to s.
Anders Viklund
Why shouldnt leftShift work in this case?
> Date: Fri, 12 Mar 2010 21:31:09 +0000
> From: john.h...@gmail.com
> To: us...@groovy.codehaus.org
> Subject: Re: [groovy-user] Reading from a socket
>
John Hartnup
> well, the data arrives to its target destination, but the stream never
> closes, thats the problem.
>
> Why shouldnt leftShift work in this case?
Ah, I was going by the JavaDoc for Socket, which doesn't really
specify what leftShift(Object) does in detail:
"Overloads the left shift operator to provide an append mechanism to
add things to the output stream of a socket"
That doesn't document the fairly specialised case of an InputStream.
Could do better!
Without any documented semantics for leftShift(InputStream), I don't
know what you're trying to do, nor whether it should work.
Does the InputStream eventually signal EOF? It would seem reasonable
for leftShift() to return only when that happens. But again, nothing
*says* that it will. At least not in the JavaDoc.
Anders Viklund
So far I have managed to get the request through to its destination.
Then it stalls because it never stops reading from the request input stream.
> Date: Sat, 13 Mar 2010 07:33:53 +0000
> From: john.h...@gmail.com
> To: us...@groovy.codehaus.org
> Subject: Re: [groovy-user] Reading from a socket
>
Hotmail: Trusted email with powerful SPAM protection. Sign up now.
John Hartnup
> I am trying to create a http proxy, just like tcpmon.
>
> So far I have managed to get the request through to its destination.
> Then it stalls because it never stops reading from the request input stream.
And how would it know to stop reading from the inputstream, unless you
call a method that
So, do a loop. It's only using Java stuff; no Groovy tricks, but it's
simple and it works.
byte[] buffer
def len = input.read(buffer)
while(len >=0) {
output.write(buffer,0,len)
len = input.read(buffer)
}
Of course you probably also want to do the same to transmit data in
the opposite direction. You could do this in another thread:
// Wrap the code above in a StreamForwarder class that implements Runnable.
Thread c2s = new Thread(new StreamForwarder(clientInStream,serverOutStream));
StreamForwarder s2c = new StreamForwarder(serverInStream,clientOutStream);
// Run one in a new thread
2s.start();
// Run the other in this thread
s2c.run();
Or you could read up on select() and do it all in one thread.
Potentially better scalability. Arguably less easy for people to
understand, if they later read the code and are not familiar with
select().
Anders Viklund
byte[] buffer
def len = input.read(buffer) // <-- now its stalling here, why?
while(len >=0) {
output.write(buffer,0,len)
len = input.read(buffer)
}
> From: john.h...@gmail.com
> To: us...@groovy.codehaus.org
> Subject: Re: [groovy-user] Reading from a socket
>
Hotmail: Powerful Free email with security by Microsoft. Get it now.
John Hartnup
> def len = input.read(buffer) // <-- now its stalling here, why?
http://java.sun.com/j2se/1.4.2/docs/api/java/io/InputStream.html#read(byte[])
"This method blocks until input data is available, end of file is
detected, or an exception is thrown."
So, it's waiting for the other end to either write to the socket, or close it.
It seems as if by trying to write a proxy, you're running before you
can walk. Go through the Java sockets tutorials.
http://java.sun.com/docs/books/tutorial/networking/sockets/index.html
Write a simple client/server pair and make sure you understand how it works.
Groovy doesn't change anything much, other than give you a few
convenience methods for common cases (such as
ServerSocket.accept(Closure ) and Socket.withStreams(Closure)
Anders Viklund
This is so wierd,
here, the request gets forwarded to the server, but the the client never sends an end of file character in the stream so it stalls.
def server = new ServerSocket(8226)
while(true) {
server.accept() { socket ->
socket.withStreams { input, output ->
def s = new Socket("localhost", 9090)
s << input
...
but, if I do like this, the request does not even get forwarded to the server, why?
def server = new ServerSocket(8226)
while(true) {
server.accept() { socket ->
socket.withStreams { input, output ->
def s = new Socket("localhost", 9090)
def thRequest = Thread.start {
s << input
}
..
> Date: Sat, 13 Mar 2010 10:27:20 +0000
> From: john.h...@gmail.com
> To: us...@groovy.codehaus.org
> Subject: Re: [groovy-user] Reading from a socket
>
Anders Viklund
java.net.SocketException: socket closed
Why is this?
def server = new ServerSocket(8235)
while(true) {
server.accept() { socket ->
socket.withStreams { input, output ->
def s = new Socket("localhost", 9090)
def thRequest = Thread.start {
try{
outStream << input
}catch (Exception e){println e}
}
}
}
}
}
From: viklund...@hotmail.com
To: us...@groovy.codehaus.org
Date: Sat, 13 Mar 2010 11:09:41 +0000
Subject: RE: [groovy-user] Reading from a socket
Hotmail: Trusted email with Microsoft’s powerful SPAM protection. Sign up now.
John Hartnup
> Putting the left shift into a try/catch results in this exception
>
> java.net.SocketException: socket closed
>
> Why is this?
Because one of your sockets is closed. Since you've bundled the read
and the write into one statement, and you've thrown away the stack
trace, it's not possible to tell which one. Either the client has
closed the socket you're reading from, or the server has closed the
socket you're writing to. Your code is going to have to deal with
that, and it cannot assume that they'll happen in a particular order.
Use e.printStackTrace(), and stop desperately clinging onto your
desire to use '<<', since nobody who's prepared to comment at this
time, knows how it's supposed to behave.
I don't understand why you didn't see this exception until you
explicitly caught it. Are you working in some wider environment that
catches and ignores exceptions? Or did you just fail to tell us about
the exceptions when they were reported?
hix li
I would like to allocate more memory for running groovy.
Is it by changing the JAVA_OPTS?
I have changed the groovy/bin/startGroovy.bat file, placing the command:
set JAVA_OPTS="-Xmx512m" But the resource monitor tells that only 55M is allocated to groovy when it's running in windows.
Is there any other place that I should make change in order to allocate more memory for running groovy?
Thanks!
Hix
|
|
|
Anders Viklund
I have selected "Capture Standard Output" and "Show Full Stack Traces" under view tab in the console and I am also surprised that no errors were printed to the console until I used try/catch.
The reason why I am using << is that I can see that the target server can receive the request ok and its less code to write.
Anyway, here is the full stack trace.
java.net.SocketException: socket closed
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.leftShift(DefaultGroovyMethods.java:5011)
at org.codehaus.groovy.runtime.dgm$304.invoke(Unknown Source)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:270)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:52)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:43)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:124)
at HttpProxy3$_run_closure1_closure2_closure3_closure4.doCall(HttpProxy3.groovy:9)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:234)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:272)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:892)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:66)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:142)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:150)
at HttpProxy3$_run_closure1_closure2_closure3_closure4.doCall(HttpProxy3.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:234)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:272)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:892)
at groovy.lang.Closure.call(Closure.java:279)
at groovy.lang.Closure.call(Closure.java:274)
at groovy.lang.Closure.run(Closure.java:355)
at java.lang.Thread.run(Thread.java:619)
> Date: Sat, 13 Mar 2010 11:59:55 +0000
> From: john.h...@gmail.com
> To: us...@groovy.codehaus.org
> Subject: Re: [groovy-user] Reading from a socket
>
Anders Viklund
see here:
http://jmol.sourceforge.net/docs/JmolUserGuide/ch02s07.html
Date: Sat, 13 Mar 2010 04:01:55 -0800
From: hix...@yahoo.ca
To: us...@groovy.codehaus.org
CC: hix...@yahoo.ca
Subject: [groovy-user] How to setup the JAVA_OPTS in groovy?
John Hartnup
> java.net.SocketException: socket closed
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:129)
> at org.codehaus.groovy.runtime.DefaultGroovyMethods.leftShift(DefaultGroovyMethods.java:5011)
Which of your two sockets do you think that applies to (there is
enough info there to tell)
What do you think the state of that socket is when the read happens?
(the exception message tells you)
How do you want your proxy to behave when that state occurs? (this is
your implementation decision)
Note: "less code" is great, but only if it does what you need it to!
Anders Viklund
But why is the exception thrown for left shift inside the thread and not outside?
> Date: Sat, 13 Mar 2010 12:30:53 +0000
> From: john.h...@gmail.com
> To: us...@groovy.codehaus.org
> Subject: Re: [groovy-user] Reading from a socket
>
Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. Sign up now.
John Hartnup
> I guess its the left shift thats causing this problem.
>
> But why is the exception thrown for left shift inside the thread and not
> outside?
I just reproduced it using a simpler case (always a good idea).
When you think it through, it makes sense.
What does withStreams() do?
A1. Get a pair of Input/Output streams from the socket
A2. Pass the streams to the closure provided as a parameter
A3. When the closure returns, close the streams (and therefore the socket)
Now, what does your closure do:
B1. spawns a thread
B2. returns
... and within the thread:
C1. Try to read from the InputStream
...
What you're seeing is that A3 happens while C1 is in progress. So C1
gets the socket closed exception.
To avoid this, you need a wait() for the thread to complete, before
your withStreams closure returns.
Here's some Java I have lying around. It will probably work as-is
within Groovy. I see no reason to change it.
public void blindlyForward() {
Thread c2s = new Thread(new StreamForwarder(clientInStream,serverOutStream));
StreamForwarder s2c = new StreamForwarder(serverInStream,clientOutStream);
// Run one in a new thread
c2s.start();
// Run the other in this thread
s2c.run();
boolean done = false;
while(!done) {
try {
c2s.join();
done = true;
} catch (InterruptedException e) {
// Shouldn't happen. Thread has no wait() or sleep() calls.
System.out.println(e.getMessage());
}
}
}
... and StreamForwarder itself:
public class StreamForwarder implements Runnable {
private InputStream input;
private OutputStream output;
public static final int BLOCKSIZE = 1024;
/**
* Constructor
* @param input Something to read from
* @param output Something to write to
*/
public StreamForwarder(InputStream input, OutputStream output) {
this.input = input;
this.output = output;
}
/**
* Run method - read a KB, write a KB, until either an IOException is thrown
* or we reach EOF.
*/
public void run() {
byte[] block = new byte[BLOCKSIZE];
boolean done = false;
while(!done) {
try {
int size = input.read(block);
if(size >0) {
output.write(block,0,size);
}
if(size == -1) {
done = true;
}
} catch (IOException e) {
System.err.println(e.getMessage());
done = true;
}
Thread.yield(); // polite, but surely redundant in this day and age?
Anders Viklund
This is now working, now I just have to figure out how to copy the input/output streams into separate strings on their way through.
def server = new ServerSocket(8235)
while(true) {
server.accept() { socket ->
socket.withStreams { input, output ->
def s = new Socket("localhost", 9090)
s.withStreams { inStream, outStream ->
blindlyForward(inStream, output,input, outStream)
}catch(Exception e){e.printStackStrace()}
}
}
}
}
public void blindlyForward(InputStream clientInStream, OutputStream serverOutStream, InputStream serverInStream, OutputStream clientOutStream) {
Thread c2s = new Thread(new StreamForwarder(clientInStream,serverOutStream));
StreamForwarder s2c = new StreamForwarder(serverInStream,clientOutStream);
// Run one in a new thread
c2s.start();
// Run the other in this thread
s2c.run();
boolean done = false;
while(!done) {
try {
c2s.join();
done = true;
} catch (InterruptedException e) {
// Shouldn't happen. Thread has no wait() or sleep() calls.
System.out.println(e.getMessage());
}
}
}
> From: john.h...@gmail.com
> To: us...@groovy.codehaus.org
> Subject: Re: [groovy-user] Reading from a socket
>
hix li
I checked the link you mentioned. Yes, Xms is for the max memory capacity, not the running memory. Any way to specify the running memory?
|
The new Internet Explorer® 8 - Faster, safer, easier. Optimized for Yahoo! Get it Now for Free!
Tim Yates
It does not set how much memory your app will allocate on startup, just the level it's allowed to grow to.
I assume you're running out of memory?
Tim
hix li
I see then. My program is not out of memory, but it's slow. So I am thinking of increase the memory capacity for the program to see if it would make it faster.
:(
Hix
|
Tim Yates
Otherwise, if you have some example code, we may be able to see a way of speeding things up?
Tim
hix li
Unfortunately, it's under windows vista.
:(
Actually I am considering changing to linux like ubuntu as an alternative if things can't be speed up in windows. |
Tim Yates
If it's not a memory issue, attacking the problem in a different way might do though
Tim
Victor Grazi
for builders. 2 use cases:
1. Have non-programmmers prepare the builder without having to worry
about the overhead of adding import statements and public static void
main, etc
2. Prepare builder component files that can be mixed and matched into
a larger container builder.
I would like to be able to include external builders and have my
builder subclass read-in the external files and process the nodes as
if they were resident.
Any direction would be appreciated
Thanks Victor
hix li
I agree the speeding up the program is not a simple memory issue. Actually it is not a simple issue at all.
I am using groovy as front end and MS Access as database backend (in the same machine, not with network yet) to run for a simulation on the artificial financial market that involves with quite a lot of interactions among objects and data extracting from database. I am working on my economics thesis, and I am not familiar with the inside structure of the computer and complier, etc.
I was guessing the bottomneck is in the database visiting. So I have switched the database backend from MS Access to MySQL, however, it does not help in speeding up the program. Just my experience. |
Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail
Tim Yates
It could be opening a connection every time you fetch or write something to the database
Tim
Victor Grazi
Anders Viklund
The problem is that the client input stream is not closed until the outStream from target server is closed.
Thanks!
class HttpProxy implements Runnable{
...
void run() {
def server = new ServerSocket(HttpProxtPort)
while(true) {
server.accept() { socket ->
socket.withStreams { input, output ->
def s = new Socket(targetHostName, targetHostPort) //connect to target host
s.withStreams { inStream, outStream ->
blindlyForward(inStream, output,input, outStream)
}
}catch(Exception e){
// How to capture whats in the input stream and close it?
}
}
println "done"
}
}
}
..
}
> Date: Sat, 13 Mar 2010 13:14:12 +0000
> From: john.h...@gmail.com
> To: us...@groovy.codehaus.org
> Subject: Re: [groovy-user] Reading from a socket
>
John Hartnup
> How can I read the client input stream if the connection to the target host
> is not working?
>
> The problem is that the client input stream is not closed until the
> outStream from target server is closed.
>
You'd have to put some logic around the write to the outputstream, to
catch the IOException and do what you want to do (ignore it?)
Anders Viklund
The soap applications I am using can do this, and I would like to do the same in my http groovy proxy.
Thanks!
> Date: Wed, 17 Mar 2010 08:16:40 +0000
> From: john.h...@gmail.com
> To: us...@groovy.codehaus.org
> Subject: Re: [groovy-user] Reading from a socket
>
John Hartnup
> How can I search the contents "on the fly" for the start/end of each http
> request/response, while keeping the connection alive?
>
> The soap applications I am using can do this, and I would like to do the
> same in my http groovy proxy.
Work with an HTTP library, rather than at the socket level?
Or write your own - perhaps having studied the source for one of the
existing ones.
Anders Viklund
but, how to act as a Http server?
Hotmail: Powerful Free email with security by Microsoft. Get it now.
John Hartnup
> I see many examples how to build http requests with the Groovy Http library,
>
> but, how to act as a Http server?
One way is to run inside Jetty.
Anders Viklund
> Date: Wed, 17 Mar 2010 19:20:22 +0000
> From: john.h...@gmail.com
> To: us...@groovy.codehaus.org
> Subject: Re: [groovy-user] Reading from a socket
>
John Hartnup
> seems like overkill to me. There must be a way to parse the inputstream on
> the fly, but how?
Well, Jetty's tiny, so it doesn't feel like overkill to me.
Other alternatives... Observer pattern?
Anders Viklund
> Date: Wed, 17 Mar 2010 20:02:01 +0000
> From: john.h...@gmail.com
> To: us...@groovy.codehaus.org
> Subject: Re: [groovy-user] Reading from a socket
>
Hotmail: Trusted email with Microsoft’s powerful SPAM protection. Sign up now.
Gordon Ross
> seems like overkill to me. There must be a way to parse the inputstream on the fly, but how?
If Jetty seems too heavyweight for you, why not extract just the HTTP
"server" aspect of it ?
I remember there was a book sometime ago that explained how Tomcat worked.
Maybe that would help you pick out bits for Tomcat (or Jetty) ?
However, I do get the impression that you are a little out of your depth
here.
GTG
Anders Viklund
public void run() {
byte[] block = new byte[BLOCKSIZE];
boolean done = false;
FileOutputStream fos = new FileOutputStream(contentFile);
while(!done) {
try {
int size = input.read(block);
if(size >0) {
output.write(block,0,size);
fos.write(block,0,size); // How can I find start/end of the request here?
}
if(size == -1) {
done = true;
}
} catch (IOException e) {
System.err.println(e.getMessage());
done = true;
}
}
}
> From: gr...@ucs.cam.ac.uk
> To: us...@groovy.codehaus.org
> Subject: RE: [groovy-user] Reading from a socket
Hotmail: Trusted email with powerful SPAM protection. Sign up now.
Gordon Ross
>> However, I do get the impression that you are a little out of your
>> depth here.
> I agree with you, it would be good enough if I could change this code
> some how to find start/end of the http request, without closing the
> stream in between.
Have you read up on the HTTP protocol ? The first few hits from google
aren't that bad...
Anders Viklund
> Date: Wed, 17 Mar 2010 21:43:26 +0000
> From: gr...@ucs.cam.ac.uk
> To: us...@groovy.codehaus.org
> Subject: RE: [groovy-user] Reading from a socket
>
Gordon Ross
> well.. I am starting to realize that this is a dead end.
Dead end 'cause no-one can give you a 10-line snippet of code on how to do
it ? Yes.
Why has no-one given you a 10-line snipped of code to parse HTTP ?
Probably 'cause none exists.
You have two choices:
1) Write your own HTTP parsing code.
2) Re-use someone elses.
Why don't you want to re-use someone else's ?
Why have you discounted, for example, the top google hit for "java http
server" ?
Did you google something like "java http server" ?
Tim Yates
http://www.java2s.com/Code/Java/Network-Protocol/Asimpleproxyserver.htm
Which shows a proxy server in java
Why not use that as a starting point, and try to groovify it?
Tim
Anders Viklund
I guess I will have to start reading up on how the java http server works, maybe here
http://java.sun.com/javase/6/docs/jre/api/net/httpserver/spec/index.html
or here?
http://api.dpml.net/jetty/5.1.6/
Thanks for any further ideas!
> Date: Wed, 17 Mar 2010 22:53:51 +0000
> From: gr...@ucs.cam.ac.uk
> To: us...@groovy.codehaus.org
> Subject: RE: [groovy-user] Reading from a socket
>
Anders Viklund
I would like to be able to distinguish between every separate request, so I can make a copy of it. The same goes for the responses.
Probably the http server is the right way to go.
From: tim....@gmail.com
Date: Wed, 17 Mar 2010 23:11:11 +0000
To: us...@groovy.codehaus.org
Subject: Re: [groovy-user] Reading from a socket
Brian Schlining
I guess I will have to start reading up on how the java http server works, maybe here
http://java.sun.com/javase/6/docs/jre/api/net/httpserver/spec/index.html
or here?
http://api.dpml.net/jetty/5.1.6/
Thanks for any further ideas!
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
Brian Schlining
bschl...@gmail.com
Anders Viklund
Thanks!
Date: Wed, 17 Mar 2010 16:32:22 -0700
From: bschl...@gmail.com
To: us...@groovy.codehaus.org
Subject: Re: [groovy-user] Reading from a socket